Add query_dataframe function to be able to then use the labels in pandas #8
Conversation
|
Thanks for the PR! I'll need to think a little about this, since the Query returning instant vector
|
Ah, yes you are right, I didn't pay attention to this and thought the basic query always returned a vector. 😅
This is strange, considering I'm calling the For the rest of your reply, my bad, I was not thinking about these types of query. But I am completely open to another API, provided that I am able to get this kind of results for the basic vector query, as you showed it:
|
Looks like it was a copy-paste bug in my test script. Sorry to cause confusion. 🙇 I think I now understand what's going on and how I tried to tackle it when first writing the library. Potentially the right tool would have been a Query returning instant vector
|
|
This would be the alternate def to_pandas2(data: dict) -> pd.Series:
"""Convert Prometheus data object to Pandas Series."""
result_type = data['resultType']
if result_type == 'vector':
index_frame = pd.DataFrame(
r['metric'] | {'@': pd.Timestamp(r['value'][0], unit='s')}
for r in data['result'])
return pd.Series(
data=(np.float64(r['value'][1]) for r in data['result']),
index=pd.MultiIndex.from_frame(index_frame)
)
elif result_type == 'matrix':
index_frame = pd.DataFrame(
r['metric'] | {'@': pd.Timestamp(v[0], unit='s')}
for r in data['result'] for v in r['values'])
return pd.Series(
data=(np.float64(v[1]) for r in data['result'] for v in r['values']),
index=pd.MultiIndex.from_frame(index_frame)
)
elif result_type == 'scalar':
return pd.Series(
data=[np.float64(data['result'][1])],
index=[pd.Timestamp(data['result'][0], unit='s')])
elif result_type == 'string':
return pd.Series(
data=[data['result'][1]],
index=[pd.Timestamp(data['result'][0], unit='s')])
else:
raise ValueError('Unknown type: {}'.format(result_type)) |
I just tried it and works nicely for my use case. I can simply call But I think directly replacing |
|
I was working on updating the Jupyter workbook, but it turns out that the form above is pretty frustrating to work with (especially for plotting). This column 
|
|
I've created an updated |
|
Hey, I just had the time to look into this, I'm sorry but this layout makes it a pain to do what I want to do and I still didn't manage to do it. With the table layout I was able to plot the data with seaborn this way: df = p.query_dataframe(
'sum by (exp,config,seed,num) (node_network_transmit_bytes_total{device=~"link.*"})',
'1970-01-01T00:10:00Z',
'1s'
)
txbytes['value'] = txbytes['value'] / 1_000_000
# ... some more customisations ...
sns.barplot(x='exp', y="value", hue="config", data=df)
I did not yet find a way to make this work with the multiindex output. I got pretty close by transposing it and resetting the index, but then I have a column named by the timestamp that I don't know how to select (this query does not have group by df = p.query(
'sum by (config,seed,num) (node_network_transmit_bytes_total{device=~"link.*"})',
t,
'1s'
).T.reset_index()
print(df) |
2 participants
This is especially useful to use the labels later in plots, for exemple to group by configuration and such.