-
Notifications
You must be signed in to change notification settings - Fork 2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add some utils for getting traces from the langfuse client #414
Conversation
WalkthroughThe changes introduce a new file, Changes
Recent review detailsConfiguration used: CodeRabbit UI Files selected for processing (1)
Additional comments not posted (6)
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
agent_traces = [ | ||
t | ||
for t in traces.data | ||
if t.session_id is not None and agent_name in t.session_id |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In UI, they allow to filter by session id startswith and similar fancy stuff, it didn't cross my mind to check if they allow that in Python SDK as well 😮💨
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah I was surprised by that too. I would have guessed that the sdk they expose would be the same as what they're using for the web backend, but I guess not!
Or we could add agent's name as a tag to filter it by that.
I was going to suggest this! Will make a ticket
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ticket for us here: #417
Also made an issue on the langfuse github https://github.com/orgs/langfuse/discussions/3395
from prediction_market_agent_tooling.tools.utils import add_utc_timezone_validator | ||
|
||
|
||
def get_traces_for_agent( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Instead of getting traces, it might be more future-proof to get observations with name process_market
, right now, it's the same thing (trace of trader agent === observation of process_market). But it could be changed in the future and one trace could contain many process_market
observations: gnosis/prediction-market-agent#444 (comment).
However it works now, it's already implement and it's just "maybe" so good by me, just FYI.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just tried this (client.get_observations(name="process_market", ...)
) but it returns an empty list - observations have the names of spans (in red), whereas process_market
is the name of the trace.
So I've added the trace_name="process_market"
arg. But atm the only traces containing agent names (e.g. DeployablePredictionProphetGPT4TurboFinalAgent
) are already the "process_market" traces. But I guess as you say it's more future proof.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No worries - it might be something I'm doing wrong. But I'll leave it for now if that's okay :)
Example usage: