Skip to content

OPEN-5555: Added methods for sending stream data #153

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Dec 2, 2023
Merged

Conversation

Parthib
Copy link
Contributor

@Parthib Parthib commented Nov 10, 2023

Summary

  • Introduces the stream_data Python client method, which hits the /data-stream endpoint.
  • The data passed to the strea_data method should be a dict or a list of dicts (records).
  • The OpenAI LLM monitor is now using the stream_data method (instead of publish_batch_data)
  • Introduces a verbose parameter to the OpenlayerClient to avoid spamming the user with "Stream published!" messages.

Testing

This is the client I used for testing data streaming.

Copy link
Contributor

@gustavocidornelas gustavocidornelas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other than the minor naming comments, it looks good. However, I noticed that the send_stream_data method is basically identical to the publish_batch_data, other than the final upload part (i.e., endpoint, request body, and the need for a file upload). With DRY in mind, do you think there's another way to do this? Maybe having a single method publish_data that takes in an arg that flags if it's a batch or stream?

@gustavocidornelas gustavocidornelas force-pushed the dataStream branch 3 times, most recently from 822e4f6 to b89a36b Compare December 1, 2023 20:53
@whoseoyster whoseoyster merged commit b1e3212 into main Dec 2, 2023
@whoseoyster whoseoyster deleted the dataStream branch December 2, 2023 13:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants