Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DX] Save a Collection's objects into a local JSON file #1172

Open
CShorten opened this issue Jul 9, 2024 · 1 comment
Open

[DX] Save a Collection's objects into a local JSON file #1172

CShorten opened this issue Jul 9, 2024 · 1 comment

Comments

@CShorten
Copy link
Member

CShorten commented Jul 9, 2024

What?

Get the objects from a Collection in Weaviate and to a local JSON file.

Why?

This helps with quick experimentation, maybe other use cases.

How?

Thinking this API:

collection = client.collections.get("WineReview")

collection.to_json(savepath="wine_reviews.json", num_samples=100) # returns bool success/fail

That pretty much just wraps the cursor API,

collection = client.collections.get("WineReview")

for item in collection.iterator():
    print(item.uuid, item.properties)

And then dumps the JSON file with the savepath.

Maybe also makes the json Python library another dependency of Weaviate's Python Client (if not already).

@CShorten CShorten changed the title Save a Collection's objects into a local JSON file [DX] Save a Collection's objects into a local JSON file Jul 9, 2024
@CShorten
Copy link
Member Author

CShorten commented Aug 11, 2024

Hey team, this notebook contains some more investigations into this idea of Weaviate Collections <> HuggingFace Dataset Hub or JSON files - https://github.com/weaviate/recipes/blob/main/weaviate-features/crud-apis/research/weaviate-collection-interoperability.ipynb

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant