Feature/improve docs #1053
Closed
Feature/improve docs #1053
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Feature/remove extra r2r abstraction (#996) * moving kg construction to enrich-graph (#984) * checkin * up * done * formatting * Feature/update ingestion issues (#985) * udpate ingestion issues * keep unbounded limit support, but default to bounded * fix * fmt * removes an unnecessary abstraction * sync changes --------- Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> * first commit * move towards orchestration * tweaks * check in working ingestion * move * kg enrichment * update future, postgres compose * hatchetize ingestion pipeline * ready for prime time * finish --------- Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com>
* add update files workflow * rm ingestion pipeline
* add update files workflow * rm ingestion pipeline * v0 restructure orch
* add update files workflow * rm ingestion pipeline * v0 restructure orch * kg orchestration * finish kg orchestration * update service * merge * cleanups
* moving kg construction to enrich-graph (#984) * checkin * up * done * formatting * Feature/update ingestion issues (#985) * udpate ingestion issues * keep unbounded limit support, but default to bounded * fix * fmt * Add support for CharacterTextSplitter (#986) * Add support for CharacterTextSplitter Allows R2R client to override the text splitter. Example: ```python ingestion_response = client.ingest_files( file_paths=[file_path], metadatas=metadata, # optionally override chunking settings at runtime chunking_settings={ "provider": "r2r", "method": "character", "extra_fields": { "separator": "---" }, } ) ``` * fixup! Add support for CharacterTextSplitter * fixup! fixup! Add support for CharacterTextSplitter * Patch/ollama base cli (#992) * Dev (#990) * moving kg construction to enrich-graph (#984) * checkin * up * done * formatting * Feature/update ingestion issues (#985) * udpate ingestion issues * keep unbounded limit support, but default to bounded * fix * fmt * Add support for CharacterTextSplitter (#986) * Add support for CharacterTextSplitter Allows R2R client to override the text splitter. Example: ```python ingestion_response = client.ingest_files( file_paths=[file_path], metadatas=metadata, # optionally override chunking settings at runtime chunking_settings={ "provider": "r2r", "method": "character", "extra_fields": { "separator": "---" }, } ) ``` * fixup! Add support for CharacterTextSplitter * fixup! fixup! Add support for CharacterTextSplitter --------- Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> Co-authored-by: Manuel R. Ciosici <manuelrciosici@gmail.com> * fix ollama cli --------- Co-authored-by: Shreyas Pimpalgaonkar <shreyas.gp.7@gmail.com> Co-authored-by: Manuel R. Ciosici <manuelrciosici@gmail.com> * Ingestion refactor (#991) * fix test (#993) * Increase Neo4j memory limits, add GDS plugin, and update LLM concurrency limit to 256. * Update ingestion sample file, disable KG node extraction pipe, add community processing in clustering, and enhance graph clustering queries. * Update runners (#1007) * Refactor KG clustering process to simplify community processing and enhance entity-triple retrieval from Neo4j. * Refactor Neo4j configuration for memory settings and update graph clustering logic in the KG provider. * Fix pipeline by enabling node extraction and refactor community processing logic in KGClusteringPipe. * hatchet works * throw error if you run global search before enrichment * Fix communities in local search * turn off node desc embedding * fix rag endpoint * Increase hatchet msg size * Update ingestion.py * Refactor and clean up code formatting * modified workflow * Add graph creation functionality * Refactor KG parameters and logging. * review * up --------- Co-authored-by: emrgnt-cmplxty <68796651+emrgnt-cmplxty@users.noreply.github.com> Co-authored-by: emrgnt-cmplxty <owen@algofi.org> Co-authored-by: Manuel R. Ciosici <manuelrciosici@gmail.com> Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com>
* add update files workflow * rm ingestion pipeline * v0 restructure orch * kg orchestration * finish kg orchestration * update service * merge * cleanups * add hatchet api key setup * cleanup * add hatchet api key setup (#1037) * add hatchet api key setup * cleanup * fix merge * cleanups
* Update runners (#1007) * Check in logs --------- Co-authored-by: Nolan Tremelling <34580718+NolanTrem@users.noreply.github.com>
* Pull in subnet and graph PR * Add in templates
* dockerfile * Update ingestion file with new sample URL and enhance unstructured chunking configuration and error handling. * clean up * clean up dockerfile * up * Update sample file and clean code * Add hatchet-sdk dependency in project. * Update providers to include local option.
* Draft of file provider * Some cleanup * Regenearte lock * Stream it * Use document_id as primary key * Pydantic v2 * File provider finished
* Fix * Fix parsing pipeline * working
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.