Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat: provide constructs to simplify genAI patterns implementation #534

Open
vgkowski opened this issue Apr 4, 2024 · 1 comment
Open

Comments

@vgkowski
Copy link
Contributor

vgkowski commented Apr 4, 2024

The most common patterns for GenAI applications are RAG and LLM fine-tuning/training. DSF can bring some GenAI constructs to accelerate the implementation of these patterns. In details we have identified 3 constructs that could help:

  • The RAG pipeline to ingest data into vector databases and provide semantic context to GenAI applications
  • The Data API pipeline to expose data to GenAI application and provide situational context to GenAI applications
  • The data preparation pipeline to prepare data for model training or fine-tuning
@krokoko
Copy link

krokoko commented Apr 16, 2024

@vgkowski

  • The RAG pipeline: in the Generative AI CDK constructs, we do have 2 ingestion pipelines for RAG: Data ingestion pipeline - OpenSearch and Data ingestion pipeline - Kendra, what is the gap between what is available and your requirements ?
  • The Data API pipeline: do you have an example of this ?
  • The data preparation pipeline: do you have an idea of the requirements/architecture for this one ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Backlog
Development

No branches or pull requests

2 participants