Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature implementation] Enterprise RAG application upgrade functionality #34

Open
jpiaseck opened this issue Feb 27, 2025 · 0 comments
Labels
EnterpriseRAG Hackathon Issue created for OSS Hackathon

Comments

@jpiaseck
Copy link
Collaborator

Currently we are not able to upgrade Enterprise RAG from older version to never version.

There is possibility to run:

 ./install_chatqna.sh --auth --telemetry --deploy xeon_torch_llm_guard --ui --upgrade

but it would not work as:
would fail during updating deployment as istio is blocking update

Digest: sha256:4732b249409962ce3fc98fc2ec6fb1ee245273aa91c0423a3ceb389348a3634c
-->helm upgrade or installation of "fingerprint" in "fingerprint" namespace in progress ...
-->helm upgrade or installation of "fingerprint" in "fingerprint" namespace finished successfully.
Error from server (InternalError): error when applying patch:
{"metadata":{"annotations":{"kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"gmc.opea.io/v1alpha3\",\"kind\":\"GMConnector\",\"metadata\":{\"annotations\":{},\"labels\":{\"app.kubernetes.io/managed-by\":\"kus
tomize\",\"app.kubernetes.io/name\":\"gmconnector\",\"gmc/platform\":\"xeon\"},\"name\":\"chatqa\",\"namespace\":\"chatqa\"},\"spec\":{\"nodes\":{\"root\":{\"routerType\":\"Sequence\",\"steps\":[{\"internalService\":{\"config\":{
\"endpoint\":\"/v1/system_fingerprint/append_arguments\"},\"serviceName\":\"fgp-svc\"},\"name\":\"Fingerprint\"},{\"data\":\"$response\",\"internalService\":{\"config\":{\"EMBEDDING_CONNECTOR\":\"langchain\",\"EMBEDDING_MODEL_NAM
E\":\"BAAI/bge-base-en-v1.5\",\"EMBEDDING_MODEL_SERVER\":\"torchserve\",\"EMBEDDING_MODEL_SERVER_ENDPOINT\":\"torchserve-embedding-svc\",\"endpoint\":\"/v1/embeddings\"},\"serviceName\":\"embedding-svc\"},\"name\":\"Embedding\"},
{\"internalService\":{\"config\":{\"TORCHSERVE_MODEL_NAME\":\"BAAI/bge-base-en-v1.5\"},\"isDownstreamService\":true,\"serviceName\":\"torchserve-embedding-svc\"},\"name\":\"TorchserveEmbedding\"},{\"data\":\"$response\",\"interna
lService\":{\"config\":{\"EMBED_MODEL\":\"BAAI/bge-base-en-v1.5\",\"endpoint\":\"/v1/retrieval\"},\"serviceName\":\"retriever-svc\"},\"name\":\"Retriever\"},{\"internalService\":{\"isDownstreamService\":true,\"serviceName\":\"red
is-vector-db\"},\"name\":\"VectorDB\"},{\"data\":\"$response\",\"internalService\":{\"config\":{\"RERANKING_SERVICE_ENDPOINT\":\"tei-reranking-svc\",\"endpoint\":\"/v1/reranking\"},\"serviceName\":\"reranking-svc\"},\"name\":\"Re
ranking\"},{\"internalService\":{\"config\":{\"endpoint\":\"/rerank\"},\"isDownstreamService\":true,\"serviceName\":\"tei-reranking-svc\"},\"name\":\"TeiReranking\"},{\"data\":\"$response\",\"dependency\":\"Hard\",\"internalServi
ce\":{\"config\":{\"endpoint\":\"/v1/prompt_template\"},\"serviceName\":\"prompt-template-svc\"},\"name\":\"PromptTemplate\"},{\"internalService\":{\"config\":{\"endpoint\":\"/v1/completions\"},\"isDownstreamService\":true,\"serv
iceName\":\"vllm-service-m\"},\"name\":\"VLLM\"},{\"data\":\"$response\",\"internalService\":{\"config\":{\"LLM_MODEL_SERVER\":\"vllm\",\"LLM_MODEL_SERVER_ENDPOINT\":\"vllm-service-m\",\"endpoint\":\"/v1/chat/completions\"},\"ser
viceName\":\"llm-svc\"},\"name\":\"Llm\"}]}},\"routerConfig\":{\"name\":\"router\",\"serviceName\":\"router-service\"}}}\n"}},"spec":{"nodes":{"root":{"steps":[{"internalService":{"config":{"endpoint":"/v1/system_fingerprint/appe
nd_arguments"},"serviceName":"fgp-svc"},"name":"Fingerprint"},{"data":"$response","internalService":{"config":{"EMBEDDING_CONNECTOR":"langchain","EMBEDDING_MODEL_NAME":"BAAI/bge-base-en-v1.5","EMBEDDING_MODEL_SERVER":"torchserve"
,"EMBEDDING_MODEL_SERVER_ENDPOINT":"torchserve-embedding-svc","endpoint":"/v1/embeddings"},"serviceName":"embedding-svc"},"name":"Embedding"},{"internalService":{"config":{"TORCHSERVE_MODEL_NAME":"BAAI/bge-base-en-v1.5"},"isDowns
treamService":true,"serviceName":"torchserve-embedding-svc"},"name":"TorchserveEmbedding"},{"data":"$response","internalService":{"config":{"EMBED_MODEL":"BAAI/bge-base-en-v1.5","endpoint":"/v1/retrieval"},"serviceName":"retrieve
r-svc"},"name":"Retriever"},{"internalService":{"isDownstreamService":true,"serviceName":"redis-vector-db"},"name":"VectorDB"},{"data":"$response","internalService":{"config":{"RERANKING_SERVICE_ENDPOINT":"tei-reranking-svc","end
point":"/v1/reranking"},"serviceName":"reranking-svc"},"name":"Reranking"},{"internalService":{"config":{"endpoint":"/rerank"},"isDownstreamService":true,"serviceName":"tei-reranking-svc"},"name":"TeiReranking"},{"data":"$respons
e","dependency":"Hard","internalService":{"config":{"endpoint":"/v1/prompt_template"},"serviceName":"prompt-template-svc"},"name":"PromptTemplate"},{"internalService":{"config":{"endpoint":"/v1/completions"},"isDownstreamService"
:true,"serviceName":"vllm-service-m"},"name":"VLLM"},{"data":"$response","internalService":{"config":{"LLM_MODEL_SERVER":"vllm","LLM_MODEL_SERVER_ENDPOINT":"vllm-service-m","endpoint":"/v1/chat/completions"},"serviceName":"llm-sv
c"},"name":"Llm"}]}}}}
to:
Resource: "gmc.opea.io/v1alpha3, Resource=gmconnectors", GroupVersionKind: "gmc.opea.io/v1alpha3, Kind=GMConnector"
Name: "chatqa", Namespace: "chatqa"
for: "/home/ubuntu/main/applications.ai.enterprise-rag.enterprise-ai-solution/deployment/microservices-connector/config/samples/chatQnA_xeon_torch.yaml": error when patching "/home/ubuntu/main/applications.ai.enterprise-rag.enter
prise-ai-solution/deployment/microservices-connector/config/samples/chatQnA_xeon_torch.yaml": Internal error occurred: failed calling webhook "vgmconnector.gmc.opea.io": failed to call webhook: Post "https://gmc-contoller.system.
svc:9443/validate-gmc-opea-io-v1alpha3-gmconnector?timeout=10s": read tcp 10.233.45.144:50982->10.233.45.144:9443: read: connection reset by peer

similar issue would be faced when trying to update telemetry.

Goal is to have ability to update configurations/erag version without crash

@aalbersk aalbersk added the EnterpriseRAG Hackathon Issue created for OSS Hackathon label Feb 27, 2025
@aalbersk aalbersk changed the title Enterprise RAG application upgrade functionality [Feature implementation] Enterprise RAG application upgrade functionality Feb 27, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
EnterpriseRAG Hackathon Issue created for OSS Hackathon
Projects
None yet
Development

No branches or pull requests

2 participants