Intel® AI for Enterprise RAG simplifies transforming your enterprise data into actionable insights. Powered by Intel® Gaudi® AI accelerators and Intel® Xeon® processors, it integrates components from industry partners to offer a streamlined approach to deploying enterprise solutions. It scales seamlessly with proven orchestration frameworks, providing the flexibility and choice your enterprise needs.
Building on the strong foundation of OPEA, Intel® AI for Enterprise RAG extends this base with key features that enhance scalability, security, and user experience. These include integrated Service Mesh capabilities for seamless integration with modern service-based architectures, production-ready validation for pipeline reliability, and a feature-rich UI for RAG as a service to manage and monitor workflows easily. Additionally, Intel® and partner support provide access to a broad ecosystem of solutions, combined with integrated IAM with UI and application, ensuring secure and compliant operations. Programmable guardrails enable fine-grained control over pipeline behavior, allowing for customized security and compliance settings.
Note
The video provided above showcases the beta release of our project. As we transition to the 1.0 version, users can anticipate an improved UI design along with other minor enhancements. While many features remain consistent with the beta, the 1.0 release will offer a more refined user experience.
ChatQnA
The ChatQnA solution uses retrieval augmented generation (RAG) architecture, quickly becoming the industry standard for chatbot development. It combines the benefits of a knowledge base (via a vector store) and generative models to reduce hallucinations, maintain up-to-date information, and leverage domain-specific knowledge.
For the complete microservices architecture, refer here
- Deployment Guide explains how to install and configure Intel® AI for Enterprise RAG for your needs.
| Category | Details |
|---|---|
| Operating System | Ubuntu 20.04/22.04 |
| Hardware Platforms | 4th Gen Intel® Xeon® Scalable processors 5th Gen Intel® Xeon® Scalable processors 6th Gen Intel® Xeon® Scalable processors 3rd Gen Intel® Xeon® Scalable processors and Intel® Gaudi® 2 AI Accelerator 4th Gen Intel® Xeon® Scalable processors and Intel® Gaudi® 2 AI Accelerator 6th Gen Intel® Xeon® Scalable processors and Intel® Gaudi® 3 AI Accelerator |
| Kubernetes Version | 1.29.5 1.29.12 1.30.8 1.31.4 |
| Gaudi Firmware Version | 1.19.2 |
To deploy the solution on a platform with Gaudi® AI Accelerator we need to have access to instance with minimal requirements:
- logical cores: A minimum of
48logical cores - RAM memory: A minimum of
250GBof RAM though this is highly dependent on database size - Disk Space:
1TBof disk space is generally recommended, though this is highly dependent on the model size and database size - Gaudi cards:
8 - Latest Gaudi driver: To check your Gaudi version,
run hl-smi. If Gaudi version doesn't match the required version, upgrade it by following this tutorial.
If you don't have a Gaudi® AI Accelerator, you can request these instances in Intel® Tiber™ AI Cloud to run Intel® AI for Enterprise RAG.
- visit Intel® Tiber™ AI Cloud.
- In the left pane select
Catalog > Hardware. - Select
Gaudi® 2 Deep Learning ServerorGaudi® 2 Deep Learning Server - Dell. - Select the Machine image - for example:
ubuntu-2204-gaudi2-1.17.0-vm-v4withArchitecture: X86_64 (Baremetal only). Please note that minor version tags may change over time. - Upload your public key and launch the instance
- Navigate to the
Instancespage and verify that the machine has reached its ready state, then click on "How to Connect via SSH" to configure your machine correctly for further installation.
To deploy the solution on a platform using 4th or 5th generation Intel® Xeon® processors, you will need:
- access to any platform with Intel® Xeon® Scalable processors that meet bellow requirements:
- logical cores: A minimum of
80logical cores - RAM memory: A minimum of
250GBof RAM - Disk Space:
500GBof disk space is generally recommended, though this is highly dependent on the model size
Refer to the prerequisites guide for detailed instructions to install the components mentioned below:
- Kubernetes Cluster: Access to a Kubernetes v1.29-v1.31 cluster
- CSI Driver: The K8s cluster must have the CSI driver installed. Users can define their own CSI driver that will be used during EnterpriseRAG install; however StorageClass provided by CSI driver should support ReadWriteMany(RWX) in case of using a multi-node cluster.
- Current solution was tested on a single node using the CSI driver local-path-provisioner, with
local_path_provisioner_claim_rootset to/mnt. For an example of how to set up Kubernetes via Kubespray, refer to the prerequisites guide: CSI Driver. - Operating System: Ubuntu 20.04/22.04
- Hugging Face Model Access: Ensure you have the necessary access to download and use the chosen Hugging Face model. This default model used is
Mixtral-8x7Bfor which access needs to be requested. Visit Mixtral-8x7B to apply for access.
- Gaudi Software Stack: Verify that your setup uses a valid software stack for Gaudi accelerators, see Gaudi support matrix. Note that running LLM on a CPU is possible but will significantly reduce performance.
- Prepared Gaudi Node: Please refer to the Gaudi Software Stack section of the prerequisites section.
cd deployment
./one_click_chatqna.sh -g HUG_TOKEN [-p HTTP_PROXY] [-u HTTPS_PROXY] [-n NO_PROXY] -d [PIPELINE] -t [TAG] -y [REGISTRY] [--features FEATURES]Note
In case we are deploying the solution on Xeon, please change the PIPELINE parameter to the pipeline dedicated for Xeon, for example, xeon_torch_llm_guard. For more information, refer to Configure Pipeline.
Proxy variables are optional. Refer Deployment if you prefer to install with multiple options.
cd deployment
./install_chatqna.sh -caSubmit questions, feature requests, and bug reports on the GitHub Issues page.
Intel® AI for Enterprise RAG is licensed under the Apache License Version 2.0. Refer to the "LICENSE" file for the full license text and copyright notice.
This distribution includes third-party software governed by separate license terms. This third-party software, even if included with the distribution of the Intel software, may be governed by separate license terms, including without limitation, third-party license terms, other Intel software license terms, and open-source software license terms. These separate license terms govern your use of the third-party programs as set forth in the "THIRD-PARTY-PROGRAMS" file.
The Security Policy outlines our guidelines and procedures for ensuring the highest level of security and trust for our users who consume Intel® AI for Enterprise RAG.
Intel, the Intel logo, OpenVINO, the OpenVINO logo, Pentium, Xeon, and Gaudi are trademarks of Intel Corporation or its subsidiaries.
- Other names and brands may be claimed as the property of others.
© Intel Corporation