diff --git a/building-blocks/blocks/adb/integrate-genai/images/azure-resource-info.png b/building-blocks/blocks/adb/integrate-genai/images/azure-resource-info.png new file mode 100644 index 00000000..5ac13f2a Binary files /dev/null and b/building-blocks/blocks/adb/integrate-genai/images/azure-resource-info.png differ diff --git a/building-blocks/blocks/adb/integrate-genai/integrate-genai.md b/building-blocks/blocks/adb/integrate-genai/integrate-genai.md index 89c5ca06..a35b692a 100644 --- a/building-blocks/blocks/adb/integrate-genai/integrate-genai.md +++ b/building-blocks/blocks/adb/integrate-genai/integrate-genai.md @@ -1,204 +1,317 @@ -# Integrate OCI Generative AI with Autonomous Database +# Integrate GenAI models with Autonomous Database ## Introduction -LLMs can produce incredibly creative responses to prompts, generate SQL from natural language, and so much more. In order to be most effective, you want to leverage LLMs with your organization's private data. The first step is to integrate your models with Autonomous Database. Here, we will connect OCI Generative AI models with Autonomous Database. +LLMs can produce incredibly creative responses to prompts, generate SQL from natural language, and so much more. In order to be most effective, you want to leverage LLMs with your organization's private data. The first step is to integrate your models with Autonomous Database. -You can use different LLMs with Autonomous Database. This lets you pick the best model for your use case. Select AI profiles encapsulate the connections to each model; you specify which profile to use when generating results. In this lab, you will enable the user **`MOVIESTREAM`** to use multiple LLMs offered by the OCI Generative AI service. +You can use different LLMs with Autonomous Database. This lets you pick the best model for your use case. Select AI profiles encapsulate the connections to each model; you specify which profile to use when generating results. In this lab, you will enable the user **`MOVIESTREAM`** to connect to models from Oracle OCI GenAI, Azure OpenAI and Google Gemini. Estimated Time: 10 minutes. ### Objectives In this lab, you will: -* Enable use of Resource Principals for MOVIESTREAM user -* Create Select AI Profile for OCI GenAI +* Connect Autonomous Database to an AI provider * Test the AI profile ### Prerequisites - This lab requires the completion of the previous labs that deployed your Autonomous Database. -## Task 1: Enable resource principal usage for user MOVIESTREAM +## Task 1: Log into SQL Worksheet +>**Note:** the **MOVIESTREAM** user and its tables were created as part of the setup. You can find the Moviestream password by navigating to **Developer Services** from the Navigation menu. Next, click **Resource Manager** > **Stacks** > Select the stack we created, **Deploy-ChatDB-Autonomous-Database...** > Select the job we created, **ormjob2024117214431** > Select **Outputs** under **Resources**. -The MOVIESTREAM user will connect to OCI Generative AI using a resource principal (for more information, see [Use Resource Principal to Access Oracle Cloud Infrastructure Resources](https://docs.oracle.com/en/cloud/paas/autonomous-database/serverless/adbsb/resource-principal.html#GUID-E283804C-F266-4DFB-A9CF-B098A21E496A)). This means that the ADB resource (i.e. your ADB instance) needs access to OCI Generative AI. The OCI policy you created in the previous lab authorized that access. +![Moviestream password](./images/moviestream-output-pswd.png "") -In ADB, enable the use of resource principals for the MOVIESTREAM user. +[](include:connect-with-sql-worksheet-non-admin.md) +## Task 2: Connect Autonomous Database to an AI provider and create an AI Profile +### Background +There are 3 things to do in order to connect Autonomous Database to an AI provider: +1. Grant the MOVIESTREAM user network access to the AI provider endpoint +2. Create a credential containing the secret used to sign requests to the AI provider +3. Create a Select AI profile (see below for more details) -1. From the **Console,** Open the **Navigation** menu and click **Oracle Database.** Under **Oracle Database,** click **Autonomous Database.** +>**Note:** All of these steps have already been done for accessing OCI GenAI when you deployed your Autonomous Database. You can review the deployment steps below. You will need to execute these steps when connecting to non-Oracle AI providers. -2. On the **Autonomous Database** page, click your **MovieStreamWorkshop** ADB instance. +A Select AI profile encapsulates connection information for an AI provider. This includes: +1. A security credential (e.g. the resource principal for OCI GenAI or a credential that captures a secret for a 3rd party AI provider) +2. The name of the provider +3. The name of the LLM (optional) +4. A list of target tables that will be used for natural language queries (required when using NL2SQL) + +You can create as many profiles as you need, which is useful when comparing the quality or performance of the results of different models. - ![Open ADB](./images/click-adb.png "") +For a complete list of the Select AI profile attributes, see the [DBMS\_CLOUD\_AI\_Package] (https://docs.oracle.com/en/cloud/paas/autonomous-database/serverless/adbsb/dbms-cloud-ai-package.html#GUID-D51B04DE-233B-48A2-BBFA-3AAB18D8C35C) in the Using Oracle Autonomous Database Serverless documentation. -3. On the **Autonomous Database details** page, click the **Database actions** drop-down list, and then click SQL. - ![Select SQL option](./images/sql-option.png "") ->**Note:** The setup script automatically enabled Resource Principals usage for MovieStream. The following 2 steps are optional. +### Connect to one of the following AI providers using each provider's default model: -4. The ADB deployment lab already set up resource principals in your database. To better understand the use of resource principals, let's undo that enablement and recreate it. Disable the use of resource principals for the MOVIESTREAM user. Copy and paste the following code into your SQL Worksheet, and then click the **Run Script (F5)** icon in the Worksheet toolbar. +
+ **OCI Generative AI** (default) +1. Grant MOVIESTREAM network access to the endpoint: - ``` + This is not required when using OCI GenAI +2. Create a credential: + + The workshop deployment step already set up a resource principal for your database and enabled the MOVIESTREAM user to use that principal (for more information, see [Use Resource Principal to Access Oracle Cloud Infrastructure Resources](https://docs.oracle.com/en/cloud/paas/autonomous-database/serverless/adbsb/resource-principal.html#GUID-E283804C-F266-4DFB-A9CF-B098A21E496A)). This means that the ADB resource (i.e. your ADB instance) needs access to OCI Generative AI. The OCI policy you created in the previous lab authorized that access. +3. Create a Select AI profile + ```sql - begin - dbms_cloud_admin.disable_resource_principal(username => 'MOVIESTREAM'); - end; - / - - ``` + begin + -- Drop the profile in case it already exists + dbms_cloud_ai.drop_profile( + profile_name => 'genai', + force => true + ); - ![Disable resource principal](./images/disable-resource.png "") + -- Create an AI profile that uses the default LLAMA model on OCI + dbms_cloud_ai.create_profile( + profile_name => 'genai', + attributes => + '{"provider": "oci", + "credential_name": "OCI$RESOURCE_PRINCIPAL", + "comments":"true", + "object_list": [ + {"owner": "MOVIESTREAM", "name": "GENRE"}, + {"owner": "MOVIESTREAM", "name": "CUSTOMER"}, + {"owner": "MOVIESTREAM", "name": "PIZZA_SHOP"}, + {"owner": "MOVIESTREAM", "name": "STREAMS"}, + {"owner": "MOVIESTREAM", "name": "MOVIES"}, + {"owner": "MOVIESTREAM", "name": "ACTORS"} + ] + }' + ); + + end; + / + + ``` +
-5. Now let's re-enable the use of the resource principal by the MOVIESTREAM user. Copy and paste the following code into your SQL Worksheet, and then click the **Run Script** icon. +
+ **OpenAI** +You will need a [paid OpenAI account](https://platform.openai.com/docs/overview) and [an API key](https://platform.openai.com/docs/quickstart) in order to use OpenAI GPT models. - ``` +1. Grant MOVIESTREAM network access to the OpenAI endpoint. + ```sql - begin - dbms_cloud_admin.enable_resource_principal(username => 'MOVIESTREAM'); - end; - / + BEGIN + DBMS_NETWORK_ACL_ADMIN.APPEND_HOST_ACE( + host => 'api.openai.com', + ace => xs$ace_type(privilege_list => xs$name_list('http'), + principal_name => 'MOVIESTREAM', + principal_type => xs_acl.ptype_db) + ); + END; + / ``` - - ![Enable resource principal](./images/enable-resource-principal.png "") -6. Sign out of the **ADMIN** user. On the **Oracle Database Actions | SQL banner**, click the drop-down list next to the **ADMIN** user, and then select **Sign Out** from the drop-down menu. - - ![Sign out from Admin](./images/sign-out.png "") - -## Task 2: Create an AI Profile for OCI Generative AI - -A Select AI profile encapsulates connection information for an AI provider. This includes: -1. A security credential (e.g. the resource principal) -2. The name of the provider -3. The name of the LLM (optional) -4. A list of target tables that will be used for natural language queries (required when using NL2SQL) - -You can create as many profiles as you need, which is useful when comparing the quality of the results of different models. - -For a complete list of the Select AI profile attributes, see the [DBMS\_CLOUD\_AI\_Package] (https://docs.oracle.com/en/cloud/paas/autonomous-database/serverless/adbsb/dbms-cloud-ai-package.html#GUID-D51B04DE-233B-48A2-BBFA-3AAB18D8C35C) in the Using Oracle Autonomous Database Serverless documentation. +2. Create a credential + ```sql + + BEGIN + dbms_cloud.create_credential ( + credential_name => 'openai_credential', + username => 'openai', + password => 'your-api-key-goes-here' + ); + END; + / + + ``` ->**Note:** The deployment script created a Select AI profile using the code below: - -``` -begin - - -- Create an AI profile that uses the default LLAMA model on OCI - dbms_cloud_ai.create_profile( - profile_name => 'genai', - attributes => - '{"provider": "oci", - "credential_name": "OCI$RESOURCE_PRINCIPAL", - "comments":"true", - "object_list": [ - {"owner": "MOVIESTREAM", "name": "GENRE"}, - {"owner": "MOVIESTREAM", "name": "CUSTOMER"}, - {"owner": "MOVIESTREAM", "name": "PIZZA_SHOP"}, - {"owner": "MOVIESTREAM", "name": "STREAMS"}, - {"owner": "MOVIESTREAM", "name": "MOVIES"}, - {"owner": "MOVIESTREAM", "name": "ACTORS"} - ] - }' - ); +3. Create a Select AI profile + ```sql + + BEGIN + -- Drop the profile in case it already exists + dbms_cloud_ai.drop_profile( + profile_name => 'genai', + force => true + ); -end; -/ -``` - -1. Sign into the SQL worksheet as the **MOVIESTREAM** user with the password **watchS0meMovies#**, and then navigate to the SQL Worksheet. + -- Create an AI profile that uses the default GPT model + dbms_cloud_ai.create_profile( + profile_name => 'genai', + attributes => + '{"provider": "openai", + "credential_name": "openai_credential", + "comments":"true", + "object_list": [ + {"owner": "MOVIESTREAM", "name": "GENRE"}, + {"owner": "MOVIESTREAM", "name": "CUSTOMER"}, + {"owner": "MOVIESTREAM", "name": "PIZZA_SHOP"}, + {"owner": "MOVIESTREAM", "name": "STREAMS"}, + {"owner": "MOVIESTREAM", "name": "MOVIES"}, + {"owner": "MOVIESTREAM", "name": "ACTORS"} + ] + }' + ); + END; + / + + ``` +
- >**Note:** the **MOVIESTREAM** user was created as part of the setup and tables that were created in that schema. You can find the Moviestream password by navigating to **Developer Services** from the Navigation menu. Next, click **Resource Manager** > **Stacks** > Select the stack we created, **Deploy-ChatDB-Autonomous-Database...** > Select the job we created, **ormjob2024117214431** > Select **Outputs** under **Resources**. - ![Moviestream password](./images/moviestream-output-pswd.png "") +
+ **Azure OpenAI** +You will need an Azure subscription and an [Azure OpenAI resource](https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/create-resource?pivots=web-portal) in order to use the GPT models. After creating the Azure OpenAI resource, navigate to the resource page and select **Resource Management -> Keys and Endpoint**. Copy its **Endpoint** (the server name only - not including "https://" or "/") and a **KEY**. For example, consider a resource named **openaigpt40** (your name will be different): +![Azure OpenAI resource](images/azure-resource-info.png) -2. Create an AI profile for the **Meta Llama 3 model**. Copy and paste the following code into your SQL Worksheet, and then click the **Run Script** icon. +You will also need the Azure OpenAI deployment name. In that same portal page, navigate to **Resource Management -> Model Deployments** and click **Manage Deployments**. Copy the **Deployment name** for your GPT model. - ``` +1. Grant MOVIESTREAM network access to the Azure OpenAI resource endpoint. + ```sql BEGIN - -- drops the profile if it already exists - DBMS_CLOUD_AI.drop_profile( - profile_name => 'ociai_llama', - force => true - ); + DBMS_NETWORK_ACL_ADMIN.APPEND_HOST_ACE( + host => 'your-azure-openai-resource-endpoint', + ace => xs$ace_type(privilege_list => xs$name_list('http'), + principal_name => 'MOVIESTREAM', + principal_type => xs_acl.ptype_db) + ); + END; + / + + ``` - -- Meta Llama 3 (this is the default model, so you could skip the model attribute if you like) - DBMS_CLOUD_AI.create_profile ( - profile_name => 'ociai_llama', - attributes => - '{"provider": "oci", - "credential_name": "OCI$RESOURCE_PRINCIPAL", - "object_list": [ - {"owner": "moviestream", "name": "GENRE"}, - {"owner": "moviestream", "name": "CUSTOMER"}, - {"owner": "moviestream", "name": "PIZZA_SHOP"}, - {"owner": "moviestream", "name": "STREAMS"}, - {"owner": "moviestream", "name": "MOVIES"}, - {"owner": "moviestream", "name": "ACTORS"} - ], - "model": "meta.llama-3-70b-instruct" - }'); - END; - / +2. Create a credential + ```sql + + BEGIN + dbms_cloud.create_credential ( + credential_name => 'azure_openai_credential', + username => 'azure', + password => 'your-api-key-goes-here' + ); + END; + / ``` - ![Create AI profile](./images/create-llama.png "") -3. Create an AI profile for the **Cohere model**. This model will not be used for SQL generation - it will only be used for generating innovative content. Copy and paste the following code into your SQL Worksheet, and then click the **Run Script** icon. - +3. Create a Select AI profile + ```sql + + begin + -- Drop the profile in case it already exists + dbms_cloud_ai.drop_profile( + profile_name => 'genai', + force => true + ); + + -- Create an AI profile that uses the default Gemini model + dbms_cloud_ai.create_profile( + profile_name => 'genai', + attributes => + '{"provider": "azure", + "azure_resource_name": "your-azure-resource-name", + "azure_deployment_name": "your-azure-deployment-name", + "credential_name": "azure_openai_credential", + "comments":"true", + "object_list": [ + {"owner": "MOVIESTREAM", "name": "GENRE"}, + {"owner": "MOVIESTREAM", "name": "CUSTOMER"}, + {"owner": "MOVIESTREAM", "name": "PIZZA_SHOP"}, + {"owner": "MOVIESTREAM", "name": "STREAMS"}, + {"owner": "MOVIESTREAM", "name": "MOVIES"}, + {"owner": "MOVIESTREAM", "name": "ACTORS"} + ] + }' + ); + end; + / + ``` +
+ +
+ **Google Gemini** +You will need a [Google AI Studio account](https://ai.google.dev) and [an API key](https://aistudio.google.com/app/apikey) in order to use Google Gemini. + +1. Grant MOVIESTREAM network access to the Google Gemini endpoint. + ```sql - BEGIN - -- drops the profile if it already exists - DBMS_CLOUD_AI.drop_profile(profile_name => 'OCIAI_COHERE', force => true); - - DBMS_CLOUD_AI.CREATE_PROFILE( - profile_name => 'OCIAI_COHERE', - attributes => '{"provider": "oci", - "credential_name": "OCI$RESOURCE_PRINCIPAL", - "model": "cohere.command-r-plus", - "oci_apiformat":"COHERE" - }'); + BEGIN + DBMS_NETWORK_ACL_ADMIN.APPEND_HOST_ACE( + host => 'generativelanguage.googleapis.com', + ace => xs$ace_type(privilege_list => xs$name_list('http'), + principal_name => 'MOVIESTREAM', + principal_type => xs_acl.ptype_db) + ); END; - / + / ``` - ![Create AI profile](./images/create-cohere.png "") +2. Create a credential + ```sql + + BEGIN + dbms_cloud.create_credential ( + credential_name => 'gemini_credential', + username => 'google', + password => 'your-api-key-goes-here' + ); + END; + / + + ``` -## Task 3: Test the AI profile +3. Create a Select AI profile + ```sql + + begin + -- Drop the profile in case it already exists + dbms_cloud_ai.drop_profile( + profile_name => 'genai', + force => true + ); + + -- Create an AI profile that uses the default Gemini model + dbms_cloud_ai.create_profile( + profile_name => 'genai', + attributes => + '{"provider": "google", + "credential_name": "gemini_credential", + "comments":"true", + "object_list": [ + {"owner": "MOVIESTREAM", "name": "GENRE"}, + {"owner": "MOVIESTREAM", "name": "CUSTOMER"}, + {"owner": "MOVIESTREAM", "name": "PIZZA_SHOP"}, + {"owner": "MOVIESTREAM", "name": "STREAMS"}, + {"owner": "MOVIESTREAM", "name": "MOVIES"}, + {"owner": "MOVIESTREAM", "name": "ACTORS"} + ] + }' + ); + end; + / + + ``` +
-We will use the PL/SQL API to generate a response from the Cohere model. This example is using the **chat** action. It is not using any private data coming from your database. -1. Test the LLM and learn about Autonomous Database as the MOVIESTREAM user using the **Cohere model**. Copy and paste the following code into your SQL Worksheet, and then click the **Run Script** icon. +## Task 3: Test the AI profile - ``` - - SELECT DBMS_CLOUD_AI.GENERATE( - prompt => 'what is oracle autonomous database', - profile_name => 'OCIAI_COHERE', - action => 'chat') - FROM dual; - - ``` - ![Test the LLM](./images/cohere-output.png "") +We will use the Select AI PL/SQL API to generate a response from the AI model. This example is using the **chat** action. It is not using any private data coming from your database. -2. Compare the Cohere model to the **Llama model**. Copy and paste the following code into your SQL Worksheet, and then click the **Run Script** icon. +1. Test the LLM and learn about Autonomous Database as the MOVIESTREAM user. Copy and paste the following code into your SQL Worksheet, and then click the **Run Script** icon. Your answer may differ! ``` SELECT DBMS_CLOUD_AI.GENERATE( prompt => 'what is oracle autonomous database', - profile_name => 'OCIAI_LLAMA', + profile_name => 'GENAI', action => 'chat') FROM dual; ``` - ![Generate sentence-like response](./images/llama-chat.png "") - + ![Test the LLM](./images/cohere-output.png "") ## Summary -You learned how to integrate Autonomous Database with OCI Generative AI. And, you chatted with different models hosted on OCI Generative AI. Next, let's see how to use our private data with LLMs. +You learned how to integrate Autonomous Database with your AI provider. And, you asked the model your first question using the "chat"action. Next, let's see how to use our private data with LLMs. You may now proceed to the next lab. @@ -211,14 +324,7 @@ You may now proceed to the next lab. ## Acknowledgements * **Author:** Marty Gubar, Product Management - * **Contributors:** - * Stephen Stuart, Cloud Engineer - * Nicholas Cusato, Cloud Engineer - * Lauran K. Serhal, Consulting User Assistance Developer - * Olivia Maxwell, Cloud Engineer - * Taylor Rees, Cloud Engineer - * Joanna Espinosa, Cloud Engineer -* **Last Updated By/Date:** Marty Gubar, July 2024 + * **Last Updated By/Date:** Marty Gubar, July 2024 Data about movies in this workshop were sourced from **Wikipedia**.