-
Notifications
You must be signed in to change notification settings - Fork 27
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
5b620c0
commit e526927
Showing
6 changed files
with
81 additions
and
0 deletions.
There are no files selected for viewing
76 changes: 76 additions & 0 deletions
76
docs/integrations/vector-databases/postgres-vector-database.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,76 @@ | ||
# Integrating with Postgres Vector Database in Orkes Conductor | ||
|
||
To effectively utilize AI and LLM tasks in Orkes Conductor, it's essential to integrate your Conductor cluster with the necessary Vector Database models. | ||
|
||
PGvector is a popular open-source extension for PostgreSQL known for its efficiency and versatility in handling large-scale vector data. This integration empowers you to access, query, and manipulate vector data effectively, enhancing the capabilities of Orkes Conductor in various natural language processing and artificial intelligence applications. | ||
|
||
This guide will provide the steps for integrating Postgres as a Vector Database with Orkes Conductor. | ||
|
||
## Steps to integrate with Postgres Database | ||
|
||
Before beginning to integrate with Postgres, you need to get specific configuration parameters from Postgres, such as database username, password, and URL. Now, let’s integrate this with Orkes Conductor. | ||
|
||
1. Navigate to **Integrations** from the left menu on your Orkes Conductor console. | ||
2. Click **+New integration** button from the top-right of your window. | ||
3. Under the **Vector Databases** section, choose **Postgres Vector Database**. | ||
4. Click **+Add** and provide the following parameters: | ||
|
||
<p align="center"><img src="/content/img/create-new-postgres-integration.png" alt="Create Postgres Integration" width="40%" height="auto"></img></p> | ||
|
||
| Parameters | Description | | ||
| ---------- | ----------- | | ||
| Integration name | Provide a name for the integration. | | ||
| Postgres Database User | Provide the database username. | | ||
| Postgres Database Password | Provide the password associated with the database username. | | ||
| Postgres Database URL | Provide the database URL.<br/><br/>**Note**: To use this integration for JDBC tasks, ensure that you provide the JDBC connection string in the format: **jdbc:database://url/databaseName**. | | ||
| Embedding dimensions | Provide the number of dimensions in the embeddings. The embedding dimensions often depend on the AI model used to generate the embeddings. | | ||
| Distance metric | Choose the distance metric, which is a metric to measure the similarity or distance between vectors. Supported values:<ul><li>Cosine Similarity</li><li>Euclidean Distance</li><li>Inner Product</li></ul> | | ||
| Indexing method | Choose the indexing method. Supported methods:<ul><li>hnsv - Hierarchical Navigable Small World graphs</li><li>ivfflat - Inverted File Flat</li></ul> | | ||
| Number of inverted lists to create for ivfflat index | The number of inverted lists to create when using the **_ivfflat_** indexing method. This parameter is only relevant if **_ivfflat_** is chosen as the indexing method. | | ||
| Description | Provide a description of your integration. | | ||
|
||
5. You can toggle-on the **Active** button to activate the integration instantly. | ||
6. Click **Save**. | ||
|
||
## Adding Indexes to Postgres Integration | ||
|
||
Now that you have integrated your Conductor console with the Postgres provider, the next step is to integrate with the specific indexes. | ||
|
||
To add a new index to the Postgres integration: | ||
|
||
1. Navigate to the integrations page and click the '+' button next to the integration you created. | ||
|
||
<p align="center"><img src="/content/img/create-new-pgvector-integration-index.png" alt="Create Indexes for Postgres Integration" width="100%" height="auto"></img></p> | ||
|
||
2. Click **+New Index**. | ||
3. Provide the index name and an optional description. | ||
|
||
<p align="center"><img src="/content/img/create-new-postgres-integration-index-model.png" alt="Create Indexes for Postgres Integration Model" width="60%" height="auto"></img></p> | ||
|
||
4. Turn on the **Active** button to enable the model immediately. | ||
5. Click **Save**. | ||
|
||
This ensures the integration model is saved for future use in LLM tasks within Orkes Conductor. | ||
|
||
## RBAC - Governance on who can use Integrations | ||
|
||
The integration with the required models is now ready. Next, we should determine the access control to these models. | ||
|
||
The permission can be granted to applications/groups within the Orkes Conductor console. | ||
|
||
To provide explicit permission to Groups: | ||
|
||
1. Navigate to **Access Control > Groups** from the left menu on your Orkes Conductor console. | ||
2. Create a new group or choose an existing group. | ||
3. Under the **Permissions** section, click **+Add Permission**. | ||
4. Under the **Integrations** tab, select the required integrations with the required permissions. | ||
|
||
<p align="center"><img src="/content/img/add-integration-permission-for-postgresvector-db.png" alt="Add Permissions for Postgres Vector Database Integration" width="60%" height="auto"></img></p> | ||
|
||
5. Click **Add Permissions**. This ensures that all the group members can access these integration models in their workflows. | ||
|
||
Similarly, you can also provide permissions to [applications](https://orkes.io/content/access-control-and-security/applications#configuring-application). | ||
|
||
:::info | ||
Once the integration is set up and the required access is granted, the integration can be utilized for [JDBC](https://orkes.io/content/reference-docs/system-tasks/jdbc) tasks within Orkes Conductor. | ||
::: |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.