[DOCS] Sample API Docs for review

[dbwiddis]

The below content is intended for publication to opensearch.org. See a similar structure at https://opensearch.org/docs/latest/ml-commons-plugin/index/

Pages are separated in this document by hard line breaks.

Please comment below on anything you think needs to be added/changed/deleted.

The link(s) to Sample Workflow Steps refer to #314

---

Flow Framework

The Flow Framework plugin simplifies complex setup and pre-processing tasks with templates for common use cases.

Automating Complex Setup Configuration

The current process of using Machine Learning (ML) offerings in OpenSearch, such as Conversational Agents and Query Generation, requires users to handle complex setup and pre-processing tasks which can be time-consuming and error-prone.

The Flow Framework plugin provides OpenSearch users with use case templates, which provide a compact description of the setup process in a JSON or YAML document. These templates describe configurations for automated workflows for conversational chat or query generation, configuring AI connectors, tools, agents and other components that prime OpenSearch as a backend to leverage generative models.

Key Features

• Use Case Templates: Get started with pre-defined templates that outline the setup process for your general use cases.
• Customizable Workflows: Customize the workflows in the template for your specific use case.
• Setup Automation: Easily configure AI connectors, tools, agents, and other components in a single API call.

For more information, see Flow Framework Overview.

Flow Framework provides its own set of REST APIs. For more information, see Flow Framework APIs.

For example workflow steps included in templates, see Example Workflow Templates. (Link TBD.)
For more information, see Integrating ML models.

---

Flow Framework Overview

Templates implement the workflow automation in Flow Framework. These templates can be in JSON or YAML format and describe one or more templates with a sequence of steps required in a particular use case. The Template includes:

• Metadata: A name, description, use case category, template version, and OpenSearch version compatibility range
• User Input: Parameters expected from the user which are common to all automation steps across all workflows, such as an index name
• Workflows: One or more workflows, containing the following elements:
  • User Input: Parameters expected from the user specific to steps in this workflow.
  • Workflow Steps: The workflow steps described as a directed acyclic graph (DAG). Nodes describe each step of the process, which may be executed in parallel. Edges sequence nodes to be executed after the previous step completes, and may be omitted for simplicity if the nodes are to be executed in sequential order.

Workflow Steps form basic “building blocks” for process automation. Most steps correspond directly to an OpenSearch or plugin API such as CRUD operations on ML Connectors, Models, and Agents. Some steps simplify the configuration of the body expected by these APIs when they would be repeated across multiple steps, such as Tools which may be used with multiple Agents.

Workflow Steps are actively being developed to expand the framework’s capabilities. Workflow Step (node) configuration includes the following fields:

• id: A user-provided ID for this step. This must be unique within a given workflow, and is useful for identifying resources created by that step. For example, a register_agent step may return an agent_id that has been registered. This ID allows users to determine which step produced which resource.
• type: The type of action to be taken, corresponding to the API it is used for, such as deploy_model. Multiple steps may share the same type, but must each have their own unique ID.
• previous node inputs: A mapping of an API body field name (such as model_id ) that will be produced as an output of a previous step in the workflow. For example, register_remote_model may produce a model_id that is required for a subsequent deploy_model step.
• user inputs: Other inputs supported by the corresponding API for this specific step. Some inputs are required for an API, while others are optional.

For some example workflow step implementations, see Example Workflow Steps.

---

Flow Framework APIs

Flow Framework supports the following APIs:

• Create Workflow
• Get Workflow
• Provision Workflow
• Get Workflow State
• Search Workflow
• Deprovision Workflow
• Delete Workflow

---

Create a Workflow

Creating a Workflow adds the content of a workflow template to the Flow Framework system index. Workflows may be in JSON (by specifying Content-Type: application/json) or YAML (by specifying Content-Type: application/yaml). By default, validation is performed on the workflow to help identify invalid configurations including:

• Workflow steps requiring an OpenSearch plugin which is not installed
• Workflow steps relying on previous node input that is provided by those steps
• Workflow step fields with invalid values
• Workflow graph (node/edge) configurations containing cycles or having duplicate IDs

Once a workflow is created, provide its workflow_id to other APIs.

Path and HTTP methods

POST /_plugins/_flow_framework/workflow

Request fields

The following table lists the available request fields.

Field	Data type	Required/Optional	Description
name	String	Required	The name of the workflow
description	String	Optional	A description of what the workflow does
use_case	String	Optional	A use case, which can be used with the Search API to find related workflows
version	Object	Optional	A key-value map with two fields: template identifying the template version, and compatibility identifying a list of minimum required versions.
workflows	Object	Optional	A map of workflows. Presently only the provision key is supported. The value for the workflow key is a key-value map which includes fields for user_params and lists of nodes and edges.

Example Request: Register and Deploy a Local Model (YAML)

YAML templates permit comments. Use Content-Type: application/yaml

# This name is required
name: createconnector-registerremotemodel-deploymodel
# Other fields are optional but useful
description: This template creates a connector to a remote model, registers it, and
  deploys that model
# Other templates with a similar use case can be searched
use_case: REMOTE_MODEL_DEPLOYMENT
version:
  # Templates may be versioned by their authors
  template: 1.0.0
  # Compatibility with OpenSearch 2.12.0 and higher and 3.0.0 and higher
  compatibility:
  - 2.12.0
  - 3.0.0
# One or more workflows can be included, presently only provision is supported
workflows:
  provision:
    # These nodes are the workflow steps corresponding to ML Commons APIs
    nodes:
    # This ID must be unique to this workflow
    - id: create_connector_1
      # There may be multiple steps with the same type
      type: create_connector
      # These inputs match the Create Connector API body
      user_inputs:
        name: OpenAI Chat Connector
        description: The connector to public OpenAI model service for GPT 3.5
        version: '1'
        protocol: http
        parameters:
          endpoint: api.openai.com
          model: gpt-3.5-turbo
        credential:
          openAI_key: '12345'
        actions:
        - action_type: predict
          method: POST
          url: https://${parameters.endpoint}/v1/chat/completions
    # This ID must be unique to this workflow
    - id: register_model_2
      type: register_remote_model
      # This step needs the connector_id produced as an output of the previous step
      previous_node_inputs:
        create_connector_1: connector_id
      # These inputs match the Register Model API body
      user_inputs:
        name: openAI-gpt-3.5-turbo
        function_name: remote
        description: test model
    # This ID must be unique to this workflow
    - id: deploy_model_3
      type: deploy_model
      # This step needs the model_id produced as an output of the previous step
      previous_node_inputs:
        register_model_2: model_id
    # Since the nodes are executed in the order specified, these are optional
    # In more complex workflows these edges define a directed graph
    edges:
    - source: create_connector_1
      dest: register_model_2
    - source: register_model_2
      dest: deploy_model_3

Example Request: Register and Deploy a Local Model (JSON)

This matches the YAML template above. Use Content-Type: application/json

{
  "name": "createconnector-registerremotemodel-deploymodel",
  "description": "This template creates a connector to a remote model, registers it, and deploys that model",
  "use_case": "REMOTE_MODEL_DEPLOYMENT",
  "version": {
    "template": "1.0.0",
    "compatibility": [
      "2.12.0",
      "3.0.0"
    ]
  },
  "workflows": {
    "provision": {
      "nodes": [
        {
          "id": "create_connector_1",
          "type": "create_connector",
          "user_inputs": {
            "name": "OpenAI Chat Connector",
            "description": "The connector to public OpenAI model service for GPT 3.5",
            "version": "1",
            "protocol": "http",
            "parameters": {
              "endpoint": "api.openai.com",
              "model": "gpt-3.5-turbo"
            },
            "credential": {
              "openAI_key": "12345"
            },
            "actions": [
              {
                "action_type": "predict",
                "method": "POST",
                "url": "https://${parameters.endpoint}/v1/chat/completions"
              }
            ]
          }
        },
        {
          "id": "register_model_2",
          "type": "register_remote_model",
          "previous_node_inputs": {
            "create_connector_1": "connector_id"
          },
          "user_inputs": {
            "name": "openAI-gpt-3.5-turbo",
            "function_name": "remote",
            "description": "test model"
          }
        },
        {
          "id": "deploy_model_3",
          "type": "deploy_model",
          "previous_node_inputs": {
            "register_model_2": "model_id"
          }
        }
      ],
      "edges": [
        {
          "source": "create_connector_1",
          "dest": "register_model_2"
        },
        {
          "source": "register_model_2",
          "dest": "deploy_model_3"
        }
      ]
    }
  }
}

Example Response

Opensearch responds with the workflow_id .

{
  "workflow_id" : "8xL8bowB8y25Tqfenm50"
}

Once you have created a workflow, you can use other Workflow APIs with the workflow_id .

---

Get a Workflow

The stored Workflow template may be retrieved by the Get API.

Path and HTTP methods

GET /_plugins/_flow_framework/workflow/<workflow_id>

Example Request:

GET /_plugins/_flow_framework/workflow/8xL8bowB8y25Tqfenm50

Example Response

Opensearch responds with the stored template which will contain the same content as the body of the Create Workflow request. The format of the template will match the specified Content-Type of the request, either Content-Type: application/json or Content-Type: application/yaml . The order of fields in the returned template may not exactly match the original template but will function identically.

---

Provision a Workflow

The workflows field of the template may contain multiple workflows. The workflow with the key provision can be executed with this API.

Provisioning refers to a one-time setup process, usually performed by a cluster administrator to create resources which will be used by end users.

Path and HTTP methods

POST /_plugins/_flow_framework/workflow/<workflow_id>/_provision

Example Request:

POST /_plugins/_flow_framework/workflow/8xL8bowB8y25Tqfenm50/_provision

Example Response

Opensearch responds with the same workflow_id that was used in the request.

{
  "workflow_id" : "8xL8bowB8y25Tqfenm50"
}

To obtain the status of the provisioning, query the Get Workflow State API.

---

Get a Workflow State

Provisioning a Workflow may take time, particularly when associated with OpenSearch indexing operations. The Workflow State API permits monitoring the deployment until it is complete.

Path and HTTP methods

GET /_plugins/_flow_framework/workflow/<workflow_id>/_status

Example Request:

GET /_plugins/_flow_framework/workflow/8xL8bowB8y25Tqfenm50/_status

Example Response

Opensearch responds with a summary of the provisioning status and a list of created resources.

Before provisioning has started, no resources are included.

{
  "workflow_id" : "8xL8bowB8y25Tqfenm50",
  "state": "NOT_STARTED"
}

While provisioning is in progress, a partial list may be returned.

{
  "workflow_id" : "8xL8bowB8y25Tqfenm50",
  "state": "PROVISIONING",
  "resources_created": [
    {
      "workflow_step_name": "create_connector",
      "workflow_step_id": "create_connector_1",
      "connector_id": "NdjCQYwBLmvn802B0IwE"
    }
  ]
}

Upon completion the full list is returned.

{
  "workflow_id" : "8xL8bowB8y25Tqfenm50",
  "state": "COMPLETED",
  "resources_created": [
    {
      "workflow_step_name": "create_connector",
      "workflow_step_id": "create_connector_1",
      "connector_id": "NdjCQYwBLmvn802B0IwE"
    },
    {
      "workflow_step_name": "register_remote_model",
      "workflow_step_id": "register_model_2",
      "model_id": "N9jCQYwBLmvn802B0oyh"
    }
  ]
}

---

Search for a Workflow

Created workflows may be retrieved by their workflow_id or searched for using a query matching a field. The use_case field is intended for use searching for similar workflows.

Path and HTTP methods

GET /_plugins/_flow_framework/workflow/_search
POST /_plugins/_flow_framework/workflow/_search

Example Request:

This request returns all created workflows

GET /_plugins/_flow_framework/workflow/_search

{
    "query": {
    "match_all": {}
  }
}

This request returns all workflows with a use_case of REMOTE_MODEL_DEPLOYMENT

GET /_plugins/_flow_framework/workflow/_search

{
    "query": {
    "match": {
      "use_case": "REMOTE_MODEL_DEPLOYMENT"
    }
  }
}

Opensearch responds with a list of workflow templates matching the search parameters.

---

Deprovision a Workflow

When a workflow is no longer required, its resources may be deprovisioned. Most workflow steps which create a resource have a corresponding workflow step to reverse that action. When executing this API, the original workflow template is parsed and the corresponding steps to deprovision resources are executed.

The workflow executes in the reverse order of provisioning and includes retry attempts when failures occur due to resource dependencies, such preventing deleting a registered model if it is still deployed. I

Path and HTTP methods

POST /_plugins/_flow_framework/workflow/<workflow_id>/_deprovision

Example Request:

POST /_plugins/_flow_framework/workflow/8xL8bowB8y25Tqfenm50/_deprovision

Example Response

If deprovisioning is successful, OpenSearch responds with the same workflow_id that was used in the request.

{
  "workflow_id" : "8xL8bowB8y25Tqfenm50"
}

If deprovisioning did not completely remove all resources, OpenSearch responds with a 202 (ACCEPTED) Status and identifies the resources which were not deprovisioned.

{
    "error": "Failed to deprovision some resources: [connector_id Lw7PX4wBfVtHp98y06wV]."
}

In some cases, the failure was due to another dependent resource which took some time to be removed, and the same request may be attempted again.

To obtain the status of the deprovisioning, query the Get Workflow State API. Upon success, the workflow returns to a NOT_STARTED state. If some resources have not yet been removed, they are provided in the response.

---

Delete a Workflow

When a workflow template is no longer needed, it may be deleted.

Note that deleting a workflow does not delete its resources, and prevents further use of the deprovision API to remove those resources.

Path and HTTP methods

DELETE /_plugins/_flow_framework/workflow/<workflow_id>

Example Request:

DELETE /_plugins/_flow_framework/workflow/8xL8bowB8y25Tqfenm50

Example Response

If the workflow exists, a delete response is returned giving the status of the deletion, with a result of deleted on success, or not_found if the workflow does not exist (may have already been deleted).

{
  "_index": ".plugins-flow_framework-templates",
  "_id": "8xL8bowB8y25Tqfenm50",
  "_version": 2,
  "result": "deleted",
  "_shards": {
    "total": 1,
    "successful": 1,
    "failed": 0
  },
  "_seq_no": 2,
  "_primary_term": 1
}