From dc8658c7259adb8de08c31b58c03a07409e2a73d Mon Sep 17 00:00:00 2001 From: Daniel Mangum Date: Wed, 31 Jul 2024 08:18:41 -0400 Subject: [PATCH 1/5] Add Replicate webhook destination example Adds an example webhook destination pipeline for Replicate. Signed-off-by: Daniel Mangum --- docs/data-routing/5-examples/18-replicate.md | 31 ++++++++++++++++++++ 1 file changed, 31 insertions(+) create mode 100644 docs/data-routing/5-examples/18-replicate.md diff --git a/docs/data-routing/5-examples/18-replicate.md b/docs/data-routing/5-examples/18-replicate.md new file mode 100644 index 00000000..4f7a1385 --- /dev/null +++ b/docs/data-routing/5-examples/18-replicate.md @@ -0,0 +1,31 @@ +--- +title: Replicate +--- +import Pipeline from '@site/src/components/usethispipeline' + +[Replicate](https://replicate.com/) is an AI platform that makes machine +learning accessible to every software developer by allowing for running and +fine-tuning open source and custom models at scale. Existing models on Replicate +suppport use cases that range from video generation to optical character +recognition (OCR). + +The following example demonstrates captioning images streamed to Golioth using +the [Salesforce BLIP model](https://replicate.com/salesforce/blip). The PNG +image is submitted in the [`webhook` +destination](/data-routing/destinations/webhook) request by encoding as a [Data +URL](https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/Data_URLs), +then embedding in a JSON object. + +Predictions on Replicate are asynchronous, meaning that the result is not +accessible directly in Pipelines. Results can be viewed in the Replicate +console, or a [webhook +URL](https://replicate.com/docs/reference/http#predictions.create--webhook) can +be included in the destination body to instruct webhook to deliver the result to +the specified URL after the prediction has completed. + +:::info Reminder +Make sure to create a [secret](/data-routing/secrets) named +`REPLICATE_TOKEN` in the format `Bearer `. +::: + + From 7e403a5a820ec0f09ddd1050fd68fac5ac788c72 Mon Sep 17 00:00:00 2001 From: Daniel Mangum Date: Wed, 31 Jul 2024 08:19:16 -0400 Subject: [PATCH 2/5] Add example hugging face pipeline Adds an example pipeline for using hugging face inference API via webhook transformer. Signed-off-by: Daniel Mangum --- .../5-examples/19-hugging-face.md | 56 +++++++++++++++++++ 1 file changed, 56 insertions(+) create mode 100644 docs/data-routing/5-examples/19-hugging-face.md diff --git a/docs/data-routing/5-examples/19-hugging-face.md b/docs/data-routing/5-examples/19-hugging-face.md new file mode 100644 index 00000000..a204976b --- /dev/null +++ b/docs/data-routing/5-examples/19-hugging-face.md @@ -0,0 +1,56 @@ +--- +title: Hugging Face +--- +import Pipeline from '@site/src/components/usethispipeline' + +[Hugging Face](https://huggingface.co/) is a platform where the machine learning +community collaborates on models, datasets, and applications. Hugging Face's +[dedicated Inference +Endpoints](https://huggingface.co/inference-endpoints/dedicated) and [Serverless +Inference API](https://huggingface.co/docs/api-inference/index) allow for +running any model via an HTTP REST API, which can be accessed from Pipelines +using the [`webhook` transformer](/data-routing/transformers/webhook). + +:::warning +The Serverless Inference API is free but rate-limited. It should only be used +for testing scenarios, and it is expected that some operations may be dropped if +the specified model is not already [loaded in +memory](https://huggingface.co/docs/api-inference/quicktour#model-loading-and-latency). +Inference Endpoints should be used in production scenarios. +::: + +The following example demonstrates using the Hugging Face Serverless Inference +API to perform sentiment analysis of audio samples streamed to Golioth. The +[Hubert-Large for Emotion Recognition +model](https://huggingface.co/superb/hubert-large-superb-er) is targeted. +Sentiment analysis results are delivered to [LightDB +Stream](/application-services/lightdb-stream) as a JSON payload. + +Example sentiment analysis result: +```json +[ + { + "score": 0.6310836672782898, + "label": "neu" + }, + { + "score": 0.2573806643486023, + "label": "sad" + }, + { + "score": 0.09393830597400665, + "label": "hap" + }, + { + "score": 0.017597444355487823, + "label": "ang" + } +] +``` + +:::info Reminder +Make sure to create a [secret](/data-routing/secrets) named `HUGGING_FACE_TOKEN` +in the format `Bearer `. +::: + + From 8aa7cac54b2dc5f0bb0fd1d177cfec7c9ceb6c09 Mon Sep 17 00:00:00 2001 From: Daniel Mangum Date: Wed, 31 Jul 2024 08:19:47 -0400 Subject: [PATCH 3/5] Add OpenAI pipeline example Adds an example pipeline for using OpenAI via a webhook transformer. Signed-off-by: Daniel Mangum --- docs/data-routing/5-examples/20-openai.md | 26 +++++++++++++++++++++++ 1 file changed, 26 insertions(+) create mode 100644 docs/data-routing/5-examples/20-openai.md diff --git a/docs/data-routing/5-examples/20-openai.md b/docs/data-routing/5-examples/20-openai.md new file mode 100644 index 00000000..dcef3c76 --- /dev/null +++ b/docs/data-routing/5-examples/20-openai.md @@ -0,0 +1,26 @@ +--- +title: OpenAI +--- +import Pipeline from '@site/src/components/usethispipeline' + +[OpenAI](https://openai.com/) is a platform that enables developers to build and +scale AI experiences powered by industry-leading models and tools. OpenAI's +suite of proprietary models support text generation, speech-to-text, vision, and +more. + +The following example demonstrates leveraging OpenAI's [GPT-4o +mini](https://platform.openai.com/docs/models/gpt-4o-mini) model via the +Pipelines [`webhook` transformer](/data-routing/transformers/webhook) to +describe an image streamed to Golioth. The image is embedded as a [Data +URL](https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/Data_URLs) +alongside the chosen model and a prompt. The resulting description is passed to +a [Slack webhook](/data-routing/examples/slack-webhook) via the [`webhook` +destination](/data-routing/destinations/webhook). + +:::info Reminder +Make sure to create a [secret](/data-routing/secrets) named `OPENAI_TOKEN` in +the format `Bearer `. In this example a `SLACK_WEBHOOK` secret is also +required. +::: + + From 6a0ea4855a6280e21d43c0a222c0cf5a30840f38 Mon Sep 17 00:00:00 2001 From: Daniel Mangum Date: Wed, 31 Jul 2024 08:20:20 -0400 Subject: [PATCH 4/5] Add pipeline example for anthropic Adds an example pipeline for using anthropic via webhook transformer. Signed-off-by: Daniel Mangum --- docs/data-routing/5-examples/21-anthropic.md | 24 ++++++++++++++++++++ 1 file changed, 24 insertions(+) create mode 100644 docs/data-routing/5-examples/21-anthropic.md diff --git a/docs/data-routing/5-examples/21-anthropic.md b/docs/data-routing/5-examples/21-anthropic.md new file mode 100644 index 00000000..808fe3ec --- /dev/null +++ b/docs/data-routing/5-examples/21-anthropic.md @@ -0,0 +1,24 @@ +--- +title: Anthropic +--- +import Pipeline from '@site/src/components/usethispipeline' + +[Anthropic](https://www.anthropic.com/) is a platform that aims to build +frontier AI systems that are reliable, interpretable, and steerable. Anthropic's +models are provided via [Claude](https://www.anthropic.com/claude), an AI system +that is capable of advanced reasoning, vision analysis, code generation, and +more. + +The following example demonstrates a simple chat interaction with Claude, in +which a text payload from a device is used as a prompt for the [Claude 3.5 +Sonnet model](https://docs.anthropic.com/en/docs/about-claude/models). The +resulting chat answer is delivered to [LightDB +Stream](/application-services/lightdb-stream) as a JSON object. + +:::info Reminder +Make sure to create a [secret](/data-routing/secrets) named `ANTHROPIC_API_KEY` +with a valid [API +key](https://docs.anthropic.com/en/api/getting-started#authentication). +::: + + From 96d509da0840078899791ab265dd4ac3b0fae221 Mon Sep 17 00:00:00 2001 From: Daniel Mangum Date: Wed, 31 Jul 2024 09:00:46 -0400 Subject: [PATCH 5/5] Add usage notice on examples overview Adds a notice on the examples overview to indicate that they may incur usage. Signed-off-by: Daniel Mangum --- docs/data-routing/5-examples/README.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/docs/data-routing/5-examples/README.md b/docs/data-routing/5-examples/README.md index 8c1b0db2..3765565a 100644 --- a/docs/data-routing/5-examples/README.md +++ b/docs/data-routing/5-examples/README.md @@ -6,6 +6,11 @@ The best way to get started with Pipelines is to take an example and modify it to fit your use case. This section contains a set of example pipelines along with explanations of their functionality. +:::usage +Many examples incur usage costs after exceeding the free tier. See [Golioth +pricing](https://golioth.io/pricing) for more information. +::: + ## Share Your Pipelines You can click the share button next to a pipeline in the console to copy a