Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: sentence transformers #517

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from
Draft

feat: sentence transformers #517

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
7f2fb87
feat: shared torch base encoder and st encoder
jamescalam Jan 19, 2025
2eb709c
feat: local intro notebook
jamescalam Jan 21, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
313 changes: 313 additions & 0 deletions docs/00a-introduction-local.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,313 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "K7NsuSPNf3px"
},
"source": [
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/aurelio-labs/semantic-router/blob/main/docs/00-introduction.ipynb) [![Open nbviewer](https://raw.githubusercontent.com/pinecone-io/examples/master/assets/nbviewer-shield.svg)](https://nbviewer.org/github/aurelio-labs/semantic-router/blob/main/docs/00-introduction.ipynb)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Am2hmLzTf3py"
},
"source": [
"# Semantic Router Intro"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "k1nRRAbYf3py"
},
"source": [
"The Semantic Router library can be used as a super fast decision making layer on top of LLMs. That means rather than waiting on a slow agent to decide what to do, we can use the magic of semantic vector space to make routes. Cutting decision making time down from seconds to milliseconds.\n",
"\n",
"In this notebook we will be introducing the library (as done in the `00-introduction.ipynb` notebook) but using the `LocalEncoder` class, allowing us to run the library locally without the need for any APIs or external services."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "NggrMQP2f3py"
},
"source": [
"## Getting Started"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "9zP-l_T7f3py"
},
"source": [
"We start by installing the library:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "4YI81tu0f3pz"
},
"outputs": [],
"source": [
"!pip install -qU \"semantic-router==0.1.0.dev6[local]\""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "HfB8252ff3pz"
},
"source": [
"We start by defining a dictionary mapping routes to example phrases that should trigger those routes."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "lslfqYOEf3pz",
"outputId": "c13e3e77-310c-4b86-e291-4b6005d698bd"
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"c:\\Users\\Siraj\\Documents\\Personal\\Work\\Aurelio\\Virtual Environments\\semantic_router_3\\Lib\\site-packages\\tqdm\\auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
" from .autonotebook import tqdm as notebook_tqdm\n"
]
}
],
"source": [
"from semantic_router import Route\n",
"\n",
"politics = Route(\n",
" name=\"politics\",\n",
" utterances=[\n",
" \"isn't politics the best thing ever\",\n",
" \"why don't you tell me about your political opinions\",\n",
" \"don't you just love the president\",\n",
" \"don't you just hate the president\",\n",
" \"they're going to destroy this country!\",\n",
" \"they will save the country!\",\n",
" ],\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "WYLHUDa1f3p0"
},
"source": [
"Let's define another for good measure:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "LAdY1jdxf3p0"
},
"outputs": [],
"source": [
"chitchat = Route(\n",
" name=\"chitchat\",\n",
" utterances=[\n",
" \"how's the weather today?\",\n",
" \"how are things going?\",\n",
" \"lovely weather today\",\n",
" \"the weather is horrendous\",\n",
" \"let's go to the chippy\",\n",
" ],\n",
")\n",
"\n",
"routes = [politics, chitchat]"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "ReN59ieGf3p0"
},
"source": [
"Now we initialize our encoder. Under-the-hood we're using the `sentence-transformers` library, which supports loading encoders from the HuggingFace Hub. We'll be using Nvidia's [nvidia/NV-Embed-v2](https://huggingface.co/nvidia/NV-Embed-v2) encoder"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "MF47W_Sof3p2"
},
"outputs": [],
"source": [
"from semantic_router.encoders import LocalEncoder\n",
"\n",
"encoder = LocalEncoder(name=\"nvidia/NV-Embed-v2\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "lYuLO0l9f3p3"
},
"source": [
"Now we define the `Router`. When called, the router will consume text (a query) and output the category (`Route`) it belongs to — to initialize a `Router` we need our `encoder` model and a list of `routes`."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "dh1U8IDOf3p3",
"outputId": "872810da-956a-47af-a91f-217ce351a88b"
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[32m2024-05-07 15:02:46 INFO semantic_router.utils.logger local\u001b[0m\n"
]
}
],
"source": [
"from semantic_router.routers import SemanticRouter\n",
"\n",
"sr = SemanticRouter(encoder=encoder, routes=routes, auto_sync=\"local\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Xj32uEF-f3p3"
},
"source": [
"Now we can test it:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "fIXOjRp9f3p3",
"outputId": "8b9b5746-ae7c-43bb-d84f-5fa7c30e423e"
},
"outputs": [
{
"data": {
"text/plain": [
"RouteChoice(name='politics', function_call=None, similarity_score=None)"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"sr(\"don't you love politics?\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "0UN2mKvjf3p4",
"outputId": "062f9499-7db3-49d2-81ef-e7d5dc9a88f6"
},
"outputs": [
{
"data": {
"text/plain": [
"RouteChoice(name='chitchat', function_call=None, similarity_score=None)"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"sr(\"how's the weather today?\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "NHZWZKoTf3p4"
},
"source": [
"Both are classified accurately, what if we send a query that is unrelated to our existing `Route` objects?"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "0WnvGJByf3p4",
"outputId": "4496e9b2-7cd8-4466-fe1a-3e6f5cf30b0d"
},
"outputs": [
{
"data": {
"text/plain": [
"RouteChoice(name=None, function_call=None, similarity_score=None)"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"sr(\"I'm interested in learning about llama 2\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"With this we see `None` is returned, ie no routes were matched."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"---"
]
}
],
"metadata": {
"colab": {
"provenance": []
},
"kernelspec": {
"display_name": "decision-layer",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.4"
}
},
"nbformat": 4,
"nbformat_minor": 0
}
3 changes: 2 additions & 1 deletion pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,8 +41,9 @@ sphinxawesome-theme = {version = "^5.2.0", optional = true}
tornado = {version = "^6.4.2", optional = true}

[tool.poetry.extras]
hybrid = ["pinecone-text"]
hybrid = ["pinecone-text"] # we need to drop this dependency
local = ["torch", "transformers", "tokenizers", "huggingface-hub", "llama-cpp-python"]
# local-cpu = ["torch"] https://download.pytorch.org/whl/cpu https://stackoverflow.com/questions/77205123/how-do-i-slim-down-sberts-sentencer-transformer-library
pinecone = ["pinecone"]
vision = ["torch", "torchvision", "transformers", "pillow"]
mistralai = ["mistralai"]
Expand Down
8 changes: 8 additions & 0 deletions semantic_router/encoders/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,10 @@
from semantic_router.encoders.google import GoogleEncoder
from semantic_router.encoders.huggingface import HuggingFaceEncoder
from semantic_router.encoders.huggingface import HFEndpointEncoder
from semantic_router.encoders.local import LocalEncoder
from semantic_router.encoders.mistral import MistralEncoder
from semantic_router.encoders.openai import OpenAIEncoder
from semantic_router.encoders.sentence_transformers import STEncoder
from semantic_router.encoders.tfidf import TfidfEncoder
from semantic_router.encoders.vit import VitEncoder
from semantic_router.encoders.zure import AzureOpenAIEncoder
Expand All @@ -23,8 +25,10 @@
"SparseEncoder",
"AzureOpenAIEncoder",
"CohereEncoder",
"LocalEncoder",
"OpenAIEncoder",
"BM25Encoder",
"STEncoder",
"TfidfEncoder",
"FastEmbedEncoder",
"HuggingFaceEncoder",
Expand All @@ -50,6 +54,10 @@ def __init__(self, type: str, name: Optional[str]):
self.model = AzureOpenAIEncoder(model=name)
elif self.type == EncoderType.COHERE:
self.model = CohereEncoder(name=name)
elif self.type == EncoderType.SENTENCE_TRANSFORMERS:
self.model = STEncoder(name=name)
elif self.type == EncoderType.LOCAL:
self.model = LocalEncoder(name=name)
elif self.type == EncoderType.OPENAI:
self.model = OpenAIEncoder(name=name)
elif self.type == EncoderType.AURELIO:
Expand Down
Loading
Loading