03_App_on_Ray_Serve.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f99a1f24-6473-4915-9d27-704ee97b9239",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from typing import Optional, Any, Dict\n",
    "from operator import add\n",
    "import requests, json\n",
    "from starlette.requests import Request\n",
    "import numpy as np\n",
    "import torch\n",
    "\n",
    "from sentence_transformers import SentenceTransformer\n",
    "from langchain.embeddings.base import Embeddings\n",
    "from langchain.text_splitter import RecursiveCharacterTextSplitter\n",
    "from langchain.vectorstores import FAISS\n",
    "from langchain.document_loaders import WikipediaLoader\n",
    "from langchain import HuggingFacePipeline\n",
    "from langchain.chains.question_answering import load_qa_chain\n",
    "from langchain.prompts import PromptTemplate\n",
    "\n",
    "from transformers import pipeline as hf_pipeline\n",
    "\n",
    "import ray\n",
    "from ray import serve"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "7a7354b9-aed8-410e-ab01-8c33db4b1a43",
   "metadata": {
    "tags": []
   },
   "source": [
    "# Productionizing LLM Q&A Application with Ray Serve\n",
    "\n",
    "In this notebook, we'll see how to productionize our Q&A application and its database service\n",
    "\n",
    "<div class=\"alert alert-block alert-info\">\n",
    "\n",
    "__Initial Productionizing Roadmap__\n",
    "1. Port the vector db service to a Ray Serve deployment and test it out\n",
    "1. Port the Q&A service to a deployment and integrate with the vector db service\n",
    "1. Extend the vector db service:\n",
    "    1. Specify an autoscaling configuration so that the service can adjust to traffic load\n",
    "    1. Build the index faster using Ray tasks to scale out\n",
    "1. Extend the Q&A service by adding replicas (service instances)\n",
    "</div>\n",
    "\n",
    "<img src='https://technical-training-assets.s3.us-west-2.amazonaws.com/LLMs/QA_App.png' width=700/>"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "a2a4245c-c65a-44e5-b2a0-653c6dff0da5",
   "metadata": {},
   "source": [
    "## Create a Ray Serve deployment for vector db functionality"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4d7cbdcb-1bd8-499f-a57e-5c7e2df6f45c",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "class LocalHuggingFaceEmbeddings(Embeddings):\n",
    "    def __init__(self, model_id):\n",
    "        self.model = SentenceTransformer(model_id)\n",
    "\n",
    "    def embed_documents(self, texts: list[str]) -> list[list[float]]:\n",
    "        embeddings = self.model.encode(texts)\n",
    "        return embeddings\n",
    "\n",
    "    def embed_query(self, text: str) -> list[float]:\n",
    "        embedding = self.model.encode(text)\n",
    "        return list(map(float, embedding))"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "cb3a7e83-6414-4576-97ee-4dd06039cb3e",
   "metadata": {},
   "source": [
    "__To convert our vector db logic into a Serve deployment__\n",
    "\n",
    "<div class=\"alert alert-block alert-info\">\n",
    "    \n",
    "1. Wrap the ad-hoc login in a Python class\n",
    "1. Add the `@serve.deployment` decorator\n",
    "1. Add conditional logic to create the index and store it locally if it has not already been created on the node\n",
    "1. Prepare to run the service -- effectively set up a constructor call -- by creating a \"bound deployment\"\n",
    "\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0aef7c83-025b-42fb-b627-308c463c584a",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# place logic inside a Python class and add Ray Serve deployment decorator\n",
    "@serve.deployment\n",
    "class VectorDBDeployment:\n",
    "    FAISS_INDEX_PATH = \"/home/ray/faiss_index\"\n",
    "\n",
    "    def __init__(self):\n",
    "        self.embeddings = LocalHuggingFaceEmbeddings(\"multi-qa-mpnet-base-dot-v1\")\n",
    "\n",
    "        # try to load the index from the local node's filesystem\n",
    "        try:\n",
    "            self.db = FAISS.load_local(self.FAISS_INDEX_PATH, self.embeddings)\n",
    "        except:\n",
    "            # if the index is not local, run the setup logic\n",
    "            self.setup_db()\n",
    "\n",
    "    def setup_db(self):\n",
    "        topics = [\"The Eras Tour\", \"2023 XFL season\"]\n",
    "        loaders = [WikipediaLoader(query=topic, load_max_docs=20) for topic in topics]\n",
    "        text_splitter = RecursiveCharacterTextSplitter(chunk_size=300, chunk_overlap=20, length_function=len,)\n",
    "        docs = add(*[loader.load() for loader in loaders])\n",
    "        print([d.metadata[\"title\"] for d in docs])\n",
    "        chunks = text_splitter.create_documents(\n",
    "            [doc.page_content for doc in docs], metadatas=[doc.metadata for doc in docs]\n",
    "        )\n",
    "        self.db = FAISS.from_documents(chunks, self.embeddings)\n",
    "        self.db.save_local(self.FAISS_INDEX_PATH)\n",
    "\n",
    "    def similarity_search(self, query):\n",
    "        return self.db.similarity_search(query)\n",
    "\n",
    "# create a \"bound deployment\" -- a wrapped (deferred) call to the constructor, which will be invoked by Ray Serve\n",
    "vecdb_deployment = VectorDBDeployment.bind()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "31b598e6-0da4-4e10-b367-1c9bd0455a40",
   "metadata": {},
   "source": [
    "We can test this service out by itself by launching a Ray Serve application"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "60a2fbc6-e653-45ce-b599-f7699ad3b2c5",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "handle = serve.run(vecdb_deployment, name=\"db\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "d984bc09-09af-4bef-a1b6-f5bcdee26308",
   "metadata": {},
   "source": [
    "Run a quick test of this module by itself"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "793d7ce9-0ce7-470c-87ee-9ba67f4eb476",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "ray.get(handle.similarity_search.remote(\"When did the XFL start?\"))"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "3ae1cbf5-f938-4adc-9997-a62b3a2f23db",
   "metadata": {},
   "source": [
    "We'll shut this down, since the db service will not be public in our final app"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "61bcb0c7-601a-4f79-81f9-573f42d4932f",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "serve.delete(\"db\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "84d41e84-f7c6-4576-b9c5-51f631009049",
   "metadata": {},
   "source": [
    "## Create a Ray Serve deployment for Q&A service functionality"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "62d7c0e1-47ae-475e-98d8-f336ae227408",
   "metadata": {},
   "source": [
    "We'll prepare the helper code and data (e.g., the prompt template)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "92bde69c-5642-4611-ab98-76cca879a137",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "class StableLMPipeline(HuggingFacePipeline):\n",
    "    # Class is temporary, we are working with the authors of LangChain to make these unnecessary.\n",
    "\n",
    "    def _call(self, prompt: str, stop: Optional[list[str]] = None) -> str:\n",
    "        response = self.pipeline(prompt, temperature=0.1, max_new_tokens=256, do_sample=True)\n",
    "        print(f\"Response is: {response}\")\n",
    "        text = response[0][\"generated_text\"][len(prompt) :]\n",
    "        return text\n",
    "\n",
    "    @classmethod\n",
    "    def from_model_id(cls, model_id: str, task: str, device: Optional[str] = None, model_kwargs: Optional[dict] = None, **kwargs: Any):\n",
    "        pipeline = hf_pipeline(model=model_id, task=task, device=device, model_kwargs=model_kwargs,)\n",
    "        return cls(pipeline=pipeline, model_id=model_id, model_kwargs=model_kwargs, **kwargs,)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "af642edf-f6e4-47d9-ad62-964298cc7414",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "template = \"\"\"\n",
    "<|SYSTEM|># StableLM Tuned (Alpha version)\n",
    "- You are a helpful, polite, fact-based agent for answering questions. \n",
    "- Your answers include enough detail for someone to follow through on your suggestions. \n",
    "<|USER|>\n",
    "If you don't know the answer, just say that you don't know. Don't try to make up an answer.\n",
    "Please answer the following question using the context provided. \n",
    "\n",
    "CONTEXT: \n",
    "{context}\n",
    "=========\n",
    "QUESTION: {question} \n",
    "ANSWER: <|ASSISTANT|>\"\"\"\n",
    "\n",
    "PROMPT = PromptTemplate(template=template, input_variables=[\"context\", \"question\"])"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "e16941b0-9891-42ec-a2b5-1d62f195b5a0",
   "metadata": {},
   "source": [
    "__To create a Ray Serve deployment from our existing QA class__\n",
    "<div class=\"alert alert-block alert-info\">\n",
    "\n",
    "1. Add the `@serve.deployment` decorator\n",
    "1. Specify `num_gpus` as 1.0 to ensure that this deployment always has access to a GPU (and all of its memory)\n",
    "1. Add the vector db service as a contructor param\n",
    "1. Prepare to provide the db service to the deployment constructor by passing it into `.bind(...)`\n",
    "\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d0dd2ba8-0d80-493e-8df4-b32f315a39a4",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# add decorator and specify GPU resource requirement\n",
    "@serve.deployment(ray_actor_options={\"num_gpus\": 1.0})\n",
    "class QADeployment:\n",
    "    # take vector db bound deployment instance as a constructor param\n",
    "    def __init__(self, db):\n",
    "        self.embeddings = LocalHuggingFaceEmbeddings(\"multi-qa-mpnet-base-dot-v1\")\n",
    "        self.db = db\n",
    "        self.llm = StableLMPipeline.from_model_id(\n",
    "            model_id=\"stabilityai/stablelm-tuned-alpha-7b\",\n",
    "            task=\"text-generation\",\n",
    "            model_kwargs={\n",
    "                \"torch_dtype\": torch.float16,\n",
    "                \"device_map\": \"auto\",\n",
    "                \"cache_dir\": \"/mnt/local_storage\",\n",
    "            },\n",
    "        )\n",
    "        self.chain = load_qa_chain(llm=self.llm, chain_type=\"stuff\", prompt=PROMPT)\n",
    "\n",
    "    async def qa(self, query):\n",
    "        # when we run the QADeployment, the vector db bound deployment will become a \"live\" serve handle, so that we can call remote methods on it\n",
    "        search_results_ref = await self.db.similarity_search.remote(query)\n",
    "\n",
    "        # return value from the call is a Ray ObjectRef (future/promise) and we await it to get the actual Python object we need\n",
    "        search_results = await search_results_ref\n",
    "        print(f\"Results from db are: {search_results}\")\n",
    "        result = self.chain({\"input_documents\": search_results, \"question\": query})\n",
    "        print(f\"Result is: {result}\")\n",
    "        return result[\"output_text\"]\n",
    "\n",
    "# create bound deployment, taking the vector db bound deployment as a parameter\n",
    "qa_deployment = QADeployment.bind(vecdb_deployment)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "58f871d4-fef0-4c91-949f-770934c5c595",
   "metadata": {},
   "source": [
    "And that's it for an initial port to Ray Serve!\n",
    "\n",
    "Let's try it out"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8bd70e92-ebe6-4216-9660-d68686bb465a",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "handle = serve.run(qa_deployment, name=\"qa\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3a7fd284-b04b-4d06-aa72-de1c458e8f20",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "ray.get(handle.qa.remote(\"How many people live in San Francisco?\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5856f613-3825-4f81-89ee-3349b1285b89",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "ray.get(handle.qa.remote(\"When did Taylor Swift's Eras tour start?\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "449cee9a-9014-45ae-8662-7f37192d9c91",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "serve.delete(\"qa\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "04bc7623-7bec-47f8-8bda-2c68b2094bce",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Extend the db service for performance and scale\n",
    "\n",
    "<div class=\"alert alert-block alert-info\">\n",
    "To provide multiple replicas -- and autoscaling -- of the db service, add an autoscaling config to the deployment decorator\n",
    "<br/><br/>\n",
    "We can also speed up the index build by splitting up (\"sharding\") the array of document vectors and defining a Ray Task to create part of the index from that shard. \n",
    "\n",
    "1. Add the `@ray.remote` decorator to the `process_shard` Python function makes it schedulable by Ray\n",
    "1. Call `process_shard.remote(...)` tells Ray to schedule these tasks -- ideally all in parallel (limited only by our compute capacity)\n",
    "1. Use `ray.get(futures)` to wait for all of the shards to be processes into index chunks\n",
    "1. Merge chunks using a local `for` loop, and write the results to disk\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f043585d-0e42-433d-9863-fbbf75d934aa",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# set the vector db service to autoscale, starting with 2 replicas and scaling between 1 and 5\n",
    "@serve.deployment(\n",
    "    autoscaling_config={\"min_replicas\": 1, \"initial_replicas\": 2, \"max_replicas\": 5}\n",
    ")\n",
    "class ParallelBuildVectorDBDeployment:\n",
    "    FAISS_INDEX_PATH = \"/home/ray/faiss_dist_built_index\"\n",
    "\n",
    "    def __init__(self):\n",
    "        self.embeddings = LocalHuggingFaceEmbeddings(\"multi-qa-mpnet-base-dot-v1\")\n",
    "        try:\n",
    "            self.db = FAISS.load_local(self.FAISS_INDEX_PATH, self.embeddings)\n",
    "        except:\n",
    "            self.setup_db()\n",
    "\n",
    "    def setup_db(self):\n",
    "        topics = [\"The Eras Tour\", \"2023 XFL season\"]\n",
    "        loaders = [WikipediaLoader(query=topic, load_max_docs=20) for topic in topics]\n",
    "        text_splitter = RecursiveCharacterTextSplitter(chunk_size=300, chunk_overlap=20, length_function=len)\n",
    "        docs = add(*[loader.load() for loader in loaders])\n",
    "        chunks = text_splitter.create_documents(\n",
    "            [doc.page_content for doc in docs], metadatas=[doc.metadata for doc in docs]\n",
    "        )\n",
    "        \n",
    "        # split dataset into chunks for parallel processing\n",
    "        db_shards = 8\n",
    "        print(f\"Loading chunks into vector store ... using {db_shards} shards\")\n",
    "        shards = np.array_split(chunks, db_shards)\n",
    "\n",
    "        # create a Ray task to generate embeddingfor a single chunk\n",
    "        @ray.remote\n",
    "        def process_shard(shard):\n",
    "            embeddings = LocalHuggingFaceEmbeddings(\"multi-qa-mpnet-base-dot-v1\")\n",
    "            result = FAISS.from_documents(shard, embeddings)\n",
    "            return result\n",
    "\n",
    "        #schedule chunk processing on Ray and obtain ObjectRefs (futures/promises)\n",
    "        futures = [process_shard.remote(shards[i]) for i in range(db_shards)]\n",
    "        \n",
    "        # wait for all chunks to be finished & retrieve Python objects\n",
    "        results = ray.get(futures)\n",
    "        \n",
    "        # combine chunks locally\n",
    "        self.db = results[0]\n",
    "        for i in range(1, db_shards):\n",
    "            self.db.merge_from(results[i])\n",
    "            \n",
    "        self.db.save_local(self.FAISS_INDEX_PATH)\n",
    "\n",
    "    def similarity_search(self, query):\n",
    "        return self.db.similarity_search(query)\n",
    "\n",
    "\n",
    "vecdb_deployment = ParallelBuildVectorDBDeployment.bind()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "9c889f35-b62b-4caf-93f2-2f56f79c8586",
   "metadata": {},
   "source": [
    "We'll spin up the whole application -- but this time using our new, faster definition of the db service"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "65f50eb9-57a2-4e4d-aa29-65f4fd8d0f80",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "qa_deployment = QADeployment.bind(vecdb_deployment)\n",
    "handle = serve.run(qa_deployment, name=\"qa\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5f2a126f-1dec-45bd-8ce5-80ece70f1280",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "ray.get(handle.qa.remote(\"How many people live in San Francisco?\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0c082ca3-c6bc-4ddb-8cde-0df617f80b7f",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "serve.delete(\"qa\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "4a6ddf37-161d-4b49-8f28-7c7526e35397",
   "metadata": {},
   "source": [
    "## Extend the Q&A service for perf, scale and HTTP traffic\n",
    "\n",
    "<div class=\"alert alert-block alert-info\">\n",
    "Specifying additional replicas in the decorator and add a JSON/HTTP handler in preparation for production deployment\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a3e4c2dc-c746-44f7-9636-b4cb36fdae44",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# specify 2 replicas of this service (in place of the default 1 replica)\n",
    "@serve.deployment(ray_actor_options={\"num_gpus\": 1.0}, num_replicas=2)\n",
    "class QADeployment:\n",
    "    def __init__(self, db):\n",
    "        self.embeddings = LocalHuggingFaceEmbeddings(\"multi-qa-mpnet-base-dot-v1\")\n",
    "        self.db = db\n",
    "        self.llm = StableLMPipeline.from_model_id(model_id=\"stabilityai/stablelm-tuned-alpha-7b\", task=\"text-generation\",\n",
    "            model_kwargs={\"torch_dtype\": torch.float16, \"device_map\": \"auto\", \"cache_dir\": \"/mnt/local_storage\",},)\n",
    "        self.chain = load_qa_chain(llm=self.llm, chain_type=\"stuff\", prompt=PROMPT)\n",
    "\n",
    "    async def qa(self, query):\n",
    "        search_results_ref = await self.db.similarity_search.remote(query)\n",
    "        search_results = await search_results_ref\n",
    "        result = self.chain({\"input_documents\": search_results, \"question\": query})\n",
    "        return result[\"output_text\"]\n",
    "\n",
    "    # add a handler for HTTP requests    \n",
    "    async def __call__(self, request: Request) -> Dict:\n",
    "\n",
    "        # decode incoming request as JSON\n",
    "        data = await request.json()\n",
    "        data = json.loads(data)\n",
    "\n",
    "        # call into existing qa method implementation and await async output\n",
    "        output = await self.qa(data[\"user_input\"])\n",
    "        return {\"result\": output}"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "0327a628-7af6-401d-a2f0-eca8713bc6b9",
   "metadata": {},
   "source": [
    "Start the service and test via Python"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2f5e1119-3769-45b9-95d1-3cb40b9ec121",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "qa_deployment = QADeployment.bind(vecdb_deployment)\n",
    "handle = serve.run(qa_deployment, name=\"qa\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c0740cde-a237-467a-80d3-5c5d5d146142",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "ray.get(handle.qa.remote(\"How many people live in San Francisco?\"))"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "adad4db0-8c85-4383-866a-17aebf7ddaa6",
   "metadata": {},
   "source": [
    "In production, we expect to receive HTTP traffic, so we'll make sure that execution path works"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bd61c9dd-e69b-44d7-b28a-2c3c63e41f80",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "message = \"When did Taylor Swift's Eras tour start?\"\n",
    "\n",
    "json_doc = json.dumps({\"user_input\": message})\n",
    "\n",
    "requests.post(\"http://localhost:8000/\", json=json_doc).json()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "65e51a68-ada4-4e1d-8ae9-428b18b71824",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "serve.delete(\"qa\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "16525a2c-287d-4144-b2c0-38e68e6e5c85",
   "metadata": {},
   "source": [
    "## Discussion ideas: extending the architecture\n",
    "\n",
    "We might want to extend our current vector db service in several ways\n",
    "* Allow indexing additional topics/documents without rebuilding the entire index and service\n",
    "* Ensure that all replicas of the db server are using the same (and most up-to-date) index\n",
    "\n",
    "The FAISS architecture as a non-distributed database makes this an interesting project: we can use Ray's Actor API and Object Store to create a single service that manages updating the index and providing the authoritative index to all vector db deployment replica.\n",
    "\n",
    "While the local FAISS architecture may not reach the performance of a fully-distributed-by-design vector database, using Ray allows us to decouple our document-serving architecture from our index update service, and lets us choose how to balance read and write performance as we consider moving to a scale-out vector db such as Milvus."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b7e98db9-52a7-4876-8aae-89e6ee87f813",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}