Skip to content

Commit

Permalink
update Xinference cases & fix nyc_flights path (#61)
Browse files Browse the repository at this point in the history 
* update xinference

* fix nyc_flights path

* fix nyc_flights path
  • Loading branch information
@luweizheng
luweizheng authored Jul 12, 2024
1 parent 3b136da commit 138abaf
Show file tree
Hide file tree
Showing 8 changed files with 36 additions and 149 deletions.
2 changes: 1 addition & 1 deletion ch-dask-dataframe/indexing.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -563,7 +563,7 @@
],
"source": [
"folder_path = nyc_flights()\n",
"file_path = os.path.join(folder_path, \"*.csv\")\n",
"file_path = os.path.join(folder_path, \"nyc-flights\", \"*.csv\")\n",
"flights_ddf = dd.read_csv(file_path,\n",
" parse_dates={'Date': [0, 1, 2]},\n",
" dtype={'TailNum': object,\n",
Expand Down
2 changes: 1 addition & 1 deletion ch-dask-dataframe/read-write.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@
"\n",
"folder_path = nyc_flights()\n",
"\n",
"file_path = os.path.join(folder_path, \"*.csv\")"
"file_path = os.path.join(folder_path, \"nyc-flights\", \"*.csv\")"
]
},
{
Expand Down
2 changes: 1 addition & 1 deletion ch-dask-ml/hyperparameter.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@
"import pandas as pd\n",
"\n",
"folder_path = nyc_flights()\n",
"file_path = os.path.join(folder_path, \"1991.csv\")"
"file_path = os.path.join(folder_path, \"nyc-flights\", \"1991.csv\")"
]
},
{
Expand Down
2 changes: 1 addition & 1 deletion ch-ray-ml/ray-tune.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -1347,7 +1347,7 @@
"from ray.tune.schedulers import PopulationBasedTraining\n",
"\n",
"folder_path = nyc_flights()\n",
"file_path = os.path.join(folder_path, \"1991.csv\")"
"file_path = os.path.join(folder_path, \"nyc-flights\", \"1991.csv\")"
]
},
{
Expand Down
2 changes: 1 addition & 1 deletion ch-ray-ml/tune-algorithm-scheduler.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@
"from ray.tune.schedulers import PopulationBasedTraining\n",
"\n",
"folder_path = nyc_flights()\n",
"file_path = os.path.join(folder_path, \"1991.csv\")"
"file_path = os.path.join(folder_path, \"nyc-flights\", \"1991.csv\")"
]
},
{
Expand Down
126 changes: 31 additions & 95 deletions ch-xorbits/xinference.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -65,42 +65,21 @@
"\n",
"Xinference 可以管理模型部署的整个生命周期：启动模型、使用模型、关闭模型。\n",
"\n",
"启动 Xinference 服务后，我们就可以启动模型并调用模型，Xinference 集成了大量开源模型，用户可以在网页中选择一个启动，Xinference 会在后台下载并启动这个模型。每个启动的模型既有网页版对话界面，又兼容 OpenAI 的 API。比如，使用 OpenAI 的 API 与某个模型交互："
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from openai import OpenAI\n",
"client = OpenAI(base_url=\"http://127.0.0.1:9997/v1\", api_key=\"not used actually\")\n",
"\n",
"response = client.chat.completions.create(\n",
" model=\"my-llama-2\",\n",
" messages=[\n",
" {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n",
" {\"role\": \"user\", \"content\": \"What is the largest animal?\"}\n",
" ]\n",
")\n",
"print(response)"
"启动 Xinference 服务后，我们就可以启动模型并调用模型，Xinference 集成了大量开源模型，用户可以在网页中选择一个启动，Xinference 会在后台下载并启动这个模型。每个启动的模型既有网页版对话界面，又兼容 OpenAI 的 API。下来我们将展示 2 个案例，介绍如何在本地管理和使用模型，如何基于 OpenAI API 与 Xinference 模型交互，如何基于 LangChain 和向量数据库构建智能系统。"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 案例\n",
"下面我们将介绍2个 Xinference 实际应用案例。\n",
"### 案例：使用通义千问（Qwen）进行简单文本生成与对话"
"## 案例：使用通义千问（Qwen）进行简单文本生成与对话"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"在开始之前，首先确保安装以下依赖包："
"在开始之前，除了安装 Xinference 外，还需要安装 openai 依赖包："
]
},
{
Expand All @@ -116,7 +95,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"首先我们启动 Xinference 的本地实例。请使用以下命令在后台运行 Xinference:"
"首先我们启动 Xinference 的本地实例。在 Jupyter Notebook 中，请使用以下命令在后台运行 Xinference，在命令行中，可以直接 `xinference-local --host 0.0.0.0 --port 9997`。"
]
},
{
Expand Down Expand Up @@ -151,9 +130,9 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Xinference 默认的主机和 IP 地址分别为127.0.0.1 和 9997。\n",
"Xinference 默认的主机和 IP 地址分别为 127.0.0.1 和 9997。\n",
"\n",
"接下来通过以下命令来启动模型。其中`size-in-billions`参数对应使用的参数规模，当前 qwen-chat 开源模型的参数规模为18亿（1.8B）、70亿（7B）、140亿（14B）和720亿（72B）。在本次案例中，我们尝试使用 Qwen-7B 。"
"接下来通过以下命令来启动通义千问模型。其中 `size-in-billions` 参数对应使用的参数规模，当前开源的 `qwen-chat` 模型参数规模为18亿（1.8B）、70亿（7B）、140亿（14B）和720亿（72B）。这里我们尝试使用 Qwen-7B 。"
]
},
{
Expand Down Expand Up @@ -183,7 +162,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"第一次启动模型时，Xinference 将自动下载该模型的参数，这可能需要一定时间。\n",
"第一次启动模型时，Xinference 将自动下载模型，这可能需要一定时间。\n",
"\n",
"由于 Xinference 提供了与 OpenAI 兼容的 API，所以可以将 Xinference 运行的模型当成 OpenAI 的本地替代。"
]
Expand All @@ -196,22 +175,23 @@
"source": [
"import openai\n",
"\n",
"client = openai.Client(api_key=\"cannot be empty\", base_url=\"http://127.0.0.1:9997/v1\")"
"client = openai.Client(api_key=\"can be empty\", base_url=\"http://127.0.0.1:9997/v1\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"接下来我们演示借助 Xinference 与 OpenAi 的 API 如何轻松调用大模型进行文本生成和上下文对话。"
"接下来我们使用 OpenAI API 轻松调用大模型进行文本生成和上下文对话。"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Completion API\n",
"首先我们使用 `client.completions.create` 进行简单的文本生成。Completion API 用来通过一段文本（也叫 prompt）进行文本生成。"
"\n",
"使用 `client.completions.create` 进行简单的文本生成。Completion API 用来通过一段文本（Prompt）引导模型进行文本生成。"
]
},
{
Expand Down Expand Up @@ -307,6 +287,7 @@
"metadata": {},
"source": [
"### Chat Completion API\n",
"\n",
"接下来我们使用 `client.chat.completions.create` 进行简单的上下文对话。\n",
"\n",
"Chat Completion API 可以为与 LLM 进行交互时提供更加结构化的方式。相比于单纯传递一段文字，我们将一个包含多个结构化信息对象的数组发送给 LLM 作为输入。这样做可以让语言模型在继续回复时有一些\"上下文\"或者\"历史\"可参考。\n",
Expand Down Expand Up @@ -448,7 +429,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"最后关闭后台运行的 Xinference 实例："
"不再需要推理服务时，可关停后台运行的 Xinference 实例："
]
},
{
Expand All @@ -465,9 +446,10 @@
"metadata": {},
"source": [
"## 案例：基于 LangChain 的文档聊天机器人\n",
"该案例将演示如何使用本地 LLM 和 LangChain 模型构建聊天机器人。通过此机器人，用户可以实现简单的文档读取，并根据文档内容进行互动对话。\n",
"\n",
"首先，我们先安装必要的库。"
"该案例将演示如何使用本地大模型和 LangChain 模型构建聊天机器人。通过此机器人，用户可以实现简单的文档读取，并根据文档内容进行互动对话。\n",
"\n",
"我们先安装必要的库："
]
},
{
Expand Down Expand Up @@ -519,8 +501,9 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"### 建立 embedding 模型\n",
"我们以马克吐温的《百万英镑》作为案例，先使用 LangChain 读取文档并进行分割。"
"### 启动向量模型\n",
"\n",
"我们以马克吐温的《百万英镑》作为案例，先使用 LangChain 读取文档并对文档中文本进行切分。"
]
},
{
Expand All @@ -531,9 +514,6 @@
"source": [
"import os\n",
"\n",
"new_directory = \"/home/u2022201752/scale-py-zh\"\n",
"os.chdir(new_directory)\n",
"\n",
"from utils import mark_twain\n",
"from langchain.document_loaders import PDFMinerLoader\n",
"from langchain.text_splitter import RecursiveCharacterTextSplitter\n",
Expand All @@ -556,7 +536,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"下面我们需要启动一个 embedding 模型将文档信息向量化："
"下面我们需要启动一个向量（Embedding）模型将文档的文本内容转化为向量："
]
},
{
Expand Down Expand Up @@ -598,10 +578,11 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"### Milvus 向量数据库建立\n",
"接下来我们使用 Milvus 向量数据库来储存 embedding 之后的信息。\n",
"### 启动向量数据库\n",
"\n",
"我们引入向量数据库，向量数据库存储了向量和文档，每个向量对应一个文档。本例中，我们使用 Milvus 向量数据库来储存向量和文档。\n",
"\n",
"Milvus 向量数据库可以通过以下命令进行安装："
"Milvus 数据库可以通过以下命令进行安装："
]
},
{
Expand Down Expand Up @@ -674,7 +655,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"这里我们可以尝试提问对文档进行检索（这里并没有使用 llm 模型，仅返回匹配的字段）："
"这里我们可以尝试提问对文档进行检索（这里并没有使用大语言模型，仅返回匹配的字段）："
]
},
{
Expand Down Expand Up @@ -704,8 +685,9 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"### 建立 llm 模型\n",
"接下来我们启动一个 llm 模型进行对话。这里我们使用 Xinference 支持的 llama-3-instruct 模型："
"### 启动大语言模型\n",
"\n",
"接下来我们启动一个大语言模型进行对话。这里我们使用 Xinference 支持的 llama-3-instruct 模型："
]
},
{
Expand Down Expand Up @@ -749,7 +731,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"现在，我们使用 llm 模型和 embedding 向量创建 `ConversationalRetrievalChain`。"
"现在，我们使用大语言模型和向量创建 `ConversationalRetrievalChain`。LangChain 连接了不同的组件，这种“连接”被称为 Chain，本例是将对话和信息检索连接起来。"
]
},
{
Expand Down Expand Up @@ -807,7 +789,6 @@
" result = chain({\"question\": query})\n",
" print(result[\"answer\"])\n",
"\n",
"\n",
"chat(\"How did people react to the protagonist carrying the million-pound banknote?\")"
]
},
Expand Down Expand Up @@ -854,54 +835,9 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"在这里 llm 模型准确识别出 “him” 指的是 “protagonist”，这说明结合 Xinference 与 LangChain 能将新的问答与聊天历史记录相关联，从而创建相互构建的响应链。\n",
"在这里大语言模型准确识别出 “him” 指的是 “protagonist”，这说明结合 Xinference 与 LangChain 能将本地知识相关联。\n",
"\n",
"我们可以看到 llm 的令人印象深刻的功能，LangChain 的“链接” `chain` 功能还允许与模型进行更连贯和上下文感知的交互。"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"最后关闭 Xinference 与 Milvus 数据库进程："
]
},
{
"cell_type": "code",
"execution_count": 36,
"metadata": {
"vscode": {
"languageId": "bat"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"Usage:\n",
" kill [options] <pid|name> [...]\n",
"\n",
"Options:\n",
" -a, --all do not restrict the name-to-pid conversion to processes\n",
" with the same uid as the present process\n",
" -s, --signal <sig> send specified signal\n",
" -q, --queue <sig> use sigqueue(2) rather than kill(2)\n",
" -p, --pid print pids without signaling them\n",
" -l, --list [=<signal>] list signal names, or convert one to a name\n",
" -L, --table list signal names and numbers\n",
"\n",
" -h, --help display this help and exit\n",
" -V, --version output version information and exit\n",
"\n",
"For more details see kill(1).\n"
]
}
],
"source": [
"!ps ax | grep xinference-local | grep -v grep | awk '{print $1}' | xargs kill -9\n",
"!ps ax | grep milvus-server | grep -v grep | awk '{print $1}' | xargs kill -9"
"两个案例展示了 Xinference 在本地构建的各类智能应用。"
]
},
{
Expand Down
48 changes: 0 additions & 48 deletions ch-xorbits/xinference.md
View file

This file was deleted.

1 change: 0 additions & 1 deletion ch-xorbits/xorbits.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,6 @@
"\n",
"import sys\n",
"sys.path.append(\"..\")\n",
"from utils import nyc_flights\n",
"from utils import nyc_taxi\n",
"\n",
"taxi_path = nyc_taxi()"
Expand Down

0 comments on commit 138abaf

Please sign in to comment.