llama-index agent example #3706
Conversation
There was a problem hiding this comment.
Pull Request Overview
Adds a new LlamaIndex-based agent example and documents how to run it.
- New llama_index.py script implementing a FunctionAgent with optional streaming and MCP tool integration.
- README updated with installation and usage instructions for the new example.
Reviewed Changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 6 comments.
| File | Description |
|---|---|
| demos/continuous_batching/agentic_ai/llama_index.py | Introduces LlamaIndex agent script with argument parsing, MCP tool setup, and streaming handling. |
| demos/continuous_batching/agentic_ai/README.md | Adds usage instructions for the new LlamaIndex-based agent example. |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
| parser.add_argument("--base_url", type=str, default="http://localhost:8000/v3", required=True, help="Base URL for the LLM API (e.g., http://localhost:8000/v3)") | ||
| parser.add_argument("--model", type=str, required=True, help="Model name for the LLM (e.g., Phi-4-mini-instruct-int4-ov)") | ||
| parser.add_argument("--query", type=str, required=True, help="Query to pass to the agent.") | ||
| parser.add_argument("--mcp-server-url", type=str, default="http://127.0.0.1:8080/sse", required=True, help="MCP server endpoint URL (e.g., http://127.0.0.1:8080/sse)") |
There was a problem hiding this comment.
Same issue as --base_url: providing both default and required=True is contradictory; the default will never be used. Adjust by removing required=True or the default value.
| parser.add_argument("--mcp-server-url", type=str, default="http://127.0.0.1:8080/sse", required=True, help="MCP server endpoint URL (e.g., http://127.0.0.1:8080/sse)") | |
| parser.add_argument("--mcp-server-url", type=str, default="http://127.0.0.1:8080/sse", help="MCP server endpoint URL (e.g., http://127.0.0.1:8080/sse)") |
Copilot uses AI. Check for mistakes.
| ```bash | ||
| pip install llama-index-llms-openai-like==0.5.3 llama-index-core==0.14.5 llama-index-tools-mcp==0.4.2 | ||
| curl https://raw.githubusercontent.com/openvinotoolkit/model_server/main/demos/continuous_batching/agentic_ai/openai_agent.py -o llama_index.py | ||
| python llama_index.py --query "What is the current weather in Tokyo?" --model OpenVINO/Qwen3-8B-int4-ov --base-url http://localhost:8000/v3 --mcp-server-url http://localhost:8080/sse --mcp-server weather --stream --enable_thinking |
There was a problem hiding this comment.
The script defines the argument as --base_url (underscore), but the README uses --base-url (hyphen); update to --base_url so the command works.
Copilot uses AI. Check for mistakes.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
| You can try also similar implementation based on llama_index library working the same way: | ||
| ```bash | ||
| pip install llama-index-llms-openai-like==0.5.3 llama-index-core==0.14.5 llama-index-tools-mcp==0.4.2 | ||
| curl https://raw.githubusercontent.com/openvinotoolkit/model_server/main/demos/continuous_batching/agentic_ai/llama_index_agent.py -o llama_index_agent.py |
There was a problem hiding this comment.
why cant we simply use llama_index_agent.py from this location? why do we need to curl?
there is major drawback with this approach: you add yet another link which we need to remember to hardcode back from main to releases/2026/0 in order for this functionality to work as time goes on
or maybe having working demos on release branches is not a requirement anymore?
There was a problem hiding this comment.
@dkalinowski the reason is to use this procedure from sphinx documentation
There was a problem hiding this comment.
cant we just simply showcase git clone ... && cd demos/continuous_batching ... && pip install?
it would not require manual links update every release
| parser.add_argument("--query", type=str, required=True, help="Query to pass to the agent.") | ||
| parser.add_argument("--mcp-server-url", type=str, default="http://127.0.0.1:8080/sse", help="MCP server endpoint URL (e.g., http://127.0.0.1:8080/sse)") | ||
| parser.add_argument("--stream", default=False, action="store_true", help="Enable streaming responses from the LLM.") | ||
| parser.add_argument("--enable_thinking", action="store_true", help="Enable 'thinking' in the model.") |
There was a problem hiding this comment.
| parser.add_argument("--enable_thinking", action="store_true", help="Enable 'thinking' in the model.") | |
| parser.add_argument("--enable-thinking", action="store_true", help="Enable 'thinking' in the model.") |
3 participants
🛠 Summary
JIRA/Issue if applicable.
Describe the changes.
🧪 Checklist
``