Agentic QA Service

Here introduce high performance asynchronous QA service built with Fastapi, Langchain and ChatOpenAI as Planner LLM.

Key features

• FastAPI: as the entry point with built in support of async/await.
• ChatOpenAI: for retrieve suitable tool based on the user query.
• Langchain: for AI orchestration
• Token usage: implement token usage both prompt_tokens and completion_tokens with help of get_openai_callback from langchain_community
• Docker: Dockerize the project for production ready.

Tech Stacks

• python3.11
• FastAPI as the framework
• pydantic for high performance schema validation.
• Langchain for AI implementation
• OpenAI gpt-4o-mini as transformer
• httpx for api calling
• duckduckgo for web_search and open-mateo for weather finder

Getting Started

• Installation & setup
  • clone the repo
    • git clone https://github.com/mohammedajmalpk/Agentic-QA-Service.git
  • configure environment variables in .env as per .env.example
    • OPENAI_API_KEY=<your api key>
  • make a virtual env
    • python3.11 -m venv venv
  • build and run docker
    • docker build -t qa-service .
    • docker run -it -p 8000:8000 qa-service
  • you may run locally
    • python main.py

API Usage

• Once the service running, you may access Swagger UI at
  • http://localhost:8000/docs
• Endpoints:
  • /chat: Ask your question to the agent
    • example request
      • http://0.0.0.0:8000/chat?query=current%20weather%20in%20doha
    • example response

      • {
         "answer": "Weather: 22.9°C, Overcast",
         "sources": [
           {
             "name": "Open-Meteo",
             "url": "https://api.open-meteo.com/v1/forecast"
           }
         ],
         "latency_ms": {
           "total": 5642,
           "by_step": {
             "retrieval": 1168,
             "llm": 4474
           }
         },
         "tokens": {
           "prompt": 216,
           "completion": 30
         }
        }
        

Project structure

.
├── app  # core project folder
│   ├── __init__.py
│   ├── agent.py  # run the agent
│   ├── planner.py  #  choose suitable tool 
│   ├── prompts.py  #  prompt template with structured prompt
│   ├── schemas.py  #  perfect pydantic schemas for validation
│   ├── tools.py  #  2 defined functions
│   └── utils  #  utilities
│       ├── __init__.py
│       └── weather_codes.py
├── .gitignore
├── Dockerfile
├── .env.example
├── .env
├── LICENSE
├── main.py  # Fastapi entry
├── README.md
├── requirements.txt
└── venv