Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
f9be937
Add files via upload
rakeshcr92 Jul 25, 2025
a60e30b
Merge branch 'odsc2015:main' into main
Agarw64 Jul 25, 2025
03c4218
feat: Add Next.js frontend application
Jul 25, 2025
8a50b91
First draft of architecture updated
shivajipanam Jul 25, 2025
ddb01e0
Merge branch 'main' of https://github.com/rakeshcr92/agents-assemble
shivajipanam Jul 25, 2025
a3c936b
Add files via upload
rakeshcr92 Jul 25, 2025
d11739f
Delete src directory
rakeshcr92 Jul 25, 2025
e0bef7a
folder structure - frontend
munishpatel Jul 25, 2025
1a8e4a0
Merge branch 'main' of https://github.com/rakeshcr92/agents-assemble
munishpatel Jul 25, 2025
1fcdf33
folder structure - frontend
munishpatel Jul 25, 2025
1487cdb
Add files via upload
rakeshcr92 Jul 25, 2025
769695b
base-agent
rakeshcr92 Jul 25, 2025
41e11f4
baseagent
rakeshcr92 Jul 25, 2025
df9049c
base_agent
rakeshcr92 Jul 25, 2025
05ba600
base_agent
rakeshcr92 Jul 25, 2025
ec2ca55
Official 'mock' draft of UI
Jul 25, 2025
2995324
Orchestrator skeleton code
shivajipanam Jul 25, 2025
0b85a3e
orchestration working code
kukunarapulikitha Jul 25, 2025
4a3a832
added the photoUpload component without api integration
ShivaniNR Jul 25, 2025
738dc91
orchestration update 2
kukunarapulikitha Jul 26, 2025
a3aa71a
created base_agent, voice_agent, response_agent, requirements and Doc…
Rakesh-b0 Jul 26, 2025
5e87bd7
Finalize UI redesign and updated overlay
Jul 26, 2025
77a2a4e
VisionAgent class added
muhamed-aroui Jul 26, 2025
0695e56
Merge photoUpload function with UI redesign
Jul 26, 2025
d84258a
mvp orchestrator single node
kukunarapulikitha Jul 26, 2025
57bcbf0
added speech-to-text feature on frontend app and modified the upload …
ShivaniNR Jul 26, 2025
0fd38b9
app deployment to google cloud using docker
Jul 26, 2025
4ec8b50
resolved the pause issue between user speech
ShivaniNR Jul 26, 2025
14f67f5
Merge pull request #1 from rakeshcr92/feature/voice-response-agents
rakeshcr92 Jul 26, 2025
d6ed7dd
Merge pull request #2 from rakeshcr92/VisionAgent
rakeshcr92 Jul 26, 2025
36a086c
Merge branch 'main' into orchestrator
kukunarapulikitha Jul 26, 2025
962b062
Finalize voice interface and memory recall flow with Jennifer convo
Jul 26, 2025
436fd65
Merge pull request #5 from rakeshcr92/orchestrator
kukunarapulikitha Jul 26, 2025
7943444
merged main branch
Jul 26, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
venv/
backend/
frontend/migrations/
frontend/node_modules/
frontend/.next/
frontend/.env.local
322 changes: 322 additions & 0 deletions ARCHITECTURE.md

Large diffs are not rendered by default.

144 changes: 144 additions & 0 deletions DEPLOYMENT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,144 @@
# Deployment Guide: agents-assemble
`Tharak Vangalapat`

This guide describes how to build, deploy, and access both backend (FastAPI) and frontend (Next.js/React) services for the agents-assemble project on Google Cloud Run using Docker.

---

## Prerequisites
- Google Cloud account and project
- `gcloud` CLI installed and authenticated
- Docker installed
- Node.js and Python installed locally for development
- Required files: `Dockerfile`, `requirements.txt` (backend), `package.json` (frontend), `.env.local` (frontend)

---

## 1. Build and Deploy Backend (FastAPI)

### a. Ensure dependencies
Your `backend/requirements.txt` should include:
```
fastapi
pydantic
uvicorn[standard]
```

### b. Build Docker image locally (optional)
```sh
cd agents-assemble/backend
docker build -t life-witness-agent-backend .
```

### c. Deploy to Google Cloud Run
From the project root, run:
```sh
gcloud builds submit --tag gcr.io/$PROJECT_ID/life-witness-agent-backend ./backend

gcloud run deploy life-witness-agent-backend \
--image gcr.io/$PROJECT_ID/life-witness-agent-backend \
--platform managed \
--region us-central1 \
--allow-unauthenticated
```

---

## 2. Build and Deploy Frontend (Next.js/React)

### a. Ensure `.env.local` is present in `frontend/` and not excluded by `.dockerignore`.

### b. Build Docker image locally (optional)
```sh
cd agents-assemble/frontend
docker build -t life-witness-agent-frontend .
```

### c. Deploy to Google Cloud Run
From the project root, run:
```sh
gcloud builds submit --tag gcr.io/$PROJECT_ID/life-witness-agent-frontend ./frontend

gcloud run deploy life-witness-agent-frontend \
--image gcr.io/$PROJECT_ID/life-witness-agent-frontend \
--platform managed \
--region us-central1 \
--allow-unauthenticated
```

---

## 3. Accessing the Services
- **Backend API Docs:**
Visit the Cloud Run URL for `life-witness-agent-backend` (e.g., `https://life-witness-agent-backend-xxxxxx.a.run.app/docs`).
- **Frontend App:**
Visit the Cloud Run URL for `life-witness-agent-frontend` (e.g., `https://life-witness-agent-frontend-xxxxxx.a.run.app`).

---

## 4. Environment Variables
- **Frontend:**
Set `NEXT_PUBLIC_AGENT_API_URL` in `.env.local` to your backend Cloud Run URL.
- **Backend:**
Use environment variables or Google Secret Manager for sensitive data if needed.

---

## 5. Troubleshooting
- **500 Internal Server Error:**
Ensure all Python dependencies are installed and your backend code is correct.
- **Memory Issues:**
Increase memory allocation with `--memory 1Gi` or higher in the deploy command.
- **.env.local not found:**
Make sure it’s present in `frontend/` and not listed in `.dockerignore`.

---

## 6. Useful Commands
- **List Cloud Run services:**
```sh
gcloud run services list --region us-central1 --platform managed
```
- **Get service URL:**
```sh
gcloud run services describe SERVICE_NAME --region us-central1 --platform managed --format 'value(status.url)'
```

---

## 7. Updating Services
After code changes, repeat the build and deploy steps for the affected service.

---

## 8. Running Deployment Scripts

### a. Local Development
To run both backend and frontend locally using Docker Compose:

```sh
cd agents-assemble
# Start both services
docker-compose up --build
```

Or, to use the provided demo script:
```sh
cd agents-assemble/scripts
./setup_dev.sh # Set up Python environment and install backend dependencies
./run_demo.sh # Start the FastAPI backend locally
```

### b. Cloud Deployment
To build and deploy both backend and frontend to Google Cloud Run using the provided script:

```sh
cd agents-assemble/scripts
./deploy.sh
```

This will build Docker images and deploy both services to Cloud Run. Make sure you have authenticated with `gcloud` and set your project.

---

For advanced topics (custom domains, load balancer, etc.), see Google Cloud Run documentation.
Empty file added backend/agents/Docker
Empty file.
Binary file not shown.
42 changes: 42 additions & 0 deletions backend/agents/base_agent.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
from abc import ABC, abstractmethod

class BaseAgent(ABC):
@abstractmethod
def run(self, input_text: str) -> str:
"""Run the agent on the input text and return a response."""
pass

class ReActAgent(BaseAgent):
def run(self, input_text: str) -> str:
# Simple demo: calculator tool
if "calculate" in input_text.lower():
expr = input_text.lower().replace("calculate", "").strip()
try:
result = eval(expr, {"__builtins__": {}})
return f"Result: {result}"
except Exception as e:
return f"Error: {str(e)}"
return "Sorry, I can only calculate for now."

def reason_and_act(self, user_input: str) -> dict:
# Simple demo: calculator tool
if "calculate" in user_input.lower():
expr = user_input.lower().replace("calculate", "").strip()
try:
result = eval(expr, {"__builtins__": {}})
return {
"thought": "I need to calculate.",
"action": "calculator",
"result": f"Result: {result}"
}
except Exception as e:
return {
"thought": "Tried to calculate but failed.",
"action": "calculator",
"result": f"Error: {str(e)}"
}
return {
"thought": "I don't know how to handle this.",
"action": None,
"result": "Sorry, I can only calculate for now."
}
Empty file added backend/agents/context_agent.py
Empty file.
Empty file added backend/agents/insight_agent.py
Empty file.
Empty file added backend/agents/memory_agent.py
Empty file.
Empty file added backend/agents/planner_agent.py
Empty file.
86 changes: 86 additions & 0 deletions backend/agents/response_agent.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
from langchain.llms.base import LLM
from langchain.chains import LLMChain
from langchain.prompts import PromptTemplate
from base_agent import BaseAgent # your custom base class
from langchain_google_vertexai import ChatVertexAI

class ResponseAgent(BaseAgent):
def __init__(self):
# Initialize the Google Vertex AI chat model (Gemini)
self.llm = ChatVertexAI(
model_name="gemini-2.5-pro", # update model if needed
project="your-project-id", # replace with your GCP project ID
location="us-central1" # update region if needed
)

# Prompt template designed for Life Witness app with memory context
prompt_template = PromptTemplate(
input_variables=["input", "context"],
template="""
You are Life Witness, a personal AI memory assistant that listens to life stories, captures moments, and helps recall memories with rich context.

You have access to the user's life memories, including events, photos, calendar entries, and relationships.

Respond in a warm, empathetic, and conversational tone — like a trusted friend who remembers important details.

Use the provided context to ground your replies, including dates, locations, people, and emotional cues.

If the user asks about past events, recall details clearly and helpfully.

If the user wants to set reminders or follow up, help create clear action items.

Do NOT say you are an AI or mention technical details.

---

Context:
{context}

User:
{input}

Life Witness:
"""
)

self.chain = LLMChain(llm=self.llm, prompt=prompt_template)

def run(self, input_text: str, context: str = "") -> str:
"""
Generate a conversational response using the input text and optional memory context.

Args:
input_text (str): The user query or statement.
context (str, optional): A string summarizing relevant memories or context. Defaults to "".

Returns:
str: The agent's generated response.
"""
return self.chain.run(input=input_text, context=context)

#Example Usage:

"""
response_agent = ResponseAgent()
response = response_agent.run(user_input, memory_context)
"""


"""
if __name__ == "__main__":
# Make sure you have authenticated your GCP credentials via:
# gcloud auth application-default login

print("Initializing ResponseAgent...")
agent = ResponseAgent()
print("Agent initialized.")

prompt = "Hello, how can you assist me today?"
print(f"Sending prompt: \"{prompt}\"")
try:
reply = agent.run(prompt)
print(f"Agent reply: {reply}")
except Exception as e:
print(f"An error occurred: {e}")
print("Check your Google Cloud authentication and project/location settings.")
"""
64 changes: 64 additions & 0 deletions backend/agents/vision_agent.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
from langchain_google_genai import ChatGoogleGenerativeAI
from langchain_core.messages import HumanMessage
from base_agent import BaseAgent
from dotenv import load_dotenv
import base64


load_dotenv()


class VisionAgent(BaseAgent):
def __init__(self, model_name: str = "gemini-2.5-flash"):
"""
Initializes the VisionAgent with a specified Google Generative AI model.

Args:
model_name (str): The name of the model to use.
"""
super().__init__()
self.model = ChatGoogleGenerativeAI(model=model_name)
print(f"VisionAgent initialized with model: {model_name}")

def describe(self, image_path: str) -> str:
"""
Creates a detailed textual description of the image at the given file path.

Args:
image_path (str): The file path to the image to be described.

Returns:
str: A textual description of the image.
"""
# Read the image file and encode it in base64
try:
with open(image_path, "rb") as image_file:
base64_image = base64.b64encode(image_file.read()).decode("utf-8")
except FileNotFoundError:
return f"Error: Image file not found at '{image_path}'"

# Prepare the message for the model
message = HumanMessage(
content=[
{"type": "text", "text": "Describe this image in detail."},
{
"type": "image_url",
"image_url": {"url": f"data:image/jpeg;base64,{base64_image}"},
},
]
)

# Invoke the model and return the content of the response
response = self.model.invoke([message])
return response.content

if __name__=="__main__":

vision_agent = VisionAgent()

image_file_path = "path_to_image"

description = vision_agent.describe(image_file_path)

print("\n--- Image Description ---")
print(description)
Loading