Awesome AI Engineer

A curated roadmap and resource collection for leveling up from AI Engineer to AI Systems Architect — with 13 production-ready portfolio projects to build along the way.

Roadmap • Projects • Learning • Frameworks • MLOps • System Design • Books • Contributing

---

Why This List?

The AI engineering landscape is evolving fast. New frameworks, models, and patterns emerge weekly. This repo cuts through the noise with:

• A clear career progression from junior AI engineer to systems architect
• 13 hands-on projects covering LLMs, agents, RAG, multi-agent systems, and protocols
• Curated resources — only the best, most relevant material for 2025-2026
• Enterprise focus — production patterns, not just tutorials

---

🗺️ Career Roadmap

┌─────────────────────────────────────────────────────────────────────┐
│                    AI ENGINEER → SYSTEMS ARCHITECT                  │
├─────────────────────────────────────────────────────────────────────┤
│                                                                     │
│  LEVEL 1: AI Engineer                                               │
│  ├── LLM fundamentals (tokenization, generation, fine-tuning)       │
│  ├── Prompt engineering & evaluation                                │
│  ├── RAG pipelines & vector databases                               │
│  ├── API design for AI services                                     │
│  └── Projects: #1 #2 #5 #7                                         │
│                                                                     │
│  LEVEL 2: Senior AI Engineer                                        │
│  ├── Agent architectures (ReAct, Plan-and-Execute)                  │
│  ├── Multi-agent orchestration patterns                             │
│  ├── Agentic frameworks (LangGraph, CrewAI, ADK)                    │
│  ├── Streaming & real-time AI systems                               │
│  └── Projects: #3 #4 #6 #8                                         │
│                                                                     │
│  LEVEL 3: AI Platform Engineer                                      │
│  ├── MLOps & model serving (vLLM, TensorRT, Triton)                │
│  ├── Kubernetes for AI workloads                                    │
│  ├── Observability & evaluation frameworks                          │
│  ├── Protocol design (MCP, A2A, UCP)                                │
│  └── Projects: #8 #9 #10                                           │
│                                                                     │
│  LEVEL 4: AI Systems Architect                                      │
│  ├── Enterprise agent framework patterns                            │
│  ├── Multi-agent system design at scale                             │
│  ├── Compliance, security & governance for AI                       │
│  ├── Cross-framework interoperability                               │
│  └── Projects: #11 #12 #13                                         │
│                                                                     │
└─────────────────────────────────────────────────────────────────────┘

---

🚀 Portfolio Projects

13 production-ready projects, each in its own repo with Docker setup, tests, CI/CD, and Kubernetes manifests. Clone any project and run it in under 2 minutes.

Foundation (Level 1-2)

#	Project	What You'll Build	Key Skills	Repo
1	LLM Playground	Interactive tokenization & generation explorer	BPE tokenization, sampling strategies, temperature
2	Customer Support Chatbot	Production chatbot with fine-tuning	LoRA/PEFT, prompt engineering, evaluation
3	Ask-the-Web Agent	Perplexity-like search & synthesis agent	LangGraph, tool use, citation generation
4	Deep Research	Multi-strategy reasoning engine	Chain-of-Thought, Tree-of-Thought, inference scaling
5	Image Generation	Multi-provider image service	Diffusion models, DALL-E, FLUX, Stable Diffusion
6	Capstone Multi-Agent	Supervisor-pattern agent platform	Multi-agent orchestration, task decomposition
7	Agent RAG	Hierarchical RAG system	Single/multi/hierarchical retrieval, query decomposition
8	MCP & A2A	Protocol implementation	MCP SDK, A2A protocol, AgentCards

Protocol & Commerce (Level 3)

#	Project	What You'll Build	Key Skills	Repo
9	UCP Merchant Server	Commerce protocol platform	Checkout state machine, AP2 verification, MCP bindings
10	UCP Shopping Agent	AI shopping assistant	Multi-merchant discovery, comparison, LangGraph

Enterprise Frameworks (Level 4)

#	Project	What You'll Build	Key Skills	Repo
11	Compliance Audit Agents	Regulatory compliance system	Microsoft Agent Framework, graph workflows, middleware
12	Incident Response ADK	IT incident orchestrator	Google ADK, Sequential/Parallel/Loop agents
13	Contract Lifecycle Crew	Contract management platform	CrewAI, role-based agents, Flow orchestration

Quick Start (Any Project)

# Clone any project
git clone https://github.com/samuelvinay91/<project-name>.git
cd <project-name>

# Option 1: Docker (recommended)
docker compose up --build

# Option 2: Local
pip install -e ".[dev]"
uvicorn src.<package>.main:app --reload --port <port>

# Run tests
pytest tests/ -v

---

📚 Foundational Learning

Courses & Tutorials

Resource	Description	Level
Neural Networks: Zero to Hero	Andrej Karpathy's legendary series — build GPT from scratch	Beginner → Intermediate
fast.ai Practical Deep Learning	Top-down practical approach to deep learning	Beginner
Stanford CS224N: NLP with Deep Learning	Comprehensive NLP & transformer theory	Intermediate
Stanford CS229: Machine Learning	Andrew Ng's foundational ML course	Beginner → Intermediate
DeepLearning.AI Short Courses	Bite-sized courses on LangChain, RAG, fine-tuning, agents	All levels
Hugging Face NLP Course	Hands-on transformers & NLP with the HF ecosystem	Beginner → Intermediate
Full Stack LLM Bootcamp	Production LLM application development	Intermediate
LLM University by Cohere	Comprehensive LLM concepts and applications	Beginner → Intermediate

Prompt Engineering

Resource	Description
Anthropic Prompt Engineering Guide	Official Claude prompt engineering best practices
OpenAI Prompt Engineering Guide	Official GPT prompt engineering techniques
Prompt Engineering Guide	Community-driven comprehensive guide to prompting techniques
DAIR.AI Prompt Engineering	Research-backed prompting patterns and techniques

---

🤖 Agent Frameworks & Tools

Core Frameworks

Framework	Best For	Maintainer	Link
LangGraph	Complex stateful agent workflows with cycles	LangChain	Docs
CrewAI	Role-based multi-agent collaboration	CrewAI	Docs
Google ADK	Agent Development Kit with sequential/parallel patterns	Google	Docs
Microsoft Agent Framework	Enterprise agents with middleware pipelines	Microsoft	GitHub
OpenAI Agents SDK	Simple agent loops with handoffs	OpenAI	Docs
LlamaIndex	Data-centric RAG and agent workflows	LlamaIndex	Docs
AutoGen	Multi-agent conversation patterns	Microsoft	Docs
Semantic Kernel	Enterprise AI orchestration for .NET/Python/Java	Microsoft	Docs

Agent Protocols & Standards

Protocol	Purpose	Link
MCP (Model Context Protocol)	Standardized tool/resource access for LLMs	Spec
A2A (Agent-to-Agent)	Inter-agent communication protocol by Google	Spec
UCP (Universal Commerce Protocol)	Agentic commerce transactions	Spec
OpenAI Function Calling	Structured tool use for LLMs	Docs

Orchestration & Deployment

Tool	Purpose	Link
LangSmith	LLM observability, tracing, and evaluation	Docs
Weights & Biases Weave	AI application tracing and evaluation	Docs
Arize Phoenix	Open-source LLM observability	GitHub
Braintrust	Eval, logging, and prompt playground	Docs

---

⚙️ MLOps & Infrastructure

Model Serving

Tool	Purpose	Link
vLLM	High-throughput LLM serving with PagedAttention	GitHub
TensorRT-LLM	NVIDIA optimized LLM inference	GitHub
Triton Inference Server	Multi-framework model serving at scale	GitHub
Ollama	Run LLMs locally with one command	GitHub
llama.cpp	CPU/GPU inference for LLMs in C++	GitHub
SGLang	Fast serving framework for LLMs and VLMs	GitHub

ML Platforms & Experiment Tracking

Tool	Purpose	Link
MLflow	Experiment tracking, model registry, deployment	Docs
Weights & Biases	Experiment tracking, dataset versioning	Docs
Ray	Distributed compute for ML training and serving	Docs
DVC	Data version control and ML pipelines	Docs
Kubeflow	ML workflows on Kubernetes	Docs

Vector Databases

Database	Strengths	Link
Qdrant	Rust-based, fast, rich filtering	Docs
Pinecone	Fully managed, serverless option	Docs
Weaviate	Multi-modal, GraphQL API	Docs
ChromaDB	Simple, lightweight, great for prototyping	Docs
Milvus	Scalable, GPU-accelerated similarity search	Docs
pgvector	PostgreSQL extension — use your existing DB	GitHub

Fine-Tuning & Training

Tool	Purpose	Link
Hugging Face TRL	RLHF, DPO, SFT training	Docs
Unsloth	2-5x faster fine-tuning with 80% less memory	GitHub
Axolotl	Streamlined fine-tuning with YAML configs	GitHub
PEFT	Parameter-efficient fine-tuning (LoRA, QLoRA)	Docs
LitGPT	Pretrain, fine-tune, deploy LLMs	GitHub

---

🏗️ System Design for AI

Architecture Patterns

Pattern	When to Use	Project Example
RAG (Retrieval-Augmented Generation)	Ground LLM responses in external data	Agent RAG (#7)
Agentic RAG	Dynamic retrieval with query routing	Agent RAG (#7)
ReAct (Reason + Act)	Tool-using agents with reasoning traces	Ask-the-Web (#3)
Supervisor Pattern	Central agent delegates to specialists	Capstone (#6)
Sequential Pipeline	Ordered multi-step processing	Incident Response (#12)
Parallel Fan-Out	Independent tasks run concurrently	Incident Response (#12)
Human-in-the-Loop	Approval gates for high-stakes decisions	Compliance Audit (#11)
Event-Driven Flow	Conditional routing based on intermediate results	Contract Lifecycle (#13)
Middleware Pipeline	Cross-cutting concerns (auth, logging, PII redaction)	Compliance Audit (#11)
State Machine	Complex multi-step transactions	UCP Merchant (#9)

System Design Resources

Resource	Description	Link
Chip Huyen — AI Engineering	Definitive guide to building AI applications	Book
Designing Machine Learning Systems	ML system design end-to-end (Chip Huyen)	Book
Eugene Yan — Applied ML	Practical patterns for production ML	Blog
The AI Engineer's Handbook	Architecture patterns for LLM applications	Newsletter
System Design for ML	Interview prep meets real-world ML design	GitHub

Evaluation & Testing

Tool	Purpose	Link
RAGAS	RAG evaluation framework	Docs
DeepEval	LLM evaluation with 14+ metrics	GitHub
Promptfoo	LLM output testing and red-teaming	Docs
Inspect AI	AI safety evaluations by AISI	GitHub
Giskard	ML model testing and vulnerability scanning	GitHub

---

☁️ Cloud & Kubernetes for AI

Cloud AI Services

Provider	Key Services	Link
AWS	Bedrock (managed LLMs), SageMaker (training/serving), Inferentia	Docs
Google Cloud	Vertex AI, Gemini API, Cloud TPUs, GKE for ML	Docs
Azure	Azure OpenAI Service, Azure ML, AKS for AI	Docs

Kubernetes for AI Workloads

Resource	Description	Link
KubeFlow	End-to-end ML platform on Kubernetes	Docs
KServe	Standardized ML model serving on K8s	Docs
NVIDIA GPU Operator	Automated GPU management in K8s	Docs
Ray on Kubernetes	Distributed ML compute on K8s with KubeRay	Docs
Kustomize	Template-free K8s configuration management	Docs

Certifications

Certification	Provider	Focus
AWS Machine Learning Specialty	AWS	ML on AWS — SageMaker, data engineering, modeling
Google Professional ML Engineer	GCP	ML systems design on Google Cloud
Azure AI Engineer Associate	Azure	AI solution design on Azure
Terraform Associate	HashiCorp	Infrastructure as Code for AI infrastructure
CKA (Certified Kubernetes Admin)	CNCF	Kubernetes administration for AI workloads

---

📖 Books

Must-Read

Book	Author	Why It Matters
AI Engineering	Chip Huyen	The book for building LLM applications in production (2025)
Designing Machine Learning Systems	Chip Huyen	End-to-end ML system design — the architect's bible
Build a Large Language Model (From Scratch)	Sebastian Raschka	Deep understanding of transformer internals
Hands-On Large Language Models	Jay Alammar, Maarten Grootendorst	Practical LLM patterns with code
Natural Language Processing with Transformers	Lewis Tunstall et al.	HuggingFace-centric NLP from the team that built it

Deep Dives

Book	Author	Why It Matters
Generative Deep Learning (2nd Ed)	David Foster	Comprehensive generative AI — VAEs, GANs, diffusion, transformers
Deep Learning	Ian Goodfellow et al.	The foundational theory reference
Designing Data-Intensive Applications	Martin Kleppmann	System design fundamentals — essential for AI architects
Building Microservices (2nd Ed)	Sam Newman	Service architecture patterns used in AI platforms
The Staff Engineer's Path	Tanya Reilly	Leadership and influence for senior technical roles

---

📰 Blogs, Newsletters & Podcasts

Blogs

Blog	Author	Focus
Lil'Log	Lilian Weng (OpenAI)	Deep technical surveys on AI topics
Chip Huyen's Blog	Chip Huyen	ML systems, AI engineering, industry trends
Eugene Yan	Eugene Yan (Amazon)	Applied ML, RecSys, production patterns
Jay Alammar	Jay Alammar	Visual explanations of transformers and LLMs
Sebastian Raschka	Sebastian Raschka	LLM research, fine-tuning, practical AI
Simon Willison	Simon Willison	LLM tools, prompt engineering, practical AI
Hamel Husain	Hamel Husain	MLOps, LLM evaluation, practical engineering

Newsletters

Newsletter	Description	Link
The Batch	Andrew Ng's weekly AI news digest	Subscribe
Ahead of AI	Sebastian Raschka's research roundup	Subscribe
The AI Engineer	Swyx's newsletter on AI engineering	Subscribe
Interconnects	Nathan Lambert on RLHF, alignment, and LLMs	Subscribe
AI Tidbits	Sahar Mor's weekly AI news	Subscribe

Podcasts

Podcast	Description	Link
Latent Space	The AI engineer podcast — deep technical interviews	Listen
Practical AI	Real-world AI/ML applications and tools	Listen
TWIML AI	This Week in Machine Learning & AI	Listen
Gradient Dissent	Weights & Biases podcast on ML engineering	Listen
Lex Fridman Podcast	Long-form interviews with AI researchers	Listen

---

🌐 Open Source Models & Datasets

Key Model Families

Model	Developer	Strengths	Link
Llama 3/4	Meta	Best open-weight general-purpose LLMs	HuggingFace
Mistral / Mixtral	Mistral AI	Excellent efficiency, MoE architecture	HuggingFace
Gemma 2/3	Google	Strong small models (2B-27B)	HuggingFace
Qwen 2.5/3	Alibaba	Competitive multilingual models	HuggingFace
DeepSeek V3/R1	DeepSeek	Strong reasoning, open-weight	HuggingFace
Phi-3/4	Microsoft	Best-in-class small language models	HuggingFace
FLUX	Black Forest Labs	State-of-the-art open image generation	HuggingFace

Datasets & Benchmarks

Resource	Purpose	Link
Hugging Face Hub	Largest open model & dataset repository	Hub
MMLU / MMLU-Pro	Massive multitask language understanding benchmark	Paper
HumanEval / SWE-bench	Code generation benchmarks	GitHub
LMSYS Chatbot Arena	Crowdsourced LLM comparison leaderboard	Leaderboard
Open LLM Leaderboard	HuggingFace's open model rankings	Leaderboard

---

🛡️ AI Safety & Governance

Resource	Description	Link
OWASP Top 10 for LLMs	Security risks in LLM applications	Docs
NIST AI Risk Management Framework	Government framework for AI risk management	Docs
Anthropic Research	AI safety research and responsible scaling	Blog
EU AI Act	European regulation for AI systems	Overview
AI Alignment Forum	Technical AI safety research discussion	Forum

---

👥 Communities & Conferences

Communities

Community	Platform	Link
Hugging Face	Discord + Forums	Join
LangChain	Discord	Join
MLOps Community	Slack	Join
r/MachineLearning	Reddit	Visit
r/LocalLLaMA	Reddit	Visit
Latent Space	Discord	Join

Conferences

Conference	Focus	Link
NeurIPS	Top ML research conference	Site
ICML	International Conference on ML	Site
AI Engineer Summit	Applied AI engineering	Site
KubeCon	Cloud-native + AI infrastructure	Site

---

🧰 Developer Tools

Tool	Purpose	Link
Claude Code	AI coding assistant in the terminal	Docs
Cursor	AI-first code editor	Site
Continue	Open-source AI code assistant	GitHub
uv	Fast Python package manager (10-100x faster than pip)	GitHub
Ruff	Extremely fast Python linter and formatter	GitHub
Docker	Containerization for reproducible AI environments	Docs

---

🗓️ Learning Path (12-Week Plan)

For those who want a structured approach:

Week	Focus	Projects	Resources
1-2	LLM Fundamentals	#1 LLM Playground	Karpathy's Zero to Hero, HF NLP Course
3-4	Prompt Engineering & Fine-Tuning	#2 Customer Support Chatbot	Anthropic/OpenAI prompt guides, PEFT docs
5-6	Agents & Tool Use	#3 Ask-the-Web, #4 Deep Research	LangGraph docs, DeepLearning.AI courses
7-8	RAG & Multi-Agent Systems	#6 Capstone, #7 Agent RAG	LlamaIndex docs, RAG survey papers
9	Protocols & Interop	#8 MCP & A2A	MCP spec, A2A spec
10	Commerce & Real-World AI	#9 UCP Merchant, #10 Shopping Agent	UCP spec, state machine design
11	Enterprise Frameworks	#11 Compliance, #12 Incident Response	MS Agent Framework, Google ADK docs
12	Advanced Patterns & Portfolio	#13 Contract Lifecycle, polish portfolio	CrewAI docs, system design resources

---

🤝 Contributing

Contributions welcome! This list is community-maintained.

How to Contribute

1. Fork this repository
2. Add your resource in the appropriate category
3. Submit a pull request with a clear description

Guidelines

• Resources must be high-quality and actively maintained
• Prefer free/open-source resources, but paid resources are OK if they're exceptional
• Each entry needs a working link and brief description
• Follow the existing table format
• No duplicates — check existing entries first

See CONTRIBUTING.md for detailed guidelines.

---

📊 Star History

If you find this useful, give it a star! It helps others discover these resources.

---

License

This work is licensed under CC0 1.0 Universal. To the extent possible under law, the author has waived all copyright and related rights to this work.

---

Built with 💡 by samuelvinay91 — from the AI Engineer Portfolio project