Skip to content

Module08

Directory actions

More options

Directory actions

More options

Latest commit

 

History

History

Module08

README.md

Module 08: Hands on With Microsoft Foundry Local - Complete Developer Toolkit

Overview

Microsoft Foundry Local represents the next generation of edge AI development, providing developers with powerful tools to build, deploy, and scale AI applications locally while maintaining seamless integration with Azure AI Foundry. This module provides comprehensive coverage of Foundry Local from installation to advanced agent development.

Key Technologies:

  • Microsoft Foundry Local CLI and SDK
  • Azure AI Foundry integration
  • On-device model inference
  • Local model caching and optimization
  • Agent-based architectures

Learning Objectives

By completing this module, you will:

  • Master Foundry Local: Install, configure, and optimize for Windows 11 development
  • Deploy Diverse Models: Run phi, qwen, deepseek, and GPT models locally with CLI commands
  • Build Production Solutions: Create AI applications with advanced prompt engineering and data integration
  • Leverage Open-Source Ecosystem: Integrate Hugging Face models and community contributions
  • Develop AI Agents: Build intelligent agents with grounding and orchestration capabilities
  • Implement Enterprise Patterns: Create modular, scalable AI solutions for production deployment

Session Structure

1: Getting Started with Foundry Local

Focus: Installation, CLI setup, model deployment, and hardware optimization

Key Topics: Complete installation • CLI commands • Model caching • Hardware acceleration • Multi-model deployment

Sample: REST Chat Quickstart • OpenAI SDK Integration • Model Discovery & Benchmarking

Duration: 2-3 hours | Level: Beginner

2: Build AI Solutions with Azure AI Foundry

Focus: Advanced prompt engineering, data integration, and cloud connectivity

Key Topics: Prompt engineering • Data integration • Azure workflows • Performance optimization • Monitoring

Sample: Chainlit RAG Application

Duration: 2-3 hours | Level: Intermediate

3: Open-Source Models Foundry Local

Focus: Hugging Face integration, BYOM strategies, and community models

Key Topics: HuggingFace integration • Bring-your-own-model • Model Mondays insights • Community contributions • Model selection

Sample: Multi-Agent Orchestration

Duration: 2-3 hours | Level: Intermediate

4: Explore Cutting-Edge Models

Focus: LLMs vs SLMs, EdgeAI implementation, and advanced demos

Key Topics: Model comparison • Edge vs cloud inference • Phi + ONNX Runtime • Chainlit RAG app • WebGPU optimization

Sample: Models-as-Tools Router

Duration: 3-4 hours | Level: Advanced

5: Build AI-Powered Agents Fast

Focus: Agent architectures, system prompts, grounding, and orchestration

Key Topics: Agent design patterns • System prompt engineering • Grounding techniques • Multi-agent systems • Production deployment

Sample: Multi-Agent Orchestration • Advanced Multi-Agent System

Duration: 3-4 hours | Level: Advanced

6: Foundry Local - Models as Tools

Focus: Modular AI solutions, enterprise scaling, and production patterns

Key Topics: Models as tools • On-device deployment • SDK/API integration • Enterprise architectures • Scaling strategies

Sample: Models-as-Tools Router • Foundry Tools Framework

Duration: 3-4 hours | Level: Expert

7: Direct API Integration Patterns

Focus: Pure REST API integration without SDK dependencies for maximum control

Key Topics: HTTP client implementation • Custom authentication • Model health monitoring • Streaming responses • Production error handling

Sample: Direct API Client

Duration: 2-3 hours | Level: Intermediate

8: Windows 11 Native Chat Application

Focus: Building modern native chat applications with Foundry Local integration

Key Topics: Electron development • Fluent Design System • Native Windows integration • Real-time streaming • Chat interface design

Sample: Windows 11 Chat Application

Duration: 3-4 hours | Level: Advanced

9: Advanced Multi-Agent Orchestration

Focus: Sophisticated agent coordination, specialized task delegation, and collaborative AI workflows

Key Topics: Intelligent agent coordination • Function calling patterns • Cross-agent communication • Workflow orchestration • Quality assurance mechanisms

Sample: Advanced Multi-Agent System

Duration: 4-5 hours | Level: Expert

10: Foundry Local as Tools Framework

Focus: Tool-first architecture for integrating Foundry Local into existing applications and frameworks

Key Topics: LangChain integration • Semantic Kernel functions • REST API frameworks • CLI tools • Jupyter integration • Production deployment patterns

Sample: Foundry Tools Framework

Duration: 4-5 hours | Level: Expert

Prerequisites

System Requirements

  • Operating System: Windows 11 (22H2 or later)
  • Memory: 16GB RAM (32GB recommended for larger models)
  • Storage: 50GB free space for model caching
  • Hardware: NPU-enabled device recommended (Copilot+ PC), GPU optional
  • Network: High-speed internet for initial model downloads

Development Environment

  • Visual Studio Code with AI Toolkit extension
  • Python 3.10+ and pip
  • Git for version control
  • PowerShell or Command Prompt
  • Azure CLI (optional for cloud integration)

Knowledge Prerequisites

  • Basic understanding of AI/ML concepts
  • Command line familiarity
  • Python programming basics
  • REST API concepts
  • Basic knowledge of prompting and model inference

Module Timeline

Total Estimated Time: 30-38 hours

Session Focus Area Samples Time Complexity
1 Setup & Basics 01, 02, 03 2-3 hours Beginner
2 AI Solutions 04 2-3 hours Intermediate
3 Open Source 05 2-3 hours Intermediate
4 Advanced Models 06 3-4 hours Advanced
5 AI Agents 05, 09 3-4 hours Advanced
6 Enterprise Tools 06, 10 3-4 hours Expert
7 Direct API Integration 07 2-3 hours Intermediate
8 Windows 11 Chat App 08 3-4 hours Advanced
9 Advanced Multi-Agent 09 4-5 hours Expert
10 Tools Framework 10 4-5 hours Expert

Key Resources

Official Documentation:

Community & Support:

Learning Outcomes

Upon completing this module, you will be equipped to:

Technical Mastery

  • Deploy and Manage: Foundry Local installations across development and production environments
  • Integrate Models: Seamlessly work with diverse model families from Microsoft, Hugging Face, and community sources
  • Build Applications: Create production-ready AI applications with advanced features and optimizations
  • Develop Agents: Implement sophisticated AI agents with grounding, reasoning, and tool integration

Strategic Understanding

  • Architecture Decisions: Make informed choices between local vs cloud deployment
  • Performance Optimization: Optimize inference performance across different hardware configurations
  • Enterprise Scaling: Design applications that scale from local prototypes to enterprise deployments
  • Privacy and Security: Implement privacy-preserving AI solutions with local inference

Innovation Capabilities

  • Rapid Prototyping: Quickly build and test AI application concepts across all 10 sample patterns
  • Community Integration: Leverage open-source models and contribute to the ecosystem
  • Advanced Patterns: Implement cutting-edge AI patterns including RAG, agents, and tool integration
  • Framework Mastery: Expert-level integration with LangChain, Semantic Kernel, Chainlit, and Electron
  • Production Deployment: Deploy scalable AI solutions from local prototypes to enterprise systems
  • Future-Ready Development: Build applications ready for emerging AI technologies and patterns

Getting Started

  1. Environment Setup: Ensure Windows 11 with recommended hardware (see Prerequisites)
  2. Install Foundry Local: Follow Session 1 for complete installation and configuration
  3. Run Sample 01: Start with basic REST API integration to verify setup
  4. Progress Through Samples: Complete samples 01-10 for comprehensive mastery

Success Metrics

Track your progress through all 10 comprehensive samples:

Foundation Level (Samples 01-03)

  • Successfully install and configure Foundry Local
  • Complete REST API integration (Sample 01)
  • Implement OpenAI SDK compatibility (Sample 02)
  • Perform model discovery and benchmarking (Sample 03)

Application Level (Samples 04-06)

  • Deploy and run at least 4 different model families
  • Build a functional RAG chat application (Sample 04)
  • Create multi-agent orchestration system (Sample 05)
  • Implement intelligent model routing (Sample 06)

Advanced Integration Level (Samples 07-10)

  • Build production-ready API client (Sample 07)
  • Develop Windows 11 native chat application (Sample 08)
  • Implement advanced multi-agent system (Sample 09)
  • Create comprehensive tools framework (Sample 10)

Mastery Indicators

  • Successfully run all 10 samples without errors
  • Customize at least 3 samples for specific use cases
  • Deploy 2+ samples in production-like environments
  • Contribute improvements or extensions to sample code
  • Integrate Foundry Local patterns into personal/professional projects

Quick Start Guide - All 10 Samples

Environment Setup (Required for All Samples)

# 1. Clone and navigate to Module08
cd Module08

# 2. Create Python virtual environment
py -m venv .venv
.\.venv\Scripts\activate

# 3. Install base dependencies
pip install -r requirements.txt

# 4. Install Foundry Local (if not already installed)
winget install Microsoft.FoundryLocal

# 5. Verify Foundry Local installation
foundry --version
foundry model list

Core Foundation Samples (01-06)

Sample 01: REST Chat Quickstart

# Start Foundry Local service
foundry model run phi-4-mini

# Run REST chat demo
python samples/01/chat_quickstart.py

Sample 02: OpenAI SDK Integration

# Ensure model is running
foundry status

# Run SDK demo
python samples/02/sdk_quickstart.py

Sample 03: Model Discovery & Benchmarking

# Run comprehensive model testing
samples/03/list_and_bench.cmd

# Or run individual components
foundry model list --available
foundry model download qwen2.5-0.5b
foundry model benchmark phi-4-mini

Sample 04: Chainlit RAG Application

# Install Chainlit dependencies
pip install chainlit langchain chromadb

# Start RAG chat application
chainlit run samples/04/app.py -w
# Opens browser at http://localhost:8000

Sample 05: Multi-Agent Orchestration

# Run agent coordinator demo
python -m samples.05.agents.coordinator

# Run specific agent examples
python samples/05/examples/specialists_demo.py

Sample 06: Models-as-Tools Router

# Configure environment
set BASE_URL=http://localhost:8000
set GENERAL_MODEL=phi-4-mini
set CODE_MODEL=qwen2.5-7b

# Run intelligent router
python samples/06/router.py "Analyze this Python code for performance issues"

Advanced Integration Samples (07-10)

Sample 07: Direct API Client

# Navigate to sample directory
cd samples/07

# Install additional dependencies
pip install -r requirements.txt

# Run basic API examples
python examples/basic_usage.py

# Try streaming responses
python examples/streaming.py

# Test production patterns
python examples/production.py

Sample 08: Windows 11 Chat Application

# Navigate to sample directory
cd samples/08

# Install Node.js dependencies
npm install

# Start Electron application
npm start

# Or build for production
npm run build

Sample 09: Advanced Multi-Agent System

# Navigate to sample directory
cd samples/09

# Install agent system dependencies
pip install -r requirements.txt

# Run basic coordination example
python examples/basic_coordination.py

# Try complex workflow
python examples/complex_workflow.py

# Interactive agent demo
python examples/interactive_demo.py

Sample 10: Foundry Tools Framework

# Navigate to sample directory
cd samples/10

# Install framework dependencies
pip install -r requirements.txt

# Run basic tools demo
python examples/basic_tools.py

# Start REST API server
python examples/rest_api_server.py
# API available at http://localhost:8080

# Try CLI application
python examples/cli_application.py --help

# Launch Jupyter notebook
jupyter notebook examples/jupyter_notebook.ipynb

# Test LangChain integration
python examples/langchain_demo.py

Troubleshooting Common Issues

Foundry Local Connection Errors

# Check service status
foundry status

# Restart if needed
foundry restart

# Verify endpoint accessibility
curl http://localhost:5273/v1/models

Model Loading Issues

# Check available models
foundry model list --cached

# Download missing models
foundry model download phi-4-mini
foundry model download qwen2.5-0.5b

# Force reload if needed
foundry model unload --all
foundry model run phi-4-mini

Dependency Issues

# Upgrade pip and reinstall
python -m pip install --upgrade pip
pip install -r requirements.txt --force-reinstall

# For Node.js samples
npm cache clean --force
npm install

Summary

This module represents the cutting edge of edge AI development, combining Microsoft's enterprise-grade tools with the flexibility and innovation of the open-source ecosystem. By mastering Foundry Local through all 10 comprehensive samples, you'll be positioned at the forefront of AI application development.

Complete Learning Path:

  • Foundation (Samples 01-03): API integration and model management
  • Applications (Samples 04-06): RAG, agents, and intelligent routing
  • Advanced (Samples 07-10): Production frameworks and enterprise integration

For Azure OpenAI integration (Session 2), see the individual sample README files for required environment variables and API version settings.