Varlord ⚙️

Stop wrestling with configuration chaos. Start with Varlord.

Varlord is a battle-tested Python configuration management library that eliminates the pain of managing configuration from multiple sources. Born from real-world production challenges, it provides a unified, type-safe, and elegant solution for configuration management.

🎯 The Problem We Solve

Real-World Configuration Nightmares

Every Python developer has faced these frustrating scenarios:

❌ The Configuration Spaghetti

# Your code becomes a mess of conditionals and parsing
host = os.getenv("HOST", "127.0.0.1")
port = int(os.getenv("PORT", "8000"))  # What if PORT is not a number?
debug = os.getenv("DEBUG", "false").lower() == "true"  # Really?
if "--host" in sys.argv:
    host = sys.argv[sys.argv.index("--host") + 1]  # Error-prone parsing
# ... and it gets worse with nested configs, validation, etc.

❌ Priority Confusion

"Does CLI override env? Or env overrides CLI? Wait, what about the config file? Which one wins?"

❌ Type Conversion Hell

# String "true" vs boolean True vs "1" vs 1
# "8000" vs 8000
# Missing values, None handling, type errors at runtime...

❌ The Restart Tax

"I just need to change one config value. Why do I have to restart the entire service?"

❌ Silent Failures

"The config looks wrong, but the app starts anyway. Users report bugs 3 hours later."

✅ Varlord's Solution

One unified interface. Multiple sources. Clear priority. Built-in diagnostics.

from dataclasses import dataclass, field
from pathlib import Path
from typing import Optional
from varlord import Config, sources

@dataclass(frozen=True)
class AppConfig:
    """Application configuration with clear structure and validation."""
    
    # Required field - must be provided
    api_key: str = field(metadata={"description": "API key for authentication"})
    
    # Optional fields with sensible defaults
    host: str = field(default="127.0.0.1", metadata={"description": "Server host address"})
    port: int = field(default=8000, metadata={"description": "Server port number"})
    debug: bool = field(default=False, metadata={"description": "Enable debug mode"})
    timeout: float = field(default=30.0, metadata={"description": "Request timeout in seconds"})
    hello_message: Optional[str] = field(
        default=None, metadata={"description": "Optional greeting message"}
    )

def main():
    # Define configuration sources with clear priority order
    # Priority: CLI (highest) > User Config > App Config > System Config > Env > Defaults (lowest)
    cfg = Config(
        model=AppConfig,
        sources=[
            # System-wide configuration (lowest priority, rarely overridden)
            sources.YAML("/etc/myapp/config.yaml"),  # System config
            
            # Application-level configuration
            sources.JSON(Path(__file__).parent / "config.json"),  # App directory
            
            # User-specific configuration (overrides system and app configs)
            sources.YAML(Path.home() / ".config" / "myapp" / "config.yaml"),  # User directory
            sources.TOML(Path.home() / ".myapp.toml"),  # Alternative user config
            
            # Environment variables (common in containers/CI)
            sources.Env(),
            sources.DotEnv(".env"),  # Local development
            
            # Command-line arguments (highest priority, for debugging/overrides)
            sources.CLI(),
        ],
    )
    
    # One line to add comprehensive CLI management: --help, --check-variables, etc.
    # This single call adds:
    #   - --help / -h: Auto-generated help from your model metadata
    #   - --check-variables / -cv: Complete configuration diagnostics
    #   - Automatic validation and error reporting
    #   - Exit handling (exits if help/cv is requested)
    cfg.handle_cli_commands()  # Handles --help, -cv automatically, exits if needed
    
    # Load configuration - type-safe, validated, ready to use
    app = cfg.load()
    
    # Your application code
    print(f"Starting server on {app.host}:{app.port}")
    print(f"Debug: {app.debug}, Timeout: {app.timeout}s")

if __name__ == "__main__":
    main()

What just happened?

✅ Multiple Sources, Unified Interface: System config, app config, user config, env vars, CLI - all handled the same way
✅ Clear Priority: Later sources override earlier ones - no confusion
✅ Automatic Type Conversion: Strings from files/env → proper types (int, bool, float)
✅ Model-Driven Filtering: Each source only reads fields defined in your model
✅ Built-in Diagnostics: --check-variables shows exactly what's loaded from where
✅ Zero Boilerplate: No parsing, no type conversion code, no priority logic

Try it:

# See comprehensive configuration diagnostics
python app.py --check-variables
# or short form
python app.py -cv

# See help with all sources and priority
python app.py --help

# Run normally
python app.py --api-key your_key

The --check-variables output shows everything:

When you run python app.py -cv, you get a comprehensive diagnostic report:

+---------------+----------+---------------+----------+-----------+
| Variable      | Required | Status        | Source   | Value     |
+---------------+----------+---------------+----------+-----------+
| api_key       | Required | Missing       | defaults | None      |
| host          | Optional | Loaded        | dotenv   | localhost |
| port          | Optional | Loaded        | dotenv   | 7000      |
| debug         | Optional | Loaded        | dotenv   | true      |
| timeout       | Optional | Loaded        | dotenv   | 20.0      |
| hello_message | Optional | Using Default | defaults | None      |
+---------------+----------+---------------+----------+-----------+

Configuration Source Priority and Details:

+------------+-------------+-----------+----------------------------------------+--------+----------------+---------------+-------------+
| Priority   | Source Name | Source ID | Instance                               | Status | Load Time (ms) | Watch Support | Last Update |
+------------+-------------+-----------+----------------------------------------+--------+----------------+---------------+-------------+
| 1 (lowest) | defaults    | defaults  | <Defaults(model=AppConfig)>            | Active | 0.00           | No            | N/A         |
| 2          | yaml        | yaml      | <YAML(/etc/myapp/config.yaml)>         | Active | 0.15           | No            | N/A         |
| 3          | json        | json      | <JSON(config.json)>                    | Active | 0.08           | No            | N/A         |
| 4          | yaml        | yaml      | <YAML(~/.config/myapp/config.yaml)>    | Active | 0.12           | No            | N/A         |
| 5          | toml        | toml      | <TOML(~/.myapp.toml)>                  | Active | 0.05           | No            | N/A         |
| 6          | env         | env       | <Env(model-based)>                     | Active | 0.05           | No            | N/A         |
| 7          | dotenv      | dotenv    | <DotEnv(.env)>                         | Active | 0.03           | No            | N/A         |
| 8 (highest)| cli         | cli       | <CLI()>                                | Active | 0.20           | No            | N/A         |
+------------+-------------+-----------+----------------------------------------+--------+----------------+---------------+-------------+


Note: Later sources override earlier ones (higher priority).

⚠️  Missing required fields: api_key
   Exiting with code 1. Please provide these fields and try again.
   For help, run: python app.py --help

What this tells you:

Variable Status: See which fields are required vs optional, loaded vs missing
Source Tracking: Know exactly which source (defaults/env/cli/file) provided each value
Priority Order: Understand the resolution chain - later sources override earlier
Performance: Load times for each source (useful for optimization)
Validation: Missing required fields are caught immediately with clear error messages

Key Benefits:

🔍 Complete Visibility: See exactly which source provides each value - no more guessing where config comes from
📊 Priority Visualization: Understand the resolution order at a glance - see which source wins for each field
⚡ Performance Metrics: Load times for each source - identify slow config sources
🛡️ Validation: Missing required fields are caught before app starts - fail fast with clear errors
📝 Self-Documenting: Help text generated from your model metadata - no manual documentation needed
🎯 Zero Configuration: handle_cli_commands() adds all this with one line - no boilerplate

Real-World Scenarios:

Debugging: "Why is my app using the wrong port?" → python app.py -cv shows port comes from env, not CLI
Onboarding: New team member runs python app.py --help → sees all config options with descriptions
CI/CD: Missing required field? → -cv shows exactly what's missing before deployment fails
Multi-Environment: See which config file (system/user/app) is actually being used

That's it. No parsing, no type conversion, no priority confusion. Just clean, type-safe configuration with built-in diagnostics.

🌟 Why Varlord?

🎯 Core Value Propositions

Problem	Varlord Solution	Impact
Config scattered everywhere	Unified interface for all sources	Single source of truth
Priority confusion	Simple rule: later sources override earlier	Predictable behavior
Type conversion errors	Automatic conversion with validation	Catch errors early
No runtime updates	Optional etcd watch for dynamic updates	Zero-downtime config changes
Repetitive boilerplate	Model-driven, auto-filtering	90% less code
Silent failures	Built-in validation framework	Fail fast, fail clear

💡 Key Differentiators

🎯 Model-Driven Design: Define your config once as a dataclass, and Varlord handles the rest
🔄 Smart Auto-Filtering: Sources automatically filter by model fields - no prefix management needed
⚡ Zero Boilerplate: Model defaults are automatic, model is auto-injected to sources
🛡️ Type Safety First: Full type hints support with automatic conversion and validation
🚀 Production Ready: Thread-safe, fail-safe, battle-tested in production environments

🚀 Quick Start

Installation

pip install varlord

# With optional features
pip install varlord[dotenv,etcd]

Basic Usage (30 seconds)

from dataclasses import dataclass, field
from varlord import Config, sources

@dataclass(frozen=True)
class AppConfig:
    host: str = field(default="127.0.0.1")
    port: int = field(default=8000)
    debug: bool = field(default=False)

# Create config - that's it!
cfg = Config(
    model=AppConfig,
    sources=[
        sources.Env(),   # Reads HOST, PORT, DEBUG from environment
        sources.CLI(),   # Reads --host, --port, --debug from CLI
    ],
)

app = cfg.load()  # Type-safe, validated config object
print(f"Server: {app.host}:{app.port}, Debug: {app.debug}")

Run it:

# Use defaults
python app.py
# Output: Server: 127.0.0.1:8000, Debug: False

# Override with env
export HOST=0.0.0.0 PORT=9000
python app.py
# Output: Server: 0.0.0.0:9000, Debug: False

# Override with CLI (highest priority)
python app.py --host 192.168.1.1 --port 8080 --debug
# Output: Server: 192.168.1.1:8080, Debug: True

One-Liner Convenience Method

# Even simpler for common cases
cfg = Config.from_model(AppConfig, cli=True, dotenv=".env")
app = cfg.load()

💼 Real-World Use Cases

Use Case 1: Microservice Configuration

Problem: Your microservice needs config from multiple sources, and you're tired of writing parsing code.

Solution:

@dataclass(frozen=True)
class ServiceConfig:
    db_host: str = field(default="localhost")
    db_port: int = field(default=5432)
    api_key: str = field()  # Required - must be provided
    log_level: str = field(default="INFO")
    max_workers: int = field(default=4)

cfg = Config(
    model=ServiceConfig,
    sources=[
        sources.Env(),           # Production: from environment
        sources.DotEnv(".env"),  # Development: from .env file
        sources.CLI(),           # Override: from command line
    ],
)

config = cfg.load()  # Validated, type-safe, ready to use

Benefits:

✅ Same code works in dev (.env), staging (env vars), and prod (env vars)
✅ CLI overrides for debugging: python service.py --log-level DEBUG
✅ Type safety: max_workers is always an int, never a string
✅ Validation: Missing api_key fails fast with clear error

Use Case 2: Dynamic Configuration Updates

Problem: You need to change configuration without restarting the service.

Solution:

def on_config_change(new_config, diff):
    print(f"Config updated: {diff}")
    # Update your app's behavior based on new config

cfg = Config(
    model=AppConfig,
    sources=[
        sources.Env(),
        sources.Etcd(
            host="etcd.example.com",
            prefix="/app/config/",
            watch=True,  # Enable dynamic updates
        ),
    ],
)

store = cfg.load_store()  # Returns ConfigStore for dynamic updates
store.subscribe(on_config_change)

# Thread-safe access to current config
current = store.get()

Benefits:

✅ Zero-downtime configuration updates
✅ Thread-safe concurrent access
✅ Automatic validation on updates
✅ Change notifications via callbacks

Use Case 3: Multi-Environment Deployment

Problem: Different configs for dev, staging, and production, but you want one codebase.

Solution:

# Development: .env file
# Staging: Environment variables
# Production: etcd + environment variables

cfg = Config(
    model=AppConfig,
    sources=[
        sources.DotEnv(".env"),  # Dev only (file may not exist in prod)
        sources.Env(),            # All environments
        sources.Etcd.from_env() if os.getenv("ETCD_HOST") else None,  # Prod only
        sources.CLI(),           # Override for debugging
    ],
)

Benefits:

✅ One codebase, multiple environments
✅ Environment-specific sources automatically handled
✅ Clear priority: CLI > etcd > env > .env > defaults

Use Case 4: Complex Nested Configuration

Problem: Your config has nested structures (database, cache, API keys, etc.).

Solution:

@dataclass(frozen=True)
class DatabaseConfig:
    host: str = field(default="localhost")
    port: int = field(default=5432)
    name: str = field(default="mydb")

@dataclass(frozen=True)
class AppConfig:
    db: DatabaseConfig = field(default_factory=DatabaseConfig)
    api_key: str = field()
    cache_ttl: int = field(default=3600)

cfg = Config(
    model=AppConfig,
    sources=[
        sources.Env(),  # Reads DB__HOST, DB__PORT, DB__NAME automatically
        sources.CLI(),  # Reads --db-host, --db-port, etc.
    ],
)

config = cfg.load()
# Access: config.db.host, config.db.port, config.api_key

Benefits:

✅ Automatic nested key mapping (DB__HOST → db.host)
✅ Type-safe nested access
✅ Validation at all levels

🎨 Key Features

1. Multiple Sources, Unified Interface

sources = [
    sources.Defaults(),      # From model defaults (automatic)
    sources.Env(),           # From environment variables
    sources.CLI(),           # From command-line arguments
    sources.DotEnv(".env"),  # From .env files
    sources.YAML("config.yaml"),  # From YAML files
    sources.TOML("config.toml"),  # From TOML files
    sources.Etcd(...),       # From etcd (optional)
]

2. Simple Priority Rule

Later sources override earlier ones. That's it.

cfg = Config(
    model=AppConfig,
    sources=[
        sources.Env(),   # Priority 1 (lowest)
        sources.CLI(),   # Priority 2 (highest - overrides env)
    ],
)

3. Automatic Type Conversion

# Environment variables are strings, but Varlord converts them automatically
export PORT=9000 DEBUG=true TIMEOUT=30.5

@dataclass(frozen=True)
class Config:
    port: int = 8000        # "9000" → 9000
    debug: bool = False     # "true" → True
    timeout: float = 30.0   # "30.5" → 30.5

4. Model-Driven Filtering

# Your model defines what config you need
@dataclass(frozen=True)
class Config:
    host: str = "127.0.0.1"
    port: int = 8000
    # ... only these fields

# Sources automatically filter - no prefix management needed
# Env source only reads HOST and PORT, ignores everything else
# CLI source only parses --host and --port, ignores other args

5. Built-in Validation

from varlord.validators import validate_range, validate_regex

@dataclass(frozen=True)
class Config:
    port: int = field(default=8000)
    host: str = field(default="127.0.0.1")
    
    def __post_init__(self):
        validate_range(self.port, min=1, max=65535)
        validate_regex(self.host, r'^\d+\.\d+\.\d+\.\d+$')

6. Dynamic Updates (Optional)

store = cfg.load_store()  # Enable watch if sources support it
store.subscribe(lambda new_config, diff: print(f"Updated: {diff}"))

# Config updates automatically in background
# Thread-safe access: current = store.get()

📚 Documentation

📖 Full Documentation: https://varlord.readthedocs.io
🚀 Quick Start Guide: Quick Start
💡 Examples: Examples Directory
🎯 API Reference: API Documentation

🧠 Memory Aids (Quick Reference)

The Varlord Mantra

"Define once, use everywhere. Later overrides earlier. Types are automatic."

Priority Cheat Sheet

Defaults < .env < Environment < YAML/TOML < etcd < CLI
  (lowest priority)                          (highest priority)

Common Patterns

# Pattern 1: Simple (most common)
Config(model=AppConfig, sources=[sources.Env(), sources.CLI()])

# Pattern 2: With .env file
Config(model=AppConfig, sources=[sources.DotEnv(".env"), sources.Env(), sources.CLI()])

# Pattern 3: Dynamic updates
Config(model=AppConfig, sources=[sources.Env(), sources.Etcd(..., watch=True)])
store = cfg.load_store()

# Pattern 4: One-liner
Config.from_model(AppConfig, cli=True, dotenv=".env")

🏢 Production Proven

Varlord is part of the Agentsmith ecosystem, battle-tested in production environments:

✅ Deployed in multiple highway management companies
✅ Used by securities firms and regulatory agencies
✅ Handles high-throughput microservices
✅ Thread-safe and production-ready

🌟 Agentsmith Open-Source Projects

Varlord ⚙️ - Configuration management (this project)
Routilux ⚡ - Event-driven workflow orchestration
Serilux 📦 - Flexible serialization framework
Lexilux 🚀 - Unified LLM API client

🤝 Contributing

We welcome contributions! See CONTRIBUTING.md for guidelines.

📄 License

Licensed under the Apache License 2.0. See LICENSE for details.

🎯 TL;DR

Varlord solves configuration management once and for all:

✅ Define your config as a dataclass - type-safe, validated
✅ Add sources in priority order - later overrides earlier
✅ Call load() - get a type-safe config object
✅ Optional: Enable dynamic updates - zero-downtime config changes

No more parsing. No more type conversion. No more priority confusion.

# Before: 50+ lines of parsing, type conversion, validation
# After: 3 lines
cfg = Config(model=AppConfig, sources=[sources.Env(), sources.CLI()])
app = cfg.load()

That's the Varlord promise. 🚀

Name		Name	Last commit message	Last commit date
Latest commit History 51 Commits
.github		.github
docs		docs
examples		examples
scripts		scripts
tests		tests
varlord		varlord
.env.example		.env.example
.gitignore		.gitignore
.readthedocs.yml		.readthedocs.yml
CHANGELOG.md		CHANGELOG.md
CLAUDE.md		CLAUDE.md
CONTRIBUTING.md		CONTRIBUTING.md
LICENSE		LICENSE
MANIFEST.in		MANIFEST.in
Makefile		Makefile
README.md		README.md
REFACTORING_PROGRESS.md		REFACTORING_PROGRESS.md
SETUP.md		SETUP.md
pyproject.toml		pyproject.toml
pytest.ini		pytest.ini
setup.py		setup.py
uv.lock		uv.lock

Folders and files

Latest commit

History

Repository files navigation

Varlord ⚙️

🎯 The Problem We Solve

Real-World Configuration Nightmares

❌ The Configuration Spaghetti

❌ Priority Confusion

❌ Type Conversion Hell

❌ The Restart Tax

❌ Silent Failures

✅ Varlord's Solution

🌟 Why Varlord?

🎯 Core Value Propositions

💡 Key Differentiators

🚀 Quick Start

Installation

Basic Usage (30 seconds)

One-Liner Convenience Method

💼 Real-World Use Cases

Use Case 1: Microservice Configuration

Use Case 2: Dynamic Configuration Updates

Use Case 3: Multi-Environment Deployment

Use Case 4: Complex Nested Configuration

🎨 Key Features

1. Multiple Sources, Unified Interface

2. Simple Priority Rule

3. Automatic Type Conversion

4. Model-Driven Filtering

5. Built-in Validation

6. Dynamic Updates (Optional)

📚 Documentation

🧠 Memory Aids (Quick Reference)

The Varlord Mantra

Priority Cheat Sheet

Common Patterns

🏢 Production Proven

🌟 Agentsmith Open-Source Projects

🤝 Contributing

📄 License

🎯 TL;DR

About

Topics

Resources

License

Contributing

Uh oh!

Stars

Watchers

Forks

Releases 10

Contributors

Uh oh!

Languages