GitHub - Kirky-X/inklog: A high-performance, secure, and feature-rich logging solution built on Tokio. Features multi-target output (console, file, database, S3), AES-256-GCM encryption, Regex-based PII masking, log rotation & compression, structured logging, health monitoring, and metrics.

Enterprise-grade Rust Logging Infrastructure

✨ Features • 🚀 Quick Start • 📚 Documentation • 💻 Examples • 🤝 Contributing

🎯 A high-performance, secure, and feature-rich logging infrastructure built on Tokio

Inklog provides a comprehensive logging solution for enterprise applications:

⚡ High Performance	🔒 Security First	🌐 Multi-Target	📊 Observability
Async I/O with Tokio	AES-256-GCM encryption	Console, File, DB, S3	Health monitoring
Batch writes & compression	Zeroized secret memory	Automatic rotation	Metrics & tracing

use inklog::{InklogConfig, LoggerManager};
use std::path::PathBuf;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let config = InklogConfig {
        file_sink: Some(inklog::FileSinkConfig {
            enabled: true,
            path: "logs/app.log".into(),
            max_size: "100MB".into(),
            compress: true,
            ..Default::default()
        }),
        ..Default::default()
    };

    let _logger = LoggerManager::with_config(config).await?;

    log::info!("Application started successfully");
    log::error!("Something went wrong with error details");

    Ok(())
}

📋 Table of Contents

📑 Table of Contents (Click to expand)

✨ Features
🚀 Quick Start
🎨 Feature Flags
📚 Documentation
💻 Examples
🏗️ Architecture
🔒 Security
🧪 Testing
🤝 Contributing
📄 License
🙏 Acknowledgments

✨ Features

🎯 Core Features	⚡ Enterprise Features
Always Available	Optional

🎯 Core Features (Always Available)

Status	Feature	Description
✅	Async I/O	Tokio-powered non-blocking logging
✅	Multi-Target Output	Console, file, database, custom sinks
✅	Structured Logging	tracing ecosystem integration
✅	Custom Formatting	Template-based log format
✅	File Rotation	Size-based and time-based rotation
✅	Data Masking	Regex-based PII redaction
✅	Health Monitoring	Sink status and metrics tracking
✅	CLI Tools	decrypt, generate, validate commands

⚡ Enterprise Features

Status	Feature	Description
🔍	Compression	ZSTD, GZIP, Brotli, LZ4 support (`zstd`, `flate2`, etc.)
🔒	Encryption	AES-256-GCM file encryption (`aes-gcm`)
🗄️	Database Sink	PostgreSQL, MySQL, SQLite via Sea-ORM
☁️	S3 Archive	Cloud log archival with AWS SDK S3 (`aws` feature)
📊	Parquet Export	Analytics-ready log format (`parquet` feature)
🌐	HTTP Endpoint	Axum-based health check server (`http` feature)
📅	Scheduled Tasks	Cron-based archive scheduling
🔧	CLI Tools	Utility commands for log management (`cli` feature)
📝	TOML Config	External configuration support (`confers` feature)

📦 Feature Presets

Preset	Features	Use Case
minimal	No optional features	Core logging only
standard	`http`, `cli`	Standard development setup
full	All default features	Production-ready logging

🚀 Quick Start

📦 Installation

Add this to your Cargo.toml:

[dependencies]
inklog = "0.1"

For full feature set:

[dependencies]
inklog = { version = "0.1", features = ["default"] }

💡 Basic Usage

🎬 5-Minute Quick Start

Step 1: Initialize Logger

use inklog::LoggerManager;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let _logger = LoggerManager::new().await?;

    log::info!("Logger initialized");
    Ok(())
}

Step 2: Log Messages

use inklog::LoggerManager;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let _logger = LoggerManager::new().await?;

    log::trace!("Trace message");
    log::debug!("Debug message");
    log::info!("Info message");
    log::warn!("Warning message");
    log::error!("Error message");

    Ok(())
}

Step 3: File Logging

use inklog::{FileSinkConfig, InklogConfig, LoggerManager};

let config = InklogConfig {
    file_sink: Some(FileSinkConfig {
        enabled: true,
        path: "logs/app.log".into(),
        max_size: "10MB".into(),
        rotation_time: "daily".into(),
        keep_files: 7,
        compress: true,
        ..Default::default()
    }),
    ..Default::default()
};

let _logger = LoggerManager::with_config(config).await?;

Step 4: Database Logging

use inklog::{DatabaseSinkConfig, InklogConfig, config::DatabaseDriver};

let config = InklogConfig {
    database_sink: Some(DatabaseSinkConfig {
        enabled: true,
        driver: DatabaseDriver::SQLite,
        url: "sqlite://logs/app.db".to_string(),
        pool_size: 5,
        batch_size: 100,
        flush_interval_ms: 1000,
        ..Default::default()
    }),
    ..Default::default()
};

let _logger = LoggerManager::with_config(config).await?;

🔧 Advanced Configuration

Encrypted File Logging

use inklog::{FileSinkConfig, InklogConfig};

// Set encryption key from environment
std::env::set_var("INKLOG_ENCRYPTION_KEY", "base64-encoded-32-byte-key");

let config = InklogConfig {
    file_sink: Some(FileSinkConfig {
        enabled: true,
        path: "logs/encrypted.log.enc".into(),
        max_size: "10MB".into(),
        encrypt: true,
        encryption_key_env: Some("INKLOG_ENCRYPTION_KEY".into()),
        compress: false, // Don't compress encrypted logs
        ..Default::default()
    }),
    ..Default::default()
};

let _logger = LoggerManager::with_config(config).await?;

S3 Cloud Archiving

use inklog::{InklogConfig, S3ArchiveConfig};

let config = InklogConfig {
    s3_archive: Some(S3ArchiveConfig {
        enabled: true,
        bucket: "my-log-bucket".to_string(),
        region: "us-west-2".to_string(),
        archive_interval_days: 7,
        local_retention_days: 30,
        prefix: "logs/".to_string(),
        compression: inklog::archive::CompressionType::Zstd,
        ..Default::default()
    }),
    ..Default::default()
};

let manager = LoggerManager::with_config(config).await?;
manager.start_archive_service().await?;

Custom Log Format

use inklog::{InklogConfig, config::GlobalConfig};

let format_string = "[{timestamp}] [{level:>5}] {target} - {message} | {file}:{line}";

let config = InklogConfig {
    global: GlobalConfig {
        level: "debug".into(),
        format: format_string.to_string(),
        masking_enabled: true,
        ..Default::default()
    },
    ..Default::default()
};

let _logger = LoggerManager::with_config(config).await?;

🎨 Feature Flags

Default Features

inklog = "0.1"  # Includes: aws, http, cli

Optional Features

# Cloud & Storage
inklog = { version = "0.1", features = [
    "aws",        # AWS S3 archive support
] }

# HTTP Server
inklog = { version = "0.1", features = [
    "http",       # Axum HTTP health endpoint
] }

# CLI Tools
inklog = { version = "0.1", features = [
    "cli",        # decrypt, generate, validate commands
] }

# Configuration
inklog = { version = "0.1", features = [
    "confers",    # TOML configuration support
] }

# Development
inklog = { version = "0.1", features = [
    "test-local", # Local testing mode
    "debug",     # Additional security audit logging
] }

Feature Details

Feature	Dependencies	Description
aws	aws-sdk-s3, aws-config, aws-types	AWS S3 cloud archive
http	axum	HTTP health check endpoint
cli	clap, glob, toml	Command-line utilities
confers	confers, toml	External TOML configuration support
test-local	-	Local testing mode
debug	-	Security audit logging

📚 Documentation

📘 API Reference

Complete API docs

💻 Examples

Working code examples

📖 Guides

In-depth guides

📖 Additional Resources

Resource	Description
📘 API Reference	Complete API documentation on docs.rs
🏗️ Architecture	System architecture and design decisions
🔒 Security	Security best practices and features
📦 Examples	Working code examples for all features

💻 Examples

💡 Real-world Examples

📝 Basic Logging use inklog::LoggerManager; #[tokio::main] async fn main() -> Result<(), Box<dyn std::error::Error>> { let _logger = LoggerManager::new().await?; log::info!("Application started"); log::error!("An error occurred: {}", err); Ok(()) }	📁 File Logging with Rotation use inklog::{FileSinkConfig, InklogConfig, LoggerManager}; let config = InklogConfig { file_sink: Some(FileSinkConfig { enabled: true, path: "logs/app.log".into(), max_size: "10MB".into(), rotation_time: "daily".into(), keep_files: 7, compress: true, ..Default::default() }), ..Default::default() }; let _logger = LoggerManager::with_config(config).await?;
🔒 Encrypted Logging use inklog::{FileSinkConfig, InklogConfig}; std::env::set_var("INKLOG_ENCRYPTION_KEY", "base64-encoded-key"); let config = InklogConfig { file_sink: Some(FileSinkConfig { enabled: true, path: "logs/encrypted.log".into(), encrypt: true, encryption_key_env: Some("INKLOG_ENCRYPTION_KEY".into()), ..Default::default() }), ..Default::default() }; let _logger = LoggerManager::with_config(config).await?;	🗄️ Database Logging use inklog::{DatabaseSinkConfig, InklogConfig, config::DatabaseDriver}; let config = InklogConfig { database_sink: Some(DatabaseSinkConfig { enabled: true, driver: DatabaseDriver::PostgreSQL, url: "postgresql://localhost/logs".to_string(), pool_size: 10, batch_size: 100, flush_interval_ms: 1000, ..Default::default() }), ..Default::default() }; let _logger = LoggerManager::with_config(config).await?;
☁️ S3 Cloud Archive use inklog::{InklogConfig, S3ArchiveConfig}; let config = InklogConfig { s3_archive: Some(S3ArchiveConfig { enabled: true, bucket: "my-log-bucket".to_string(), region: "us-west-2".to_string(), archive_interval_days: 7, local_retention_days: 30, prefix: "logs/".to_string(), compression: inklog::archive::CompressionType::Zstd, ..Default::default() }), ..Default::default() }; let manager = LoggerManager::with_config(config).await?; manager.start_archive_service().await?;	🏥 HTTP Health Endpoint use axum::{routing::get, Json, Router}; use inklog::LoggerManager; use std::sync::Arc; let logger = Arc::new(LoggerManager::new().await?); let app = Router::new().route( "/health", get({ let logger = logger.clone(); \|\| async move { Json(logger.get_health_status()) } }), ); // Start HTTP server...
🎨 Custom Format use inklog::{InklogConfig, config::GlobalConfig}; let format_string = "[{timestamp}] [{level:>5}] {target} - {message}"; let config = InklogConfig { global: GlobalConfig { level: "debug".into(), format: format_string.to_string(), masking_enabled: true, ..Default::default() }, ..Default::default() }; let _logger = LoggerManager::with_config(config).await?;	🔍 Data Masking use inklog::{InklogConfig, config::GlobalConfig}; let config = InklogConfig { global: GlobalConfig { level: "info".into(), format: "{timestamp} {level} {message}".to_string(), masking_enabled: true, // Enable PII masking ..Default::default() }, ..Default::default() }; let _logger = LoggerManager::with_config(config).await?; // Sensitive data will be automatically masked log::info!("User email: user@example.com"); // Output: User email: *@.**

📂 View all examples →

🏗️ Architecture

🏗️ System Architecture

┌─────────────────────────────────────────────────┐
│           Application Layer                    │
│  (Your code using log! macros)             │
└────────────────┬────────────────────────────┘
                 │
┌────────────────▼────────────────────────────┐
│         Inklog API Layer                  │
│  - LoggerManager, LoggerBuilder          │
│  - Configuration management               │
│  - Health monitoring                     │
└────────────────┬────────────────────────────┘
                 │
┌────────────────▼────────────────────────────┐
│         Sink Abstraction Layer             │
│  - ConsoleSink                          │
│  - FileSink (rotation, compression)     │
│  - DatabaseSink (batch writes)           │
│  - AsyncFileSink                        │
│  - RingBufferedFileSink                 │
└────────────────┬────────────────────────────┘
                 │
┌────────────────▼────────────────────────────┐
│         Core Processing Layer              │
│  - Log formatting & templates            │
│  - Data masking (PII redaction)         │
│  - Encryption (AES-256-GCM)             │
│  - Compression (ZSTD, GZIP, Brotli)    │
└────────────────┬────────────────────────────┘
                 │
┌────────────────▼────────────────────────────┐
│         Concurrency & I/O                 │
│  - Tokio async runtime                  │
│  - Crossbeam channels                  │
│  - Rayon parallel processing            │
└────────────────┬────────────────────────────┘
                 │
┌────────────────▼────────────────────────────┐
│         Storage & External Services        │
│  - Filesystem                          │
│  - Database (PostgreSQL, MySQL, SQLite)  │
│  - AWS S3 (cloud archive)              │
│  - Parquet (analytics)                 │
└───────────────────────────────────────────┘

Layer-by-Layer Explanation

Application Layer

Application code uses standard log! macros from the log crate
Compatible with existing Rust logging patterns

Inklog API Layer

LoggerManager: Main orchestrator for all logging operations
LoggerBuilder: Fluent builder pattern for configuration
Health status tracking and metrics collection

Sink Abstraction Layer

Multiple sink implementations for different output targets
Console output for development
File output with rotation, compression, and encryption
Database output with batch writes (PostgreSQL, MySQL, SQLite)
Async and buffered file sinks for high-throughput scenarios

Core Processing Layer

Template-based log formatting
Regex-based PII data masking (emails, SSNs, credit cards)
AES-256-GCM encryption for sensitive logs
Multiple compression algorithms (ZSTD, GZIP, Brotli, LZ4)

Concurrency & I/O Layer

Tokio async runtime for non-blocking I/O
Crossbeam channels for inter-task communication
Rayon for CPU-intensive parallel processing

Storage & External Services Layer

Local filesystem access
Database connectivity via Sea-ORM
AWS S3 integration for cloud archival
Parquet format for analytics workflows

🔒 Security

🛡️ Security Features

Inklog is built with security as a top priority:

🔒 Encryption

AES-256-GCM: Military-grade encryption for log files
Key Management: Environment variable-based key injection
Zeroized Memory: Secrets are securely cleared after use via zeroize crate
SHA-256 Hashing: Integrity verification for encrypted logs

🎭 Data Masking

Regex-Based Patterns: Automatic PII detection and redaction
Email Masking: user@example.com → ***@***.***
SSN Masking: Credit card and social security number redaction
Custom Patterns: Configurable regex patterns for sensitive data

🔐 Secure Key Handling

// Set encryption key securely from environment
std::env::set_var("INKLOG_ENCRYPTION_KEY", "base64-encoded-32-byte-key");

// Key is automatically zeroized after use
// Never hardcode keys in your application

🛡️ Security Best Practices

No hardcoded secrets: Keys loaded from environment variables
Minimal privileged operations: Only necessary file/database access
Audit logging: Debug feature for security audit trails
Compliance-ready: Supports GDPR, HIPAA, PCI-DSS logging requirements

🧪 Testing

🎯 Run Tests

# Run all tests with default features
cargo test --all-features

# Run tests with specific features
cargo test --features "aws,http,cli"

# Run tests in release mode
cargo test --release

# Run benchmarks
cargo bench

Test Coverage

Inklog targets 95%+ code coverage:

# Generate coverage report
cargo tarpaulin --out Html --all-features

Linting and Formatting

# Format code
cargo fmt --all

# Check formatting without changes
cargo fmt --all -- --check

# Run Clippy (warnings as errors)
cargo clippy --all-targets --all-features -- -D warnings

Security Audit

# Run cargo deny for security checks
cargo deny check

# Check for advisories
cargo deny check advisories

# Check for banned licenses
cargo deny check bans

Integration Tests

# Run integration tests
cargo test --test '*'

# Run with Docker services (PostgreSQL, MySQL)
docker-compose up -d
cargo test --all-features
docker-compose down

🤝 Contributing

Contributions are welcome! Please see CONTRIBUTING.md for guidelines.

Development Setup

# Clone repository
git clone https://github.com/Kirky-X/inklog.git
cd inklog

# Install pre-commit hooks (if available)
./scripts/install-pre-commit.sh

# Run tests
cargo test --all-features

# Run linter
cargo clippy --all-features

# Format code
cargo fmt --all

Pull Request Process

Fork the repository
Create a feature branch (git checkout -b feature/amazing-feature)
Make your changes
Run tests and ensure all pass (cargo test --all-features)
Run clippy and fix warnings (cargo clippy --all-features)
Commit your changes (git commit -m 'Add amazing feature')
Push to the branch (git push origin feature/amazing-feature)
Open a Pull Request

Code Style

Follow Rust naming conventions (snake_case for variables, PascalCase for types)
Use thiserror for error types
Use anyhow for error contexts
Add doc comments to all public APIs
Run cargo fmt before committing

📄 License

This project is dual-licensed under MIT / Apache-2.0:

🙏 Acknowledgments

🌟 Built on Excellent Tools

Inklog wouldn't be possible without these amazing projects:

tracing - The foundation of Rust structured logging
tokio - Async runtime for Rust
Sea-ORM - Async ORM for database operations
AWS SDK for Rust - AWS S3 integration
axum - Web framework for HTTP endpoints
serde - Serialization framework
The entire Rust ecosystem for amazing tools and libraries

📞 Support

📋 Issues

Report bugs and issues

💬 Discussions

Ask questions and share ideas

🐙 GitHub

View source code

⭐ Star History

💝 Support This Project

If you find this project useful, please consider giving it a ⭐️!

Built with ❤️ by Inklog Team

⬆ Back to Top

Name		Name	Last commit message	Last commit date
Latest commit History 48 Commits
.github/workflows		.github/workflows
benches		benches
docs		docs
examples		examples
resource		resource
src		src
tests		tests
.gitignore		.gitignore
.pre-commit-config.yaml		.pre-commit-config.yaml
.secrets.baseline		.secrets.baseline
CHANGELOG.md		CHANGELOG.md
Cargo.lock		Cargo.lock
Cargo.toml		Cargo.toml
LICENSE		LICENSE
README.md		README.md
README_zh.md		README_zh.md
deny.toml		deny.toml

License

Kirky-X/inklog

Folders and files

Latest commit

History

Repository files navigation

🎯 A high-performance, secure, and feature-rich logging infrastructure built on Tokio

📋 Table of Contents

✨ Features

🎯 Core Features (Always Available)

⚡ Enterprise Features

📦 Feature Presets

🚀 Quick Start

📦 Installation

💡 Basic Usage

🎬 5-Minute Quick Start

🔧 Advanced Configuration

Encrypted File Logging

S3 Cloud Archiving

Custom Log Format

🎨 Feature Flags

Default Features

Optional Features

Feature Details

📚 Documentation

📖 Additional Resources

💻 Examples

💡 Real-world Examples

📝 Basic Logging

📁 File Logging with Rotation

🔒 Encrypted Logging

🗄️ Database Logging

☁️ S3 Cloud Archive

🏥 HTTP Health Endpoint

🎨 Custom Format

🔍 Data Masking

🏗️ Architecture

🏗️ System Architecture

Layer-by-Layer Explanation

🔒 Security

🛡️ Security Features

🔒 Encryption

🎭 Data Masking

🔐 Secure Key Handling

🛡️ Security Best Practices

🧪 Testing

🎯 Run Tests

Test Coverage

Linting and Formatting

Security Audit

Integration Tests

🤝 Contributing

Development Setup

Pull Request Process

Code Style

📄 License

🙏 Acknowledgments

🌟 Built on Excellent Tools

📞 Support

⭐ Star History

💝 Support This Project

About

Topics

Resources

License

Contributing

Security policy

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Languages

Packages