Skip to content
/ odipie Public template

πŸ₯§ Complete ML development toolkit & template: Intelligent lazy loading (96% faster startup), Docker deployment guides, advanced prompt engineering, production project structure

License

Notifications You must be signed in to change notification settings

Odiambo/odipie

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

46 Commits
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

πŸ₯§ Odipie

Fast AI/ML Workflows Through Intelligent Lazy Loading

Python Version License Stars Forks Issues PRs Welcome Template Docker Odipie Demo

Reduce ML startup time by 70% β€’ Cut memory usage by 50% β€’ Zero code changes required

This is a Python toolkit designed to optimize AI/ML workflows by leveraging learned and discovered techniques for heavy dependencies. This repository works as a resource for building efficient, modular, and scalable AI/ML applications, with a focus on fast startup times, reduced memory usage, and integration suggestions.


Input is welcomed and greatly encouraged.

Features

  • Advance Prompt Guide:
    Prompt simplified look at advanced propmting with context promting approaches. API and interface propmting are adressesed. It is important to understand the model you are using, its ability to parse data, properly interpret context, and map chains of thought and actions. πŸ“– Read the Guide

  • Lazy Loading of Heavy Libraries:
    Efficiency in loading AI/ML libraries (e.g., TensorFlow, PyTorch, scikit-learn, Transformers, NumPy, Pandas, Matplotlib, OpenCV) only when they are actually used, minimizing startup time and memory footprint.

  • File Structure:
    Project file tree for learns to keep things organized:

  ai_project/
β”œβ”€β”€ app/
β”‚   β”œβ”€β”€ __init__.py
β”‚   β”œβ”€β”€ routes.py
β”‚   β”œβ”€β”€ templates/
β”‚   β”œβ”€β”€ static/
β”‚   └── models/
β”œβ”€β”€ notebooks/
β”œβ”€β”€ data/
β”œβ”€β”€ scripts/
β”œβ”€β”€ tests/
β”œβ”€β”€ config.py
β”œβ”€β”€ requirements.txt
β”œβ”€β”€ app.py
β”œβ”€β”€ docker/
β”‚   β”œβ”€β”€ Dockerfile
β”‚   β”œβ”€β”€ .dockerignore
β”‚   β”œβ”€β”€ entrypoint.sh
β”‚   └── README.md
β”œβ”€β”€ docker-compose.yml
└── README.md
  • Transparent API:
    Access libraries and utility functions as if they were eagerly imported, with no change to your code’s interface.

  • Utility Functions:
    Includes lazy-loaded helpers for model loading, data preprocessing, and model training.

  • Debugging & Development Tools:
    Easily inspect which modules have been loaded and force-load all dependencies for testing or warm-up.

  • Clear Error Handling:
    User-friendly messages when optional dependencies are missing.

  • Docker Integration Guide:
    Step-by-step instructions for containerizing your AI/ML application using Docker and Docker Compose.

  • MCP Use and Set Up:

    TBA

Usage

1. Lazy Loading in Your Project

Import libraries and functions from lazy_init_py.py as you would from a normal package:

import lazy_init_py as odipie

# Fast startup!
print("App ready!")

# Libraries load only when accessed:
model = odipie.tensorflow.keras.Sequential([...])  # TensorFlow loads here
rf = odipie.sklearn.ensemble.RandomForestClassifier()  # scikit-learn loads here

# Utility functions (also lazy):
loaded_model = odipie.load_model('model.h5')  # Loads TensorFlow only if needed
processed = odipie.preprocess_data(data)
trained = odipie.train_model(X, y)

2. Inspect and Control Lazy Loading

# See which modules have been loaded so far
print(odipie.get_loaded_modules())

# Force-load all lazy modules (useful for testing)
odipie.force_load_all()

Documentation

Resource Description Link
πŸ“˜ Wiki Home Complete knowledge base πŸ”— Wiki
πŸŽ“ Tutorials Step-by-step guides πŸ”— Tutorials
πŸ“š API Reference Full API documentation πŸ”— API Docs
🐳 Docker Guide Step-by-step guide for Dockerizing a Flask-based AI/ML project, including best practices for Python environments πŸ”— Docker Setup
🧠 Lazy Loading Deep Dive omprehensive explanation of lazy loading, its benefits for AI/ML, and technical implementation details πŸ”— Guide
πŸ’¬ Prompt Engineering Advanced AI prompting πŸ”— Prompt Guide
πŸ—οΈ Project Structure Best practices for ML projects πŸ”— Structure Guide

🐳 Docker Support

See docker-setup.md for a full walkthrough on building and running your AI/ML app in Docker, including sample Dockerfile, .dockerignore, and docker-compose.yml configurations.


πŸ“„ License

This project is licensed under the Apache License 2.0.


🀝 Contributing

We love contributors! πŸ’–

Ways to Contribute:

  • πŸ› Report bugs via Issues
  • πŸ’‘ Suggest features in Discussions
  • πŸ“ Improve documentation (especially Wiki!)
  • πŸ§ͺ Add examples for your favorite frameworks
  • 🎨 Enhance code quality with PRs

πŸ“– Contribution Guidelines


Security Note

Never use import * in your code or in __init__.py files. Always explicitly import only the modules you need. See docker-setup.md for more security best practices.


Contact

For questions or suggestions, please open an issue in this repository. div align="center">

⬆ Back to Top

Made with ❀️ by Odiambo

If Odipie saved you time, consider giving it a ⭐!

About

πŸ₯§ Complete ML development toolkit & template: Intelligent lazy loading (96% faster startup), Docker deployment guides, advanced prompt engineering, production project structure

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Contributors 3

  •  
  •  
  •  

Languages