This is a template repository for AI/ML model projects.
- AI/ML model
- Jupyter notebook
- Research
- Python module/package
- Project Structure
- Boilerplate/Template
- Best Practices
- Configuration
- Test
- Build
- Documentation
- Scripts
- Examples
- CI/CD
-
You can use this template repository as reference to create a new repository with the same structure or clone the repository to start a new project. It will help you to organize your project structure and files. It works out of the box for most of the AI/ML projects.
-
You can customize (remove, modify or add) the files and directories as needed to meet your project requirements.
-
If you want to use the template repository directly, just click the Use this template button and follow the instructions.
-
You can use cookiecutter to generate a new project from cookiecutter branch:
# Clone the cookiecutter branch: git clone -b cookiecutter https://github.com/bybatkhuu/model.python-template.git # Install cookiecutter: pip install cookiecutter # Generate a new project from the cookiecutter template: cookiecutter -f .
- Install Python (>= v3.9) and pip (>= 23):
- [RECOMMENDED] Miniconda (v3)
- [arm64/aarch64] Miniforge (v3)
- [Python virutal environment] venv
- [OPTIONAL] For GPU (NVIDIA):
- NVIDIA GPU driver (>= v452.39)
- NVIDIA CUDA (>= v11) and cuDNN (>= v8)
[OPTIONAL] For DEVELOPMENT environment:
- Install git
- Setup an SSH key (video tutorial)
Tip
Skip this step, if you're going to install the package directly from GitHub repository.
2.1. Prepare projects directory (if not exists):
# Create projects directory:
mkdir -pv ~/workspaces/projects
# Enter into projects directory:
cd ~/workspaces/projects2.2. Follow one of the below options [A], [B] or [C]:
OPTION A. Clone the repository:
git clone https://github.com/bybatkhuu/model.python-template.git && \
cd model.python-templateOPTION B. Clone the repository (for DEVELOPMENT: git + ssh key):
git clone git@github.com:bybatkhuu/model.python-template.git && \
cd model.python-templateOPTION C. Download source code:
- Download archived zip file from releases.
- Extract it into the projects directory.
Note
Choose one of the following methods to install the package [A ~ E]:
OPTION A. Install directly from GitHub repository:
pip install git+https://github.com/bybatkhuu/model.python-template.gitOPTION B. Install from the downloaded source code:
# Install directly from the source code:
pip install .
# Or install with editable mode:
pip install -e .OPTION C. Install for DEVELOPMENT environment:
pip install -r ./requirements/requirements.dev.txtOPTION D. Install from pre-built package files (for PRODUCTION):
- Download
.whlor.tar.gzfile from releases. - Install with pip:
# Install from .whl file:
pip install ./simple_model-[VERSION]-py3-none-any.whl
# Or install from .tar.gz file:
pip install ./simple_model-[VERSION].tar.gzOPTION E. Copy the module into the project directory (for testing):
# Install python dependencies:
pip install -r ./requirements.txt
# Copy the module source code into the project:
cp -r ./src/simple_model [PROJECT_DIR]
# For example:
cp -r ./src/simple_model /some/path/project/## Standard libraries
import os
import sys
import logging
import pathlib
from typing import Any
## Third-party libraries
import numpy as np
from numpy.typing import NDArray
## Internal modules
from simple_model import SimpleModel
logger = logging.getLogger(__name__)
def main() -> None:
logging.basicConfig(
stream=sys.stdout,
level=logging.INFO,
datefmt="%Y-%m-%d %H:%M:%S %z",
format="[%(asctime)s | %(levelname)s | %(filename)s:%(lineno)d]: %(message)s",
)
# Pre-defined variables (for customizing and testing)
_parent_dir = pathlib.Path(__file__).parent.resolve()
_models_dir = str(_parent_dir.parent.parent / "models")
if not os.path.isdir(_models_dir):
os.makedirs(_models_dir, exist_ok=True)
_model_name = "linear_regression.v0.0.1-24"
_X_train = np.array([[1], [2], [3], [4], [5]])
_y_train = np.array([2, 4, 6, 8, 10])
_X_test = np.array([[6], [7], [8]])
_y_test = np.array([10, 14, 16])
# Create the model instance
_config = {"models_dir": _models_dir, "model_name": _model_name}
_model = SimpleModel(config=_config)
# Train or load the model
if not SimpleModel.is_model_files_exist(**_config):
_model.train(X=_X_train, y=_y_train)
else:
_model.load()
# Predict the target values
_y_pred: NDArray[Any] = _model.predict(X=_X_test)
logger.info(f"Predicted values for {_X_test.flatten()}: {_y_pred.flatten()}")
# Evaluate the model
_r2_score: float = _model.score(y_true=_y_test, y_pred=_y_pred)
logger.info(f"R^2 score: {_r2_score}")
_is_similar: bool = _model.is_similar(X=_X_test, y=_y_test)
logger.info(f"Is similar: {_is_similar}")
# Save the model
if _model.is_trained() and (not SimpleModel.is_model_files_exist(**_config)):
_model.save()
logger.info("Done!\n")
return
if __name__ == "__main__":
main()👍
simple_model: # Just an example to group the configs (Not necessary)
models_dir: "./models" # Directory where the models are saved
model_name: "linear_regression.v0.0.1-240101" # Name of the model as sub-directory
threshold: 0.5 # Threshold for similarity check# ENV=development
# DEBUG=trueTo run tests, run the following command:
# Install python test dependencies:
pip install -r ./requirements/requirements.test.txt
# Run tests:
python -m pytest -sv -o log_cli=true
# Or use the test script:
./scripts/test.sh -l -v -cTo build the python package, run the following command:
# Install python build dependencies:
pip install -r ./requirements/requirements.build.txt
# Build python package:
python -m build
# Or use the build script:
./scripts/build.shTo build the documentation, run the following command:
# Install python documentation dependencies:
pip install -r ./requirements/requirements.docs.txt
# Serve documentation locally (for development):
mkdocs serve
# Or use the docs script:
./scripts/docs.sh
# Or build documentation:
mkdocs build
# Or use the docs script:
./scripts/docs.sh -b