Skip to content

Commit

Permalink
Add github action
Browse files Browse the repository at this point in the history
grelinfo committed Nov 20, 2024
1 parent a222ff2 commit 2fa44ed
Showing 8 changed files with 526 additions and 10 deletions.
8 changes: 5 additions & 3 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
name: Publish

on:
release:
types:
@@ -9,14 +10,15 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: Install uv
uses: astral-sh/setup-uv@v3
with:
enable-cache: true

cache-dependency-glob: uv.lock

- name: Set up Python
uses: actions/setup-python@v5
run: uv python install

- name: Build
run: uv build
43 changes: 43 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
name: Test

on:
push:
branches:
- main
pull_request:
branches:
- main

jobs:
test:
name: Python ${{ matrix.python-version }} on ${{ startsWith(matrix.os, 'macos-') && 'macOS' || startsWith(matrix.os, 'windows-') && 'Windows' || 'Linux' }}
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, windows-latest, macos-latest]
python-version: ["3.11", "3.12", "3.13"]

steps:
- uses: actions/checkout@v4

- name: Install uv
uses: astral-sh/setup-uv@v3
with:
enable-cache: true
cache-dependency-glob: uv.lock

- name: Set up Python ${{ matrix.python-version }}
run: uv python install ${{ matrix.python-version }}

- name: Install dependencies
run: uv sync --all-extra

- name: Run pre-commit
run: uv run pre-commit run --all-files

- name: Upload coverage reports to Codecov
uses: codecov/codecov-action@v4.0.1
with:
token: ${{ secrets.CODECOV_TOKEN }}
file: ./coverage.xml
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -2,7 +2,7 @@

Grelmicro is a lightweight framework/toolkit for building async microservices in Python.

It is the perfect companion to FastAPI and FastStream, providing essential tools like leader election, distributed lock, and task scheduling for creating robust distributed systems.
It is the perfect companion to build cloud-native app with FastAPI and FastStream, providing essential tools like leader election, distributed lock, and task scheduling for creating robust distributed systems.

[![PyPI - Version](https://img.shields.io/pypi/v/grelmicro)](https://pypi.org/project/grelmicro/)
[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/grelmicro)](https://pypi.org/project/grelmicro/)
139 changes: 139 additions & 0 deletions docs/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,139 @@
# Grelmicro

Grelmicro is a lightweight framework/toolkit for building async microservices in Python.

It is the perfect companion to build cloud-native app with FastAPI and FastStream, providing essential tools like leader election, distributed lock, and task scheduling for creating robust distributed systems.

[![PyPI - Version](https://img.shields.io/pypi/v/grelmicro)](https://pypi.org/project/grelmicro/)
[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/grelmicro)](https://pypi.org/project/grelmicro/)
[![uv](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/uv/main/assets/badge/v0.json)](https://github.com/astral-sh/uv)
[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
[![mypy](https://www.mypy-lang.org/static/mypy_badge.svg)](https://mypy-lang.org/)

---

**Source Code**: [https://github.com/grelinfo/grelmicro](https://github.com/grelinfo/grelmicro)

---

## Toolkit Modules

Each Grelmicro module addresses common problems in building microservices and distributed systems.

The modules are fast, lightweight and fully typed, offering non-blocking operations by default using AnyIO for concurrency and async/await. Adapters are provided to run synchronous code in AnyIO Worker Threads, ensuring compatibility with both synchronous and asynchronous functions.

### `sync` Module

The `sync` module provides synchronization primitives for distributed systems.

The primitives are technology agnostic, supporting multiple backends (see `backends` module).

The available primitives are:

- **Leader Election**: A single worker is elected as the leader for performing tasks only once in a cluster.
- **Lock**: A distributed lock that can be used to synchronize access to shared resources.

### `task` Module

The `task` module provides a simple task scheduler that can be used to run tasks periodically.

> **Note**: This is not a replacement for bigger tools like Celery, taskiq, or APScheduler. It is just lightweight, easy to use, and safe for running tasks in a distributed system with synchronization.
The key features are:

- **Fast & Easy**: Offers simple decorators to define and schedule tasks effortlessly.
- **Interval Task**: Allows tasks to run at specified intervals.
- **Synchronization**: Controls concurrency using synchronization primitives to manage simultaneous task execution (see the `sync` module).
- **Dependency Injection**: Use [FastDepends](https://lancetnik.github.io/FastDepends/) library to inject dependencies into tasks.
- **Error Handling**: Catches and logs errors, ensuring that task execution failures do not stop the scheduling.

### `backends` Module

The `backends` module is the key of technology agnostic design of Grelmicro.

For now, it provides only a lock backend with Redis, PostgreSQL, and Memory for testing purposes.

> **Important**: Although Grelmicro use AnyIO for concurrency, the backends generally depend on `asyncio`, therefore Trio is not supported.
> **Note**: Feel free to create your own backend and contribute it. In the `abc` module, you can find the protocol for creating new backends.
## Installation

```bash
pip install grelmicro
```

## Examples

### FastAPI Integration

* Create a file `main.py` with:

```python
from contextlib import asynccontextmanager
import typer
from fastapi import FastAPI

from grelmicro.backends.redis import RedisLockBackend
from grelmicro.sync import LeaderElection, Lock
from grelmicro.task import TaskManager

# === FastAPI ===
@asynccontextmanager
async def lifespan(app):
# Start the lock backend and task manager
async with lock_backend, task:
yield

app = FastAPI(lifespan=lifespan)

@app.get("/")
def read_root():
return {"Hello": "World"}

# === Grelmicro ===
task = TaskManager()
lock_backend = RedisLockBackend("redis://localhost:6379/0")

# --- Ensure that only one say hello world at the same time ---
lock = Lock("say_hello_world")

@task.interval(1, sync=lock)
def say_hello_world_every_second():
typer.echo("Hello World")


@task.interval(1, sync=lock)
def say_as_well_hello_world_every_second():
typer.echo("Hello World")


# --- Ensure that only one worker is the leader ---
leader_election = LeaderElection("leader-election")
task.add_task(leader_election)

@task.interval(10, sync=leader_election)
def say_hello_leader_every_ten_seconds():
typer.echo("Hello Leader")
```

## Dependencies

Grelmicro depends on Pydantic v2+, AnyIO v4+, and FastDepends.

### `redis` Dependencies

When you install Grelmicro with `pip install grelmicro[redis]` it comes with:

- `redis-py`: The Python interface to the Redis key-value store (the async interface depends on `asyncio`).

### `postgres` Dependencies

When you install Grelmicro with `pip install grelmicro[postgres]` it comes with:

- `asyncpg`: The Python `asyncio` interface for PostgreSQL.


## License

This project is licensed under the terms of the MIT license.
4 changes: 3 additions & 1 deletion grelmicro/__init__.py
Original file line number Diff line number Diff line change
@@ -1 +1,3 @@
"""Grelmicro Cloud-Native Toolkit."""
"""Grelmicro is a lightweight framework/toolkit for building async microservices in Python."""

__version__ = "0.1.2"
4 changes: 4 additions & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
site_name: Grelmicro
site_url: https://grelmicro.grel.info
theme:
name: material
18 changes: 14 additions & 4 deletions pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
[project]
name = "grelmicro"
version = "0.1.2"
description = "Grelmicro is a tool-kit designed to streamline the development cloud-native applications with FastAPI."
description = "Grelmicro is a lightweight framework/toolkit for building async microservices in Python."
license = "MIT"
authors = [{ name = "Loïc Gremaud", email = "grelinfo@gmail.com"}]
readme = "README.md"
@@ -31,9 +30,11 @@ classifiers = [
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
]

dynamic = ["version"]

requires-python = ">=3.11"


dependencies = [
"anyio>=4.0.0",
"pydantic>=2.0.0",
@@ -70,12 +71,21 @@ dev = [
"fastapi>=0.115.5",
"fastapi-cli>=0.0.5",
]
doc = [
"mkdocs-material>=9.5.44",
]


[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[tool.hatch.build]
skip-excluded-dirs = true
exclude = ["/tests", "/docs", "/examples"]

[tool.hatch.version]
path = "grelmicro/__init__.py"

[tool.ruff]
target-version = "py311"
@@ -132,7 +142,7 @@ disallow_untyped_defs = false
[tool.pytest.ini_options]
addopts = """
--cov=grelmicro
--cov-fail-under=100
--cov-fail-under=90
--cov-report term:skip-covered
--cov-report xml:coverage.xml
--strict-config
318 changes: 317 additions & 1 deletion uv.lock

Large diffs are not rendered by default.

0 comments on commit 2fa44ed

Please sign in to comment.