Skip to content

README.md

Latest commit

 

History

History
117 lines (78 loc) · 3.71 KB

README.md

File metadata and controls

117 lines (78 loc) · 3.71 KB

A professional README is the storefront of your project. It’s what makes a recruiter or a potential investor say, "This developer knows what they’re doing."

Here is a comprehensive, high-quality README.md for SyncraFlow, written in English as requested.

SyncraFlow Engine

SyncraFlow is a high-performance, event-driven automation backend designed to orchestrate data between different APIs. It allows users to create custom workflows triggered by Webhooks, process data asynchronously, and execute multi-step actions with reliability and speed.

Key Features

  • Event-Driven Architecture: Non-blocking request handling using FastAPI and Celery.
  • Dynamic Webhooks: Unique endpoint generation for every user workflow.
  • Asynchronous Task Processing: Offloads heavy data processing to background workers via Redis.
  • Native PostgreSQL Support: Optimized for Supabase with raw SQL queries for maximum performance.
  • JWT Authentication: Secure user management and scoped access to workflows.
  • Detailed Execution Logs: Full traceability for every event processed by the engine.
  • AI-Ready: Modular architecture designed to integrate LLM (OpenAI/Anthropic) steps seamlessly in the future.

Tech Stack

Architecture Overview

  1. Ingestion Layer: FastAPI receives incoming webhooks and validates the payload.
  2. Messaging Layer: The payload is pushed to a Redis queue.
  3. Processing Layer: Celery Workers fetch the workflow configuration from Supabase and execute the defined steps (e.g., Data transformation, HTTP requests).
  4. Logging Layer: Every success or failure is logged back to PostgreSQL for real-time monitoring.

Getting Started

1. Prerequisites

  • Docker & Docker Compose
  • A Supabase account (PostgreSQL)
  • Python 3.12+

2. Environment Setup

Create a .env file in the root directory:

DATABASE_URL=postgresql://postgres:[PASSWORD]@[HOST]:5432/postgres
REDIS_URL=redis://localhost:6379/0
SECRET_KEY=your_super_secret_jwt_key

3. Installation

# Clone the repository
git clone https://github.com/yourusername/syncraflow.git
cd syncraflow

# Create a virtual environment
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

4. Running the Application

# Start Redis and Postgres (if using local Docker)
docker-compose up -d

# Start the FastAPI server
uvicorn app.main:app --reload

# Start the Celery Worker (in a new terminal)
celery -A app.worker worker --loglevel=info

Roadmap

  • Core Webhook ingestion engine
  • Native PostgreSQL integration
  • Multi-step logic execution
  • User Dashboard API
  • AI-powered data transformation steps (Future)
  • Stripe integration for SaaS monetization

TODO:

  • Add more detailed API documentation (Swagger/OpenAPI)
  • setup CI/CD pipeline for automated testing and deployment, DOCKERIZE the application for easier deployment, python package distribution, and add more comprehensive unit and integration tests.
  • Python formatting and linting with Black and Flake8.
  • Implement a retry mechanism for failed tasks in Celery.
  • Implement a real database and remove mock data generation and stockage functions
  • Implement a user-friendly dashboard for workflow management and monitoring.