Distributed, scalable Task Scheduler built for high throughput.
1. Schedule delayed job - Will execute after the time duration is passed.
2. Schedule bounded recurring tasks - tasks will execute till specified when to stop.
3. Schedule unbounded recurring tasks - will run indefinitely.
4. Chain Plans - What if a 'task execution' triggers, triggering a new plan? Read more about Chained Plans
Note: This may result in exponentially scheduling tasks. But Karya can be configured to adjust the tree-depth of chained plans.
5. Attach hooks - Hooks can be attached to each job, to perform any reactive action.
6. Trigger multiple Actions in hook - One can attach multiple actions to a plan's hook.
7. Extending Hooks - What if one could trigger a whole new plan instead of taking just reactive actions?
8. Multiple Actions to Configure - Configure from a range of actions on what to do once a task is to be executed.
Karya Nodes can be run completely as docker containers.
Karya currently requires the following external components to work: a SQL-based database, a key-value-based one, and a queue.
Karya provides out-of-the-box integration with the below given components with more on the way. For more details, refer - Data Adapters
| Repo Adapter | Locks Adapter | Queue Adapter |
|---|---|---|
| Postgres | Redis | RabbitMQ |
| AWS Aurora | ElastiCache | SQS |
Users can use the following clients to interact with Karya from their preferred environment.
This section will help set up Karya locally so that you can start scheduling jobs in no time!
A few things to note: If you haven't already gone through the architecture overview, Karya requires an SQL database, a queue, and a key-value store to work with. For our local setup, we will be using Postgres, Redis, and RabbitMQ, respectively.
For more detailed local setup, please take a look at the local setup documentation.
Docker and Docker Compose should be installed on your machine, and the engine is running.
Run the below command to set up and run Postgres, Redis, and RabbitMQ containers.
docker-compose -f ./docs/local-setup/providers.docker-compose.yml up -dCreate the following .yml files on your machine:
| File Name | Documentation | Description | Example |
|---|---|---|---|
| providers.yml | Data Adapters | Contains the connection details for Postgres, Redis and RabbitMQ | psql-redis-rabbitmq.providers.yml |
| server.yml | Karya Server | Contains the server configuration details. This will serve as the client interfacing side | server.yml |
| executor.yml | Karya Executor Connectors | Contains the executor configuration details. This is what will provide the functionality to Karya | executor.yml |
| scheduler.yml | Karya Scheduler | Contains the scheduler configuration details. This is the heart of Karya | scheduler.yml |
Run the below command to start the Karya servers via docker-compose
NOTE : Make sure to change the paths while mounting the volumes in the below command to the paths where you have stored the .yml files.
docker-compose -f ./docs/local-setup/karya.docker-compose.yml up -dUse the client of your choice and start scheduling tasks:
There are several task schedulers out there. Why to choose Karya? Here are the reasons:
- High Throughput – Designed for infinite horizontal scaling.
- Fault-Tolerant – Uses stateless nodes with external data-adapters.
- Highly Customizable – Configurable with YAML files for data-adapters and connectors.
- Fast & Safe – Written in Kotlin with structured concurrency using coroutines.