Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
71 changes: 70 additions & 1 deletion docs/guides/enqueueing-tasks.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,16 @@

runqy uses Redis for task storage. This guide shows how to enqueue tasks from various languages.

## Quick Comparison

| Method | Throughput | Use Case |
|--------|------------|----------|
| HTTP API (`POST /queue/add`) | ~800-1,000/s | Simple integrations |
| HTTP Batch API (`POST /queue/add-batch`) | ~35,000-50,000/s | High-throughput from any language |
| Direct Redis (pipelined) | ~40,000-80,000/s | Maximum performance |

For most use cases, the **Batch API** offers the best balance of simplicity and performance.

## Redis Key Format

runqy uses an asynq-compatible key format:
Expand Down Expand Up @@ -48,7 +58,66 @@ This works in the API, CLI, and direct Redis operations (the server normalizes t
!!! warning "Queue must exist"
If the resolved queue (e.g., `inference.default`) doesn't exist in the configuration, the operation fails with an error.

## Examples
## HTTP Batch API

The batch endpoint is the recommended approach for high-throughput job submission. It uses Redis pipelining internally for optimal performance.

### Python (using SDK)

```python
from runqy_python import RunqyClient

client = RunqyClient("http://localhost:3000", api_key="your-api-key")

# Submit 1000 jobs in one request
jobs = [{"prompt": f"Generate image {i}"} for i in range(1000)]
result = client.enqueue_batch("inference.default", jobs)

print(f"Enqueued: {result.enqueued} jobs")
```

### cURL

```bash
curl -X POST http://localhost:3000/queue/add-batch \
-H "Authorization: Bearer your-api-key" \
-H "Content-Type: application/json" \
-d '{
"queue": "inference.default",
"jobs": [
{"data": {"prompt": "Hello"}},
{"data": {"prompt": "World"}}
]
}'
```

### Node.js

```javascript
const response = await fetch('http://localhost:3000/queue/add-batch', {
method: 'POST',
headers: {
'Authorization': 'Bearer your-api-key',
'Content-Type': 'application/json',
},
body: JSON.stringify({
queue: 'inference.default',
jobs: [
{ data: { prompt: 'Hello' } },
{ data: { prompt: 'World' } },
],
}),
});

const result = await response.json();
console.log(`Enqueued: ${result.enqueued}`);
```

---

## Direct Redis Access

For maximum performance or when you need direct Redis access, you can enqueue tasks directly.

### Redis CLI

Expand Down
22 changes: 22 additions & 0 deletions docs/python-sdk/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ The `runqy-python` SDK provides simple decorators for writing task handlers that

- **`RunqyClient` class**: HTTP client for enqueuing tasks
- **`enqueue()` function**: Quick enqueue without creating a client instance
- **`enqueue_batch()` function**: High-throughput batch enqueue (35,000+ jobs/s)

## Installation

Expand Down Expand Up @@ -68,6 +69,27 @@ result = client.get_task(task.task_id)
print(f"State: {result.state}, Result: {result.result}")
```

## Batch Enqueue Example

For high-throughput scenarios, use `enqueue_batch()` to submit thousands of jobs per second:

```python
from runqy_python import RunqyClient

client = RunqyClient("http://localhost:3000", api_key="your-api-key")

# Enqueue 1000 jobs in a single request
jobs = [{"input": f"job-{i}"} for i in range(1000)]
result = client.enqueue_batch("inference.default", jobs)

print(f"Enqueued: {result.enqueued}")
print(f"Failed: {result.failed}")
print(f"First 5 task IDs: {result.task_ids[:5]}")
```

!!! tip "Performance"
Batch enqueue achieves ~35,000-50,000 jobs/s compared to ~800-1,000 jobs/s with individual `enqueue()` calls.

## Source Code

```
Expand Down
56 changes: 56 additions & 0 deletions docs/server/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,62 @@ Enqueue a new task.
}
```

### `POST /queue/add-batch`

Enqueue multiple tasks in a single request. Uses Redis pipelining for high-throughput job submission.

**Request Body:**

```json
{
"queue": "inference.high",
"timeout": 300,
"jobs": [
{"data": {"prompt": "Hello"}},
{"data": {"prompt": "World"}},
{"data": {"prompt": "Foo"}, "timeout": 600}
]
}
```

| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `queue` | string | Yes | Target queue name |
| `timeout` | int | No | Default timeout for all jobs (seconds) |
| `jobs` | array | Yes | Array of job objects |
| `jobs[].data` | object | Yes | Task payload |
| `jobs[].timeout` | int | No | Override timeout for this job |

**Response:**

```json
{
"enqueued": 3,
"failed": 0,
"task_ids": ["uuid-1", "uuid-2", "uuid-3"],
"errors": []
}
```

| Field | Type | Description |
|-------|------|-------------|
| `enqueued` | int | Number of successfully enqueued tasks |
| `failed` | int | Number of failed tasks |
| `task_ids` | array | List of task IDs (in order) |
| `errors` | array | Error messages for failed tasks |

**Performance:**

| Method | Throughput |
|--------|------------|
| Single `POST /queue/add` calls | ~800-1,000 jobs/s |
| Batch endpoint (100 jobs/request) | ~35,000-50,000 jobs/s |

!!! tip "Optimal batch size"
Batch sizes of 50-200 jobs per request offer the best balance of throughput and latency.

---

### `GET /queue/{task_id}`

Get task status and result. The queue is automatically determined from the task's metadata stored in Redis.
Expand Down