From 0718646cf310693556afac49a1583261bca34160 Mon Sep 17 00:00:00 2001 From: kitsune-hash Date: Mon, 9 Feb 2026 09:00:17 +0100 Subject: [PATCH] docs: add batch endpoint documentation - Add POST /queue/add-batch to API reference - Add batch enqueue example to Python SDK docs - Add HTTP Batch API section to enqueueing-tasks guide - Include performance comparison table --- docs/guides/enqueueing-tasks.md | 71 ++++++++++++++++++++++++++++++++- docs/python-sdk/index.md | 22 ++++++++++ docs/server/api.md | 56 ++++++++++++++++++++++++++ 3 files changed, 148 insertions(+), 1 deletion(-) diff --git a/docs/guides/enqueueing-tasks.md b/docs/guides/enqueueing-tasks.md index 00d0d1d..f424db7 100644 --- a/docs/guides/enqueueing-tasks.md +++ b/docs/guides/enqueueing-tasks.md @@ -2,6 +2,16 @@ runqy uses Redis for task storage. This guide shows how to enqueue tasks from various languages. +## Quick Comparison + +| Method | Throughput | Use Case | +|--------|------------|----------| +| HTTP API (`POST /queue/add`) | ~800-1,000/s | Simple integrations | +| HTTP Batch API (`POST /queue/add-batch`) | ~35,000-50,000/s | High-throughput from any language | +| Direct Redis (pipelined) | ~40,000-80,000/s | Maximum performance | + +For most use cases, the **Batch API** offers the best balance of simplicity and performance. + ## Redis Key Format runqy uses an asynq-compatible key format: @@ -48,7 +58,66 @@ This works in the API, CLI, and direct Redis operations (the server normalizes t !!! warning "Queue must exist" If the resolved queue (e.g., `inference.default`) doesn't exist in the configuration, the operation fails with an error. -## Examples +## HTTP Batch API + +The batch endpoint is the recommended approach for high-throughput job submission. It uses Redis pipelining internally for optimal performance. + +### Python (using SDK) + +```python +from runqy_python import RunqyClient + +client = RunqyClient("http://localhost:3000", api_key="your-api-key") + +# Submit 1000 jobs in one request +jobs = [{"prompt": f"Generate image {i}"} for i in range(1000)] +result = client.enqueue_batch("inference.default", jobs) + +print(f"Enqueued: {result.enqueued} jobs") +``` + +### cURL + +```bash +curl -X POST http://localhost:3000/queue/add-batch \ + -H "Authorization: Bearer your-api-key" \ + -H "Content-Type: application/json" \ + -d '{ + "queue": "inference.default", + "jobs": [ + {"data": {"prompt": "Hello"}}, + {"data": {"prompt": "World"}} + ] + }' +``` + +### Node.js + +```javascript +const response = await fetch('http://localhost:3000/queue/add-batch', { + method: 'POST', + headers: { + 'Authorization': 'Bearer your-api-key', + 'Content-Type': 'application/json', + }, + body: JSON.stringify({ + queue: 'inference.default', + jobs: [ + { data: { prompt: 'Hello' } }, + { data: { prompt: 'World' } }, + ], + }), +}); + +const result = await response.json(); +console.log(`Enqueued: ${result.enqueued}`); +``` + +--- + +## Direct Redis Access + +For maximum performance or when you need direct Redis access, you can enqueue tasks directly. ### Redis CLI diff --git a/docs/python-sdk/index.md b/docs/python-sdk/index.md index 5ee8ca9..1f5f0a0 100644 --- a/docs/python-sdk/index.md +++ b/docs/python-sdk/index.md @@ -15,6 +15,7 @@ The `runqy-python` SDK provides simple decorators for writing task handlers that - **`RunqyClient` class**: HTTP client for enqueuing tasks - **`enqueue()` function**: Quick enqueue without creating a client instance +- **`enqueue_batch()` function**: High-throughput batch enqueue (35,000+ jobs/s) ## Installation @@ -68,6 +69,27 @@ result = client.get_task(task.task_id) print(f"State: {result.state}, Result: {result.result}") ``` +## Batch Enqueue Example + +For high-throughput scenarios, use `enqueue_batch()` to submit thousands of jobs per second: + +```python +from runqy_python import RunqyClient + +client = RunqyClient("http://localhost:3000", api_key="your-api-key") + +# Enqueue 1000 jobs in a single request +jobs = [{"input": f"job-{i}"} for i in range(1000)] +result = client.enqueue_batch("inference.default", jobs) + +print(f"Enqueued: {result.enqueued}") +print(f"Failed: {result.failed}") +print(f"First 5 task IDs: {result.task_ids[:5]}") +``` + +!!! tip "Performance" + Batch enqueue achieves ~35,000-50,000 jobs/s compared to ~800-1,000 jobs/s with individual `enqueue()` calls. + ## Source Code ``` diff --git a/docs/server/api.md b/docs/server/api.md index b9eea89..8d7de70 100644 --- a/docs/server/api.md +++ b/docs/server/api.md @@ -110,6 +110,62 @@ Enqueue a new task. } ``` +### `POST /queue/add-batch` + +Enqueue multiple tasks in a single request. Uses Redis pipelining for high-throughput job submission. + +**Request Body:** + +```json +{ + "queue": "inference.high", + "timeout": 300, + "jobs": [ + {"data": {"prompt": "Hello"}}, + {"data": {"prompt": "World"}}, + {"data": {"prompt": "Foo"}, "timeout": 600} + ] +} +``` + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| `queue` | string | Yes | Target queue name | +| `timeout` | int | No | Default timeout for all jobs (seconds) | +| `jobs` | array | Yes | Array of job objects | +| `jobs[].data` | object | Yes | Task payload | +| `jobs[].timeout` | int | No | Override timeout for this job | + +**Response:** + +```json +{ + "enqueued": 3, + "failed": 0, + "task_ids": ["uuid-1", "uuid-2", "uuid-3"], + "errors": [] +} +``` + +| Field | Type | Description | +|-------|------|-------------| +| `enqueued` | int | Number of successfully enqueued tasks | +| `failed` | int | Number of failed tasks | +| `task_ids` | array | List of task IDs (in order) | +| `errors` | array | Error messages for failed tasks | + +**Performance:** + +| Method | Throughput | +|--------|------------| +| Single `POST /queue/add` calls | ~800-1,000 jobs/s | +| Batch endpoint (100 jobs/request) | ~35,000-50,000 jobs/s | + +!!! tip "Optimal batch size" + Batch sizes of 50-200 jobs per request offer the best balance of throughput and latency. + +--- + ### `GET /queue/{task_id}` Get task status and result. The queue is automatically determined from the task's metadata stored in Redis.