Skip to content

Commit

Permalink
docs: improve RC release status description, clean up
Browse files Browse the repository at this point in the history
  • Loading branch information
weyoss committed Mar 26, 2024
1 parent cb18d44 commit 98910dc
Show file tree
Hide file tree
Showing 6 changed files with 76 additions and 55 deletions.
41 changes: 30 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,20 +3,19 @@
> The master branch always reflects the most recent changes. To view the latest release reference see [RedisSMQ v7.2.3](https://github.com/weyoss/redis-smq/tree/v7.2.3)
<div align="center" style="text-align: center">
<p><a href="https://github.com/weyoss/redis-smq"><img alt="RedisSMQ" src="./logo.png?v=202312182134" /></a></p>
<p>A simple high-performance Redis message queue for Node.js.</p>
<p><a href="https://github.com/weyoss/redis-smq"><img alt="RedisSMQ" src="./logo.png?v=202312182134" /></a></p>
<p>A simple high-performance Redis message queue for Node.js.</p>
<p>
<a href="https://github.com/weyoss/redis-smq/actions/workflows/tests.yml"><img src="https://github.com/weyoss/redis-smq/actions/workflows/tests.yml/badge.svg" alt="Tests" style="max-width:100%;" /></a>
<a href="https://github.com/weyoss/redis-smq/actions/workflows/codeql.yml" rel="nofollow"><img src="https://github.com/weyoss/redis-smq/actions/workflows/codeql.yml/badge.svg" alt="Code quality" /></a>
<a href="https://codecov.io/github/weyoss/redis-smq?branch=master" rel="nofollow"><img src="https://img.shields.io/codecov/c/github/weyoss/redis-smq" alt="Coverage Status" /></a>
<a href="https://npmjs.org/package/redis-smq" rel="nofollow"><img src="https://img.shields.io/npm/v/redis-smq.svg" alt="NPM version" /></a>
<a href="https://npmjs.org/package/redis-smq" rel="nofollow"><img src="https://img.shields.io/npm/dm/redis-smq.svg" alt="NPM downloads" /></a>
</p>
</div>

# RedisSMQ

<p>
<a href="https://github.com/weyoss/redis-smq/actions/workflows/tests.yml"><img src="https://github.com/weyoss/redis-smq/actions/workflows/tests.yml/badge.svg" alt="Tests" style="max-width:100%;" /></a>
<a href="https://github.com/weyoss/redis-smq/actions/workflows/codeql.yml" rel="nofollow"><img src="https://github.com/weyoss/redis-smq/actions/workflows/codeql.yml/badge.svg" alt="Code quality" /></a>
<a href="https://codecov.io/github/weyoss/redis-smq?branch=master" rel="nofollow"><img src="https://img.shields.io/codecov/c/github/weyoss/redis-smq" alt="Coverage Status" /></a>
<a href="https://npmjs.org/package/redis-smq" rel="nofollow"><img src="https://img.shields.io/npm/v/redis-smq.svg" alt="NPM version" /></a>
<a href="https://npmjs.org/package/redis-smq" rel="nofollow"><img src="https://img.shields.io/npm/dm/redis-smq.svg" alt="NPM downloads" /></a>
</p>

RedisSMQ is a Node.js library for queuing messages (aka jobs) and processing them asynchronously with consumers. Backed by Redis, it allows scaling up your typical applications with ease of use.

## Features
Expand All @@ -43,7 +42,27 @@ RedisSMQ is a Node.js library for queuing messages (aka jobs) and processing the

## What's new?

:rocket: RedisSMQ v8 is coming soon!
:rocket: RC's are now available for RedisSMQ v8! The v8 release will bring major improvements and new features. Some of them are:

- [x] Message queue codebase refactoring and optimizations.
- [x] Message storage and handling improvements.
- [x] Message status which allows to retrieve, at any time, the status of a message by its ID.
- [x] [Pub/Sub Delivery Model and Consumer Groups](docs/queue-delivery-models.md#pubsub-delivery-model).
- [x] Message handlers sandboxing and message processing performance improvement with [Message Handler Worker Threads](docs/message-handler-worker-threads.md).
- [x] Cross-system event propagation based on [EventBus](docs/event-bus.md).
- [x] Better error handling aiming at reporting fatal errors to the application whenever it is possible and without crashing the main process.
- [x] [ESM Modules Support](docs/esm-cjs-modules.md).

Current RedisSMQ v8 RC status:

- [x] RedisSMQ Common Library
- [x] RedisSMQ
- [ ] HTTP API (WIP)
- [ ] Web UI (WIP)

If you wish to get the latest updates early feel free to try RedisSMQ v8 RC. Do not hesitate to report any bug or issue if encountered.

Otherwise, stay with [RedisSMQ v7](https://github.com/weyoss/redis-smq/tree/v7.2.3) if you are looking for a fully working release with an HTTP API and a Web UI.

## Installation

Expand Down
16 changes: 8 additions & 8 deletions docs/messages.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

# Messages

[`ProducibleMessage`](api/classes/ProducibleMessage.md) class is responsible for carrying application data, sometimes referred to as `message payload`, which may be delivered to a consumer to be processed asynchronously.
[`ProducibleMessage Class`](api/classes/ProducibleMessage.md) is responsible for carrying application data, sometimes referred to as `message payload`, which may be delivered to a consumer to be processed asynchronously.

The message payload can be of any valid JSON data type. It may be a simple text message like `Hello world` or a complex data type like `{hello: 'world'}`.

Expand All @@ -17,10 +17,10 @@ The `ProducibleMessage` class provides many methods for setting up different del

See:

- [ProducibleMessage Class](api/classes/ProducibleMessage.md) - To set up various message parameters
- [Message Class](api/classes/Message.md) - To fetch/delete a message or a list of messages
- [QueueMessages Class](api/classes/QueueMessages.md) - To browse all queue messages
- [QueuePendingMessages Class](api/classes/QueuePendingMessages.md) - To browse queue pending messages
- [QueueAcknowledgedMessages Class](api/classes/QueueAcknowledgedMessages.md) - To browse/requeue/delete queue acknowledged messages
- [QueueDeadLetteredMessages Class](api/classes/QueueDeadLetteredMessages.md) - To browse/requeue/delete queue dead-lettered messages
- [QueueScheduledMessages Class](api/classes/QueueScheduledMessages.md) - To browse/delete queue scheduled messages
- [ProducibleMessage Class](api/classes/ProducibleMessage.md) - To set up various message parameters before publishing it;
- [Message Class](api/classes/Message.md) - To fetch/delete a message or a list of messages from a queue;
- [QueueMessages Class](api/classes/QueueMessages.md) - To manage queue messages;
- [QueuePendingMessages Class](api/classes/QueuePendingMessages.md) - To manage queue pending messages
- [QueueAcknowledgedMessages Class](api/classes/QueueAcknowledgedMessages.md) - To manage queue acknowledged messages
- [QueueDeadLetteredMessages Class](api/classes/QueueDeadLetteredMessages.md) - To manage queue dead-lettered messages
- [QueueScheduledMessages Class](api/classes/QueueScheduledMessages.md) - To manage queue scheduled messages
20 changes: 10 additions & 10 deletions docs/queue-delivery-models.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,13 @@

# Queue Delivery Models

When it comes to message delivery RedisSMQ provides 2 reliable models: Point-2-Point and Pub/Sub.
When it comes to message delivery RedisSMQ provides 2 reliable models: `Point-2-Point` and `Pub/Sub`.

## Point-2-Point Delivery Model

![RedisSMQ Point-2-Point Delivery Model](redis-smq-point-2-point-delivery-model.png)

In the Point-2-Point model, a message is produced to a Point-2-Point queue and then delivered to and consumed by at a time only one consumer.
In the `Point-2-Point` model, a message is produced to a `Point-2-Point` queue and then delivered to and consumed by at a time only one consumer.

### Creating a Point-2-Point Queue

Expand Down Expand Up @@ -73,25 +73,25 @@ consumer.run((err) => {

![RedisSMQ Pub/Sub Delivery Model High-level View](redis-smq-pubsub-delivery-model-highlevel-view.png)

In the Pub/Sub model, on the other hand, a message is delivered to and consumed by all consumers of a Pub/Sub queue.
In the `Pub/Sub` model, on the other hand, a message is delivered to and consumed by all consumers of a `Pub/Sub` queue.

From a high-level view the Pub/Sub model maybe illustrated as shown in the diagram above.
From a high-level view the `Pub/Sub` model maybe illustrated as shown in the diagram above.

Each consumer receives and processes a copy of the produced message.

### Consumer Groups

![RedisSMQ Pub/Sub Delivery Model](redis-smq-pubsub-delivery-model.png)

To consume messages from a Pub/Sub queue, a consumer group is required.
To consume messages from a `Pub/Sub` queue, a consumer group is required.

In fact, when publishing a message to a pub/sub queue, the message is published to all consumer groups of the given queue.
In fact, when publishing a message to a `Pub/Sub` queue, the message is published to all consumer groups of the given queue.

Within a consumer group, only one consumer receives the message.

For a consumer group, unacknowledged messages are retried in the same manner as for a Point-2-Point queue.
For a consumer group, unacknowledged messages are retried in the same manner as for a `Point-2-Point` queue.

When `retryTreshold` is exceeded failed messages from all consumer groups are stored, if configured to do so, in the dead-letter-queue of the given Pub/Sub queue.
When `retryTreshold` is exceeded failed messages from all consumer groups are stored, if configured to do so, in the dead-letter-queue of the given `Pub/Sub` queue.

### Creating a Pub/Sub Queue

Expand Down Expand Up @@ -139,7 +139,7 @@ producer.run((err) => {
});
```

When producing a message to a Pub/Sub queue, if the queue has no consumer groups an error will be returned.
When producing a message to a `Pub/Sub` queue, if the queue has no consumer groups an error will be returned.

So make sure the queue has at least one consumer group before publishing messages.

Expand All @@ -166,4 +166,4 @@ consumer.run((err) => {
});
```

Please do not forget to provide the consumer group ID when consuming messages from a Pub/Sub queue.
Please do not forget to provide the consumer group ID when consuming messages from a `Pub/Sub` queue.
10 changes: 6 additions & 4 deletions docs/queue-rate-limiting.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[RedisSMQ](../README.md) / [Docs](README.md) / Queue Rate limiting
[RedisSMQ](../README.md) / [Docs](README.md) / Queue Rate Limiting

# Queue Rate limiting
# Queue Rate Limiting

In some cases consuming messages with a high message rate may be not desirable. For example:

Expand All @@ -9,7 +9,7 @@ In some cases consuming messages with a high message rate may be not desirable.
- Your application is using an external API which is rate-limiting client requests and consuming messages with a high rate could make your service banned for a certain time or maybe permanently.
- Etc.

RedisSMQ allows you, in such cases, to control the rate at which the messages are consumed by setting a rate limit for a given queue.
RedisSMQ allows you, in such cases, to control the rate at which the messages are consumed by setting a rate limit for a queue.

**Example**

Expand All @@ -23,6 +23,8 @@ queueRateLimit.set('notifications', { limit: 200, interval: 60000 }, (err) => {
});
```

To configure and view rate limiting parameters for a queue, the [QueueRateLimit Class](api/classes/QueueRateLimit.md).
To configure and manage rate limiting for a queue see:

-[QueueRateLimit Class](api/classes/QueueRateLimit.md).

Queue rate limiting parameters can be also configured using the [HTTP API Interface](https://github.com/weyoss/redis-smq-monitor) or from your browser with the help of the [Web UI](https://github.com/weyoss/redis-smq-monitor-client).
30 changes: 15 additions & 15 deletions docs/queues.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@ All RedisSMQ queue types are **reliable**. A queue is said to be reliable, when

In a typical use case, both LIFO and FIFO queues use [brpoplpush](https://redis.io/commands/brpoplpush), which blocks the connection to the Redis server until a message is received. However, priority queues use pooling and lua scripting which introduce a little of overhead on the MQ and therefore priority queues are less performant than other queue types.

In the next examples for simplicity we are going to use a `EQueueDeliveryModel.POINT_TO_POINT` [delivery model](queue-delivery-models.md) for all the queues. But you may choose any combination of `EQueueType` and `EQueueDeliveryModel` that fits your application.

## LIFO (Last In, First Out) queues

![RedisSMQ LIFO Queuing](redis-smq-lifo-queuing.png)
Expand Down Expand Up @@ -119,23 +121,21 @@ A queue name is required to fulfill the following requirements:

## Managing Queues and Namespaces

Queues and Namespaces can be managed using the [Queue Class](api/classes/Queue.md) and [Namespace Class](api/classes/Namespace.md) which provide the following methods:
To manage queues and namespaces RedisSMQ provides:

1. [Namespace.getNamespaces()](api/classes/Namespace.md#getnamespaces): To retrieve the list of namespaces.
2. [Namespace.getNamespaceQueues()](api/classes/Namespace.md#getnamespacequeues): To retrieve the list of queues of a given namespace.
3. [Namespace.delete()](api/classes/Namespace.md#delete): To delete a namespace alongside with its queues.
4. [Queue.save()](api/classes/Queue.md#save): To create a queue.
5. [Queue.list()](api/classes/Queue.md#getqueues): To retrieve the list of queues from all namespaces.
6. [Queue.delete()](api/classes/Queue.md#delete): To delete a queue.
7. [Queue.exists()](api/classes/Queue.md#exists): To check of a queue exists.
8. [Queue.getProperties()](api/classes/Queue.md#getproperties): To retrieve properties of a given queue.
- [Namespace Class](api/classes/Namespace.md)
- [Queue Class](api/classes/Queue.md)

## Queue Messages

RedisSMQ provides:
To manage queue messages use:

- [QueueMessages Class](api/classes/QueueMessages.md)
- [QueuePendingMessages Class](api/classes/QueuePendingMessages.md)
- [QueueAcknowledgedMessages Class](api/classes/QueueAcknowledgedMessages.md)
- [QueueDeadLetteredMessages Class](api/classes/QueueDeadLetteredMessages.md)
- [QueueScheduledMessages Class](api/classes/QueueScheduledMessages.md)

Please note that queue acknowledged messages and dead-lettered messages are not saved by default.

- [QueueMessages Class](api/classes/QueueMessages.md) - To browse all queue messages
- [QueuePendingMessages Class](api/classes/QueuePendingMessages.md) - To browse queue pending messages
- [QueueAcknowledgedMessages Class](api/classes/QueueAcknowledgedMessages.md) - To browse/requeue/delete queue acknowledged messages
- [QueueDeadLetteredMessages Class](api/classes/QueueDeadLetteredMessages.md) - To browse/requeue/delete queue dead-lettered messages
- [QueueScheduledMessages Class](api/classes/QueueScheduledMessages.md) - To browse/delete queue scheduled messages
To manage acknowledged and dead-lettered messages you need to enable acknowledged/dead-lettered messages storage from your [RedisSMQ Configuration](configuration.md).
14 changes: 7 additions & 7 deletions docs/scheduling-messages.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,7 @@ To set up scheduling parameters for a given message, the [ProducibleMessage Clas
- [ProducibleMessage.setScheduledRepeat()](api/classes/ProducibleMessage.md#setscheduledrepeat)
- [ProducibleMessage.setScheduledRepeatPeriod()](api/classes/ProducibleMessage.md#setscheduledrepeatperiod)

To schedule your message, you can publish it, as any other message, from your [Producer Class](api/classes/Producer.md)
using the [Producer.produce()](api/classes/Producer.md#produce) method.
You can publish scheduled messages, as any other message, using the [Producer.produce()](api/classes/Producer.md#produce) method.

```javascript
'use strict';
Expand All @@ -24,14 +23,15 @@ msg
.setScheduledCRON(`0 0 * * * *`)
.setQueue('test_queue');

producer.produce(msg, (err) => {
producer.produce(msg, (err, messageIds) => {
if (err) console.error(err);
else console.log('Message has been successfully scheduled');
else console.log('Message has been successfully scheduled. Message IDs are: ', messageIds.join(','));
});
```

For managing scheduled messages see [QueueScheduledMessages Class](api/classes/QueueScheduledMessages.md).
For managing scheduled messages see:

To delete a scheduled message see [Message.deleteMessageById()](api/classes/Message.md#deletemessagebyid).
- [QueueScheduledMessages Class](api/classes/QueueScheduledMessages.md)
- [Message Class](api/classes/Message.md)

Scheduled messages can be also managed using the [HTTP API Interface](https://github.com/weyoss/redis-smq-monitor) or from your browser with the help of the [Web UI](https://github.com/weyoss/redis-smq-monitor-client).
Scheduled messages may be also managed using the [HTTP API Interface](https://github.com/weyoss/redis-smq-monitor) or from your browser with the help of the [Web UI](https://github.com/weyoss/redis-smq-monitor-client).

0 comments on commit 98910dc

Please sign in to comment.