How to implement message queue based on redis

Message Queue, Message Queue, is often used to solve resource consistency problems in concurrent systems, improve peak processing capabilities, and ensure the orderliness, recoverability, and must-delivery of messages. , decouple applications, or implement asynchronous communication, etc. (Recommended learning: Redis video tutorial)

There are many MQ applications on the market (for example: Kafka, RabbitMQ, Disque), and they can also be implemented based on Redis, which is more typical The solutions include:

Implementation of LPUSH BRPOP based on List

PUB/SUB, subscription/publishing mode

Implementation based on Sorted-Set

Implementation based on Stream type

In the use of message queues, there are producers and consumers. Producers are responsible for generating messages, and consumers are responsible for processing messages.

Production refers to putting messages into the message queue.

Consumption refers to reading and processing messages. Usually after a message is consumed, it should be deleted from the message queue.

Implementation of LPUSH BRPOP based on List

The typical command is:

LPUSH，将消息队列
BRPOP，从队列中取出消息，阻塞模式

is a typical solution based on FIFL queue. Among them, LPUSH is what the producer does, and BRPOP is what the consumer does.

This model has many advantages:

Simple implementation

Reids supports persistent messages, which means that messages will not be lost and can be viewed repeatedly (note Not consuming, just watching and using, LRANGE class instructions).

The order can be ensured, and the LPUSH command can be used to ensure the order of the messages.

Using RPUSH, the message can be placed at the beginning of the queue to achieve the purpose of prioritizing messages and realizing simple messages. Priority queue.

At the same time, there are some disadvantages:

It is more troublesome to do consumption confirmation ACK, that is, there is no guarantee that the consumer will have unprocessed downtime problems after reading. Resulting in unexpected loss of messages. Usually you need to maintain a Pending list yourself to ensure that the message is processed and confirmed.

Cannot do broadcast mode, such as the typical Pub/Discribe mode.

Cannot be consumed repeatedly, once consumed it will be deleted

Group consumption is not supported, you need to solve it yourself at the business logic layer

PUB/SUB, subscription/publishing Pattern

SUBSCRIBE，用于订阅信道
PUBLISH，向信道发送消息
UNSUBSCRIBE，取消订阅

Producers and consumers interact through the same channel (Channel). A channel is actually a queue. There are usually multiple consumers. Multiple consumers subscribe to the same channel. When a producer publishes a message to the channel, the channel will immediately publish the message to each consumer one by one. It can be seen that the channel is a divergent channel for consumers, and each consumer can get the same message. Typical one-to-many relationship.

The typical advantages are:

Typical broadcast mode, a message can be published to multiple consumers

Multi-channel subscription, consumers can Subscribe to multiple channels at the same time to receive multiple types of messages

Messages are sent immediately. The message does not need to wait for the consumer to read. The consumer will automatically receive the message published by the channel

There are also some Disadvantages:

Once the message is published, it cannot be received. In other words, if the client is not online when publishing, the message will be lost and cannot be retrieved

There is no guarantee that the time received by each consumer is consistent

If a message appears on the consumer client The backlog, to a certain extent, will be forcibly disconnected, resulting in unexpected loss of messages. It usually occurs when the production of messages is much faster than the consumption speed

It can be seen that the Pub/Sub mode is not suitable for message storage and message backlog services, but is good at processing broadcast, instant messaging, and instant feedback services.

Implementation based on SortedSet ordered set

ZADD KEY score member，压入集合
ZRANGEBYSCORE，依据score获取成员

The scheme of ordered set is more commonly used when determining the message order ID by yourself, using the Score of the set member as the message ID, guarantees the order, and can also ensure the monotonous increase of the message ID. Usually a timestamp sequence number scheme can be used. The monotonic increase of message ID is ensured, and an ordered message queue can be created by using SortedSet's feature of sorting according to Score.

Compared with the above solution, the advantage is that the message ID can be customized, which is more important when the message ID is meaningful. The disadvantages are also obvious. Duplicate messages are not allowed (think of them as collections). At the same time, errors in determining the message ID will lead to errors in the order of the messages.

So, if you don’t need to customize the message ID, this solution seems a bit tasteless...

Implementation based on Stream type

This Stream type redis is In order to implement message queue. Supports core message queue functions such as automatic generation of message ID, group consumption, ACK, message transfer, queue monitoring, etc.

For more Redis-related technical articles, please visit the Redis Getting Started Tutorial column to learn!

The above is the detailed content of How to implement message queue based on redis. For more information, please follow other related articles on the PHP Chinese website!