How redis message queue prevents data loss

Redis implements message queues in two forms:

Broadcast subscription mode: Based on the Pub/Sub mechanism of Redis, once a client goes Publish a message in a certain key, and all subscribe clients will trigger the event cluster subscription mode: based on Redis List bidirectional atomicity BRPOP (recommended learning: Redis video tutorial)

When using the Redis message queue, messages may be lost when Redis goes down (it also depends on the persistence strategy). If the message recipient does not have a resending and verification mechanism, the data in Redis will be lost. So, using Redis as a message queue is usually a scenario where the accuracy of the message is not particularly high.

If you absolutely guarantee the final consistency of the data and ensure that the message is 100% not lost, then you need:

1.Requirements when writing Enable transaction processing to ensure successful writing.

2. Redis is configured so that any changes must be persisted in real time. For example, if the storage end is a disk, each change will be written to the disk synchronously before it is completed. Redis supports configuration in this way, but doing so will make its in-memory database feature completely disappear and its performance become very low.

3. The consumer side must also implement transaction mode. After the processing is completed, it will come back to actually delete the message.

4. Concurrent processing of multiple threads or multiple terminals at the same time can be avoided by locking.

3 The requirements of 4 need to be implemented by yourself. You can consider them together. You can also use another queue to implement them, but a better way is to implement a counter inside the queue. For hash format, add a field and add a value. For list, add a value first to base it. If you add a value and a separator to the head of string, you can make a simple counter. Although it is simple, it is practical enough.

Except for specific systems, such strong consistency is generally not required. It is not difficult to implement, but the performance will be very poor.

Banking payment businesses require strict transaction consistency, while Internet businesses generally use a tricky approach that can tolerate a small amount of data loss in a very short period of time in exchange for higher performance.

For example, the above redis processing can be changed to actually write to the disk when 1000 data changes. In extreme cases, such as a sudden power outage, there is a risk of losing these 1,000 pieces of data. Of course, the probability of this happening is also very low (stay away from Lanxiang excavator?), so it is acceptable in most scenarios.

For more Redis-related technical articles, please visit the Redis Getting Started Tutorial column to learn!

The above is the detailed content of How redis message queue prevents data loss. For more information, please follow other related articles on the PHP Chinese website!