How to ensure double-write consistency between Redis and MySQL? (Meituan Ermian)

前言

四月份的时候，有位朋友去美团面试，他说被问到Redis与MySQL双写一致性如何保证？ 这道题其实就是在问缓存和数据库在双写场景下，一致性是如何保证的？本文将跟大家一起来探讨如何回答这个问题。

• github地址，感谢每一颗star

谈谈一致性

一致性就是数据保持一致，在分布式系统中，可以理解为多个节点中数据的值是一致的。

• Strong consistency: This consistency level is most in line with user intuition. Whatever it requires the system to write will be read out. The user experience is good, but it is difficult to implement. It often has a great impact on the performance of the system
• Weak consistency: This consistency level restricts the system from being able to read the written value immediately after the write is successful, nor does it guarantee that the written value can be read immediately. Promise how long it will take for the data to be consistent, but we will try our best to ensure that the data can reach a consistent state after a certain time level (such as the second level)
• Eventual Consistency: Final Consistency It is a special case of weak consistency. The system will ensure that a data consistency state can be achieved within a certain period of time. The reason why final consistency is mentioned separately here is because it is a very respected consistency model in weak consistency, and it is also a model that is highly respected in the industry for data consistency in large distributed systems

Three classic caching modes

Caching can improve performance and relieve database pressure, but using cache can also lead to data inconsistency problems. How do we generally use cache? There are three classic caching patterns:

• Cache-Aside Pattern
• Read-Through/Write through
• Write behind

Cache -Aside Pattern

Cache-Aside Pattern, that is, Bypass cache mode, is proposed to solve the problem of data inconsistency between the cache and the database as much as possible.

Cache-Aside read process

The read request process of Cache-Aside Pattern is as follows:

1. When reading, read the cache first. If the cache hits, the data will be returned directly.
2. If the cache does not hit, read the database, retrieve the data from the database, put it into the cache, and return the response at the same time.

Cache-Aside write process

The write request process of Cache-Aside Pattern is as follows:

When updating, first update the database and then delete the cache .

Read-Through/Write-Through (read-write penetration)

In Read/Write Through mode, the server uses the cache as the main data storage. The interaction between the application and the database cache is completed through the Abstract Cache Layer.

Read-Through

The brief process of Read-Through is as follows

1. Read from cache If the data cannot be read, it will be returned directly. If it cannot be read, it will be loaded from the database, written to the cache, and then the response will be returned.
2. Is this brief process very similar to

Cache-Aside

? In fact, Read-Through is just an extra layer of Cache-Provider, and the process is as follows:

Read-Through is actually just

Cache-Aside

has a layer of encapsulation on top, which will make the program code more concise and reduce the load on the data source. Write-Through

Write-Through

In mode, when a write request occurs, the data source and cached data are also completed by the Cache Abstraction Layer The update process is as follows: Write behind (asynchronous cache writing)

Write behind

followed by Read-Through/Write-ThroughThere are similarities. Cache Provider is responsible for reading and writing cache and database. There is a big difference between them: Read/Write Through updates the cache and data synchronously, while Write Behind only updates the cache and does not directly update the database, through Batch asynchronous way to update the database.

In this method, the consistency between the cache and the database is not strong.

Systems with high consistency requirements should be used with caution

. But it is suitable for frequent writing scenarios. MySQL's InnoDB Buffer Pool mechanism uses this mode. When operating the cache, should I delete the cache or update the cache?

In general business scenarios, we use the

Cache-Aside

mode. Some friends may ask, Cache-AsideWhen writing a request, why delete the cache instead of updating the cache?

When we operate the cache, should we delete the cache or update the cache? Let’s look at an example first:

1. Thread A first initiates a write operation, and the first step is to update the database
2. Thread B then initiates a write operation Write operation, the second step updates the database
3. Due to network and other reasons, thread B updates the cache first
4. Thread A updates the cache.

At this time, the cache saves A's data (old data), and the database saves B's data (new data). The data is inconsistent, and dirty data appears. . If deletes the cache instead of updating the cache, this dirty data problem will not occur.

Updating the cache has two disadvantages compared to deleting the cache:

• If the cache value you write is obtained after complex calculations. If the cache is updated frequently, performance will be wasted.
• When there are many database writing scenarios and few data reading scenarios, the data is often updated before it is read, which also wastes performance (actually, in scenarios where there is a lot of writing, It is not very cost-effective to use cache)

In the case of double writing, should the database be operated first or the cache first?

Cache-AsideIn the cache mode, some friends still have questions. When writing a request, why operate the database first? Why not operate the cache first?

Suppose there are two requests, A and B, requesting A to do the update operation and requesting B to do the query and read operation.

1. Thread A initiates a write operation, the first step is del cache
2. At this time, thread B initiates a read operation, cache miss
3. Thread B continues Read DB, read out an old data
4. Then thread B sets the old data into cache
5. Thread A writes the latest data in DB

Jiang Zi has a problem La, The cache and database data are inconsistent. The cache stores old data, and the database stores new data. Therefore, Cache-Aside cache mode chooses to operate the database first instead of the cache first.

Cache Delayed Double Delete

Some friends may say that it is not necessary to operate the database first, just use the Cache Delayed Double Delete strategy? What is delayed double deletion?

1. Delete the cache first
2. Then update the database
3. Sleep for a while (such as 1 second) and delete the cache again.

How long does it usually take to sleep for a while? Are they all 1 second?

This sleep time = the time it takes to read business logic data is several hundred milliseconds. In order to ensure that the read request ends, the write request can delete cached dirty data that may be brought by the read request.

Delete cache retry mechanism

Whether it is delayed double deletion or Cache-Aside first operates the database and then deletes the cache, If the second step of deleting the cache fails, the deletion failure will result in dirty data~

If the deletion fails, delete it a few more times to ensure that the cache deletion is successful~So you can introduce Delete cache Retry mechanism

1. Write request to update the database
2. The cache failed to delete for some reason
3. Put the key that failed to be deleted into the message queue
4. Consume messages from the message queue and obtain the key to be deleted
5. Retry the deletion cache operation

Read the biglog Asynchronous cache deletion

The retry deletion cache mechanism is okay, but it will cause a lot of business code intrusion. In fact, you can also eliminate key asynchronously through the binlog of the database.

Taking mysql as an example, you can use Alibaba's canal to collect binlog logs and send them to the MQ queue, and then confirm and process the update message through the ACK mechanism, delete the cache, and ensure data Cache consistency

Recommended learning: "Redis Video Tutorial"