What should I do if the double-write caches of Redis and MySQL are inconsistent? Solution sharing

What should I do if the double-write cache of Redis and MySQL is inconsistent? This article will share with you how to solve the problem of cache double-write inconsistency. I hope it can help you!

redis and mysql double-write cache are inconsistent:

But in terms of updating the cache, for the update is complete Database, should we update the cache or delete the cache? Or delete the cache first and then update the database. In fact, there is a lot of controversy. Currently there is no comprehensive blog that analyzes these solutions. So the blogger wrote this article tremblingly and risking being criticized by everyone.

Text

Set the expiration time for cached data

Let me make an explanation first. In theory, set the cache Expiration time is a solution to ensure eventual consistency. Under this solution, we can set the expiration time for the data stored in the cache. All write operations are subject to the database, and we only need to do our best for cache operations. That is to say, if the database write is successful and the cache update fails, then as long as the expiration time is reached, subsequent read requests will naturally read new values ​​from the database and backfill the cache. Therefore, the ideas discussed next do not rely on the solution of setting an expiration time for the cache.

Here, we first discuss three update strategies:

1. Update the database first, then update the cache
2. Delete the cache first, and then Update the database
3. Update the database first, then delete the cache

Update the database first, then update the cache

This scheme is generally opposed by everyone . why? There are two points as follows:

• Reason one (thread safety perspective)

(1) Thread A updated the database
(2) Thread B updated the database
(3) Thread B updated the cache
(4) Thread A updated the cache

This means that requesting A to update the cache should be earlier than requesting B to update the cache, but due to network and other reasons, But B updated the cache earlier than A. This results in dirty data and is therefore not considered.

• Reason Two (Business Scenario Perspective)

(1) If you have business needs with more database writing scenarios and fewer data reading scenarios, use this This solution will cause the cache to be frequently updated before the data is read, which wastes performance.

(2) If the value you write to the database is not written directly into the cache, it must go through a series of complex calculations and then be written into the cache. Then, calculating the value written to the cache again every time it is written to the database is undoubtedly a waste of performance. Obviously, deleting the cache is more appropriate.

Delete the cache first, and then update the database

The reason why this solution will cause inconsistency is. At the same time, one requests A to perform an update operation, and the other requests B to perform a query operation. Then the following situation will occur:

(1) Request A to perform a write operation and delete the cache
(2) Request B to query and find that the cache does not exist
(3) Request B to query the database to obtain the old value
(4) Request B to write the old value into the cache
(5) Request A to write the new value into the database

The above situation will lead to inconsistencies. Moreover, if you do not set an expiration time strategy for the cache, the data will always be dirty data.

So, how to solve it? Adopt delayed double deletion strategy

Cache delayed double deletion

public class CacheServiceImpl implements ICacheService {

    @Resource
    private RedisOperator redisOperator;
    
    @Autowired
    private IShopService shopService;

    //1. 采用延时双删，解决数据库和缓存的一致性
    @Override
    public void updateHotCount(String id) {
        try {
            //删除缓存
            redisOperator.del("redis_key_" + id);
            //更新数据库
            shopService.updataHotShop();
            Thread.sleep(1000);//休眠1秒
            //延时删除
            redisOperator.del("redis_key_" + id);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }


    }

    @Override
    public Integer getHotCount(String id) {
        return null;
    }}

Explanation:

1. Eliminate the cache first
2. Write the database again
3. Sleep for 1 second, and then eliminate the cache (By doing this, you can delete the cached dirty data caused within 1 second again.)

In view of the above situation, readers should evaluate the time-consuming data reading business logic of their own projects. Then the sleep time for writing data is based on the time consuming for reading data business logic, plus a few hundred milliseconds. The purpose of this is to ensure that the read request ends, and the write request can delete the cached dirty data caused by the read request.

What if the database adopts a read-write separation architecture? (The main library is responsible for writing operations, and the slave library is responsible for reading operations)

ok, in this case, the reasons for data inconsistency are as follows, there are still two requests and one request A performs an update operation, and the other requests B to perform a query operation.

(1) Request A to perform a write operation, delete the cache, and request A to write data to the main library. Synchronization from the slave library has not started yet

(2)(Within 1s) Request B to query the cache , the cache is not found, and B is requested to query the slave database. At this time, the master-slave synchronization has not been completed, and the old value is found, and the old value is written to the cache.

(3) The master database completes the master-slave synchronization, and the slave database changes to the new value

The above process is the problem of data inconsistency, and a double-deletion delay strategy is also used. However, the sleep time is modified to be based on the master-slave synchronization delay time, plus a few hundred ms

If this synchronization elimination strategy is adopted, what should I do if the throughput is reduced?

ok, then treat the second deletion as asynchronous. Start a thread yourself and delete it asynchronously. In this way, the written request does not have to sleep for a period of time before returning. Doing so increases throughput.

What should I do if the deletion fails for the second time?

This is a very good question, because if the deletion fails for the second time, the following situation will occur. There are still two requests, one requests A to perform an update operation, and the other requests B to perform a query operation. For convenience, it is assumed to be a single database:

(1) Request A to perform a write operation and delete the cache
( 2) Request B to query and find that the cache does not exist
(3) Request B to query the database to get the old value
(4) Request B to write the old value into the cache
(5) Request A to write the new value Database
(6) Request A tried to delete the cache value written by request B, but failed.

ok, that is to say. If the cache deletion fails for the second time, the cache and database inconsistency will occur again.

How to solve it?

For specific solutions, let’s look at the blogger’s analysis of the update strategy of updating the database first and then deleting the cache.

Delete cache retry mechanism

Whether it is Delayed double deletion or Cache-Aside first operates the database and then deletes the cache, there may be data inconsistency problems caused by failure to delete the cache in the second step. You can use this solution to optimize: if the deletion fails, delete it a few more times to ensure that the cache deletion is successful~ So you can introduce a delete cache retry mechanism

1. (1) Update database data;
   (2) Cache deletion failed due to various problems
   (3) Send the key that needs to be deleted to the message queue
   (4) Consume the message yourself, Obtain the key that needs to be deleted
   (5) Continue to retry the deletion operation until it succeeds

However, this solution has a shortcoming, causing a lot of intrusion into the business line code. So we have the second option. In the second option, start a subscription program to subscribe to the binlog of the database to obtain the data that needs to be operated. In the application, start a new program to obtain the information from this subscription program and delete the cache.

Read biglog and delete cache asynchronously

The process is as shown in the figure below:

(1) Update database Data
(2) The database will write the operation information into the binlog log
(3) The subscription program extracts the required data and key
(4) Start a new section of non-business code to obtain the information
(5) Try to delete the cache operation and find that the deletion failed
(6) Send the information to the message queue
(7) Re-obtain the data from the message queue and retry the operation.

Remarks: The above binlog subscription program has a ready-made middleware called canal in mysql, which can complete the function of subscribing to binlog logs. As for Oracle, the blogger currently does not know if there is any ready-made middleware that can be used. In addition, as for the retry mechanism, the blogger uses a message queue. If the consistency requirements are not very high, just start a new thread in the program and try again every once in a while. You can use this flexibly, but I just provide an idea.

This article is actually a summary of the existing consistency solutions in the Internet. Regarding the update strategy of first deleting the cache and then updating the database, there is also a plan to maintain a memory queue. The blogger looked at it and felt that the implementation is extremely complicated and unnecessary, so there is no need to give it in the article. Finally, I hope you all gained something.

[Related recommendations: mysql video tutorial]

The above is the detailed content of What should I do if the double-write caches of Redis and MySQL are inconsistent? Solution sharing. For more information, please follow other related articles on the PHP Chinese website!