Does redis have a scheduled deletion function?

There are three possible answers to Redis timeout deletion, which represent three different deletion strategies:

Timed deletion: While setting the expiration time of the key, Create a timer (timer) and let the timer immediately perform the deletion of the key when the key's expiration time comes.

Lazy deletion: Let the key expire, but every time you get the key from the key space, check whether the obtained key has expired. If it has expired, delete the key; if it has not expired, return the key .

Regular deletion: Every once in a while, the program checks the database and deletes the expired keys. It's up to the algorithm to decide how many expired keys to delete and how many databases to check.

Among these three strategies, the first and third are active deletion strategies, while the second is a passive deletion strategy.

Scheduled deletion:

The scheduled deletion strategy is the most memory-friendly: by using a timer, the scheduled deletion strategy can ensure that expired keys will Removed as quickly as possible and free the memory occupied by expired keys.

On the other hand, the disadvantage of the scheduled deletion strategy is that it is the least friendly to CPU time: when there are many expired keys, deleting expired keys may take up a considerable part of the CPU time. When memory is not tight but CPU time is very tight, using CPU time to delete expired keys that are irrelevant to the current task will undoubtedly affect the server's response time and throughput.

For example, if there are a large number of command requests waiting for the server to process, and the server is not currently short of memory, then the server should prioritize using CPU time to process the client's command requests instead of deleting expired keys. above.

In addition, creating a timer requires the use of time events in the Redis server. The current time event is implemented as an unordered linked list. The time complexity of finding an event is O(N). ——Cannot handle large amounts of time events efficiently.

Therefore, it is not realistic at this stage to ask the server to create a large number of timers to implement a scheduled deletion strategy.

Lazy deletion:

The lazy deletion strategy is the most CPU time friendly: the program will only delete the key when it is removed Expiration check is performed, which ensures that deletion of expired keys will only be done when necessary, and the deletion target is limited to the currently processed keys. This strategy will not spend any CPU time on deleting other irrelevant expired keys. .

The disadvantage of the lazy deletion strategy is that it is the least friendly to memory: if a key has expired, and the key is still retained in the database, then as long as the expired key is not deleted, it will occupy The memory will not be released.

When using the lazy deletion strategy, if there are a lot of expired keys in the database, and these expired keys happen to not be accessed, then they may never be deleted (unless the user manually executes FLUSHDB ), we can even regard this situation as a memory leak - useless garbage data occupies a large amount of memory, but the server will not release them by itself. This is a problem for the Redis server whose running status is very dependent on memory. That's definitely not good news.

For example, for some time-related data, such as logs, after a certain point in time, access to them will be greatly reduced, or even no longer accessed. If such expired data A large number of keys are backlogged in the database. Users think that the server has automatically deleted them, but in fact these keys still exist, and the memory occupied by the keys has not been released. The consequences must be very serious.

Regular deletion:

From the above discussion of scheduled deletion and lazy deletion, these two deletion methods are very effective when used alone. All have obvious flaws:

·Timed deletion takes up too much CPU time, affecting the server's response time and throughput.

·Lazy deletion wastes too much memory and risks memory leaks.

The periodic deletion strategy is an integration and compromise of the first two strategies:

·The periodic deletion strategy performs the deletion of expired keys every once in a while and performs it by limiting the deletion operation. Duration and frequency to reduce the impact of deletion operations on CPU time.

·In addition, by regularly deleting expired keys, the regular deletion strategy effectively reduces the memory waste caused by expired keys.

The difficulty of the periodic deletion strategy is to determine the length and frequency of the deletion operation:

·If the deletion operation is performed too frequently, or the execution time is too long, the periodic deletion strategy will degenerate into The scheduled deletion strategy consumes too much CPU time in deleting expired keys.

·If the deletion operation is performed too rarely, or the execution time is too short, the regular deletion strategy will be the same as the lazy deletion strategy, resulting in a waste of memory.

The periodic deletion strategy of expired keys is implemented by the redis.c/activeExpireCycle function. Whenever the Redis server periodically operates the redis.c/serverCron function, the activeExpireCycle function will be called. It is called at the specified time. Within, each database in the server is traversed multiple times, the expiration time of some keys is randomly checked from the expires dictionary of the database, and the expired keys are deleted.

The entire process can be described in pseudo code as follows:

#
默认每次检查的数据库数量
DEFAULT_DB_NUMBERS = 16
#
默认每个数据库检查的键数量
DEFAULT_KEY_NUMBERS = 20
#
全局变量，记录检查进度
current_db = 0
def activeExpireCycle():
    #
初始化要检查的数据库数量
    #
如果服务器的数据库数量比 DEFAULT_DB_NUMBERS
要小
    #
那么以服务器的数据库数量为准
    if server.dbnum < DEFAULT_DB_NUMBERS:
        db_numbers = server.dbnum
    else:
        db_numbers = DEFAULT_DB_NUMBERS
    #
遍历各个数据库
    for i in range(db_numbers):
        #
如果current_db
的值等于服务器的数据库数量
        #
这表示检查程序已经遍历了服务器的所有数据库一次
        #
将current_db
重置为0
，开始新的一轮遍历
        if current_db == server.dbnum:
            current_db = 0
        #
获取当前要处理的数据库
        redisDb = server.db[current_db]
        #
将数据库索引增1
，指向下一个要处理的数据库
        current_db += 1
        #
检查数据库键
        for j in range(DEFAULT_KEY_NUMBERS):
            #
如果数据库中没有一个键带有过期时间，那么跳过这个数据库
            if redisDb.expires.size() == 0: break
            #
随机获取一个带有过期时间的键
            key_with_ttl = redisDb.expires.get_random_key()
            #
检查键是否过期，如果过期就删除它
            if is_expired(key_with_ttl):
                delete_key(key_with_ttl)
            #
已达到时间上限，停止处理
            if reach_time_limit(): return

The working mode of the activeExpireCycle function can be summarized as follows:

·Every time the function runs, it starts from a certain number of databases A certain number of random keys are taken out for inspection and expired keys are deleted.

·The global variable current_db will record the progress of the current activeExpireCycle function check, and the next time the activeExpireCycle function is called, the previous progress will be processed. For example, if the current activeExpireCycle function returns when traversing database No. 10, then the next time the activeExpireCycle function is executed, it will search for and delete expired keys starting from database No. 11.

·As the activeExpireCycle function continues to execute, all databases in the server will be checked. At this time, the function resets the current_db variable to 0, and then starts a new round of checking work again.

For more Redis related knowledge, please visit the Redis usage tutorial column!

The above is the detailed content of Does redis have a scheduled deletion function?. For more information, please follow other related articles on the PHP Chinese website!