Three methods of Redis deletion strategy and analysis of eviction algorithm examples

1. Preface

Before the article begins, let me ask you a question: When we use the command: expire key secondSet the expiration time for a key, once the expiration time is up, Is the expired data corresponding to this key really deleted immediately by the server? The answer is that it will not be deleted immediately. Knowing this answer, let's take a look at how expired data is handled in Redis.

2. Data characteristics in Redis

Redis is a memory-level database. All data is stored in the memory. The data in the memory can obtain its status through TTL instructions. There are three states:

Instruction result	State
XX	Time-sensitive data
-1	Permanently valid data
-2	has been Expired data or deleted data or undefined data

3. Time-sensitive data storage structure

When we use instructions to set expired data, the address corresponding to the data will be placed in the expires space, and the storage method is hash. , the stored value is the expiration time.

4. Data deletion strategy

Data deletion strategy goal: Find a balance between memory usage and CPU usage. Failure to consider one or the other will result in a decline in overall redis performance. It may even cause server downtime or memory leaks

1. Scheduled deletion

• Create a timer. When the key is set to have an expiration time and the expiration time is reached, the timer will be used. The server task immediately executes the deletion operation on the key

• Advantages:Save memory, delete it when the time comes, and quickly release unnecessary memory usage

• Disadvantages: The CPU is under great pressure. No matter how high the CPU load is at this time, the CPU is occupied, which will affect the redis server response time and instruction throughput

2. Lazy deletion

• No processing will be done if the data reaches the expiration time. The next time the data is accessed, the expireIfNeeded() function will be called to determine whether the data has expired: If it has not expired, return the data, find it has expired, delete it, and return it does not exist

• Advantages: Save CPU performance, delete only when you find it must be deleted

• Disadvantages: The memory pressure is very high, and it appears Data that occupies memory for a long time

3. Periodic deletion

Process analysis :

• When the Redis server starts and initializes, read the value of configuration server.hz, the default is 10

• and then every second Execute the serverCron() method 10 times, which is used to detect the server

• serverCron() method will call databasesCron( ) method, this method is used to traverse the database. Redis has 16 databases by default, starting from the first database

• databasesCron() method The activeExpireCycle() method will be called again, which will detect each expires[*] one by one, executing 250ms/server.hz ​​each time; for a certain expires [*]During detection, W keys are randomly selected for detection

• If the key timeout is detected, the key will be deleted; if the number of keys deleted in one round > W25% , loop the process; if the number of keys deleted in one round is ≤W25%, check the next expires[*], 0-15 loop. Where W value = ACTIVE_EXPIRE_CYCLE_LOOKUPS_PER_LOOP attribute value

• If the execution time of activeExpireCycle() expires, the execution will continue from current_db next time

• Parameters current_db is used to record which expires[*] activeExpireCycle() enters to execute

• ##Summary: cycle Consistently poll the timely data in the redis database, adopt a random extraction strategy, and use the proportion of expired data to control the frequency of deletion

• The memory pressure is not very high, and the memory will be occupied for a long time Cold data will be continuously cleaned

5. Comparison of deletion strategies

Regular deletionSave memory, no OccupyOccupies CPU resources regardless of time period, high frequencyExchange time for spaceLazy deletionSevere memory usage Delayed execution, high CPU utilizationTrade space for timeRegular deletionRegular random cleaning of memory Spend a fixed amount of CPU resources per second to maintain memory Random and focused checks

6. Eviction algorithm

1. Concept introduction

When new data enters redis, what should I do if there is insufficient memory? Redis uses memory to store data. Before executing each command, it will call freeMemoryIfNeeded() to check whether the memory is sufficient. If the memory does not meet the minimum storage requirements for newly added data, redis will temporarily delete some data to clear storage space for the current instruction. The strategy for cleaning data is called the eviction algorithm.

Note: The process of evicting data is not 100% able to clear out enough usable memory space. If it is unsuccessful, it will be executed repeatedly. After trying all data, if the memory cleanup requirements cannot be met, an error message will appear.

2. Eight configurations

Detect volatile data (Data sets that may expire server.db[i].expires )

volatile-lru	Select the least recently used data to eliminate
volatile-lfu	Select the least recently used data to eliminate
volatile-ttl	Select the data that is about to expire
volatile-random	Randomly select data elimination

Detect the entire database data (all data sets server.db[i].dict )

allkeys-lru	Select the least recently used data to eliminate
allkeys-lfu	Select the least recently used data to eliminate
allkeys-random	Select any data to eliminate

Give up data eviction:

no-enviction	Disable eviction of data (default policy in redis4.0)

The above is the detailed content of Three methods of Redis deletion strategy and analysis of eviction algorithm examples. For more information, please follow other related articles on the PHP Chinese website!