Where to configure cache cleaning policy in redis

When using Redis as a cache, if the memory space is full, old data will be automatically evicted. Memcached works this way by default, and most developers are familiar with it. (Recommended study: Redis Video Tutorial)

LRU is the only recycling algorithm supported by Redis. This article introduces in detail the maxmemory instruction used to limit the maximum memory usage, and explains in depth what Redis uses. Approximate LRU algorithm.

maxmemory configuration directive

maxmemory is used to specify the maximum memory that Redis can use. It can be set in the redis.conf file or dynamically modified during operation through the CONFIG SET command.

For example, to set a memory limit of 100MB, you can configure it in the redis.conf file like this:

maxmemory 100mb

Set maxmemory to 0, which means no memory limit. Of course, there is an implicit limitation for 32-bit systems: up to 3GB of RAM.

When memory usage reaches the maximum limit, if new data needs to be stored, Redis may directly return an error message or delete some old data depending on the configured policies.

Eviction policy

When the maximum memory limit (maxmemory) is reached, Redis determines the specific behavior based on the policy configured by maxmemory-policy.

The current version, the strategies supported by Redis 3.0 include:

noeviction: Do not delete the strategy. When the maximum memory limit is reached, if more memory is needed, directly Return error message. Most write commands will cause more memory to be occupied (with rare exceptions, such as DEL).

allkeys-lru: Common to all keys; delete the least recently used (LRU) keys first.

volatile-lru: Only the part with expire set; delete the least recently used (LRU) key first.

allkeys-random: Common to all keys; randomly delete some keys.

volatile-random: Only limited to the part where expire is set; randomly delete a part of the key.

volatile-ttl: Only limited to the part where expire is set; keys with short remaining time (time to live, TTL) will be deleted first.

If the expire key is not set and the prerequisites are not met; then the behavior of volatile-lru, volatile-random and volatile-ttl strategies is basically the same as noeviction (no deletion).

You need to choose an appropriate eviction strategy based on the characteristics of the system. Of course, you can also dynamically set the eviction policy through commands during operation, and monitor cache misses and hits through the INFO command for tuning.

Generally speaking:

If it is divided into hot data and cold data, it is recommended to use the allkeys-lru strategy. That is, some of the keys are often read and written. If you are not sure about the specific business characteristics, then allkeys-lru is a good choice.

If you need to read and write all keys in a loop, or the access frequency of each key is similar, you can use the allkeys-random strategy, that is, the probability of reading and writing all elements is almost the same.

If you want Redis to filter keys that need to be deleted based on TTL, please use the volatile-ttl strategy.

The main application scenarios of volatile-lru and volatile-random strategies are: instances with both cache and persistent keys. Generally speaking, for scenarios like this, two separate Redis instances should be used.

It is worth mentioning that setting expire will consume additional memory, so using the allkeys-lru strategy can make more efficient use of memory, because this way you no longer need to set the expiration time.

Internal implementation of eviction

The eviction process can be understood as follows:

The client executes a command, resulting in Redis Data increases and takes up more memory.

Redis checks the memory usage. If it exceeds the maxmemory limit, some keys will be cleared according to the policy.

Continue to execute the next command, and so on.

During this process, the memory usage will continuously reach the limit value, then exceed it, and then delete some keys, and the usage will drop below the limit value again.

If a certain command causes a large amount of memory usage (such as saving a large set through a new key), the memory usage may significantly exceed the maxmemory limit for a period of time.

Approximate LRU algorithm

Redis does not use the complete LRU algorithm. The automatically evicted key is not necessarily the one that best satisfies the LRU characteristics. Instead, a small number of key samples are extracted through the approximate LRU algorithm, and then the key with the oldest access time is deleted.

The eviction algorithm has been greatly optimized since Redis 3.0, using pool as a candidate. This greatly improves the algorithm efficiency and is closer to the real LRU algorithm.

In the Redis LRU algorithm, the accuracy of the algorithm can be tuned by setting the number of samples. Configure through the following instructions:

maxmemory-samples 5

Why not use full LRU implementation? The reason is to save memory. But the behavior of Redis is basically equivalent to LRU. The following is a behavioral comparison chart between Redis LRU and the complete LRU algorithm.

During the test, access starts from the first key, so the first key is the best eviction object.

You can see three types of points from the picture, forming three different strips.

The light gray part indicates the evicted object.

The gray part indicates "not evicted" objects.

The green part indicates the objects added later.

In the pure LRU algorithm implementation, the first half of the old keys are released. The LRU algorithm of Redis only releases longer keys probabilistically.

As you can see, in Redis 3.0, the effect of 5 samples is much better than that of Redis 2.8. Of course, Redis 2.8 is also good, the last accessed key is basically still in the memory. When using 10 samples in Redis 3.0, it is very close to the pure LRU algorithm.

Note that LRU is only a probability model used to predict that a certain key may continue to be accessed in the future. In addition, if the data access situation conforms to the power law distribution (power law), then for most requests , LRU will perform well.

In the simulation, we found that if power law access is used, the results of pure LRU and Redis are very different, or even invisible.

Of course, you can also increase the number of samples to 10, at the cost of consuming some additional CPU, so that the results are closer to the real LRU, and the difference can be judged through cache miss statistics.

It is easy to set the sample size, just use the command CONFIG SET maxmemory-samples

For more Redis related technical articles, please visit the Redis Getting Started Tutorial column Get studying!

The above is the detailed content of Where to configure cache cleaning policy in redis. For more information, please follow other related articles on the PHP Chinese website!