How to cluster redis

Redis Sharding Cluster

Redis Sharding is a client-side Sharding sharding technology.

Redis Sharding can be said to be a multi-Redis instance cluster method commonly used in the industry before Redis Cluster came out. The main idea is to use a hash algorithm to hash the key of Redis data. Through the hash function, a specific key will be mapped to a specific Redis node. (Recommended learning: Redis video tutorial)

In this way, the client knows which Redis node to operate data on. It should be noted that this is on the client side. Completed.

java redis client jedis already supports Redis Sharding function, namely ShardedJedis and ShardedJedisPool combined with cache pool. Jedis's Redis Sharding implementation has the following characteristics:

1. Use consistent hashing algorithm (consistent hashing)

Hash the key and node name at the same time, and then Mapping matching, the algorithm used is MURMUR_HASH. The main reason for consistent hashing is that when adding or subtracting nodes, there will be no rehashing caused by rematching. Consistent hashing only affects the key allocation of adjacent nodes, and the impact is small. For more introduction to consistent hashing algorithms, please refer to: http://blog.csdn.net/cywosp/article/details/23397179/

2. Virtual node

ShardedJedis will virtualize each Redis node into 160 virtual nodes based on its name for hashing. By using virtual nodes for mapping and matching, when adding or reducing Redis nodes, keys can be moved more evenly among Redis nodes, instead of only adjacent nodes being affected. As shown in the figure, Redis node 1 is virtualized into NODE1-1 and NODE1-2, which are in the hash ring. In this way, when object1 and object2 are hashed, the nearest nodes NODE1-1 and NODE1-2 are selected, and NODE1-1 and NODE1-2 are virtual nodes of the NODE node, that is, they are actually stored on the NODE node.

Adding virtual nodes can ensure balance, that is, each Redis machine stores similar data, instead of one machine storing more data and others less.

3. ShardedJedis supports keyTagPattern mode

Extract part of the keyTag for sharding, so that by naming the key reasonably, a group of related keys can be put into the same Redis nodes to avoid cross-node access. That is, the client specifies and stores the key values ​​of the same rules on the same Redis node.

When adding or reducing nodes?

Redis Sharding adopts the client Sharding method, and the Redis on the server is still a relatively independent Redis instance node. At the same time, we do not need to add additional intermediate processing components. This is a very lightweight and flexible Redis multi-instance cluster solution.

Of course, this lightweight and flexible approach must compromise other capabilities of the cluster. For example, when scaling up, when you want to add Redis nodes, even if consistent hashing is used, different keys will be distributed to different Redis nodes.

When we need to expand, add machines to the shard list. At this time, the client calculates based on the key and lands on a different machine than the original one. In this way, if a certain value is to be obtained, it may not be obtained.

For this situation, the general approach is to reload the data directly from the back-end database after it cannot be retrieved. However, sometimes, breaking through the cache layer and directly accessing the database layer will cause great harm to system access. huge pressure.

The author of Redis gave a way - presharding.

is an online expansion method. The principle is to run multiple Redis instances with different ports on each physical machine. For example, if there are three physical machines, each physical machine will run three Redis instances. , then there are actually 9 Redis instances in our sharding list. When we need to expand, add a physical machine. The steps are as follows:

1. Run Redis-server

2. The Redis-server belongs to (slaveof) a Redis-Server in the shard list (assumed to be called RedisA).

3. After the master-slave replication (Replication) is completed, change the IP and port of RedisA in the client shard list to the IP and port of Redis-Server on the new physical machine.

4. Stop RedisA

This is equivalent to transferring a certain Redis-Server to a new machine. However, it still relies heavily on the replication function of Redis itself. If the main database snapshot data file is too large, the replication process will take a long time and will also put pressure on the main Redis. Therefore, it is best to choose a low-business access method for this split process. Conducted during peak hours.

When a node fails

It is not only the addition or deletion of Redis nodes that causes key value loss. The bigger obstacle comes from the sudden downtime of the Redis node.

In order not to affect the performance of Redis, try not to enable the AOF and RDB file saving functions. Therefore, it is necessary to construct a Redis active and standby mode. If the active Redis is down, the standby Redis will have a backup, and the data will not be lost.

Sharding evolved as follows:

In this way, our architectural model becomes a Redis node slice containing a primary Redis and a standby Redis. The active and standby together form a Redis node, through automatic failover , ensuring the high availability of the node.

Redis Sentinel

Provides functions such as Redis monitoring and failover in active and backup modes to achieve high system availability.

Read and write separation

Under high access time, even if Sharding is used, a single node still bears a lot of access pressure. At this time, we need to further decompose it.

Normally, reading is often several times faster than writing. At this time, we can separate reading and writing, and reading provides more instances. The master-slave mode is used to realize the separation of reading and writing. The master is responsible for writing and the slave is responsible for reading only. At the same time, one master can connect multiple slaves. Under Redis Sentinel monitoring, automatic monitoring of node failures can also be ensured.

For more Redis-related technical articles, please visit the Introduction to Using Redis Database Tutorial column to learn!

The above is the detailed content of How to cluster redis. For more information, please follow other related articles on the PHP Chinese website!