Redis sharding (distributed cache)

Partitioning is the process of splitting your data into multiple Redis instances, so that each instance will only contain a subset of all keys. (Related recommendations: Redis Tutorial)

1 What is the use of sharding

Redis’ sharding has two main goals:

• Allows the combined memory of many computers to be used to support larger databases. Without sharding, you are limited to the amount of memory a single machine can support.

• Allows scaling computing power to multiple cores or multiple servers, and network bandwidth to multiple servers or multiple network adapters.

2 Sharding Basics

There are many different sharding criteria (criteria).

Suppose we have 4 Redis instances R0, R1, R2, R3, and many more keys representing users, like user:1, user:2, ... etc. We can find different ways to choose which instance a specific key is stored in. In other words, there are many different ways to map a key to a specific Redis server.

One of the simplest ways to perform sharding is range partitioning, which completes sharding by mapping the range of an object to a specified Redis instance. For example, I can assume that the user enters instance R0 from ID 0 to ID 10000, and the user enters instance R1 from ID 10001 to ID 20000.

This approach works and is actually used in practice, however , this has the disadvantage that it requires a table that maps ranges to instances.

This table needs to be managed, and different types of objects require a table, so range sharding is often not advisable in Redis because This is much less efficient than other sharding alternatives.

An alternative to range sharding is hash partitioning.

This mode works with any key, and does not require the key to be in the form of object_name:, just It's as simple as this

• Use a hash function (for example, the crc32 hash function) to convert the key name to a number. For example, if the key is foobar, crc32(foobar) will output something like 93024922.

• Modulo this data to convert it to a number between 0 and 3 so that the number can be mapped to one of my 4 Redis instances. 93024922 modulo 4 equals 2, so I know my key foobar should be stored to the R2 instance. Note: The modulo operation returns the remainder of the division operation, which is always implemented as the % operator in many programming languages.

There are many other ways to shard, as you can see from these two examples. An advanced form of hash sharding is called consistent hashing and is implemented by some Redis clients and brokers.

3 Sharding Implementation (Theory)

Sharding can be undertaken by different parts of the software stack.

•Client side partitioning

The client directly selects the correct node to write and read the specified key. Many Redis clients implement client side partitioning.

• Proxy assisted partitioning

Our client sends the request to a proxy that can understand the Redis protocol. Instead of sending the request directly to the Redis instance.

The proxy will ensure that our requests are forwarded to the correct Redis instance according to the configured sharding mode, and return a response to the client.

Redis and Memcached's proxy Twemproxy implements proxy-assisted splitting Piece.

• Query routing

You can send your query to a random instance, and this instance will guarantee to forward your query to the correct node.

Redis Cluster implements a hybrid form of query routing with the help of the client (the request is not forwarded directly from the Redis instance to another, but the client receives a redirect to the correct node).

4 Disadvantages of sharding

Some features of Redis do not play well with sharding

• Operations involving multiple keys are generally not supported. For example, you cannot perform an intersection on keys mapped on two different Redis instances (in fact there is a way to do it, but not directly).

• Transactions involving multiple keys cannot use

• The granularity of sharding is the key, so you cannot use a large key to shard the data set, such as a large ordered set

• When sharding is used, the data The processing becomes more complex, for example, you need to process multiple RDB/AOF files, and when backing up data you need to aggregate persistent files from multiple instances and hosts

• Adding and removing capacity is also complex. For example, Redis Cluster has the ability to dynamically add and remove nodes at runtime to support transparent rebalancing of data, but other methods, such as client-side sharding and proxies, do not support this feature. However, there is a technology called presharding that can help at this point.

5 Storage OR Cache

Although Redis's sharding concept is the same whether you use Redis as a data store or cache, there is an important limitation when used as a data store. When Redis is used as a data store, a given key is always mapped to the same Redis instance. When Redis is used as a cache, it is not a big problem if one node is unavailable and another node is used. Changing the mapping of keys and instances according to our wishes improves the availability of the system (that is, the system's ability to answer our queries) .

Consistent hashing implementations are often able to switch to other nodes if the preferred node for a given key is unavailable. Similarly, if you add a new node, some data will start to be stored in this new node.

The main concepts here are as follows:

• If Redis is used as a cache, it is easy to use consistent hashing to achieve scaling up and down.

• If Redis is used as storage, a fixed key-to-node mapping is used, so the number of nodes must be fixed and cannot be changed. Otherwise, when adding or deleting nodes, you need a system that supports rebalancing keys between nodes. Currently, only Redis cluster can do this.

6 Pre-sharding

We already know a problem with sharding. Unless we use Redis as a cache, adding and removing nodes is a tricky thing. It is much simpler to use fixed key and instance mapping.

However, data storage needs may be changing all the time. Today I can live with 10 Redis nodes (instances), but tomorrow I may need 50 nodes.

Because Redis has a fairly small memory footprint and is lightweight (an idle instance only uses 1MB of memory), a simple solution is to start many instances from the beginning. Even if you start with just one server, you can decide on day one to live in a distributed world and use sharding to run multiple Redis instances on a single server.

You can choose a large number of instances from the beginning. For example, 32 or 64 instances will satisfy most users and provide enough room for future growth.

This way, when your data storage needs to grow and you need more Redis servers, all you have to do is simply move the instance from one server to another. When you add the first server, you need to move half of the Redis instances from the first server to the second, and so on.

Using Redis replication, you can move data with little or no downtime:

• Start an empty instance on your new server.

• Move data and configure the new instance as a slave service of the source instance.

• Stop your client.

• Update the server IP address configuration of the moved instance.

• Send the SLAVEOF NO ONE command to the slave node on the new server.

• Start your client with the new updated configuration.

• Finally, close the instances on the old server that are no longer in use.

7 Sharding Implementation (Practice)

So far, we have discussed Redis sharding in theory, but what is the practical situation? What system should you use?

7.1 Redis Cluster

Redis Cluster is the preferred method for automatic sharding and high availability.

Once Redis Cluster is available, and supports Redis Cluster Once the client is available, Redis Cluster will become the de facto standard for Redis sharding.

Redis cluster is a hybrid model of query routing and client sharding.

7.2 Twemproxy

Twemproxy is a proxy developed by Twitter that supports Memcached ASCII and Redis protocols. It is single-threaded, written in C, and runs very fast. Open source project under the Apache 2.0 license.

Twemproxy supports automatic sharding across multiple Redis instances, and optional node exclusion support if the node is unavailable (this will change the mapping of keys to instances, so you should only use Redis as a cache Only use this feature).

This is not a single point of failure because you can start multiple proxies and have your clients connect to the first proxy that accepts the connection.

Fundamentally, Twemproxy is a middle layer between the client and the Redis instance, allowing us to reliably handle our shards with minimal additional complexity. This is the currently recommended way of handling Redis sharding.

7.3 Clients that support consistent hashing

An alternative to Twemproxy is to use the implementation Clients with client-side sharding, through consistent hashing or other similar algorithms. There are several Redis clients that support consistent hashing, such as redis-rb and Predis.

Please check the complete list of Redis clients to see if there is a mature client that supports your programming language and implements consistent hashing.

The above is the detailed content of Redis sharding (distributed cache). For more information, please follow other related articles on the PHP Chinese website!