What is the method of designing the Pika architecture of the Redis storage system?

Pika is an efficient, stable, simple and reliable open source NoSQL database product jointly developed by the 360 ​​infrastructure team and the DBA team. It is fully compatible with the Redis protocol and supports 5 data structures (string, hash, list, set, zset). The data is persisted to RocksDB. Compared with the Redis memory storage method, it can greatly reduce the occupation of server resources and enhance the data security. reliability. It can be deployed in two modes: stand-alone and cluster. The Pika project was launched in 2015 and was subsequently open sourced on Github. It currently has 3,700 stars and 35 contributors. The community has a large number of online businesses using Pika.

Compare Redis

Storage capacity: Redis stores in memory, with high hardware costs and high downtime recovery delays; Pika borrows RocksDB to store to disk, and the amount of data accommodated by a single server is Redis. Dozens of times faster recovery from downtime.

Throughput: Redis QPS is higher, a single server has a million-level QPS; Pika QPS is relatively low, a single server has hundreds of thousands, and Redis is 3 to 5 times that of Pika.

Access latency: Redis should be within 1ms; Pika latency is slightly higher, within 3ms.

Operation and maintenance deployment: Redis supports two methods: stand-alone master-slave and cluster; Pika also supports two methods of deployment.

Applicable scenarios

If the business scenario data volume is relatively large (> 50GB) and the data reliability requirements are high, then Pika can solve your problem.

Scenario 1: Intermediate result storage for large-scale data processing systems

Scenario 2: Business systems using Redis/Redis Cluster for persistent storage

Scenario 3: Large-scale distribution Metadata storage of the traditional system

Architecture design

Pika can choose to run classic mode (Classic) or distributed by setting the instance-mode configuration item in the configuration file to classic and sharding. Sharding of Pika.

• Classic mode architecture

##Classic mode (Classic): that is, 1 master N slave synchronization mode, One master instance stores all data, and N slave instances completely mirror and synchronize the data of the master instance. Each instance supports multiple DBs. Pika's configuration item databases allows you to set the maximum number of DBs that can be created, starting from 0 by default. The physical form of DB on Pika is a file directory.

• Distributed mode architecture

##Distributed mode (Sharding): In Sharding mode, users The stored data set is called a Table. Each Table is divided into multiple shards, and each shard is called a Slot. The data of a certain KEY is calculated by a hash algorithm to determine which Slot it belongs to. Distribute all Slots and their copies to all Pika instances according to a certain strategy. Each Pika instance has a part of the master Slot and a part of the slave Slot. In Sharding mode, Slot is used to divide master and slave, and Pika instances are no longer used. The physical form of slot on Pika is a file directory.

The above is the detailed content of What is the method of designing the Pika architecture of the Redis storage system?. For more information, please follow other related articles on the PHP Chinese website!