Let's talk about transactions in Redis: transaction mode, Lua script

This article will give you a thorough understanding of Redis transactions and compare the two modes of Redis transactions (transaction mode and Lua script). I hope it will be helpful to everyone!

To be precise, Redis transactions include two modes: Transaction mode and Lua script.

Let’s talk about the conclusion first:

Redis’ transaction mode has the following characteristics:

• Isolation is guaranteed;
• Durability cannot be guaranteed;
• It has a certain degree of atomicity, but does not support rollback;
• The concept of consistency is different. It is assumed that the core of consistency is the semantics of constraints, and Redis transactions can guarantee consistency.

But Lua script has more practical scenarios. It is another form of transaction. It has a certain degree of atomicity, but when the script reports an error, the transaction will not be rolled back. Lua scripts can ensure isolation, and can perfectly support the subsequent steps depending on the results of the previous steps. [Related recommendations: Redis Video Tutorial]

Lua script mode is almost everywhere, such as distributed locks, delay queues, red envelope grabbing and other scenarios.

1 Transaction Principle

Redis transactions include the following commands:

The transaction consists of three stages:

0. Transaction is opened, using MULTI, this command marks the client executing the command switching from the non-transaction state to the transaction state;
1. Command After entering the queue, MULTI starts the transaction, the client's command will not be executed immediately, but will be put into a transaction queue;
2. Execute the transaction or discard it. If an EXEC command is received, the command in the transaction queue will be executed. If it is DISCARD, the transaction will be discarded.

The following shows an example of a transaction.

1
 redis> MULTI 
2
 OK
3
 redis> SET msg "hello world"
4
 QUEUED
5
 redis> GET msg
6
 QUEUED
7
 redis> EXEC
8
 1) OK
9
 1) hello world

Have a question here? Can the Redis key be modified when starting a transaction?

Before the transaction executes the EXEC command, the Redis key can still be modified.

Before the transaction is started, we can use the watch command to monitor the Redis key. Before the transaction is executed, we modify the key value. The transaction execution fails and nil is returned.

Through the above example, the watch command can achieve an effect similar to optimistic locking.

2 ACID of transaction

2.1 Atomicity

Atomicity refers to: a transaction All operations in are either completed or not completed, and will not end at any intermediate stage. If an error occurs during the execution of the transaction, it will be rolled back to the state before the transaction started, as if the transaction had never been executed.

First example:

Before executing the EXEC command, the operation command sent by the client is wrong, such as syntax error or a non-existent command.

1
 redis> MULTI
2
 OK
3
 redis> SET msg "other msg"
4
 QUEUED
5
 redis> wrongcommand  ### 故意写错误的命令
6
 (error) ERR unknown command 'wrongcommand' 
7
 redis> EXEC
8
 (error) EXECABORT Transaction discarded because of previous errors.
9
 redis> GET msg
10
 "hello world"

In this example, we used a command that does not exist, causing the queue to fail and the entire transaction will not be executed.

Second example:

When the transaction operation is enqueued, the data types of the command and operation do not match. The enqueue is normal, but the execution of the EXEC command is abnormal.

1
 redis> MULTI  
2
 OK
3
 redis> SET msg "other msg"
4
 QUEUED
5
 redis> SET mystring "I am a string"
6
 QUEUED
7
 redis> HMSET mystring name  "test"
8
 QUEUED
9
 redis> SET msg "after"
10
 QUEUED
11
 redis> EXEC
12
 1) OK
13
 2) OK
14
 3) (error) WRONGTYPE Operation against a key holding the wrong kind of value
15
 4) OK
16
 redis> GET msg
17
 "after"

In this example, if an error occurs when Redis executes the EXEC command, Redis will not terminate the execution of other commands, and the transaction will not be rolled back because a command fails to execute.

To sum up, my understanding of the atomicity of Redis transactions is as follows:

0. When the command is added to the queue, an error will be reported, and transaction execution will be abandoned to ensure atomicity;
1. Command It is normal when joining the queue, but an error is reported after executing the EXEC command, and atomicity is not guaranteed;

That is: Redis transactions only have certain atomicity under specific conditions.

2.2 Isolation

The isolation of the database means that the database allows multiple concurrent transactions to read, write and modify its data at the same time. Ability and isolation can prevent data inconsistency due to cross execution when multiple transactions are executed concurrently.

Transaction isolation is divided into different levels, namely:

• Read uncommitted (read uncommitted)
• Read committed (read committed)
• Repeatable read
• Serializable

First of all, it needs to be clear: Redis does not have the concept of transaction isolation level. Here we discuss the isolation of Redis: In concurrent scenarios, whether transactions can avoid interfering with each other.

We can divide transaction execution into two stages: before the EXEC command is executed and after the EXEC command is executed, and discuss them separately.

0. Before executing the EXEC command

In the section on transaction principles, we found that the Redis key can still be modified before the transaction is executed. At this time, you can use WATCH mechanism to achieve the effect of optimistic locking.

0. After the EXEC command is executed

Because Redis is a single-threaded execution command, after the EXEC command is executed, Redis will ensure that all commands in the command queue are executed. This ensures transaction isolation.

2.3 Persistence

The persistence of the database means: after the transaction is completed, the modification of the data is permanent, even if the system fails It won't be lost either.

Whether Redis data is persisted depends on the persistence configuration mode of Redis.

0. Without RDB or AOF configured, the durability of the transaction cannot be guaranteed;
1. If the RDB mode is used, after a transaction is executed and before the next RDB snapshot is executed, if If an instance crashes, the durability of the transaction cannot be guaranteed;
2. AOF mode is used; the three configuration options of AOF mode, no and everysec, will cause data loss. always can guarantee transaction durability, but because its performance is too poor, it is generally not recommended for use in production environments.

To sum up, the durability of redis transactions cannot be guaranteed.

2.4 Consistency

The concept of consistency has always been confusing. In the information I searched, there are two different definitions. .

0. Wikipedia

Let’s first look at the definition of consistency on Wikipedia:

Consistency ensures that a transaction can only bring the database from one valid state to another, maintaining database invariants: any data written to the database must be valid according to all defined rules, including constraints, cascades, triggers, and any combination thereof. This prevents database corruption by an illegal transaction, but does not guarantee that a transaction is correct. Referential integrity guarantees the primary key – foreign key relationship.

In this text, the core of consistency is "constraint", " any data written to the database must be valid according to all defined rules”.

How to understand constraints? Here is a quote from Zhihu question How to understand the internal consistency and external consistency of the database, answered by Han Fusheng, an OceanBase R&D expert at Ant Financial:

"Constraints" are determined by the database's The user tells the database that the user requires that the data must comply with this or that constraint. When the data is modified, the database will check whether the data still meets the constraints. If the constraints are no longer satisfied, the modification operation will not occur.

The two most common types of constraints in relational databases are "unique constraints" and "integrity constraints". The primary keys and unique keys defined in the table ensure that the specified data items will never be repeated. Inter-defined referential integrity also ensures the consistency of the same attribute in different tables.

"Consistency in ACID" is so easy to use that it has melted into the blood of most users. Users will consciously add the required constraints when designing tables, and the database will also This constraint will be strictly enforced.

So the consistency of the transaction is related to the pre-defined constraints, ensuring the constraints ensures the consistency .

Let’s take a closer look at this sentence: This prevents database corruption by an illegal transaction, but does not guarantee that a transaction is correct.

Maybe everyone is still a little confused after writing this. Let’s take the classic transfer case.

We start a transaction. The initial balances on Zhang San’s and Li Si’s accounts are both 1,000 yuan, and there are no constraints on the balance field. Zhang San transferred 1,200 yuan to Li Si. Zhang San’s balance is updated to -200, and Li Si’s balance is updated to 2200.

From the application level, this transaction is obviously illegal, because in real scenarios, the user balance cannot be less than 0, but it completely follows the constraints of the database, so from the database level, this transaction is still guaranteed consistency.

The transaction consistency of Redis means that the Redis transaction complies with the constraints of the database during execution and does not contain illegal or invalid error data.

We discuss three exception scenarios respectively:

0. Before executing the EXEC command, the operation command sent by the client is wrong, the transaction is terminated, and the data remains consistent;

1. After executing the EXEC command, the data types of the command and operation do not match. An error will be reported for the wrong command, but the transaction will not be terminated due to the wrong command, but will continue to be executed. The correct command is executed normally, and the wrong command reports an error. From this perspective, the data can also maintain consistency;

2. During the execution of the transaction, the Redis service goes down. Here you need to consider the persistence mode of the service configuration.

   • Memory mode without persistence: After the service restarts, the database does not maintain data, so the data remains consistent;
   • RDB / AOF mode: After the service restarts, Redis Restore data via RDB/AOF files and the database is restored to a consistent state.

To sum up, Under the semantics that the core of consistency is constraint, Redis transactions can guarantee consistency.

0. "Designing Data-Intensive Applications"

This book is a magic book for getting started with distributed systems. There is an explanation about ACID in the transaction chapter:

Atomicity, isolation, and durability are properties of the database,whereas consistency (in the ACID sense ) is a property of the application. The application may rely on the database's atomicity and isolation properties in order to achieve consistency, but it's not up to the database alone. Thus, the letter C doesn't really belong in ACID.

Atomicity, isolation, and durability are properties of the database, while consistency (in the ACID sense) is a property of the application. Applications may rely on the atomicity and isolation properties of the database to achieve consistency, but this does not depend solely on the database. Therefore, the letter C does not belong to ACID.

Many times, the consistency we have been struggling with actually refers to consistency in line with the real world. Consistency in the real world is the ultimate goal of affairs.

In order to achieve consistency in the real world, the following points need to be met:

0. Guarantee atomicity, durability and isolation. If these characteristics cannot be guaranteed, then the consistency of the transaction cannot be guaranteed;
1. The constraints of the database itself, such as the string length cannot exceed the column restrictions or uniqueness constraints;
2. The business level also needs to be guaranteed.

2.5 Transaction Characteristics

We usually call Redis an in-memory database, which is different from traditional relational databases. In order to provide higher Performance, faster writing speed, some balances have been made at the design and implementation levels, and it cannot fully support ACID of transactions.

Redis transactions have the following characteristics:

• ensures isolation;
• cannot guarantee durability;
• has a certain degree of atomicity. But rollback is not supported;
• The concept of consistency is different. It is assumed that Redis transactions can guarantee consistency under the semantics that the core of consistency is constraint.

From an engineering perspective, assuming that each step in a transaction operation needs to rely on the result returned by the previous step, optimistic locking needs to be implemented through watch.

3 Lua Script

3.1 Introduction

Lua is written in standard C. The code is simple and beautiful, and can be compiled and run on almost all operating systems and platforms. Lua scripts can be easily called by C/C code, and can also call C/C functions in turn, which makes Lua widely used in applications.

Lua scripts have made a big splash in the game field. The well-known "Westward Journey II" and "World of Warcraft" both use Lua scripts extensively. Lua scripts can be seen in api gateways that Java back-end engineers have come into contact with, such as Openresty and Kong.

Starting from Redis version 2.6.0, Redis’s built-in Lua interpreter can run Lua scripts in Redis.

Benefits of using Lua scripts:

• Reduce network overhead. Send multiple requests at once in the form of a script to reduce network latency.
• Atomic operations. Redis will execute the entire script as a whole, and no other commands will be inserted in the middle.
• Reuse. The script sent by the client will be permanently stored in Redis, and other clients can reuse this script without using code to complete the same logic.

Common commands for Redis Lua scripts:

3.2 EVAL 命令

命令格式：

1
 EVAL script numkeys key [key ...] arg [arg ...]

说明：

• script是第一个参数，为 Lua 5.1脚本；
• 第二个参数numkeys指定后续参数有几个 key；
• key [key ...]，是要操作的键，可以指定多个，在 Lua 脚本中通过KEYS[1], KEYS[2]获取；
• arg [arg ...]，参数，在 Lua 脚本中通过ARGV[1], ARGV[2]获取。

简单实例：

1
 redis> eval "return ARGV[1]" 0 100 
2
 "100"
3
 redis> eval "return {ARGV[1],ARGV[2]}" 0 100 101
4
 1) "100"
5
 2) "101"
6
 redis> eval "return {KEYS[1],KEYS[2],ARGV[1]}" 2 key1 key2 first second
7
 1) "key1"
8
 2) "key2"
9
 3) "first"
10
 4) "second"

下面演示下 Lua 如何调用 Redis 命令 ，通过redis.call()来执行了 Redis 命令 。

1
 redis> set mystring 'hello world'
2
 OK
3
 redis> get mystring
4
 "hello world"
5
 redis> EVAL "return redis.call('GET',KEYS[1])" 1 mystring
6
 "hello world"
7
 redis> EVAL "return redis.call('GET','mystring')" 0
8
 "hello world"

3.3 EVALSHA 命令

使用 EVAL 命令每次请求都需要传输 Lua 脚本 ，若 Lua 脚本过长，不仅会消耗网络带宽，而且也会对 Redis 的性能造成一定的影响。

思路是先将 Lua 脚本先缓存起来 , 返回给客户端 Lua 脚本的 sha1 摘要。 客户端存储脚本的 sha1 摘要 ，每次请求执行 EVALSHA 命令即可。

EVALSHA 命令基本语法如下：

1
 redis> EVALSHA sha1 numkeys key [key ...] arg [arg ...]

实例如下：

1
 redis> SCRIPT LOAD "return 'hello world'"
2
 "5332031c6b470dc5a0dd9b4bf2030dea6d65de91"
3
 redis> EVALSHA 5332031c6b470dc5a0dd9b4bf2030dea6d65de91 0
4
 "hello world"

4 事务 VS Lua 脚本

从定义上来说， Redis 中的脚本本身就是一种事务， 所以任何在事务里可以完成的事， 在脚本里面也能完成。 并且一般来说， 使用脚本要来得更简单，并且速度更快。

因为脚本功能是 Redis 2.6 才引入的， 而事务功能则更早之前就存在了， 所以 Redis 才会同时存在两种处理事务的方法。

不过我们并不打算在短时间内就移除事务功能， 因为事务提供了一种即使不使用脚本， 也可以避免竞争条件的方法， 而且事务本身的实现并不复杂。

-- redis.io/

Lua 脚本是另一种形式的事务，他具备一定的原子性，但脚本报错的情况下，事务并不会回滚。Lua 脚本可以保证隔离性，而且可以完美的支持后面的步骤依赖前面步骤的结果。

Lua 脚本模式的身影几乎无处不在，比如分布式锁、延迟队列、抢红包等场景。

不过在编写 Lua 脚本时，要注意如下两点：

0. 为了避免 Redis 阻塞，Lua 脚本业务逻辑不能过于复杂和耗时；
1. 仔细检查和测试 Lua 脚本 ，因为执行 Lua 脚本具备一定的原子性，不支持回滚。

更多编程相关知识，请访问：编程视频！！

The above is the detailed content of Let's talk about transactions in Redis: transaction mode, Lua script. For more information, please follow other related articles on the PHP Chinese website!