Detailed explanation of pipeline in redis

1. The background of pipeline appearance:

Redis has four processes to execute a command: sending command, command queuing, command execution, and returning results;

This process is called Round trip time (RTT for short, round trip time). mget mset effectively saves RTT, but most commands (such as hgetall, but not mhgetall) do not support batch operations and need to consume N times of RTT. At this time, a pipeline is needed to solve this problem.

2. Performance of pipeline

1. No pipeline is used to execute N commands

2. Pipeline is used to execute N commands

3. Performance comparison between the two

Summary: This is a set of statistical data, using Pipeline The execution speed is faster than the execution one by one, especially the greater the network delay between the client and the server, the more obvious the performance will be.

The test code is posted below to analyze the performance difference between the two:

	@Test
	public void pipeCompare() {
		Jedis redis = new Jedis("192.168.1.111", 6379);
		redis.auth("12345678");//授权密码 对应redis.conf的requirepass密码
		Map<String, String> data = new HashMap<String, String>();
		redis.select(8);//使用第8个库
		redis.flushDB();//清空第8个库所有数据
		// hmset
		long start = System.currentTimeMillis();
		// 直接hmset
		for (int i = 0; i < 10000; i++) {
			data.clear();  //清空map
			data.put("k_" + i, "v_" + i);
			redis.hmset("key_" + i, data); //循环执行10000条数据插入redis
		}
		long end = System.currentTimeMillis();
		System.out.println("    共插入:[" + redis.dbSize() + "]条 .. ");
		System.out.println("1,未使用PIPE批量设值耗时" + (end - start) / 1000 + "秒..");
		redis.select(8);
		redis.flushDB();
		// 使用pipeline hmset
		Pipeline pipe = redis.pipelined();
		start = System.currentTimeMillis();
		//
		for (int i = 0; i < 10000; i++) {
			data.clear();
			data.put("k_" + i, "v_" + i);
			pipe.hmset("key_" + i, data); //将值封装到PIPE对象，此时并未执行，还停留在客户端
		}
		pipe.sync(); //将封装后的PIPE一次性发给redis
		end = System.currentTimeMillis();
		System.out.println("    PIPE共插入:[" + redis.dbSize() + "]条 .. ");
		System.out.println("2,使用PIPE批量设值耗时" + (end - start) / 1000 + "秒 ..");
//--------------------------------------------------------------------------------------------------
		// hmget
		Set<String> keys = redis.keys("key_*"); //将上面设值所有结果键查询出来
		// 直接使用Jedis hgetall
		start = System.currentTimeMillis();
		Map<String, Map<String, String>> result = new HashMap<String, Map<String, String>>();
		for (String key : keys) {
			//此处keys根据以上的设值结果，共有10000个，循环10000次
			result.put(key, redis.hgetAll(key)); //使用redis对象根据键值去取值，将结果放入result对象
		}
		end = System.currentTimeMillis();
		System.out.println("    共取值:[" + redis.dbSize() + "]条 .. ");
		System.out.println("3,未使用PIPE批量取值耗时 " + (end - start) / 1000 + "秒 ..");

		// 使用pipeline hgetall
		result.clear();
		start = System.currentTimeMillis();
		for (String key : keys) {
			pipe.hgetAll(key); //使用PIPE封装需要取值的key,此时还停留在客户端，并未真正执行查询请求
		}
		pipe.sync();  //提交到redis进行查询
		
		end = System.currentTimeMillis();
		System.out.println("    PIPE共取值:[" + redis.dbSize() + "]条 .. ");
		System.out.println("4,使用PIPE批量取值耗时" + (end - start) / 1000 + "秒 ..");

		redis.disconnect();
	}

3. Comparison between native batch commands (mset, mget) and Pipeline

1. The native batch command is atomic, and the pipeline is non-atomic

(The concept of atomicity: a transaction is an indivisible minimum unit of work, either all succeed or all fail. Atomic operations refer to your A business logic must be indivisible. Processing one thing either succeeds or fails, and atoms cannot be split)

2. The native batch command commands multiple keys, but the pipeline supports multiple commands. (Transaction exists), non-atomic

3. The native batch command is implemented on the server side, while the pipeline needs to be completed by the server and the client together

4. The correct way to use Pipeline

The number of commands assembled using pipeline cannot be too large, otherwise the amount of data will be too large, which will increase the waiting time of the client and may cause network congestion. A large number of commands can be split into multiple small pipeline commands to complete.

1. How to use pipeline in Jedis

As we all know, redis provides mset and mget methods, but does not provide mdel method. If you want to achieve it, you can use pipeline to achieve it.

2. Steps to use pipeline in Jedis:

• Obtain the jedis object (usually obtained from the connection pool)

• Get the pipeline object of the jedis object

• Add instructions

• Execute instructions

Test class method:

	 @Test
	public void testCommond() {
		// 工具类初始化
		JedisUtils jedis = new JedisUtils("192.168.1.111", 6379, "12345678");

		for (int i = 0; i < 100; i++) {
			// 设值
			jedis.set("n" + i, String.valueOf(i));
		}
		System.out.println("keys from redis return =======" + jedis.keys("*"));

	}

	// 使用pipeline批量删除
	 @Test
	public void testPipelineMdel() {
		// 工具类初始化
		JedisUtils jedis = new JedisUtils("192.168.1.111", 6379, "12345678");
		List<String> keys = new ArrayList<String>();
		for (int i = 0; i < 100; i++) {
			keys.add("n" + i);
		}
		jedis.mdel(keys);
		System.out.println("after mdel the redis return ---------" + jedis.keys("*"));
	}

mdel method under JedisUtils:

	/**
	 * 删除多个字符串key 并释放连接
	 * 
	 * @param keys*
	 * @return 成功返回value 失败返回null
	 */
	public boolean mdel(List<String> keys) {
		Jedis jedis = null;
		boolean flag = false;
		try {
			jedis = pool.getResource();//从连接借用Jedis对象
			Pipeline pipe = jedis.pipelined();//获取jedis对象的pipeline对象
			for(String key:keys){
				pipe.del(key); //将多个key放入pipe删除指令中
			}
			pipe.sync(); //执行命令，完全此时pipeline对象的远程调用 
			flag = true;
		} catch (Exception e) {
			pool.returnBrokenResource(jedis);
			e.printStackTrace();
		} finally {
			returnResource(pool, jedis);
		}
		return flag;
	}

Use pipeline to submit all operations and return execution results:

@Test
	public void testPipelineSyncAll() {
		// 工具类初始化
		Jedis jedis = new Jedis("192.168.1.111", 6379);
		jedis.auth("12345678");
		// 获取pipeline对象
		Pipeline pipe = jedis.pipelined();
		pipe.multi();
		pipe.set("name", "james"); // 调值
		pipe.incr("age");// 自增
		pipe.get("name");
		pipe.discard();
		// 将不同类型的操作命令合并提交，并将操作操作以list返回
		List<Object> list = pipe.syncAndReturnAll();

		for (Object obj : list) {
			// 将操作结果打印出来
			System.out.println(obj);
		}
		// 断开连接，释放资源
		jedis.disconnect();
	}

5. redis transaction

Pipeline is multiple A combination of commands, in order to ensure its atomicity, redis provides simple transactions.

1. Simple transaction of redis,

A group of commands that need to be executed together are placed between the multi and exec commands, where multi represents the start of the transaction and exec represents the end of the transaction.

2. Stop the transaction discard

3. The command is wrong and the syntax is incorrect, causing the transaction to not end normally

4. Running error, the syntax is correct, but the type is wrong, the transaction can end normally

5. watch command:

After using watch, multi fails and the transaction fails

The mechanism of WATCH is: when the transaction EXEC command is executed, Redis will check the WATCH key. EXEC will be executed only if the key being WATCH has not changed since the start of WATCH. If the WATCH key changes between the WATCH command and the EXEC command, the EXEC command will return failure.

For more redis knowledge, please pay attention to the redis introductory tutorial column.

The above is the detailed content of Detailed explanation of pipeline in redis. For more information, please follow other related articles on the PHP Chinese website!