1. Written before
After graduation and working, I finally got to go home today. I thought back on some of the content I have done in the past six months, and I always feel that it is still at such a basic level, and I am still solving various problems. In the process of solving problems, I have fewer innovative ideas and rely more on searching. I don’t want to make a summary of 2016. I hope that in 2017 I can learn more technologies I love, read more open source code, learn more design ideas and coding ideas, and update my understanding of code in the past two years. .
This sharing is mainly to make up for the shortcomings of my previous RedisRepository.
Half a year ago, due to insufficient reading of the StackExchange.Redis document, the RedisRepository I shared was wrong. Here are my main mistakes:
Error 1, there is no singleton ConnectionMultiplexer Redis connection object, and I naively thought that locking the singleton object would limit the performance of Redis in concurrent situations.
Error 2. In the master-slave situation, I thought that when manual switching occurs, we need to subscribe to the switching event and dynamically change the Endpoint pointed to by the connection object after the event occurs.
When I read the document carefully again, I realized my mistake. This was a late correction, but I felt that my own repository still had many shortcomings, so I really needed an experienced driver. guidance and suggestions.
Correction 1. The cost of creating a Redis connection object is very high, and singleton locking will not affect Redis performance because the connection object is not waiting during the network request.
Correction 2, when Redis is master-slave, after the sentinel switches the master-slave relationship, StackExchange.Redis will identify the new master-slave for us without any operation on our part.
So far I still have two questions.
Question 1, there is no clear result after reading the document. When the master-slave read-write separation is performed, when we add multiple nodes to the Endpoint Collection, the read-write separation will be automatically performed? Or do we need to specify CommandFlags.PreferSlave in the method of reading the command? I think it’s the latter? So I specify PreferSlave in all my read methods. What do the veteran drivers say?
Question 2, I use LuaScript.Prepare(lua) and then load it. Executing lua always has no effect, and LuaScript.GetCachedScriptCount() is 0. However, if I use ScriptEvaluateAsync directly, it is easy to use. If the driver has a good example, I hope the experienced driver can give some guidance or share it.
2. Code structure, for reference only
The structure is roughly like this. All classes under RedisAsyncHelper are partial classes, and their class name is RedisHelper . They jointly implemented the IRedisHelper interface and left detailed comments.
The directory structure of the synchronous version and the asynchronous version is the same.
3. Preparatory phase
Two helper classes in CommonHelper:
RedisInnerTypeHelper.cs
using StackExchange.Redis; using System.Collections.Generic; using System.Linq; namespace Fantasy.RedisRepository.CommonHelper { internal class RedisInnerTypeHelper { public static ListRedisValuesToGenericList (RedisValue[] redisValues) { var result = new List (); redisValues.ToList().ForEach(r => result.Add(SerializeHelper.Deserialize (r))); return result; } public static RedisValue[] GenericListToRedisValues (List values) { var redisValues = new List (); values.ForEach(v => redisValues.Add(SerializeHelper.Serialize(values))); return redisValues.ToArray(); } public static RedisKey[] GenericListToRedisKeys(List keys) { var redisKeys = new List (); keys.ForEach(k => redisKeys.Add(k)); return redisKeys.ToArray(); } } }
SerializeHelper.cs
configuration inusing System.IO; using System.Runtime.Serialization.Formatters.Binary; namespace Fantasy.RedisRepository.CommonHelper { internal static class SerializeHelper { ////// 字节数组序列化 /// /// ///internal static byte[] Serialize(object o) { if (o == null) { return null; } BinaryFormatter binaryFormatter = new BinaryFormatter(); using (MemoryStream memoryStream = new MemoryStream()) { binaryFormatter.Serialize(memoryStream, o); byte[] objectDataAsStream = memoryStream.ToArray(); return objectDataAsStream; } } /// /// 字节数组反序列化 /// ////// /// internal static T Deserialize (byte[] stream) { if (stream == null) { return default(T); } BinaryFormatter binaryFormatter = new BinaryFormatter(); using (MemoryStream memoryStream = new MemoryStream(stream)) { T result = (T)binaryFormatter.Deserialize(memoryStream); return result; } } } }
using System; using System.Configuration; namespace Fantasy.RedisRepository.Config { internal class ConfigHelper { internal static T Get(string appSettingsKey, T defaultValue) { string text = ConfigurationManager.AppSettings[appSettingsKey]; if (string.IsNullOrWhiteSpace(text)) return defaultValue; try { var value = Convert.ChangeType(text, typeof(T)); return (T)value; } catch { return defaultValue; } } } }
RedisClientConfig.cs
namespace Fantasy.RedisRepository.Config { internal class RedisClientConfig { private static string _server = ConfigHelper.Get("RedisServer", "115.xx.xx.31"); ////// 节点IP /// public static string Server { get { return _server; } set { _server = value; } } private static int _port = ConfigHelper.Get("RedisPort", 6380); ////// 节点端口 /// public static int Port { get { return _port; } set { _port = value; } } private static string _slaveServer = ConfigHelper.Get("SlaveServer", "115.xx.xx.31"); ////// 节点IP /// public static string SlaveServer { get { return _slaveServer; } set { _slaveServer = value; } } private static int _slavePort = ConfigHelper.Get("SlavePort", 6381); ////// 节点端口 /// public static int SlavePort { get { return _slavePort; } set { _slavePort = value; } } private static string _auth = ConfigHelper.Get("RedisAuth", "fantasy.."); ////// 节点密码 /// public static string RedisAuth { get { return _auth; } set { _auth = value; } } private static int _defaultDatabase = ConfigHelper.Get("RedisDataBase", 0); ////// redis默认0号库 /// public static int DefaultDatabase { get { return _defaultDatabase; } set { _defaultDatabase = value; } } private static int _connectTimeout = 10000; public static int ConnectTimeout { get { return _connectTimeout; } set { _connectTimeout = value; } } private static int _connectRetry = 3; public static int ConnectRetry { get { return _connectRetry; } set { _connectRetry = value; } } private static bool _preserveAsyncOrder = false; public static bool PreserveAsyncOrder { get { return _preserveAsyncOrder; } set { _preserveAsyncOrder = value; } } } }
RedisConnection.cs
using Fantasy.RedisRepository.Config; using StackExchange.Redis; namespace Fantasy.RedisRepository { ////// Redis连接类 /// public static class RedisConnection { private static ConnectionMultiplexer _connection; private static readonly object SyncObject = new object(); ////// redis连接对象,单例加锁不影响性能 /// public static ConnectionMultiplexer GenerateConnection { get { if (_connection == null || !_connection.IsConnected) { lock (SyncObject) { var configurationOptions = new ConfigurationOptions() { Password = RedisClientConfig.RedisAuth, EndPoints = { {RedisClientConfig.Server, RedisClientConfig.Port}, {RedisClientConfig.SlaveServer, RedisClientConfig.SlavePort} } }; _connection = ConnectionMultiplexer.Connect(configurationOptions); } } return _connection; } } } }
4. RedisHelper
is actually just a layer serialization wrapper .
IRedisHelper:
using System; using StackExchange.Redis; using System.Collections.Generic; using System.Threading.Tasks; namespace Fantasy.RedisRepository.RedisHelpers { ////// 异步方法接口 --Author 吴双 www.cnblogs.com/tdws /// 存入数据均为方法内部序列化后的byte,所以取数据的时候需要反序列化时,请指定正确的数据类型 /// public partial interface IRedisHelper { #region Redis数据类型—String ////// 将任何数据以redis string存储 /// ////// /// /// /// Task StringSetAsync (string key, T value, TimeSpan? timeout = null); /// /// 对数值进行减法操作,默认-1 /// /// /// ///操作后的结果 TaskStringDecrementAsync(string key, long value = 1L); /// /// 对数值进行加法操作,默认+1 /// /// /// ///操作后的结果 TaskStringIncrementAsync(string key, long value = 1L); /// /// 从redis string中以指定类型取出 /// ////// /// Task StringGetAsync (string key); #endregion #region Redis数据类型—Hash /// /// 向Hash key中存储任意类型任意值 /// ////// /// /// /// 是否成功 TaskHashSetAsync (string key, string field, T value); /// /// 批量 向Hash key中存储任意类型任意值 /// ////// /// /// 无返回值 Task HashMultiSetAsync(string key, Dictionary hashFields); /// /// 对指定hash key中制定field做数量增加操作 默认自增1 /// 如果此操作前key不存在 则创建。 如果此操作前该field不存在或者非数字 则先被置0,再被继续操作 /// /// /// /// ///操作后的结果 TaskHashIncrementAsync(string key, string field, long incrCount = 1); /// /// 对指定hash key中制定field做数量增加操作 默认自减1 /// 如果此操作前key不存在 则创建。 如果此操作前该field不存在或者非数字 则先被置0,再被继续操作 /// /// /// /// ///操作后的结果 TaskHashDecrementAsync(string key, string field, long decrCount = 1); /// /// 从指定Hash中 删除指定field /// 如果key或者field不存在,则false /// /// /// ///是否成功 TaskHashDeleteFieldAsync(string key, string field); /// /// 从指定Hash key中 批量删除指定field /// 如果key或者field不存在,则false /// /// /// ///移除数量 TaskHashMultiDeleteFieldAsync(string key, List fields); /// /// 从指定Hash key中获取指定field值 /// ////// /// /// Task HashGetAsync (string key, string field); /// /// 从指定Hash key中判断field是否存在 /// /// /// ///Task HashFieldExistAsync(string key, string field); /// /// 获取指定Hash key中的所有field的值 /// ////// /// Task > HashValuesAsync
(string key); /// /// 获取指定Hash key中所有 field名称及其Value /// ////// /// Task > HashGetAllAsync (string key); /// /// 获取指定Hash key中所有field /// /// ///Task > HashFieldsAsync(string key); #endregion #region Redis数据类型—List ///
/// 在指定pivot后插入value, 如果pivot不存在,则返回-1, 如果key不存在,则返回0 /// 如果存在多个相同指定的的pivot,则插入第一个指定pivot后面. /// 即链表从左向右查找,遇到指定pivot,则确定位置 /// ////// /// list中的一个值 /// /// Task ListInsertAfterAsync (string key, string pivot, T value); /// /// 在指定pivot前插入value, 如果pivot不存在,则返回-1, 如果key不存在,则返回0 /// 如果存在多个相同指定的的pivot,则插入第一个指定pivot前面. /// 即链表从左向右查找,遇到指定pivot,则确定位置 /// ////// /// /// /// Task ListInsertBeforeAsync (string key, string pivot, T value); /// /// 从链表左侧弹出第一个元素(弹出能获取到该元素并且被删除) /// 如果key不存在 或者链表为空 则为null /// ////// /// Task ListLeftPopAsync (string key); /// /// 从链表左侧增加一个元素,key不存在则被创建 /// ////// /// /// 返回操作后的链表长度 TaskListLeftPushAsync (string key, T value); /// /// 从链表左侧批量增加元素,如果 a b c 则c会在链表左侧第一位 b第二位 a第三位 /// ////// /// /// 返回操作后的链表长度 TaskListLeftMultiPushAsync (string key, List values); /// /// 获取链表长度,不存在key则为0 /// ////// /// Task ListLengthAsync (string key); /// /// 获取链表中所有数据,从左侧start开始到stop结束,从0—-1则认为获取全部,默认获取全部 /// start为负数则代表从链表右侧开始,-1为右侧第一位,-2为右侧第二位 /// start要小于stop,否则返回null /// ////// /// /// /// Task > ListRangeAsync
(string key, long start = 0L, long stop = -1L); /// /// 从链表中一处count数量的value. count大于0则从左至右,count小于0则从右至左,count=0则移除全部 /// ////// /// /// /// Task ListRemoveAsync (string key, T value, long count = 0L); /// /// 从右侧弹出第一个元素(弹出能获取到该元素并且被删除) /// ////// /// Task ListRightPopAsync (string key); /// /// 从链表右侧加入元素,如果 rpush a b c 则c为右侧第一位 b第二位 c第三位 /// ////// /// /// Task ListRightPushAsync (string key, T value); /// /// 从右侧批量插入,和左侧相反 /// ////// /// /// Task ListRightMultiPushAsync (string key, List values); /// /// 在链表指定索引处,插入元素 /// 正数索引从0开始,代表左侧。负数从-1开始 代表从右侧。-1为右侧第一位 /// ////// /// /// /// Task ListSetByIndexAsync (string key, int index, T value); /// /// 留下start到stop之间的数据。负数代表从右侧寻找 -1为右侧第一位 /// /// /// /// ///Task ListTrimAsync(string key, long start, long stop); /// /// 获取指定index的值,负数代表从右侧寻找 -1为右侧第一位 /// ////// /// /// Task ListGetByIndexAsync (string key, long index); #endregion #region Redis数据类型—Set /// /// 向指定集合中增加一个元素 /// ////// /// /// Task SetAddAsync (string key, T value); /// /// 指定集合计算操作operation枚举,指定计算结果将存的目标destKey,指定需要参与计算的多个key /// /// /// /// ///Task SetCombineAndStoreAsync(SetOperation operation, string destKey, List combineKeys); /// /// 指定集合计算操作operation枚举,指定需要参与计算的多个key /// ////// /// /// Task > SetCombineAsync
(SetOperation operation, List combineKeys); /// /// 指定值是否存在于指定集合中 /// ////// /// /// Task SetContainsAsync (string key, T value); /// /// 获取指定集合中元素个数 /// /// ///Task SetLengthAsync(string key); /// /// 获取指定集合中的所有元素 /// ////// /// /// Task > SetMembersAsync
(string key, T value); /// /// 从sourceKey移除指定value到目标distKey集合当中 /// 如果sourceKey存在指定value则返回true,否则不做任何操作返回false /// ////// /// /// /// Task SetMoveAsync (string sourcekey, string distKey, T value); /// /// 从指定集合当中随机取出一个元素 /// ////// /// Task SetRandomMemberAsync (string key); /// /// 从指定集合随机弹出(删除并获取)一个元素 /// ////// /// Task SetPopAsync (string key); /// /// 从集合中随机弹出(删除并获取)多个元素 /// ////// /// Task > SetRandomMembersAsync
(string key); /// /// 从集合中移除指定元素 /// ////// /// /// Task SetRemoveAsync (string key, T value); /// /// 从集合中批量移除元素 /// ////// /// /// Task SetMultiRemoveAsync (string key, List values); #endregion #region Redis数据类型—SortSet #endregion #region Redis Key操作 /// /// 删除指定key /// /// ///Task KeyDeleteAsync(string key); /// /// 设置key过期时间具体DateTime /// /// /// ///Task KeyExpireAtAsync(string key, DateTime expireAt); /// /// 设置key在将来的timeout后过期(TimeSpan) /// /// /// ///Task KeyExpireInAsync(string key, TimeSpan timeout); /// /// key重命名 /// /// /// ///Task KeyRenameAsync(string key, string newKey); /// /// 判断key是否已存在 /// /// ///Task KeyExistsAsync(string key); #endregion #region Redis Transcation /// /// 在事务中执行一系列redis命令。注意:在委托中的一系列命令的所有 值 都需要进行字节数组序列化 /// /// ///Task DoInTranscationAsync(Action ranOperations); #endregion Task Test(); } }
RedisHelper partial class RedisStringHelperAsync.cs
using System; using Fantasy.RedisRepository.CommonHelper; using StackExchange.Redis; using System.Threading.Tasks; namespace Fantasy.RedisRepository.RedisHelpers { ////// Redis异步操作类 String部分类 /// internal partial class RedisHelper// : IRedisHelper { private static IDatabase _client; internal RedisHelper() { _client = RedisConnection.GenerateConnection.GetDatabase(); } #region String 写操作 ////// 将任何数据添加到redis中 /// ////// /// /// /// public async Task StringSetAsync (string key, T value, TimeSpan? timeout = null) { return await _client.StringSetAsync(key, SerializeHelper.Serialize(value), timeout); } public async Task StringDecrementAsync(string key, long value = 1L) { return await _client.StringDecrementAsync(key, value); } public async Task StringIncrementAsync(string key, long value = 1L) { return await _client.StringIncrementAsync(key, value); } #endregion #region String 读操作 /// /// 根据key获取指定类型数据 /// ////// /// public async Task StringGetAsync (string key) { return SerializeHelper.Deserialize (await _client.StringGetAsync(key, CommandFlags.PreferSlave)); } #endregion } }
RedisHelper partial class RedisHashHelperAsync.cs
using Fantasy.RedisRepository.CommonHelper; using StackExchange.Redis; using System.Collections.Generic; using System.Linq; using System.Threading.Tasks; namespace Fantasy.RedisRepository.RedisHelpers { ////// Redis异步操作类 Hash部分类 /// internal partial class RedisHelper { #region Hash 写操作 public async TaskHashSetAsync (string key, string field, T value) { return await _client.HashSetAsync(key, field, SerializeHelper.Serialize(value)); } public async Task HashMultiSetAsync (string key, Dictionary hashFields) { List entries = new List (); hashFields.ToList().ForEach(d => entries.Add(new HashEntry(d.Key, SerializeHelper.Serialize(d.Value)))); await _client.HashSetAsync(key, entries.ToArray()); } public async Task HashIncrementAsync(string key, string field, long incrCount = 1) { return await _client.HashIncrementAsync(key, field, incrCount); } public async Task HashDecrementAsync(string key, string field, long decrCount = 1) { return await _client.HashDecrementAsync(key, field, decrCount); } public async Task HashDeleteFieldAsync(string key, string field) { return await _client.HashDeleteAsync(key, field); } public async Task HashMultiDeleteFieldAsync(string key, List fields) { List values = new List (); fields.ForEach(f => values.Add(f)); return await _client.HashDeleteAsync(key, values.ToArray()); } #endregion #region Hash 读操作 /// /// Redis 指定hash类型key中field是否存在 /// /// /// ///public async Task HashFieldExistAsync(string key, string field) { return await _client.HashExistsAsync(key, field, CommandFlags.PreferSlave); } public async Task > HashFieldsAsync(string key) { RedisValue[] values = await _client.HashKeysAsync(key, CommandFlags.PreferSlave); return RedisInnerTypeHelper.RedisValuesToGenericList
(values); } public async Task > HashValuesAsync
(string key) { var values = await _client.HashValuesAsync(key, CommandFlags.PreferSlave); return RedisInnerTypeHelper.RedisValuesToGenericList (values); } public async Task HashGetAsync (string key, string field) { return SerializeHelper.Deserialize (await _client.HashGetAsync(key, field, CommandFlags.PreferSlave)); } public async Task > HashGetAllAsync (string key) { HashEntry[] entries = await _client.HashGetAllAsync(key, CommandFlags.PreferSlave); Dictionary dic = new Dictionary (); entries.ToList().ForEach(e => dic.Add(e.Name, SerializeHelper.Deserialize (e.Value))); return dic; } #endregion } }
RedisHelper partial class RedisListHelperAsync.cs
using Fantasy.RedisRepository.CommonHelper; using StackExchange.Redis; using System.Collections.Generic; using System.Linq; using System.Threading.Tasks; namespace Fantasy.RedisRepository.RedisHelpers { ////// Redis异步操作类 Hash部分类 /// internal partial class RedisHelper { #region Hash 写操作 public async TaskHashSetAsync (string key, string field, T value) { return await _client.HashSetAsync(key, field, SerializeHelper.Serialize(value)); } public async Task HashMultiSetAsync (string key, Dictionary hashFields) { List entries = new List (); hashFields.ToList().ForEach(d => entries.Add(new HashEntry(d.Key, SerializeHelper.Serialize(d.Value)))); await _client.HashSetAsync(key, entries.ToArray()); } public async Task HashIncrementAsync(string key, string field, long incrCount = 1) { return await _client.HashIncrementAsync(key, field, incrCount); } public async Task HashDecrementAsync(string key, string field, long decrCount = 1) { return await _client.HashDecrementAsync(key, field, decrCount); } public async Task HashDeleteFieldAsync(string key, string field) { return await _client.HashDeleteAsync(key, field); } public async Task HashMultiDeleteFieldAsync(string key, List fields) { List values = new List (); fields.ForEach(f => values.Add(f)); return await _client.HashDeleteAsync(key, values.ToArray()); } #endregion #region Hash 读操作 /// /// Redis 指定hash类型key中field是否存在 /// /// /// ///public async Task HashFieldExistAsync(string key, string field) { return await _client.HashExistsAsync(key, field, CommandFlags.PreferSlave); } public async Task > HashFieldsAsync(string key) { RedisValue[] values = await _client.HashKeysAsync(key, CommandFlags.PreferSlave); return RedisInnerTypeHelper.RedisValuesToGenericList
(values); } public async Task > HashValuesAsync
(string key) { var values = await _client.HashValuesAsync(key, CommandFlags.PreferSlave); return RedisInnerTypeHelper.RedisValuesToGenericList (values); } public async Task HashGetAsync (string key, string field) { return SerializeHelper.Deserialize (await _client.HashGetAsync(key, field, CommandFlags.PreferSlave)); } public async Task > HashGetAllAsync (string key) { HashEntry[] entries = await _client.HashGetAllAsync(key, CommandFlags.PreferSlave); Dictionary dic = new Dictionary (); entries.ToList().ForEach(e => dic.Add(e.Name, SerializeHelper.Deserialize (e.Value))); return dic; } #endregion } }
RedisLuaHelper.cs I plan to install some functional Lua scripts here. Externally, parameters such as keys are still passed. This is incomplete and is just an example.
using StackExchange.Redis; using System.Threading.Tasks; namespace Fantasy.RedisRepository.RedisHelpers { internal partial class RedisHelper { public async TaskLuaMutilGetHash() { string lua = @"local result={} for i, v in ipairs(KEYS) do result[i] = redis.call('hgetall',v) end return result"; var res = await _client.ScriptEvaluateAsync(lua, new RedisKey[] { "people:1", "people:2", "people:3" }); var res1= LuaScript.GetCachedScriptCount(); return res; } } }
Regarding Transcation encapsulation, I personally don’t have any good method. I provide such a method
public async TaskDoInTranscationAsync(Action runOperations) { var tran = RedisConnection.GenerateConnection.GetDatabase().CreateTransaction(); runOperations(tran); return await tran.ExecuteAsync(); }
RedisFactory.cs
using Fantasy.RedisRepository.RedisHelpers; namespace Fantasy.RedisRepository { public class RedisFactory { ////// 外部访问redis入口,暂时只暴露异步方法 /// ///public static IRedisHelper CreateRedisRepository() { return new RedisHelper(); } } }
The above is The entire content of this article, I hope that the content of this article can bring some help to everyone's study or work, and I also hope to support the PHP Chinese website!
For more articles related to RedisRepository sharing and error correction, please pay attention to the PHP Chinese website!