Home > Backend Development > PHP Tutorial > memcache分布式 [一致性hash算法] 的php实现

memcache分布式 [一致性hash算法] 的php实现

WBOY
Release: 2016-06-23 13:37:29
Original
737 people have browsed it

最近在看一些分布式方面的文章,所以就用php实现一致性hash来练练手,以前一般用的是最原始的hash取模做分布式,当生产过程中添加或删除一台memcache都会造成数据的全部失效,一致性hash就是为了解决这个问题,把失效数据降到最低,相关资料可以google一下!

php实现效率有一定的缺失,如果要高效率,还是写扩展比较好
经测试,5个memcache,每个memcache生成100个虚拟节点,set加get1000次,与单个memcache直接set加get慢5倍,所以效率一般,有待优化!
实现过程:

memcache的配置 ip+端口+虚拟节点序列号 做hash,使用的是crc32,形成一个闭环。

对要操作的key进行crc32

二分法在虚拟节点环中查找最近的一个虚拟节点

从虚拟节点中提取真实的memcache ip和端口,做单例连接

<?php/*** 一致性哈希memcache分布式,采用的是虚拟节点的方式解决分布均匀性问题,查找节点采用二分法快速查找* the last known user to change this file in the repository  <$LastChangedBy: nash.xiong [        DISCUZ_CODE_0        ]gt;* @author nash.xiong <nash.xiong@gmail.com>* @copyright Copyright &copy; 2003-2012 phpd.cn* @license */class memcacheHashMap {        private $_node = array();        private $_nodeData = array();        private $_keyNode = 0;        private $_memcache = null;                //每个物理服务器生成虚拟节点个数 [注:节点数越多,cache分布的均匀性越好,同时set get操作时,也更耗资源,10台物理服务器,采用200较为合理]        private $_virtualNodeNum = 200;                 private function __construct() {                /* 放入配置文件 */                $config = array(                                                '127.0.0.1:11211',                                                '127.0.0.1:11212',                                                '127.0.0.1:11213',                                                '127.0.0.1:11214',                                                '127.0.0.1:11215'                                        );                                                        if (!$config) throw new Exception('Cache config NULL');                foreach ($config as $key => $value) {                        for ($i = 0; $i < $this->_virtualNodeNum; $i++) {                                $this->_node[sprintf("%u", crc32($value . '_' . $i))] = $value . '_' . $i;                        }                }                ksort($this->_node);        }        private function __clone(){}                /**         * 单例,保证只有一个实例         */        static public function getInstance() {                static $memcacheObj = null;                if (!is_object($memcacheObj)) {                        $memcacheObj = new self();                }                return $memcacheObj;        }                /**         * 根据key做一致性hash后连接到一台物理memcache服务器         * @param string $key         */        private function _connectMemcache($key) {                $this->_nodeData = array_keys($this->_node);                $this->_keyNode = sprintf("%u", crc32($key));                $nodeKey = $this->_findServerNode();                //如果超出环,从头再用二分法查找一个最近的,然后环的头尾做判断,取最接近的节点                if ($this->_keyNode > end($this->_nodeData)) {                        $this->_keyNode -= end($this->_nodeData);                        $nodeKey2 = $this->_findServerNode();                        if (abs($nodeKey2 - $this->_keyNode) < abs($nodeKey - $this->_keyNode))  $nodeKey = $nodeKey2;                }                var_dump($this->_node[$nodeKey]);                list($config, $num) = explode('_', $this->_node[$nodeKey]);                if (!$config) throw new Exception('Cache config Error');                if (!isset($this->_memcache[$config])) {                        $this->_memcache[$config] = new Memcache;                        list($host, $port) = explode(':', $config);                        $this->_memcache[$config]->connect($host, $port);                }                return $this->_memcache[$config];        }                /**         * 采用二分法从虚拟memcache节点中查找最近的节点         * @param unknown_type $m         * @param unknown_type $b         */        private function _findServerNode($m = 0, $b = 0) {            $total = count($this->_nodeData);            if ($total != 0 && $b == 0) $b = $total - 1;            if ($m < $b){                $avg = intval(($m+$b) / 2);                if ($this->_nodeData[$avg] == $this->_keyNode) return $this->_nodeData[$avg];                elseif ($this->_keyNode < $this->_nodeData[$avg] && ($avg-1 >= 0)) return $this->_findServerNode($m, $avg-1);                else return $this->_findServerNode($avg+1, $b);            }                if (abs($this->_nodeData[$b] - $this->_keyNode) < abs($this->_nodeData[$m] - $this->_keyNode))  return $this->_nodeData[$b];                else return $this->_nodeData[$m];        }                public function set($key, $value, $expire = 0) {                return $this->_connectMemcache($key)->set($key, json_encode($value), 0, $expire);        }                public function add($key, $value, $expire = 0) {                return $this->_connectMemcache($key)->add($key, json_encode($value), 0, $expire);        }                public function get($key) {                return json_decode($this->_connectMemcache($key)->get($key), true);        }                public function delete($key) {                return $this->_connectMemcache($key)->delete($key);        }        }
Copy after login
$runData['BEGIN_TIME'] = microtime(true);//测试一万次set加getfor($i=0;$i<10000;$i++) {        $key = md5(mt_rand());        $b = memcacheHashMap::getInstance()->set($key, time(), 10);}var_dump(number_format(microtime(true) - $runData['BEGIN_TIME'],6));$runData['BEGIN_TIME'] = microtime(true); $m= new Memcache;$m->connect('127.0.0.1', 11211); for($i=0;$i<10000;$i++) {        $key = md5(mt_rand());        $b = $m->set($key, time(), 0, 10);}var_dump(number_format(microtime(true) - $runData['BEGIN_TIME'],6));
Copy after login

//测试结果,采用一致性哈希分布效率比原生单台速度相差5倍左右

source:php.cn
Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn
Popular Tutorials
More>
Latest Downloads
More>
Web Effects
Website Source Code
Website Materials
Front End Template