4.1路由验证
4.1.1测试通过MySQL Router连接6446 RW端口，查看连接的是否为primary
[root@node4 mysqlrouter]# for ((i=0;i<=5;i++));do mysql -h172.16.8.101 -uroot -p123456 -P6446 -e"select @@hostname;";done;
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| oratest51 |
+------------+
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| oratest51 |
+------------+
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| oratest51 |
+------------+
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| oratest51 |
+------------+
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| oratest51 |
+------------+
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| oratest51 |
+------------+
ログイン後にコピー
测试结果： 使用6446 RW端口会访问到后端InnoDB Cluster的primary
4.1.2测试通过MySQL Router连接6447 RO端口，查看连接的是否为seconde节点，rr算法是否生效
[root@node4 mysqlrouter]# for ((i=0;i<=5;i++));do mysql -h172.16.8.101 -uroot -p123456 -P6447 -e"select @@hostname;";done;
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| oratest52 |
+------------+
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| test61 |
+------------+
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| oratest52 |
+------------+
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| test61 |
+------------+
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| oratest52 |
+------------+
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| test61 |
+------------+
ログイン後にコピー
测试结果： 使用6447 RO端口会连接到后端InnoDB Cluster集群中所有的seconde节点，并且以rr(round-robin)的策略调度
4.2高可用测试
4.2.1模拟其中一台seconde节点宕机，查看MySQL Router是否会自动取消该节点的路由
(1) 停掉一台seconde节点
[root@oratest52 ~]# systemctl status mysql
[root@oratest52 ~]# systemctl stop mysql
[root@oratest52 ~]# systemctl status mysql
ログイン後にコピー
(2) 在InnoDB Cluster中查看集群状态，可以看到oratest52已经处于MISSING状态
MySQL 172.16.9.51:3306 JS > cluster.status()
{
"clusterName": "st",
"defaultReplicaSet": {
"name": "default",
"primary": "oratest51:3306",
"ssl": "DISABLED",
"status": "OK_NO_TOLERANCE",
"statusText": "Cluster is NOT tolerant to any failures. 1 member is not active",
"topology": {
"oratest51:3306": {
"address": "oratest61:3306",
"mode": "R/O",
"readReplicas": {},
"role": "HA",
"status": "ONLINE"
},
"oratest52:3306": {
"address": "oratest52:3306",
"mode": "n/a",
"readReplicas": {},
"role": "HA",
"shellConnectError": "MySQL Error 2003 (HY000): Can't connect to MySQL server on 'oratest52' (111)",
"status": "(MISSING)"
},
"test61:3306": {
"address": "test51:3306",
"mode": "R/W",
"readReplicas": {},
"role": "HA",
"status": "ONLINE"
}
},
"topologyMode": "Single-Primary"
},
"groupInformationSourceMember": "test61:3306"
}ログイン後にコピー
(3) 在MGR中查看GR状态，可以看到已经没有oratest52这台机器
mysql> select * from performance_schema.replication_group_members;
+---------------------------+--------------------------------------+-------------+-------------+--------------+
| CHANNEL_NAME | MEMBER_ID | MEMBER_HOST | MEMBER_PORT | MEMBER_STATE |
+---------------------------+--------------------------------------+-------------+-------------+--------------+
| group_replication_applier | 48cdca42-a386-11e9-95f1-0050569d2587 | test61 | 3306 | ONLINE |
| group_replication_applier | e9ee5267-a386-11e9-ac35-0050569d9ef5 | oratest51 | 3306 | ONLINE |
+---------------------------+--------------------------------------+-------------+-------------+--------------+
ログイン後にコピー
(4) 测试通过6447 RO端口连接，查看是否会自动剔除宕机的oratest52节点
[root@node4 ~]# for ((i=0;i<=5;i++));do mysql -h172.16.8.101 -uroot -p123456 -P6447 -e"select @@hostname;";done;
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| test61 |
+------------+
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| test61 |
+------------+
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| test61 |
+------------+
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| test61 |
+------------+
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| test61 |
+------------+
ログイン後にコピー
测试结果： MySQL Router会自动剔除宕机的oratest52节点，read请求只会转发到后端正常的MySQL实例，过程无需人工干预。
4.2.2启动上一步中停止的MySQL实例，确认MGR和InnoDB Cluster一切正常后，查看MySQL Router是否会自动加入这台已恢复的实例进行转发
(1) 启动oratest52的MySQL服务
[root@oratest52 ~]# systemctl start mysql
ログイン後にコピー
(2)启动MGR，并确认MGR状态正常
mysql> start group_replication;
Query OK, 0 rows affected (2.92 sec)
mysql> select * from performance_schema.replication_group_members;
+---------------------------+--------------------------------------+-------------+-------------+--------------+
| CHANNEL_NAME | MEMBER_ID | MEMBER_HOST | MEMBER_PORT | MEMBER_STATE |
+---------------------------+--------------------------------------+-------------+-------------+--------------+
| group_replication_applier | 48cdca42-a386-11e9-95f1-0050569d2587 | test61 | 3306 | ONLINE |
| group_replication_applier | 50bc1160-a386-11e9-92c9-0050569dc0da | oratest52 | 3306 | ONLINE |
| group_replication_applier | e9ee5267-a386-11e9-ac35-0050569d9ef5 | oratest51 | 3306 | ONLINE |
+---------------------------+--------------------------------------+-------------+-------------+--------------+
ログイン後にコピー
(3) 检查InnoDB Cluster是否正常
MySQL 172.16.9.51:3306 JS > cluster.status()
{
"clusterName": "st",
"defaultReplicaSet": {
"name": "default",
"primary": "oratest51:3306",
"ssl": "DISABLED",
"status": "OK",
"statusText": "Cluster is ONLINE and can tolerate up to ONE failure.",
"topology": {
"oratest51:3306": {
"address": "oratest61:3306",
"mode": "R/O",
"readReplicas": {},
"role": "HA",
"status": "ONLINE"
},
"oratest52:3306": {
"address": "oratest52:3306",
"mode": "R/O",
"readReplicas": {},
"role": "HA",
"status": "ONLINE"
},
"test61:3306": {
"address": "test51:3306",
"mode": "R/W",
"readReplicas": {},
"role": "HA",
"status": "ONLINE"
}
},
"topologyMode": "Single-Primary"
},
"groupInformationSourceMember": "test61:3306"
}ログイン後にコピー
(4) 查看恢复正常的实例，能不能自动加入MySQL Router进行转发，可以看到oratest52可以被正常调度
[root@node4 mysqlrouter]# for ((i=0;i<=5;i++));do mysql -h172.16.8.101 -uroot -p123456 -P6447 -e"select @@hostname;";done;
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| oratest52 |
+------------+
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| test61 |
+------------+
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| oratest52 |
+------------+
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| test61 |
+------------+
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| oratest52 |
+------------+
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| test61 |
+------------+
ログイン後にコピー
测试结果： 恢复正常的MySQL实例可以自动加入MySQL Router并进行转发，无需人工干预。
4.2.3模拟停掉primary节点，测试MySQL Router如何处理发送RW端口的请求，MySQL Router怎样获取新的primary节点
(1) 当前primary节点是oratest51，在primary节点上停止MySQL，确认停止成功
[root@oratest51 data]# systemctl status mysql
[root@oratest51 data]# systemctl stop mysql
[root@oratest51 data]# systemctl status mysql
[root@oratest51 data]# ps -ef |grep mysql
ログイン後にコピー
(2) 查看InnoDB Cluster状态，可以看到oratest51已经处于MISSING状态，primary节点已经切换到了test61
MySQL 172.16.9.61:3306 JS > cluster.status();
{
"clusterName": "st",
"defaultReplicaSet": {
"name": "default",
"primary": "test61:3306",
"ssl": "DISABLED",
"status": "OK_NO_TOLERANCE",
"statusText": "Cluster is NOT tolerant to any failures. 1 member is not active",
"topology": {
"oratest51:3306": {
"address": "oratest51:3306",
"mode": "n/a",
"readReplicas": {},
"role": "HA",
"shellConnectError": "MySQL Error 2003 (HY000): Can't connect to MySQL server on 'oratest51' (111)",
"status": "(MISSING)"
},
"oratest52:3306": {
"address": "oratest52:3306",
"mode": "R/O",
"readReplicas": {},
"role": "HA",
"status": "ONLINE"
},
"test61:3306": {
"address": "test61:3306",
"mode": "R/W",
"readReplicas": {},
"role": "HA",
"status": "ONLINE"
}
},
"topologyMode": "Single-Primary"
},
"groupInformationSourceMember": "test61:3306"
}ログイン後にコピー
(3) 测试连接到RW 6446端口，查看是否可以正常连接到新的primary节点test61，并且写入数据否正常
[root@node2 mysqlrouter]# for ((i=0;i<=5;i++));do mysql -h172.16.8.101 -uroot -p123456 -P6446 -e"select @@hostname;";done;
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| test61 |
+------------+
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| test61 |
+------------+
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| test61 |
+------------+
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| test61 |
+------------+
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| test61 |
+------------+
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| test61 |
+------------+
ログイン後にコピー
测试结果： 当primary节点挂掉之后，MySQL Router会自动去连接新的primary节点
4.2.4启动上一步中停掉的oratest51，将状态恢复正常，测试MySQL Router能不能自动加入这台已恢复的节点并调度
(1) 启动oratest51，将状态恢复正常
[root@oratest51 ~]# systemctl start mysql
[root@oratest51 ~]# systemctl status mysql
[root@oratest51 ~]# mysql -uroot -p123456
mysql> start group_replication;
mysql> select * from performance_schema.replication_group_members;
+---------------------------+--------------------------------------+------------+-------------+--------------+
| CHANNEL_NAME | MEMBER_ID |MEMBER_HOST | MEMBER_PORT | MEMBER_STATE |
+---------------------------+--------------------------------------+------------+-------------+--------------+
| group_replication_applier | 48cdca42-a386-11e9-95f1-0050569d2587 |test61 | 3306 | ONLINE |
| group_replication_applier | 50bc1160-a386-11e9-92c9-0050569dc0da |oratest52 | 3306 | ONLINE |
| group_replication_applier | e9ee5267-a386-11e9-ac35-0050569d9ef5 |oratest51 | 3306 | ONLINE |
+---------------------------+--------------------------------------+------------+-------------+--------------+
3 rows in set (0.00 sec)
mysql> quit
[root@oratest51 local]# mysqlsh
MySQL JS > shell.connect('root@172.16.9.51:3306')
MySQL 172.16.9.51:3306 JS > var cluster = dba.getCluster()
MySQL 172.16.9.51:3306 JS > cluster.status();
{
"clusterName": "st",
"defaultReplicaSet": {
"name": "default",
"primary": "test61:3306",
"ssl": "DISABLED",
"status": "OK",
"statusText": "Cluster is ONLINE and can tolerate up to ONE failure.",
"topology": {
"oratest51:3306": {
"address": "oratest51:3306",
"mode": "R/O",
"readReplicas": {},
"role": "HA",
"status": "ONLINE"
},
"oratest52:3306": {
"address": "oratest52:3306",
"mode": "R/O",
"readReplicas": {},
"role": "HA",
"status": "ONLINE"
},
"test61:3306": {
"address": "test61:3306",
"mode": "R/W",
"readReplicas": {},
"role": "HA",
"status": "ONLINE"
}
},
"topologyMode": "Single-Primary"
},
"groupInformationSourceMember": "test61:3306"
}ログイン後にコピー
(2) 测试RO端口的Router调度
[root@node4 mysqlrouter]# for ((i=0;i<=5;i++));do mysql -h172.16.8.101 -uroot -p123456 -P6447 -e"select @@hostname;";done;
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| oratest52 |
+------------+
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| oratest51 |
+------------+
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| oratest52 |
+------------+
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| oratest51 |
+------------+
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| oratest52 |
+------------+
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+
| @@hostname |
+------------+
| test61 |
+------------+
ログイン後にコピー
测试结果： 恢复正常的实例加入集群后，会自动加入到MySQL Router并进行调度
五、运维相关
5.1数据库节点变更
MySQL Router在初始化配置的时候是连接到集群节点读取集群的元数据的。如果在集群中新增或减少节点，需要同步更新MySQL Router的配置，否则无法生效
# mysqlrouter --bootstrap 172.16.9.51:3306 --directory /data/mysqlrouter --user=root --conf-use-sockets --force
ログイン後にコピー
更新了MySQL Router的配置的配置，需要重启MySQL Router：
#systemctl restart mysqlrouter
#systemctl status mysqlrouter
ログイン後にコピー
六、问题汇总
6.1初始化MySQL Router前，需要停止group_replication
报错如下
[root@oratest51 data]# mysqlrouter --bootstrap root@172.16.9.51:3310 --directory /data/mysqlrouter --conf-use-sockets --user=root
Please enter MySQL password for root:
Error: Unable to connect to the metadata server: Error connecting to MySQL server at 172.16.9.51:3310: Can't connect to MySQL server on '172.16.9.51' (111) (2003)
ログイン後にコピー
原因： 需要配置连接用户的权限
解决办法： 给连接用户配置所有权限，如下：
mysql> grant all privileges on *.* to root@'%' identified by "123456";
mysql> flush privileges;
ログイン後にコピー
[root@oratest51 data]# mysqlrouter --bootstrap root@172.16.9.51:3306 --directory /data/mysqlrouter --conf-use-sockets --user=root
Please enter MySQL password for root:
Error: Expected MySQL Server '172.16.9.51:3306' to contain the metadata of MySQL InnoDB Cluster, but the schema does not exist.
Checking version of the metadata schema failed with: Error executing MySQL query: Table 'mysql_innodb_cluster_metadata.schema_version' doesn't exist (1146)
See https://dev.mysql.com/doc/refman/en/mysql-innodb-cluster-creating.html for instructions on setting up a MySQL Server to act as an InnoDB Cluster Metadata server
ログイン後にコピー
原因： 没有创建InnoDB Cluster集群，创建InnoDB Cluster集群后会自动生成这张表。mysql_innodb_cluster_metadata.schema_version表的用途是MySQL Router在进行调度分配的时候，需要读取这张表的内容来做调度策略。
解决办法： 创建InnoDB Cluster集群
6.4启动MySQL Router hang住不动，查看日志报错Error: bootstrap_server_addresses is not allowed when dynamic state file is used
具体如下：
[root@node2 log]# tailf /root/mysqlrouter/log/mysqlrouter.log
2019-08-02 15:37:52 routing INFO [7f9721e2d700] [routing:st_default_ro] started: listening using /root/mysqlrouter/mysqlro.sock
2019-08-02 15:37:52 routing INFO [7f972162c700] [routing:st_default_rw] started: listening using /root/mysqlrouter/mysql.sock
2019-08-02 15:37:52 routing INFO [7f9720e2b700] [routing:st_default_x_ro] started: listening using /root/mysqlrouter/mysqlxro.sock
2019-08-02 15:37:52 routing INFO [7f9713fff700] [routing:st_default_x_rw] started: listening on 0.0.0.0:64460, routing strategy = first-available
2019-08-02 15:37:52 routing INFO [7f9713fff700] [routing:st_default_x_rw] started: listening using /root/mysqlrouter/mysqlx.sock
2019-08-02 15:37:52 routing INFO [7f9713fff700] [routing:st_default_x_rw] stopped
2019-08-02 15:37:52 routing INFO [7f9721e2d700] [routing:st_default_ro] stopped
2019-08-02 15:37:52 routing INFO [7f972162c700] [routing:st_default_rw] stopped
2019-08-02 15:37:52 routing INFO [7f9720e2b700] [routing:st_default_x_ro] stopped
2019-08-02 15:37:52 main ERROR [7f9726f1b880] Error: bootstrap_server_addresses is not allowed when dynamic state file is used
ログイン後にコピー
原因： 开启了dynamic_state
解决办法： 在配置文件中注释掉dynamic_state所在行，例如下列第11行
1 # File automatically generated during MySQL Router bootstrap
2 [DEFAULT]
3 user=root
4 logging_folder=/root/mysqlrouter/log
5 runtime_folder=/root/mysqlrouter/run
6 data_folder=/root/mysqlrouter/data
7 keyring_path=/root/mysqlrouter/data/keyring
8 master_key_path=/root/mysqlrouter/mysqlrouter.key
9 connect_timeout=15
10 read_timeout=30
11 #dynamic_state=/root/mysqlrouter/data/state.json
ログイン後にコピー
6.5启动MySQL Router hang住不动，查看日志报错Unable to fetch live group_replication member data from any server in replicaset 'default'
具体如下：
[root@node2 log]# tailf /root/mysqlrouter/log/mysqlrouter.log
2019-08-02 15:46:41 metadata_cache WARNING [7f3030405700] While updating metadata, could not establish a connection to replicaset 'default' through test61:3306
2019-08-02 15:46:51 metadata_cache WARNING [7f3030405700] While updating metadata, could not establish a connection to replicaset 'default' through oratest52:3306
2019-08-02 15:47:01 metadata_cache WARNING [7f3030405700] While updating metadata, could not establish a connection to replicaset 'default' through oratest51:3306
2019-08-02 15:47:01 metadata_cache ERROR [7f3030405700] Unable to fetch live group_replication member data from any server in replicaset 'default'
ログイン後にコピー
原因： MySQL Router所在节点没有配置本机DNS解析
解决办法： 在/etc/hosts中添加MGR实例的解析
七、个人总结
- MySQL Router有两种部署模式：
- bootstrap模式：支持failover，必须结合InnoDB Cluster使用，在
--directory指定的路径下自动生成安装目录，配置文件里的端口为6446和6447。
- 简单模式：不支持failover，无需结合InnoDB Cluster使用，一般在主从复制或者主主复制等模式下使用，使用
/usr/local/mysqlrouter/share/doc/mysqlrouter/sample_mysqlrouter.conf这个配置文件部署。
- MySQL Router使用的是一个IP地址(可以理解为VIP)加一个RW端口和一个RO端口实现读写分离，Router自身不能判断应用程序的请求是读还是写，因此做读写分离时，必须指定两个端口，如果应用程序不便指定两个端口，也可以全部指定RW端口，这样的话从节点就只作高可用，不做读写分离。
- MySQL Router自身存在单点故障隐患，官方推荐在每个应用程序所在机器上部署Router，本机器连接本机器的Router。也可以在Router的上层在搭建一个高可用服务，如果是自建机房可以选择Keepalived、pacemaker等方案；
- 通过bootstrap生成的配置文件，只需要在
bootstrap_server_addresses这一栏里配置就可以了，格式如下：bootstrap_server_addresses=mysql://oratest51:3306,mysql://oratest52:3306,mysql://test61:3306
ログイン後にコピー
- MySQL Router的后端MySQL实例挂掉之后，无需人工干预，MySQL Router会自动剔除挂了的MySQL实例，当挂了的MySQL实例恢复后会自动加入MySQL Router的后端服务器，也无需人工干预。
- MGR架构可以实现高可用，但是要实现failover，则需要安装InnoDB Cluster了。MySQL Shell和MySQL Router是InnoDB Cluster集群的一部分
- MySQL Router非常轻量级，性能损耗小于1%，官方建议每台应用上部署一个mysqlrouter节点，优点是节省网络带宽。缺点是mysqlrouter太轻量级了，只能提供简单的基于端口的读写路由(Port based routing)和基于权重轮询的负载均衡(Weighted Balance)，不支持基于SQL的读写分离(Query based routing)和空闲优先的负载均衡(Application Layer balancer)
【相关推荐：mysql视频教程】
![[Web フロントエンド] Node.js クイック スタート](https://img.php.cn/upload/course/000/000/067/662b5d34ba7c0227.png)
