Possible MySQL server UUID duplication for server

黄舟原创: 2017-02-15 10:40:28 1092浏览

在mysql enterprise monitor监控过程中出现这样的event事件，Topic: Possible MySQL server UUID duplication for server 事件，从该提示的描述来看貌似是存在重复的uuid，而实际上主从关系并不存在重复的uuid。主从关系是通过xtrabackup来构建的。那到底是哪里的问题呢？下文是描述基于xtrabackup复制时导致监控出现重复uuid的问题。

1、故障现象

Topic: Possible MySQL server UUID duplication for server afd6bca4-6636-11e3-9d60-74867ae1c47c: NOTICE
Categories:Monitoring and Support Services 
Current State:Open
Auto-Closes by Default:Yes
Advisor:Duplicate MySQL Server UUID 
Current Status:Notice
Last Checked:May 4, 2015 2:18:02 PM

###提示uuid改变了N多次，在MySQL Instances监控见面会出现这2个主机配置在不停的切换中

MySQL server afd6bca4-6636-11e3-9d60-74867ae1c47c changed its hostname 181 times between the following hostnames:DBSRV-TXT01DBSRV-SLAVE02
MySQL server afd6bca4-6636-11e3-9d60-74867ae1c47c changed its connection TCP properties 96 times between the following TCP properties:
127.0.0.1:3306127.0.0.1:3307
AdviceCheck that you are not monitoring more than one instance with the following server UUID: afd6bca4-6636-11e3-9d60-74867ae1c47c. 
This can be caused by server or host cloning. If this is expected (example: HA scenarios), then please ignore this notice.

2、校验uuid
###经google文章描述，mysql.inventory保存了被mysql enterpirse moitor监控用到的uuid信息
###查看主库上的uuid及mysql.inventory表

[root@DBSRV-TXT01 ~]# mysql -uroot -p -e "
show variables like 'server_uuid';
select * from mysql.inventory;" -S /tmp/mysql3307.sock 
Enter password: 
+---------------+--------------------------------------+
| Variable_name | Value                                |
+---------------+--------------------------------------+
| server_uuid   | 1ed85852-dd27-11e4-aa4a-44a8420ba7a5 |
+---------------+--------------------------------------+
+--------+-------------------------------------------------------+
| name   | value                                                 |
+--------+-------------------------------------------------------+
| uuid   | afd6bca4-6636-11e3-9d60-74867ae1c47c                  |
| hostid | ssh:{8a:c7:a9:42:3a:6b:06:ad:fa:ed:04:ac:a5:fa:f0:b5} |
+--------+-------------------------------------------------------+

###查看从库上的uuid及mysql.inventory表

[root@DBSRV-SLAVE02 ~]# mysql -uroot -p -e "    ---Author : Leshami
> show variables like 'server_uuid';            ---Blog   : 
//m.sbmmt.com/
> select * from mysql.inventory;"
Enter password: 
+---------------+--------------------------------------+
| Variable_name | Value                                |
+---------------+--------------------------------------+
| server_uuid   | f7e00194-2f59-11e4-bcf6-b82a72d46b21 |
+---------------+--------------------------------------+
+--------+-------------------------------------------------------+
| name   | value                                                 |
+--------+-------------------------------------------------------+
| uuid   | afd6bca4-6636-11e3-9d60-74867ae1c47c                   |
| hostid | ssh:{8a:c7:a9:42:3a:6b:06:ad:fa:ed:04:ac:a5:fa:f0:b5} |
+--------+-------------------------------------------------------+

###从上面的查询结果得知，mysql.inventory表里边确实保存了相同的uuid
###这个相同的uuid是由于使用了实例级别的热备，所以2个实例具有相同的uuid

###清空mysql.inventory，然后重启监控agent(略)，问题解决

[root@DBSRV-SLAVE02 ~]# mysql -uroot -p -e "truncate table mysql.inventory" -S /tmp/mysql3307.sock
Enter password:

3、关于MySQL MEM UUID duplication

MySQL Enterprise Monitor uses a number of unique values known as       
UUIDs to identify the different      
components, including the MySQL instance being monitored. UUID      
values related to the MySQL instance and the host on which it runs      
are stored in a table mysql.inventory within      
the instance. MySQL Enterprise Monitor creates this table if it does not exist      
already.

Each MySQL Server has a UUID, stored in the mysql.inventory table, that uniquely identifies the MySQL server to the rest of MEM. The server UUID is used to collate information about a single MySQL instance.
Each host (the machine on which the agent is running) has a UUID to uniquely identify the host to the rest of MySQL Enterprise Monitor. This is used to collate the OS information (such as CPU, RAM and disk data). The host ID also determines whether the MySQL server is on the same host as it was before, to identify when data has been moved between machines, or when a machine has been upgraded. The host UUID is stored within the hostid row within the mysql.inventory table.
Each agent has a UUID to identify the agent to MEM. The agent UUID is defined within the agent-uuid parameter within the agent configuration file.

These UUIDs are used in combination to register and collate information, and to determine the location and source of an issue.

Note

Because each host must be unique, be careful when restoring from a backup so you do not have hosts with duplicated SSH keys or UUIDs.

The MySQL Enterprise Monitor Agent and MySQL Enterprise Service Manager use the unique host ID, stored within the mysql.inventory table on the monitored MySQL Server, to determine whether the instance being monitored is a clone. The host ID of the current server is checked against the stored value when the agent starts. If the generated host ID and stored host ID do not match, you get an error similar to the following in the agent log file:

%s: [%s] the hostid from mysql.inventory doesn't match our agent's host-id (%s != %s)
We assume that this is a cloned host and shutdown now.
Please TRUNCATE TABLE mysql.inventory on this mysql-instance and restart the agent.
If this is a master for replication, please also run SET SQL_LOG_BIN = 0; first.

To fix the problem, connect to the MySQL server using the credentials configured when you installed the agent, and then truncate the mysql.inventory table: