MySQL異步複製和半同步複製怎麼實現

PHPz
發布: 2023-06-01 10:28:26
轉載
1279 人瀏覽過

MySQL異步複製和半同步複製怎麼實現

非同步複製

MySQL異步複製和半同步複製怎麼實現

在非同步複製(async replication)中,Master不用關心Slave是否接收到二進位日誌,所以Master與Slave沒有任何的依賴關係。 Master和Slave可以被視為兩個獨立工作的伺服器,在二進位日誌的作用下,它們最終能夠保持資料的一致性。

非同步複製的效能最好,因為它對資料庫本身幾乎沒有任何開銷,除非主從延遲非常大,Dump Thread需要讀取大量二進位日誌檔案。

如果業務對於資料一致性要求不高,當發生故障時,能容忍資料的遺失,甚至大量的遺失,推薦用非同步複製,這樣效能最好(例如像微博這樣的業務,雖然它對效能的要求極高,但對於資料遺失,通常可以容忍)。但往往核心業務系統最關心的就是資料安全,例如監控業務、警報系統。

異步複製環境的規劃:

  • master(docker),連接埠3310

  • slave(docker),連接埠3311

master的設定

設定檔my.cnf

$ sudo cat /home/mysql/docker-data/3311/conf/my.cnf # For advice on how to change settings please see # http://dev.mysql.com/doc/refman/5.7/en/server-configuration-defaults.html [mysqld] # # Remove leading # and set to the amount of RAM for the most important data # cache in MySQL. Start at 70% of total RAM for dedicated server, else 10%. # innodb_buffer_pool_size = 128M # # Remove leading # to turn on a very important data integrity option: logging # changes to the binary log between backups. # log_bin # # Remove leading # to set options mainly useful for reporting servers. # The server defaults are faster for transactions and fast SELECTs. # Adjust sizes as needed, experiment to find the optimal values. # join_buffer_size = 128M # sort_buffer_size = 2M # read_rnd_buffer_size = 2M #datadir=/home/mysql/docker-data/3307/data #socket=/home/mysql/docker-data/3307/mysql.sock character_set_server=utf8 init_connect='SET NAMES utf8' # Disabling symbolic-links is recommended to prevent assorted security risks symbolic-links=0 #log-error=/home/mysql/docker-data/3307/logs/mysqld.log #pid-file=/home/mysql/docker-data/3307/mysqld.pid lower_case_table_names=1 # 表名是否小写 server-id=1403311 log-bin=mysql-bin # 开启binlog binlog-format=ROW # binlog的格式 auto_increment_increment=1 # 自增的步长,适用于主主复制,为了避免id冲突,步长设置为master的个数 auto_increment_offset=1 # 自增的偏移,主主复制每个master的偏移需要不一致 # binlog-do-db=mstest # 要同步的数据库 # binlog-ignore-db=mysql # 要忽略的数据库 #rpl_semi_sync_master_enabled=1 #rpl_semi_sync_master_timeout=10000
登入後複製

啟動mysql:

$ docker run --name mysql3310 -p 3310:3306 --privileged=true -ti -e MYSQL_ROOT_PASSWORD=root -e MYSQL_DATABASE=order -e MYSQL_USER=user -e MYSQL_PASSWORD=pass -v /home/mysql/docker-data/3310/conf:/etc/mysql/conf.d -v /home/mysql/docker-data/3310/data/:/var/lib/mysql -v /home/mysql/docker-data/3310/logs/:/var/log/mysql -d mysql:5.7
登入後複製

建立用於同步的使用者:

mysql> GRANT REPLICATION SLAVE,FILE,REPLICATION CLIENT ON *.* TO 'repluser'@'%' IDENTIFIED BY '123456'; Query OK, 0 rows affected, 1 warning (0.01 sec) mysql> FLUSH PRIVILEGES; Query OK, 0 rows affected (0.01 sec)
登入後複製

在master上查看master的二進位日誌:

mysql> show master status; +------------------+----------+--------------+------------------+-------------------+ | File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set | +------------------+----------+--------------+------------------+-------------------+ | mysql-bin.000003 | 1147 | | | | +------------------+----------+--------------+------------------+-------------------+ 1 row in set (0.00 sec)
登入後複製

查看master上的進程清單:

mysql> show processlist; +----+----------+------------------+-------+-------------+------+---------------------------------------------------------------+------------------+ | Id | User | Host | db | Command | Time | State | Info | +----+----------+------------------+-------+-------------+------+---------------------------------------------------------------+------------------+ | 2 | root | localhost | order | Query | 0 | starting | show processlist | | 6 | repluser | 172.17.0.1:48450 | NULL | Binlog Dump | 448 | Master has sent all binlog to slave; waiting for more updates | NULL | +----+----------+------------------+-------+-------------+------+---------------------------------------------------------------+------------------+ 2 rows in set (0.00 sec)
登入後複製

slave的設定

設定檔my. cnf與master的配置一致,除了server-id欄位需唯一。

啟動mysql:

$ docker run --name mysql3311 -p 3311:3306 --privileged=true -ti -e MYSQL_ROOT_PASSWORD=root -e MYSQL_DATABASE=order -e MYSQL_USER=user -e MYSQL_PASSWORD=pass -v /home/mysql/docker-data/3311/conf:/etc/mysql/conf.d -v /home/mysql/docker-data/3311/data/:/var/lib/mysql -v /home/mysql/docker-data/3311/logs/:/var/log/mysql -d mysql:5.7
登入後複製

在slave中設定master的資訊:

# master_log_file和master_log_pos取上面show master status显示的值mysql> change master to master_host='172.23.252.98',master_port=3310,master_user='repluser',master_password='123456',master_log_file='mysql-bin.000003',master_log_pos=1147;
登入後複製

開啟slave,啟動SQL和IO執行緒:

mysql> start slave;Query OK, 0 rows affected (0.00 sec)
登入後複製

查看slave的狀態:

mysql> show slave status\G;*************************** 1. row *************************** Slave_IO_State: Waiting for master to send event Master_Host: 192.168.54.214 Master_User: repluser Master_Port: 3310 Connect_Retry: 60 Master_Log_File: mysql-bin.000003 Read_Master_Log_Pos: 1147 Relay_Log_File: 2da789531bf3-relay-bin.000002 Relay_Log_Pos: 320 Relay_Master_Log_File: mysql-bin.000003 Slave_IO_Running: Yes Slave_SQL_Running: Yes Replicate_Do_DB: Replicate_Ignore_DB: Replicate_Do_Table: Replicate_Ignore_Table: Replicate_Wild_Do_Table: Replicate_Wild_Ignore_Table: Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Master_Log_Pos: 1147 Relay_Log_Space: 534 Until_Condition: None Until_Log_File: Until_Log_Pos: 0 Master_SSL_Allowed: No Master_SSL_CA_File: Master_SSL_CA_Path: Master_SSL_Cert: Master_SSL_Cipher: Master_SSL_Key: Seconds_Behind_Master: 0Master_SSL_Verify_Server_Cert: No Last_IO_Errno: 0 Last_IO_Error: Last_SQL_Errno: 0 Last_SQL_Error: Replicate_Ignore_Server_Ids: Master_Server_Id: 1403311 Master_UUID: cd2eaa0a-7a59-11ec-b3b4-0242ac110002 Master_Info_File: /var/lib/mysql/master.info SQL_Delay: 0 SQL_Remaining_Delay: NULL Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates Master_Retry_Count: 86400 Master_Bind: Last_IO_Error_Timestamp: Last_SQL_Error_Timestamp: Master_SSL_Crl: Master_SSL_Crlpath: Retrieved_Gtid_Set: Executed_Gtid_Set: Auto_Position: 0 Replicate_Rewrite_DB: Channel_Name: Master_TLS_Version:1 row in set (0.00 sec)
登入後複製

只有新增的資料才會進行複製,存量的資料需要手動使用工具(如mysqldump)進行同步。

檢視slave上的行程清單:

mysql> show processlist; +----+-------------+-----------+-------+---------+------+--------------------------------------------------------+------------------+ | Id | User | Host | db | Command | Time | State | Info | +----+-------------+-----------+-------+---------+------+--------------------------------------------------------+------------------+ | 4 | root | localhost | order | Query | 0 | starting | show processlist | | 7 | system user | | NULL | Connect | 533 | Waiting for master to send event | NULL | | 8 | system user | | NULL | Connect | 127 | Slave has read all relay log; waiting for more updates | NULL | +----+-------------+-----------+-------+---------+------+--------------------------------------------------------+------------------+ 3 rows in set (0.00 sec)
登入後複製

半同步複製

半同步複製是MySQL 5.7版本前的半同步複製機制。

MySQL異步複製和半同步複製怎麼實現

半同步複製要求Master事務提交過程中,至少有N個Slave接收到二進位日誌,這樣就能保證當Master發生宕機,至少有N 台Slave伺服器中的資料是完整的。

半同步複製並不是MySQL內建的功能,而是要安裝半同步插件,並啟用半同步複製功能,設定N個Slave接受二進位日誌成功。

缺點:假設user1在主庫插入了一條數據,正在等待數據返回,user2這時就能查詢到這條數據了,如果此時master掛了,user2又查不到這條數據了,產生了類似幻讀的現象。

增強半同步複製

##增強半同步複製解決了半同步複製的缺點,WAIT和ACK發生在交易提交之前,因此即使Slave沒有收到二進位日誌,但是Master宕機了,由於最後一個交易還沒提交,所以本身這個資料對外也不可見,不存在遺失的問題。

MySQL異步複製和半同步複製怎麼實現

所以,對於任何有資料一致性要求的業務,如電商的核心訂單業務、銀行、保險、證券等與資金密切相關的業務,請務必使用增強半同步複製。即使出現宕機,從機也會有完整的資料備份,因此資料將是安全且可靠的。

增強半同步複製環境是基於非同步複製的基礎上進行的。

安裝插件,建議master和slave都安裝,因為會有主從切換的情境:

# master mysql> install plugin rpl_semi_sync_master soname 'semisync_master.so'; # slave mysql> install plugin rpl_semi_sync_slave soname 'semisync_slave.so';
登入後複製

確保插件安裝成功:

mysql> show plugins; ... ... | rpl_semi_sync_master | ACTIVE | REPLICATION | semisync_master.so | GPL | | rpl_semi_sync_slave | ACTIVE | REPLICATION | semisync_slave.so | GPL | +----------------------------+----------+--------------------+--------------------+---------+ 46 rows in set (0.00 sec)
登入後複製

先啟動slave上的半同步:

mysql> set global rpl_semi_sync_slave_enabled = {0|1}; # 1:启用,0:禁止
登入後複製

再啟動master上的半同步:

mysql> set global rpl_semi_sync_master_enabled = {0|1}; # 1:启用,0:禁止 # mysql> set global rpl_semi_sync_master_timeout = 10000; # 单位为ms,默认为10s
登入後複製

從庫重啟io_thread:

mysql> stop slave io_thread; mysql> start slave io_thread;
登入後複製

半同步的參數為什麼不建議寫入設定檔

參數寫入設定檔的話,會使實例啟動後立即進入半同步模式,如果發生長時間斷連的實例重新運行啟動,有可能導致主庫被拖垮。

長時間斷開的從庫,重新連接後,要等待追完全部事務後,手動開啟半同步模式,而不是啟動後直接切換,防止衝擊主庫。

查詢主庫狀態資訊

mysql> show global status like "%semi%"; +--------------------------------------------+-------+ | Variable_name | Value | +--------------------------------------------+-------+ | Rpl_semi_sync_master_clients | 0 | | Rpl_semi_sync_master_net_avg_wait_time | 0 | | Rpl_semi_sync_master_net_wait_time | 0 | | Rpl_semi_sync_master_net_waits | 0 | | Rpl_semi_sync_master_no_times | 0 | | Rpl_semi_sync_master_no_tx | 0 | | Rpl_semi_sync_master_status | ON | | Rpl_semi_sync_master_timefunc_failures | 0 | | Rpl_semi_sync_master_tx_avg_wait_time | 0 | | Rpl_semi_sync_master_tx_wait_time | 0 | | Rpl_semi_sync_master_tx_waits | 0 | | Rpl_semi_sync_master_wait_pos_backtraverse | 0 | | Rpl_semi_sync_master_wait_sessions | 0 | | Rpl_semi_sync_master_yes_tx | 0 | | Rpl_semi_sync_slave_status | ON | +--------------------------------------------+-------+ 15 rows in set (0.00 sec)
登入後複製

重要參數說明:

  • Rpl_semi_sync_master_clients:支援與註冊半同步複製的已連Slave數

  • #Rpl_semi_sync_master_no_times:master關閉半同步複製的次數

  • Rpl_semi_sync_master_no_tx:master沒有收到slave的回應而提交的次數,可以理解為master等待超時的次數,即半同步模式不成功提交數量

  • Rpl_semi_sync_master_status:ON是活动状态(半同步),OFF是非活动状态(异步),用于表示主服务器使用的是异步复制模式,还是半同步复制模式

  • Rpl_semi_sync_master_tx_avg_wait_time:master花在每个事务上的平均等待时间

  • Rpl_semi_sync_master_tx_waits:master等待成功的次数,即master没有等待超时的次数,也就是成功提交的次数

  • Rpl_semi_sync_master_yes_tx:master成功接收到slave的回复的次数,即半同步模式成功提交数量。

查询主库参数信息

mysql> show global variables like '%sync%'; +-------------------------------------------+------------+ | Variable_name | Value | +-------------------------------------------+------------+ | binlog_group_commit_sync_delay | 0 | | binlog_group_commit_sync_no_delay_count | 0 | | innodb_flush_sync | ON | | innodb_sync_array_size | 1 | | innodb_sync_spin_loops | 30 | | rpl_semi_sync_master_enabled | ON | | rpl_semi_sync_master_timeout | 10000 | | rpl_semi_sync_master_trace_level | 32 | | rpl_semi_sync_master_wait_for_slave_count | 1 | | rpl_semi_sync_master_wait_no_slave | ON | | rpl_semi_sync_master_wait_point | AFTER_SYNC | | rpl_semi_sync_slave_enabled | ON | | rpl_semi_sync_slave_trace_level | 32 | | sync_binlog | 1 | | sync_frm | ON | | sync_master_info | 10000 | | sync_relay_log | 10000 | | sync_relay_log_info | 10000 | +-------------------------------------------+------------+ 18 rows in set (0.01 sec)
登入後複製

重要参数说明:

  • rpl_semi_sync_master_enabled:(主库)是否启动半同步

  • rpl_semi_sync_master_timeout:等待多时毫秒后变成异步复制,默认是10000ms

  • rpl_semi_sync_master_wait_point:5.7默认AFTER_SYNC(增强版半同步复制,无损复制模式),在得到slave的应答后再commit,可选值AFTER_COMMIT,在master提交后同步数据给slave,然后master等待slave应答,应答成功返回客户端。

可以在slave端执行stop slave,测试master端会发生什么情况?

在master上执行下面的sql:

mysql> insert into t_order values(3,"C");Query OK, 1 row affected (10.05 sec)
登入後複製

会发现这条语句会阻塞10s(对应上面的参数rpl_semi_sync_master_timeout),然后执行成功,最后看看看master的日志描述:

2022-01-25T02:31:57.016430Z 4 [Note] Start binlog_dump to master_thread_id(4) slave_server(1403312), pos(mysql-bin.000005, 154) 2022-01-25T02:31:57.016515Z 4 [Note] Start asynchronous binlog_dump to slave (server_id: 1403312), pos(mysql-bin.000005, 154) 2022-01-25T02:33:32.183819Z 2 [Note] Semi-sync replication initialized for transactions. 2022-01-25T02:33:32.183865Z 2 [Note] Semi-sync replication enabled on the master. 2022-01-25T02:33:32.184004Z 0 [Note] Starting ack receiver thread 2022-01-25T02:33:59.644444Z 5 [Note] While initializing dump thread for slave with UUID , found a zombie dump thread with the same UUID. Master is killing the zombie dump thread(4). 2022-01-25T02:33:59.644612Z 5 [Note] Start binlog_dump to master_thread_id(5) slave_server(1403312), pos(mysql-bin.000005, 154) 2022-01-25T02:33:59.644632Z 4 [Note] Stop asynchronous binlog_dump to slave (server_id: 1403312) 2022-01-25T02:33:59.644727Z 5 [Note] Start semi-sync binlog_dump to slave (server_id: 1403312), pos(mysql-bin.000005, 154) 2022-01-25T02:38:02.847879Z 0 [ERROR] mysqld: Got an error reading communication packets 2022-01-25T02:38:27.228952Z 2 [Warning] Timeout waiting for reply of binlog (file: mysql-bin.000005, pos: 684), semi-sync up to file mysql-bin.000005, position 419. 2022-01-25T02:38:27.229063Z 2 [Note] Semi-sync replication switched OFF. 2022-01-25T02:39:47.230189Z 5 [Note] Stop semi-sync binlog_dump to slave (server_id: 1403312)
登入後複製

可以发现半同步关闭了,变成异步模式。

以上是MySQL異步複製和半同步複製怎麼實現的詳細內容。更多資訊請關注PHP中文網其他相關文章!

相關標籤:
來源:yisu.com
本網站聲明
本文內容由網友自願投稿,版權歸原作者所有。本站不承擔相應的法律責任。如發現涉嫌抄襲或侵權的內容,請聯絡admin@php.cn
最新下載
更多>
網站特效
網站源碼
網站素材
前端模板
關於我們 免責聲明 Sitemap
PHP中文網:公益線上PHP培訓,幫助PHP學習者快速成長!