One 、Orchestrator brief introduction

Orchestrator It's an open source , Yes MySQL Replication provides high availability 、 Visual topology management tool , use go Language writing , It can actively discover the current topology and master-slave replication state , Support MySQL The adjustment of master slave replication topology 、 Support MySQL Automatic failover of main database (failover)、 Manual master-slave switching (switchover) And so on .

Orchestrator The backstage depends on MySQL perhaps SQLite Store metadata , Can provide Web Interface display MySQL Cluster topology and instance status , adopt Web The interface can be changed MySQL Some configuration information of the instance , Command line and api Interface , In order to more flexible automatic operation and maintenance management .Orchestrator Yes MySQL The main database is divided into automatic switchover and manual switchover . Manual switching is divided into recover、force-master-failover、force-master-takeover as well as graceful-master-takeover.

Compared with MHA,Orchestrator More emphasis on the management of replication topology , Can achieve MySQL Adjustment of any replication topology , And on this basis , Realization MySQL High availability . in addition ,Orchestrator You can also deploy multiple nodes on your own , adopt raft Distributed consistency protocol , Ensure your own high availability .

Orchestrator It has the following characteristics :

  • Automatically monitor the structure and status of database replication
  • Provides GUI,CLI,API Check the status of the replication topology and do some adjustment operations
  • Support for automatic master failover, When copying the structure of server After hanging up ( Whether it's manual or automatic ), Be able to reshape the replication topology
  • Not dependent on specific server Version or branch (MySQL, Percona Server, MariaDB or even MaxScale binlog servers)
  • Supports multiple types of topologies , Whether it's a single master-slave or hundreds of them server The multi-level replication of the composition is no problem
  • its GUI Not just to you report Topology state , You can also do it in Orchestrator web The page changes the replication topology by dragging or deleting nodes (CLI and API Can also do )

Orchestrator Of GitHub Address :

https://github.com/openark/orchestrator

https://github.com/outbrain/orchestrator/wiki/Orchestrator-Manual

https://github.com/github/orchestrator/tree/master/docs

Orchestrator All parameters of :

https://github.com/github/orchestrator/blob/master/go/config/config.go

An example of an official suggested production configuration :

https://github.com/github/orchestrator/blob/master/docs/configuration-sample.md

Two 、 Quickly build Orchestrator Environmental Science

--  Pull the mirror image of the wheat seedling
docker pull registry.cn-hangzhou.aliyuncs.com/lhrbest/lhrorchestrator:1.0
--  Create a container
docker run -d --name lhrorchestrator -h lhrorchestrator \
  -p 233:22 -p 3087:3389 -p 3000:3000 \
  --privileged=true lhrbest/lhrorchestrator:1.0 \
  /usr/sbin/init
  
  
--  Into the container  
docker exec -it lhrorchestrator bash
--  Start back end storage MySQL library
systemctl status mysqld
MySQL User name and password :root/lhr
--  start-up orchestrator
systemctl start orchestrator
systemctl status orchestrator
-- web visit :
http://192.168.66.35:3000
User name and password :lhr/lhr
--  journal :
tailf /var/log/messages
--  Parameter file
/usr/local/orchestrator/orchestrator.conf.json

web The interface is as follows :

3、 ... and 、 Add a master-slave environment

Add the configured replication instance to the orchestrator, because orch Automatically discover all instances of the entire topology , So just add any instance .

here , I have a set 1 Lord 2 From our environment , The topology is as follows :

[root@docker35 log]# mysqlrplshow --master=root:lhr@172.72.0.2:3306 --discover-slaves-login=root:lhr --verbose
WARNING: Using a password on the command line interface can be insecure.
# master on 172.72.0.2: ... connected.
# Finding slaves for master: 172.72.0.2:3306
# Replication Topology Graph
172.72.0.2:3306 (MASTER)
   |
   +--- 172.72.0.3:3306 [IO: Yes, SQL: Yes] - (SLAVE)
   |
   +--- 172.72.0.4:3306 [IO: Yes, SQL: Yes] - (SLAVE)

Let's start adding this environment to orchestrator in :

Add success .

Let's add a dual master and dual slave environment :

Choose any one MySQL Server run Discover operation , When I'm done , Refresh the page as follows :

Four 、 Modify the cluster alias

After the modification is completed , Refresh the page :

Be careful : Cluster alias does not support Chinese .

5、 ... and 、 Drag the interface to modify the master-slave relationship

Manually modify the master-slave relationship :

6、 ... and 、 Automatically failover

To achieve automatic failover, Need modification 2 Parameters :

[root@lhrorchestrator orchestrator]# more orchestrator.conf.json | grep  ClusterFilters
  "RecoverMasterClusterFilters": ["*"],
  "RecoverIntermediateMasterClusterFilters": ["*"],

this 2 A parameter defines which clusters can be automatically failover and recover.

Automatically failover Before :

Take down the main library :

[root@lhrorchestrator /]# mysql -uroot -plhr -h172.72.0.2 -P3306
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 11357
Server version: 5.7.30-log MySQL Community Server (GPL)
Copyright (c) 2000, 2020, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> shutdown ;
Query OK, 0 rows affected (0.00 sec)

Refresh the page :

172.72.0.2 Automatically eliminated , The new main library becomes 172.72.0.3.

172.72.0.4 Points to the new main library 172.72.0.3.

172.72.0.2 Down , The border is black :

Next launch 172.72.0.2, After startup :

We need to join the cluster manually :

[root@docker35 log]#  mysql -uroot -plhr -h172.72.0.2 -P3306
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 13
Server version: 5.7.30-log MySQL Community Server (GPL)
Copyright (c) 2000, 2020, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> show slave hosts;
Empty set (0.00 sec)
mysql> show slave status;
Empty set (0.00 sec)
mysql> change master to
    -> master_host='172.72.0.3',
    -> master_port=3306,
    -> master_user='repl',
    -> master_password='lhr',
    -> master_auto_position=1;
show slave status \G;
Query OK, 0 rows affected, 1 warning (0.02 sec)
mysql> 
mysql> start slave; 
Query OK, 0 rows affected (0.01 sec)
mysql> show slave status \G;
*************************** 1. row ***************************
               Slave_IO_State: Checking master version
                  Master_Host: 172.72.0.3
                  Master_User: repl
                  Master_Port: 3306
                Connect_Retry: 60
              Master_Log_File: 
          Read_Master_Log_Pos: 4
               Relay_Log_File: master-relay-bin.000001
                Relay_Log_Pos: 4
        Relay_Master_Log_File: 
             Slave_IO_Running: Yes
            Slave_SQL_Running: Yes
              Replicate_Do_DB: 
          Replicate_Ignore_DB: 
           Replicate_Do_Table: 
       Replicate_Ignore_Table: 
      Replicate_Wild_Do_Table: 
  Replicate_Wild_Ignore_Table: 
                   Last_Errno: 0
                   Last_Error: 
                 Skip_Counter: 0
          Exec_Master_Log_Pos: 0
              Relay_Log_Space: 154
              Until_Condition: None
               Until_Log_File: 
                Until_Log_Pos: 0
           Master_SSL_Allowed: No
           Master_SSL_CA_File: 
           Master_SSL_CA_Path: 
              Master_SSL_Cert: 
            Master_SSL_Cipher: 
               Master_SSL_Key: 
        Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
                Last_IO_Errno: 0
                Last_IO_Error: 
               Last_SQL_Errno: 0
               Last_SQL_Error: 
  Replicate_Ignore_Server_Ids: 
             Master_Server_Id: 0
                  Master_UUID: 
             Master_Info_File: mysql.slave_master_info
                    SQL_Delay: 0
          SQL_Remaining_Delay: NULL
      Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates
           Master_Retry_Count: 86400
                  Master_Bind: 
      Last_IO_Error_Timestamp: 
     Last_SQL_Error_Timestamp: 
               Master_SSL_Crl: 
           Master_SSL_Crlpath: 
           Retrieved_Gtid_Set: 
            Executed_Gtid_Set: bacfcde8-69dc-11eb-b240-0242ac480002:1-70574,
bc6d26f0-69dc-11eb-bfa7-0242ac480003:1-5
                Auto_Position: 1
         Replicate_Rewrite_DB: 
                 Channel_Name: 
           Master_TLS_Version: 
1 row in set (0.00 sec)
ERROR: 
No query specified
mysql>

Refresh the page :

You can see , Back to the one Lord 2 From our environment .