10.3. Preparing to Deploy Geo-replication
10.3. Preparing to Deploy Geo-replication This section provides an overview of geo-replication deployment scenarios, lists prerequisites, and describes how to setup the environment for geo-replication session. Section 10.3.1, "Exploring Geo-replication Deployment Scenarios" Section 10.3.2, "Geo-replication Deployment Overview" Section 10.3.3, "Prerequisites" Section 10.3.4.2, "Setting Up your Environment for a Secure Geo-replication Slave" Section 10.3.4.1, "Setting Up your Environment for Geo-replication Session" Section 10.3.5, "Configuring a Meta-Volume" 10.3.1. Exploring Geo-replication Deployment Scenarios Geo-replication provides an incremental replication service over Local Area Networks (LANs), Wide Area Network (WANs), and the Internet. This section illustrates the most common deployment scenarios for geo-replication, including the following: Geo-replication over LAN Geo-replication over WAN Geo-replication over the Internet Multi-site cascading geo-replication Geo-replication over LAN Geo-replication over WAN Geo-replication over Internet Multi-site cascading Geo-replication 10.3.2. Geo-replication Deployment Overview Deploying geo-replication involves the following steps: Verify that your environment matches the minimum system requirements. See Section 10.3.3, "Prerequisites" . Determine the appropriate deployment scenario. See Section 10.3.1, "Exploring Geo-replication Deployment Scenarios" . Start geo-replication on the master and slave systems. For manual method, see Section 10.4, "Starting Geo-replication" . For gdeploy method, see Starting a geo-replication session in Section 10.5.3, "Controlling geo-replication sessions using gdeploy" . 10.3.3. Prerequisites The following are prerequisites for deploying geo-replication: Note that these prerequisites only need to be carried out once from one cluster to another cluster, so if you are syncing multiple volumes from the same master cluster to the same slave cluster, you need only perform these prerequisites once. The master and slave volumes must use the same version of Red Hat Gluster Storage. Nodes in the slave volume must not be part of the master volume. Two separate trusted storage pools are required. Disable the performance.quick-read option in the slave volume using the following command: Time must be synchronized between all master and slave nodes before geo-replication is configured. Red Hat recommends setting up a network time protocol service to keep time synchronized between bricks and servers, and avoid out-of-time synchronization errors. See Network Time Protocol Setup for more information. Add the required port for geo-replication from the ports listed in the Section 3.1.2, "Port Access Requirements" . Key-based SSH authentication without a password is required between one node of the master volume (the node from which the geo-replication create command will be executed), and one node of the slave volume (the node whose IP/hostname will be mentioned in the slave name when running the geo-replication create command). Create the public and private keys using ssh-keygen (without passphrase) on the master node: Copy the public key to the slave node using the following command: If you are setting up a non-root geo-replicaton session, then copy the public key to the respective user location. Note - Key-based SSH authentication without a password is only required from the master node to the slave node; the slave node does not need this level of access. - ssh-copy-id command does not work if ssh authorized_keys file is configured in the custom location. You must copy the contents of .ssh/id_rsa.pub file from the Master and paste it to authorized_keys file in the custom location on the Slave node. Gsyncd also requires key-based SSH authentication without a password between every node in the master cluster to every node in the slave cluster. The gluster system:: execute gsec_create command creates secret-pem files on all the nodes in the master, and is used to implement the SSH authentication connection. The push-pem option in the geo-replication create command pushes these keys to all slave nodes. For more information on the gluster system::execute gsec_create and push-pem commands, see Section 10.3.4.1, "Setting Up your Environment for Geo-replication Session" . 10.3.4. Setting Up your Environment You can set up your environment for a geo-replication session in the following ways: Section 10.3.4.1, "Setting Up your Environment for Geo-replication Session" - In this method, the slave mount is owned by the root user. Section 10.3.4.2, "Setting Up your Environment for a Secure Geo-replication Slave" - This method is more secure as the slave mount is owned by a normal user. 10.3.4.1. Setting Up your Environment for Geo-replication Session Creating Geo-replication Sessions To create a common pem pub file, run the following command on the master node where the key-based SSH authentication connection is configured: Alternatively, you can create the pem pub file by running the following command on the master node where the key-based SSH authentication connection is configured. This alternate command generates Geo-rep session specific ssh-keys in all the master nodes and collects public keys from all peer nodes. It also provides a detailed view of the command status. Create the geo-replication session using the following command. The push-pem option is needed to perform the necessary pem-file setup on the slave nodes. For example: Note There must be key-based SSH authentication access between the node from which this command is run, and the slave host specified in the above command. This command performs the slave verification, which includes checking for a valid slave URL, valid slave volume, and available space on the slave. If the verification fails, you can use the force option which will ignore the failed verification and create a geo-replication session. The slave volume is in read-only mode by default. However, in case of a failover-failback situation, the original master is made read-only by default as the session is from the original slave to the original master. Enable shared storage for master and slave volumes: For more information on shared storage, see Section 11.12, "Setting up Shared Storage Volume" . Configure the meta-volume for geo-replication: For example: For more information on configuring meta-volume, see Section 10.3.5, "Configuring a Meta-Volume" . Start the geo-replication by running the following command on the master node: For example, Verify the status of the created session by running the following command: 10.3.4.2. Setting Up your Environment for a Secure Geo-replication Slave Geo-replication supports access to Red Hat Gluster Storage slaves through SSH using an unprivileged account (user account with non-zero UID). This method is more secure and it reduces the master's capabilities over slave to the minimum. This feature relies on mountbroker , an internal service of glusterd which manages the mounts for unprivileged slave accounts. You must perform additional steps to configure glusterd with the appropriate mountbroker's access control directives. The following example demonstrates this process: Perform the following steps on all the Slave nodes to setup an auxiliary glusterFS mount for the unprivileged account : In all the slave nodes, create a new group. For example, geogroup . Note You must not use multiple groups for the mountbroker setup. You can create multiple user accounts but the group should be same for all the non-root users. In all the slave nodes, create a unprivileged account. For example, geoaccount . Add geoaccount as a member of geogroup group. On any one of the Slave nodes, run the following command to set up mountbroker root directory and group. For example, On any one of the Slave nodes, run the following commands to add volume and user to the mountbroker service. For example, Check the status of the setup by running the following command: The output displays the mountbroker status for every peer node in the slave cluster. Restart glusterd service on all the Slave nodes. After you setup an auxiliary glusterFS mount for the unprivileged account on all the Slave nodes, perform the following steps to setup a non-root geo-replication session. : Setup key-based SSH authentication from one of the master nodes to the user on one of the slave nodes. For example, to setup key-based SSH authentication to the user geoaccount . Create a common pem pub file by running the following command on the master nodes, where the key-based SSH authentication connection is configured to the user on the slave nodes: Create a geo-replication relationship between the master and the slave to the user by running the following command on the master node: For example, If you have multiple slave volumes and/or multiple accounts, create a geo-replication session with that particular user and volume. For example, Enable shared storage for master and slave volumes: For more information on shared storage, see Section 11.12, "Setting up Shared Storage Volume" . On the slave node, which is used to create relationship, run /usr/libexec/glusterfs/set_geo_rep_pem_keys.sh as a root with user name, master volume name, and slave volume names as the arguments. For example, Configure the meta-volume for geo-replication: For example: For more information on configuring meta-volume, see Section 10.3.5, "Configuring a Meta-Volume" . Start the geo-replication with slave user by running the following command on the master node: For example, Verify the status of geo-replication session by running the following command on the master node: Deleting a mountbroker geo-replication options after deleting session After mountbroker geo-replicaton session is deleted, you must remove the volumes per mountbroker user. Important You must first stop and delete the geo-replication session before you can delete volumes from the mountbroker. For more information, see Section 10.4.5, "Stopping a Geo-replication Session" and Section 10.4.6, "Deleting a Geo-replication Session" To remove the volumes per mountbroker user: For example, If the volume to be removed is the last one for the mountbroker user, the user is also removed. Important If you have a secured geo-replication setup, you must ensure to prefix the unprivileged user account to the slave volume in the command. For example, to execute a geo-replication status command, run the following: In this command, geoaccount is the name of the unprivileged user account. 10.3.5. Configuring a Meta-Volume Meta-volume aka gluster_shared_storage is the gluster volume used for internal purposes. Setting use_meta_volume to true enables geo-replication to use shared volume in order to store lock file(s) which helps in handling worker fail-overs. For effective handling of node fail-overs in Master volume, geo-replication requires this shared storage to be available across all nodes of the cluster. Hence, ensure that a gluster volume named gluster_shared_storage is created in the cluster, and is mounted at /var/run/gluster/shared_storage on all the nodes in the cluster. For more information on setting up shared storage volume, see Section 11.12, "Setting up Shared Storage Volume" . Note With the release of 3.5 Batch Update 3, the mount point of shared storage is changed from /var/run/gluster/ to /run/gluster/ . Configure the meta-volume for geo-replication: For example: Important On RHEL 8, ensure the booleans rsync_full_access on and rsync_client on booleans are set to ON to prevent file permission issues during rsync required by geo-replication.
gluster volume set slavevol performance.quick-read off ssh-keygen ssh-copy-id -i identity_file root@slave_node_IPaddress/Hostname gluster system:: execute gsec_create gluster-georep-sshkey generate +--------------+-------------+---------------+ | NODE | NODE STATUS | KEYGEN STATUS | +--------------+-------------+---------------+ | node1 | UP | OK | | node2 | UP | OK | | node3 | UP | OK | | node4 | UP | OK | | node5 | UP | OK | | localhost | UP | OK | +--------------+-------------+---------------+ gluster volume geo-replication MASTER_VOL SLAVE_HOST :: SLAVE_VOL create push-pem [force] gluster volume geo-replication Volume1 storage.backup.com::slave-vol create push-pem gluster volume set all cluster.enable-shared-storage enable gluster volume geo-replication MASTER_VOL SLAVE_HOST::SLAVE_VOL config use_meta_volume true gluster volume geo-replication Volume1 storage.backup.com::slave-vol config use_meta_volume true gluster volume geo-replication MASTER_VOL SLAVE_HOST :: SLAVE_VOL start [force] gluster volume geo-replication MASTER_VOL SLAVE_HOST :: SLAVE_VOL status gluster-mountbroker setup <MOUNT ROOT> <GROUP> gluster-mountbroker setup /var/mountbroker-root geogroup gluster-mountbroker add <VOLUME> <USER> gluster-mountbroker add slavevol geoaccount gluster-mountbroker status NODE NODE STATUS MOUNT ROOT GROUP USERS --------------------------------------------------------------------------------------- localhost UP /var/mountbroker-root(OK) geogroup(OK) geoaccount(slavevol) node2 UP /var/mountbroker-root(OK) geogroup(OK) geoaccount(slavevol) service glusterd restart ssh-keygen ssh-copy-id -i identity_file geoaccount@slave_node_IPaddress/Hostname gluster system:: execute gsec_create gluster volume geo-replication MASTERVOL geoaccount@SLAVENODE::slavevol create push-pem gluster volume geo-replication MASTERVOL geoaccount2@SLAVENODE::slavevol2 create push-pem gluster volume set all cluster.enable-shared-storage enable /usr/libexec/glusterfs/set_geo_rep_pem_keys.sh geoaccount MASTERVOL SLAVEVOL_NAME gluster volume geo-replication MASTER_VOL SLAVE_HOST::SLAVE_VOL config use_meta_volume true gluster volume geo-replication Volume1 storage.backup.com::slave-vol config use_meta_volume true gluster volume geo-replication MASTERVOL geoaccount@SLAVENODE::slavevol start gluster volume geo-replication MASTERVOL geoaccount@SLAVENODE::slavevol status gluster-mountbroker remove [--volume volume ] [--user user ] gluster-mountbroker remove --volume slavevol --user geoaccount gluster-mountbroker remove --user geoaccount gluster-mountbroker remove --volume slavevol gluster volume geo-replication MASTERVOL geoaccount@SLAVENODE::slavevol status gluster volume geo-replication MASTER_VOL SLAVE_HOST::SLAVE_VOL config use_meta_volume true gluster volume geo-replication Volume1 storage.backup.com::slave-vol config use_meta_volume true