Chapter 9. Live migration
Chapter 9. Live migration 9.1. Virtual machine live migration 9.1.1. Understanding live migration Live migration is the process of moving a running virtual machine instance (VMI) to another node in the cluster without interrupting the virtual workload or access. If a VMI uses the LiveMigrate eviction strategy, it automatically migrates when the node that the VMI runs on is placed into maintenance mode. You can also manually start live migration by selecting a VMI to migrate. Virtual machines must have a persistent volume claim (PVC) with a shared ReadWriteMany (RWX) access mode to be live migrated. Note Live migration is not supported for virtual machines that are attached to an SR-IOV network interface. 9.1.2. Updating access mode for live migration For live migration to function properly, you must use the ReadWriteMany (RWX) access mode. Use this procedure to update the access mode, if needed. Procedure To set the RWX access mode, run the following oc patch command: USD oc patch -n openshift-cnv \ cm kubevirt-storage-class-defaults \ -p '{"data":{"'USD<STORAGE_CLASS>'.accessMode":"ReadWriteMany"}}' Additional resources: Migrating a virtual machine instance to another node Live migration limiting Storage defaults for data volumes 9.2. Live migration limits and timeouts Live migration limits and timeouts are applied so that migration processes do not overwhelm the cluster. Configure these settings by editing the kubevirt-config configuration file. 9.2.1. Configuring live migration limits and timeouts Configure live migration limits and timeouts for the cluster by adding updated key:value fields to the kubevirt-config configuration file, which is located in the openshift-cnv namespace. Procedure Edit the kubevirt-config configuration file and add the necessary live migration parameters. The following example shows the default values: USD oc edit configmap kubevirt-config -n openshift-cnv Example configuration file apiVersion: v1 data: default-network-interface: masquerade feature-gates: DataVolumes,SRIOV,LiveMigration,CPUManager,CPUNodeDiscovery,Sidecar,Snapshot migrations: |- parallelMigrationsPerCluster: "5" parallelOutboundMigrationsPerNode: "2" bandwidthPerMigration: "64Mi" completionTimeoutPerGiB: "800" progressTimeout: "150" machine-type: pc-q35-rhel8.3.0 selinuxLauncherType: virt_launcher.process smbios: |- Family: Red Hat Product: Container-native virtualization Manufacturer: Red Hat Sku: 2.6.0 Version: 2.6.0 kind: ConfigMap metadata: creationTimestamp: "2021-03-26T18:01:04Z" labels: app: kubevirt-hyperconverged name: kubevirt-config namespace: openshift-cnv resourceVersion: "15371295" selfLink: /api/v1/namespaces/openshift-cnv/configmaps/kubevirt-config uid: <uuid> 9.2.2. Cluster-wide live migration limits and timeouts Table 9.1. Migration parameters Parameter Description Default parallelMigrationsPerCluster Number of migrations running in parallel in the cluster. 5 parallelOutboundMigrationsPerNode Maximum number of outbound migrations per node. 2 bandwidthPerMigration Bandwidth limit of each migration, in MiB/s. 64Mi completionTimeoutPerGiB The migration is canceled if it has not completed in this time, in seconds per GiB of memory. For example, a virtual machine instance with 6GiB memory times out if it has not completed migration in 4800 seconds. If the Migration Method is BlockMigration , the size of the migrating disks is included in the calculation. 800 progressTimeout The migration is canceled if memory copy fails to make progress in this time, in seconds. 150 9.3. Migrating a virtual machine instance to another node Manually initiate a live migration of a virtual machine instance to another node using either the web console or the CLI. 9.3.1. Initiating live migration of a virtual machine instance in the web console Migrate a running virtual machine instance to a different node in the cluster. Note The Migrate Virtual Machine action is visible to all users but only admin users can initiate a virtual machine migration. Procedure In the OpenShift Virtualization console, click Workloads Virtualization from the side menu. Click the Virtual Machines tab. You can initiate the migration from this screen, which makes it easier to perform actions on multiple virtual machines in the one screen, or from the Virtual Machine Overview screen where you can view comprehensive details of the selected virtual machine: Click the Options menu at the end of virtual machine and select Migrate Virtual Machine . Click the virtual machine name to open the Virtual Machine Overview screen and click Actions Migrate Virtual Machine . Click Migrate to migrate the virtual machine to another node. 9.3.2. Initiating live migration of a virtual machine instance in the CLI Initiate a live migration of a running virtual machine instance by creating a VirtualMachineInstanceMigration object in the cluster and referencing the name of the virtual machine instance. Procedure Create a VirtualMachineInstanceMigration configuration file for the virtual machine instance to migrate. For example, vmi-migrate.yaml : apiVersion: kubevirt.io/v1alpha3 kind: VirtualMachineInstanceMigration metadata: name: migration-job spec: vmiName: vmi-fedora Create the object in the cluster by running the following command: USD oc create -f vmi-migrate.yaml The VirtualMachineInstanceMigration object triggers a live migration of the virtual machine instance. This object exists in the cluster for as long as the virtual machine instance is running, unless manually deleted. Additional resources: Monitoring live migration of a virtual machine instance Cancelling the live migration of a virtual machine instance 9.4. Monitoring live migration of a virtual machine instance You can monitor the progress of a live migration of a virtual machine instance from either the web console or the CLI. 9.4.1. Monitoring live migration of a virtual machine instance in the web console For the duration of the migration, the virtual machine has a status of Migrating . This status is displayed in the Virtual Machines tab or in the Virtual Machine Overview screen for the migrating virtual machine. Procedure In the OpenShift Virtualization console, click Workloads Virtualization from the side menu. Click the Virtual Machines tab. Select a virtual machine to open the Virtual Machine Overview screen. 9.4.2. Monitoring live migration of a virtual machine instance in the CLI The status of the virtual machine migration is stored in the Status component of the VirtualMachineInstance configuration. Procedure Use the oc describe command on the migrating virtual machine instance: USD oc describe vmi vmi-fedora Example output ... Status: Conditions: Last Probe Time: <nil> Last Transition Time: <nil> Status: True Type: LiveMigratable Migration Method: LiveMigration Migration State: Completed: true End Timestamp: 2018-12-24T06:19:42Z Migration UID: d78c8962-0743-11e9-a540-fa163e0c69f1 Source Node: node2.example.com Start Timestamp: 2018-12-24T06:19:35Z Target Node: node1.example.com Target Node Address: 10.9.0.18:43891 Target Node Domain Detected: true 9.5. Cancelling the live migration of a virtual machine instance Cancel the live migration so that the virtual machine instance remains on the original node. You can cancel a live migration from either the web console or the CLI. 9.5.1. Cancelling live migration of a virtual machine instance in the web console You can cancel a live migration of the virtual machine instance using the Options menu found on each virtual machine in the Virtualization Virtual Machines tab, or from the Actions menu available on all tabs in the Virtual Machine Overview screen. Procedure In the OpenShift Virtualization console, click Workloads Virtualization from the side menu. Click the Virtual Machines tab. You can cancel the migration from this screen, which makes it easier to perform actions on multiple virtual machines, or from the Virtual Machine Overview screen where you can view comprehensive details of the selected virtual machine: Click the Options menu at the end of virtual machine and select Cancel Virtual Machine Migration . Select a virtual machine name to open the Virtual Machine Overview screen and click Actions Cancel Virtual Machine Migration . Click Cancel Migration to cancel the virtual machine live migration. 9.5.2. Cancelling live migration of a virtual machine instance in the CLI Cancel the live migration of a virtual machine instance by deleting the VirtualMachineInstanceMigration object associated with the migration. Procedure Delete the VirtualMachineInstanceMigration object that triggered the live migration, migration-job in this example: USD oc delete vmim migration-job 9.6. Configuring virtual machine eviction strategy The LiveMigrate eviction strategy ensures that a virtual machine instance is not interrupted if the node is placed into maintenance or drained. Virtual machines instances with this eviction strategy will be live migrated to another node. 9.6.1. Configuring custom virtual machines with the LiveMigration eviction strategy You only need to configure the LiveMigration eviction strategy on custom virtual machines. Common templates have this eviction strategy configured by default. Procedure Add the evictionStrategy: LiveMigrate option to the spec.template.spec section in the virtual machine configuration file. This example uses oc edit to update the relevant snippet of the VirtualMachine configuration file: USD oc edit vm <custom-vm> -n <my-namespace> apiVersion: kubevirt.io/v1alpha3 kind: VirtualMachine metadata: name: custom-vm spec: template: spec: evictionStrategy: LiveMigrate ... Restart the virtual machine for the update to take effect: USD virtctl restart <custom-vm> -n <my-namespace>
oc patch -n openshift-cnv cm kubevirt-storage-class-defaults -p '{"data":{"'USD<STORAGE_CLASS>'.accessMode":"ReadWriteMany"}}' oc edit configmap kubevirt-config -n openshift-cnv apiVersion: v1 data: default-network-interface: masquerade feature-gates: DataVolumes,SRIOV,LiveMigration,CPUManager,CPUNodeDiscovery,Sidecar,Snapshot migrations: |- parallelMigrationsPerCluster: "5" parallelOutboundMigrationsPerNode: "2" bandwidthPerMigration: "64Mi" completionTimeoutPerGiB: "800" progressTimeout: "150" machine-type: pc-q35-rhel8.3.0 selinuxLauncherType: virt_launcher.process smbios: |- Family: Red Hat Product: Container-native virtualization Manufacturer: Red Hat Sku: 2.6.0 Version: 2.6.0 kind: ConfigMap metadata: creationTimestamp: "2021-03-26T18:01:04Z" labels: app: kubevirt-hyperconverged name: kubevirt-config namespace: openshift-cnv resourceVersion: "15371295" selfLink: /api/v1/namespaces/openshift-cnv/configmaps/kubevirt-config uid: <uuid> apiVersion: kubevirt.io/v1alpha3 kind: VirtualMachineInstanceMigration metadata: name: migration-job spec: vmiName: vmi-fedora oc create -f vmi-migrate.yaml oc describe vmi vmi-fedora Status: Conditions: Last Probe Time: <nil> Last Transition Time: <nil> Status: True Type: LiveMigratable Migration Method: LiveMigration Migration State: Completed: true End Timestamp: 2018-12-24T06:19:42Z Migration UID: d78c8962-0743-11e9-a540-fa163e0c69f1 Source Node: node2.example.com Start Timestamp: 2018-12-24T06:19:35Z Target Node: node1.example.com Target Node Address: 10.9.0.18:43891 Target Node Domain Detected: true oc delete vmim migration-job oc edit vm <custom-vm> -n <my-namespace> apiVersion: kubevirt.io/v1alpha3 kind: VirtualMachine metadata: name: custom-vm spec: template: spec: evictionStrategy: LiveMigrate virtctl restart <custom-vm> -n <my-namespace>