Chapter 3. Live migration of images
Chapter 3. Live migration of images As a storage administrator, you can live-migrate RBD images between different pools or even with the same pool, within the same storage cluster. You can migrate between different images formats and layouts and even from external data sources. When live migration is initiated, the source image is deep copied to the destination image, pulling all snapshot history while preserving the sparse allocation of data where possible. Important Currently, the krbd kernel module does not support live migration. Prerequisites A running Red Hat Ceph Storage cluster. 3.1. The live migration process By default, during the live migration of the RBD images with the same storage cluster, the source image is marked read-only. All clients redirect the Input/Output (I/O) to the new target image. Additionally, this mode can preserve the link to the source image's parent to preserve sparseness, or it can flatten the image during the migration to remove the dependency on the source image's parent. You can use the live migration process in an import-only mode, where the source image remains unmodified. You can link the target image to an external data source, such as a backup file, HTTP(s) file, or an S3 object. The live migration copy process can safely run in the background while the new target image is being used. The live migration process consists of three steps: Prepare Migration : The first step is to create new target image and link the target image to the source image. If the import-only mode is not configured, the source image will also be linked to the target image and marked read-only. Attempts to read uninitialized data extents within the target image will internally redirect the read to the source image, and writes to uninitialized extents within the target image will internally deep copy, the overlapping source image extents to the target image. Execute Migration : This is a background operation that deep-copies all initialized blocks from the source image to the target. You can run this step when clients are actively using the new target image. Finish Migration : You can commit or abort the migration, once the background migration process is completed. Committing the migration removes the cross-links between the source and target images, and will remove the source image if not configured in the import-only mode. Aborting the migration remove the cross-links, and will remove the target image. 3.2. Formats You can use the native format to describe a native RBD image within a Red Hat Ceph Storage cluster as the source image. The source-spec JSON document is encoded as: Syntax Note that the native format does not include the stream object since it utilizes native Ceph operations. For example, to import from the image rbd/ns1/image1@snap1 , the source-spec could be encoded as: Example You can use the qcow format to describe a QEMU copy-on-write (QCOW) block device. Both the QCOW v1 and v2 formats are currently supported with the exception of advanced features such as compression, encryption, backing files, and external data files. You can link the qcow format data to any supported stream source: Example You can use the raw format to describe a thick-provisioned, raw block device export that is rbd export -export-format 1 SNAP_SPEC . You can link the raw format data to any supported stream source: Example The inclusion of the snapshots array is optional and currently only supports thick-provisioned raw snapshot exports. 3.3. Streams File stream You can use the file stream to import from a locally accessible POSIX file source. Syntax For example, to import a raw-format image from a file located at /mnt/image.raw , the source-spec JSON file is: Example HTTP stream You can use the HTTP stream to import from a remote HTTP or HTTPS web server. Syntax For example, to import a raw-format image from a file located at http://download.ceph.com/image.raw , the source-spec JSON file is: Example S3 stream You can use the s3 stream to import from a remote S3 bucket. Syntax For example, to import a raw-format image from a file located at http://s3.ceph.com/bucket/image.raw , its source-spec JSON is encoded as follows: Example 3.4. Preparing the live migration process You can prepare the default live migration process for RBD images within the same Red Hat Ceph Storage cluster. The rbd migration prepare command accepts all the same layout options as the rbd create command. The rbd create command allows changes to the on-disk layout of the immutable image. If you only want to change the on-disk layout and want to keep the original image name, skip the migration_target argument. All clients using the source image must be stopped before preparing a live migration. The prepare step will fail if it finds any running clients with the image open in read/write mode. You can restart the clients using the new target image once the prepare step is completed. Note You cannot restart the clients using the source image as it will result in a failure. Prerequisites A running Red Hat Ceph Storage cluster. Two block device pools. One block device image. Procedure Prepare the live migration within the storage cluster: Syntax Example OR If you want to rename the source image: Syntax Example In the example, newsourceimage1 is the renamed source image. You can check the current state of the live migration process with the following command: Syntax Example Important During the migration process, the source image is moved into the RBD trash to prevent mistaken usage. Example Example 3.5. Preparing import-only migration You can initiate the import-only live migration process by running the rbd migration prepare command with the --import-only and either, --source-spec or --source-spec-path options, passing a JSON document that describes how to access the source image data directly on the command line or from a file. Prerequisites A running Red Hat Ceph Storage cluster. A bucket and an S3 object are created. Procedure Create a JSON file: Example Prepare the import-only live migration process: Syntax Example Note The rbd migration prepare command accepts all the same image options as the rbd create command. You can check the status of the import-only live migration: Example 3.6. Executing the live migration process After you prepare for the live migration, you must copy the image blocks from the source image to the target image. Prerequisites A running Red Hat Ceph Storage cluster. Two block device pools. One block device image. Procedure Execute the live migration: Syntax Example You can check the feedback on the progress of the migration block deep-copy process: Syntax Example 3.7. Committing the live migration process You can commit the migration, once the live migration has completed deep-copying all the data blocks from the source image to the target image. Prerequisites A running Red Hat Ceph Storage cluster. Two block device pools. One block device image. Procedure Commit the migration, once deep-copying is completed: Syntax Example Verification Committing the live migration will remove the cross-links between the source and target images, and also removes the source image from the source pool: Example 3.8. Aborting the live migration process You can revert the live migration process. Aborting live migration reverts the prepare and execute steps. Note You can abort only if you have not committed the live migration. Prerequisites A running Red Hat Ceph Storage cluster. Two block device pools. One block device image. Procedure Abort the live migration process: Syntax Example Verification When the live migration process is aborted, the target image is deleted and access to the original source image is restored in the source pool: Example
{ "type": "native", "pool_name": " POOL_NAME ", ["pool_id": " POOL_ID ",] (optional, alternative to " POOL_NAME " key) ["pool_namespace": " POOL_NAMESPACE ",] (optional) "image_name": " IMAGE_NAME >", ["image_id": " IMAGE_ID ",] (optional, useful if image is in trash) "snap_name": " SNAP_NAME ", ["snap_id": " SNAP_ID ",] (optional, alternative to " SNAP_NAME " key) } { "type": "native", "pool_name": "rbd", "pool_namespace": "ns1", "image_name": "image1", "snap_name": "snap1" } { "type": "qcow", "stream": { "type": "file", "file_path": "/mnt/image.qcow" } } { "type": "raw", "stream": { "type": "file", "file_path": "/mnt/image-head.raw" }, "snapshots": [ { "type": "raw", "name": "snap1", "stream": { "type": "file", "file_path": "/mnt/image-snap1.raw" } }, ] (optional oldest to newest ordering of snapshots) } { <format unique parameters> "stream": { "type": "file", "file_path": " FILE_PATH " } } { "type": "raw", "stream": { "type": "file", "file_path": "/mnt/image.raw" } } { <format unique parameters> "stream": { "type": "http", "url": " URL_PATH " } } { "type": "raw", "stream": { "type": "http", "url": "http://download.ceph.com/image.raw" } } { <format unique parameters> "stream": { "type": "s3", "url": " URL_PATH ", "access_key": " ACCESS_KEY ", "secret_key": " SECRET_KEY " } } { "type": "raw", "stream": { "type": "s3", "url": "http://s3.ceph.com/bucket/image.raw", "access_key": "NX5QOQKC6BH2IDN8HC7A", "secret_key": "LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn" } } rbd migration prepare SOURCE_POOL_NAME / SOURCE_IMAGE_NAME TARGET_POOL_NAME / SOURCE_IMAGE_NAME rbd migration prepare sourcepool1/sourceimage1 targetpool1/sourceimage1 rbd migration prepare SOURCE_POOL_NAME / SOURCE_IMAGE_NAME TARGET_POOL_NAME / NEW_SOURCE_IMAGE_NAME rbd migration prepare sourcepool1/sourceimage1 targetpool1/newsourceimage1 rbd status TARGET_POOL_NAME / SOURCE_IMAGE_NAME rbd status targetpool1/sourceimage1 Watchers: none Migration: source: sourcepool1/sourceimage1 (adb429cb769a) destination: targetpool2/testimage1 (add299966c63) state: prepared rbd info sourceimage1 rbd: error opening image sourceimage1: (2) No such file or directory rbd trash ls --all sourcepool1 adb429cb769a sourceimage1 cat testspec.json { "type": "raw", "stream": { "type": "s3", "url": "http:10.74.253.18:80/testbucket1/image.raw", "access_key": "RLJOCP6345BGB38YQXI5", "secret_key": "oahWRB2ote2rnLy4dojYjDrsvaBADriDDgtSfk6o" } rbd migration prepare --import-only --source-spec-path " JSON_FILE " TARGET_POOL_NAME rbd migration prepare --import-only --source-spec-path "testspec.json" targetpool1 rbd status targetpool1/sourceimage1 Watchers: none Migration: source: {"stream":{"access_key":"RLJOCP6345BGB38YQXI5","secret_key":"oahWRB2ote2rnLy4dojYjDrsvaBADriDDgtSfk6o","type":"s3","url":"http://10.74.253.18:80/testbucket1/image.raw"},"type":"raw"} destination: targetpool1/sourceimage1 (b13865345e66) state: prepared rbd migration execute TARGET_POOL_NAME / SOURCE_IMAGE_NAME rbd migration execute targetpool1/sourceimage1 Image migration: 100% complete...done. rbd status TARGET_POOL_NAME / SOURCE_IMAGE_NAME rbd status targetpool1/sourceimage1 Watchers: none Migration: source: sourcepool1/testimage1 (adb429cb769a) destination: targetpool1/testimage1 (add299966c63) state: executed rbd migration commit TARGET_POOL_NAME / SOURCE_IMAGE_NAME rbd migration commit targetpool1/sourceimage1 Commit image migration: 100% complete...done. rbd trash list --all sourcepool1 rbd migration abort TARGET_POOL_NAME / SOURCE_IMAGE_NAME rbd migration abort targetpool1/sourceimage1 Abort image migration: 100% complete...done. rbd ls sourcepool1 sourceimage1