Chapter 2. LVM-VDO requirements
Chapter 2. LVM-VDO requirements VDO on LVM has certain requirements on its placement and your system resources. 2.1. VDO memory requirements Each VDO volume has two distinct memory requirements: The VDO module VDO requires a fixed 38 MB of RAM and several variable amounts: 1.15 MB of RAM for each 1 MB of configured block map cache size. The block map cache requires a minimum of 150 MB of RAM. 1.6 MB of RAM for each 1 TB of logical space. 268 MB of RAM for each 1 TB of physical storage managed by the volume. The UDS index The Universal Deduplication Service (UDS) requires a minimum of 250 MB of RAM, which is also the default amount that deduplication uses. You can configure the value when formatting a VDO volume, because the value also affects the amount of storage that the index needs. The memory required for the UDS index is determined by the index type and the required size of the deduplication window. The deduplication window is the amount of previously written data that VDO can check for matching blocks. Index type Deduplication window Dense 1 TB per 1 GB of RAM Sparse 10 TB per 1 GB of RAM Note The minimal disk usage for a VDO volume using default settings of 2 GB slab size and 0.25 dense index, requires approx 4.7 GB. This provides slightly less than 2 GB of physical data to write at 0% deduplication or compression. Here, the minimal disk usage is the sum of the default slab size and dense index. Additional resources Examples of VDO requirements by physical size 2.2. Differences between the sparse and dense index The UDS Sparse Indexing feature relies on the temporal locality of data and attempts to retain only the most relevant index entries in memory. With the sparse index, UDS can maintain a deduplication window that is ten times larger than with the dense index while using the same amount of memory. The sparse index requires ten times more physical space for metadata. Although the sparse index provides the greatest coverage, the dense index provides more deduplication advice. For most workloads, given the same amount of memory, the difference in deduplication rates between dense and sparse indexes is negligible. 2.3. VDO storage space requirements You can configure a VDO volume to use up to 256 TB of physical storage. Only a certain part of the physical storage is usable to store data. VDO requires storage for two types of VDO metadata and for the UDS index. Use the following calculations to determine the usable size of a VDO-managed volume: The first type of VDO metadata uses approximately 1 MB for each 4 GB of physical storage plus an additional 1 MB per slab. The second type of VDO metadata consumes approximately 1.25 MB for each 1 GB of logical storage , rounded up to the nearest slab. The amount of storage required for the UDS index depends on the type of index and the amount of RAM allocated to the index. For each 1 GB of RAM, a dense UDS index uses 17 GB of storage, and a sparse UDS index will use 170 GB of storage. Additional resources Examples of VDO requirements by physical size Slab size in VDO 2.4. Examples of VDO requirements by physical size The following tables provide approximate system requirements of VDO based on the physical size of the underlying volume. Each table lists requirements appropriate to the intended deployment, such as primary storage or backup storage. The exact numbers depend on your configuration of the VDO volume. Primary storage deployment In the primary storage case, the UDS index is between 0.01% to 25% the size of the physical size. Table 2.1. Examples of storage and memory configurations for primary storage Physical size RAM usage: UDS RAM usage: VDO Disk usage Index type 1 TB 250 MB 472 MB 2.5 GB Dense 10 TB 1 GB 3 GB 10 GB Dense 250 MB 22 GB Sparse 50 TB 1 GB 14 GB 85 GB Sparse 100 TB 3 GB 27 GB 255 GB Sparse 256 TB 5 GB 69 GB 425 GB Sparse Backup storage deployment In the backup storage case, the deduplication window must be larger than the backup set. If you expect the backup set or the physical size to grow in the future, factor this into the index size. Table 2.2. Examples of storage and memory configurations for backup storage Deduplication window RAM usage: UDS Disk usage Index type 1 TB 250 MB 2.5 GB Dense 10 TB 2 GB 21 GB Dense 50 TB 2 GB 170 GB Sparse 100 TB 4 GB 340 GB Sparse 256 TB 8 GB 700 GB Sparse 2.5. Placement of LVM-VDO in the storage stack You must place certain storage layers under a VDO logical volume and others on top of it. You can place thick-provisioned layers on top of VDO, but you cannot rely on the guarantees of thick provisioning in that case. Since the VDO layer is thin-provisioned, the effects of thin provisioning apply to all layers. If you do not monitor the VDO volume, you might run out of physical space on thick-provisioned volumes on top of VDO. Since the supported placement of the following layers is under VDO, do not place them on top of VDO: DM Multipath DM Crypt Software RAID (LVM or MD RAID) The following configurations are not supported: VDO on top of a loopback device Encrypted volumes on top of VDO Partitions on a VDO volume RAID, such as LVM RAID, MD RAID, or any other type, on top of a VDO volume Deploying Ceph Storage on LVM-VDO Additional resources Stacking LVM volumes (Red Hat Knowledgebase) 2.6. LVM-VDO deployment scenarios You can deploy VDO on LVM (LVM-VDO) in a variety of ways to provide deduplicated storage for: block access file access local storage remote storage Because LVM-VDO exposes its deduplicated storage as a regular logical volume (LV), you can use it with standard file systems, iSCSI and FC target drivers, or as unified storage. KVM You can deploy LVM-VDO on a KVM server configured with Direct Attached Storage. File systems You can create file systems on top of a VDO LV and expose them to NFS or CIFS users with the NFS server or Samba. iSCSI target You can export the entirety of the VDO LV as an iSCSI target to remote iSCSI initiators. Encryption Device Mapper (DM) mechanisms such as DM Crypt are compatible with VDO. Encrypting VDO LV volumes helps ensure data security, and any file systems above the VDO LV are still deduplicated. Important Applying the encryption layer above the VDO LV results in little if any data deduplication. Encryption makes duplicate blocks different before VDO can deduplicate them. Always place the encryption layer below the VDO LV.