Chapter 20. Using NVDIMM persistent memory storage
Chapter 20. Using NVDIMM persistent memory storage You can enable and manage various types of storage on Non-Volatile Dual In-line Memory Modules (NVDIMM) devices connected to your system. For installing Red Hat Enterprise Linux 9 on NVDIMM storage, see Installing to an NVDIMM device instead. 20.1. The NVDIMM persistent memory technology Non-Volatile Dual In-line Memory Modules (NVDIMM) persistent memory, also called storage class memory or pmem , is a combination of memory and storage. NVDIMM combines the durability of storage with the low access latency and the high bandwidth of dynamic RAM (DRAM). The following are the other advantages of using NVDIMM: NVDIMM storage is byte-addressable, which means it can be accessed by using the CPU load and store instructions. In addition to the read() and write() system calls, which are required for accessing traditional block-based storage, NVDIMM also supports direct load and a store programming model. The performance characteristics of NVDIMM are similar to DRAM with very low access latency, typically in the tens to hundreds of nanoseconds. Data stored on NVDIMM is preserved when the power is off, similar to a persistent memory. With the direct access (DAX) technology, applications to memory map storage directly are possible without going through the system page cache. This frees up DRAM for other purposes. NVDIMM is beneficial in use cases such as: Databases The reduced storage access latency on NVDIMM improves database performance. Rapid restart Rapid restart is also called the warm cache effect. For example, a file server has none of the file contents in memory after starting. As clients connect and read or write data, that data is cached in the page cache. Eventually, the cache contains mostly hot data. After a reboot, the system must start the process again on traditional storage. With NVDIMM, it is possible for an application to keep the warm cache across reboots if the application is designed properly. In this example, there would be no page cache involved: the application would cache data directly in the persistent memory. Fast write-cache File servers often do not acknowledge a client write request until the data is on durable media. Using NVDIMM as a fast write-cache, enables a file server to acknowledge the write request quickly, and results in low latency. 20.2. NVDIMM interleaving and regions Non-Volatile Dual In-line Memory Modules (NVDIMM) devices support grouping into interleaved regions. NVDIMM devices can be grouped into interleave sets in the same way as regular dynamic RAM (DRAM). An interleave set is similar to a RAID 0 level (stripe) configuration across multiple DIMMs. An Interleave set is also called a region. Interleaving has the following advantages: NVDIMM devices benefit from increased performance when they are configured into interleave sets. Interleaving can combine multiple smaller NVDIMM devices into a larger logical device. NVDIMM interleave sets are configured in the system BIOS or UEFI firmware. Red Hat Enterprise Linux creates one region device for each interleave set. 20.3. NVDIMM namespaces Non-Volatile Dual In-line Memory Modules (NVDIMM) regions can be divided into one or more namespaces depending on the size of the label area. Using namespaces, you can access the device using different methods, based on the access modes of the namespace such as sector , fsdax , devdax , and raw . For more information, NVDIMM access modes . Some NVDIMM devices do not support multiple namespaces on a region: If your NVDIMM device supports labels, you can subdivide the region into namespaces. If your NVDIMM device does not support labels, the region can only contain a single namespace. In that case, Red Hat Enterprise Linux creates a default namespace that covers the entire region. 20.4. NVDIMM access modes You can configure Non-Volatile Dual In-line Memory Modules (NVDIMM) namespaces to use either of the following modes: sector Presents the storage as a fast block device. This mode is useful for legacy applications that are not modified to use NVDIMM storage, or for applications that use the full I/O stack, including Device Mapper. A sector device can be used in the same way as any other block device on the system. You can create partitions or file systems on it, configure it as part of a software RAID set, or use it as the cache device for dm-cache . Devices in this mode are available as /dev/pmem N s . After creating the namespace, see the listed blockdev value. devdax , or device direct access (DAX) With devdax , NVDIMM devices support direct access programming as described in the Storage Networking Industry Association (SNIA) Non-Volatile Memory (NVM) Programming Model specification. In this mode, I/O bypasses the storage stack of the kernel. Therefore, no Device Mapper drivers can be used. Device DAX provides raw access to NVDIMM storage by using a DAX character device node. Data on a devdax device can be made durable using CPU cache flushing and fencing instructions. Certain databases and virtual machine hypervisors might benefit from this mode. File systems cannot be created on devdax devices. Devices in this mode are available as /dev/dax N . M . After creating the namespace, see the listed chardev value. fsdax , or file system direct access (DAX) With fsdax , NVDIMM devices support direct access programming as described in the Storage Networking Industry Association (SNIA) Non-Volatile Memory (NVM) Programming Model specification. In this mode, I/O bypasses the storage stack of the kernel, and many Device Mapper drivers therefore cannot be used. You can create file systems on file system DAX devices. Devices in this mode are available as /dev/pmem N . After creating the namespace, see the listed blockdev value. raw Presents a memory disk that does not support DAX. In this mode, namespaces have several limitations and should not be used. Devices in this mode are available as /dev/pmem N . After creating the namespace, see the listed blockdev value. 20.5. Installing ndctl You can install the ndctl utility to configure and monitor Non-Volatile Dual In-line Memory Modules (NVDIMM) devices. Procedure Install the ndctl utility: 20.6. Creating a sector namespace on an NVDIMM to act as a block device You can configure a Non-Volatile Dual In-line Memory Modules (NVDIMM) device in sector mode, also called legacy mode, to support traditional, block-based storage. You can either: reconfigure an existing namespace to sector mode, or create a new sector namespace if there is available space. Prerequisites An NVDIMM device is attached to your system. 20.6.1. Reconfiguring an existing NVDIMM namespace to sector mode You can reconfigure an Non-Volatile Dual In-line Memory Modules (NVDIMM) namespace to sector mode for using it as a fast block device. Warning Reconfiguring a namespace deletes previously stored data on the namespace. Prerequisites The ndctl utility is installed. For more information, see Installing ndctl . Procedure View the existing namespaces: Reconfigure the selected namespace to the sector mode: Example 20.1. Reconfiguring namespace1.0 in sector mode The reconfigured namespace is now available under the /dev directory as the /dev/pmem1s file. Verification Verify if the existing namespace on your system is reconfigured: Additional resources ndctl-create-namespace(1) man page on your system 20.6.2. Creating a new NVDIMM namespace in sector mode You can create a Non-Volatile Dual In-line Memory Modules (NVDIMM) namespace in sector mode for using it as a fast block device if there is available space in the region. Prerequisites The ndctl utility is installed. For more information, see Installing ndctl . The NVDIMM device supports labels to create multiple namespaces in a region. You can check this using the following command: This indicates that it read the label of one NVDIMM device. If the value is 0 , it implies that your device does not support labels. Procedure List the pmem regions on your system that have available space. In the following example, space is available in the region1 and region0 regions: Allocate one or more namespaces on any of the available regions: Example 20.2. Creating a 36-GiB sector namespace on region0 The new namespace is now available as /dev/pmem0.1s . Verification Verify if the new namespace is created in the sector mode: Additional resources ndctl-create-namespace(1) man page on your system 20.7. Creating a device DAX namespace on an NVDIMM Configure the NVDIMM device that is attached to your system, in device DAX mode to support character storage with direct access capabilities. Consider the following options: Reconfiguring an existing namespace to device DAX mode. Creating a new device DAX namespace, if there is space available. 20.7.1. NVDIMM in device direct access mode Device direct access (device DAX, devdax ) provides a means for applications to directly access storage, without the involvement of a file system. The benefit of device DAX is that it provides a guaranteed fault granularity, which can be configured using the --align option of the ndctl utility. For the Intel 64 and AMD64 architecture, the following fault granularities are supported: 4 KiB 2 MiB 1 GiB Device DAX nodes support only the following system calls: open() close() mmap() You can view the supported alignments of your NVDIMM device using the ndctl list --human --capabilities command. For example, to view it for the region0 device, use the ndctl list --human --capabilities -r region0 command. Note The read() and write() system calls are not supported because the device DAX use case is tied to the SNIA Non-Volatile Memory Programming Model. 20.7.2. Reconfiguring an existing NVDIMM namespace to device DAX mode You can reconfigure an existing Non-Volatile Dual In-line Memory Modules (NVDIMM) namespace to device DAX mode. Warning Reconfiguring a namespace deletes previously stored data on the namespace. Prerequisites The ndctl utility is installed. For more information, see Installing ndctl . Procedure List all namespaces on your system: Reconfigure any namespace: Example 20.3. Reconfiguring a namespace as device DAX The following command reconfigures namespace0.1 for data storage that supports DAX. It is aligned to a 2-MiB fault granularity to ensure that the operating system faults in 2-MiB pages at a time: The namespace is now available at the /dev/dax0.1 path. Verification Verify if the existing namespaces on your system is reconfigured: Additional resources ndctl-create-namespace(1) man page on your system 20.7.3. Creating a new NVDIMM namespace in device DAX mode You can create a new device DAX namespace on an Non-Volatile Dual In-line Memory Modules (NVDIMM) device if there is available space in the region. Prerequisites The ndctl utility is installed. For more information, see Installing ndctl . The NVDIMM device supports labels to create multiple namespaces in a region. You can check this using the following command: This indicates that it read the label of one NVDIMM device. If the value is 0 , it implies that your device does not support labels. Procedure List the pmem regions on your system that have available space. In the following example, space is available in the region1 and region0 regions: Allocate one or more namespaces on any of the available regions: Example 20.4. Creating a namespace on a region The following command creates a 36-GiB device DAX namespace on region0. It is aligned to a 2-MiB fault granularity to ensure that the operating system faults in 2-MiB pages at a time: The namespace is now available as /dev/dax0.2 . Verification Verify if the new namespace is created in device DAX mode: Additional resources ndctl-create-namespace(1) man page on your system 20.8. Creating a file system DAX namespace on an NVDIMM Configure an NVDIMM device that is attached to your system, in file system DAX mode to support a file system with direct access capabilities. Consider the following options: Reconfiguring an existing namespace to file system DAX mode. Creating a new file system DAX namespace if there is available space. 20.8.1. NVDIMM in file system direct access mode When an NVDIMM device is configured in file system direct access (file system DAX, fsdax ) mode, you can create a file system on top of it. Any application that performs an mmap() operation on a file on this file system gets direct access to its storage. This enables the direct access programming model on NVDIMM. The following new -o dax options are now available, and direct access behavior can be controlled through a file attribute if required: -o dax=inode This is the default option when you do not specify any dax option while mounting a file system. Using this option, you can set an attribute flag on files to control if the dax mode can be activated. If required, you can set this flag on individual files. You can also set this flag on a directory and any files in that directory will be created with the same flag. You can set this attribute flag by using the xfs_io -c 'chattr +x' directory-name command. -o dax=never With this option, the dax mode will not be enabled even if the dax flag is set to an inode mode. This means that the per-inode dax attribute flag is ignored, and files set with this flag will never be direct-access enabled. -o dax=always This option is equivalent to the old -o dax behavior. With this option, you can activate direct access mode for any file on the file system, regardless of the dax attribute flag. Warning In further releases, -o dax might not be supported and if required, you can use -o dax=always instead. In this mode, every file might be in the direct-access mode. Per-page metadata allocation This mode requires allocating per-page metadata in the system DRAM or on the NVDIMM device itself. The overhead of this data structure is 64 bytes per each 4-KiB page: On small devices, the amount of overhead is small enough to fit in DRAM with no problems. For example, a 16-GiB namespace only requires 256 MiB for page structures. Since NVDIMM devices are usually small and expensive, storing the page tracking data structures in DRAM is preferable. On NVDIMM devices that are be terabytes in size or larger, the amount of memory required to store the page tracking data structures might exceed the amount of DRAM in the system. One TiB of NVDIMM requires 16 GiB for page structures. As a result, storing the data structures on the NVDIMM itself is preferable in such cases. You can configure where per-page metadata are stored using the --map option when configuring a namespace: To allocate in the system RAM, use --map=mem . To allocate on the NVDIMM, use --map=dev . 20.8.2. Reconfiguring an existing NVDIMM namespace to file system DAX mode You can reconfigure an existing Non-Volatile Dual In-line Memory Modules (NVDIMM) namespace to file system DAX mode. Warning Reconfiguring a namespace deletes previously stored data on the namespace. Prerequisites The ndctl utility is installed. For more information, see Installing ndctl . Procedure List all namespaces on your system: Reconfigure any namespace: Example 20.5. Reconfiguring a namespace as file system DAX To use namespace0.0 for a file system that supports DAX, use the following command: The namespace is now available at the /dev/pmem0 path. Verification Verify if the existing namespaces on your system is reconfigured: Additional resources ndctl-create-namespace(1) man page on your system 20.8.3. Creating a new NVDIMM namespace in file system DAX mode You can create a new file system DAX namespace on an Non-Volatile Dual In-line Memory Modules (NVDIMM) device if there is available space in the region. Prerequisites The ndctl utility is installed. For more information, see Installing ndctl . The NVDIMM device supports labels to create multiple namespaces in a region. You can check this using the following command: This indicates that it read the label of one NVDIMM device. If the value is 0 , it implies that your device does not support labels. Procedure List the pmem regions on your system that have available space. In the following example, space is available in the region1 and region0 regions: Allocate one or more namespaces on any of the available regions: Example 20.6. Creating a namespace on a region The following command creates a 36-GiB file system DAX namespace on region0 : The namespace is now available as /dev/pmem0.3 . Verification Verify if the new namespace is created in file system DAX mode: Additional resources ndctl-create-namespace(1) man page on your system 20.8.4. Creating a file system on a file system DAX device You can create a file system on a file system DAX device and mount the file system. After creating a file system, application can use persistent memory and create files in the mount-point directory, open the files, and use the mmap operation to map the files for direct access. Procedure Optional: Create a partition on the file system DAX device. For more information, see Creating a partition with parted . Note When creating partitions on an fsdax device, partitions must be aligned on page boundaries. On the Intel 64 and AMD64 architecture, at least 4 KiB alignment is required for the start and end of the partition. 2 MiB is the preferred alignment. By default, the parted tool aligns partitions on 1 MiB boundaries. For the first partition, specify 2 MiB as the start of the partition. If the size of the partition is a multiple of 2 MiB, all other partitions are also aligned. Create an XFS or ext4 file system on the partition or the NVDIMM device: Note The dax-capable and reflinked files can now co-exist on the file system. However, for an individual file, dax and reflink are mutually exclusive. Also, in order to increase the likelihood of large page mappings, set the stripe unit and stripe width. Mount the file system: There is no need to mount a file system with the dax option to enable direct access mode. When you do not specify any dax option while mounting, the file system is in the dax=inode mode. Set the dax option on the file before direct access mode is activated. Additional resources mkfs.xfs(8) man page on your system NVDIMM in file system direct access mode 20.9. Monitoring NVDIMM health using S.M.A.R.T. Some Non-Volatile Dual In-line Memory Modules (NVDIMM) devices support Self-Monitoring, Analysis and Reporting Technology (S.M.A.R.T.) interfaces for retrieving health information. Important Monitor NVDIMM health regularly to prevent data loss. If S.M.A.R.T. reports problems with the health status of an NVDIMM device, replace it as described in Detecting and replacing a broken NVDIMM device . Prerequisites Optional: On some systems, upload the acpi_ipmi driver to retrieve health information using the following command: Procedure Access the health information: Additional resources ndctl-list(1) man page on your system 20.10. Detecting and replacing a broken NVDIMM device If you find error messages related to Non-Volatile Dual In-line Memory Modules (NVDIMM) reported in your system log or by S.M.A.R.T., it might mean an NVDIMM device is failing. In that case, it is necessary to: Detect which NVDIMM device is failing Back up data stored on it Physically replace the device Procedure Detect the broken device: Find the phys_id attribute of the broken NVDIMM: From the example, you know that nmem0 is the broken NVDIMM. Therefore, find the phys_id attribute of nmem0 . Example 20.7. The phys_id attributes of NVDIMMs In the following example, the phys_id is 0x10 : Find the memory slot of the broken NVDIMM: In the output, find the entry where the Handle identifier matches the phys_id attribute of the broken NVDIMM. The Locator field lists the memory slot used by the broken NVDIMM. Example 20.8. NVDIMM Memory Slot Listing In the following example, the nmem0 device matches the 0x0010 identifier and uses the DIMM-XXX-YYYY memory slot: Back up all data in the namespaces on the NVDIMM. If you do not back up the data before replacing the NVDIMM, the data will be lost when you remove the NVDIMM from your system. Warning In some cases, such as when the NVDIMM is completely broken, the backup might fail. To prevent this, regularly monitor your NVDIMM devices using S.M.A.R.T. as described in Monitoring NVDIMM health using S.M.A.R.T. and replace failing NVDIMMs before they break. List the namespaces on the NVDIMM: Example 20.9. NVDIMM namespaces listing In the following example, the nmem0 device contains the namespace0.0 and namespace0.2 namespaces, which you need to back up: Replace the broken NVDIMM physically. Additional resources ndctl-list(1) and dmidecode(8) man pages on your system
dnf install ndctl ndctl list --namespaces --idle [ { "dev":"namespace1.0", "mode":"raw", "size":34359738368, "state":"disabled", "numa_node":1 }, { "dev":"namespace0.0", "mode":"raw", "size":34359738368, "state":"disabled", "numa_node":0 } ] ndctl create-namespace --force --reconfig= namespace-ID --mode=sector ndctl create-namespace --force --reconfig=namespace1.0 --mode=sector { "dev":"namespace1.0", "mode":"sector", "size":"755.26 GiB (810.95 GB)", "uuid":"2509949d-1dc4-4ee0-925a-4542b28aa616", "sector_size":4096, "blockdev":"pmem1s" } ndctl list --namespace namespace1.0 [ { "dev":"namespace1.0", "mode":"sector", "size":810954706944, "uuid":"2509949d-1dc4-4ee0-925a-4542b28aa616", "sector_size":4096, "blockdev":"pmem1s" } ] ndctl read-labels nmem0 >/dev/null read 1 nmem ndctl list --regions [ { "dev":"region1", "size":2156073582592, "align":16777216, "available_size":2117418876928, "max_available_extent":2117418876928, "type":"pmem", "iset_id":-9102197055295954944, "badblock_count":1, "persistence_domain":"memory_controller" }, { "dev":"region0", "size":2156073582592, "align":16777216, "available_size":2143188680704, "max_available_extent":2143188680704, "type":"pmem", "iset_id":736272362787276936, "badblock_count":3, "persistence_domain":"memory_controller" } ] ndctl create-namespace --mode=sector --region=region N --size=namespace-size ndctl create-namespace --mode=sector --region=region0 --size=36G { "dev":"namespace0.1", "mode":"sector", "size":"35.96 GiB (38.62 GB)", "uuid":"ff5a0a16-3495-4ce8-b86b-f0e3bd9d1817", "sector_size":4096, "blockdev":"pmem0.1s" } ndctl list -RN -n namespace0.1 { "regions":[ { "dev":"region0", "size":2156073582592, "align":16777216, "available_size":2104533975040, "max_available_extent":2104533975040, "type":"pmem", "iset_id":736272362787276936, "badblock_count":3, "persistence_domain":"memory_controller", "namespaces":[ { "dev":"namespace0.1", "mode":"sector", "size":38615912448, "uuid":"ff5a0a16-3495-4ce8-b86b-f0e3bd9d1817", "sector_size":4096, "blockdev":"pmem0.1s" } ] } ] } ndctl list --namespaces --idle [ { "dev":"namespace1.0", "mode":"raw", "size":34359738368, "uuid":"ac951312-b312-4e76-9f15-6e00c8f2e6f4" "state":"disabled", "numa_node":1 }, { "dev":"namespace0.0", "mode":"raw", "size":38615912448, "uuid":"ff5a0a16-3495-4ce8-b86b-f0e3bd9d1817", "state":"disabled", "numa_node":0 } ] ndctl create-namespace --force --mode=devdax --reconfig= namespace-ID ndctl create-namespace --force --mode=devdax --align=2M --reconfig=namespace0.1 { "dev":"namespace0.1", "mode":"devdax", "map":"dev", "size":"35.44 GiB (38.05 GB)", "uuid":"426d6a52-df92-43d2-8cc7-046241d6d761", "daxregion":{ "id":0, "size":"35.44 GiB (38.05 GB)", "align":2097152, "devices":[ { "chardev":"dax0.1", "size":"35.44 GiB (38.05 GB)", "target_node":4, "mode":"devdax" } ] }, "align":2097152 } ndctl list --namespace namespace0.1 [ { "dev":"namespace0.1", "mode":"devdax", "map":"dev", "size":38048628736, "uuid":"426d6a52-df92-43d2-8cc7-046241d6d761", "chardev":"dax0.1", "align":2097152 } ] ndctl read-labels nmem0 >/dev/null read 1 nmem ndctl list --regions [ { "dev":"region1", "size":2156073582592, "align":16777216, "available_size":2117418876928, "max_available_extent":2117418876928, "type":"pmem", "iset_id":-9102197055295954944, "badblock_count":1, "persistence_domain":"memory_controller" }, { "dev":"region0", "size":2156073582592, "align":16777216, "available_size":2143188680704, "max_available_extent":2143188680704, "type":"pmem", "iset_id":736272362787276936, "badblock_count":3, "persistence_domain":"memory_controller" } ] ndctl create-namespace --mode=devdax --region=region N --size= namespace-size ndctl create-namespace --mode=devdax --region=region0 --align=2M --size=36G { "dev":"namespace0.2", "mode":"devdax", "map":"dev", "size":"35.44 GiB (38.05 GB)", "uuid":"89d13f41-be6c-425b-9ec7-1e2a239b5303", "daxregion":{ "id":0, "size":"35.44 GiB (38.05 GB)", "align":2097152, "devices":[ { "chardev":"dax0.2", "size":"35.44 GiB (38.05 GB)", "target_node":4, "mode":"devdax" } ] }, "align":2097152 } ndctl list -RN -n namespace0.2 { "regions":[ { "dev":"region0", "size":2156073582592, "align":16777216, "available_size":2065879269376, "max_available_extent":2065879269376, "type":"pmem", "iset_id":736272362787276936, "badblock_count":3, "persistence_domain":"memory_controller", "namespaces":[ { "dev":"namespace0.2", "mode":"devdax", "map":"dev", "size":38048628736, "uuid":"89d13f41-be6c-425b-9ec7-1e2a239b5303", "chardev":"dax0.2", "align":2097152 } ] } ] } ndctl list --namespaces --idle [ { "dev":"namespace1.0", "mode":"raw", "size":34359738368, "uuid":"ac951312-b312-4e76-9f15-6e00c8f2e6f4" "state":"disabled", "numa_node":1 }, { "dev":"namespace0.0", "mode":"raw", "size":38615912448, "uuid":"ff5a0a16-3495-4ce8-b86b-f0e3bd9d1817", "state":"disabled", "numa_node":0 } ] ndctl create-namespace --force --mode=fsdax --reconfig= namespace-ID ndctl create-namespace --force --mode=fsdax --reconfig=namespace0.0 { "dev":"namespace0.0", "mode":"fsdax", "map":"dev", "size":"11.81 GiB (12.68 GB)", "uuid":"f8153ee3-c52d-4c6e-bc1d-197f5be38483", "sector_size":512, "align":2097152, "blockdev":"pmem0" } ndctl list --namespace namespace0.0 [ { "dev":"namespace0.0", "mode":"fsdax", "map":"dev", "size":12681478144, "uuid":"f8153ee3-c52d-4c6e-bc1d-197f5be38483", "sector_size":512, "align":2097152, "blockdev":"pmem0" } ] ndctl read-labels nmem0 >/dev/null read 1 nmem ndctl list --regions [ { "dev":"region1", "size":2156073582592, "align":16777216, "available_size":2117418876928, "max_available_extent":2117418876928, "type":"pmem", "iset_id":-9102197055295954944, "badblock_count":1, "persistence_domain":"memory_controller" }, { "dev":"region0", "size":2156073582592, "align":16777216, "available_size":2143188680704, "max_available_extent":2143188680704, "type":"pmem", "iset_id":736272362787276936, "badblock_count":3, "persistence_domain":"memory_controller" } ] ndctl create-namespace --mode=fsdax --region=region N --size= namespace-size ndctl create-namespace --mode=fsdax --region=region0 --size=36G { "dev":"namespace0.3", "mode":"fsdax", "map":"dev", "size":"35.44 GiB (38.05 GB)", "uuid":"99e77865-42eb-4b82-9db6-c6bc9b3959c2", "sector_size":512, "align":2097152, "blockdev":"pmem0.3" } ndctl list -RN -n namespace0.3 { "regions":[ { "dev":"region0", "size":2156073582592, "align":16777216, "available_size":2027224563712, "max_available_extent":2027224563712, "type":"pmem", "iset_id":736272362787276936, "badblock_count":3, "persistence_domain":"memory_controller", "namespaces":[ { "dev":"namespace0.3", "mode":"fsdax", "map":"dev", "size":38048628736, "uuid":"99e77865-42eb-4b82-9db6-c6bc9b3959c2", "sector_size":512, "align":2097152, "blockdev":"pmem0.3" } ] } ] } mkfs.xfs -d su=2m,sw=1 fsdax-partition-or-device mount f_sdax-partition-or-device mount-point_ modprobe acpi_ipmi ndctl list --dimms --health [ { "dev":"nmem1", "id":"8089-a2-1834-00001f13", "handle":17, "phys_id":32, "security":"disabled", "health":{ "health_state":"ok", "temperature_celsius":36.0, "controller_temperature_celsius":37.0, "spares_percentage":100, "alarm_temperature":false, "alarm_controller_temperature":false, "alarm_spares":false, "alarm_enabled_media_temperature":true, "temperature_threshold":82.0, "alarm_enabled_ctrl_temperature":true, "controller_temperature_threshold":98.0, "alarm_enabled_spares":true, "spares_threshold":50, "shutdown_state":"clean", "shutdown_count":4 } }, [...] ] ndctl list --dimms --regions --health { "dimms":[ { "dev":"nmem1", "id":"8089-a2-1834-00001f13", "handle":17, "phys_id":32, "security":"disabled", "health":{ "health_state":"ok", "temperature_celsius":35.0, [...] } [...] } ndctl list --dimms --human ndctl list --dimms --human [ { "dev":"nmem1", "id":"XXXX-XX-XXXX-XXXXXXXX", "handle":"0x120", "phys_id":"0x1c" }, { "dev":"nmem0", "id":"XXXX-XX-XXXX-XXXXXXXX", "handle":"0x20", "phys_id":"0x10", "flag_failed_flush":true, "flag_smart_event":true } ] dmidecode dmidecode Handle 0x0010, DMI type 17, 40 bytes Memory Device Array Handle: 0x0004 Error Information Handle: Not Provided Total Width: 72 bits Data Width: 64 bits Size: 125 GB Form Factor: DIMM Set: 1 Locator: DIMM-XXX-YYYY Bank Locator: Bank0 Type: Other Type Detail: Non-Volatile Registered (Buffered) ndctl list --namespaces --dimm= DIMM-ID-number ndctl list --namespaces --dimm=0 [ { "dev":"namespace0.2", "mode":"sector", "size":67042312192, "uuid":"XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX", "raw_uuid":"XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX", "sector_size":4096, "blockdev":"pmem0.2s", "numa_node":0 }, { "dev":"namespace0.0", "mode":"sector", "size":67042312192, "uuid":"XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX", "raw_uuid":"XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX", "sector_size":4096, "blockdev":"pmem0s", "numa_node":0 } ]