8.3. Kernel Same-page Merging (KSM)
8.3. Kernel Same-page Merging (KSM) Kernel same-page Merging (KSM), used by the KVM hypervisor, allows KVM guests to share identical memory pages. These shared pages are usually common libraries or other identical, high-use data. KSM allows for greater guest density of identical or similar guest operating systems by avoiding memory duplication. The concept of shared memory is common in modern operating systems. For example, when a program is first started, it shares all of its memory with the parent program. When either the child or parent program tries to modify this memory, the kernel allocates a new memory region, copies the original contents and allows the program to modify this new region. This is known as copy on write. KSM is a Linux feature which uses this concept in reverse. KSM enables the kernel to examine two or more already running programs and compare their memory. If any memory regions or pages are identical, KSM reduces multiple identical memory pages to a single page. This page is then marked copy on write. If the contents of the page is modified by a guest virtual machine, a new page is created for that guest. This is useful for virtualization with KVM. When a guest virtual machine is started, it only inherits the memory from the host qemu-kvm process. Once the guest is running, the contents of the guest operating system image can be shared when guests are running the same operating system or applications. KSM allows KVM to request that these identical guest memory regions be shared. KSM provides enhanced memory speed and utilization. With KSM, common process data is stored in cache or in main memory. This reduces cache misses for the KVM guests, which can improve performance for some applications and operating systems. Secondly, sharing memory reduces the overall memory usage of guests, which allows for higher densities and greater utilization of resources. Note In Red Hat Enterprise Linux 7, KSM is NUMA aware. This allows it to take NUMA locality into account while coalescing pages, thus preventing performance drops related to pages being moved to a remote node. Red Hat recommends avoiding cross-node memory merging when KSM is in use. If KSM is in use, change the /sys/kernel/mm/ksm/merge_across_nodes tunable to 0 to avoid merging pages across NUMA nodes. This can be done with the virsh node-memory-tune --shm-merge-across-nodes 0 command. Kernel memory accounting statistics can eventually contradict each other after large amounts of cross-node merging. As such, numad can become confused after the KSM daemon merges large amounts of memory. If your system has a large amount of free memory, you may achieve higher performance by turning off and disabling the KSM daemon. See Chapter 9, NUMA " for more information on NUMA. Important Ensure the swap size is sufficient for the committed RAM even without taking KSM into account. KSM reduces the RAM usage of identical or similar guests. Overcommitting guests with KSM without sufficient swap space may be possible, but is not recommended because guest virtual machine memory use can result in pages becoming unshared. Red Hat Enterprise Linux uses two separate methods for controlling KSM: The ksm service starts and stops the KSM kernel thread. The ksmtuned service controls and tunes the ksm service, dynamically managing same-page merging. ksmtuned starts the ksm service and stops the ksm service if memory sharing is not necessary. When new guests are created or destroyed, ksmtuned must be instructed with the retune parameter to run. Both of these services are controlled with the standard service management tools. Note KSM is off by default on Red Hat Enterprise Linux 6.7. 8.3.1. The KSM Service The ksm service is included in the qemu-kvm package. When the ksm service is not started, Kernel same-page merging (KSM) shares only 2000 pages. This default value provides limited memory-saving benefits. When the ksm service is started, KSM will share up to half of the host system's main memory. Start the ksm service to enable KSM to share more memory. The ksm service can be added to the default startup sequence. Make the ksm service persistent with the systemctl command. 8.3.2. The KSM Tuning Service The ksmtuned service fine-tunes the kernel same-page merging (KSM) configuration by looping and adjusting ksm . In addition, the ksmtuned service is notified by libvirt when a guest virtual machine is created or destroyed. The ksmtuned service has no options. The ksmtuned service can be tuned with the retune parameter, which instructs ksmtuned to run tuning functions manually. The /etc/ksmtuned.conf file is the configuration file for the ksmtuned service. The file output below is the default ksmtuned.conf file: Within the /etc/ksmtuned.conf file, npages sets how many pages ksm will scan before the ksmd daemon becomes inactive. This value will also be set in the /sys/kernel/mm/ksm/pages_to_scan file. The KSM_THRES_CONST value represents the amount of available memory used as a threshold to activate ksm . ksmd is activated if either of the following occurs: The amount of free memory drops below the threshold, set in KSM_THRES_CONST . The amount of committed memory plus the threshold, KSM_THRES_CONST , exceeds the total amount of memory. 8.3.3. KSM Variables and Monitoring Kernel same-page merging (KSM) stores monitoring data in the /sys/kernel/mm/ksm/ directory. Files in this directory are updated by the kernel and are an accurate record of KSM usage and statistics. The variables in the list below are also configurable variables in the /etc/ksmtuned.conf file, as noted above. Files in /sys/kernel/mm/ksm/ : full_scans Full scans run. merge_across_nodes Whether pages from different NUMA nodes can be merged. pages_shared Total pages shared. pages_sharing Pages currently shared. pages_to_scan Pages not scanned. pages_unshared Pages no longer shared. pages_volatile Number of volatile pages. run Whether the KSM process is running. sleep_millisecs Sleep milliseconds. These variables can be manually tuned using the virsh node-memory-tune command. For example, the following specifies the number of pages to scan before the shared memory service goes to sleep: KSM tuning activity is stored in the /var/log/ksmtuned log file if the DEBUG=1 line is added to the /etc/ksmtuned.conf file. The log file location can be changed with the LOGFILE parameter. Changing the log file location is not advised and may require special configuration of SELinux settings. 8.3.4. Deactivating KSM Kernel same-page merging (KSM) has a performance overhead which may be too large for certain environments or host systems. KSM may also introduce side channels that could be potentially used to leak information across guests. If this is a concern, KSM can be disabled on per-guest basis. KSM can be deactivated by stopping the ksmtuned and the ksm services. However, this action does not persist after restarting. To deactivate KSM, run the following in a terminal as root: Stopping the ksmtuned and the ksm deactivates KSM, but this action does not persist after restarting. Persistently deactivate KSM with the systemctl commands: When KSM is disabled, any memory pages that were shared prior to deactivating KSM are still shared. To delete all of the PageKSM in the system, use the following command: After this is performed, the khugepaged daemon can rebuild transparent hugepages on the KVM guest physical memory. Using # echo 0 >/sys/kernel/mm/ksm/run stops KSM, but does not unshare all the previously created KSM pages (this is the same as the # systemctl stop ksmtuned command).
systemctl start ksm Starting ksm: [ OK ] systemctl enable ksm systemctl start ksmtuned Starting ksmtuned: [ OK ] Configuration file for ksmtuned. How long ksmtuned should sleep between tuning adjustments KSM_MONITOR_INTERVAL=60 Millisecond sleep between ksm scans for 16Gb server. Smaller servers sleep more, bigger sleep less. KSM_SLEEP_MSEC=10 KSM_NPAGES_BOOST - is added to the `npages` value, when `free memory` is less than `thres`. KSM_NPAGES_BOOST=300 KSM_NPAGES_DECAY - is the value given is subtracted to the `npages` value, when `free memory` is greater than `thres`. KSM_NPAGES_DECAY=-50 KSM_NPAGES_MIN - is the lower limit for the `npages` value. KSM_NPAGES_MIN=64 KSM_NPAGES_MAX - is the upper limit for the `npages` value. KSM_NPAGES_MAX=1250 KSM_THRES_COEF - is the RAM percentage to be calculated in parameter `thres`. KSM_THRES_COEF=20 KSM_THRES_CONST - If this is a low memory system, and the `thres` value is less than `KSM_THRES_CONST`, then reset `thres` value to `KSM_THRES_CONST` value. KSM_THRES_CONST=2048 uncomment the following to enable ksmtuned debug information LOGFILE=/var/log/ksmtuned DEBUG=1 virsh node-memory-tune --shm-pages-to-scan number systemctl stop ksmtuned Stopping ksmtuned: [ OK ] systemctl stop ksm Stopping ksm: [ OK ] systemctl disable ksm systemctl disable ksmtuned echo 2 >/sys/kernel/mm/ksm/run