By Kappa Category linux
Tags thp memory benchmarking
Estimated Read Time: 5 min read
Ninja Level:
Photo credit: Kimberly Farmer (Unsplash license)
Introduction of THP
A typical page in Linux memory management system is 4KB. To enhance the efficiency of allocating large amount of memory pages, the OS could make use of huge memory pages. Instead of using 4KB as a page, the OS could allocate a bigger block of page size in 2MB or 1GB. To use huge memory pages, the OS need to be configured so that the memory is preserved or allocated. Refer to the system documentation for details.
Transparent Huge Pages (THP) is a memory subsystem technology to use huge memory pages without the need of pre-configuration. As far as I know, it is enabled by default in most Linux distribution like RedHat Enterprise Linux, SUSE Enterprise Linux, Debian and etc. However, if you are using Oracle, Couchbase and some products, they requested the system to disable THP explicitly.
To check if a current system has used any THP:
1 2 | $ grep AnonHugePages /proc/meminfo AnonHugePages: 3610624 kB |
So, the system has used about 3.6GB of memory for the THP. If you want to know which processes have used THP, you could refer to this article.
A small experiment
In this experiment, I would allocate a file of 15GB in /tmp inside a Linux KVM guest. For modern Linux, /tmp or tmpfs is backed by SHM (POSIX shared memory), so this would test the speed of memory allocation. We could enable or disable THP in the host and guest independently.
Here's is my testing result. I run each test three times.
| Host THP status | KVM guest THP status | First run | Second run | Third run |
|---|---|---|---|---|
| Disabled | Disabled | 14.1 seconds | 5.43 seconds | 5.36 seconds |
| Disabled | Enabled | 4.9 seconds | 4.8 seconds | 4.8 seconds |
| Enabled | Disabled | 6.0 seconds | 5.37 seconds | 5.3 seconds |
| Enabled | Enabled | 4.69 seconds | 4.75 seconds | 4.67 seconds |
So, let's take a look on the testing result. If both the host and KVM guest had disable THP, the KVM guest would take the longest time to allocate memory on the first run. Either enabling THP in the host or KVM guest would enhance the memory allocation efficiency. The memory allocation performance would be most efficient if both host and KVM guest enabled THP.