неділя, 6 січня 2013 р.

HugePages on Oracle Linux 64-bit


Applies to:

Linux OS - Version Enterprise Linux 4.0 to Oracle Linux 6.0 with Unbreakable Enterprise Kernel [2.6.32] [Release RHEL4 to OL6]
Oracle Server - Enterprise Edition - Version 9.2.0.1 and later
Linux x86-64
Oracle Linux
Red Hat Enterprise Linux (RHEL)
SUSE Linux Enterprise Server (SLES)

Why Do You Need HugePages?

HugePages is crucial for faster Oracle database performance on Linux if you have a large RAM and SGA. If your combined database SGAs is large (like more than 8GB, can even be important for smaller), you will need HugePages configured. Note that the size of the SGA matters. Advantages of HugePages are:
  • Larger Page Size and Less # of Pages: Default page size is 4K whereas the HugeTLB size is 2048K. That means the system would need to handle 512 times less pages.
  • No Page Table Lookups: Since the HugePages are not subject to replacement (despite regular pages), page table lookups are not required.
  • Better Overall Memory Performance: On virtual memory systems (any modern OS) each memory operation is actually two abstract memory operations. With HugePages, since there are less number of pages to work on, the possible bottleneck on page table access is clearly avoided.
  • No Swapping: We must avoid swapping to happen on Linux OS at all Document 1295478.1. HugePages are not swappable (whereas regular pages are). Therefore there is no page replacement mechanism overhead. HugePages are universally regarded as pinned.
  • No 'kswapd' Operations: kswapd will get very busy if there is a very large area to be paged (i.e. 13 million page table entries for 50GB memory) and will use an incredible amount of CPU resource. When HugePages are used, kswapd is not involved in managing them. See also Document 361670.1 

How to Configure

The configuration steps below will guide you to do a persistent system configuration where you would need to do a complete reboot of the system. Please plan your operations accordingly:
Step 1: Have the memlock user limit set in /etc/security/limits.conf file. Set the value (in KB) slightly smaller than installed RAM. e.g. If you have 64GB RAM installed, you may set:
*   soft   memlock    60397977
*   hard   memlock    60397977
There is no harm in setting this value large than your SGA requirements.

The parameters will be set by default on:
  • Oracle Linux with oracle-validated package (See Document 437743.1) installed.
  • Oracle Exadata DB compute nodes
$ ulimit -l
60397977
Step 3: If you have Oracle Database 11g or later, the default database created uses the Automatic Memory Management (AMM) feature which is incompatible with HugePages. Disable AMM before proceeding. To disable, set the initialization parameters MEMORY_TARGET and MEMORY_MAX_TARGET to 0 (zero). Please see Document 749851.1 for further information.

Step 4: Make sure that all your database instances are up (including ASM instances) as they would run on production. Use the script hugepages_settings.sh in Document 401749.1 to calculate the recommended value for the vm.nr_hugepages kernel parameter. e.g.:
Note: You can also calculate a proper value for the parameter yourself but that is not advised if you do not have extensive experience with HugePages and concepts.

Step 5: Edit the file /etc/sysctl.conf and set the vm.nr_hugepages parameter there:
This will make the parameter to be set properly with each reboot.

Step 6: Stop all the database instances and reboot the server

Check and Validate the Configuration

After the system is rebooted, make sure that your database instances (including the ASM instances) are started. Automatic startup via OS configuration or CRS, or manual startup (whichever method you use) should have been performed. Check the HugePages state from /proc/meminfo. e.g.:

# grep HugePages /proc/meminfo
HugePages_Total:    1496
HugePages_Free:      485
HugePages_Rsvd:      446
HugePages_Surp:        0
The values in the output will vary. To make sure that the configuration is valid, the HugePages_Free value should be smaller than HugePages_Total and there should be some HugePages_Rsvd. The sum of Hugepages_Free and HugePages_Rsvd may be smaller than your total combined SGA as instances allocate pages dynamically and proactively as needed.
[ID 361468.1]


 Document 749851.1 HugePages and Oracle Database 11g Automatic Memory Management (AMM) on Linux


With AMM all SGA memory is allocated by creating files under /dev/shm. When Oracle DB does SGA allocations that way HugePages are not reserved/used. The use of AMM is absolutely incompatible with HugePages. (Please see references at the end of the document for further information on HugePages)

always make a backup using hugepages

REFERENCES

Немає коментарів: