Tuning Linux Parameters
Tuning System Limits and Kernel Parameter on RHEL and CentOS.
Chris GThe default limits and kernel parameters on Linux are fine for a basic, relatively low usage system, but these parameters rapidly become inadequate on more robust and busier implementations.
The first issue that most people encounter is file descriptors. By default, Linux systems limit the number of file descriptors that any one process may open to 1024 per process. File descriptors are used for many things in Linux, from reading and writing files to maintaining TCP connections, and this can quickly become a bottleneck that will cripple your application.
You can use the following command to view your limits:
ulimit -aH
core file size (blocks) unlimited
data seg size (kbytes) unlimited
file size (blocks) unlimited
max locked memory (kbytes) unlimited
max memory size (kbytes) unlimited
open files 1024
pipe size (512 bytes) 8
stack size (kbytes) unlimited
cpu time (seconds) unlimited
max user processes 4094
virtual memory (kbytes) unlimitedChanging the system limits
It is no longer recommended to edit limits in /etc/security/limits.conf directly. Instead, you should create files in /etc/security/limits.d/ (the lowest numbers in the file name overwrite values from higher numbers in case of conflicting entries).
The following are examples of a script that will add values for the user root and for all other users during the provisioning of a new server. It is assumed that applications are running under their own user account and that the root account is only used for the operating system.
echo "root soft nofile 4096" > /etc/security/limits.d/20-nproc.conf
echo "root hard nofile 65535" >> /etc/security/limits.d/20-nproc.conf
echo "* soft nofile 131072" >> /etc/security/limits.d/20-nproc.conf
echo "* hard nofile 131072" >> /etc/security/limits.d/20-nproc.conf
echo "* soft nproc 131072" >> /etc/security/limits.d/20-nproc.conf
echo "* hard nproc 131072" >> /etc/security/limits.d/20-nproc.confAdvanced Kernel Parameters
In addition to limits, Linux Kernel parameters can significantly influence how the system behaves, especially under heavy load.
⚠ Warning:
Applying an invalid Kernel Parameter in Linux will prevent your system from booting. Make sure to backup your current configuration and make sure to have a valid snapshot/backup of your system before you make any changes. While it may be possible to boot your system in recovery mode to edit the configuration file, it is not easy and not a recommended approach. Hope is not a strategy.
For some applications such as running a busy Oracle database server, it is recommended to turn OFF some of the self-tuning services such as tuned or ktune to apply your own tuning.
The following script will tune some of the recommended values based on the amount of available memory on the system, please adapt it for your own use. The sed commands are used to delete lines and avoid duplicate entries in the configuration file. Note that these parameters are only applied at boot time, while the sysctl commands at the end of the script may apply these values immediately, rebooting the system is the best way to ensure that the new values take effect.
service tuned stop
chkconfig tuned off
service ktune stop
chkconfig ktune off
echo "fs.file-max=65536" > /etc/sysctl.conf
echo "vm.pagecache=40" >> /etc/sysctl.conf
echo "kernel.panic_on_oops=1" >> /etc/sysctl.conf
echo "fs.file-max=6815744" >> /etc/sysctl.conf
echo "fs.aio-max-nr=1048576" >> /etc/sysctl.conf
echo "net.core.rmem_default=262144" >> /etc/sysctl.conf
echo "net.core.rmem_max=4194304" >> /etc/sysctl.conf
echo "net.core.wmem_default=262144" >> /etc/sysctl.conf
echo "net.core.wmem_max=1048576" >> /etc/sysctl.conf
echo "net.bridge.bridge-nf-call-arptables=0" >> /etc/sysctl.conf
echo "net.bridge.bridge-nf-call-ip6tables=0" >> /etc/sysctl.conf
echo "net.bridge.bridge-nf-call-iptables=0" >> /etc/sysctl.conf
echo "net.ipv4.ip_local_port_range=9000 65500" >> /etc/sysctl.conf
echo "net.ipv4.tcp_fin_timeout=30" >> /etc/sysctl.conf
sed -i -e '/kernel.shmmax/d' /etc/sysctl.conf
sed -i -e '/kernel.shmall/d' /etc/sysctl.conf
#Obtain the total memory in the system in bytes:
mem=$(free -b | awk '/Mem/ {print $2}')
#Get page size in bytes:
page=$(getconf PAGE_SIZE)
#Calculate 75% of the total memory in the system in pages for SHMALL:
all=$(expr $mem \* 75 / 100 / $page + 1)
#Multiply the SHMALL value by the page size to get SHMMAX:
max=$(expr $all \* $page)
#Set the SHMMAX and the SHMALL values in the /etc/sysctl.conf file:
echo "kernel.shmmax=$max" >> /etc/sysctl.conf
echo "kernel.shmall=$all" >> /etc/sysctl.conf
sed -i -e '/kernel.shmmni/d' /etc/sysctl.conf
#Set the maximum number of shared memory segments with SHMMNI in the /etc/sysctl.conf file:
echo "kernel.shmmni=4096" >> /etc/sysctl.conf
sed -i -e '/kernel.sem/d' /etc/sysctl.conf
#Recommended minimums for semaphore operations:
#The first value, SEMMSL, is the maximum number of semaphores per semaphore set
#The second value, SEMMNS, defines the total number of semaphores for the system
#The third value, SEMOPM, defines the maximum number of semaphore operations per semaphore call
#The last value, SEMMNI, defines the number of entire semaphore sets for the system
sysctl -w "kernel.sem=250 32000 100 128"
echo "kernel.sem=250 32000 100 128" >> /etc/sysctl.conf
sysctl --system
sysctl -p
free -hThe following parameters are specific to a performant Oracle application server since at times the OS and application processes can have a hard time reclaiming huge amounts of memory that becomes fragmented over time.
############ VARIABLES TO CHANGE
# System Global Area (SGA) values from Oracle in Gig
SGA=8
# Oracle user ID
oracle_user=oracle
oracle_user_oid=$(id -g $oracle_user)
Hugepagesize=$(grep Hugepagesize /proc/meminfo | awk '/Hugepagesize: / {print $2}')
nr_hugepages=$(expr $SGA \* 1024 \* 1024 \/ $Hugepagesize)
sed -i -e '/vm.nr_hugepages/d' /etc/sysctl.conf
echo "vm.nr_hugepages=$nr_hugepages" >> /etc/sysctl.conf
sed -i -e '/vm.hugetlb_shm_group/d' /etc/sysctl.conf
echo "vm.hugetlb_shm_group=$oracle_user_oid" >> /etc/sysctl.conf
sed -i -e '/vm.dirty_background_ratio/d' /etc/sysctl.conf
echo "vm.dirty_background_ratio=3" >> /etc/sysctl.conf
sed -i -e '/vm.swappiness/d' /etc/sysctl.conf
echo "vm.swappiness=10" >> /etc/sysctl.conf
sed -i -e '/vm.dirty_expire_centisecs/d' /etc/sysctl.conf
echo "vm.dirty_expire_centisecs=500" >> /etc/sysctl.conf
sed -i -e '/vm.dirty_writeback_centisecs/d' /etc/sysctl.conf
echo "vm.dirty_writeback_centisecs=100" >> /etc/sysctl.conf
sed -i -e '/vm.min_free_kbytes/d' /etc/sysctl.conf
echo "vm.min_free_kbytes=65536" >> /etc/sysctl.confTagged with:
performance