Error: Too many open files in Linux

“Too many open files” problem occurs when a program opens a large number of file descriptors(files, in simple terms). So, a program is out of limit. In Linux there are rules on possible number of open files. This applies to each process, user session and system wide for protection purposes.

Let’s start our research, we’ll explore how to predict it and how to deal with it.

Check current and maximum number of open files for user and system

To get the maximum number of open files in current file system, use the following command:

cat /proc/sys/fs/file-max
...
400396

Or by using sysctl:

sudo sysctl -a | grep file-max
...
fs.file-max = 400396

Get the number of current open files and its maximum number with one command:

cat /proc/sys/fs/file-nr
...
2656	0	400396

Or with sysctl:

sudo sysctl fs.file-nr
...
fs.file-nr = 2720	0	400396

Explanation:

  • 2720 The number of allocated file handles.
  • 0 The number of unused-but-allocated file handles.
  • 400396 The system-wide maximum number of file handles.

We can view the current limit on the number of open files for a user with the following command:

ulimit -n
...
1024

With this command, we can get how many open files we currently have on the user:

lsof -u your_username | awk '{print $9}' | sort | uniq | wc -l
...
455

Forked processes can share file handles, so by using these filters we get the number of unique files.

Soft and hard limit of open files

ulimit utility returns two types of restrictions – hard and soft. Soft value can be changed in any direction as long as it doesn’t exceed hard value. Hard value (without sudo) can be changed only in the smaller direction. As root, we can change both hard and soft values in any direction. By default, ulimit utility returns soft value:

ulimit -S -n
...
1024

We can use -H option to get hard value:

ulimit -H -n
...
1048576

Change open files limit

With ulimit for a user

Just pass the new value:

ulimit -n 2048

With sysctl.conf for the system

Open /etc/sysctl.conf:

nano /etc/sysctl.conf

Then add new line at the file end:

fs.file-max = 200000

Then make sysctl -p to load new settings:

sysctl -p

Change soft and hard limit

To change the user’s max file limit, we need to edit /etc/security/limits.conf. It has the following format:

username restriction_type restriction_name value

Instead of the username, we can use * character to apply changes to all users in the system. The type of restriction must be soft or hard. Restriction name – in our case is nofile. And the last one – the desired value.

Open the file:

sudo nano /etc/security/limits.conf

Go down and add this lines with your desired values:

* hard nofile 1048576
* soft nofile 1048576

If you want the changes to take effect make sure there is the following line in /etc/pam.d/common-session presented:

session required pam_limits.so

If this line is missing, add it to the end of file.

Change open files limit for a certain service

If you need to add restrictions only for a specific service, for example, Apache or MySQL, then it is not necessary to affect the whole system. You can do it by using systemctl. Perform the following command:

sudo systemctl edit service_name

Then add the following lines to the opened file:

[Service]
LimitNOFILE=120000
LimitNOFILESoft=120000

Don’t forget to replace the values with yours.

Here we set the maximum value for both hard and soft restrictions. Next we need to save & close this file and update systemctl:

sudo systemctl daemon-reload

Then restart your service which you gave the settings above.

sudo systemctl restart service_name

You can make sure that the necessary restrictions have been applied to your service by checking /proc/pid_service/limits. First we look at the PID of the service we need:

sudo systemctl status mariadb
...
Main PID: 2169 (mysqld)
...

Then check the process info:

cat /proc/2169/limits | grep "Max open files"
...
Max open files            120000                 120000              files
Updated: September 2, 2019 — 7:28 am