Error: Too many open files in Linux

“Too many open files” problem occurs when the program opens a large number of file descriptors(files, in simple terms). So, the program is out of limit. In Linux there are strict restrictions on the number of open files for each process and user, for protection purposes.

Let’s start our research, we’ll explore how to predict it and how to deal with it.

Check current and maximum number of open files for user or system

To get the maximum number of open files in current file system, use

cat /proc/sys/fs/file-max
...
400396

Or with sysctl

sudo sysctl -a | grep file-max
...
fs.file-max = 400396

Get the number of current open files and its maximum number with one command

cat /proc/sys/fs/file-nr
...
2656	0	400396

Or with sysctl

sudo sysctl fs.file-nr
...
fs.file-nr = 2720	0	400396

Where:

  • 2720 The number of allocated file handles.
  • 0 The number of unused-but-allocated file handles.
  • 400396 The system-wide maximum number of file handles.

We can view the current limits on the number of open files for a user with the command

ulimit -n
...
1024

With this command, we can get how many open files we currently have on the user

lsof -u your_username | awk '{print $9}' | sort | uniq | wc -l
...
455

It looks weird, but it filters duplicates (forked processes can share file handles etc). So, you get the number of unique files.

Soft and hard limit of open files

ulimit utility returns two types of constraints – hard and soft. Restriction soft you can change in any direction as long as it doesn’t exceed the hard. Restriction hard without sudo can be changed only in the smaller side. As root, you can change both kinds of restrictions as needed. By default, ulimit utility returns soft value:

ulimit -S -n
...
1024

We can use -H option to get hard value

ulimit -H -n
...
1048576

Change max open files limit

With ulimit for user

Just pass the new value

ulimit -n 2048

With sysctl.conf for system

Open the /etc/sysctl.conf file

nano /etc/sysctl.conf

Then add you value in it

fs.file-max = 200000

Then make sysctl load the settings from it

sysctl -p

Change the soft and hard value

Since the hard limit may be small, you cannot set a limit greater than hard value. To change the user’s limit settings, you need to edit the /etc/security/limits.conf file. It has the following format

username restriction_type restriction_name value

Instead of a username, you can use * character to apply changes to all users in the system. The type of restriction must be soft or hard. Name – in our case nofile. And the last one – the desired value.

Now open the file

sudo nano /etc/security/limits.conf

Go down and add this lines with your desired values

* hard nofile 1048576
* soft nofile 1048576

If you want the changes to take effect. Also make sure that there is a line in the /etc/pam.d/common-session file

session required pam_limits.so

If this line is missing, add it to the end. It is necessary for your restrictions to be loaded when the user logs in system.

Change open files limit for a certain service

If you need to add restrictions only for a specific service, for example, Apache or MySQL, then it is not necessary to affect all settings in the system. You can do it using systemctl. Just follow

sudo systemctl edit service_name

And add the following lines in the opened file

[Service]
LimitNOFILE=120000
LimitNOFILESoft=120000

Don’t forget to replace the value with your desired.

Here we set the maximum possible limit for both hard- and soft-value. Next you need to close this file and update the configuration of services

sudo systemctl daemon-reload

Then restart youir service

sudo systemctl restart service_name

You can make sure that the necessary restrictions have been applied to your service by opening the file along the path /proc/pid_service/limits. First we look at the PID of the service we need

sudo systemctl status mariadb
...
Main PID: 2169 (mysqld)
...

And check the process info

cat /proc/2169/limits | grep "Max open files"
...
Max open files            120000                 120000              files
Updated: April 1, 2019 — 2:02 pm

Leave a Reply

Your email address will not be published.