Wednesday 27 March 2013

“Too many open files” error in highly multithreaded application


Error:
Caused by: java.io.FileNotFoundException: errors.txt (Too many open files)
    at java.io.FileOutputStream.openAppend(Native Method)
    at java.io.FileOutputStream.<init>(FileOutputStream.java:192)
    at java.io.FileOutputStream.<init>(FileOutputStream.java:116)
    at hydra.FileUtil.appendToFile(FileUtil.java:528)
    ... 5 more


If we see “Too many open files” error in highly multi- threaded application, this means we have reached to maximum number of file handles.



To see where you are at with file handles on Linux:

Run the command


sysctl fs.file-nr


From this command you will get 3 numbers. First is the number of used file descriptor the second is the number of allocated but not used file descriptor and the last is the system max file descriptor.




To up this limit:
You will need to add something similar to the following in the file /etc/sysctl.conf:


fs.file-max = 204708



Also there is an individual users file handle limit, that is where the "ulimit -n" command. If you are running to this limit you will have to add entries to the /etc/security/limits.conf file for the user like the following:

@users hard nofile 81920
@users soft nofile 8192
@users hard nproc unlimited
@users soft nproc 501408
@users soft core unlimited
@users hard core unlimited


Apply the changes using :

sysctl -a

No comments:

Post a Comment