Java problem -- Too many open files

Written by Mike Noel

Thursday, 23 September 2004

Recently I was working on a Java project. The program was intended to run for a long time (weeks at a time) and process incoming requests. I started the overnight test to see how it would do receiving requests over a 12 hour period. After five hours the program crashed with this message: java.io.IOException: Too many open files So I had to dig in and figure out what was causing this. First of all, the number of open files allowed is an Operating System dependent setting. This is a limited resource that is shared by all processes on the machine. The standard programming practice is to make sure to close all files that the program opens. On Unix systems this applies to socket connections as well. I inspected my code and looked for all open statements. I made sure that each open statement had the accompanying close. After looking over my code I was certain that everything was closed. But I was still seeing the accumulation of open files as messages were handled. Usually in these type of situations I turn to Google to see if someone else has dealt with this problem too. After a bit of searching I found the clue. My program uses the Runtime.getProcess() method to get a process and then exec() an external script. What I didn't know was that the API automatically opens three streams (stdout, stderr, stdin) each time the getProcess() is called. It is the responsibility of the caller to close those streams when done. I added the code to close those and everything was fine. Of course, after doing this I read the documentation for the Runtime class. Sure enough, it mentions the three streams... It's amazing what a little bit of knowledge can do.