Change Erlang file handle limit? - erlang

Question

I'm running into trouble with an Erlang OTP + Cowboy app that does not allow me to open enough files simultaneously.
How do I change the number of open file handles allowed in the BEAM?
Potentially, I'll need about 500 small text files open at the same time, but it appears that the file limit is 224. I've got the value of 224 from this little test program:
-module(test_fds).
-export([count/0]).
count() -> count(1, []).
count(N, Fds) ->
case file:open(integer_to_list(N), [write]) of
{ok, F} ->
count(N+1, [F| Fds]);
{error, Err} ->
[ file:close(F) || F <- Fds ],
delete(N-1),
{Err, N}
end.
delete(0) -> ok;
delete(N) ->
case file:delete(integer_to_list(N)) of
ok -> ok;
{error, _} -> meh
end,
delete(N-1).
This gives
$ erl
Erlang/OTP 20 [erts-9.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:10] [kernel-poll:false]
Eshell V9.2 (abort with ^G)
1> c(test_fds).
{ok,test_fds}
2> test_fds:count().
{emfile,224}
3>
This seems to be an Erlang problem rather than a Mac OSX problem since from the command line, I get:
$ sysctl -h kern.maxfiles
kern.maxfiles: 49,152
$ sysctl -h kern.maxfilesperproc
kern.maxfilesperproc: 24,576

Answer 1

The number of open file descriptors is most likely being limited by your shell. You can increase it by running ulimit -n 1000 (or more) in your shell before invoking erl. On my system, the default value was 7168 and your script could open 7135 files before returning emfile. Here's the output of me running your script with different ulimit values:
$ ulimit -n 64; erl -noshell -eval 'io:format("~p~n", [test_fds:count()]), erlang:halt()'
{emfile,32}
$ ulimit -n 128; erl -noshell -eval 'io:format("~p~n", [test_fds:count()]), erlang:halt()'
{emfile,96}
$ ulimit -n 256; erl -noshell -eval 'io:format("~p~n", [test_fds:count()]), erlang:halt()'
{emfile,224}
$ ulimit -n 512; erl -noshell -eval 'io:format("~p~n", [test_fds:count()]), erlang:halt()'
{emfile,480}
erl is most likely opening 32 file descriptors before it starts evaluating our code which explains the constant difference of 32 in the ulimit and the output.