To learn Erlang I am trying to implement a tiny web server based on gen_tcp. Unfortunately, my code seems to trigger some wired behaviour. To demonstrate the problem I have attached a minimised version of my implementation which is sufficient to reproduce the problem. It is just delivering a static 200 OK, no matter what the HTTP request was.
The problem arises when I try to run ab (Apache HTTP server benchmarking) against my web server (using loopback interface). Without any concurrent requests (-c) everything is running just fine. However, if I use -c 8 or -c 16, the call to gen_tcp:accept/1 seems to fail on some sockets as I see a number of request: closed lines in the shell.
What makes the whole story even weirder is, that I see different behaviours on different operating systems:
OS X+Erlang/OTP 18: ab reports "Connection reset by peer" almost immediately after starting.
Debian+Erlang R15B01: All but two of the HTTP requests seem to run through. But then, ab hangs for a few seconds and reports "The timeout specified has expired, Total of 4998 requests completed", when i run ab with -n 5000. Similarly, 14998 is reported when I run 15000 tests.
This one does not seem to be the problem. I am honestly quite lost and therefore appreciate any help! :) Thanks!
server(Port) ->
Opt = [list, {active, false}, {reuseaddr, true}],
case gen_tcp:listen(Port, Opt) of
{ok, Listen} ->
handler(Listen),
gen_tcp:close(Listen),
ok;
{error, Error} ->
io:format("init: ~w~n", [Error])
end.
handler(Listen) ->
case gen_tcp:accept(Listen) of
{ok, Client} ->
request(Client),
handler(Listen);
{error, Error} ->
io:format("request: ~w~n", [Error])
end.
request(Client) ->
Recv = gen_tcp:recv(Client, 0),
case Recv of
{ok, _} ->
Response = reply(),
gen_tcp:send(Client, Response);
{error, Error} ->
io:format("request: ~w~n", [Error])
end,
gen_tcp:close(Client).
reply() ->
"HTTP/1.0 200 OK\r\n" ++
"Content-Length: 7\r\n\r\n"
"static\n".
When you increase the number of concurrent requests sent with ab -c N it will immediately open multiple TCP sockets to the server.
By default a socket opened with gen_tcp:listen/2 will support only five outstanding connection requests. Increase the number of connection requests outstanding with the {backlog, N} option to gen_tcp:listen/2.
I tested your code on OS X with ab and saw this resolve the prolem with "Connection reset by peer".
Related
Does it make a difference if I register a newly spawned process using register(atom, spawn..) or if I do Pid = spawn..?
To take an example, I just did this with an old program from the Programming Erlang book:
Let's first make a simple server loop:
-module(geometry_server).
-export([loop/0]).
loop() ->
receive
{Client, {square, S} = Tuple} ->
io:format("Server: Area of square of Side ~p is ~p and Client was ~p~n", [S, S*S, Client]),
Client ! {self(), Tuple, S*S},
loop()
end.
Now a client:
-module(geometry_client).
-export([client/2, start_server/0]).
client(Pid_server, Geom_tuple) ->
Pid_server ! {self(), Geom_tuple},
receive
{Pid_server, Geom_tuple, Area} -> io:format("Client: Area of ~p is ~p and server was ~p~n", [Geom_tuple, Area, Pid_server])
after 1000 ->
io:format("~p~n",["received nothing from server"] )
end.
start_server() -> spawn(geometry_server, loop, []).
After compiling both, I do
register(q, Q = geometry_client:start_server()).
Then I call them and get results as follows:
5> geometry_client:client(Q, {square,2}).
Server: Area of square of Side 2 is 4 and Client was <0.60.0>
Client: Area of {square,2} is 4 and server was <0.77.0>
ok
6> geometry_client:client(q, {square,2}).
Server: Area of square of Side 2 is 4 and Client was <0.60.0>
"received nothing from server"
ok
Why does the client not receive anything from the server when I use the registered atom?? The server obviously received the message from the client.
I can confirm that the server sent a message, because after the above if I do
7> geometry_client:client(whereis(q), {square,2}).
Client: Area of {square,2} is 4 and server was <0.77.0>
Server: Area of square of Side 2 is 4 and Client was <0.60.0>
ok
12>
So I conclude that the mailbox already has the message from the server from the previous command, which is why the Client output gets printed before the Server message has been received and printed...
What am I missing?? Why is there a problem receiving the message when I use the registered atom?
The receive in your client/2 function waits for a message matching {Pid_server, Geom_tuple, Area}. When you pass q as the argument, Pid_server is q, but the message the server sends back to the client is a tuple with the first element always being the actual PID of the server, not its name, which means your receive ends up in the after block.
There are many ways to solve this. You can modify client/2 to use whereis/1 to get the PID of the registered process and use that in receive if Pid_server is an atom.
The best way would be to use references here (see make_ref/0). You create a reference when sending a message to the server, and the server sends it back in the response. This way you're guaranteed that you're receiving the response for the request you just sent, since every reference returned by make_ref/0 is guaranteed to be unique.
In client/2, do:
client(Pid_server, Geom_tuple) ->
Ref = make_ref(),
Pid_server ! {Ref, self(), Geom_tuple},
receive
{Ref, Geom_tuple, Area} -> io:format("Client: Area of ~p is ~p and server was ~p~n", [Geom_tuple, Area, Pid_server])
after 1000 ->
io:format("~p~n",["received nothing from server"] )
end.
And in the server:
loop() ->
receive
{Ref, Client, {square, S} = Tuple} ->
io:format("Server: Area of square of Side ~p is ~p and Client was ~p~n", [S, S*S, Client]),
Client ! {Ref, Tuple, S*S},
loop()
end.
I tried using the following code for a simple test to send packets across the Internet. I did check to see if the localhost version works with the following commands and it did work but it doesn't work if I replace localhost (127.0.0.1) with a real internet address, mine. I just get 0 on the client side and nothing changes with the server side, although using localhost is the same with the server side.
Server side:
udp_test:start_server().
Client side:
udp_test:client(40).
Simple enough but when I replace 127.0.0.1 with my ip address I get nothing ( I turned off the firewall too). The basic topology of my home network consists of a wireless modem (using dsl) and my computer is connected via a wireless usb card. There are about two other active computers on the network. I thought that may have something to do with the connection.
I also wondered how could I modify the code so I can type udp_test:client(40,127.0.0.1).
instead and change the client function to accept two arguments in udp_test? I tried simply doing client(N,ip) and changing the function gen_udp:send(Socket, "ip",,) but that came up with a no matching clause error.
-module(udp_test).
-export([start_server/0, client/1]).
start_server() ->
spawn(fun() -> server(4000) end).
%% The server
server(Port) ->
{ok, Socket} = gen_udp:open(Port, [binary]),
io:format("server opened socket:~p~n",[Socket]),
loop(Socket).
loop(Socket) ->
receive
{udp, Socket, Host, Port, Bin} = Msg ->
io:format("server received:~p~n",[Msg]),
N = binary_to_term(Bin),
Fac = fac(N),
gen_udp:send(Socket, Host, Port, term_to_binary(Fac)),
loop(Socket)
end.
fac(0) -> 1;
fac(N) -> N * fac(N-1).
%% The client
client(N) ->
{ok, Socket} = gen_udp:open(0, [binary]),
io:format("client opened socket=~p~n",[Socket]),
ok = gen_udp:send(Socket, "127.0.0.1", 4000,
term_to_binary(N)),
Value = receive
{udp, Socket, _, _, Bin} = Msg ->
io:format("client received:~p~n",[Msg]),
binary_to_term(Bin)
after 2000 ->
0
end,
gen_udp:close(Socket),
Value.
write {127,0,0,1} instead of "127.0.0.1".
Ip address
I am writing some code which sends data over ssl sockets.
The sending part is inside a gen_server:call/3 as:
handle_call({send, Data}, _From, #state{socket=Socket} = State) ->
Reply = case ssl:send(Socket, Data) of
ok ->
ok;
{error, Error} ->
{error, Error}
end,
{reply, Reply, State}.
the problem is that if i kill the application which behaves as server at the other side of the connection, the result of the call is 'ok' but the Data is not sent. Does that mean that the socket is viewed as alive untile {ssl_closed, S} is received by the process?
It was my mistake, data is actually sent but never recovered by peer.
I am implementing as exercise a gen_server which behaves as interface towards an ssl authentication server. The ssl server severes the connection if a packet received is wrong (e.g. wrong username and password). The connection must be persistent.
In my gen_server, I open the ssl connection towards the server with an handle_cast/2:
handle_cast(connect, State) ->
......
case ssl:connect(Address, Port, Options, Timeout) of
{ok, NewSocket} ->
{noreply, State#state{socket=NewSocket}};
{error, Reason} ->
gen_server:cast(?SERVER, connect),
{noreply, State#state{socket=undefined}};
and then I wait for other other messages in handle_cast/2 which can be sent for example using:
gen_server:cast(Pid, {authenticate, User, Password}).
Whenever I receive such a cast message I spawn a new function which recovers the SSL socket from the server state using a gen_server:call/3 and sends the authentication message to the SSL server. If the sending part returns an error I try to reconnect, otherwise I read for a while on the socket, to be sure that the socket does not go down, and if it does I reconnect.
send_auth(_, _, 0) ->
{error, max_num_reached};
send_auth(User, Password, Num) ->
Socket = gen_server:call(?SERVER, socket),
%% also a check that socket is not 'undefined'
case ssl:send(Socket, AuthMessage) of
ok ->
case ssl:recv(Socket, 0, 2000) of
{error, timeout} ->
ok;
_ ->
gen_server:cast(?SERVER, connect),
send_auth(User, Password, Num-1)
end,
{error, closed} ->
gen_server:cast(?SERVER, connect),
send_auth(User, Password, Num-1)
end.
I made many tests, but every time, if one message (not the last) is wrong, none of the following messages is actually delivered.
How can I grant that all valid authentication messages are delivered to the authentication server? Moreover how can be sure that the server will connect only if it is not already trying to do so? Otherwise that would be like DOS attack!
As I can understand there's only one authentication server in the system so why can't you connect to it right in the gen_server's init? If the connection is somewhat reliable, you can just kill gen_server if there are any issues with connection and let supervisor re-launch it. It probably will solve your DOS concern.
When you spawn additional processes with send_auth you can just pass socket connection in arguments, instead of doing gen_server call.
To guarantee the delivery you'll add some kind of acknowledgement into your protocol. Auth server should reply to send_auth with something confirming that it got it. And send_auth should retry until it receives that acknowledgement or have some other fallback behavior in case it never does.
Does anyone know how to enable active instead of passive sockets in a Mochiweb application. Specifically, I am trying to adapt http://www.metabrew.com/article/a-million-user-comet-application-with-mochiweb-part-2 so that when a client disconnects, it will immediately "logout".
I have tried setting:
start(Options) ->
{DocRoot, Options1} = get_option(docroot, Options),
Loop = fun (Req) ->
Socket = Req:get(socket),
inet:setopts(Socket, [{active, once}]),
?MODULE:loop(Req, DocRoot)
end,
but that seems to not be working. I still only get updates in my receive after I am sent a new message.
Thoughts? Thanks!
I solved this for my Erlang comet app, parts of which I show in this blog post. Basically, you don't want the socket to be in active mode all the time; you just want it in active mode after you've read the client's request and before you return a response.
Here's a sample request handler:
comet(Req) ->
Body = Req:recv_body(),
io:format("~nBody: ~p~n", [Body]),
Socket = Req:get(socket),
inet:setopts(Socket, [{active, once}]),
Response = connection:handle_json(Body),
inet:setopts(Socket, [{active, false}]),
io:format("~nSending Response: ~s~n", [Response]),
Req:ok({"application/json", [], Response}).
The io:format call is just console logging for my benefit. The important part is that I set {active, once} on the socket after reading the body from the request and just before calling the function which holds the request and returns data. I also turn active mode back off; the socket may be reused in certain HTTP modes.