I want to log all requests along with responses to db. I'm using hooks for that. But it looks like I can't get request body in 'onresponse' hook, it's always <<>>. In 'onrequest' hook I can get request body.
My hooks defined as:
request_hook(Req) ->
%% All is OK: ReqBody contains what I sent:
{ok, ReqBody, Req2} = cowboy_req:body(Req),
io:format("request_hook: body = ~p", [ReqBody]),
Req2.
response_hook(_Status, _Headers, _Body, Req) ->
%% ReqBody is always <<>> at this point. Why?
{ok, ReqBody, Req2} = cowboy_req:body(Req),
io:format("response_hook: body = ~p", [ReqBody]),
Req2.
Is this a bug in cowboy or normal behaviour?
I'm using the latest cowboy available at the time of writing this post (commit: aab63d605c595d8d0cd33646d13942d6cb372b60).
The latest version of Cowboy (as I know from v0.8.2) use following approach to increase performance - cowboy_req:body(Req) return Body and NewReq structure without request body. In other word it is a normal behaviour and you able to retrieve request body only once.
Cowboy does not receive request body as it can be huge, body placed in socket until it became necessary (until cowboy_req:body/1 call).
Also after you retrieve body, it become not available in handler.
So if you want to implement logging and make body available in handler, you can save body on request to shared location and explicitly remove it on response.
request_hook(Req) ->
%% limit max body length for security reasons
%% here we expects that body less than 80000 bytes
{ok, Body, Req2} = cowboy_req:body(80000, Req),
put(req_body, Body), %% put body to process dict
Req2.
response_hook(RespCode, RespHeaders, RespBody, Req) ->
ReqBody = get(req_body),
Req2.
%% Need to cleanup body record in proc dict
%% since cowboy uses one process per several
%% requests in keepalive mode
terminate(_Reason, _Req, _St) ->
put(req_body, undefined),
ok.
Related
This seems to cause strange behaviour when PUT has a body larger than a certain length (in my case it is 902 bytes), i.e. ejabberd trims the body (in my case it receives malformed JSON).
Github reference: https://github.com/processone/ejabberd/blob/master/src/ejabberd_http.erl#L403
If I change the case statement to:
case Method of
_AnyMethod ->
case recv_data(State) of
{ok, Data} ->
LQuery = case catch parse_urlencoded(Data) of
{'EXIT', _Reason} -> [];
LQ -> LQ
end,
{State, {LPath, LQuery, Data}};
error ->
{State, false}
end
end
then the body is parsed correctly.
Is this a configuration issue? How can I force Ejabberd to correctly parse the JSON body?
Looks like you've found a bug.
As you've noticed, for POST requests the function recv_data is called, which checks the Content-Length header and reads that many bytes from the socket. For PUT requests however, it only uses Trail, which is the data that has been already received while reading HTTP request headers. (This happens in the receive_headers function, which sends a length of 0 to the recv function, meaning that it won't wait for any specific amount of data.)
How much of the request body is received is going to depend on the size of the headers, as well as the way the client sends the request. If for example the client first sends the headers in one network packet, and then the request body in the next network packet, ejabberd wouldn't pick up the request body at all.
I am new with Erlang and my doubt is how to create a JSON Object in Erlang and pass that object in REST API call. I have read so many posts but didn't get any satisfying answer.
Edit
Here I am calling the API:
offline_message(From, To, #message{type = Type, body = Body}) ->
Type = xml:get_text(Type),
Body = xml:get_text(Body),
Token = gen_mod:get_module_opt(To#jid.lserver, ?MODULE, auth_token, fun(S) -> iolist_to_binary(S) end, list_to_binary("")),
PostUrl = gen_mod:get_module_opt(To#jid.lserver, ?MODULE, post_url, fun(S) -> iolist_to_binary(S) end, list_to_binary("")),
to = To#jid.luser
from = From#jid.luser
if
(Type == <<"chat">>) and (Body /= <<"">>) ->
Sep = "&",
Post = {
"token":binary_to_list(Token),
"from":binary_to_list(from),
"to":binary_to_list(to),
"body":binary_to_list(Body)
},
?INFO_MSG("Sending post request to ~s with body \"~s\"", [PostUrl, Post]),
httpc:request(post, {binary_to_list(PostUrl), [], "application/json", binary_to_list(Post)},[],[]),
ok;
true ->
ok
end.
Is everything ok here regarding JSON String. I am trying to modify this module.
How to create a JSON Object in Erlang
There are no such things as objects in erlang, so the simple answer is: you can't. However, the things you send over the wire are just strings, and you can certainly create strings using erlang.
To make things easier, you can use an erlang module like jsx to create the json formatted strings that you want to send in your request. However, in order to use erlang modules you will have to learn a little about rebar3, which is erlang's package installer (see What is the easiest way for beginners to install a module?).
Remember that an http request is just a string that is formatted a certain way. An http request starts with a line like:
POST /some/path HTTP/1.1
Then there are some lines of text called headers, which look like:
User-Agent: Mozilla-yah-yah-bah
Content-Type: application/json
Content-Length: 103
Then there are a couple of newlines followed by some additional text, which is called the post body, which can be in several different formats (the format should be declared in the Content-Type header):
Format Content-Type
------ -----------
"x=1&y=2" application/x-www-form-urlencoded
"{x:1, y:2}" application/json
"more complex string" multipart/form-data
To make it easier to assemble an http request and send it to a server, erlang has a builtin http client called inets, which you can read about in the docs here. For an example that uses inets, see here. Because inets is a bit cumbersome to use, alternatively you can use a third party http client like hackney. Once again, though, you will need to be able to install hackney with rebar3.
Once you send the request, it's up to the server to decipher the request and take the necessary course of action.
Does it make a difference if I register a newly spawned process using register(atom, spawn..) or if I do Pid = spawn..?
To take an example, I just did this with an old program from the Programming Erlang book:
Let's first make a simple server loop:
-module(geometry_server).
-export([loop/0]).
loop() ->
receive
{Client, {square, S} = Tuple} ->
io:format("Server: Area of square of Side ~p is ~p and Client was ~p~n", [S, S*S, Client]),
Client ! {self(), Tuple, S*S},
loop()
end.
Now a client:
-module(geometry_client).
-export([client/2, start_server/0]).
client(Pid_server, Geom_tuple) ->
Pid_server ! {self(), Geom_tuple},
receive
{Pid_server, Geom_tuple, Area} -> io:format("Client: Area of ~p is ~p and server was ~p~n", [Geom_tuple, Area, Pid_server])
after 1000 ->
io:format("~p~n",["received nothing from server"] )
end.
start_server() -> spawn(geometry_server, loop, []).
After compiling both, I do
register(q, Q = geometry_client:start_server()).
Then I call them and get results as follows:
5> geometry_client:client(Q, {square,2}).
Server: Area of square of Side 2 is 4 and Client was <0.60.0>
Client: Area of {square,2} is 4 and server was <0.77.0>
ok
6> geometry_client:client(q, {square,2}).
Server: Area of square of Side 2 is 4 and Client was <0.60.0>
"received nothing from server"
ok
Why does the client not receive anything from the server when I use the registered atom?? The server obviously received the message from the client.
I can confirm that the server sent a message, because after the above if I do
7> geometry_client:client(whereis(q), {square,2}).
Client: Area of {square,2} is 4 and server was <0.77.0>
Server: Area of square of Side 2 is 4 and Client was <0.60.0>
ok
12>
So I conclude that the mailbox already has the message from the server from the previous command, which is why the Client output gets printed before the Server message has been received and printed...
What am I missing?? Why is there a problem receiving the message when I use the registered atom?
The receive in your client/2 function waits for a message matching {Pid_server, Geom_tuple, Area}. When you pass q as the argument, Pid_server is q, but the message the server sends back to the client is a tuple with the first element always being the actual PID of the server, not its name, which means your receive ends up in the after block.
There are many ways to solve this. You can modify client/2 to use whereis/1 to get the PID of the registered process and use that in receive if Pid_server is an atom.
The best way would be to use references here (see make_ref/0). You create a reference when sending a message to the server, and the server sends it back in the response. This way you're guaranteed that you're receiving the response for the request you just sent, since every reference returned by make_ref/0 is guaranteed to be unique.
In client/2, do:
client(Pid_server, Geom_tuple) ->
Ref = make_ref(),
Pid_server ! {Ref, self(), Geom_tuple},
receive
{Ref, Geom_tuple, Area} -> io:format("Client: Area of ~p is ~p and server was ~p~n", [Geom_tuple, Area, Pid_server])
after 1000 ->
io:format("~p~n",["received nothing from server"] )
end.
And in the server:
loop() ->
receive
{Ref, Client, {square, S} = Tuple} ->
io:format("Server: Area of square of Side ~p is ~p and Client was ~p~n", [S, S*S, Client]),
Client ! {Ref, Tuple, S*S},
loop()
end.
Now that Im using cowboy to serve as a websocket server, I understand that the handler implements a behavior called cowboy_websocket_handler and that the websocket_handle/3 function is called every time we receive a request and that to reply back to the request, we reply using {reply, X, _}. However since WebSocket is a bi-directional protocol and that server can reach to a client without a request, how do I send some data to the client, not in the web_socket_handle.
I am expecting something in the handler along the lines of
send(Client, Data). Am I thinking in the right direction? ? Thanks!
If yes, does cowboy provide some API to do so?
To quote the docs:
Cowboy will call websocket_info/2 whenever an Erlang message arrives.
The handler can handle or ignore the messages. It can also send frames
to the client or stop the connection.
The following snippet forwards log messages to the client and ignores
all others:
websocket_info({log, Text}, State) ->
{reply, {text, Text}, State};
websocket_info(_Info, State) ->
{ok, State}.
So all you have to do is send a message to your handler from another process (or from itself if you wish), and implement websocket_info as above to send a frame to the client.
Does anyone know how to enable active instead of passive sockets in a Mochiweb application. Specifically, I am trying to adapt http://www.metabrew.com/article/a-million-user-comet-application-with-mochiweb-part-2 so that when a client disconnects, it will immediately "logout".
I have tried setting:
start(Options) ->
{DocRoot, Options1} = get_option(docroot, Options),
Loop = fun (Req) ->
Socket = Req:get(socket),
inet:setopts(Socket, [{active, once}]),
?MODULE:loop(Req, DocRoot)
end,
but that seems to not be working. I still only get updates in my receive after I am sent a new message.
Thoughts? Thanks!
I solved this for my Erlang comet app, parts of which I show in this blog post. Basically, you don't want the socket to be in active mode all the time; you just want it in active mode after you've read the client's request and before you return a response.
Here's a sample request handler:
comet(Req) ->
Body = Req:recv_body(),
io:format("~nBody: ~p~n", [Body]),
Socket = Req:get(socket),
inet:setopts(Socket, [{active, once}]),
Response = connection:handle_json(Body),
inet:setopts(Socket, [{active, false}]),
io:format("~nSending Response: ~s~n", [Response]),
Req:ok({"application/json", [], Response}).
The io:format call is just console logging for my benefit. The important part is that I set {active, once} on the socket after reading the body from the request and just before calling the function which holds the request and returns data. I also turn active mode back off; the socket may be reused in certain HTTP modes.