It's my first attempt to write anything in Erlang, so maybe the question is silly.
I'm writing a quite simple HTTP server using cowboy
db_name() -> "DB_test".
timestamp() ->
calendar:datetime_to_gregorian_seconds(calendar:universal_time()).
sha(Str) ->
<<X:256/big-unsigned-integer>> = crypto:hash(sha256, Str),
lists:flatten(io_lib:format("~64.16.0b", [X])).
handle_post(Req0, State) ->
Link = binary_to_list(cowboy_req:header(<<"link">>, Req0)),
dets:open_file(db_name(), []),
dets:insert(db_name(), {hashed_url(Link), Link, timestamp()}),
Req = cowboy_req:reply(200,
#{<<"content-type">> => <<"text/plain">>},
sha(Link),
Req0),
{ok, Req, State}.
The idea is that a POST HTTP request contains a 'link' header with some link. After recieving such request my server should store it's hash in dets table along with the link and its timestamp. The problem is that the "DB_test" file isn't created. Why?
Based on your example code, it's hard to say exactly why, since you're ignoring the return values from both dets:open_file/2 and dets:insert/2.
Both of them return different values for the success and failure cases; but do not throw exceptions.
See the official documentation for more details: http://erlang.org/doc/man/dets.html
The simplest solution to this is to crash the cowboy connection handling process in non-success cases. You can do that by doing something like the following:
{ok, Ref} = dets:open_file(db_name(), []),
ok = dets:insert(Ref, {hashed_url(Link), Link, timestamp()}),
This will crash with a badmatch exception in the failure cases, when the value returned cannot be pattern matched to the left-hand side of the statement, subsequently causing cowboy to return HTTP 500 to the client.
You'll then see details on what the actual error was in the stacktrace logged
A second solution would be to explicitly handle the failure cases, you can use the 'case' keyword for that.
An example would be something like:
case dets:open_file(db_name(), []) of
{ok, Ref} ->
do_success_things();
{error, Reason}=E ->
io:format("Unable to open database file: ~p~n", [E]),
do_failure_things();
end
For further reading, I'd highly recommend the Syntax in functions and Errors and exceptions chapters of Learn you some Erlang for great good: http://learnyousomeerlang.com/
Related
I'm trying to subscribe to a RabbitMQ queue like explained here. I can get the messages in my process but somehow it doesn't get pattern matched to the first function clause.
handle_info({#'basic.deliver'{delivery_tag = Tag}, #amqp_msg{payload = Payload}}, {Channel, Tag, Module} = State) ->
amqp_channel:cast(Channel, #'basic.ack'{delivery_tag = Tag}),
gen_server:start_link(Module, Payload, []),
{noreply, State};
handle_info(Message, State) ->
lager:info("~p", [Message]),
{noreply, State}.
My code looks identical to a few examples I found on Github but the message always ends up in the second clause. Here's the log message.
[info]
{{'basic.deliver',<<"amq.ctag-7EUw07D8SQ5cSX9DaVBznw">>,1,true,<<"trail">>,<<"SlackUserCreated">>},{amqp_msg,{'P_basic',undefined,undefined,undefined,undefined,undefined,undefined,undefined,undefined,undefined,undefined,undefined,undefined,undefined,undefined},<<"{\"userId\":32,\"returnUrl\":\"http://example.org\"}">>}}
Clearly I'm missing something obvious here. This is the RabbitMQ client I'm using.
In the function head, you're using the variable Tag twice, once in the basic.deliver record and once in the state. It looks like this message should match otherwise, so probably the tag in the state is not equal to the delivery tag.
I am a little confused with gproc and Pub/Sub methods( https://github.com/uwiger/gproc#use-case-pubsub-patterns ).
I can't understand how to receive messages from another process.
Example:
-module(ws_handler).
-export([init/2]).
init(Req, Opts) ->
lager:info("WS: init ws handler"),
gproc:reg({p, l, {?MODULE, WSNewMsgKey}}),
{cowboy_websocket, Req, Opts}.
process_data(Data) ->
lager:info("WS: start processing of json data"),
gproc:send({p, l, WSNewMsgKey}, {self(), WSNewMsgKey, Data}).
There are 2 processes, both of them are registered as subscribers. They should share incoming data with each other. I guess that i have to implement some interface/function but docs don't tell which exactly.
I've never used gproc for this, but it certainly seems like two things are missing: a definition of WSNewMsgKey (its never in scope in your snippet above) and a receive clause somewhere to accept the messages sent:
-module(ws_handler).
-export([init/2]).
init(Req, Opts) ->
gproc:reg({p, l, {?MODULE, ws_event}}),
{some_state_blah, Req, Opts}.
notify_peers(Event) ->
gproc:send({p, l, ws_event}, {self(), ws_event, Event}).
...and elsewhere either
handle_info({From, ws_event, Event}, State) ->
ok = handle_ws_event(From, Event).
or in your loop (if you wrote your process by hand):
loop(State) ->
receive
{From, ws_event, Event} ->
ok = handle_ws_event(From, Event),
loop(State);
Whatever ->
% other stuff...
end.
I'm not sure if the message that is sent would be sent by as a call, a cast, or a normal message (I'm assuming either an OTP generic cast, or normal message) -- but it seems that this is what should happen. In all cases, though, you need a well-defined key to identify the category of message being sent, and here I've used the atom 'ws_event' to make this explicit.
As for the details of the snippet above... you appear to be broadcasting the same JSON message to a bunch of processes at once for some sort of processing? I'm not sure what this would do for you -- I can't think of any case where broadcasting raw JSON would be beneficial (unless maybe if the need is to broadcast the JSON outside of the system and you are broadcasting to a bunch of subscribed client socket handlers?). So I'm confused at the context (what are you trying to achieve?).
This appears to be the way the docs intend this to be used -- but I'd have to actually play with it to be certain.
E.g. suppose I have a list that looks something roughly like this:
Handlers = [{foo, FooHandler}, {bar, BarHandler} | Etc()]
The best that I can come up with is this:
receive
Message ->
Handler = find_matching_handler(Message, Handlers),
Handler(Message)
end
The problem with this is that if Message does not match anything in Handlers, it's too late: I've taken it out of the mailbox.
I guess if there's a way to put a message back into the mailbox (into the save queue) without reordering, then that would take care of it. Simply resending to self() would reorder. It would also not restart the receive, and even if it did, you might get stuck in a spin loop until a message of interest arrives. Is there a way to put a message into the mailbox's save queue?
Another near solution that I thought of was to use match guard, but IIUC, you can only use BIFs in guards, which seems to preclude using find_matching_handler (unless there is a BIF for that).
Another near solution: map matching:
receive
M when Handlers#{M := Handler} -> Handler(M) % booyah?
end
Alas, I have not found an incantation that satisfies Erlang...
Match on the message:
loop() ->
receive
{foo, Data} ->
handle_foo(Data),
loop();
{bar, Data} ->
handle_bar(Data),
loop()
end.
This is the basic way of distinguishing between message forms.
You can also be less direct and match in a function head you pass all messages to:
loop() ->
receive
Message ->
handle_message(Message),
loop()
end.
handle_message({foo, Data}) ->
foo(Data),
ok;
handle_message({bar, Data}) ->
bar(Data),
ok.
A combination of the first and second forms is sort of the way gen_server type callback modules are structured in OTP. The message handlers receive a slightly more complex set of arguments and exist in their own module (the part you write), and the actual receive occurs in the generic gen_server module.
You can use a selective receive pattern to periodcally scan the mailbox for handler messages. Something like this:
check_msg_handlers(Handlers) ->
[check_handler(X) || X <- Handlers],
timer:sleep(500),
check_msg_handlers(Handlers).
check_handler(Handler) ->
receive
{_Handler={M,F}, Msg} ->
M:F(Msg)
after
0 ->
no_msg
end.
Note the receive X -> Y after -> N no_msg end, this is the selective receive. When using a timeout of N=0 it effectively becomes a scan of the mailbox to see if the X message is present or not, i.e. it becomes a non-blocking receive. The order of the messages is preserved after the scan as required in your case.
The LYSE chapter More On Multiprocessing has a section on selective receives that is very good.
(edit: i miss some reputation to create the bitcask tag so ...)
(tl;dr => bitcask:get/2 doesn't work and raises badarg in bitcask_nifs:keydir_get_int)
I would like to know how to use bitcask without riak the right way.
First, i was trying this:
bitcask:put(Handle, 3, {this, is, data}).
bitcask:get(Handle, 3).
This two calls raise the same error : badarg with erlang:size/1
The problem is erlang:size/1 accepts only binaries or tuples.
So i was trying this :
bitcask:put(Handle, {thing, 3}, {this, is, data}).
bitcask:get(Handle, {thing, 3}).
A new badarg error then, with erlang:crc32 and the Value i want to store.
So now i use this code, bucket is the atom name of a registered gen_server
which keeps the handle in its state. cask_wrapper is the code for theese
gen_servers. The code below is the acces to theese gen servers.
-module(sr_db).
...
get(Type, Key) when not is_binary(Key) ->
?MODULE:get(Type, term_to_binary(Key));
get(Type, Key) ->
Bucket = type2bucket(Type),
cask_wrapper:get(Bucket, {get, Key}).
put(Type, Key, Data) when not is_binary(Key) ->
?MODULE:put(Type, term_to_binary(Key), Data);
put(Type, Key, Data) when not is_binary(Data) ->
?MODULE:put(Type, Key, term_to_binary(Data));
put(Type, Key, Data) ->
Bucket = type2bucket(Type),
cask_wrapper:put(Bucket, Key, Data),
ok.
%% syncput(Type, Key, Data) -> call au lieu de cast
type2bucket(user) -> users_cask.
I use this code like this:
sr_db:get(user, 3).
%% then a call is made to cask_wrapper:get(users_cask, {get, 3}).
there are the cask_wrapper functions
get(Bucket, Key) ->
gen_server:call(Bucket, {get, Key}).
handle_call({get, Key}, _From, State) ->
Fetch = bitcask:get(State#state.handle, Key),
{reply, Fetch, State}.
I use the same mechanism with the put function. (but with gen_server:cast)
My first question is : is doing term_to_binary conversion in every call
a good practice, or is it slow ? I will have to convert back to erlang
terms the values that i fetch.
At the moment, the put operation returns 'ok'. It works. But the get
operation doesn't work yet. This is the error:
=ERROR REPORT==== 29-Jan-2012::20:21:24 ===
** Generic server users_cask terminating
** Last message in was {get,{get,<<131,97,3>>}}
** When Server state == {state,#Ref<0.0.0.353>}
** Reason for termination ==
** {badarg,[{bitcask_nifs,keydir_get_int,[<<>>,{get,<<131,97,3>>}]},
{bitcask_nifs,keydir_get,2},
{bitcask,get,3},
{cask_wrapper,handle_call,3},
{gen_server,handle_msg,5},
{proc_lib,init_p_do_apply,3}]}
Bitcask dir : "/home/niahoo/src/skyraiders/priv/bitcasks/users"
options : [read_write]** exception exit: {{badarg,
[{bitcask_nifs,keydir_get_int,
[<<>>,{get,<<131,97,3>>}]},
{bitcask_nifs,keydir_get,2},
{bitcask,get,3},
{cask_wrapper,handle_call,3},
{gen_server,handle_msg,5},
{proc_lib,init_p_do_apply,3}]},
{gen_server,call,[users_cask,{get,{get,<<131,97,3>>}}]}}
in function gen_server:call/2
I can't figure out why it does not work and would appreciate some help.
Thank you
Bitcask expects the key and the value both to be binaries (as you already noticed).
I don't really know how fast term_to_binary/binary_to_term is, but there is really no way around it if you want to store terms on disk. You could of course roll you own code to convert you keys and values to/from binaries, but I doubt that it will be significantly fast than the builtin functions and certainly less flexible.
But at the end of the day you have to measure the profile your application, and decide if term_to_binary/binary_to_term is a hotspot in your total system. I would be very surprised if that is the case in any real application where data has to be written to disk.
Now to the error when calling sr_db:get/2. You are wrapping the key twice inside a {get, Key} tuple, once inside sr_db:get/2 and another time in cask_wrapper:get/2, but you unwrap it only once, by matching in cask_wrapper:handle_call/3.
You can immediately spot this in the error report in those two lines:
** Last message in was {get,{get,<<131,97,3>>}}
and
{gen_server,call,[users_cask,{get,{get,<<131,97,3>>}}]}}
I'm trying to write a simple Thrift server in Erlang that takes a string and returns a string.
Everything seems to be working up to the point of calling my function:
handle_function(Function, Args) when is_atom(Function), is_tuple(Args) ->
case apply(?MODULE, Function, tuple_to_list(Args)) of
ok -> ok;
Reply -> {reply, Reply}
end.
test([X]) ->
"You sent: " ++ X.
I'm getting a function_clause. The stack trace shows the following:
{function_clause, [{server, test,
[<<"w00t">>]},
{server,handle_function, 2}, ...
My handle_function is copied from the tutorial file so I won't be surprised if I need to tweak it. Any ideas?
That last argument of apply should be a list of arguments to 'test', e.g., if tuple_to_list(Args) resulted in:
[1]
...then:
test(1)
If tuple_to_list(Args) resulted in:
[1,2]
...then:
test(1,2)
So, if {<<"woot">>} is being passed to tuple_to_list, that's going to be:
[<<"woot">>]
...so:
test(<<"woot">>)
...but test's signature asks for a list as the argument, so there's a mismatch.