(edit: i miss some reputation to create the bitcask tag so ...)
(tl;dr => bitcask:get/2 doesn't work and raises badarg in bitcask_nifs:keydir_get_int)
I would like to know how to use bitcask without riak the right way.
First, i was trying this:
bitcask:put(Handle, 3, {this, is, data}).
bitcask:get(Handle, 3).
This two calls raise the same error : badarg with erlang:size/1
The problem is erlang:size/1 accepts only binaries or tuples.
So i was trying this :
bitcask:put(Handle, {thing, 3}, {this, is, data}).
bitcask:get(Handle, {thing, 3}).
A new badarg error then, with erlang:crc32 and the Value i want to store.
So now i use this code, bucket is the atom name of a registered gen_server
which keeps the handle in its state. cask_wrapper is the code for theese
gen_servers. The code below is the acces to theese gen servers.
-module(sr_db).
...
get(Type, Key) when not is_binary(Key) ->
?MODULE:get(Type, term_to_binary(Key));
get(Type, Key) ->
Bucket = type2bucket(Type),
cask_wrapper:get(Bucket, {get, Key}).
put(Type, Key, Data) when not is_binary(Key) ->
?MODULE:put(Type, term_to_binary(Key), Data);
put(Type, Key, Data) when not is_binary(Data) ->
?MODULE:put(Type, Key, term_to_binary(Data));
put(Type, Key, Data) ->
Bucket = type2bucket(Type),
cask_wrapper:put(Bucket, Key, Data),
ok.
%% syncput(Type, Key, Data) -> call au lieu de cast
type2bucket(user) -> users_cask.
I use this code like this:
sr_db:get(user, 3).
%% then a call is made to cask_wrapper:get(users_cask, {get, 3}).
there are the cask_wrapper functions
get(Bucket, Key) ->
gen_server:call(Bucket, {get, Key}).
handle_call({get, Key}, _From, State) ->
Fetch = bitcask:get(State#state.handle, Key),
{reply, Fetch, State}.
I use the same mechanism with the put function. (but with gen_server:cast)
My first question is : is doing term_to_binary conversion in every call
a good practice, or is it slow ? I will have to convert back to erlang
terms the values that i fetch.
At the moment, the put operation returns 'ok'. It works. But the get
operation doesn't work yet. This is the error:
=ERROR REPORT==== 29-Jan-2012::20:21:24 ===
** Generic server users_cask terminating
** Last message in was {get,{get,<<131,97,3>>}}
** When Server state == {state,#Ref<0.0.0.353>}
** Reason for termination ==
** {badarg,[{bitcask_nifs,keydir_get_int,[<<>>,{get,<<131,97,3>>}]},
{bitcask_nifs,keydir_get,2},
{bitcask,get,3},
{cask_wrapper,handle_call,3},
{gen_server,handle_msg,5},
{proc_lib,init_p_do_apply,3}]}
Bitcask dir : "/home/niahoo/src/skyraiders/priv/bitcasks/users"
options : [read_write]** exception exit: {{badarg,
[{bitcask_nifs,keydir_get_int,
[<<>>,{get,<<131,97,3>>}]},
{bitcask_nifs,keydir_get,2},
{bitcask,get,3},
{cask_wrapper,handle_call,3},
{gen_server,handle_msg,5},
{proc_lib,init_p_do_apply,3}]},
{gen_server,call,[users_cask,{get,{get,<<131,97,3>>}}]}}
in function gen_server:call/2
I can't figure out why it does not work and would appreciate some help.
Thank you
Bitcask expects the key and the value both to be binaries (as you already noticed).
I don't really know how fast term_to_binary/binary_to_term is, but there is really no way around it if you want to store terms on disk. You could of course roll you own code to convert you keys and values to/from binaries, but I doubt that it will be significantly fast than the builtin functions and certainly less flexible.
But at the end of the day you have to measure the profile your application, and decide if term_to_binary/binary_to_term is a hotspot in your total system. I would be very surprised if that is the case in any real application where data has to be written to disk.
Now to the error when calling sr_db:get/2. You are wrapping the key twice inside a {get, Key} tuple, once inside sr_db:get/2 and another time in cask_wrapper:get/2, but you unwrap it only once, by matching in cask_wrapper:handle_call/3.
You can immediately spot this in the error report in those two lines:
** Last message in was {get,{get,<<131,97,3>>}}
and
{gen_server,call,[users_cask,{get,{get,<<131,97,3>>}}]}}
Related
In this program, I cannot for the life of me figure out how to access the value of the counter in a process.
-module(counter).
-export([start/0,loop/1,increment/1,value/1,stop/1]).
%% First the interface functions.
start() ->
spawn(counter, loop, [0]).
increment(Counter) ->
Counter ! increment.
value(Counter) ->
Counter ! {self(),value},
receive
{Counter,Value} ->
Value
end.
stop(Counter) ->
Counter ! stop.
%% The counter loop.
loop(Val) ->
receive
increment ->
loop(Val + 1);
{From,value} ->
From ! {self(),Val},
loop(Val);
stop -> % No recursive call here
true;
Other -> % All other messages
loop(Val)
end.
I assume it's:
{From,value} ->
From ! {self(),Val},
loop(Val);
which returns the value of the counter, but every time I use PID ! {PID,value}, or something similar to that it returns the thing after !, e.g. {<0.57.0>, value}.
TL;DR
You shouldn't use ! operator explicitly, it is considered an anti-pattern. You could run into some problems with typos in atoms or some bad data, just like you did this time.
To ensure correct communication with you one usually create some wrapper functions witch handle correct data format and communication with process. Function just like increment/1 value/1 and stop/1. In fact if you would use those, you would get expected results; in your case, assuming that PID is your counter, just call counter:value(PID).
Let me explain
There are few thing you seem to getting little bit wrong.
First of all ! will send message to another process. And that's all it does. Since everything in Erlang is expression (needs to return something, have a value) each call to ! will return right hand side of !. PID ! ok. will return ok, no matter what (there is slight chance that it will fail, but lets no go there). You send your message, and go on with your life, or execution.
Than, some process after receiving your message might decide to send you some message back. In case of {From, value} it will, in case of increment it wont. If you are expecting to get message back you need to wait for it and retrieve it from your mailbox. receive clause will do both waiting and retrieving. So if you decise to use ! on your own you should fallow it with receive with correct pattern match. You can see that value/1 function does just that.
Third thing is correct use of process ID's. I guess you started your counter correctly and you have it's Pid, and you can send messages to it with !. But if you expect it to send something back it needs to know your process ID, your address if you will. So you should have called PID ! {MyPid, values}. How to get MyPid? With self() function. Again, just like in value/1 function.
And last thing many people get wrong at the begging. counter module is just a file with some functions, it's not whole actor/process, and it's not an object. Fact that some value/1 and stop/1 are implemented in it, it doesn't mean that they will be run on counter actor/process. They are functions like any other, and if you call them they will be evaluated in your actor/process, on your stack (same goes for calling them from shell, shell is just another actor). You can spawn new process and tell it to run loop/1 function, but that's all it does. All increment/1 value/1 and stop/1 calls will be executed "on your side".
If this is somewhat confusing try to imagine some simpler function inside counter module, like
add(A, B) ->
A + B.
You can execute it from shell even without any counter process started. It will be created in your process, on your stack, it will add two numbers and return result.
This is important because when you call counter:value(Counter). it will execute Counter ! {self(),value}, "on your side", on your process, so self() will return Pid of your process, not the Pid of counter.
In theory you don't need to know this if you are just using those wrapper function (API or interface if you will), but since you are learning Erlang I would guess you will soon have to write such wrapper. Understanding what happens where is crucial then. Just remember that there is no magic in modules, no secret binding or special execution. Those are just plain old functions and they will be behaving just like in any other language. Only spawn, receive and maybe ! are little different.
It's my first attempt to write anything in Erlang, so maybe the question is silly.
I'm writing a quite simple HTTP server using cowboy
db_name() -> "DB_test".
timestamp() ->
calendar:datetime_to_gregorian_seconds(calendar:universal_time()).
sha(Str) ->
<<X:256/big-unsigned-integer>> = crypto:hash(sha256, Str),
lists:flatten(io_lib:format("~64.16.0b", [X])).
handle_post(Req0, State) ->
Link = binary_to_list(cowboy_req:header(<<"link">>, Req0)),
dets:open_file(db_name(), []),
dets:insert(db_name(), {hashed_url(Link), Link, timestamp()}),
Req = cowboy_req:reply(200,
#{<<"content-type">> => <<"text/plain">>},
sha(Link),
Req0),
{ok, Req, State}.
The idea is that a POST HTTP request contains a 'link' header with some link. After recieving such request my server should store it's hash in dets table along with the link and its timestamp. The problem is that the "DB_test" file isn't created. Why?
Based on your example code, it's hard to say exactly why, since you're ignoring the return values from both dets:open_file/2 and dets:insert/2.
Both of them return different values for the success and failure cases; but do not throw exceptions.
See the official documentation for more details: http://erlang.org/doc/man/dets.html
The simplest solution to this is to crash the cowboy connection handling process in non-success cases. You can do that by doing something like the following:
{ok, Ref} = dets:open_file(db_name(), []),
ok = dets:insert(Ref, {hashed_url(Link), Link, timestamp()}),
This will crash with a badmatch exception in the failure cases, when the value returned cannot be pattern matched to the left-hand side of the statement, subsequently causing cowboy to return HTTP 500 to the client.
You'll then see details on what the actual error was in the stacktrace logged
A second solution would be to explicitly handle the failure cases, you can use the 'case' keyword for that.
An example would be something like:
case dets:open_file(db_name(), []) of
{ok, Ref} ->
do_success_things();
{error, Reason}=E ->
io:format("Unable to open database file: ~p~n", [E]),
do_failure_things();
end
For further reading, I'd highly recommend the Syntax in functions and Errors and exceptions chapters of Learn you some Erlang for great good: http://learnyousomeerlang.com/
I am using observer in elixir and the following is the snapshot of an Application [under applications tab]:
I need to exit these processes once their work is done. Somehow, I am not able to figure out where some of the processes are originating. Is there a way in elixir/erlang to figure out the module/function where a particular process was created?
Suggestions will be highly appreciated. Thanks.
First you must always have the process's PID or its reference name.
Process.info/2
will give you information about that Process. You may get more documentation and information on how this function works in the Erlang's function it is calling:
process_info-2
There are also arity 1 variants: Process Docs
[erlang:process_info(Pid, initial_call) || Pid <- erlang:processes()].
But note that gen_server, etc., all have the same initial call, so you need to dig a little deeper.
The following is adapted from https://gist.github.com/rlipscombe/a8e87583d47799170f8b:
lists:map(
fun(Pid) ->
InitialCall = case erlang:process_info(Pid, initial_call) of
{initial_call,{proc_lib,init_p,A}} ->
case erlang:process_info(Pid, dictionary) of
{dictionary, D} ->
proplists:get_value('$initial_call', D, undefined);
_ ->
{proc_lib,init_p,A}
end;
{initial_call,{erlang,apply,A}} ->
case erlang:process_info(Pid, current_function) of
{current_function,MFA} -> MFA;
_ -> {erlang,apply,A}
end;
{initial_call,IC} ->
IC;
Other ->
Other
end,
{Pid, InitialCall}
end, erlang:processes()).
Using process_info/1 you can get a list of process information from which initial_call and current_function could help you to find the initial function call with which the process was spawned and the current function call of the process respectively.
Also process_info(Pid, initial_call) and process_info(Pid, current_function) functions are using as shortcut.
I have skimmed through the Mochiweb code, but have not found any sign of the State variable.
Does something similar to gen_server's State variable exist in Mochiweb?
I need to store some small amount of state-related server-side (not session-related) data on the server and I do not want to use ETS or Mnesia for that.
I think you have somewhat a misunderstanding of what gen_server state is.
First, let me explain briefly how mochiweb works.
Mochiweb doesn't produce a gen_server process per client. Instead, it just spawns a new process using proc_lib:spawn/3 and creates a parametrized module, which is, basically, a tuple of the following kind:
{mochiweb_request, #Port<0.623>, get, "/users", {1, 1}, []}
which is
{mochiweb_request, Socket, Method, RawPath, HTTPVersion, Headers}
This tuple is used as an argument to a function that you pass as a loop parameter to mochiweb_http:start/1. So, when this "loop" function is called, it will look like this:
handle_request(Req) ->
%% The pattern matching below just shows what Req really is
{mochiweb_request, _, _, _, _, _} = Req,
...
Now, to explanation of gen_server state.
Basically, gen_server is a process with approximately the following structure. Of course, IRL it's more complicated, but this should give you the general idea:
init(Options)
State = ...
loop(Module, State).
loop(Module, State)
NewState = receive
{call, Msg, From} -> Module:handle_call(Msg, From, State)
{cast, Msg} -> Module:handle_cast(Msg, State)
Info -> Module:handle_info(Info, State)
end,
loop(Module, NewState).
So, state is just an argument that you drag through all the function calls and change inside your loop. It doesn't actually matter if your process is a gen_server or not, it doesn't have what lifetime it has. In the following example the term [1, 2, 3] is a state too:
a() ->
b([1, 2, 3], now()).
b(State, Timestamp) ->
Result = do_something(Timestamp)
c(State, Result).
c(State, Payload) ->
exit({State, Payload}).
Now, back to mochiweb.
If you need to create a state of your own, you can just add an extra function argument:
handle_request(Req) ->
User = Req:get(path),
UserData = load_user_data(User),
handle_request(Req, UserData).
handle_request(Req, UserData) ->
...
Now UserData is a state too. You can loop this process, or let it respond and end right away – but you won't lose UserData as long as you pass it as an argument.
Finally, if you really want to make this process a gen_server (which is really unreasonable in most cases), you can use gen_server:enter_loop/3 function that will make your current process a gen_server. And The 3rd argument of this function will be your state that will be stored inside the started gen_server.
I am trying to use Erlang/ets to store/update various informations by pattern matching received data. Here is the code
start() ->
S = ets:new(test,[]),
register(proc,spawn(fun() -> receive_data(S) end)).
receive_data(S) ->
receive
{see,A} -> ets:insert(S,{cycle,A}) ;
[[f,c],Fcd,Fca,_,_] -> ets:insert(S,{flag_c,Fcd,Fca});
[[b],Bd,Ba,_,_] -> ets:insert(S,{ball,Bd,Ba})
end,
receive_data(S).
Here A is cycle number, [f,c] is center flag , [b] is ball and Fcd,Fca, Bd, Ba are directions and angle of flag and ball from player.
Sender process is sending these informations. Here, pattern matching is working correctly which I checked by printing values of A, Fcd,Fca..etc. I believe there is something wrong with the use of Erlang/ets.
When I run this code I get error like this
Error in process <0.48.0> with exit value: {badarg,[{ets,insert,[16400,{cycle,7}]},{single,receive_data,1}]
Can anybody tell me what's wrong with this code and how to correct this problem?
The problem is that the owner of the ets-table is the process running the start/1 function and the default behavior for ets is to only allow the owner to write and other processes to read, aka protected. Two solutions:
Create the ets table as public
S = ets:new(test,[public]).
Set the owner to your newly created process
Pid = spawn(fun() -> receive_data(S) end,
ets:give_away(test, Pid, gift)
register(proc,Pid)
Documentation for give_away/3