Related
E.g.
make_orphan() ->
P = spawn(...),
ok
.
Is there a way for P to receive a message some time after make_orphan returns? Or is P destined to haunt the system (using up precious resources) for all eternity, unless it exits on its own?
A straightforward way to:
receive a message some time after make_orphan returns
is with a monitor.
make_orphan() ->
Parent = self(),
P = spawn(fun() -> monitor(process, Parent), ... end),
ok
P will then get a {'DOWN', Ref, process, Parent, Reason} message when Parent dies. Even if Parent exits before monitor/2 is called, the message will contain the reason noproc.
Communicate P to some process somewhere, register P in some way (register, global, gproc, pg2, some homebrew solution, etc.), have someone monitor it, etc. So sure, several ways. But a fundamental principle of an OTP program is that every process belongs to a supervision tree somewhere, so this becomes less of a problem.
Unless you are modeling a system that falls way outside the assumptions of OTP (like peer supervision among cellular automata) then you don't ever want to create the opportunity for orphans to exist. Orphan processes are the Erlang equivalent to memory leaks -- and that is never a good thing.
For some background information on some of the implications of writing OTP processes versus raw Erlang stuff where you're much more likely to leak processes, read the documentation for proc_lib and the "Sys and Proc_Lib" chapter of the OTP Design Principles docs.
I'm still in the learning fase of Erlang, so I might be wrong, but this is how I understood a process' message queue.
A process could be in it's main receive loop, receiving certain types of messages, while later it could be put in a waiting loop to deal with a different kind of message in the second loop. If the process would receive messages intended for the first loop in the second loop, it would just put them in the queue, ignore them for the time being and only process those message that it can match against in the current loop it is in. Now if it would enter the first receive loop again, it would start from the beginning and again process the messages that it can match against.
Now my question would be, if this is how Erlang works and I understood this correctly, then what happens when a malicious process would send a bunch of messages that the process will never process. Will the queue eventually overflow, resulting in a crash for the process or how should I deal with this? I'll type out an example to illustrate what I mean.
Now if a malicious program would get a hold of the Pid and would go Pid ! {malicioudata, LotsOfData} repeatedly, would those messages be filtered out since they will never possibly be processed or would they just stack up in the queue?
startproc() -> firstloop(InitValues).
firstloop(Values) ->
receive
retrieveinformation ->
WaitingList=askforinformation(),
retrieveloop(WaitingList);
dostuff ->
NewValues=doingstuff(),
firstloop(NewValues);
sendmeyourdata ->
sendingdata(Values),
firstloop(Values)
end.
retrieveloop([],Values) -> firstloop(Values).
retrieveloop(WaitingList,Values) ->
receive
{hereismyinformation,Id,MyInfo} ->
NewValues=dosomethingwithinfo(Id,MyInfo),
retrieveloop(lists:remove(Id,1,WaitingList),NewValues);
end.
There is not a hard limit on message counts, and there is not a fixed amount of memory you are limited to, but you can certainly run out of memory if you have billions of messages (or a few super huge ones, maybe).
Long before you OOM because of a huge mailbox you will notice either selective receives taking a long time (not that "selective receive" is a good pattern to follow much of the time...) or innocently peek into a process mail queue and realized you've opened Pandora's Box in your terminal.
This is usually treated as a throttling and monitoring issue in the Erlang world. If you aren't able to keep up and your problem is parallelizable then you need more workers. If you are maxing out your hardware then you need more efficiency. If you are still maxing out your hardware, can't get any more, and you're still overwhelmed then you need to decide how to implement pushback or load shedding.
Unfortunately there is no "message queue overflow" and it's going to grow until VM crashes due to memory allocation error.
Solution is to drop any invalid messages in main loop, because you are not suppose to receive any of {hereismyinformation, _,_} nor one you get in askforinformation() due to blocking nature of your process.
startproc() -> firstloop(InitValues).
firstloop(Values) ->
receive
retrieveinformation ->
WaitingList=askforinformation(),
retrieveloop(WaitingList, Values); % i assume you meant that
dostuff ->
NewValues=doingstuff(),
firstloop(NewValues);
sendmeyourdata ->
sendingdata(Values),
firstloop(Values);
_ ->
firstloop(Values) % you can't get {hereismyinformation, _,_} here so we can drop any invalid message
end.
retrieveloop([],Values) -> firstloop(Values).
retrieveloop(WaitingList,Values) ->
receive
{hereismyinformation,Id,MyInfo} ->
NewValues=dosomethingwithinfo(Id,MyInfo),
retrieveloop(lists:remove(Id,1,WaitingList),NewValues);
end.
It's not really a problem with unexpected messages because it's easily avoidable but when process queue is growing faster than it's processed. For this specific problem there is a nice jobs framework for production systems.
Detecting cycles in a single linked list is a well known problem. I know that this question has been asked a zillion times all over the internet. The reason why I am asking it again is I thought of a solution which I did not encounter at other places. (I admit I haven't searched that deeply either).
My solution is:
Given a linked list and pointer to some node, break the link between node and node->next();
Then start at node->next() and traverse till either you hit an end (which means there was no loop) or till you reach at node which means there was a loop.
Is there anything wrong/good about above solution ?
Note: Do join the link back once you are done.
That will work to detect complete cycles (i.e., cycles with a period of the whole list), e.g.:
A -> B -> C -> D -> A
But what if we have a cycle somewhere else in the list?
e.g.,
A -> B -> C -> D -> E -> C
I can't see that your algorithm will detect the cycle in this case.
Keep in mind that to detect the first case, we need not even break the link. We could just traverse the list and keep comparing the next link for each node with the head element to see if we'd started back at the start yet (or hit the end).
I guess the most trivial approach (not necessarily the best, but one that everybody should know how to implement in Java in a few lines of code) is to build a Hash Set of the nodes, start adding them until you find one that you already saw before. Takes extra memory though.
If you can mark nodes, start marking them until you find one you marked before (the hash map is essentially an external marker).
And check the usual graph theory books...
You are not allowed to break a link, even if you join it back at the end. What if other programs read the list at the same time ?
The algorithm must not damage the list while working on it.
HOW I WISH I HAD PHRASED MY QUESTION TO BEGIN WITH
Take a table with 26 keys, a-z and let them have integer values.
Create a process, Ouch, that does two things over and over again
In one transaction, write random values for a, b, and c such that those values always sum to 10
In another transaction, read the values for a, b and c and complain if their values do not sum to 10
If you spin-up even a few of these processes you will see that very quickly a, b and c are in a state where their values do not sum to 10. I believe there is no way to ask mnesia to "lock these 3 records before starting the writes (or reads)", one can only have mnesia lock the records as it gets to them (so to speak) which allows for the set of records' values to violate my "must sum to 10" constraint.
If I am right, solutions to this problem include
lock the entire table before writing (or reading) the set of 3 records -- I hate to lock whole table for 3 recs,
Create a process that keeps track of who is reading or writing which keys and protects bulk operations from anyone else writing or reading until the operation is completed. Of course I would have to make sure that all processes made use of this... crap, I guess this means writing my own AccessMod as the fourth parameter to activity/4 which seems like a non-trivial exercise
Some other thing that I am not smart enough to figure out.
thoughts?
Ok, I'm an ambitious Erlang newbee, so sorry if this is a dumb question, but
I am building an application-specific, in-memory distributed cache and I need to be able to write sets of Key, Value pairs in one transaction and also retrieve sets of values in one transaction. In other words I need to
1) Write 40 key,value pairs into the cache and ensure that no one else can read or write any of these 40 keys during this multi-key write operation; and,
2) Read 40 keys in one operation and get back 40 values knowing that all 40 values have been unchanged from the moment that this read operation started until it ended.
The only way I can think of doing this is to lock the entire table at the beginning of the fetch_keylist([ListOfKeys]) or at the beginning of the write_keylist([KeyValuePairs], but I don't want to do this because I have many processes simultaneously doing their own multi_key reads and writes and I don't want to lock the entire table any time any process needs to read/write a relatively small subset of records.
Help?
Trying to be more clear: I do not think this is just about using vanilla transactions
I think I am asking a more subtle question than this. Imagine that I have a process that, within a transaction, iterates through 10 records, locking them as it goes. Now imagine this process starts but before it iterates to the 3rd record ANOTHER process updates the 3rd record. This will be just fine as far as transactions go because the first process hadn't locked the 3rd record (yet) and the OTHER process modified it and released it before the first process got to it. What I want is to be guaranteed that once my first process starts that no other process can touch the 10 records until the first process is done with them.
PROBLEM SOLVED - I'M AN IDIOT... I guess...
Thank you all for your patients, especially Hynek -Pichi- Vychodil!
I prepared my test code to show the problem, and I could in fact reproduce the problem. I then simplified the code for readability and the problem went away. I was not able to again reproduce the problem. This is both embarrassing and mysterious to me since I had this problem for days. Also mnesia never complained that I was executing operations outside of a transaction and I have no dirty transactions anywhere in my code, I have no idea how I was able to introduce this bug into my code!
I have pounded the notion of Isolation into my head and will not doubt that it exists again.
Thanks for the education.
Actually, turns out the problem was using try/catch around mnesia operations within a transaction. See here for more.
Mnesia transaction will do exactly this thing for you. It is what is transaction for unless you do dirty operations. So just place your write and read operations to one transaction a mnesia will do rest. All operations in one transaction is done as one atomic operation. Mnesia transaction isolation level is what is sometimes known as "serializable" i.e. strongest isolation level.
Edit:
It seems you missed one important point about concurrent processes in Erlang. (To be fair it is not only true in Erlang but in any truly concurrent environment and when someone arguing else it is not really concurrent environment.) You can't distinguish which action happen first and which happen second unless you do some synchronization. Only way you can do this synchronization is using message passing. You have guaranteed only one thing about messages in Erlang, ordering of messages sent from one process to other process. It means when you send two messages M1 and M2 from process A to process B they arrives in same order. But if you send message M1 from A to B and message M2 from C to B they can arrive in any order. Simply because how you can even tell which message you sent first? It is even worse if you send message M1 from A to B and then M2 from A to C and when M2 arrives to C send M3 from C to B you don't have guarantied that M1 arrives to B before M3. Even it will happen in one VM in current implementation. But you can't rely on it because it is not guaranteed and can change even in next version of VM just due message passing implementation between different schedulers.
It illustrates problems of event ordering in concurrent processes. Now back to the mnesia transaction. Mnesia transaction have to be side effect free fun. It means there may not be any message sending outside from transaction. So you can't tell which transaction starts first and when starts. Only thing you can tell if transaction succeed and they order you can only determine by its effect. When you consider this your subtle clarification makes no sense. One transaction will read all keys in atomic operation even it is implemented as reading one key by one in transaction implementation and your write operation will be also performed as atomic operation. You can't tell if write to 4th key in second transaction was happen after you read 1st key in first transaction because there it is not observable from outside. Both transaction will be performed in particular order as separate atomic operation. From outside point of view all keys will be read in same point of time and it is work of mnesia to force it. If you send message from inside of transaction you violate mnesia transaction property and you can't be surprised it will behave strange. To be concrete, this message can be send many times.
Edit2:
If you spin-up even a few of these processes you will see that very
quickly a, b and c are in a state where their values do not sum to 10.
I'm curious why you think it would happen or you tested it? Show me your test case and I will show mine:
-module(transactions).
-export([start/2, sum/0, write/0]).
start(W, R) ->
mnesia:start(),
{atomic, ok} = mnesia:create_table(test, [{ram_copies,[node()]}]),
F = fun() ->
ok = mnesia:write({test, a, 10}),
[ ok = mnesia:write({test, X, 0}) || X <-
[b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y,z]],
ok
end,
{atomic, ok} = mnesia:transaction(F),
F2 = fun() ->
S = self(),
erlang:send_after(1000, S, show),
[ spawn_link(fun() -> writer(S) end) || _ <- lists:seq(1,W) ],
[ spawn_link(fun() -> reader(S) end) || _ <- lists:seq(1,R) ],
collect(0,0)
end,
spawn(F2).
collect(R, W) ->
receive
read -> collect(R+1, W);
write -> collect(R, W+1);
show ->
erlang:send_after(1000, self(), show),
io:format("R: ~p, W: ~p~n", [R,W]),
collect(R, W)
end.
keys() ->
element(random:uniform(6),
{[a,b,c],[a,c,b],[b,a,c],[b,c,a],[c,a,b],[c,b,a]}).
sum() ->
F = fun() ->
lists:sum([X || K<-keys(), {test, _, X} <- mnesia:read(test, K)])
end,
{atomic, S} = mnesia:transaction(F),
S.
write() ->
F = fun() ->
[A, B ] = L = [ random:uniform(10) || _ <- [1,2] ],
[ok = mnesia:write({test, K, V}) || {K, V} <- lists:zip(keys(),
[10-A-B|L])],
ok
end,
{atomic, ok} = mnesia:transaction(F),
ok.
reader(P) ->
case sum() of
10 ->
P ! read,
reader(P);
_ ->
io:format("ERROR!!!~n",[]),
exit(error)
end.
writer(P) ->
ok = write(),
P ! write,
writer(P).
If it would not work it would be really serious problem. There are serious applications including payment systems which rely on it. If you have test case which shows it is broken, please report bug at erlang-bugs#erlang.org
Have you tried mnesia Events ? You can have the reader subscribe to mnesia's Table Events especially write events so as not to interrupt the process doing the writing. In this way, mnesia just keeps sending a copy of what has been written in real-time to the other process which checks what the values are at any one time. take a look at this:
subscriber()->
mnesia:subscribe({table,YOUR_TABLE_NAME,simple}),
%% OR mnesia:subscribe({table,YOUR_TABLE_NAME,detailed}),
wait_events().
wait_events()->
receive
%% For simple events
{mnesia_table_event,{write, NewRecord, ActivityId}} ->
%% Analyse the written record as you wish
wait_events();
%% For detailed events
{mnesia_table_event,{write, YOUR_TABLE, NewRecord, [OldRecords], ActivityId}} ->
%% Analyse the written record as you wish
wait_events();
_Any -> wait_events()
end.
Now you spawn your analyser as a process like this:
spawn(?MODULE,subscriber,[]).
This makes the whole process to run without any process being blocked, mnesia needs not lock any tabel or record because now what you have is a writer process and an analyser process. The whole thing will run in real-time. Remember that there are many other events that you can make use of if you wish by pattern matching them in the subscriber wait_events() receive body.
Its possible to build a heavy duty gen_server or complete application intended for reception and analysis of all your mnesia events. Its usually better to have one capable subscriber than many failing event subscribers. If i have understood you question well, this unblocking solution fits your requirements.
mnesia:read/3 with write locks seems to be suffient.
Mnesia's transaction is implemented by read-write lock and locks are well-formed (holding lock untill the end of transaction). So the isolation level is serializable.
The granularity of locks are per record as long as you access by primary key.
i am writing records into mnesia which should be kept there
only for an allowed time (24 hours). after 24 hours, before a user modifies part of them,
the system should remove them automatically. forexample, a user is given free airtime (for voice calls)
which they should use in a given time. if they do not use it, after 24 hours, the system should
remove these resource reservation from the users record.
Now, this has brought in timers. an example of a record structure is:
-record(free_airtime,
{
reference_no,
timer_object, %% value returned by timer:apply_after/4
amount
}).
The timer object in the record is important because in case the user
finally puts to use the resources reserved before they are timed out
(or if they time out),the system can call timer:cancel/1 so as to relieve
the timer server from this object.
Now the problem, i have two ways of handling timers on these records:
Option 1: timers handled within the transaction
reserve_resources(Reference_no,Amnt)->
F = fun(Ref_no,Amount) ->
case mnesia:read({free_airtime,Ref_no}) of
[] ->
case mnesia:write(#free_airtime{reference_no = Ref_no,amount = Amount}) == ok of
true ->
case timer:apply_after(timer:hours(24),?MODULE,reference_no_timed_out,[Ref_no]) of
{ok,Timer_obj} ->
[Obj] = mnesia:read({free_airtime,Ref_no}),
mnesia:write(Obj#free_airtime{timer_object = Timer_obj});
_ -> mnesia:abort({error,failed_to_time_object})
end;
false -> mnesia:abort({error,write_failed})
end;
[_] -> mnesia:abort({error,exists,Ref_no})
end
end,
mnesia:activity(transaction,F,[Reference_no,Amnt],mnesia_frag).
About the above option.
Mnesia docs say that transactions maybe repeated by the tm manager (due to some reason)
until they are successful, and so when you put code which is io:format/2 or any other which has nothing to do with
writes or reads, it may get executed several times. This statement made me pause at this point
and think of a way of handling timers out of the transaction it self, so i modified the code as
follows:
Option 2: timers handled outside the transaction
reserve_resources(Reference_no,Amnt)->
F = fun(Ref_no,Amount) ->
case mnesia:read({free_airtime,Ref_no}) of
[] ->
P = #free_airtime{reference_no = Ref_no,amount = Amount},
ok = mnesia:write(P),
P;
[_] -> mnesia:abort({error,exists,Ref_no})
end
end,
Result = try mnesia:activity(transaction,F,[Reference_no,Amnt],mnesia_frag) of
Any -> Any
catch
exit:{aborted,{error,exists,XX}} -> {exists,XX}
E1:E2 -> {error,{E1,E2}}
end,
on_reservation(Result).
on_reservation(#free_airtime{reference_no = Some_Ref})->
case timer:apply_after(timer:hours(24),?MODULE,reference_no_timed_out,[Some_Ref]) of
{ok,Timer_obj} ->
[Obj] = mnesia:activity(transaction,fun(XX) -> mnesia:read({free_airtime,XX}) end,[Some_Ref],mnesia_frag),
ok = mnesia:activity(transaction,fun(XX) -> mnesia:write(XX) end,[Obj#free_airtime{timer_object = Timer_obj}],mnesia_frag);
_ ->
ok = mnesia:activity(transaction,fun(XX) -> mnesia:delete({free_airtime,XX}) end,[Some_Ref],mnesia_frag),
{error,failed_to_time_object}
end;
on_reservation(Any)-> Any.
The code to handle time out of the reservation:
reference_no_timed_out(Ref_no)->
do_somethings_here.....
then later remove this reservation from the database....below..
ok = mnesia:activity(transaction,fun(XX) -> mnesia:delete({free_airtime,XX}) end,[Ref_no],mnesia_frag).
Now i thought that in option 2, i am safer by keeping the timer processing
code out, even when mnesia_tm re-executes the transaction due to its reasons
, this piece of code is not run twice (i avoid having several timer objects
against the same record).
Question 1: Which of these two implementations is right? and/or wrong? Tell me (also)
wether both of them are wrong
Question 2: The module timer, is it well suited for handling large numbers of timer
jobs in production?
Question 3: As compared to Sean Hinde's timer_mn-1.1,
which runs on top of mnesia, is the timer module (possibly running on top of Ets tables) less
capable (for real) in production?
(am asking this because using Sean Hinde's timer_mn on a system which itself is using mnesia appears
to be a problem in terms schema changes, node problems e.t.c)
If any one has another way of handling timer related problems with mnesia, update me
thanx guys...
Question 1:
Handle the timer outside the transaction. When transactions collide in Mnesia, they are simply repeated. That would give you more than one timer reference and two triggers of the timer. It is not a problem per se, but if you wait until the success of the transaction before installing the timer, you can avoid the problem.
The second solution is what I would do. If the TX is okay, you can install a timer on it. If the timer triggers and there is no reference to the object, it doesn't matter. You are only to worry about if this situation happens a lot since you would then have a large number of stray timers.
Question 2:
The timer module is neat, but the performance guide recommends you use the erlang:start_timer BIFs instead, see
http://www.erlang.org/doc/efficiency_guide/commoncaveats.html#id58959
I would introduce a separate process as a gen_server which handles the timing stuff. You send it a remove(timer:hours(24), RefNo) message and then it starts up a timer, gets a TRef and installs a mapping {TRef, RefNo, AuxData} in either Mnesia or ETS. When the timer trigger, the process can spawn a helper removing the RefNo entry from the main table.
At this point, you must wonder about crashes. The removal gen_server may crash. Also, the whole node may crash. How you want to reinstall timers in the case this happens is up to you, but you ought to ponder on it happening so you can solve it. Suppose we come up again and the timer information is loaded in from disk. How do you plan on reinstalling the timers?
One way is to have AuxData contain information about the timeout point. Every hour or 15 minutes, you scan all of the table, removing guys that shouldn't be there. In fact, you could opt for this being the main way to remove timer structures. Yes, you will give people 15 minutes of extra time in the worst case, but it may be easier to handle code-wise. At least it better handles the case where the node (and thus the timers) die.
Another option again is to cheat and only store timings rougly in a data structure which makes it very cheap to find all expired RefNo's in the last 5 minutes and then run that every 5 minutes. Doing stuff in bulk is probably going to be more effective. This kind of bulk-handling is used a lot by operating system kernels for instance.
Question 3
I know nothing about timer-tm, sorry :)