I am developing a simple framework in Erlang to handle 2-player turn-based games. The code is the following:
-module(game).
-export([start_server/0,generate_server/0,add_player/0,remove_player/0]).
generate_server() ->
Table_num = 0,
Player_num = 0,
io:format("Server generated...~n", []),
io:format("The current number of tables is ~w~n", [Table_num]),
io:format("The current number of players is ~w~n", [Player_num]),
receive
login ->
io:format("A new player has connected!~n", []),
New = Player_num + 1,
io:format("The current number of players is ~w~n", [New]);
logout ->
io:format("You have beeen succesfully disconnected~n", [])
end.
start_server() ->
io:format("Welcome player!~nInitializing game...~n", []),
io:format("Generating server...~n", []),
register(server,spawn(game, generate_server, [])).
add_player() ->
server ! login.
remove_player() ->
server ! logout.
There are two main problems when I run this code:
When I execute add_player() and then remove_player(), this second function crashes with an exception
If I launch the program on one terminal window and then execute add_player() on a second terminal windows, I get an error. What should I do to be able to run it on more than one terminal window?
Any help would be highly appreciated.
1/
There is no loop in your server. When you start it, after some print, it waits on the receive statement.
When it receives the login message, it executes the operations, and then the server code is finished; the process die and is unregistered. All variables disappear and the VM clean up the memory...
So, later on, any process sending a message to the server will crash because you are using a name which is no more registered.
To make it works you should keep a list of connected players (in list for example) and recall the server loop with this list as parameter.
generate_server(Tlist,Plist) ->
io:format("The current number of tables is ~w~n", [length(Tlist)]),
io:format("The current number of players is ~w~n", [length(Plist)]),
receive
{login,Name} ->
io:format("A new player ~p has connected!~n", [Name]),
generate_server(Tlist,[Name|Plist]);
{logout,Name} ->
io:format("You have beeen succesfully disconnected~n", []),
generate_server(Tlist,lists:delete(Name,Plist))
end.
and the call to generate_server is done by
register(server,spawn(game, generate_server, [[],[]]))
2/
in order to use erlang messages between 2 different nodes, you need to:
share the same erlang coockie
discover the nodes (using netadm for example)
get the server pid or use golbal registered name
see example at http://learnyousomeerlang.com/distribunomicon#alone-in-the-dark
You missed out a loop body for the server. Your program crashes because the server receives only one message and exits. Consider another version of the server below:
generate_server() ->
Table_num = 0,
Player_num = 0,
io:format("Server generated...~n", []),
io:format("The current number of tables is ~w~n", [Table_num]),
io:format("The current number of players is ~w~n", [Player_num]),
loop([]).
loop(Players)->
receive
{From,{login,PlayerId}} ->
io:format("A new player has connected!~n", []),
io:format("The current number of players is ~w~n", [New]),
NewPlayers = case lists:member(PlayerId,Players) of
true ->
From ! {login_failed,exists},
Players;
false ->
From ! {login_success,true},
[PlayerId|Players]
end,
loop(NewPlayers);
{From,{logout,PlayerId}} ->
NewPlayers = case lists:member(PlayerId,Players) of
true ->
From ! {logout,ok},
Players -- [PlayerId];
false ->
From ! {logout,failed},
Players
end, loop(NewPlayers);
_ -> loop(Players)
end.
There; that looks much better.
Related
I am writing a program that solves producers-consumers problem using Erlang multiprocessing with one process responsible for handling buffer to which I produce/consume and many producers and many consumers processes. To simplify I assume producer/consumer does not know that his operation has failed (that it is impossible to produce or consume because of buffer constraints), but the server is prepared to do this.
My code is:
Server code
server(Buffer, Capacity, CountPid) ->
receive
%% PRODUCER
{Pid, produce, InputList} ->
NumberProduce = lists:flatlength(InputList),
case canProduce(Buffer, NumberProduce, Capacity) of
true ->
NewBuffer = append(InputList, Buffer),
CountPid ! lists:flatlength(InputList),
Pid ! ok,
server(NewBuffer,Capacity, CountPid);
false ->
Pid ! tryagain,
server(Buffer, Capacity, CountPid)
end;
%% CONSUMER
{Pid, consume, Number} ->
case canConsume(Buffer, Number) of
true ->
Data = lists:sublist(Buffer, Number),
NewBuffer = lists:subtract(Buffer, Data),
Pid ! {ok, Data},
server(NewBuffer, Capacity,CountPid);
false ->
Pid ! tryagain,
server(Buffer, Capacity, CountPid)
end
end.
Producer and consumer
producer(ServerPid) ->
X = rand:uniform(9),
ToProduce = [rand:uniform(500) || _ <- lists:seq(1, X)],
ServerPid ! {self(),produce,ToProduce},
producer(ServerPid).
consumer(ServerPid) ->
X = rand:uniform(9),
ServerPid ! {self(),consume,X},
consumer(ServerPid).
Starting and auxiliary functions (I enclose as I don't know where exactly my problem is)
spawnProducers(Number, ServerPid) ->
case Number of
0 -> io:format("Spawned producers");
N ->
spawn(zad2,producer,[ServerPid]),
spawnProducers(N - 1,ServerPid)
end.
spawnConsumers(Number, ServerPid) ->
case Number of
0 -> io:format("Spawned producers");
N ->
spawn(zad2,consumer,[ServerPid]),
spawnProducers(N - 1,ServerPid)
end.
start(ProdsNumber, ConsNumber) ->
CountPid = spawn(zad2, count, [0,0]),
ServerPid = spawn(zad2,server,[[],20, CountPid]),
spawnProducers(ProdsNumber, ServerPid),
spawnConsumers(ConsNumber, ServerPid).
canProduce(Buffer, Number, Capacity) ->
lists:flatlength(Buffer) + Number =< Capacity.
canConsume(Buffer, Number) ->
lists:flatlength(Buffer) >= Number.
append([H|T], Tail) ->
[H|append(T, Tail)];
append([], Tail) ->
Tail.
I am trying to count number of elements using such process, server sends message to it whenever elements are produced.
count(N, ThousandsCounter) ->
receive
X ->
if
N >= 1000 ->
io:format("Yeah! We have produced ~p elements!~n", [ThousandsCounter]),
count(0, ThousandsCounter + 1000);
true -> count(N + X, ThousandsCounter)
end
end.
I expect this program to work properly, which means: it produces elements, increase of produced elements depends on time like f(t) = kt, k-constant and the more processes I have the faster production is.
ACTUAL QUESTION
I launch program:
erl
c(zad2)
zad2:start(5,5)
How the program behaves:
The longer production lasts the less elements in the unit of time are being produced (e.g. in first second 10000, in next 5000, in 10th second 1000 etc.
The more processes I have, the slower production is, in start(10,10) I need to wait about a second for first thousand, whereas for start(2,2) 20000 appears almost immediately
start(100,100) made me restart my computer (I work on Ubuntu) as the whole CPU was used and there was no memory available for me to open terminal and terminate erlang machine
Why does my program not behave like I expect? Am I doing something wrong with Erlang programming or is this the matter of OS or anything else?
The producer/1 and consumer/1 functions as written above don't ever wait for anything - they just loop and loop, bombarding the server with messages. The server's message queue is filling up very quickly, and the Erlang VM will try to grow as much as it can, stealing all your memory, and the looping processes will steal all available CPU time on all cores.
I'm trying to understand request order in erlang, but I can't seem to grasp it very well. Here's the example code:
test() ->
Server = start(),
spawn(fun() -> write(Server, 2),
io:format(”Child 1 read ~p~n”, [read(Server)]) end),
write(Server, 3),
write2(Server, 5),
spawn(fun() -> write2(Server, 1),
io:format(”Child 2 read ~p~n”, [read(Server)]) end),
io:format(”Parent read ~p~n”, [read(Server)]).
And here's the server:
start() ->
spawn(fun() -> init() end).
init() -> loop(0).
loop(N) ->
receive
{read, Pid} ->
Pid ! {value, self(), N},
loop(N);
{write, Pid, M} ->
Pid ! {write_reply, self()},
loop(M);
{write2, _Pid, M} -> loop(M)
end.
read(Serv) ->
Serv ! {read, self()},
receive {value, Serv, N} -> N end.
write(Serv, N) ->
Serv ! {write, self(), N},
receive {write_reply, Serv} -> ok end.
write2(Serv, N) ->
Serv ! {write2, self(), N},
ok.
I understand that different values could be printed by the three different processes created in test/0, but I'm trying to figure out the lowest and highest values that could be printed by those Parent, Child1 and Child2 processes. The answer states:
Parent: lowest 1, highest 5
Child1: lowest 1, highest 5
Child2: lowest 1, highest 2
Can somebody explain this?
Keep in mind that Erlang guarantees message order only from one process to another. If process A sequentially sends message 1 and then message 2 to process B, then B will receive them in that order. But Erlang guarantees no specific ordering of messages arriving at B if multiple concurrent processes are sending them. In this example, Parent, Child1, and Child2 all run concurrently and all send messages concurrently to Server.
The Parent process performs the following sequential steps:
Spawns the Server process. This eventually sets the initial value in the Server loop to 0.
Spawns Child1. This eventually writes the value 2 to Server, then reads from Server and prints the result.
Uses write/2 to send the value 3 to Server. The write case in the loop/1 function first replies to the caller and then installs the value 3 for its next iteration.
Uses write2/2 to send 5 to Server. The write2/2 function just sends a message to Server and does not await a reply. The write2 case in the loop/1 function just installs the value 5 for its next iteration.
Spawns Child2, which eventually calls write2/2 with the value 1, and then reads Server and prints the result.
Reads Server and prints the result.
For Parent, step 3 sends the value 3 to Server, so as far as Parent is concerned, Server now has the value 3. In step 4, Parent calls write2/2 to send 5 to the server, and that message must arrive at Server sometime after the message sent in step 3. In step 6, Parent performs a read, but all we know is that this message has to arrive at Server after the write message in step 4. This message ordering means the highest value Parent can see is 5.
The lowest value Parent can see is 1 because if Child2 gets its write message of 1 to Server after the Parent write message of 5 but before the final Parent read message, then Parent will see the 1.
For Child1, the highest value it can see is 5 because it runs concurrently with Parent and the two messages sent by Parent could arrive at Server before its write message of 2. The lowest Child1 can see is 1 because the Child2 write message of 1 can arrive before the Child1 read message.
For Child2, the lowest value it can see is its own write of 1. The highest value it can see is 2 from the write of Child1 because the Parent writes of 3 and 5 occur before Child2 is spawned, thus Child1 is the only process writing concurrently and so only it has a chance of interleaving its write message between the Child2 write and read messages.
Hi I'm trying to make a cursor in mnesia from a remote node, for example:
I have the node which is the mnesia database owner and runs important processes in a dedicated server machine, and another node which have a process running in other computer and have to go through all the items to make some simple operations with the data. The thing is that I can make the miscellaneous process work if runs in the same node of mnesia but not remotely.
This is my code that runs locally:
make_cursor_local() ->
QD = qlc:sort(mnesia:table(customer, [{traverse, select}])),
mnesia:activity(async_dirty, fun() -> qlc:cursor(QD) end, mnesia_frag).
get_next_local(Cursor) ->
Get = fun() -> qlc:next_answers(Cursor,100) end,
mnesia:activity(async_dirty, Get, mnesia_frag).
del_cursor_local(Cursor) ->
qlc:delete_cursor(Cursor).
This is my actual code using rpc:
make_cursor() ->
Sort = rpc:call(?NamespaceNode, mnesia, table, [customer, [{traverse, select}]]),
QD = rpc:call(?NamespaceNode, qlc, sort, [Sort]),
CursorCreation = fun() -> qlc:cursor(qlc:sort(Sort)) end,
Cursor = rpc:call(?NamespaceNode, mnesia, activity, [async_dirty, CursorCreation, mnesia_frag]),
Cursor.
get_next(Cursor) ->
Get = fun() -> rpc:call(?NamespaceNode, qlc, next_answers, [Cursor, 100]) end,
Next = rpc:call(?NamespaceNode, mnesia, activity, [async_dirty, Get, mnesia_frag]),
Next.
del_cursor(Cursor) ->
rpc:call(?NamespaceNode, qlc, delete_cursor, [Cursor]).
This code is generating this error making the mnesia activity call in the make_cursor function:
{badrpc,
{'EXIT',
{undef,
[{#Fun<cleaner_app.2.116369932>,[],[]},
{mnesia_tm,non_transaction,5,
[{file,"mnesia_tm.erl"},{line,738}]},
{rpc,'-handle_call_call/6-fun-0-',5,
[{file,"rpc.erl"},{line,205}]}]}}} {badrpc,{'EXIT',{undef,[{#Fun<misc_app.2.116369932>,[],
[]},
{mnesia_tm,non_transaction,5,
[{file,"mnesia_tm.erl"},{line,738}]},
{rpc,'-handle_call_call/6-fun-0-',5,
[{file,"rpc.erl"},{line,205}]}]}}}
I found that the remote node can't execute anonymous functions created in another node so the error posted can be solved this way:
make_cursor() ->
Sort = rpc:call(?NamespaceNode, mnesia, table, [customer, [{traverse, select}]]),
QD = rpc:call(?NamespaceNode, qlc, sort, [Sort]),
rpc:call(?NamespaceNode, mnesia, activity, [sync_transaction, fun qlc:cursor/1, [QD], mnesia_frag]).
I found the answer here what kind of types can be sent on an erlang message?
Now I have to solve the Cursor ownership because I cant execute it from the remote node.
This is the error:
{badrpc,
{'EXIT',
{aborted,
{not_cursor_owner,
[{qlc,next_answers,
[{qlc_cursor,{<6920.991.0>,<6920.990.0>}},100],
[{file,"qlc.erl"},{line,515}]},
{mnesia_tm,apply_fun,3,[{file,"mnesia_tm.erl"},{line,833}]},
{mnesia_tm,execute_transaction,5,
[{file,"mnesia_tm.erl"},{line,813}]},
{mnesia,wrap_trans,6,[{file,"mnesia.erl"},{line,394}]},
{rpc,'-handle_call_call/6-fun-0-',5,
[{file,"rpc.erl"},{line,205}]}]}}}}
I am learning Erlang from a Ruby background and having some difficulty grasping the thought process. The problem I am trying to solve is the following:
I need to make the same request to an api, each time I receive a unique ID in the response which I need to pass into the next request until there is not ID returned. From each response I need to extract certain data and use it for other things as well.
First get the iterator:
ShardIteratorResponse = kinetic:get_shard_iterator(GetShardIteratorPayload).
{ok,[{<<"ShardIterator">>,
<<"AAAAAAAAAAGU+v0fDvpmu/02z5Q5OJZhPo/tU7fjftFF/H9M7J9niRJB8MIZiB9E1ntZGL90dIj3TW6MUWMUX67NEj4GO89D"...>>}]}
Parse out the shard_iterator..
{_, [{_, ShardIterator}]} = ShardIteratorResponse.
Make the request to kinesis for the streams records...
GetRecordsPayload = [{<<"ShardIterator">>, <<ShardIterator/binary>>}].
[{<<"ShardIterator">>,
<<"AAAAAAAAAAGU+v0fDvpmu/02z5Q5OJZhPo/tU7fjftFF/H9M7J9niRJB8MIZiB9E1ntZGL90dIj3TW6MUWMUX67NEj4GO89DETABlwVV"...>>}]
14> RecordsResponse = kinetic:get_records(GetRecordsPayload).
{ok,[{<<"NextShardIterator">>,
<<"AAAAAAAAAAFy3dnTJYkWr3gq0CGo3hkj1t47ccUS10f5nADQXWkBZaJvVgTMcY+nZ9p4AZCdUYVmr3dmygWjcMdugHLQEg6x"...>>},
{<<"Records">>,
[{[{<<"Data">>,<<"Zmlyc3QgcmVjb3JkISEh">>},
{<<"PartitionKey">>,<<"BlanePartitionKey">>},
{<<"SequenceNumber">>,
<<"49545722516689138064543799042897648239478878787235479554">>}]}]}]}
What I am struggling with is how do I write a loop that keeps hitting the kinesis endpoint for that stream until there are no more shard iterators, aka I want all records. Since I can't re-assign the variables as I would in Ruby.
WARNING: My code might be bugged but it's "close". I've never ran it and don't see how last iterator can look like.
I see you are trying to do your job entirely in shell. It's possible but hard. You can use named function and recursion (since release 17.0 it's easier), for example:
F = fun (ShardIteratorPayload) ->
{_, [{_, ShardIterator}]} = kinetic:get_shard_iterator(ShardIteratorPayload),
FunLoop =
fun Loop(<<>>, Accumulator) -> % no clue how last iterator can look like
lists:reverse(Accumulator);
Loop(ShardIterator, Accumulator) ->
{ok, [{_, NextShardIterator}, {<<"Records">>, Records}]} =
kinetic:get_records([{<<"ShardIterator">>, <<ShardIterator/binary>>}]),
Loop(NextShardIterator, [Records | Accumulator])
end,
FunLoop(ShardIterator, [])
end.
AllRecords = F(GetShardIteratorPayload).
But it's too complicated to type in shell...
It's much easier to code it in modules.
A common pattern in erlang is to spawn another process or processes to fetch your data. To keep it simple you can spawn another process by calling spawn or spawn_link but don't bother with links now and use just spawn/3.
Let's compile simple consumer module:
-module(kinetic_simple_consumer).
-export([start/1]).
start(GetShardIteratorPayload) ->
Pid = spawn(kinetic_simple_fetcher, start, [self(), GetShardIteratorPayload]),
consumer_loop(Pid).
consumer_loop(FetcherPid) ->
receive
{FetcherPid, finished} ->
ok;
{FetcherPid, {records, Records}} ->
consume(Records),
consumer_loop(FetcherPid);
UnexpectedMsg ->
io:format("DROPPING:~n~p~n", [UnexpectedMsg]),
consumer_loop(FetcherPid)
end.
consume(Records) ->
io:format("RECEIVED:~n~p~n",[Records]).
And fetcher:
-module(kinetic_simple_fetcher).
-export([start/2]).
start(ConsumerPid, GetShardIteratorPayload) ->
{ok, [ShardIterator]} = kinetic:get_shard_iterator(GetShardIteratorPayload),
fetcher_loop(ConsumerPid, ShardIterator).
fetcher_loop(ConsumerPid, {_, <<>>}) -> % no clue how last iterator can look like
ConsumerPid ! {self(), finished};
fetcher_loop(ConsumerPid, ShardIterator) ->
{ok, [NextShardIterator, {<<"Records">>, Records}]} =
kinetic:get_records(shard_iterator(ShardIterator)),
ConsumerPid ! {self(), {records, Records}},
fetcher_loop(ConsumerPid, NextShardIterator).
shard_iterator({_, ShardIterator}) ->
[{<<"ShardIterator">>, <<ShardIterator/binary>>}].
As you can see both processes can do their job concurrently.
Try from your shell:
kinetic_simple_consumer:start(GetShardIteratorPayload).
Now your see that your shell process turns to consumer and you will have your shell back after fetcher will send {ItsPid, finished}.
Next time instead of
kinetic_simple_consumer:start(GetShardIteratorPayload).
run:
spawn(kinetic_simple_consumer, start, [GetShardIteratorPayload]).
You should play with spawning processes - it's erlang main strength.
In Erlang, you can write loop using tail recursive functions. I don't know the kinetic API, so for simplicity, I just assume, that kinetic:next_iterator/1 return {ok, NextIterator} or {error, Reason} when there are no more shards.
loop({error, Reason}) ->
ok;
loop({ok, Iterator}) ->
do_something_with(Iterator),
Result = kinetic:next_iterator(Iterator),
loop(Result).
You are replacing loop with iteration. First clause deals with case, where there are no more shards left (always start recursion with the end condition). Second clause deals with case, where we got some iterator, we do something with it and call next.
The recursive call is last instruction in the function body, which is called tail recursion. Erlang optimizes such calls - they don't use call stack, so they can run infinitely in constant memory (you will not get anything like "Stack level too deep")
In most applications, its hard to avoid the need to query large amounts of information which a user wants to browse through. This is what led me to cursors. With mnesia, cursors are implemented using qlc:cursor/1 or qlc:cursor/2. After working with them for a while and facing this problem many times,
11> qlc:next_answers(QC,3).
** exception error: {qlc_cursor_pid_no_longer_exists,<0.59.0>}
in function qlc:next_loop/3 (qlc.erl, line 1359)
12>
It has occured to me that the whole cursor thing has to be within one mnesia transaction: executes as a whole once. like this below
E:\>erl
Eshell V5.9 (abort with ^G)
1> mnesia:start().
ok
2> rd(obj,{key,value}).
obj
3> mnesia:create_table(obj,[{attributes,record_info(fields,obj)}]).
{atomic,ok}
4> Write = fun(Obj) -> mnesia:transaction(fun() -> mnesia:write(Obj) end) end.
#Fun<erl_eval.6.111823515>
5> [Write(#obj{key = N,value = N * 2}) || N <- lists:seq(1,100)],ok.
ok
6> mnesia:transaction(fun() ->
QC = cursor_server:cursor(qlc:q([XX || XX <- mnesia:table(obj)])),
Ans = qlc:next_answers(QC,3),
io:format("\n\tAns: ~p~n",[Ans])
end).
Ans: [{obj,20,40},{obj,21,42},{obj,86,172}]
{atomic,ok}
7>
When you attempt to call say: qlc:next_answers/2 outside a mnesia transaction, you will get an exception. Not only just out of the transaction, but even if that method is executed by a DIFFERENT process than the one which created the cursor, problems are bound to happen.
Another intresting finding is that, as soon as you get out of a mnesia transaction, one of the processes which are involved in a mnesia cursor (apparently mnesia spawns a process in the background), exits, causing the cursor to be invalid. Look at this below:
-module(cursor_server).
-compile(export_all).
cursor(Q)->
case mnesia:is_transaction() of
false ->
F = fun(QH)-> qlc:cursor(QH,[]) end,
mnesia:activity(transaction,F,[Q],mnesia_frag);
true -> qlc:cursor(Q,[])
end.
%% --- End of module -------------------------------------------
Then in shell, i use that method:
7> QC = cursor_server:cursor(qlc:q([XX || XX <- mnesia:table(obj)])).
{qlc_cursor,{<0.59.0>,<0.30.0>}}
8> erlang:is_process_alive(list_to_pid("<0.59.0>")).
false
9> erlang:is_process_alive(list_to_pid("<0.30.0>")).
true
10> self().
<0.30.0>
11> qlc:next_answers(QC,3).
** exception error: {qlc_cursor_pid_no_longer_exists,<0.59.0>}
in function qlc:next_loop/3 (qlc.erl, line 1359)
12>
So, this makes it very Extremely hard to build a web application in which a user needs to browse a particular set of results, group by group say: give him/her the first 20, then next 20 e.t.c. This involves, getting the first results, send them to the web page, then wait for the user to click NEXT then ask qlc:cursor/2 for the next 20 and so on. These operations cannot be done, while hanging inside a mnesia transaction !!! The only possible way, is by spawning a process which will hang there, receiving and sending back next answers as messages and receiving the next_answers requests as messages like this:
-define(CURSOR_TIMEOUT,timer:hours(1)).
%% initial request is made here below
request(PageSize)->
Me = self(),
CursorPid = spawn(?MODULE,cursor_pid,[Me,PageSize]),
receive
{initial_answers,Ans} ->
%% find a way of hidding the Cursor Pid
%% in the page so that the subsequent requests
%% come along with it
{Ans,pid_to_list(CursorPid)}
after ?CURSOR_TIMEOUT -> timedout
end.
cursor_pid(ParentPid,PageSize)->
F = fun(Pid,N)->
QC = cursor_server:cursor(qlc:q([XX || XX <- mnesia:table(obj)])),
Ans = qlc:next_answers(QC,N),
Pid ! {initial_answers,Ans},
receive
{From,{next_answers,Num}} ->
From ! {next_answers,qlc:next_answers(QC,Num)},
%% Problem here ! how to loop back
%% check: Erlang Y-Combinator
delete ->
%% it could have died already, so we be careful here !
try qlc:delete_cursor(QC) of
_ -> ok
catch
_:_ -> ok
end,
exit(normal)
after ?CURSOR_TIMEOUT -> exit(normal)
end
end,
mnesia:activity(transaction,F,[ParentPid,PageSize],mnesia_frag).
next_answers(CursorPid,PageSize)->
list_to_pid(CursorPid) ! {self(),{next_answers,PageSize}},
receive
{next_answers,Ans} ->
{Ans,pid_to_list(CursorPid)}
after ?CURSOR_TIMEOUT -> timedout
end.
That would create a more complex problem of managing process exits, tracking / monitoring e.t.c. I wonder why the mnesia implementers didnot see this !
Now, that brings me to my questions. I have been walking around the web for solutions and you could please check out these links from which the questions arise: mnemosyne, Ulf Wiger's Solution to Cursor Problems, AMNESIA - an RDBMS implementation of mnesia.
1. Does anyone have an idea on how to handle mnesia query cursors in a different way from what is documented, and is worth sharing ?
2. What are the reasons why mnesia implemeters decided to force the cursors within a single transaction: even the calls for the next_answers ?
3. Is there anything, from what i have presented, that i do not understand clearly (other than my bad buggy illustration code - please ignore those) ?
4. AMNESIA (on section 4.7 of the link i gave above), has a good implementation of cursors, because the subsequent calls for the next_answers, do not need to be in the same transaction, NOR by the same process. Would you advise anyone to switch from mnesia to amnesia due to this and also, is this library still supported ?
5. Ulf Wiger , (the author of many erlang libraries esp. GPROC), suggests the use of mnesia:select/4. How would i use it to solve cursor problems in a web application ? NOTE: Please do not advise me to leave mnesia and use something else, because i want to use mnesia for this specific problem. I appreciate your time to read through all this question.
The motivation is that transaction grabs (in your case) read locks.
Locks can not be kept outside of transactions.
If you want, you can run it in a dirty_context, but you loose the
transactional properties, i.e. the table may change between invocations.
make_cursor() ->
QD = qlc:sort(mnesia:table(person, [{traverse, select}])),
mnesia:activity(async_dirty, fun() -> qlc:cursor(QD) end, mnesia_frag).
get_next(Cursor) ->
Get = fun() -> qlc:next_answers(Cursor,5) end,
mnesia:activity(async_dirty, Get, mnesia_frag).
del_cursor(Cursor) ->
qlc:delete_cursor(Cursor).
I think this may help you :
use async_dirty instead of transaction
{Record,Cont}=mnesia:activity(async_dirty, fun mnesia:select/4,[md,[{Match_head,[Guard],[Result]}],Limit,read])
then read next Limit number of records:
mnesia:activity(async_dirty, fun mnesia:select/1,[Cont])
full code:
-record(md,{id,name}).
batch_delete(Id,Limit) ->
Match_head = #md{id='$1',name='$2'},
Guard = {'<','$1',Id},
Result = '$_',
{Record,Cont} = mnesia:activity(async_dirty, fun mnesia:select/4,[md,[{Match_head,[Guard],[Result]}],Limit,read]),
delete_next({Record,Cont}).
delete_next('$end_of_table') ->
over;
delete_next({Record,Cont}) ->
delete(Record),
delete_next(mnesia:activity(async_dirty, fun mnesia:select/1,[Cont])).
delete(Records) ->
io:format("delete(~p)~n",[Records]),
F = fun() ->
[ mnesia:delete_object(O) || O <- Records]
end,
mnesia:transaction(F).
remember you can not use cursor out of one transaction