I'm trying to save offline ejabberd messages in Riak. I earlier had problems connecting to Riak but those are resolved now with the help of this forum. But now, with my limited Erlang / ejabberd understanding, I'm failing to get the ejabberd packet saved as a string and then put on Riak. Essentially, when the offline_message_hook is latched, I take the Packet argument and then want to put a backslash for each double quote, so that I can then take this revised string and save as a string value on Riak. However I seem to be struggling with modifying the incoming packet to replace the " chars with \".
Is this the right approach? Or am I missing something here in my design? My application relies on the xml format, so should I instead parse the packet using the p1_xml module and reconstruct the xml using the extracted data elements before storing it on Riak.
Apologies for the very basic and multiple questions but appreciate if someone can throw some light here!
The code that I use to try and replace the " with \" in the incoming packet is: (it doesnt quite work):
NewPacket = re:replace(Packet, "\"", "\\\"", [{return, list}, global]),
So essentially, I would be passing the NewPacket as a value to my Riak calls.
ejabberd is compliant with Riak and it is already storing packets in Riak. For example, mod_offline does that.
You can look directly in ejabberd code to find how to do that. For example, in mod_offline, here is how ejabberd store the offline message:
store_offline_msg(Host, {User, _}, Msgs, Len, MaxOfflineMsgs,
riak) ->
Count = if MaxOfflineMsgs =/= infinity ->
Len + count_offline_messages(User, Host);
true -> 0
end,
if
Count > MaxOfflineMsgs ->
discard_warn_sender(Msgs);
true ->
lists:foreach(
fun(#offline_msg{us = US,
timestamp = TS} = M) ->
ejabberd_riak:put(M, offline_msg_schema(),
[{i, TS}, {'2i', [{<<"us">>, US}]}])
end, Msgs)
end.
The code of ejabberd_riak:put/3 is:
put(Rec, RecSchema, IndexInfo) ->
Key = encode_key(proplists:get_value(i, IndexInfo, element(2, Rec))),
SecIdxs = [encode_index_key(K, V) ||
{K, V} <- proplists:get_value('2i', IndexInfo, [])],
Table = element(1, Rec),
Value = encode_record(Rec, RecSchema),
case put_raw(Table, Key, Value, SecIdxs) of
ok ->
ok;
{error, _} = Error ->
log_error(Error, put, [{record, Rec},
{index_info, IndexInfo}]),
Error
end.
put_raw(Table, Key, Value, Indexes) ->
Bucket = make_bucket(Table),
Obj = riakc_obj:new(Bucket, Key, Value, "application/x-erlang-term"),
Obj1 = if Indexes /= [] ->
MetaData = dict:store(<<"index">>, Indexes, dict:new()),
riakc_obj:update_metadata(Obj, MetaData);
true ->
Obj
end,
catch riakc_pb_socket:put(get_random_pid(), Obj1).
You should have already the proper API to do what you want in ejabberd regarding Riak packet storage.
Related
I am using Erlang to send message between an email server and Spamassassin.
What I want to achieve is retrieving the tests done by SA to generate a report (I am doing some kind of mail-tester program)
When SpamAssassin answers (through raw TCP) it sends a binary string like this one:
<<"SPAMD/1.1 0 EX_OK\r\nContent-length: 728\r\nSpam: True ; 6.3 / 5.0\r\n\r\nReceived: from localhost by debpub1.cs2cloud.internal\r\n\twith SpamAssassin (version 3.4.2);\r\n\tSat, 04 Jan 2020 18:24:37 +0100\r\nFrom: bibi <bibi#XXXXX.local>\r\nTo: <aZphki8N05#XXXXXXXX>\r\nSubject: i\r\nDate: Sat, 4 Jan 2020 18:24:36 +0100\r\nMessage-Id: <3b68dede-f1c3-4f04-62dc-f0b2de6e980a#PPPPPP.local>\r\nX-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on\r\n\tdebpub1.cs2cloud.internal\r\nX-Spam-Flag: YES\r\nX-Spam-Level: ******\r\nX-Spam-Status: Yes, score=6.3 required=5.0 tests=BODY_SINGLE_WORD,\r\n\tDKIM_ADSP_NXDOMAIN,DOS_RCVD_IP_TWICE_C,HELO_MISC_IP,\r\n\tNO_FM_NAME_IP_HOSTN autolearn=no autolearn_force=no version=3.4.2\r\nMIME-Version: 1.0\r\nContent-Type: multipart/mixed; boundary=\"----------=_5E10CA56.0200B819\"\r\n\r\n">>
I put in bold the items I want to pick up:
BODY_SINGLE_WORD
DKIM_ADSP_NXDOMAIN
DOS_RCVD_IP_TWICE_C
HELO_MISC_IP
NO_FM_NAME_IP_HOSTN
I then want to serialize like that:
[<<"DKIM_ADSP_NXDOMAIN">>,<<"DOS_RCVD_IP_TWICE_C">>,…]
But that's not easy, terms have no regular "delimitors", has \r\n or \r\n\t
I give a start with that expression (splitting on ',' on a binary string) but result is incomplete
split(BinaryString, ",", all),
case lists:member(<<"HELO_MISC_IP">>, Data3 ) of
true -> ; %push the result in a database
false -> ok
end;
I wish I could take another start, and using looping through recursion (and becausee it is a clean and nice way to loop) but it looks pointless to me regarding that scenario …
split(BinaryString, Idx, Acc) ->
case BinaryString of
<<"tests=",_This:Idx/binary, Char, Tail/binary>> ->
case lists:member(Char, BinaryString ) of
false ->
split(BinaryString, Idx+1, Acc);
true ->
case Tail of
<<Y/binary, _Tail/binary>> ->
%doing something
<<_Yop2/binary>> ->
%doing somethin else
end
end;
The thing is I don't see how achieve this in a acceptable and clean way
If anyone could give me a hand that would be very very appreciable.
Yours
One solution is to match the parts of the binary you're looking for:
Data = <<"SPAMD/1.1 0 EX_OK\r\nContent-length: 728\r\nSpam: True ; 6.3 / 5.0\r\n\r\nReceived: from localhost by debpub1.cs2cloud.internal\r\n\twith SpamAssassin (version 3.4.2);\r\n\tSat, 04 Jan 2020 18:24:37 +0100\r\nFrom: bibi <bibi#XXXXX.local>\r\nTo: <aZphki8N05#XXXXXXXX>\r\nSubject: i\r\nDate: Sat, 4 Jan 2020 18:24:36 +0100\r\nMessage-Id: <3b68dede-f1c3-4f04-62dc-f0b2de6e980a#PPPPPP.local>\r\nX-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on\r\n\tdebpub1.cs2cloud.internal\r\nX-Spam-Flag: YES\r\nX-Spam-Level: ******\r\nX-Spam-Status: Yes, score=6.3 required=5.0 tests=BODY_SINGLE_WORD,\r\n\tDKIM_ADSP_NXDOMAIN,DOS_RCVD_IP_TWICE_C,HELO_MISC_IP,\r\n\tNO_FM_NAME_IP_HOSTN autolearn=no autolearn_force=no version=3.4.2\r\nMIME-Version: 1.0\r\nContent-Type: multipart/mixed; boundary=\"----------=_5E10CA56.0200B819\"\r\n\r\n">>,
Matches = binary:compile_pattern([<<"BODY_SINGLE_WORD">>,<<"DKIM_ADSP_NXDOMAIN">>,<<"DOS_RCVD_IP_TWICE_C">>,<<"HELO_MISC_IP">>,<<"NO_FM_NAME_IP_HOSTN">>]),
[binary:part(Data, PosLen) || PosLen <- binary:matches(Data, Matches)].
Executing the three lines above in an Erlang shell returns:
[<<"BODY_SINGLE_WORD">>,<<"DKIM_ADSP_NXDOMAIN">>, <<"DOS_RCVD_IP_TWICE_C">>,<<"HELO_MISC_IP">>, <<"NO_FM_NAME_IP_HOSTN">>]
This provides the desired result, but it might not be safe since it doesn't do anything to try to verify whether the input is valid or whether the matches occur on valid boundaries.
A potentially safer approach relies on the fact that the input binary resembles an HTTP result, and so it can be partially parsed with built-in Erlang decoders. The parse/1,2 functions below use erlang:decode_packet/3 to extract information from the input:
parse(Data) ->
{ok, Line, Rest} = erlang:decode_packet(line, Data, []),
parse(Line, Rest).
parse(<<"SPAMD/", _/binary>>, Data) ->
parse(Data, []);
parse(<<>>, Hdrs) ->
Result = [{Key,Value} || {http_header, _, Key, _, Value} <- Hdrs],
process_results(Result);
parse(Data, Hdrs) ->
case erlang:decode_packet(httph, Data, []) of
{ok, http_eoh, Rest} ->
parse(Rest, Hdrs);
{ok, Hdr, Rest} ->
parse(Rest, [Hdr|Hdrs]);
Error ->
Error
end.
The parse/1 function initially decodes the first line of the input using the line decoder, passing the results to parse/2. The first clause of parse/2 matches the "SPAMD/" prefix of the initial line of the input data just to verify we're looking in the right place, then recursively invokes parse/2 passing the remaining Data and an empty accumulator list. The second and third clauses of parse/2 treat the data as HTTP headers. The second clause of parse/2 matches when the input data is exhausted; it maps the accumulated header list to a list of {Key,Value} pairs and passes it to a process_results/1 function, described below, to finish the data extraction. The third clause of parse/2 tries to decode the data via the httph HTTP header decoder, accumulating each matched header and ignoring any http_eoh end-of-headers markers that result from "\r\n" sequences embedded at odd places in the input.
For the input data provided in the question, the parse/1,2 functions ultimately pass the following list of key-value pairs to process_results/1:
[{'Content-Type',"multipart/mixed; boundary=\"----------=_5E10CA56.0200B819\""},{"Mime-Version","1.0"},{"X-Spam-Status","Yes, score=6.3 required=5.0 tests=BODY_SINGLE_WORD,\r\n\tDKIM_ADSP_NXDOMAIN,DOS_RCVD_IP_TWICE_C,HELO_MISC_IP,\r\n\tNO_FM_NAME_IP_HOSTN autolearn=no autolearn_force=no version=3.4.2"},{"X-Spam-Level","******"},{"X-Spam-Flag","YES"},{"X-Spam-Checker-Version","SpamAssassin 3.4.2 (2018-09-13) on\r\n\tdebpub1.cs2cloud.internal"},{"Message-Id","<3b68dede-f1c3-4f04-62dc-f0b2de6e980a#PPPPPP.local>"},{'Date',"Sat, 4 Jan 2020 18:24:36 +0100"},{"Subject","i"},{"To","<aZphki8N05#XXXXXXXX>"},{'From',"bibi <bibi#XXXXX.local>"},{"Received","from localhost by debpub1.cs2cloud.internal\r\n\twith SpamAssassin (version 3.4.2);\r\n\tSat, 04 Jan 2020 18:24:37 +0100"},{"Spam","True ; 6.3 / 5.0"},{'Content-Length',"728"}]
The process_results/1,2 functions first match the key of interest, which is "X-Spam-Status", and then extract the desired data from its value. The three functions below implement process_results/1 to look for that key and process it, or return {error, not_found} if no such key is seen. The second clause matches the desired key, splits its associated value on the space, comma, carriage return, newline, tab, and equal sign characters, and passes the split result along with an empty accumulator to process_results/2:
process_results([]) ->
{error, not_found};
process_results([{"X-Spam-Status", V}|_]) ->
process_results(string:lexemes(V, " ,\r\n\t="), []);
process_results([_|T]) ->
process_results(T).
For the input data in the question, the list of strings passed to process_results/2 is
["Yes","score","6.3","required","5.0","tests","BODY_SINGLE_WORD","\r\n","DKIM_ADSP_NXDOMAIN","DOS_RCVD_IP_TWICE_C","HELO_MISC_IP","\r\n","NO_FM_NAME_IP_HOSTN","autolearn","no","autolearn_force","no","version","3.4.2"]
The clauses of process_results/2 below recursively walk this list of strings and accumulate the matched results. Each of the second through sixth clauses matches one of the values we seek, and each converts the matched string to a binary before accumulating it.
process_results([], Results) ->
{ok, lists:reverse(Results)};
process_results([V="BODY_SINGLE_WORD"|T], Results) ->
process_results(T, [list_to_binary(V)|Results]);
process_results([V="DKIM_ADSP_NXDOMAIN"|T], Results) ->
process_results(T, [list_to_binary(V)|Results]);
process_results([V="DOS_RCVD_IP_TWICE_C"|T], Results) ->
process_results(T, [list_to_binary(V)|Results]);
process_results([V="HELO_MISC_IP"|T], Results) ->
process_results(T, [list_to_binary(V)|Results]);
process_results([V="NO_FM_NAME_IP_HOSTN"|T], Results) ->
process_results(T, [list_to_binary(V)|Results]);
process_results([_|T], Results) ->
process_results(T, Results).
The final clause ignores unwanted data. The first clause of process_results/2 is invoked when the list of strings is empty, and it just returns the reversed accumulator. For the input data in the question, the final result of process_results/2 is:
{ok, [<<"BODY_SINGLE_WORD">>,<<"DKIM_ADSP_NXDOMAIN">>,<<"DOS_RCVD_IP_TWICE_C">>,<<"HELO_MISC_IP">>,<<"NO_FM_NAME_IP_HOSTN">>]}
Hi I'm a newbie in Erlang and I just started learning about processes. Here I have a typical process loop:
loop(X,Y,Z) ->
receive
{do} ->
NewX = X+1,
NewY = Y+1,
NewZ = Z+1,
Product = NewX * NewY * NewZ,
% do something
loop(NewX,NewY,NewZ)
end.
How do I get the latest value of Product from a function let's say get_product()? I know that message passing will be the logical option but is there a more optimal way of extracting the value?
Here are methods to communicate between Erlang processes I am aware of, and my (possibly wrong) assessment of theirs relative performance.
Message passing. This method will suit most of your needs. I don't know how it is actually implemented, but from my point of view it should be as fast as putting a pointer into a queue and retrieving it back.
Exterior methods, e.g. sockets, files, pipes. These methods might be faster for communicating between different nodes, depending on a problem you solve, your solution and environment your program will be executed in. Inter-node communication in Erlang is done via TCP connections, so if you want to use self written code to communicate via TCP sockets, you should try really hard to outperform Erlang's implementation.
ETS, Dets. These methods won't be faster than message passing (ETS) or file (Dets) assuming best possible implementation.
NIF. You can write one method to save value in your NIF library and one to retrieve it. This one has a potential to outperform message passing since you can just save a value into a variable and return it back when needed and it has no overhead on pattern matching in receive.
Process dictionary. You can get another process dictionary using erlang:process_info(Pid, dictionary) call, in the Pid process you can put value in that dictionary using put(Key, Value) call.
Also, if you want to speed up your Erlang application take a look at HiPE, it might help.
Before switching from message passing to anything from this list to gain in speed you should measure it first!
I assumed this is what you want:
-module(lab).
-compile(export_all).
start() ->
InitialState = {1,1,1},
Pid = spawn(?MODULE, loop, [InitialState]),
register(server, Pid).
loop(State) ->
{X, Y, Z} = State,
receive
tick ->
NewX = X+1,
NewY = Y+1,
NewZ = Z+1,
NewState = {NewX, NewY, NewZ},
loop(NewState);
{get_product, From} ->
Product = X * Y * Z,
From ! Product,
loop(State);
_ ->
io:format("Unknown message received.~n"),
loop(State)
end.
get_product() ->
server ! {get_product, self()},
receive
Product ->
Product
end.
tick() ->
server ! tick.
From within the Erlang shell:
1> c(lab).
{ok,lab}
2> lab:start().
true
3> lab:get_product().
1
4> lab:tick().
tick
5> lab:get_product().
8
6> lab:tick().
tick
7> lab:tick().
tick
8> lab:get_product().
64
So I need to write this function that receives a number from three different processes and saves them. I can't get it to work, I get the "variable unbound" error.
serverB2(Time, Client1, Client2, Client3) ->
receive
{From, TimeClient} ->
if
From == Client1 ->
TimeClient1 = TimeClient;
From == Client2 ->
TimeClient2 = TimeClient;
From == Client3 ->
TimeClient3 = TimeClient;
end,
serverB2(Time, Client1, Client2, Client3)
end,
List = [Time, TimeClient1, TimeClient2, TimeClient3],
io:format("~w \n", List).
You are getting variable unbound error because your code has three paths and in each path you will bound only one of variables TimeClient1, TimeClient2 and TimeClient3 so you have always two of them unbound. What's worse your code never stop. You will receive message, then evaluate if statement and then recursive call serverB2 code and again and again and again. There is not any code path to your statements after receive.
I would write this in this way
serverB2(Time, Client1, Client2, Client3) ->
List = [ Time
| [ receive {Client, TimeClient} -> TimeClient end
|| Client <- [Client1, Client2, Client3]]],
io:format("~w~n", [List]).
or in more conventional way
serverB2(Time, Client1, Client2, Client3) ->
List = [ Time | collect_times([Client1, Client2, Client3])],
io:format("~w~n", [List]).
collect_times([]) -> [];
collect_times([Client|Tail]) ->
TimeClient = receive {Client, Time} -> Time end,
[ TimeClient | collect_times(Tail) ].
Which is how list comprehension above would be translated by compiler.
The trick what I'm using here is that I receive messages in order what I want instead of order in which they are arrived to mailbox. I'm using selective receive there because variable Client is bound in receive pattern in both examples above. There is more complicated way how receive messages in order how they arrive which you should use only with very good reason (performance for example).
My scenario is as follows -
I have a client C with function foo() which performs some computation.
I'd like a server S, which doesn't know about foo(), to execute this function instead, and send the result back to the client.
I am trying to determine the best way to perform this in Erlang. I am considering:
Hot code swapping - i.e. "upgrade" code in S such that it has the function foo(). Execute and send back to the client.
In a distributed manner where nodes are all appropriately registered, do something along the lines of S ! C:foo() - for the purpose of "sending" the function to process/node S
Are there other methods (or features of the language) that I am not thinking of?
Thanks for the help!
If the computation function is self contained i.e. does not depend on any other modules or functions on the client C, then what you need to do is a fun (Functional Objects). A fun can be sent across the network and applied by a remote machine and in side the fun, the sender has embedded their address and a way of getting the answer back. So the executor may only see a fun to which they may or may not give an argument, yet inside the fun, the sender has forced a method where by the answer will automatically be sent back. The fun is an abstraction of very many tasks within one thing, and it can be moved around as arguments.
At the client, you can have code like this:
%% somewhere in the client
%% client runs on node() == 'client#domain.com'
-module(client).
-compile(export_all).
-define(SERVER,{server,'server#domain.com'}).
give_a_server_a_job(Number)-> ?SERVER ! {build_fun(),Number}.
build_fun()->
FunObject = fun(Param)->
Answer = Param * 20/1000, %% computation here
rpc:call('client#domain.com',client,answer_ready,[Answer])
end,
FunObject.
answer_ready(Answer)->
%%% use Answer for all sorts of funny things....
io:format("\n\tAnswer is here: ~p~n",[Answer]).
The server then has code like this:
%%% somewhere on the server
%%% server runs on node() == 'server#domain.com'
-module(server).
-compile(export_all).
start()-> register(server,spawn(?MODULE,loop,[])).
loop()->
receive
{Fun,Arg} ->
Fun(Arg), %% server executes job
%% job automatically sends answer back
%% to client
loop();
stop -> exit(normal);
_ -> loop()
end.
In this way, the job executor needs not know about how to send back the reply, The job itself comes knowing how it will send back the answer to however sent the job!. I have used this method of sending functional objects across the network in several project, its so cool !!!
#### EDIT #####
If you have a recursive problem, You manipulate recursion using funs. However, you will need at least one library function at the client and/or the server to assist in recursive manipulations. Create a function which should be in the code path of the client as well as the server.
Another option is to dynamically send code from the server to the client and then using the library: Dynamic Compile erlang to load and execute erlang code at the server from the client. Using dynamic compile, here is an example:
1> String = "-module(add).\n -export([add/2]). \n add(A,B) -> A + B. \n".
"-module(add).\n -export([add/2]). \n add(A,B) -> A + B. \n"
2> dynamic_compile:load_from_string(String).
{module,add}
3> add:add(2,5).
7
4>
What we see above is a piece of module code that is compiled and loaded dynamically from a string. If the library enabling this is available at the server and client , then each entity can send code as a string and its loaded and executed dynamically at the other. This code can be unloaded after use. Lets look at the Fibonacci function and how it can be sent and executed at the server:
%% This is the normal Fibonacci code which we are to convert into a string:
-module(fib).
-export([fib/1]).
fib(N) when N == 0 -> 0;
fib(N) when (N < 3) and (N > 0) -> 1;
fib(N) when N > 0 -> fib(N-1) + fib(N-2).
%% In String format, this would now become this piece of code
StringCode = " -module(fib).\n -export([fib/1]). \nfib(N) when N == 0 -> 0;\n fib(N) when (N < 3) and (N > 0) -> 1;\n fib(N) when N > 0 -> fib(N-1) + fib(N-2). \n".
%% Then the client would send this string above to the server and the server would %% dynamically load the code and execute it
send_fib_code(Arg)->
{ServerRegName,ServerNode} ! {string,StringCode,fib,Arg},
ok.
get_answer({fib,of,This,is,That}) ->
io:format("Fibonacci (from server) of ~p is: ~p~n",[This,That]).
%%% At Server
loop(ServerState)->
receive
{string,StringCode,Fib,Arg} when Fib == fib ->
try dynamic_compile:load_from_string(StringCode) of
{module,AnyMod} ->
Answer = AnyMod:fib(Arg),
%%% send answer back to client
%%% should be asynchronously
%%% as the channels are different & not make
%% client wait
rpc:call('client#domain.com',client,get_answer,[{fib,of,Arg,is,Answer}])
catch
_:_ -> error_logger:error_report(["Failed to Dynamic Compile & Load Module from client"])
end,
loop(ServerState);
_ -> loop(ServerState)
end.
That piece of rough code can show you what am trying to say. However, you remember to unload all un-usable dynamic modules. Also you can a have a way in which the server tries to check wether such a module was loaded already before loading it again. I advise that you donot copy and paste the above code. Look at it and understand it and then write your own version that can do the job. success !!!
If you do S ! C:foo() it will compute on client side function foo/1 from module C and send its result to process S. It doesn't seem like what you want to do. You should do something like:
% In client
call(S, M, F, A) ->
S ! {do, {M, F, A}, self()},
receive
{ok, V} -> V
end.
% In server
loop() ->
receive
{do, {M, F, A}, C} ->
C ! {ok, apply(M, F, A)},
loop()
end.
But in real scenario you would have to do a lot more work e.g. mark your client message to perform selective receive (make_ref/0), catch error in server and send it back to client, monitor server from client to catch server down, add some timeout and so. Look how are gen_server:call/2 and rpc:call/4,5 implemented and it is reason why there is OTP to save you from most of gotcha.
am josh in Uganda. i created a mnesia fragmented table (64 fragments), and managed to populate it upto 9948723 records. Each fragment was a disc_copies type, with two replicas.
Now, using qlc (query list comprehension), was too slow in searching for a record, and was returning inaccurate results.
I found out that this overhead is that qlc uses the select function of mnesia which traverses the entire table in order to match records. i tried something else below.
-define(ACCESS_MOD,mnesia_frag).
-define(DEFAULT_CONTEXT,transaction).
-define(NULL,'_').
-record(address,{tel,zip_code,email}).
-record(person,{name,sex,age,address = #address{}}).
match()-> Z = fun(Spec) -> mnesia:match_object(Spec) end,Z.
match_object(Pattern)->
Match = match(),
mnesia:activity(?DEFAULT_CONTEXT,Match,[Pattern],?ACCESS_MOD).
Trying this functionality gave me good results. But i found that i have to dynamically build patterns for every search that may be made in my stored procedures.
i decided to go through the havoc of doing this, so i wrote functions which will dynamically build wild patterns for my records depending on which parameter is to be searched.
%% This below gives me the default pattern for all searches ::= {person,'_','_','_'}
pattern(Record_name)->
N = length(my_record_info(Record_name)) + 1,
erlang:setelement(1,erlang:make_tuple(N,?NULL),Record_name).
%% this finds the position of the provided value and places it in that
%% position while keeping '_' in the other positions.
%% The caller function can use this function recursively until
%% it has built the full search pattern of interest
pattern({Field,Value},Pattern_sofar)->
N = position(Field,my_record_info(element(1,Pattern_sofar))),
case N of
-1 -> Pattern_sofar;
Int when Int >= 1 -> erlang:setelement(N + 1,Pattern_sofar,Value);
_ -> Pattern_sofar
end.
my_record_info(Record_name)->
case Record_name of
staff_dynamic -> record_info(fields,staff_dynamic);
person -> record_info(fields,person);
_ -> []
end.
%% These below,help locate the position of an element in a list
%% returned by "-record_info(fields,person)"
position(_,[]) -> -1;
position(Value,List)->
find(lists:member(Value,List),Value,List,1).
find(false,_,_,_) -> -1;
find(true,V,[V|_],N)-> N;
find(true,V,[_|X],N)->
find(V,X,N + 1).
find(V,[V|_],N)-> N;
find(V,[_|X],N) -> find(V,X,N + 1).
This was working very well though it was computationally intensive.
It could still work even after changing the record definition since at compile time, it gets the new record info
The problem is that when i initiate even 25 processes on a 3.0 GHz pentium 4 processor running WinXP, It hangs and takes a long time to return results.
If am to use qlc in these fragments, to get accurate results, i have to specify which fragment to search in like this.
find_person_by_tel(Tel)->
select(qlc:q([ X || X <- mnesia:table(Frag), (X#person.address)#address.tel == Tel])).
select(Q)->
case ?transact(fun() -> qlc:e(Q) end) of
{atomic,Val} -> Val;
{aborted,_} = Error -> report_mnesia_event(Error)
end.
Qlc was returning [], when i search for something yet when i use match_object/1 i get accurate results. I found that using match_expressions can help.
mnesia:table(Tab,Props).
where Props is a data structure that defines the match expression, the chunk size of return values e.t.c
I got a problem when i tried building match expressions dynamically.
Function mnesia:read/1 or mnesia:read/2 requires that you have the primary key
Now am asking myself, how can i efficiently use QLC to search for records in a large fragmented table? Please help.
I know that using tuple representation of records makes code hard to upgrade. This is why
i hate using mnesia:select/1, mnesia:match_object/1 and i want to stick to QLC. QLC is giving me wrong results in my queries from a mnesia table of 64 fragments even on the same node.
Has anyone ever used QLC to query a fragmented table?, please help
Do you invoke the qlc in the activity context?
tfn_match(Id) ->
Search = #person{address=#address{tel=Id, _ = '_'}, _ = '_'},
trans(fun() -> mnesia:match_object(Search) end).
tfn_qlc(Id) ->
Q = qlc:q([ X || X <- mnesia:table(person), (X#person.address)#address.tel == Id]),
trans(fun() -> qlc:e(Q) end).
trans(Fun) ->
try Res = mnesia:activity(transaction, Fun, mnesia_frag),
{atomic, Res}
catch exit:Error ->
{aborted, Error}
end.