how to use mnesia:select/4 and mnesia:select/1 for paging query - mnesia

a table named "md" with structure {id,name},I want read records from md use paging query,I tried mnesia:select/4 and mnesia:select/1 as below:
%% first use select/2: "ID < 10",returned [1,2,4,3,8,5,9,7,6]
(ejabberd#localhost)5> mnesia:activity(transaction,fun mnesia:select/2,md, [{{md,'$1','_'},[{'<','$1',10}],['$1']}]).
{atomic,[1,2,4,3,8,5,9,7,6]}
%%but when query with select/4,returned [6], why?
(ejabberd#localhost)7> {atomic,{R1,C1}}=mnesia:activity(transaction,fun mnesia:select/4,md,[{{md,'$1','_'},[{'<','$1',10}],['$1']}],5,read).
{atomic,{[6],
{mnesia_select,md,
{tid,10535470,<0.460.0>},
ejabberd#localhost,disc_only_copies,
{dets_cont,select,5,
<<0,0,0,29,18,52,86,120,0,0,0,21,131,104,3,...>>,
{141720,148792,<<>>},
md,<0.130.0>,<<>>},
[],undefined,undefined,
[{{md,'$1','_'},[{'<','$1',10}],['$1']}]}}}
%% and then use mnesia:select/1 with continuation "C1",got wrong_transaction
(ejabberd#localhost)8> mnesia:activity(transaction,fun mnesia:select/1,C1).
{aborted,wrong_transaction}
how to use mnesia:select/4 and mnesia:select/1 for paging query?

You will have to call select/1 inside the same transaction.
Otherwise the table can change between invocations to select/4 and select/1.
You must use a dirty context if you want to use is as written above.

here is my solution:
use async_dirty instead of transaction
{Record,Cont}=mnesia:activity(async_dirty, fun mnesia:select/4,[md,[{Match_head,[Guard],[Result]}],Limit,read])
then read next Limit number of records:
mnesia:activity(async_dirty, fun mnesia:select/1,[Cont])
full code:
-record(md,{id,name}).
batch_delete(Id,Limit) ->
Match_head = #md{id='$1',name='$2'},
Guard = {'<','$1',Id},
Result = '$_',
{Record,Cont} = mnesia:activity(async_dirty, fun mnesia:select/4,[md,[{Match_head,[Guard],[Result]}],Limit,read]),
delete_next({Record,Cont}).
delete_next('$end_of_table') ->
over;
delete_next({Record,Cont}) ->
delete(Record),
delete_next(mnesia:activity(async_dirty, fun mnesia:select/1,[Cont])).
delete(Records) ->
io:format("delete(~p)~n",[Records]),
F = fun() ->
[ mnesia:delete_object(O) || O <- Records]
end,
mnesia:transaction(F).

Related

Write F# DataTable to CSV file

I am trying to write an F# DataTable to csv (or output in a txt). The table I have is defined as follows:
let setup_report_tbl ( tbl : DataTable ) =
ignore( tbl.Columns.Add("business_date", typeof<System.Int32>) )
ignore( tbl.Columns.Add("ticker", typeof<System.String>) )
ignore( tbl.Columns.Add("price", typeof<System.String>) )
ignore( tbl.Columns.Add("rate", typeof<System.Boolean>) )
ignore( tbl.Columns.Add("range", typeof<System.Double>) )
My goal is to write this empty table with headers into a csv or txt. I'm new to F# and not quite sure where to start here, any help is appreciated thanks!
To write a DataTable as CSV, I would do something like this:
open System
open System.IO
open System.Data
let writeCsv (wtr : StreamWriter) (tbl : DataTable) =
let writeValues (values : seq<_>) =
String.Join(',', values)
|> wtr.WriteLine
tbl.Columns
|> Seq.cast<DataColumn>
|> writeValues
for row in tbl.Rows do
row.ItemArray |> writeValues
Note that I haven't done anything to check for special characters in the values, such commas or quotes.
Example:
let tbl = new DataTable()
setup_report_tbl tbl
tbl.Rows.Add(1, "moo", "baa", true, 2.0) |> ignore
use wtr = new StreamWriter(Console.OpenStandardOutput())
writeCsv wtr tbl
Output is:
business_date,ticker,price,rate,range
1,moo,baa,True,2
Update
To avoid compiler error, perhaps try this:
let writeValues (values : seq<_>) =
let s = String.Join(',', values)
wtr.WriteLine(s)
Note that s is a string, so there should be no ambiguity in which version of WriteLine is called.
If you wanted to use an existing library rather than writing your own CSV encoding (which may get tricky when you need to escape things), you could use Deedle which has an easy way to create data frame from a DataTable and save it to a CSV file:
#r "nuget: Deedle"
open Deedle
open System.Data
// Setup table using your function and add some data
let tbl = new DataTable()
setup_report_tbl tbl
tbl.Rows.Add(1, "very\",evil'ticker", "$42", false, 1.23)
// Turn it into a dataframe and save it
let df = Frame.ReadReader(tbl.CreateDataReader())
df.SaveCsv("C:/temp/test.csv")
As a bonus point, you could see if the data frame type from Deedle lets you do some of the other things you want to do with the data - but this depends on your scenario.

Why cyrsasl_scram mechanism is not allowing base64 GUID?

I am writing a chat communication app.
If the user's unique id is given in Base 64 GUID format, it is throwing bad_username error.
In this file: https://pow.gs/mirror/ejabberd/-/blob/fd8e07af4789be362a61755ea47f216baeb64989/src/cyrsasl_scram.erl, there is a method to remove the == from username:
unescape_username(<<"">>) -> <<"">>;
unescape_username(EscapedUsername) ->
Pos = str:str(EscapedUsername, <<"=">>),
if Pos == 0 -> EscapedUsername;
true ->
Start = str:substr(EscapedUsername, 1, Pos - 1),
End = str:substr(EscapedUsername, Pos),
EndLen = byte_size(End),
if EndLen < 3 -> error;
true ->
case str:substr(End, 1, 3) of
<<"=2C">> ->
<<Start/binary, ",",
(unescape_username(str:substr(End, 4)))/binary>>;
<<"=3D">> ->
<<Start/binary, "=",
(unescape_username(str:substr(End, 4)))/binary>>;
_Else -> error
end
end
end.
I don't know why this has been written. If I remove this particular code, the connection is working fine. Please let me know why it is restricted.
If the users' unique id is given in Base 64 GUID format, it is throwing bad_username error.
Right:
call xmpp_sasl_scram:unescape_username(<<"user1">>)
returned from xmpp_sasl_scram:unescape_username/1 -> <<"user1">>
call xmpp_sasl_scram:unescape_username(<<"user3==ABC">>)
returned from xmpp_sasl_scram:unescape_username/1 -> error
call xmpp_sasl_scram:unescape_username(<<"user4=DEF">>)
returned from xmpp_sasl_scram:unescape_username/1 -> error
call xmpp_sasl_scram:unescape_username(<<"user5=">>)
returned from xmpp_sasl_scram:unescape_username/1 -> error
I don't know why this has been written. If I remove this particular code, the connection is working fine. Please let me know why it is restricted.
I don't know, either. But that code exists since nine years ago:
https://github.com/processone/ejabberd/commit/e80b92b48148505b44c6a378db36badfe60fce79#diff-5c51943c1268ffe26fe3b041b20675c6R136
Whatever reason there is for it, it's obviously a good reason.

Erlang:creating list of tuples within lists:foreach

I query the list of users from Mnesia Database in Chicagoboss. I'm getting the error when I try to add the Lists within lists:foreach with ++ operator. My aim is, based on userid I will do ets:lookup to my cache and create a List like - [{{<<"name">>,<<"Batman">>}, {<<"steps">>,2552}, {<<"distance">>,2050}}].
For each user I'll create this list and add with the previous List. So that ultimately I can can sort on <<"steps">> and convert the binary list by json encoding and send it to the client via Websockets.
I'm getting the error at this line:
Reading1 = Reading2 ++ Currentlist
as I've decalred Reading1 as an Empty list.
My question is how can I manipulate the lists within the lists:foreach and then send the result List via websocket?
BelugaUsers = boss_db:find(users, [{accesstoken, 'not_equals', ''}]),
Reading1 = [],
Reading2 = [],
lists:foreach(fun(X) ->
{_,_,BEmail,BName,_,_,BAccessToken,_} = X,
UserKey = BEmail ++ "-" ++ ?MYAPICALL1,
io:format("UserKey for Leader Board: ~n~p~n",[UserKey]),
[Reading] = ets:lookup(myapi_cache, list_to_binary(UserKey)),
{_,Result} = Reading,
ActivitySummary = proplists:get_value(<<"activitySummary">>, Result),
%Print ActivitySummary for the user ....printing fine
io:format("ActivitySummary ==========: ~n~p~n",[ActivitySummary]),
%Create a list of the format
%[{{<<"name">>,<<"Batman">>}, {<<"steps">>,2552}, {<<"distance">>,2050}}]
Currentlist = [{{<<"name">>, list_to_binary(BName)}, {<<"steps">>, proplists:get_value(<<"steps">>, ActivitySummary)}, {<<"distance">>, proplists:get_value(<<"distance">>, ActivitySummary)}}],
%% HERE I'M GETTING error%%
Reading1 = Reading2 ++ Currentlist
end, BelugaUsers),
%sort the list
Reading3 = lists:keysort(2, Reading1),
%reverse the list
Reading4 = lists:reverse(Reading3),
WebSocketId ! {text, jsx:encode(Reading4)},
Erlang variables are single-assignment; once bound to a value, they can't be re-bound to a different value.
The lists:foreach/2 function is not useful for this problem because it can't create a new value and return it to its caller. You should instead use lists:map/2, perhaps like this:
BelugaUsers = boss_db:find(users, [{accesstoken, 'not_equals', ''}]),
Reading = lists:map(
fun(X) ->
{_,_,BEmail,BName,_,_,BAccessToken,_} = X,
UserKey = BEmail ++ "-" ++ ?MYAPICALL1,
io:format("UserKey for Leader Board: ~n~p~n",[UserKey]),
{_,Result} = hd(ets:lookup(myapi_cache, list_to_binary(UserKey))),
ActivitySummary = proplists:get_value(<<"activitySummary">>, Result),
%%Print ActivitySummary for the user ....printing fine
io:format("ActivitySummary ==========: ~n~p~n",[ActivitySummary]),
%%Create a tuple of the format
%%{{<<"name">>,<<"Batman">>}, {<<"steps">>,2552}, {<<"distance">>,2050}}
{{<<"name">>, list_to_binary(BName)},
{<<"steps">>, proplists:get_value(<<"steps">>, ActivitySummary)},
{<<"distance">>, proplists:get_value(<<"distance">>, ActivitySummary)}}
end, BelugaUsers),
%%sort the list
Reading2 = lists:keysort(2, Reading),
%%reverse the list
Reading3 = lists:reverse(Reading2),
WebSocketId ! {text, jsx:encode(Reading3)}.
The lists:map/2 function applies a function to each value in a list to a produce a potentially different value and returns a new list consisting of those new values. This is essentially what you were trying to do with lists:foreach/2 and your attempt to use imperative assignment to add each element to an already-existing list.
You could alternatively use a list comprehension but I think lists:map/2 is clearer in this situation.

Rewrite variable in Erlang

I am playing with records and list. Please, I want to know how to use one variable twice. When I assign any values into variable _list and after that I try rewrite this variable then raising error:
** exception error: no match of right hand side value
-module(hello).
-author("anx00040").
-record(car, {evc, type, color}).
-record(person, {name, phone, addresa, rc}).
-record(driver, {rc, evc}).
-record(list, {cars = [], persons = [], drivers = []} ).
%% API
-export([helloIF/1, helloCase/1, helloResult/1, helloList/0, map/2, filter/2, helloListCaA/0, createCar/3, createPerson/4, createDriver/2, helloRecords/0, empty_list/0, any_data/0, del_Person/1, get_persons/1, do_it_hard/0, add_person/2]).
createCar(P_evc, P_type, P_color) -> _car = #car{evc = P_evc, type = P_type, color = P_color}, _car
.
createPerson(P_name, P_phone, P_addres, P_rc) -> _person= #person{name = P_name, phone = P_phone, addresa = P_addres, rc = P_rc}, _person
.
createDriver(P_evc, P_rc) -> _driver = #driver{rc = P_rc, evc = P_evc}, _driver
.
empty_list() ->
#list{}.
any_data() ->
_car1 = hello:createCar("BL 4", "Skoda octavia", "White"),
_person1 = hello:createPerson("Eduard B.","+421 917 111 711","Kr, 81107 Bratislava1", "8811235"),
_driver1 = hello:createDriver(_car1#car.evc, _person1#person.rc),
_car2 = hello:createCar("BL 111 HK", "BMW M1", "Red"),
_person2 = hello:createPerson("Lenka M","+421 917 111 111","Krizn0, 81107 Bratislava1", "8811167695"),
_driver2 = hello:createDriver(_car2#car.evc, _person2#person.rc),
_car3 = hello:createCar("BL 123 AB", "Audi A1 S", "Black"),
_person3 = hello:createPerson("Stela Ba.","+421 918 111 711","Azna 20, 81107 Bratislava1", "8811167695"),
_driver3 = hello:createDriver(_car3#car.evc, _person3#person.rc),
_list = #list{
cars = [_car1,_car2,_car3],
persons = [_person1, _person2, _person3],
drivers = [_driver1, _driver2, _driver3]},
_list.
add_person(List, Person) ->
List#list{persons = lists:append([Person], List#list.persons) }.
get_persons(#list{persons = P}) -> P.
do_it_hard()->
empty_list(),
_list = add_person(any_data(), #person{name = "Test",phone = "+421Test", addresa = "Testova 20 81101 Testovo", rc =88113545}),
io:fwrite("\n"),
get_persons(add_person(_list, #person{name = "Test2",phone = "+421Test2", addresa = "Testova 20 81101 Testovo2", rc =991135455}))
.
But it raising error when i use variable _list twice:
do_it_hard()->
empty_list(),
_list = add_person(any_data(), #person{name = "Test",phone = "+421Test", addresa = "Testova 20 81101 Testovo", rc =88113545}),
_list =add_person(_list, #person{name = "Test2",phone = "+421Test2", addresa = "Testova 20 81101 Testovo2", rc =991135455}),
get_persons(_list)
.
In the REPL, it can be convenient to experiment with things while re-using variable names. There, you can do f(A). to have Erlang "forget" the current assignment of A.
1> Result = connect("goooogle.com").
{error, "server not found"}
2> % oops! I misspelled the server name
2> f(Result).
ok
3> Result = connect("google.com").
{ok, <<"contents of the page">>}
Note that this is only a REPL convenience feature. You can't do this in actual code.
In actual code, variables can only be assigned once. In a procedural language (C, Java, Python, etc), the typical use-case for reassignment is loops:
for (int i = 0; i < max; i++) {
conn = connect(servers[i]);
reply = send_data(conn);
print(reply);
}
In the above, the variables i, conn, and reply are reassigned in each iteration of the loop.
Functional languages use recursion to perform their loops:
send_all(Max, Servers) ->
send_loop(1, Max, Servers).
send_loop(Current, Max, _Servers) when Current =:= Max->
ok;
send_loop(Current, Max, Servers) ->
Conn = connect(lists:nth(Current, Servers)),
Reply = send_data(Conn),
print(Reply).
This isn't very idiomatic Erlang; I'm trying to make it mirror the procedural code above.
As you can see, I'm getting the same effect, but my assignments within a function are fixed.
As a side note, you are using a lot of variable names beginning with underscore. In Erlang this is a way of hinting that you will not be using the value of these variables. (Like in the above example, when I've reached the end of my list, I don't care about the list of servers.) Using a leading underscore as in your code turns off some useful compiler warnings and will confuse any other developers who look at your code.
In some situations it is convenient to use use SeqBind:
SeqBind is a parse transformation that auto-numbers all occurrences of these bindings following the suffix # (creating L#0, L#1, Req#0, Req#1) and so on.
Simple example:
...
-compile({parse_transform,seqbind}).
...
List# = lists:seq(0, 100),
List# = lists:filter(fun (X) -> X rem 2 == 0 end, List#)
...
I used google...
Erlang is a single-assignment language. That is, once a variable has been given a value, it cannot be given a different value. In this sense it is like algebra rather than like most conventional programming languages.
http://www.cis.upenn.edu/~matuszek/General/ConciseGuides/concise-erlang.html

Custom concatenation of inner list Erlang

I have a big list of some forms with data that needs to be concated with other data from others forms with the same name.
The list format is quite complex and looks like this:
[[{EVAL_SEQ_1, {FORMNAME, ListDataToConcat1}}], [{EVAL_SEQ_2, {FORMNAME, ListDataToConcat2}}], ...]
This is the output that I want:
[[{EVAL_SEQ_1, {FORMNAME, ListDataToConcat1 + ListDataToConcat2}}]}}] ...]
Where:
EVAL_SEQ_1 = Form Sequence Number,
FORMNAME = Form Name
ListDataToConcat = List that Needs to concat
eg.Here is my sample data:
[[{"eval_data_12",
{<<"prvl_mobable_asset_0000_h200401">>,
[{'F_01_0100',[1]},
{'F_01_0090',["3"]},
{'F_01_0080',[]},
{'F_01_0070',[9999]},
{'F_01_0060',[{era,0},{year,[]},{month,[]}]},
{'F_01_0050',[]},
{'F_01_0040',[]},
{'F_01_0030',[]},
{'F_01_0020',<<>>},
{'F_01_0010',<<"4 - 8">>}]}}],
[{"eval_data_11",
{<<"prvl_mobable_asset_0000_h200401">>,
[{'F_01_0100',[]},
{'F_01_0090',["2"]},
{'F_01_0080',[]},
{'F_01_0070',[22222]},
{'F_01_0060',[{era,0},{year,[]},{month,[]}]},
{'F_01_0050',[]},
{'F_01_0040',[]},
{'F_01_0030',[]},
{'F_01_0020',<<>>},
{'F_01_0010',<<"4 - 1">>}]}}], ...]
I want the resultant output like this:
[{"eval_data_11",
{<<"prvl_mobable_asset_0000_h200401">>,
[{'F_01_0100',[[], [1]]},
{'F_01_0090',[["2"], ["3"]]},
{'F_01_0080',[[], []]},
{'F_01_0070',[[22222], [9999]]},
{'F_01_0060',[[{era,0},{year,[]},{month,[]}], [{era,0},{year,[]},{month,[]}]]},
{'F_01_0050',[[], []]},
{'F_01_0040',[[], []]},
{'F_01_0030',[[], []]},
{'F_01_0020',[[<<>>], [<<>>]]},
{'F_01_0010',[[<<"4 - 1">>], [<<"4 - 8">>]}]}}]
I propose you this solution:
[edit]
I modified the code to answer your last comment, there are still fuzzy things:
if Assets are different, creates different record list?
if not what should be done with Asset name? I have chosen to keep the "smallest one"
is the order of records important - I decided no
One remark, I am missing some context, but If I add to collect such informations, I would store it in an ets table. It is faster to update, easy to traverse, and easy to transform into list if needed.
-module (t).
-compile([export_all]).
% rec = {atom,term}
% reclist = [rec,...]
% asset = {bin,reclist}
% eval_data = [{list,asset}]
% eval_set = [eval_data,...]
% recs = {atom,[term]}
% recslist = [recs,...]
addrec({Key,Val},Recslist) ->
Val_list = proplists:get_value(Key, Recslist, []),
[{Key,[Val|Val_list]}|proplists:delete(Key,Recslist)].
merge_rec(Reclist,Recslist) -> lists:foldl(fun(Rec,Acc) -> addrec(Rec,Acc) end,Recslist,Reclist).
merge_eval([{Eval,{Asset,Reclist}}],[]) ->
[{Eval,{Asset,[{Key,[Val]} || {Key,Val} <- Reclist]}}];
merge_eval([{Eval,{Asset,Reclist}}],[{Eval_low,{Asset_low,Recslist}}]) ->
[{min(Eval,Eval_low),{min(Asset,Asset_low),merge_rec(Reclist,Recslist)}}].
merge_set(Eval_set) -> lists:foldl(fun(Eval_data,Acc) -> merge_eval(Eval_data,Acc) end,[],Eval_set).
test() ->
Eval_set = [[{"eval_data_10",
{<<"prvl_mobable_asset_0000_h200401">>,
[{'F_01_0100',[1]},
{'F_01_0090',["3"]},
{'F_01_0080',[]},
{'F_01_0070',[9999]},
{'F_01_0060',[{era,0},{year,[]},{month,[]}]},
{'F_01_0050',[]},
{'F_01_0040',[]},
{'F_01_0030',[]},
{'F_01_0020',<<>>},
{'F_01_0010',<<"4 - 8">>}]}}],
[{"eval_data_11",
{<<"prvl_mobable_asset_0000_h200401">>,
[{'F_01_0100',[]},
{'F_01_0090',["2"]},
{'F_01_0080',[]},
{'F_01_0070',[22222]},
{'F_01_0060',[{era,0},{year,[]},{month,[]}]},
{'F_01_0050',[]},
{'F_01_0040',[]},
{'F_01_0030',[]},
{'F_01_0020',<<>>},
{'F_01_0010',<<"4 - 1">>}]}}]],
merge_set(Eval_set).
#trex:
FormList = [[{EVAL_SEQ_1, {FORMNAME, ListDataToConcat1}}], [{EVAL_SEQ_2, {FORMNAME, ListDataToConcat2}}], ....]
You will need to get a separate the list which has common form names as <<"prvl_mobable_asset_0000_h200401">>
{MobableList, EvalDataList} = lists:partition(fun([{EVAL_SEQ, {FORMNAME, ListData}}]) ->
FORMNAME == <<"prvl_mobable_asset_0000_h200401">>
end, FormList),
Then, Get a separate Tuple of Form sequences and Lists. To separate them
{EvalSeq, MergingList} = lists:foldl(fun(X, {EvalNames, OutList}) ->
[{EVAL_SEQ, {FORMNAME, ListData}}] = X,
{[EVAL_SEQ|EvalNames], [ListData|OutList]}
end, {[], []}, MobableList),
Hence, you will get the new tuple as :
{[EVAL_SEQ_1, EVAL_SEQ_2, EVAL_SEQ_3, ...], [ListDataToConcat1, ListDataToConcat2, ListDataToConcat3,...]}
I'm not sure which sequence Number you want as you havn't mentioned it clearly, Here is the way you can get the minimum Sequence Number.
Evalsequence = lists:min(EvalSeq),
Now merge your code using merge function as shown below or you can refer merging inner list Merge inner lists of a list erlang:
MergedList = merge(MergingList),
And finally a separate list as:
[{Evalsequence, {<<"prvl_mobable_asset_0000_h200401">>, MergedList}}].
merge(ListOfLists) ->
Combined = lists:append(ListOfLists),
Fun = fun(Key) -> {Key,proplists:get_all_values(Key,Combined)} end,
lists:map(Fun,proplists:get_keys(Combined)).

Resources