When reading rabbitmq's rabbit.erl，it contain hipe compilation related code.
hipe_compile() ->
Count = length(?HIPE_WORTHY),
io:format("HiPE compiling: |~s|~n |",
[string:copies("-", Count)]),
T1 = erlang:now(),
PidMRefs = [spawn_monitor(fun () -> [begin
{ok, M} = hipe:c(M, [o3]),
io:format("#")
end || M <- Ms]
end) ||
Ms <- split(?HIPE_WORTHY, ?HIPE_PROCESSES)],
[receive
{'DOWN', MRef, process, _, normal} -> ok;
{'DOWN', MRef, process, _, Reason} -> exit(Reason)
end || {_Pid, MRef} <- PidMRefs],
T2 = erlang:now(),
io:format("|~n~nCompiled ~B modules in ~Bs~n",
[Count, timer:now_diff(T2, T1) div 1000000]).
But there is no explanation about hipe in the erlang's reference doc. What's the meaning of 'o3'?
(emacs#chen-yumatoMacBook-Pro.local)51> hipe:c(xx_reader,[o3]).
{ok,xx_reader}
After I use hipe:c as above, No new compiled file can be found the in the pwd() directory?
Where it is?
o3 indicates the optimization level used by the compiler. There're also levels o0, o1, o2. Details of the levels are as follows:
o1 = [inline_fp,pmatch,peephole],
o2 = [icode_range,icode_ssa_const_prop,icode_ssa_copy_prop,icode_type,
icode_inline_bifs,rtl_lcm,rtl_ssa,rtl_ssa_const_prop,spillmin_color,
use_indexing,remove_comments,concurrent_comp,binary_opt] ++ o1,
o3 = [{regalloc,coalescing},icode_range] ++ o2.
You can use hipe:help_option(Option) to further investigate the meanings of different options. For example,
3> hipe:help_option(regalloc).
regalloc - Select register allocation algorithm. Used as {regalloc, METHOD}.
Currently available methods:
naive - spills everything (for debugging and testing)
linear_scan - fast; not so good if few registers available
graph_color - slow, but gives OK performance
coalescing - slower, tries hard to use registers
optimistic - another variant of a coalescing allocator
ok
4> hipe:help_option(icode_range).
icode_range - Performs integer range analysis on the Icode level
ok
I think HiPE is JIT compilation, just as the one used in Java. The native parts are available only in runtime, so there should be no explicit representation in your file system.
Also, hipe:c do require a .beam file is present. For example, if you create a test.erl with some stuff, and without compiling it to a .beam file, call hipe:c directly will lead to an error:
1> hipe:c(test, [o3]).
<HiPE (v 3.9.3)> EXITED with reason {cant_find_beam_file,test} #hipe:419
=ERROR REPORT==== 29-Nov-2012::17:03:02 ===
<HiPE (v 3.9.3)> Error: [hipe:418]: Cannot find test.beam file.** exception error: {hipe,419,{cant_find_beam_file,test}}
in function hipe:beam_file/1 (hipe.erl, line 419)
in call from hipe:c/2 (hipe.erl, line 313)
2> c(test).
{ok,test}
3> hipe:c(test, [o3]).
{ok,test}
There is some in erlang's doc. See here. But the doc is not much indeed. The index page of HiPE only updated recently.
Also, you can check some help in erlang shell.
> hipe:help().
> hipe:help_options().
> hipe:help_option(Option).
Related
I wanted to send a message to a process after a delay, and discovered erlang:send_after/4.
When looking at the docs it looked like this is exactly what I wanted:
erlang:send_after(Time, Dest, Msg, Options) -> TimerRef
Starts a timer. When the timer expires, the message Msg is sent to the
process identified by Dest.
However, it doesn't seem to work when the destination is running on another node - it tells me one of the arguments are bad.
1> P = spawn('node#host', module, function, [Arg]).
<10585.83.0>
2> erlang:send_after(1000, P, {123}).
** exception error: bad argument
in function erlang:send_after/3
called as erlang:send_after(1000,<10585.83.0>,{123})
Doing the same thing with timer:send_after/3 appears to work fine:
1> P = spawn('node#host', module, function, [Arg]).
<10101.10.0>
2> timer:send_after(1000, P, {123}).
{ok,{-576458842589535,#Ref<0.1843049418.1937244161.31646>}}
And, the docs for timer:send_after/3 state almost the same thing as the erlang version:
send_after(Time, Pid, Message) -> {ok, TRef} | {error, Reason}
Evaluates Pid ! Message after Time milliseconds.
So the question is, why do these two functions, which on the face of it do the same thing, behave differently? Is erlang:send_after broken, or mis-advertised? Or maybe timer:send_after isn't doing what I think it is?
TL;DR
Your assumption is correct: these are intended to do the same thing, but are implemented differently.
Discussion
Things in the timer module such as timer:send_after/2,3 work through the gen_server that defines that as a service. Like any other service, this one can get overloaded if you assign a really huge number of tasks (timers to track) to it.
erlang:send_after/3,4, on the other hand, is a BIF implemented directly within the runtime and therefore have access to system primitives like the hardware timer. If you have a ton of timers this is definitely the way to go. In most programs you won't notice the difference, though.
There is actually a note about this in the Erlang Efficiency Guide:
3.1 Timer Module
Creating timers using erlang:send_after/3 and erlang:start_timer/3 , is much more efficient than using the timers provided by the timer module in STDLIB. The timer module uses a separate process to manage the timers. That process can easily become overloaded if many processes create and cancel timers frequently (especially when using the SMP emulator).
The functions in the timer module that do not manage timers (such as timer:tc/3 or timer:sleep/1), do not call the timer-server process and are therefore harmless.
A workaround
A workaround to gain the efficiency of the BIF without the same-node restriction is to have a process of your own that does nothing but wait for a message to forward to another node:
-module(foo_forward).
-export([send_after/3, cancel/1]).
% Obviously this is an example only. You would want to write this to
% be compliant with proc_lib, write a proper init/N and integrate with
% OTP. Note that this snippet is missing the OTP service functions.
start() ->
spawn(fun() -> loop(self(), [], none) end).
send_after(Time, Dest, Message) ->
erlang:send_after(Time, self(), {forward, Dest, Message}).
loop(Parent, Debug, State) ->
receive
{forward, Dest, Message} ->
Dest ! Message,
loop(Parent, Debug, State);
{system, From, Request} ->
sys:handle_msg(Request, From, Parent, ?MODULE, Debug, State);
Unexpected ->
ok = log(warning, "Received message: ~tp", [Unexpected]),
loop(Parent, Debug, State)
end.
The above example is a bit shallow, but hopefully it expresses the point. It should be possible to get the efficiency of the BIF erlang:send_after/3,4 but still manage to send messages across nodes as well as give you the freedom to cancel a message using erlang:cancel_timer/1
But why?
The puzzle (and bug) is why erlang:send_after/3,4 does not want to work across nodes. The example you provided above looks a bit odd as the first assignment to P was the Pid <10101.10.0>, but the crashed call was reported as <10585.83.0> -- clearly not the same.
For the moment I do not know why erlang:send_after/3,4 doesn't work, but I can say with confidence that the mechanism of operation between the two is not the same. I'll look into it, but I imagine that the BIF version is actually doing some funny business within the runtime to gain efficiency and as a result signalling the target process by directly updating its mailbox instead of actually sending an Erlang message on the higher Erlang-to-Erlang level.
Maybe it is good that we have both, but this should definitely be clearly marked in the docs, and it evidently is not (I just checked).
There is some difference in timeout order if you have many timers.
The example below shows erlang:send_after does not guarantee order, but
timer:send_after does.
1> A = lists:seq(1,10).
[1,2,3,4,5,6,7,8,9,10]
2> [erlang:send_after(100, self(), X) || X <- A].
...
3> flush().
Shell got 2
Shell got 3
Shell got 4
Shell got 5
Shell got 6
Shell got 7
Shell got 8
Shell got 9
Shell got 10
Shell got 1
ok
4> [timer:send_after(100, self(), X) || X <- A].
...
5> flush().
Shell got 1
Shell got 2
Shell got 3
Shell got 4
Shell got 5
Shell got 6
Shell got 7
Shell got 8
Shell got 9
Shell got 10
ok
I am trying to extend the Erlang Mixer Library (https://github.com/opscode/mixer) to pass the "-spec()." lines for the functions that it is adding to a module. However I am unclear on how erlc places specs into the core erlang code.
I started with a really simple (test) module:
-module(mix1).
-export([square/1]).
-spec(square(number()) -> number()).
square(X) -> X * X.
And compiled it with "erlc +debug_info -S mix1.erl" and got this (module_info functions removed):
{module, mix1}. %% version = 0
{exports, [{module_info,0},{module_info,1},{square,1}]}.
{attributes, []}.
{labels, 7}.
{function, square, 1, 2}.
{label,1}.
{line,[{location,"mix1.erl",7}]}.
{func_info,{atom,mix1},{atom,square},1}.
{label,2}.
{line,[{location,"mix1.erl",8}]}.
{gc_bif,'*',{f,0},1,[{x,0},{x,0}],{x,0}}.
return.
I am trying to figure out how the "-spec()" are translated and I don't see them there, any ideas? What am I missing. The eventual goal here is to use this for a parse transform.
I believe your output is not exactly AST, but some normalized form of AST.
{ok, Forms} = epp:parse_file("mix1.erl",[],[]).
{ok,[{attribute,1,file,{"mix1.erl",1}},
{attribute,1,module,mix1},
{attribute,2,export,[{square,1}]},
{attribute,3,spec,
{{square,1},
[{type,3,'fun',
[{type,3,product,[{type,3,number,[]}]},
{type,3,number,[]}]}]}},
{function,4,square,1,
[{clause,4,
[{var,4,'X'}],
[],
[{op,4,'*',{var,4,'X'},{var,4,'X'}}]}]},
{eof,6}]}
I work in erlang
Now, I have a big problem
I want to have a log from a table mnesia and this log should be write in excel file
So the goal is write data from table mnesia to the excel file
I think and this is related to some code find in this forum that the best way is to write .txt file then transfer data from .txt file to excel file
I find this code in this forum in this link.
exporttxt()->
F = fun(T) -> mnesia:foldl(fun(X,Acc) -> [X|Acc] end, [],T) end,
{atomic,L} = mnesia:transaction(F(user)),
file:write_file("test.txt",[io_lib:format("~p\t~p\t~p~n",[F1,F2,F3]) ||
#user{id = F1,adress = F2,birthday = F3} <- L]).
But this code produces an error
As commented, the problem is clearly explained in the link itself. If you want the code then here it is. But please understand before directly jumping into the code.
exporttxt()->
F = fun() -> mnesia:foldl(fun(X,Acc) -> [X|Acc] end, [],user) end,
{atomic,L} = mnesia:transaction(F),
file:write_file("test.txt",[io_lib:format("~p\t~p\t~p~n",[F1,F2,F3]) ||
#user{id = F1,adress = F2,birthday = F3} <- L]).
in the subject you mention, I said that I didn't test the code, and of course there was a syntax error.
Here is a code that run.
1> ok = mnesia:create_schema([node()]).
ok
2> rd(my_user,{firstname,lastname,age}).
my_user
3> ok =application:start(mnesia).
ok
4> {atomic,ok} = mnesia:create_table(my_user,[{attributes,record_info(fields,my_user)},{disc_copies,[node()]},{type,bag}]).
{atomic,ok}
5> Add_user = fun(Fn,Ln,Ag) ->
5> F = fun() -> mnesia:write(my_user,#my_user{firstname=Fn,lastname=Ln,age=Ag},write) end,
5> mnesia:activity(transaction,F)
5> end.
#Fun<erl_eval.18.82930912>
6> ok = Add_user("Georges","Boy",25).
ok
7> ok = Add_user("Joe","Armstrong",52).
ok
8> ok = Add_user("Pascal","Me",55).
ok
9> F = fun(T) -> mnesia:foldl(fun(X,Acc) -> [X|Acc] end, [],T) end.
#Fun<erl_eval.6.82930912>
10> {atomic,L} = mnesia:transaction(F,[my_user]).
{atomic,[#my_user{firstname = "Pascal",lastname = "Me",
age = 55},
#my_user{firstname = "Joe",lastname = "Armstrong",age = 52},
#my_user{firstname = "Georges",lastname = "Boy",age = 25}]}
11> ok = file:write_file("test.txt",[io_lib:format("~p\t~p\t~p~n",[F1,F2,F3]) || #my_user{firstname = F1, lastname = F2, age = F3} <- L]).
ok
12>
you will have in your working directory a file named test.txt containing
"Pascal" "Me" 55
"Joe" "Armstrong" 52
"Georges" "Boy" 25
and if you open it with excel you will get
But this is not a code sequence you should use directly.
line 1 should take place in a code use for the deployment of your application.
line 2 is a record definition, necessary for the shell to understand the next lines. It should be replaced by a -record(...) in a module or an included file.
lines 3 and 4 should take place in the init function of one of the higher level supervisor (with some test to check already started application, existing table...)
line 5 should be in the interface definition of a server
line 6,7,8 should be generated by a user interface in some client
and last line 9,10,11 in another interface (for admin?).
Message to Lost_with_coding and his fellows,
If I may give you my opinion, you are burning steps. You should focus on mastering the Erlang syntax, the concept of pattern matching and variable bounding... after you should look at more advance construction such as list and binary comprehensions. Take time to look at error messages and use them to solve simple problems. The official Erlang documentation is great for this purpose. I always have it open in my browser and sometimes I also use this link http://erldocs.com/R15B/ when I look for function I don't know.
Next will come higher order functions, error handling, processes, concurrency, OTP... plus the usage of the efficient but not sexy Erlang tools (tv, appmon, debugger...).
I recommend it very often, but use the fantastic Fred Hebert's web site http://learnyousomeerlang.com/ and follow it step by step, rewriting the code, not copy/paste; it really worths the effort.
In most applications, its hard to avoid the need to query large amounts of information which a user wants to browse through. This is what led me to cursors. With mnesia, cursors are implemented using qlc:cursor/1 or qlc:cursor/2. After working with them for a while and facing this problem many times,
11> qlc:next_answers(QC,3).
** exception error: {qlc_cursor_pid_no_longer_exists,<0.59.0>}
in function qlc:next_loop/3 (qlc.erl, line 1359)
12>
It has occured to me that the whole cursor thing has to be within one mnesia transaction: executes as a whole once. like this below
E:\>erl
Eshell V5.9 (abort with ^G)
1> mnesia:start().
ok
2> rd(obj,{key,value}).
obj
3> mnesia:create_table(obj,[{attributes,record_info(fields,obj)}]).
{atomic,ok}
4> Write = fun(Obj) -> mnesia:transaction(fun() -> mnesia:write(Obj) end) end.
#Fun<erl_eval.6.111823515>
5> [Write(#obj{key = N,value = N * 2}) || N <- lists:seq(1,100)],ok.
ok
6> mnesia:transaction(fun() ->
QC = cursor_server:cursor(qlc:q([XX || XX <- mnesia:table(obj)])),
Ans = qlc:next_answers(QC,3),
io:format("\n\tAns: ~p~n",[Ans])
end).
Ans: [{obj,20,40},{obj,21,42},{obj,86,172}]
{atomic,ok}
7>
When you attempt to call say: qlc:next_answers/2 outside a mnesia transaction, you will get an exception. Not only just out of the transaction, but even if that method is executed by a DIFFERENT process than the one which created the cursor, problems are bound to happen.
Another intresting finding is that, as soon as you get out of a mnesia transaction, one of the processes which are involved in a mnesia cursor (apparently mnesia spawns a process in the background), exits, causing the cursor to be invalid. Look at this below:
-module(cursor_server).
-compile(export_all).
cursor(Q)->
case mnesia:is_transaction() of
false ->
F = fun(QH)-> qlc:cursor(QH,[]) end,
mnesia:activity(transaction,F,[Q],mnesia_frag);
true -> qlc:cursor(Q,[])
end.
%% --- End of module -------------------------------------------
Then in shell, i use that method:
7> QC = cursor_server:cursor(qlc:q([XX || XX <- mnesia:table(obj)])).
{qlc_cursor,{<0.59.0>,<0.30.0>}}
8> erlang:is_process_alive(list_to_pid("<0.59.0>")).
false
9> erlang:is_process_alive(list_to_pid("<0.30.0>")).
true
10> self().
<0.30.0>
11> qlc:next_answers(QC,3).
** exception error: {qlc_cursor_pid_no_longer_exists,<0.59.0>}
in function qlc:next_loop/3 (qlc.erl, line 1359)
12>
So, this makes it very Extremely hard to build a web application in which a user needs to browse a particular set of results, group by group say: give him/her the first 20, then next 20 e.t.c. This involves, getting the first results, send them to the web page, then wait for the user to click NEXT then ask qlc:cursor/2 for the next 20 and so on. These operations cannot be done, while hanging inside a mnesia transaction !!! The only possible way, is by spawning a process which will hang there, receiving and sending back next answers as messages and receiving the next_answers requests as messages like this:
-define(CURSOR_TIMEOUT,timer:hours(1)).
%% initial request is made here below
request(PageSize)->
Me = self(),
CursorPid = spawn(?MODULE,cursor_pid,[Me,PageSize]),
receive
{initial_answers,Ans} ->
%% find a way of hidding the Cursor Pid
%% in the page so that the subsequent requests
%% come along with it
{Ans,pid_to_list(CursorPid)}
after ?CURSOR_TIMEOUT -> timedout
end.
cursor_pid(ParentPid,PageSize)->
F = fun(Pid,N)->
QC = cursor_server:cursor(qlc:q([XX || XX <- mnesia:table(obj)])),
Ans = qlc:next_answers(QC,N),
Pid ! {initial_answers,Ans},
receive
{From,{next_answers,Num}} ->
From ! {next_answers,qlc:next_answers(QC,Num)},
%% Problem here ! how to loop back
%% check: Erlang Y-Combinator
delete ->
%% it could have died already, so we be careful here !
try qlc:delete_cursor(QC) of
_ -> ok
catch
_:_ -> ok
end,
exit(normal)
after ?CURSOR_TIMEOUT -> exit(normal)
end
end,
mnesia:activity(transaction,F,[ParentPid,PageSize],mnesia_frag).
next_answers(CursorPid,PageSize)->
list_to_pid(CursorPid) ! {self(),{next_answers,PageSize}},
receive
{next_answers,Ans} ->
{Ans,pid_to_list(CursorPid)}
after ?CURSOR_TIMEOUT -> timedout
end.
That would create a more complex problem of managing process exits, tracking / monitoring e.t.c. I wonder why the mnesia implementers didnot see this !
Now, that brings me to my questions. I have been walking around the web for solutions and you could please check out these links from which the questions arise: mnemosyne, Ulf Wiger's Solution to Cursor Problems, AMNESIA - an RDBMS implementation of mnesia.
1. Does anyone have an idea on how to handle mnesia query cursors in a different way from what is documented, and is worth sharing ?
2. What are the reasons why mnesia implemeters decided to force the cursors within a single transaction: even the calls for the next_answers ?
3. Is there anything, from what i have presented, that i do not understand clearly (other than my bad buggy illustration code - please ignore those) ?
4. AMNESIA (on section 4.7 of the link i gave above), has a good implementation of cursors, because the subsequent calls for the next_answers, do not need to be in the same transaction, NOR by the same process. Would you advise anyone to switch from mnesia to amnesia due to this and also, is this library still supported ?
5. Ulf Wiger , (the author of many erlang libraries esp. GPROC), suggests the use of mnesia:select/4. How would i use it to solve cursor problems in a web application ? NOTE: Please do not advise me to leave mnesia and use something else, because i want to use mnesia for this specific problem. I appreciate your time to read through all this question.
The motivation is that transaction grabs (in your case) read locks.
Locks can not be kept outside of transactions.
If you want, you can run it in a dirty_context, but you loose the
transactional properties, i.e. the table may change between invocations.
make_cursor() ->
QD = qlc:sort(mnesia:table(person, [{traverse, select}])),
mnesia:activity(async_dirty, fun() -> qlc:cursor(QD) end, mnesia_frag).
get_next(Cursor) ->
Get = fun() -> qlc:next_answers(Cursor,5) end,
mnesia:activity(async_dirty, Get, mnesia_frag).
del_cursor(Cursor) ->
qlc:delete_cursor(Cursor).
I think this may help you :
use async_dirty instead of transaction
{Record,Cont}=mnesia:activity(async_dirty, fun mnesia:select/4,[md,[{Match_head,[Guard],[Result]}],Limit,read])
then read next Limit number of records:
mnesia:activity(async_dirty, fun mnesia:select/1,[Cont])
full code:
-record(md,{id,name}).
batch_delete(Id,Limit) ->
Match_head = #md{id='$1',name='$2'},
Guard = {'<','$1',Id},
Result = '$_',
{Record,Cont} = mnesia:activity(async_dirty, fun mnesia:select/4,[md,[{Match_head,[Guard],[Result]}],Limit,read]),
delete_next({Record,Cont}).
delete_next('$end_of_table') ->
over;
delete_next({Record,Cont}) ->
delete(Record),
delete_next(mnesia:activity(async_dirty, fun mnesia:select/1,[Cont])).
delete(Records) ->
io:format("delete(~p)~n",[Records]),
F = fun() ->
[ mnesia:delete_object(O) || O <- Records]
end,
mnesia:transaction(F).
remember you can not use cursor out of one transaction
Is it possible to know the name of a function in itself ?
a_function()->
io:format("I am ~p!", [????]). % What to use here?
Use the macro ?FUNCTION_NAME to get the name of the current function as an atom, and ?FUNCTION_ARITY to get the arity as an integer.
Example:
-module(test).
-export([a_function/2]).
a_function(_Foo, _Bar) ->
io:write("I am ~p/~p!",[?FUNCTION_NAME, ?FUNCTION_ARITY]).
1> c(test).
{ok,test}
2> test:a_function(a, b).
I am a_function/2!
This was implemented in EEP-0045.
For Erlang Versions 18 and Older
In older Erlang versions, there's no simple way to get the current function name at compile time. You can however retrieve it at runtime:
{current_function, {M, F, A}} = process_info(self(), current_function)
Where A is the arity (number of arguments), not the actual arguments. The first argument to process_info/2 is a process ID which can be either the current process (self()) or an other process. For example:
1> process_info(self(), current_function).
{current_function,{erl_eval,do_apply,5}}
Note however, that while this would be functionally equivalent to the ?FUNCTION_NAME macro, it's much slower because it is evaluated in runtime.
at runtime, you could throw an exception and check the top of the stacktrace.
foo() ->
catch throw(away),
[{Module, Fun, Arity} | _] = erlang:get_stacktrace(),
io:format("I am ~p:~p/~p!~n",[Module, Fun, Arity]).