I have a Rails/SQLite3 setup which is getting exceptions
ActiveRecord::StatementInvalid (SQLite3::BusyException: database is locked):
Now I understand that SQLite3 cannot do parallel writes, I'd be happy if they were blocked until the other write was finished, so done serially, rather than raising. This is something to do with default SQLite3 handler in C falling foul of Ruby's GIL, apparently. The fix, according to this post, is to install one's own "busy handler" for the SQLite3 connection, and I have this code in a Rails initializer
# config/initializers/sqlite3.rb
if ActiveRecord::Base.connection.adapter_name == 'SQLite' then
if raw_connection = ActiveRecord::Base.connection.raw_connection then
puts 'installing busy handler'
raw_connection.busy_handler do |count|
puts 'QUACK'
end
puts 'done'
else
raise RuntimeError 'no DB raw connection!'
end
end
On starting Rails (Puma in the console) I get the expected
installing busy handler
done
but on running parallel requests I get the exception with no QUACK, i.e., it seems that my handler has not been called. Is this a misunderstanding of the scope of initializers? Is there a "correct" way to get this handler installed?
(Obviously the real handler will not just say QUACK, but I find that my "real" handler has no effect so replace it by a debugging version.)
say I have a class which traps SIGTERM and I want to write a spec to verify that that specific code is ran when SIGTERM is received. What is the proper way of doing it?
I've followed the answer for this topic: How to test signal handling in RSpec, particularly handling of SIGTERM?, but rspec is terminated on Process.kill happens.
I've also tried it like this:
raise SignalException.new('TERM')
But it doesn't seem to do anything (trap is not triggered). Finnaly, I've tried using 'allow' to substitute a method which is called during the spec to raise the signal or call Process.kill like this:
allow(<Class>).to receive(<method>).and_raise(SignalException.new('TERM'))
allow(<Class>).to receive(<method>).and_return(Process.kill 'TERM',0)
When raising the signal it also doesn't seem to do anything, and calling Process.kill simply ends rspec without a stack trace, just the word 'Terminated').
The trap code is like this:
trap('SIGTERM') {
Rails.logger.error('term')
#received_sigterm = true
}
I have an operation that I need to execute in my rails application that before my Rails app dies. Is there a hook I can utilize in Rails for this? Something similar to at_exit I guess.
Ruby itself supports two hooks, BEGIN and END, which are run at the start of a script and as the interpreter stops running it.
See "What does Ruby's BEGIN do?" for more information.
The BEGIN documentation says:
Designates, via code block, code to be executed unconditionally before sequential execution of the program begins. Sometimes used to simulate forward references to methods.
puts times_3(gets.to_i)
BEGIN {
def times_3(n)
n * 3
end
}
The END documentations says:
Designates, via code block, code to be executed just prior to program termination.
END { puts "Bye!" }
Okay so I am making no guarantees as to impact because I have not tested this at all but you could define your own hook e.g.
ObjectSpace.define_finalizer(YOUR_RAILS_APP::Application, proc {puts "exiting now"})
Note this will execute after at_exit so the rails application server output will look like
Stopping ...
Exiting
exiting now
With Tin Man's solution included
ObjectSpace.define_finalizer(YOUR_RAILS_APP::Application, proc {puts "exiting now"})
END { puts "exiting again" }
Output is
Stopping ...
Exiting
exiting again
exiting now
I've a rails app that I'm crashing on purpose.. it's local and I'm just hitting ctrl + c and killing it mid way through processing records..
To my mind the records in the block shouldn't have been committed.. Is this a postgres "error" or a rails "error", or a dave ERROR?
ActiveRecord::Base.transaction do
UploadStage.where("id in (#{ids.join(',')})").update_all(:status => 2);
records.each do |record|
record.success = process_line(record.id, klas, record.hash_value).to_s[0..250]
record.status = 1000
record.save();
end
end
I generate my ids by reading out all the records where the status is 1.
Nothing but this function sets the status to 1000..
If the action crashes for what ever reason, I'd expect there to be no records in the database with status = 2...
This is not what I'm seeing though. Half the records have status 1000, the other half have status 2.. .
Am I missing something?
How can I make sure there are no 2's if the app crashes?
EDIT:
I found this link http://coderrr.wordpress.com/2011/05/03/beware-of-threadkill-or-your-activerecord-transactions-are-in-danger-of-being-partially-committed/
As I suspected and as confirmed by dave's update, it looks like ActiveRecord will commit a half-finished transaction under some circumstances when you kill a thread. Woo, safe! See dave's link for detailed explanation and mitigation options.
If you're simulating hard crash (host OS crash or plug-pull), control-C is absolutely not the right approach. Use Control-\ to send a SIGQUIT, which is generally not handled, or use kill -KILL to hard-kill the process with no opportunity to do cleanup. Control-C sends SIGINT which is a gentle signal that's usually attached to a clean shutdown handler.
In general, if you're debugging issues like this, you should enable detailed query logging and see what Rails is doing. Use log_statement = 'all' in postgresql.conf then examine the PostgreSQL logs.
Could you tell me when to use throw, exit and error?
1> catch throw ({aaa}).
{aaa}
2> catch exit ({aaa}).
{'EXIT',{aaa}}
3> catch gen_server:call(aaa,{aaa}).
{'EXIT',{noproc,{gen_server,call,[aaa,{aaa}]}}}
4> catch exit("jaj")
{'EXIT',"jaj"}
There are 3 classes which can be caught with a try ... catch: throw, error and exit.
throw is generated using throw/1 and is intended to be used for non-local returns and does not generate an error unless it is not caught (when you get a nocatch error).
error is generated when the system detects an error. You can explicitly generate an error using error/1. The system also includes a stacktrace in the generated error value, for example {badarg,[...]}.
exit is generated using exit/1 and is intended to signal that this process is to die.
The difference between error/1 and exit/1 is not that great, it more about intention which the stacktrace generated by errors enhances.
The difference between them is actually more noticeable when doing catch ...: when throw/1 is used then the catch just returns the thrown value, as is expected from a non-local return; when an error/1 is used then the catch returns {'EXIT',Reason} where Reason contains the stacktrace; while from exit/1 catch also returns {'EXIT',Reason} but Reason only contains the actual exit reason. try ... catch looks like it equates them, but they are/were very different.
[UPDATED]
I glossed over the important difference between throw and error, pointed out by Robert Virding. This edit is just for the record!
throw error is to be used where one would use throw in other languages. An error in a running process has been detected by your code, which signals an exception with error/1. The same process catches it (possibly higher up in the stack), and the error is to be handled within the same process. error always brings with it a stacktrace.
throw is to be used not to signal an error, but just to return a value from a deeply nested function.
Since it unwinds the stack, calling throw returns the thrown value to the place it was caught. As in the case of error, we're catching stuff that was thrown, only what was thrown wasn't an error but rather just a value passed up the stack. This is why throw does not bring with it a stacktrace.
As a contrived example, if we wanted to implement an exists function for lists, (similar to what list:any does) and as an exercise without doing the recursing ourselves, and using just list:foreach, then throw could be used here:
exists(P, List) ->
F = fun(X) ->
case P(X) of
true -> throw(true);
Whatever -> Whatever
end
end,
try lists:foreach(F, List) of
ok -> false
catch
true -> true
end.
A value thrown but not caught is treated as an error: a nocatch exception will be generated.
EXIT is to be signaled by a process when it 'gives up'. The parent process handles the EXIT, while the child process just dies. This is the Erlang let-it-crash philosophy.
So exit/1's EXIT is not to be caught within the same process, but left to the parent. error/1's errors are local to the process - i.e., a matter of what happens and how it is handled by the process itself; throw/1 is used for control flow across the stack.
[UPDATE]
This tutorial explains it well: http://learnyousomeerlang.com/errors-and-exceptions
Note there is also a exit/2 - called with a Pid of a process to send the EXIT to.
exit/1 implies the parent process.
I'm new to Erlang, but here's how I think about what these things are, their differences, what they're used for, etc.:
throw: a condition that should be handled locally (i.e. within the current process). E.g. caller is looking for an element in a collection, but does not know if the collection actually contains such an element; then, the callee could throw if such an element is not present, and the caller detect absence by using try[/of]/catch. If caller neglects to do this, then this gets turned into an nocatch error (explained below).
exit: The current process is done. E.g. it has simply finished (in that case, you'd pass normal, which is treated the the same as the original function returning), or its operation was cancelled (E.g. it normally loops indefinitely but has just received a shut_down message).
error: the process has done something and/or reached a state that the programmer did not take into account (E.g. 1/0), believes is impossible (E.g. case ... of encounters a value that does not match any case), or some precondition is not met (E.g. input is nonempty). In this case, local recovery doesn't make sense. Therefore, neither throw nor exit is appropriate. Since this is unexpected, a stack trace is part of the Reason.
As you can see, the above list is in escalating order:
throw is for sane conditions that the caller is expected to handle. I.e. handling occurs within the current process.
exit is also sane, but should end the current process simply because the process is done.
error is insane. Something happened that can't reasonably be recovered from (usually a bug?), and local recovery would not be appropriate.
vs. other languages:
throw is analogous to the way checked exceptions are used in Java. Whereas, error is used in a manner more analogous to unchecked exceptions. Checked exceptions are exceptions you want the caller to handle. Java requires you to either wrap calls in try/catch or declare that your method throws such exceptions. Whereas, unchecked exceptions generally propagate to the outermost caller.
exit does not have a good analog in more "conventional" languages like Java, C++, Python, JavaScript, Ruby, etc. exit vaguely like an uber-return: instead of returning at the end, you can return from the middle of a function, except you don't just return from the current function, you return from them ALL.
exit Example
serve_good_times() ->
receive
{top_of_the_mornin, Sender} ->
Sender ! and_the_rest_of_the_day_to_yourself;
{you_suck, Sender} ->
Sender ! take_a_chill_pill;
% More cases...
shut_down ->
exit(normal)
end,
serve_good_times()
end
Since serve_good_times calls itself after almost all messages, the programmer has decided that we don't want to repeat that call in every receive case. Therefore, she has put that call after the receive. But then, what if serve_good_times decides to stop calling itself? This is where exit comes to the rescue. Passing normal to exit causes the process to terminate just as though the last function call has returned.
As such, it's generally inappropriate to call exit in a general purpose library, like lists. It's none of the library's business whether the process should end; that should be decided by application code.
What About Abnormal exit?
This matters if another process (the "remote" process) is linked to the "local" process that calls exit (and process_flag(trap_exit, true) was not called): Just like the last function returning, exit(normal) does not cause remote process to exit. But if the local process makes a exit(herp_derp) call, then the remote process also exits with Reason=herp_derp. Of course, if the remote process is linked to yet more processes, they also get exit signal with Reason=herp_derp. Therefore, non-normal exits result in a chain reaction.
Let's take a look at this in action:
1> self().
<0.32.0>
2> spawn_link(fun() -> exit(normal) end).
<0.35.0>
3> self().
<0.32.0>
4>
4>
4> spawn_link(fun() -> exit(abnormal) end).
** exception exit: abnormal
5> self().
<0.39.0>
6>
The first process that we spawned did not cause the shell to exit (we can tell, because self returned the same pid before and after spawn_link). BUT the second process did cause the shell to exit (and the system replaced the shell process with a new one).
Of course, if the remote process uses process_flag(trap_exit, true) then it just gets a message, regardless of whether the local process passes normal or something else to exit. Setting this flag stops the chain reaction.
6> process_flag(trap_exit, true).
false
7> spawn_link(fun() -> exit(normal) end).
<0.43.0>
8> self().
<0.39.0>
9> flush().
Shell got {'EXIT',<0.43.0>,normal}
ok
10>
10>
10> spawn_link(fun() -> exit(abnormal) end).
<0.47.0>
11> self().
<0.39.0>
12> flush().
Shell got {'EXIT',<0.47.0>,abnormal}
Recall that I said that exit(normal) is treated like the original function returning:
13> spawn_link(fun() -> ok end).
<0.51.0>
14> flush().
Shell got {'EXIT',<0.51.0>,normal}
ok
15> self().
<0.39.0>
What do you know: the same thing happened as when exit(normal) was called. Wonderful!