In my rails application, I have a background process runner, model name Worker, that checks for new tasks to run every 10 seconds. This check generates two SQL queries each time - one to look for new jobs, one to delete old completed ones.
The problem with this - the main log file gets spammed for each of those queries.
Can I direct the SQL queries spawned by the Worker model into a separate log file, or at least silence them? Overwriting Worker.logger does not work - it redirects only the messages that explicitly call logger.debug("something").
The simplest and most idiomatic solution
logger.silence do
do_something
end
See Logger#silence
Queries are logged at Adapter level as I demonstrated here.
How do I get the last SQL query performed by ActiveRecord in Ruby on Rails?
You can't change the behavior unless tweaking the Adapter behavior with some really really horrible hacks.
class Worker < ActiveRecord::Base
def run
old_level, self.class.logger.level = self.class.logger.level, Logger::WARN
run_outstanding_jobs
remove_obsolete_jobs
ensure
self.class.logger.level = old_level
end
end
This is a fairly familiar idiom. I've seen it many times, in different situations. Of course, if you didn't know that ActiveRecord::Base.logger can be changed like that, it would have been hard to guess.
One caveat of this solution: this changes the logger level for all of ActiveRecord, ActionController, ActionView, ActionMailer and ActiveResource. This is because there is a single Logger instance shared by all modules.
Related
I am developing a Rails app for network automation. Part of app consists logic to run operations, part are operations themselves. Operation is simply a ruby class that performs several commands for network device (router, switch etc).
Right now, operation is simply part of Rails app repo. But in order to make development process more agile, I would like to decouple app and operations. I would have 2 repos - one for app and one for operations. App deploy would follow standard procedure, but operation would sync every time something is pushed to master. And what is more important, I don't want to restart app after operations repo update.
So my question is:
How to exclude several classes (or namespaces) from being cashed in production Rails app - I mean every time I call this class it would be reread file from disk. What could be potential dangers of doing so?
Some code example:
# Example operation - I would like to add or modify such classes withou
class FooOperation < BaseOperation
def perform(host)
conn = new_connection(host) # method from BaseOperation
result = conn.execute("foo")
if result =~ /Error/
# retry, its known bug in device foo
conn.execute("foo")
else
conn.exit
return success # method from BaseOperation
end
end
end
# somewhere in admin panel I would do so:
o = Operations.create(name: "Foo", class_name: "Foo")
o.id # => 123 # for next example
# Ruby worker which actually runs an operation
class OperationWorker
def perform(operation_id, host)
operation = Operation.find(operation_id)
# here, everytime I load this I want ruby to search for implementation on filesystem, never cache
klass = operation.class_name.constantize
class.new(host).perform #
end
end
i think you have quite a misunderstanding about how ruby code loading and interpretation works!
the fact that rails reloads classes at development time is kind of a "hack" to let you iterate on the code while the server has already loaded, parsed and executed parts of your application.
in order to do so, it has to implement quite some magic to unload your code and reload parts of it on change.
so if you want to have up-to-date code when executing an "operation" you are probably best of by spawning a new process. this will guarantee that your new code is read and parsed properly when executed with a blank state.
another thing you can do is use load instead of require because it will actually re-read the source on subsequent requests. you have to keep in mind, that subsequent calls to load just add to the already existing code in the ruby VM. so you need to make sure that every change is compatible with the already loaded code.
this could be circumvented by some clever instance_eval tricks, but i'm not sure that is what you want...
I have a model called SystemSettings with a name on and a value. It is where I store the majority of my configuration for my app. I need to be able to access it in my production.rb inside my rails 3.2 app. How would you go about doing this?
Since the Rails config such as production.rbis read before ActiveRecord is initialised you would need to use a callback:
Rails.application.configure do
ActiveSupport.on_load(:active_record) do
config.custom_variable = SystemSettings.find_by(name: "Foo").value
end
end
But since the callback executes later when ActiveRecord is ready you can't immediately use its value which is why your approach may be flawed due to race conditions.
Unless you are building something like a CMS where you need to provide a user interface to edit system settings you will be better off using environmental variables. They are immediately available from memory and do not have the overhead of a database query.
http://guides.rubyonrails.org/v3.2.9/initialization.html
I have a rails application with a dynamically configured time zone. It is stored in a database table containing other options, and the rails application itself is configured to UTC (default).
I've made the application itself aware of the timezone with a simple around filter using Time.use_zone(..., &block).
I would like to do something similar for my Sidekiq workers. Some of them process data that has timezone relevance, so they need it. I don't see any filtering options available in Sidekiq itself, no callbacks, before/after type things I can hook into. My current solution is to a prepend a module, like so:
module TimeZoneAwareWorker
def perform(*args)
Time.use_zone(Options.time_zone) do
super
end
end
end
and mixed in:
class MyWorker
include Sidekiq::Worker
prepend TimeZoneAwareWorker
...
end
This works fine for simple workers, but breaks down if the prepend occurs in the same class as the include Sidekiq::Worker. If the worker is subclassed, the hierarchy doesn't work out for the prepended perform to wrap the implementation.
Is there a better way? Ultimately it seems what I really want is a foolproof method of wrapping a single method with another method, and yielding the wrapped implementation.
I know my other option is monkeypatching before/after/around type callbacks into Sidekiq's implementation, but I'd like to only go there if forced.
Sidekiq has its own middleware solution:
Sidekiq has a similar notion of middleware to Rack: these are small
bits of code that can implement functionality. Sidekiq breaks
middleware into client-side and server-side.
Client-side middleware runs before the pushing of the job to Redis and allows you to modify/stop the job before it gets pushed. Client
middleware may receive the class argument as a Class object or a
String containing the name of the class.
Server-side middleware runs 'around' job processing. Sidekiq's retry feature is implemented as a simple middleware.
You can easily create your own middleware agent to add the timezone awareness code.
I have read the documentation on this guide and the class. I wish to create a logger which take logging informations each day and after let's say a week, delete the oldest logging information automatically each time.
logfile = File.open(RAILS_ROOT + '/log/'+ (Date.today << 1).to_s + '_custom.log', 'a') #create log file
logfile.sync = true #automatically flushes data to file
CUSTOM_LOGGER = CustomLogger.new(logfile, 'daily') #constant accessible anywhere
Plus, I wish to create a custom logging, so for instance something that looks like this (format):
class MyLogger < Logger
def format_message(severity, timestamp, progname, msg)
"#{timestamp} : #{msg}\n"
end
end
So basically, I would like to have a better idea where to place everything correctly under which directory. For instance, where MyLogger should be logically placed... (anywhere? A helper? or under app/config/ ?
Is that a valid way to implement this?
I made it by putting everything in config/initializers and creating a file named my_logger.rb. I'm still stuck at deleting/managing log files.
Does the server handle that part with a log rotation ( I know there's something with logrotation from the linux OS)? Or Rails can handle that internally?
Where should MyLogger be logically placed?
Probably put it under /lib. You can then require it from the initializer where you set the custom logger.
How can you periodically delete the oldest logging information?
There are a countless ways you can do this and choosing will be based on your constraints. You haven't spoken much about your constraints, so it's going to be hard to give you the just-right answer. E.g. you could clean up old logs every time you add a new log entry, you could run a cron job, you could install some non-Rails software that does log rotation and other log maintenance, you could use Papertrail, if you use Heroku you could look up https://devcenter.heroku.com/articles/scheduled-jobs-custom-clock-processes.
Remember Rails is designed more to handle requests and respond to them in the context of that request, than to run maintenance outside of the context of receiving a request. You could do maintenance as a side-effect of every format_message request to MyLogger, checking for the oldest logging entry and if you find one older than a week, delete them. You haven't given a constraint why you can't do this in-process, and if you're prototyping something early and portable, then this would get you going fast.
I have a scenario outline with multiple scenarios. I'd like my Before hook to run only once so I can bootstrap the ActiveRecord objects I need to run against all of the scenarios. The problem is if I use
Before do
# my code here
end
This will execute before each Scenario. Is there anyway to run it once for the entire Outline?
I think if you simply create the objects in a file in features/support they will be persisted:
ImportantThing.create(:name => "USEFUL THING")
This is because before every Scenario Cucumber will start a database transaction and then rollback to its prior status, which should contain the objects you've loaded.
I had the same problem, where I needed to create a subscriber manager once for all of my event logging tests. If I just used a before hook or a regular step (e.g. a Given), the manager would be created before each scenario.
My solution was ultimately to use a tagged before hook on my first scenario.
Before('#first_logging_scenario') do
# do something useful
end
To shutdown my manager, I used a tagged After hook with my last scenario
After('#last_logging_scenario') do
# do something useful
end