large rails application boot time - ruby-on-rails

I have a very very large rails application and I've read every post there on how to reduce boot time, each suggesting to cut down on models or controllers or gems, but all of them are being used.
Problem i'm facing is that heroku pushes R10 error code since my app takes more than 60s to load.
I've been trying one thing, almost successfully. I'm trying to run Bundler.require and Application.inialize! in threads (latter thread waiting on the first one to finish). Advantage of this being that thin server boots almost instantly.
Problem is, when someone hits the app with a request, the initialization process stays incomplete. Any idea how i can achieve this?

You can take control of the initialization process by extending the initialize! method in config/application.rb.
Look at this sample application.rb file from a github project. You should be able to extend the initialize! method inside of the class Application < Rails::Application scope.
You can take the base methods from the Rails Initialization docs in section 2.3 railties/lib/rails/application.rb. So here is what you'll have if you extend the methods and leave them as they are implemented by default.
class Application < Rails::Application
def initialize!(group=:default) #:nodoc:
raise "Application has been already initialized." if #initialized
run_initializers(group, self)
#initialized = true
self
end
def run_initializers(group=:default, *args)
return if instance_variable_defined?(:#ran)
initializers.tsort_each do |initializer|
initializer.run(*args) if initializer.belongs_to?(group)
end
#ran = true
end
end
At this point you can add print statements and determine which piece you're getting stuck on. For example, is your #initialized variable never being set to true? If so, why? Perhaps you are getting stuck inside of run_intializers. If that is the case, then which initializer are you getting stuck inside of?

I've realized what i'm trying to do is not possible. So i'm ending the thread.
What i'm doing:
In my environment.rb,
def init_bundler
# If you want your assets lazily compiled in production, use this line
s = Time.now
_rails_groups = (ENV["RAILS_GROUPS"]||"").split(",")
if _rails_groups.length == 1 and _rails_groups.first.to_s == "assets"
Bundler.require(:assets) #load ONLY assets ..
elsif _rails_groups.length == 1 and _rails_groups.first.to_s == "clock"
Bundler.require(:clock) #load ONLY assets ..
else
Bundler.require(:default, *_rails_groups, Rails.env.to_s)
end
puts "bundler required in #{Time.now - s}s"
end
puts "starting t1"
$_t1 = Thread.new{ init_bundler }
In my environment.rb:
def init_app
# Initialize the rails application
puts "initialize started"
s = Time.now
Hedgepo::Application.initialize!
puts "initialize done in #{Time.now - s}s"
end
puts "starting t2"
$_t2 = Thread.new do
loop do
if defined?($_t1) and !$_t1.status
init_app
break
else
puts "waiting for bundler"
end
sleep(1)
end
end
the middleware i'm using above everyone else:
class MyInit
def initialize(app)
#app = app
end
def call(env)
loop do
if defined?($_t2) and !$_t2.status
break
else
puts "waiting for app init to finish...."
end
sleep(1)
end
#app.call(env)
end
end
But the problem is that Rack::Server allocates an 'app' variable and .start!s the server when the hit comes. So whatever I do, i cannot change the app that was initialized BEFORE this #app variable was populated.

Related

Code refactoring for environment conditional in Rails application

In my Rails app, I am using sidekiq for job scheduling to lift heavy tasks. So, I have this code in a lot of places:
if Rails.env.development? or Rails.env.test?
#Call the method directly
else #Rails.env.production?
#Call the job via sidekiq that calls the said method
end
Is there any way to clean this out? I am reading software design patterns these days. I am not able to apply that knowledge here. Can you suggest how can this be cleaned up or written in a way that is more manageable?
You can put
require 'sidekiq/testing'
Sidekiq::Testing.inline!
in your development.rb and test.rb config files to get the behaviour you are after. In your applications business logic you would remove the environment conditional and just call the worker (which will now run synchronously in test and development).
How about trying refactoring like following?
module Util
extend self
def execute_method_or_delay_its_execution(obj:, method_name:)
if Rails.env.development? or Rails.env.test?
obj.send(method_name)
else #Rails.env.production?
#Call the job via sidekiq that calls the said method
end
end
end
MyClass1
def my_method
Util.execute_method_or_delay_its_execution(obj: self, method_name: :my_method)
end
end
MyClass2
def my_method
Util.execute_method_or_delay_its_execution(obj: self, method_name: :my_method)
end
end
and then just invoke the methods on object as it should be and the internal delegation should take care of your desired direct execution or delayed execution
mc_1 = MyClass1.new
mc_1.my_method
mc_2 = MyClass2.new
mc_2.my_method
Hope that helps. Thanks.

Rails reload! resets class variables, need to rerun some initializers

Suppose I need to parse some configuration to instanciate some Service Singletons (that could be used with or without Rails).
A sample code example:
#services/my_service.rb
module MyService
#config = nil
def self.load_config(config)
#config = config
end
When using with Rail (or Capistrano, SInatra, etc.) I would use an initializer to boot up the service
#initializers/svc.rb
MyService.load_config(Rails.application.secrets.my_service.credentials)
But when used specifically with Rails, on every rails console restart!, this #config variable is cleared which is a problem...
Are there
after-reload! hooks that I could use to re-run the initializer ?
other types of variables that would be preserved during a restart!
that I could use here ?
You could define config method as:
def config
#config ||= Rails.application.secrets.my_service.credentials
end
And call this method instead of #config, so when the config variable is unset, it will be set again, otherwise it will return the value.
Seeing that people are still reading this, here is implementation I ended up with
# config/initializers/0_service_activation.rb
# Activation done at the end of file
module ServiceActivation
def self.with_reload
ActiveSupport::Reloader.to_prepare do
yield
end
end
module Slack
def self.service
::SlackConnector
end
def self.should_be_activated?
Utility.production? ||
Utility.staging? ||
(
Utility.development? &&
ENV['ENABLE_SLACK'] == 'true'
)
end
def self.activate
slack = service
slack.webhook = Rails.application.secrets.slack&.dig(:slack_webhooks)
Rails.application.secrets&.dig(:slack, :intercept_channel).try do |channel|
slack.intercept_channel = channel if channel.present?
end
slack.activate
slack
end
end
[
ServiceActivation::Intercom,
ServiceActivation::Slack,
ServiceActivation::Google
] .each do |activator|
ServiceActivation.with_reload do
activator.activate if activator.should_be_activated?
activator.service.status_report
end
end
I am not showing my connector class SlackConnector, but basically you can probably guess the interface from the way it is called. You need to set the webhook url, and do other stuff. The implementation is decoupled, so it's possible to use the same SlackConnector in Rails and in Capistrano for deployment, so it's basically in the lib/ folder

Reloading rails middleware without restarting the server in development

I have a rails 4 app with middleware located at lib/some/middleware.rb which is currently injected into the stack though an initializer like so:
MyApp::Application.configure.do |config|
config.middleware.use 'Some::Middleware'
end
Unfortunately, any time I change something I need to restart the server. How can I reload it on each request in development mode? I've seen similar questions about reloading lib code with either autoloading or wrapping code in a to_prepare block but I'm unsure how that could be applied in this scenario.
Thanks,
- FJM
Update #1
If I try to delete the middleware and then re-add it in a to_prepare block I get an error "Can't modify frozen array".
I thought that at some point Rails was smart enough replacing middleware code at runtime, but I may be wrong.
Here is what I came up with, circumventing Ruby class loading craziness and leveraging Rails class reloading.
Add the middleware to the stack:
# config/environments/development.rb
[...]
config.middleware.use "SomeMiddleware", "some_additional_paramter"
Make use of auto-reloading, but make sure that the running rails instance and the already initialized middleware object keep "forgetting" about the actual code that is executed:
# app/middlewares/some_middleware.rb
class SomeMiddleware
def initialize(*args)
#args = args
end
def call(env)
"#{self.class}::Logic".constantize.new(*#args).call(env)
end
class Logic
def initialize(app, additional)
#app = app
#additional = additional
end
def call(env)
[magic]
#app.call(env)
end
end
end
Changes in Logic should be picked up by rails auto reloading on each request.
I think that this actually might become a useful gem!
Building up on #phoet's answer we can actually wrap any middleware with this kind of lazy loading, which I found even more useful:
class ReloadableMiddleware
def initialize(app, middleware_module_name, *middleware_args)
#app = app
#name = middleware_module_name
#args = middleware_args
end
def call(env)
# Lazily initialize the middleware item and call it immediately
#name.constantize.new(#app, *#args).call(env)
end
end
It can be then hooked into the Rails config with any other middleware as its first argument, given as a string:
Rails.application.config.middleware.use ReloadableMiddleware, 'YourMiddleware'
Alternatively - I packaged it into a gem called reloadable_middleware, which can be used like so:
Rails.application.config.middleware.use ReloadableMiddleware.wrap(YourMiddleware)
In Rails 6 with the new default Zeitwork code loader, this works for me:
# at the top of config/application.rb, after Bundler.require
# Load the middleware. It will later be hot-reloaded in config.to_prepare
Dir["./app/middleware/*.rb"].each do |middleware|
load middleware
end
Below it in the section that configures your class Application, add hot-reloading in config.to_prepare:
middleware = "#{Rails.root}/app/middleware"
Rails.autoloaders.main.ignore(middleware)
# Run before every request in development mode, or before the first request in production
config.to_prepare do
Dir.glob("#{middleware}/*.rb").each do |middleware|
load middleware
end
end
Can you not simply use shotgun? If I understand your question you want to ensure the environment reloads on every change you make to your code. That is what shotgun will do.

delayed_job not logging

I cannot log messages from my delayed_job process. Here is the job that is being run.
class MyJob
def initialize(blahblah)
#blahblah = blahblah
#logger = Logger.new(File.join(Rails.root, 'log', 'delayed_job.log'))
end
def perform
#logger.add Logger::INFO, "logging from delayed_job"
#do stuff
end
end
I've tried various logging levels, and I have config.log_level = :debug in my environment configuration. I run delayed_job from monit. I'm using delayed_job 3.0.1 with ruby 1.9.3 and rails 3.0.10.
An explation could be that the job gets initialized only once on producer side. Then it gets serialized, delivered through the queue (database for example) and unserialized in the worker. But the initialize method is not being called in the worker process again. Only the perform method is called via send.
However you can reuse the workers logger to write to the log file:
class MyJob
def perform
say "performing like hell"
end
def say(text)
Delayed::Worker.logger.add(Logger::INFO, text)
end
end
Don't forget to restart the workers.
In RAILS_ROOT/config/initializers have a file called delayed_job_config.rb with these lines:
Delayed::Worker.logger = Rails.logger
Delayed::Worker.logger.auto_flushing = true
Remember to re-start your workers after doing this.
Let me know if this helps
I don't see why you would set the logger in the job. When I've done this I set the worker to to use a specific file on start e.g. Logger.new("log/worker_#{worker_number}") which ensures that each worker outputs to it's own file and you don't have to worry about multiple workers writing to the same file at the same time (messy).
Also, in plain ol' ruby you can call #logger.info "logging from delayed_job".
Finally, i'm pretty sure that 'perform' is called directly by your worker and instantiated, so you can refactor to:
class MyJob
def perform(blahblah)
#logger.add Logger::INFO, "logging from delayed_job"
#blahblah = blahblah
#do stuff
end
end
This is working just fine for me in Rails 3.2:
class FiveMinuteAggregateJob < Struct.new(:link, :timestamp)
def perform
Rails.logger.info 'yup.'
end
end

getting in-out from ruby methods

I am using jruby to run bunch of ruby scripts, though I am using pure ruby part of it.
It sometimes gets difficult to follow from output what exactly is happening or where exactly something went wrong.
I wanted to get something like this in my std out for every method:
entered in method A
out of method A
Now I can surely go and put those comments in every method ..which feels very wrong. Is there a way to run ruby in a little verbose more to get this information in my log. Hopefully I would avoid using a lot of gems etc .. since these are on some managed servers and I will have to spend some time to just get more s/f on it. Hoping something would be avail as part of jruby itself
Thanks!
You could use this code:
module InOutHook
module ClassMethods
def setup_hooks(*syms)
syms.each do |sym| # For each symbol
str_id = "__#{sym}__hooked__"
unless private_instance_methods.include?(str_id)
alias_method str_id, sym # Backup original method
private str_id # Make backup private
define_method sym do |*args| # Replace method
puts ">>> #{self.class}\##{sym} >>>"
ret = __send__ str_id, *args # Invoke backup
puts "<<< #{self.class}\##{sym} >>>"
ret
end
end
end
end
end
def InOutHook.included(base)
base.extend(ClassMethods)
end
end
class TestClass
def test1
puts "test!"
end
def test2(v)
puts "Value is #{v}"
end
include InOutHook
setup_hooks(:test1, :test2)
end
# works on existing classes too:
class Array
include InOutHook
setup_hooks(:[])
end
tc = TestClass.new
tc.test1
tc.test2(10)
ary = [1,2,3]
puts ary[1..2]
In case you want to add a hoot to every method, just add a splat asterisk:
setup_hooks(*[].methods)

Resources