Background:
I am getting some reloading errors in development when switching to Zeitwerk for a rails 6.0 application that uses an engine (thredded). I'm also a developer on thredded, so want to understand this fully before committing any apparent fixes:
I've already read:
https://guides.rubyonrails.org/autoloading_and_reloading_constants
https://guides.rubyonrails.org/engines.html#overriding-models-and-controllers (and implemented these fixes in the main app)
https://github.com/fxn/zeitwerk/blob/main/README.md (amazingly informative)
https://github.com/fxn/zeitwerk/issues/143
First Question (separation of autoloading and engines): Are engines autoloaded in the same zeitwerk instance/loader as the main app, or are they somehow loaded separately? That is to say does wrapping some code with Rails.application.reloader.to_prepare do ensure that code is run before both the main app and the engine are reloaded.
Second Question (engine code reloading): Are engine's constants reloaded when the main app reloads? (my understanding is yes).
Third Question (configuring engines): Currently Thredded's docs suggest that the configuration of Thredded happens in an initializer -- e.g. Thredded.some_configuration_option = value - but I think that would get wiped away with autoloading? So therefore probably needs to be wrapped with (I think) Rails.application.reloader.to_prepare do (but this isn't what e.g. Devise recommends (see https://github.com/heartcombo/devise/blob/main/lib/generators/templates/devise.rb) and seems to conflict with https://github.com/fxn/zeitwerk/issues/143). What have I misunderstood here?
Fourth Question (all these to_prepares): Can someone explain or point me to docs that clarify the lifecycle difference between:
Rails.application.reloader.to_prepare do (and is the block run at least once even when it is eager loaded in production with reloading on)
Rails.application.config.to_prepare do
SomeEngine::Engine.config.to_prepare do
Any answers would be great. Kind of a long Q with multiple parts. Happy to split this into multiple StackOverflow Qs if appropriate but they seem quite linked.
Answering my question for the future based on the discussion in comments above and the discussion in https://github.com/fxn/zeitwerk/issues/240.
(1) Yes, the main autoloader manages both the application and all engines.
(2) Yes, engine's constants are reloaded when the main app reloads.
(3) By default only what is in the engine's app directory is autoloaded (unless config.autoload_paths is changed in for example the engines engine.rb) and the engine's top-level constant (in this case the Thredded module) is typically defined outside of app, in lib (e.g. lib/thredded.rb).
(4) approximately definitive answers (any updates appreciated)
Rails.application.reloader.to_prepare is run when the app boots (even when eager loaded) and each time it is reloaded (in development) -- see https://guides.rubyonrails.org/v6.1.0/autoloading_and_reloading_constants.html#autoloading-when-the-application-boots
Rails.application.config.to_prepare and SomeEngine::Engine.config.to_prepare are run on boot, and after each reload see https://guides.rubyonrails.org/autoloading_and_reloading_constants.html#use-case-1-during-boot-load-reloadable-code (and I assume after reloader.to_prepare - the relative order of application.config.to_prepare and ...Engine.config.to_prepare will depend on their relative load orders (see https://api.rubyonrails.org/classes/Rails/Engine.html#class-Rails::Engine-label-Loading+priority HT https://stackoverflow.com/a/73982363/109175)
Additionally I discovered you can define initializers that your engine needs (and will be run on boot time only before autoloading) with
initializer "some explanation of what is for" do
# code that occurs on initialization
end
(See https://guides.rubyonrails.org/engines.html#separate-assets-and-precompiling for an example of this initializer method)
I found this is particularly useful for defining "overrides" when one engine wants to monkey-patch another engine:
initializer "let the main autoloader ignore this engine's overrides" do
overrides = root.join("app/overrides")
Rails.autoloaders.main.ignore(overrides)
end
This is parallel to what you do in the main app when monkey-patching autoloaded constants in an engine (https://guides.rubyonrails.org/engines.html#overriding-models-and-controllers).
Related
I have a rails which seemed to be working previously but now after some changes when I go to the root page it takes it infinitely to load it, it just doesn't load it. There're nothing useful in the console either. How can I find out what prevents it from loading the main page? Is it about profiling?
Check your Rails logs, eg. development.rb. You can put logger.info, or puts statements in your environment.rb, development.rb and application.rb files to see how far Rails is getting in the boot process. You can also create a dumb initializer named 00_start_init.rb with a logger.info or puts statement to see if you're getting as far as initialization. I've found that useful before.
To really understand where you application is hanging, you need to understand the Rails initialization process. Here is the documentation for Rails version 4.2. http://guides.rubyonrails.org/v4.2/initialization.html. Similar documentation exists for every version of Rails. You can take advantage of understanding the boot sequence by placing log statement at various point in the process.
I'm assuming you're in the development environment. If so, and the console loads, it's likely not a configuration problem. It's more likely a problem with your controllers or models. If the console won't load to a prompt, then it's likely a configuration problem in application.rb, development.rb, an initializer, etc.
You mention profiling, but provide no details about it. I can't even guess what you're referring to, so the answer is "maybe?". If you can post the code changes you made since the app last loaded in the browser, that would make it much easier to help you trouble-shoot.
I have an app with a rails engine i use. I've stripped them down pretty much to the bone, but i get a fault when I start the engine up with multi-threading enabled. I faults out with missing constant the model name i'm calling from the controller. I've reduced it down to just calling the Model.all.pluck(:id). And still get the fault. The models are being invoked via an Ajax call to the controller.
The app runs correctly if i refresh the page and continue, never faulting out again, that I can see.
I think it's actually a multi-treading issue, I'm using Rails 4.1, Ruby 2.12 and Puma 2.8.2 and postgress.
This is happening on my development system a macbook. I will don't see it in the heroku logs when I deploy, but ??
Now somewhat curious. I've made the issue go away, by forcing the app to load each model in the engine, one at a time. Things seem to run smoothly after that.
Any suggestions on how to dubug this
This is directly linked to config.eager_load settings of your app. In production it is set to true so you do not have the circular dependency problem, in development it is generally set to false and that's where rails auto loading systems may trigger this kind of error.
I do believe (though I'm not certain) that you can solve this by not declaring nested classes and modules in one single line
Instead of
class MyModule::MyClass
...
end
Use
module MyModule
class MyClass
...
end
end
I read a post about the rails load_paths, here is the link.
But, I am still confused about the difference between the autoload_paths and eager_load_paths:
I have tested them in a newly created Rails 4 project. It seems that they run the same way, that auto-reload in the development mode but in the production mode.
Author of the linked article here. Here's an attempt to clear up the confusion, going off of #fkreusch's answer.
In Ruby you have to require every .rb file in order to have its code run. However, notice how in Rails you never specifically require any of your models, controllers, or other files in the app/ dir. Why is that? That's because in Rails app/* is in autoload_paths. This means that when you run your rails app in development (for example via rails console) — none of the models and controllers are actually required by ruby yet. Rails uses special magical feature of ruby to actually wait until the code mentions a constant, say Book, and only then it would run require 'book' which it finds in one of the autoload_paths. This gives you faster console and server startup in development, because nothing gets required when you start it, only when code actually needs it.
Now, this behavior is good for local development, but what about production? Imagine that in production your server does the same type of magical constant loading (autoloading). It's not the end of the world really, you start your server in production, and people start browsing your pages slightly slower, because some of the files will need to be autoloaded. Yes, it's slower for those few initial requests, while the server "warms up", but it's not that bad. Except, that's not the end of the story.
If you are running on ruby 1.9.x (if I recall correctly), then auto-requiring files like that is not thread safe. So if you are using a server like puma, you will run into problems. Even if you aren't using a multi-threaded server, you are still probably better off having your whole application get required "proactively", on startup. This means that in production, you want every model, every controller, etc all fully required as you start your app, and you don't mind the longer startup time. This is called eager loading. All ruby files get eagerly loaded, get it? But how can you do that, if your rails app doesn't have a single require statement? That's where eager_load_paths come in. Whatever you put in them, all the files in all the directories underneath those paths will be required at startup in production. Hope this clears it up.
It's important to note that eager_load_paths are not active in development environment, so whatever you put in them will not be eagerly required immediately in development, only in production.
It's also important to note that just putting something into autoload_paths will not make it eager-loaded in production. Unfortunately. You have to explicitly put it into eager_load_paths as well.
Another interesting quirk is that in every rails app, all directories under app/ are automatically in both autoload_paths and eager_load_paths, meaning that adding a directory there requires no further actions.
Basically, autoload_paths are paths Rails will use to try loading your classes automatically. E.g. when you call Book, if that class isn't loaded yet, it will go through the autoload_paths and look for it in those paths.
In production, it might be better to load those upfront to avoid autoload concurrent issues. For that, it provides the eager_load_paths. Paths in that list will be required upfront when your application starts.
I am a former PHP developer learning Rails and Sinatra. In PHP, every page request loaded all of the required files. If I changed some code and refreshed the page, I could be sure that the code was fresh.
In Rails 3, Controller code is fresh on every request. However, if I modify any code in the /lib folder, I need to restart the server so the changes take effect.
Why does this happen? Is it something to do with the way Ruby is designed? Is Rails doing some optimizations to avoid reloading code on every request?
Thanks!
Edit: I am mostly interested in what is going on under the hood. Do frameworks like Rails and Sinatra do some special caching for classes? If so, what does they do? Is the default behavior in Ruby that all code gets reloaded on every request? Why do we need tools like Shotgun for Sinatra (http://sinatra-book.gittr.com/#automatic_code_reloading)?
While you are in development mode you should tell Rails not to cache your classes so they reload each time. This means that each request the classes are basically redefined in the rails interpreter. The setting in your Rails.root/config/environments/development.rb:
config.cache_classes = false
The classes the are in your lib/ dir are usually loaded through an initializer and not subject to this setting.
When you move to production you will want all of your classes to be cached so requests are faster and rails will do optimizations to things like scopes on your models.
You could put something in another initializer (maybe called Rails.root/config/initializers/development_reload.rb) that reloads the lib dir with every request in development (or just the ones you are working on):
# file development_reload.rb
if Rails.env.development?
ActionDispatch::Callbacks.after do
load 'filename_in_lib'
# or
Dir.entries("#{Rails.root}/lib").each do |entry|
load entry if entry =~ /.rb$/
end
end
end
I am calling "load" so it actually reloads the file, whereas "require" would just check if it has been loaded and determine it already has so it will not reload it. (I just threw this together and don't use it, but Ruby is extremely flexible and will allow you to do quite a bit.) Use something like this wisely and only in a dev environment.
Why code needs to be reloaded in Rails 3?
Ruby is an interpreted language (JRuby has some support for precompilation, but it's still interpreted). Interpreting the the definition of classes once on initialization is similar to compiling php and deploying in executable format (somewhat). The interpreter is not bothered with redefining classes all the time.
Forcing the explicit reload is an optimization for this type of interpreted language. (if you AOT compile in php you would need to reload the compiled "bytecode" after changes as well; default php uses on-the-fly compilation which is what you are taking advantage of)
How about a more high level approach:
ActionDispatch::Reloader.cleanup!
ActionDispatch::Reloader.prepare!
This was taken from Rails/ActiveRecord v3.2.13 - active_record/railtie.rb
The load approach didn't work for me. Just performing load caused a weird issue where it would trigger certain validators twice for me.
In order to fix that, I tried Object.send(:remove_const, User) before reloading User, but then I lost my observers on that class, so I started chasing my tail.
The above approach reloads all the classes, so maybe there is still yet a better approach to properly remove an individual class from cache and reload it...
I'm developing a ruby module that I include in my rails app. I want it to be reloaded automatically when it changes. I've done extensive googling, and looked at the various questions here that discuss it, but they all seem out of date or wrong.
How do I get an external module to be reloaded in rails when it changes? I've tried adding its name to ActiveSupport::Dependencies.unloadable_constants, but after I type reload! in the console, I can no longer refer to that symbol without a NameError: uninitialized constant foo, even if I do another require 'foo_module'. Does anyone know how to get this working?
Note: here is one possible dup, but note in the comments to the 'answer' that it never solved the problem for modules. There's also this question which has a dead link in the answer, and finally this one, which also doesn't solve it.
I found how to do this:
Make sure FooModule is in lib/foo_module.rb.
Use require_dependency to require your external library in lib/foo_module.rb.
These steps are both required, and no others are required.
There are two separate problems here.
The simpler of which is that you are using require, when you want load.
require will evaluate the code in a file once, no matter how many times that file/module is required.
load will evaluate the code in a file each time that file is loaded.
require is preferred to load used so that the files are not evaluated multiple times when many files depend on them.
The short version is that load can be used to reload modules that have already been loaded by require.
The more complicated problem is automatically reloading a module on change.
One of the possible duplicates listed in the question links to here. Which suggests prefixing any code that depends on your module with a conditional load of your module if it has changed since loading. You'll need to use a global variable to keep track of when a file was loaded.
N.B.: this should not be used on a production server, but should be fine on a development server or console.
I spent sometimes to research this issue as well.
Here's my findings on how to auto-reload require files in Rails without restarting server.
The solution is now available as a Ruby gem require_reloader.