I have a standard app that contains of sub-apps that we will slowly migrate into separate gems. The main problem is that each sub-app basically has an almost identical schema and very similar business logic that sometimes is really pretty much the same.
Currently, as a first step, we created a subfolder in lib/client with a structure like a typical rails app.
As an example, a concern for a ClientA looks like this/
module Client
module ClientA
module Concerns
module MyConcern
...
end
end
end
This all works fine and gets autoloaded by Zeitwerk.
However, when I want to create an initializer in a Rails way, I don't want to wrap it inside modules and make a class around it and this is where I am getting lib/clients/ClientA/config/initializers/custom_initializer.rb to define constant Clients::ClientA::Config::Initializers::CustomInitializer, but didn't (Zeitwerk::NameError)
This folder gets autoloaded like this
config.autoload_paths += %W(#{config.root}/lib)
config.autoload_paths += %W(#{config.root}/lib/client/**/*)
Is there a way how to
Blacklist the folder from being autoloaded by Zeitwerk?
Load it using require or in any other way without wrapping it in modules?
You can ignore parts of the project in zeitwerk using Loader#ignore. You can access rails zeitwerk autoloader via Rails.autoloaders.main.
So you should be able to add something like this to your application.rb:
Rails.autoloaders.main.ignore(Rails.root.join('lib/clients/*/config/initializers/*.rb'))
You'll then have to require them manually, maybe in an initializer in the main app directory. You could create a config/initializers/clients.rb with something like:
Dir[Rails.root.join('lib/clients/*/config/initializers/*.rb')].each do |filename|
require filename
end
Related
I'm having some trouble switching from the classic autoloader to Zeitwerk with a Rails app that's seen the light of day back in Rails 3 days – so there's some crust there.
Some model code has been extracted to modules and these modules are nested in the model class (which acts as the namespace):
# app/models/donation
class Donation < ApplicationRecord
(...)
end
# app/models/donation/download
class Donation
module Download
def csv
(...)
end
end
end
The modules are then used on the fly when needed:
donation = Donation.find(...)
donation.extend(Donation::Download).csv
Since the subdirs in app/models are not added by default, it's done explicitly in application.rb:
Dir[
"#{config.root}/app/models/*/"
].then do |paths|
config.autoload_paths += paths
config.eager_load_paths += paths
end
The eager_load_paths are required by Zeitwerk (as per the Rails guides), however, Zeitwerk doesn't seem to like this constellation:
% rails zeitwerk:check
Hold on, I am eager loading the application.
expected file app/models/donation/download.rb to define constant Download
Strange, because Download is defined there. Any idea what's going on here and how best to refactor things to work with Zeitwerk?
Thanks for your hints!
Hmmm, that should work out of the box, looks like a regular setup to me.
Since app/models is in the autoload paths, Donation and Donation::Download are going to be autoloaded just fine, no custom configuration is needed.
If they do not, the app has to be doing something funky. We could debug it.
I've multiple issues to load / require classes under my app/services folder in a Rails 5 project and I'm starting to give up on this issue.
First of all and to be clear, services/ are simple PORO classes I use throughout my project to abstract most of the business logic from the controllers, models, etc.
The tree looks like this
app/
services/
my_service/
base.rb
funny_name.rb
my_service.rb
models/
funny_name.rb
Failure #1
First, when I tried to use MyService.const_get('FunnyName') it got FunnyName from my models directory. It does not seem to have the same behavior when I do MyService::FunnyName directly though, in most of my tests and changes this was working fine, it's odd.
I realised Rails config.autoload_paths does not load things recursively ; it would makes sense that the first FunnyName to be catch is the models/funny_name.rb because it's definitely loaded but not the other.
That's ok, let's find a workaround. I added this to my application.rb :
config.autoload_paths += Dir[Rails.root.join('app', 'services', '**/')]
Which will add all the subdirectories of services into config.autoload_paths. Apparently it's not recommended to write things like that since Rails 5 ; but the idea does look right to me.
Failure #2
Now, when I start my application it crashes and output something like this
Unable to autoload constant Base, expected
/.../backend/app/services/my_service/base.rb to define it (LoadError)
Names were changed but it's the matching path from the tree I wrote previously
The thing is, base.rb is defined in the exact file the error leads me, which contains something like
class MyService
class Base
end
end
Poor solution
So I try other workaround, lots of them, nothing ever works. So I end up totally removing the autoload_paths and add this directly in the application.rb
Dir[Rails.root.join('app', 'services', '**', '*.rb')].each { |file| require file }
Now the base.rb is correctly loaded, the MyService.const_get('FunnyName') will actually return the correct class and everything works, but it's a disgusting workaround. Also, it has yet not been tested in production but it might create problems depending the environment.
Requiring the whole tree from the application.rb sounds like a bad idea and I don't think it can be kept this way.
What's the cleanest way to add custom services/ directory in Rails ? It contains multiple subdirectories and classes with simple names which are also present in other parts of the app (models, base.rb, etc.)
How do you avoid confusing the autoload_paths ? Is there something else I don't know which could do the trick ? Why did base.rb even crash here ?
Working solution
After deeper investigation and attempts, I realised that I had to eager_load the services to avoid getting wrong constants when calling meta functionalities such as const_get('MyClassWithModelName').
But here's is the thing : the classic eager_load_paths won't work because for some reason those classes will apparently be loaded before the entire core of Rails is initialized, and simple class names such as Base will actually be mixed up with the core, therefore make everything crash.
Some could say "then rename Base into something else" but should I change a class name wrapped into a namespace because Rails tell me to ? I don't think so. Class names should be kept simple, and what I do inside a custom namespace is no concern of Rails.
I had to think it through and write down my own hook of Rails configuration. We load the core and all its functionalities and then service/ recursively.
On a side note, it won't add any weight to the production environment, and it's very convenient for development.
Code to add
Place this in config/environment/development.rb and all other environment you want to eager load without Rails class conflicts (such as test.rb in my case)
# we eager load all services and subdirectories after Rails itself has been initializer
# why not use `eager_load_paths` or `autoload_paths` ? it makes conflict with the Rails core classes
# here we do eager them the same way but afterwards so it never crashes or has conflicts.
# see `initializers/after_eager_load_paths.rb` for more details
config.after_eager_load_paths = Dir[Rails.root.join('app', 'services', '**/')]
Then create a new file initializers/after_eager_load_paths.rb containing this
# this is a customized eager load system
# after Rails has been initialized and if the `after_eager_load_paths` contains something
# we will go through the directories recursively and eager load all ruby files
# this is to avoid constant mismatch on startup with `autoload_paths` or `eager_load_paths`
# it also prevent any autoload failure dû to deep recursive folders with subclasses
# which have similar name to top level constants.
Rails.application.configure do
if config.respond_to?(:after_eager_load_paths) && config.after_eager_load_paths.instance_of?(Array)
config.after_initialize do
config.after_eager_load_paths.each do |path|
Dir["#{path}/*.rb"].each { |file| require file }
end
end
end
end
Works like a charm. You can also change require by load if you need it.
When I do this (which is in all of my projects), it looks something like this:
app
|- services
| |- sub_service
| | |- service_base.rb
| | |- useful_service.rb
| |- service_base.rb
I put all common method definitions in app/services/service_base.rb:
app/services/service_base.rb
class ServiceBase
attr_accessor *%w(
args
).freeze
class < self
def call(args={})
new(args).call
end
end
def initialize(args)
#args = args
end
end
I put any methods common to the sub_services in app/services/sub_service/service_base.rb:
app/services/sub_service/service_base.rb
class SubService::ServiceBase < ServiceBase
def call
end
private
def a_subservice_method
end
end
And then any unique methods in useful_service:
app/services/sub_service/useful_service.rb
class SubService::UsefulService < SubService::ServiceBase
def call
a_subservice_method
a_useful_service_method
end
private
def a_useful_service_method
end
end
Then, I can do something like:
SubService::UsefulService.call(some: :args)
With your tree,
app/
services/
my_class/
base.rb
funny_name.rb
my_class.rb
models/
funny_name.rb
services/my_class/base.rb should look similar to:
module MyClass
class Base
services/my_class/funny_name.rb should look similar to:
module MyClass
class FunnyName
services/my_class.rb should look similar to:
class MyClass
models/funny_name.rb should look similar to:
class FunnyName
I say "should look similar to" because class/module are interchangable; Rails is merely looking for these constants to be defined in these locations.
You don't need to add anything to your autoload path. Rails automatically picks up everything in app
Anecdotal: With your services directory, it's fairly common to treat their naming convention (both name of file and underlying constant) to be "_service.rb" or "ThingService" — just like how controllers look. Models don't get this suffix because they're treated as first-class objects.
GitLab has some great file structure that is very worth a look at. https://gitlab.com/gitlab-org/gitlab-ce
Usually I add the following in config/application.rb to add autload_paths:
config.autoload_paths += Dir[Rails.root.join('app', 'poros', '{**}')]
How can I achieve the same in an engine?
It seems to work when I just use the same code in application.rb in the host app, however I think it's ugly that the code is not in the engine and needs to be added to the host app to make things work.
The only solution I found to add the load path through the engine is by adding this to lib/engine/engine.rb:
config.to_prepare do
Dir.glob(Rails.root + "../../app/poros/**/*.rb").each do |c|
require_dependency(c)
end
end
However there seems to be something fundamentally wrong with this as this leads to problems when I'm doing console reloads (e.g. it tells me that constants are already defined or that concerns can't execute the include block twice)
What is the right way to do this in the engine itself? (can't believe this is so hard/uncommon, I have really googled a lot but I can't find a solution)
According to the Rails::Engine documentation, you can add autoload paths in your Railtie like this:
class MyEngine < Rails::Engine
# Add a load path for this specific Engine
config.autoload_paths << File.expand_path("../lib/some/path", __FILE__)
initializer "my_engine.add_middleware" do |app|
app.middleware.use MyEngine::Middleware
end
end
If poros is a subdirectory of app, you do not need add it again.
All subdirectories of app in the application and engines present at boot time. For example, app/controllers. They do not need to be the default ones, any custom directories like app/workers belong automatically to autoload_paths.
I use the following line in an initializer to autoload code in my /lib directory during development:
config/initializers/custom.rb:
RELOAD_LIBS = Dir[Rails.root + 'lib/**/*.rb'] if Rails.env.development?
(from Rails 3 Quicktip: Auto reload lib folders in development mode)
It works great, but it's too inefficient to use in production- Instead of loading libs on each request, I just want to load them on start up. The same blog has another article describing how to do this:
config/application.rb:
# Custom directories with classes and modules you want to be autoloadable.
config.autoload_paths += %W(#{config.root}/lib)
config.autoload_paths += Dir["#{config.root}/lib/**/"]
However, when I switch to that, even in development, I get NoMethodErrors when trying to use the lib functions.
Example of one of my lib files:
lib/extensions.rb:
Time.class_eval do
def self.milli_stamp
Time.now.strftime('%Y%m%d%H%M%S%L').to_i
end
end
Calling Time.milli_stamp will throw NoMethodError
I realize others have answered similar questions on SO but they all seem to deal with naming conventions and other issues that I didn't to have to worry about before- My lib classes already worked for per-request loading, I just want to change it to per-startup loading. What's the right way to do this?
I think this may solve your problem:
in config/application.rb:
config.autoload_paths << Rails.root.join('lib')
and keep the right naming convention in lib.
in lib/foo.rb:
class Foo
end
in lib/foo/bar.rb:
class Foo::Bar
end
if you really wanna do some monkey patches in file like lib/extensions.rb, you may manually require it:
in config/initializers/require.rb:
require "#{Rails.root}/lib/extensions"
P.S.
Rails 3 Autoload Modules/Classes by Bill Harding.
And to understand what does Rails exactly do about auto-loading?
read Rails autoloading — how it works, and when it doesn't by Simon Coffey.
Though this does not directly answer the question, but I think it is a good alternative to avoid the question altogether.
To avoid all the autoload_paths or eager_load_paths hassle, create a "lib" or a "misc" directory under "app" directory. Place codes as you would normally do in there, and Rails will load files just like how it will load (and reload) model files.
This might help someone like me that finds this answer when searching for solutions to how Rails handles the class loading ... I found that I had to define a module whose name matched my filename appropriately, rather than just defining a class:
In file lib/development_mail_interceptor.rb (Yes, I'm using code from a Railscast :))
module DevelopmentMailInterceptor
class DevelopmentMailInterceptor
def self.delivering_email(message)
message.subject = "intercepted for: #{message.to} #{message.subject}"
message.to = "myemail#mydomain.org"
end
end
end
works, but it doesn't load if I hadn't put the class inside a module.
Use config.to_prepare to load you monkey patches/extensions for every request in development mode.
config.to_prepare do |action_dispatcher|
# More importantly, will run upon every request in development, but only once (during boot-up) in production and test.
Rails.logger.info "\n--- Loading extensions for #{self.class} "
Dir.glob("#{Rails.root}/lib/extensions/**/*.rb").sort.each do |entry|
Rails.logger.info "Loading extension(s): #{entry}"
require_dependency "#{entry}"
end
Rails.logger.info "--- Loaded extensions for #{self.class}\n"
end
I place a file name g.rb in side Rails.root/lib folder
The file content is like this:
module Google
end
Then I add
config.autoload_paths += %W(#{config.root}/lib #{Rails.root}/app/delayed_jobs)
to my Rails.root/config/application.rb
However, when I try to invoke Google from rails console, an exception is thrown. The exception goes away only if I execute require 'google'.
Why? Shouldn't my file is autoloaded and shouldn't I access the module without any extra require statement?
Hmm, I discovered an interesting thing. In order for Rails to auto load my class, the class name should be compliant to the file name and the folder structure.
For example, if I want to have Google module autoloaded, I must placed it inside google.rb, directly under /lib (incase I specify autoload from /lib).
If I want to auto load Google::Docs, then I either place it inside google.rb or google/docs.rb
I had a similar problem with getting my module to run on Heroku. In addition to the autoload naming convention stated by Stephen C, I found out that the module code must be require'd due to a threadsafe assumption made by the Rails' production environment on Heroku (even though threadsafe was commented out in my production.rb configuration file.) As soon as I require'd the module file before calling include on the module, everything started to work.
require 'mymodule'
include Mymodule
Please take a look at this excellent article on the subject of getting Modules to load correctly in Heroku (production).
That's because the point of autoload is not to 'require' everything up front (startup penalty). Classes are loaded as they are needed/referenced. In order to do this, you need some way to know where to look for the class. Otherwise, you would have to load every file in the autoload directory in advance to see what classes are declared. It's a tradeoff, but requiring everything in advance (as marbaq suggests) is not autoloading.
You can use the autoload command as provided by Ruby, which takes two arguments, the module to load (symbolized, i.e. :Google in your case), and the second argument is the filename, which would be g.rb if lib is in your load path ($:). See the Ruby docs for autoload.
Change config.autoload_paths to config.eager_load_paths
(based on Rails issue #6850 and Force reload! from lib directory in rails 3.2 console)
I faced the same problem just now, and my "solution" (or rather workaround) was to manually require every needed file from Rails.root/lib in my application.rb.
require 'lib/message'
require 'lib/store'
require 'lib/vault/vault.rb'
require 'lib/custom_loggers'
module MyApplication
class Application < Rails::Application
My next step would be to categorize the files in module folders as you mention.
i found this solution recently
config/application.rb
module AppName
class Application < Rails::Application
# Custom directories with classes and modules you want to be autoloadable.
config.autoload_paths += Dir[Rails.root.join('app', 'models', '{**}')]
config.autoload_paths += Dir[Rails.root.join('app', 'lib', 'extensions')]
end
end
the first config call induces rails to auto-load all sub-directories of the app/models directory
so now i can have /app/models/sub_directory/model.rb auto-loaded
(handy for organising an app with a large code base)
the second config call induces rails to autoload the lib/extensions directory
hope this helps
note: i believe this is rails 3 specific