I'm creating a rails engine, which I have written some rake tasks in. My rake tasks call rake tasks in a 3rd-party gem, which I've included as a dependency in my gemspec.
When I try to run my rake tasks, it fails stating it cannot find the tasks from the 3rd party gem. Which makes senses considering Rake -T shows my tasks, but not the 3rd party's.
If I include the 3rd party gem in my application gemfile, everything is perfect, but I don't want to do this.
I'm guessing I need to add another include to my engine's rake_tasks do block, but no idea what to include. I've tried a variety of paths, but none seem to work.
# lib/my_gem.rb
module MyGem
ROOT_PATH = File.expand_path "../../", __FILE__
# :: before of the module name it is not to encapsulate the module "My Gem"
module ::Rails
class Application
rake_tasks do
Dir[File.join(ROOT_PATH, "/lib/tasks/", "**/*.rake")].each do |file|
load file
end
end
end
end
end
Sounds like your third party gem may not have a railtie set up. The rake tasks from gems aren't automatically included within a Rails application. Here are two good resources which help explain how to set up a Rail tie, from which you can probably base some code to manually include the third party gem's rake tasks:
including rake tasks in gems
http://blog.nathanhumbert.com/2010/02/rails-3-loading-rake-tasks-from-gem.html
Related
I'm making a gem that executes Rails commands (rails g model Item for example). When I use it in a Rails project, everything works. The problem is testing it in development outside of a Rails project.
I'm using cucumber with aruba to test if CLI commands execute the proper rails commands and generate the expected files. Unfortunately, when I try to test the behaviour it fails because there are no rails files and the commands require to be run inside of a Rails project in order to work.
I have added a rails dependency to the gemspec:
Gem::Specification.new do |spec|
spec.add_development_dependency 'rails', '~> 5.2.4'
end
I've thought about creating a new rails project on test start and then deleting it after the tests run, but that seems highly inconvenient. Is there a better way to do this?
A technique we use for WickedPDF is in the default rake task, before we run the tests, is to delete & generate a full Rails application in a gitignored subdirectory of the gem.
As a high-level simplified example of this Rakefile, it looks something like this:
Rakefile
require 'rake'
require 'rake/testtask'
# This gets run when you run `bin/rake` or `bundle exec rake` without specifying a task.
task :default => [:generate_dummy_rails_app, :test]
desc 'generate a rails app inside the test directory to get access to it'
task :generate_dummy_rails_app do
if File.exist?('test/dummy/config/environment.rb')
FileUtils.rm_r Dir.glob('test/dummy/')
end
system('rails new test/dummy --database=sqlite3')
system('touch test/dummy/db/schema.rb')
FileUtils.cp 'test/fixtures/database.yml', 'test/dummy/config/'
FileUtils.rm_r Dir.glob('test/dummy/test/*') # clobber existing tests
end
desc 'run tests in the test directory, which includes the generated rails app'
Rake::TestTask.new(:test) do |t|
t.libs << 'lib'
t.libs << 'test'
t.pattern = 'test/**/*_test.rb'
t.verbose = true
end
Then, in test/test_helper.rb, we require the generated Rails app, which loads Rails itself and it's environment:
test/test_helper.rb
ENV['RAILS_ENV'] = 'test'
require File.expand_path('../dummy/config/environment.rb', __FILE__)
require 'test/unit' # or possibly rspec/minispec
# Tests can go here, or other test files can require this file to have the Rails environment available to them.
# Some tests may need to copy assets/fixtures/controllers into the dummy app before being run. That can happen here, or in your test setup.
You could skip parts of Rails that aren't needed by customizing the command that generates the app. For example, your gem may not need a database at all or a lot of things by default, so you command could be customized for a simpler app. Something like this maybe:
system("rails new test/dummy --skip-active-record \
--skip-active-storage --skip-action-cable --skip-webpack-install \
--skip-git --skip-sprockets --skip-javascript --skip-turbolinks")
In the WickedPDF project, we wanted to test across a wide range of "default" Rails installs, so we don't customize the command much, but that may generate much more than what you need to test some generator tasks.
WickedPDF also tests against multiple versions of Rails with TravisCI and multiple Gemfiles, but this could also be accomplished with the Appraisal gem that Luke suggested in this thread.
Check out Thoughbot's Appraisal gem:
Appraisal integrates with bundler and rake to test your library against different versions of dependencies in repeatable scenarios called "appraisals."
Here is a guide on how to set it up, including setting up a micro Rails app within your tests dir.
We have a gem which runs via a Rake task. The task is defined in lib/tasks/<namespace>.rake. After reading Rake tasks inside gem not being found I confirmed that the .gemspec includes the file defining the task; there is also a railtie which should be including the tasks as suggested in including rake tasks in gems. And yet our Rails 4.1 application doesn't seem to load the Rake task.
What am I missing?
I just successfully tested your gem and I can see no problem with the rake tasks in it:
I added gem_fresh to the Gemfile and ran bundle, the gem installed
Immediately I could see the rake present in the list of rakes:
$ rake -T outdated
rake gem_fresh:outdated # outdated
Then I updated the railtie.rb file to use the load method to load the rake defined in lib/task/metrics.rake and searched the available rakes again:
# lib/gem_fresh/railtie.rb:
rake_tasks do
namespace :gem_fresh do
desc "outdated"
task :outdated => :environment do
GemFresh::Reporter.new.report
end
end
load "tasks/metrics.rake"
end
$ rake -T outdated
rake gem_fresh:outdated # outdated
rake metrics:outdated_gems # display outdated gem version metrics
So, I can see no problem with your gem. Both methods in railtie (inline rake as well as using the load method) seem to work OK. The only difference I noticed is that I tested this on rails 4.2 but I rather doubt that would make a difference.
Did you put the gem into your Gemfile? If I remove it from there, I indeed see no gem_fresh rakes defined.
The problem was not with the gem, but with the way it was included in the app.
In the Gemfile, this works and includes the rake task:
gem 'gem_fresh'
This works but doesn't include the rake task:
group :development do
gem 'gem_fresh'
end
This seems to be due to how Bundler requires gems in Rails apps. From the documentation:
By default, a Rails generated app calls Bundler.require(:default, Rails.env) in your application.rb, which links the groups in your Gemfile to the Rails environment.
If for some reason that Rails.env argument wasn't evaluating to include the :development group, which seems to be the case when I call rake -T, the gem wouldn't be Bundler.require-d, and the rake tasks wouldn't be loaded.
The fact that it isn't including the :development group seems to be an odd Bundler "gotcha", but at least now I know that moving it to the default group solves the issue and it's not a problem with the gem.
in my case, the project requiring the gem had a rake file, and the gem I was requiring had the same name when I changed the name of the rake file I could see the tasks in the project.
my_gem had lib/tasks/XXXX.rake and my_proj also had lib/tasks/XXXX.rake,
after I changed my_gem XXXX.rake to YYYY.rake I managed to list and use the tasks in YYYY.rake
I am developing a Rails app where most of the code not specific to the app has been written inside of various gems, including some Rails engines and some 3rd party gems for which I am enhancing or fixing bugs.
gem 'mygem', path: File.expath_path('../../mygem', __FILE__)
Since a lot of the code in these gems is really part of the app, it's still changing frequently. I'd like to be able to utilize the Rails feature where code is reloaded on each request when in development (i.e. when config.cache_classes is false), but this is only done within the normal application structure by default.
How can I configure Rails to reload gem code on each request, just like with the app code?
I have found through trial and error that several steps are required, with the help of ActiveSupport.
Add activesupport as a dependency in the .gemspec files
spec.add_dependency 'activesupport'
Include ActiveSupport::Dependencies in the top-level module of your gem (this was the most elusive requirement)
require 'bundler'; Bundler.setup
require 'active_support/dependencies'
module MyGem
unloadable
include ActiveSupport::Dependencies
end
require 'my_gem/version.rb'
# etc...
Set up your gem to use autoloading. You an either manually use ruby autoload declarations to map symbols into filenames, or use the Rails-style folder-structure-to-module-hierarchy rules (see ActiveSupport #constantize)
In each module and class in your gem, add unloadable.
module MyModule
unloadable
end
In each file that depends on a module or class from the gem, including in the gem itself, declare them at the top of each file using require_dependency. Look up the path of the gem as necessary to properly resolve the paths.
require_dependency "#{Gem.loaded_specs['my_gem'].full_gem_path}/lib/my_gem/myclass"
If you get exceptions after modifying a file and making a request, check that you haven't missed a dependency.
For some interesting details see this comprehensive post on Rails (and ruby) autoloading.
The solution that I used for Rails 6, with a dedicated Zeitwerk class loader and file checker :
Add the gem to the Rails project using the path: option in Gemfile
gem 'mygem', path: 'TODO' # The root directory of the local gem
In the development.rb, setup the classloader and the file watcher
gem_path = 'TODO' # The root directory of the local gem, the same used in Gemfile
# Create a Zeitwerk class loader for each gem
gem_lib_path = gem_path.join('lib').join(gem_path.basename)
gem_loader = Zeitwerk::Registry.loader_for_gem(gem_lib_path)
gem_loader.enable_reloading
gem_loader.setup
# Create a file watcher that will reload the gem classes when a file changes
file_watcher = ActiveSupport::FileUpdateChecker.new(gem_path.glob('**/*')) do
gem_loader.reload
end
# Plug it to Rails to be executed on each request
Rails.application.reloaders << Class.new do
def initialize(file_watcher)
#file_watcher = file_watcher
end
def updated?
#file_watcher.execute_if_updated
end
end.new(file_watcher)
With this, on each request, the class loader will reload the gem classes if one of them has been modified.
For a detailed walkthrough, see my article Embed a gem in a Rails project and enable autoreload.
When creating a gem for a specific application framework, I nest a project within the gem in a test directory. For example, with a Rails specific gem I'd setup a directory structure like:
Rakefile
Gemfile
attached.gemspec
lib/attached.rb
lib/...
test/Gemfile
test/app/...
test/...
To test, I setup the nested project Gemfile using gem 'attached', path: '...' and run rake test inside the test directory. Is it possible to add a task to my main Rakefile that will allow me to run the tests in my sub project without changing into the directory first?
I always use the gem enginex to help me setup my gems with an integrated rails application.
In the root Rakefile they write:
require 'rake/testtask'
Rake::TestTask.new(:test) do |t|
t.libs << 'lib'
t.libs << 'test'
t.pattern = 'test/**/*_test.rb'
t.verbose = false
end
task :default => :test
They do not use a nested Gemfile, instead they load the Rails project inside the test_helper.rb and rails is a development/test dependency of the gem.
To check out the gem:
gem install enginex
Or check the source. This gem is included in rails 3.1 as the new plugin generator.
The simplest way (and I think the least bug prone), would be to have a task like:
task :test do
system('cd test; bundle exec rake test')
end
The other, more complicated way would be to include all "subtasks" in the root Rakefile add a prerequisite to each one of them, that will change the current directory, like so:
task :change_dir do
puts 'changing dir'
Dir.chdir('test')
end
namespace :sub do
load 'test/Rakefile'
end
Rake::Task.tasks.select{|t| t.name.start_with?('sub:')}.each do |task|
task.prerequisites.insert(0, 'change_dir')
end
I'm not sure how this will work with bundler though.
One last thing that I would recommend to you is to take a look at a project structure that bundle gem command creates. I use it for all my gems now, and I believe that taking advantage of it would make your problem go away entirely:)
What do you do when you want to use a gem for development/testing that you don't want to force other devs to use? Right now I have
begin
require 'redgreen'
rescue LoadError
end
in test_helper.rb and no gem config, but that seems like a clumsy approach, albeit a functional one. I'd like to do something like the following:
config.gem "redgreen", :optional => true
Any other suggestions? Or should I just vendor those pretty superficial gems...?
EDIT
To be clear, I am only talking about those specific gems, like redgreen, which aren't actually used in the functional code, but only in the coding process. There is no need to vendor these at all, except to avoid the conditional require.
Gems that are specific to your development environment should be installed in your gemset or local gems, but not in the Gemfile.
A classic example is the ruby-debug-base19x which Rubymine needs for debugging. This is installed in your local gemset, but not in the Gemfile because not all coders use Rubymine.
[EDIT]
Indeed, everything is run in the context of the bundle, and outside gems are not reachable. There do exist some workarounds indeed. Most of them are dirty :)
I found a lot of good solutions in this bundler issue.
The nicest solution was to add this to your .irbrc :
# Add all gems in the global gemset to the $LOAD_PATH so they can be used even
# in places like 'rails console'.
if defined?(::Bundler)
global_gemset = ENV['GEM_PATH'].split(':').grep(/ruby.*#global/).first
if global_gemset
all_global_gem_paths = Dir.glob("#{global_gemset}/gems/*")
all_global_gem_paths.each do |p|
gem_path = "#{p}/lib"
$LOAD_PATH << gem_path
end
end
end
require 'irb/completion'
require 'rubygems'
require 'wirble'
Wirble.init
Wirble.colorize
If you then install wirble to the global gemset, it can then be found.
Original source: https://gist.github.com/794915
Hope this helps.
I answered a similar question of my own here
User-level bundler Gemfile
One way to do this is to create different environments:
group :scott do
end
Then
bundle --with-env=scott
Ok, I think I've come up with something. Basically, the idea is to only execute a secondary Gemfile when a Rails app is executing. To do this we add two things:
First, we alter the rails script a little:
# in ./script/rails
Kernel::IN_RAILS_APP = true
APP_PATH = File.expand_path('../../config/application', __FILE__)
require File.expand_path('../../config/boot', __FILE__)
require 'rails/commands'
Second, we tell bundler to pull in the secondary Gemfile if we're in a rails app and a secondary file exists:
# Gemfile
if Kernel.const_defined?(:IN_RAILS_APP)
local_gemfile = File.dirname(__FILE__) + "/Gemfile.local"
if File.exists?(local_gemfile)
puts 'using local gemfile'
self.instance_eval(Bundler.read_file(local_gemfile))
end
end
Now you can add a Gemfile.local to your project and run specific gems on a per-machine basis. bundle install works normally since the IN_RAILS_APP constant doesn't exist.
** Make sure to add Gemfile.local to your .gitignore.
In my opinions this is what environments are for. Fortunately there is also a way provided to do it with what is in your Gemfile, this is also how rails use it: groups
Pretty much use the environments the same way rails use it. Here is what you could find in your Gemfile:
group :test do
# Pretty printed test output
gem 'turn', :require => false
end
And here is what you can find in your config/application.rb
Bundler.require(:default, Rails.env) if defined?(Bundler)
All you would need to do is to change your local environment settings and the others working with you won't be affected unless they decide to. Everything gets committed and nothing gets lost.
Here some links :
http://yehudakatz.com/2010/05/09/the-how-and-why-of-bundler-groups/
http://gembundler.com/groups.html
If you want it to be optional, it's better to freeze the gem as a plugin. However, it's not a good idea to use different gems than the rest of a development team, as it creates some inconsistencies in the codebase that can be hard to track down later. I would say add it to config.gem, and just tell the other developers to do:
rake gems:install
And you're done.
This is how I tackled the same problem under Rails 3.1. In my Gemfile:
if File.exists? './tmp/eric_dev_gems'
gem 'redgreen'
gem 'awesome_print'
gem 'wirble'
gem 'wirb'
gem 'hirb'
end
Create a file in ./tmp/ (or in some folder which is in your .gitignore) of your choosing. I used eric_dev_gems. This should be ignored by git, and will only exist on your system unless one of your teammates decides he wants to create that file too.
I solved it by putting this in my gem file:
$gem_names ||= ENV['GEM_PATH'].split(':').map{|g| Dir.glob("#{g}/gems/*").map{|p|p.split('/gems/').last}}.flatten
gem 'redgreen' if $gem_names.any?{|n| n=~/redgreen/ }
That way the gem will only be used if you manually installed it on your system.
This works well but has the downside that it puts the gem name in the Gemfile.lock. This is of little consequence because the gem does not get installed with bundle install but it does make your lock file a bit messy and can cause the lock file to change a bit from one developer to the next.
If that is an issue for you another option is to keep the gemfile clean and require the gem by its full path, or you can add the path for just that gem. Like this:
$gem_paths ||= ENV['GEM_PATH'].split(':').map{|g| Dir.glob("#{g}/gems/*")}.flatten
$gem_paths.grep(/redgreen/).each {|p|$LOAD_PATH << p+'/lib'}
require 'redgreen' rescue nil