I want to analyse the test coverage of our code , and therefore, installed the simplecov gem.
Our testing environement has 2 seperate project: REST API test (Java+Rest-Assured) and Web UI testing (Java-Selenium).
As you can see, we dont have unit testing inside the rails app, and we are testing using external projects.
I configured the simplecov gem as descriped in the tutorial and put this, in the rails script:
require 'simplecov'
SimpleCov.start 'rails'
puts "require simplecov"
When loading the app, I see the string I printed.
I ran both automation test projects, saw their printouts in the rails log, but I don't see any coverage of controllers/models, I see only small precentage of initializtion files of some gems and rails.
I searched the net, and tried putting the code phrase in boot.rb or even puma.rb and it returned the same results.
Any ideas?
EDIT
Nothing helped with all the comments, but I figured out something very interesting, in all cases, I only see the name of methods marked as tested, not the content (in controllers).
I tried to put the simplecov start phrase in both bin/rails, puma.rb, config.ru, environment.rb, all not given the desired results of code coverage.
I'm not sure simplecov can measure the whole rails app coverage... But I googled something that you can attach as a rack middleware:
https://github.com/danmayer/coverband
And it's output is compatible with simplecov. So it looks like it could be useful in your case.
As you mentioned in your question you're using puma. I suspect that, since it's multi-threaded, it spawns a few rails apps and their simplecov output overwrites each other's results. I'd try with the single threaded server like webrick - but this may make your tests slower (depending on how the tests are fired up really) or try the coverband gem.
Also - even if the server is single threaded - I'm not sure if each request would not overwrite simplecov's output.
Maybe you have to specify the paths
require 'simplecov'
SimpleCov.start do
# add_filter '/admin/'
add_group "Models", "app/models"
add_group "Controllers", "app/controllers"
add_group "Lib", "lib/"
add_group "Helpers", "app/helpers"
end
You need to to start SimpleCov before loading any of your files, so put these lines as early as possible in your ruby entrypoint:
require 'simplecov'
SimpleCov.start
You could see an example in one of my repos here:
https://github.com/tareksamni/DockUp/blob/master/spec/spec_helper.rb
I do autoload my ruby code after starting SimpleCov. You need to the same as well:
require 'simplecov'
SimpleCov.start
require './autoload'
Related
I'm making a gem that executes Rails commands (rails g model Item for example). When I use it in a Rails project, everything works. The problem is testing it in development outside of a Rails project.
I'm using cucumber with aruba to test if CLI commands execute the proper rails commands and generate the expected files. Unfortunately, when I try to test the behaviour it fails because there are no rails files and the commands require to be run inside of a Rails project in order to work.
I have added a rails dependency to the gemspec:
Gem::Specification.new do |spec|
spec.add_development_dependency 'rails', '~> 5.2.4'
end
I've thought about creating a new rails project on test start and then deleting it after the tests run, but that seems highly inconvenient. Is there a better way to do this?
A technique we use for WickedPDF is in the default rake task, before we run the tests, is to delete & generate a full Rails application in a gitignored subdirectory of the gem.
As a high-level simplified example of this Rakefile, it looks something like this:
Rakefile
require 'rake'
require 'rake/testtask'
# This gets run when you run `bin/rake` or `bundle exec rake` without specifying a task.
task :default => [:generate_dummy_rails_app, :test]
desc 'generate a rails app inside the test directory to get access to it'
task :generate_dummy_rails_app do
if File.exist?('test/dummy/config/environment.rb')
FileUtils.rm_r Dir.glob('test/dummy/')
end
system('rails new test/dummy --database=sqlite3')
system('touch test/dummy/db/schema.rb')
FileUtils.cp 'test/fixtures/database.yml', 'test/dummy/config/'
FileUtils.rm_r Dir.glob('test/dummy/test/*') # clobber existing tests
end
desc 'run tests in the test directory, which includes the generated rails app'
Rake::TestTask.new(:test) do |t|
t.libs << 'lib'
t.libs << 'test'
t.pattern = 'test/**/*_test.rb'
t.verbose = true
end
Then, in test/test_helper.rb, we require the generated Rails app, which loads Rails itself and it's environment:
test/test_helper.rb
ENV['RAILS_ENV'] = 'test'
require File.expand_path('../dummy/config/environment.rb', __FILE__)
require 'test/unit' # or possibly rspec/minispec
# Tests can go here, or other test files can require this file to have the Rails environment available to them.
# Some tests may need to copy assets/fixtures/controllers into the dummy app before being run. That can happen here, or in your test setup.
You could skip parts of Rails that aren't needed by customizing the command that generates the app. For example, your gem may not need a database at all or a lot of things by default, so you command could be customized for a simpler app. Something like this maybe:
system("rails new test/dummy --skip-active-record \
--skip-active-storage --skip-action-cable --skip-webpack-install \
--skip-git --skip-sprockets --skip-javascript --skip-turbolinks")
In the WickedPDF project, we wanted to test across a wide range of "default" Rails installs, so we don't customize the command much, but that may generate much more than what you need to test some generator tasks.
WickedPDF also tests against multiple versions of Rails with TravisCI and multiple Gemfiles, but this could also be accomplished with the Appraisal gem that Luke suggested in this thread.
Check out Thoughbot's Appraisal gem:
Appraisal integrates with bundler and rake to test your library against different versions of dependencies in repeatable scenarios called "appraisals."
Here is a guide on how to set it up, including setting up a micro Rails app within your tests dir.
I am using the default test for different purposes and I have decided to make a specific rspec environment configuration for running the test suite.
However, I discovered that upon changing to ENV["RAILS_ENV"] ||= rspec in my rails_helper.rb file, suddenly a LOT of things are going wrong, constants are not being loaded (FactoryGirl, DatabaseCleaner, etc. throw uninitialized constant errors)
My question is, where is the code that loads those guys in test environment ? Since I am planning to use this stage for other purposes than running automatic tests, I'm afraid this "out of nowhere" added configuration might not work well with what I am planning to do.
From the perspective of Rails, the test environment is configured and loaded like any other environment such as production or development. You can see this prefixing RAILS_ENV=test to many of the native Rails commands e.g. RAILS_ENV=test rails c will load the rails console for the test environment, and so on. Similarly, all test-specific configuration within Rails is defined in test.rb in your config/environments folder.
However, when you run your specs with rspec spec, you're actually starting the RSpec runner, which, for most intents and purposes, runs independently of Rails (even with the rspec-rails gem).
By convention, when RSpec starts the first thing it does is read command line args from the .rspec in the current directory, if it exists. Then it runs spec_helper.rb (and also rails_helper.rb for rspec-rails 3+). It's actually the spec_helper.rb which does all the heavy-lifting in loading the Rails environment for your tests, along with any of the modules you're using in tests, such as DatabaseCleaner, FactoryGirl, etc.
If you're wondering how RSpec hooks into Rails, the bulk of it is performed in this line, which bootstraps Rails.
require File.expand_path('../../config/environment', __FILE__)
Now, as to your question, without the ENV['RAILS_ENV'] ||= 'test' statement, the above line will load Rails in the default environment (development), which isn't what you want, since any gems not in the :test group will not be loaded, and environments/test.rb will not be loaded either.
TL;DR
Test configuration is handled by two files: spec/spec_helper.rb (sometimes named rails_helper.rb) and config/environments/test.rb. The former configures RSpec and any objects and modules which will be used specifically within the files used in spec, the latter configures your Rails app itself. Omitting ENV['RAILS_ENV'] ||= test loads the development environment and gemsets instead of the test environment and gemsets, which is why you're getting a ton of errors.
If you are getting uninitialized constant errors for FactoryGirl, DatabaseCleaner etc, you most likely included them to test group in your Gemfile.
You should move them to rspec group, eg:
# Gemfile
group :rspec do
gem 'factory_girl_rails', '~> 4.0'
gem 'faker'
end
I have a Rails 4.1 project that uses RSpec and Cucumber.
I've recently added fixture_builder.
fixture_builder.rb includes logic to rebuild fixtures any time the file changes.
This works fine for RSpec with require 'fixture_builder' in spec_helper.rb.
However, when running Cucumber tests fixture_builder.rb is not called, so fixtures are not updated if any changes have been made to fixture_builder.rb.
Is there an equivalent config file like spec_helper.rb for Cucumber?
By default, Cucumber loads all *.rb files in the same directory as the feature(s) it's running and all subdirectories of that directory, so you can put your require in any file you want in any of those directories.
The conventional thing to do is to put 'support' code like require in features/support/env.rb, or in another file in features/support if your env.rb gets too big.
The cucumber-rails gem provides a generator that sets up Cucumber to work with Rails. If you haven't already, install the gem and run
rails g cucumber:install
to create features/support/env.rb and the rest of the usual Rails + Cucumber setup.
I have a project in which I'm using rails 3.2.3, rspec 2.9.0, and simplecov 0.6.1, all seem to be the latest gems.
I'm getting code coverage results for my controllers and models, but not my views. I've tried adding the group in my simplecov setup:
SimpleCov.start 'rails' do
add_group 'Views', 'app/views'
end
And even explicitly saying I want .erb files included:
SimpleCov.start 'rails' do
add_group 'Views', 'app/views/*/*.erb'
end
...but no dice. The "Views" group is there in my coverage results, but no files are listed there.
Has anyone gotten this working?
Apparently, the answer to this is "you can't", due to a limitation in the standard ruby coverage library:
https://github.com/colszowka/simplecov/issues/38
I'm working on a plugin for Rails that would add limited in-memory caching to ActiveRecord's finders. The functionality itself is mature enough, but I can't for the life of me get unit tests to work with the plugin.
I now have under vendor/plugins/my_plugin/test/my_plugin_test.rb a standard subclass of ActiveSupport::TestCase with a couple of basic tests. I try running 'rake test' from the plugin directory, and I have confirmed that this task loads the ruby file with the test case, but it doesn't actually run any of the tests.
I followed the Rails plugin guide (http://guides.rubyonrails.org/plugins.html) where applicable, but it seems to be horribly outdated (it suggests things that Rails now do automatically, etc.)
The only output I get is this:
Kakadu:ingenious_record adam$ rake test
(in /Users/adam/Sites/1_PRK/vendor/plugins/ingenious_record)
/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/bin/ruby -Ilib:lib:test "/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/gems/1.8/gems/rake-0.8.3/lib/rake/rake_test_loader.rb" "test/ingenious_record_test.rb"
The simplest test case looks like this:
require 'test_helper'
require 'active_record'
class IngeniousRecordTest < ActiveSupport::TestCase
test "example" do
assert false
end
end
This should definitely produce at least some output, and the only test in that file should produce a failed assertion.
Any ideas what I could do to get Rails to run my tests?
I test my plugins using the Engines plugin's rake tasks:
rake test:plugins:all PLUGIN=myplugin
I'm sure you can do it without Engines, but it is an option. Recent versions of Rails (I'm still on 2.2.2) have Engines support integrated.
Apparently there is a little problem with rails 2.2.3, as it does not include the 'test/unit' package/file in plugins.
In order to fix your plugin problems, just add the following to the top of your test_helper.rb file:
require 'test/unit'
Found it as a ticket at lighthouse.