I am struggling to run only tests that end with *Test.java and have this in my configs but test runner does not pick anything with this config, am I missing something?
testPattern {
include '**/*Test.java'
}
It is working it was not just logging property, it works.
Related
I'm working with Rails 5 and rspec (gem version 4). I was wondering if RSpec can be configured to only run tests that have been modified within a single file when only running that file, i.e.
bundle exec rspec spec/my_spec.rb
. If my file is like this
RSpec.describe MyClass do
context "context 1" do
it "tests condition 1" do
end
it "tests condition 2" do
end
...
end
context "context 2" do
...
end
...
end
and I only update tests in "context 1," is it possible to test the single file and have only modified tests from within that file run? With respect to this answer -- Can I get RSpec to only run changed specs?, it appears that only relates to actual files that have changed when running the complete suite of rspec tests.
I think you are looking for guard.
Checkout nicely written article https://collectiveidea.com/blog/archives/2017/02/09/guard-is-your-friend
There is a Ruby Gem called retest designed to do exactly that. Just run retest, and it will watch for changes in the code or the specs themselves, and rerun just the respective spec file.
In a terminal, I can run multiple rake tasks one after another in a single line:
rake grunt war app_server:start
I can't seem to get this to work in IntelliJ IDEA when trying to create a Configuration to do it. I'm able to fill in the "Task Name" field as above, but when I run the configuration, I get the following error:
Don't know how to build task 'grunt war app_server:start' (see --tasks)
Does anyone know if I'm just doing something wrong, or is this just not possible in IDEA? Thank you.
Compounds don't work since they are run in parallel and not in the same process. This makes it impossible to execute for example environment task followed by another task such as elasticsearch:import:all.
The only way I could get it to work was to create a project-level task, for example lib/tasks/es.rake:
require 'elasticsearch/rails/tasks/import'
namespace :es do
task reindex: %w[environment elasticsearch:import:all]
end
You then execute es:reindex as a top-level task which will execute its dependencies in order.
As an option, in the Idea you can create multiple tasks and combine them in a group:
Run -> Edit Configurations -> Add -> Compound
I am using the out of the box, vanilla test suite for Rails 5 beta. I am wondering if anyone has figured out how to run a global setup, and tear down.
The reason this is required to I am spinning up a in memory Elasticsearch cluster before any test begins and stop the cluster once the tests are done.
Rspec is not an option.
Under Minitest (which is the default testing environment in Rails 4+), to get the "global setup" behavior, simply run anything in your test_helper.rb (outside the tests themselves or any setup methods), i.e. in the file where you load your testing environment from. The test helper is usually required in the tests, so its code is run once before any tests.
For a "global teardown", Minitest provides the Minitest.after_run method. Anything inside its block will be run once after all tests are finished (it uses the program exit hook). Place it e.g. in the test_helper again. For this to work you need to require 'minitest/autorun' at the beginning of the test helper file.
test/test_helper.rb
class ActiveSupport::TestCase
# Some pre-generated stuff here
setup do
do_something
end
teardown do
do_something
end
end
I have a rails application that runs parallel_test with rspec inside circleci
Looking around on Internet I added up this to the very beginnging of my spec_helper.rb file:
if ENV['COVERAGE']
require 'simplecov'
# on circleci change the output dir to the artifacts
if ENV['CIRCLE_ARTIFACTS']
dir = File.join("..", "..", "..", ENV['CIRCLE_ARTIFACTS'], "coverage")
SimpleCov.coverage_dir(dir)
SimpleCov.merge_timeout 3600
SimpleCov.command_name "rspec_#{Process.pid.to_s}#{ENV['TEST_ENV_NUMBER']}"
end
SimpleCov.start 'rails'
end
The problem is that as a result I get different folders one for each circleci instance:
What am I doing wrong ?
I work at CircleCI. Unfortunately this isn't going to work - we don't collect the artifacts directories from different builds until after all builds have finished running, so tools that try to merge them together during the build won't work. We have talked about adding capabilities to do this, but it's not currently on our feature roadmap, sorry!
For anyone who still looking for a solution to this problem, there is a new possibility: using ssh between containers to manually sync and merge reports, see the docs. This is, however, not a turnkey solution, you will have to write the necessary scripts yourself.
Otherwise, you can also use an external coverage service (we use coveralls codecov) together with CircleCI's notification webhook.
Edit
You can add the webhook like this in your circle.yml (thanks Ian):
notify:
webhooks:
- url: https://coveralls.io/webhook?repo_token=(your repo token)
To expound upon what Frank Eckert said, Coveralls can do this. However, that documentation is slightly off.
Add to you Gemfile:
gem 'coveralls', require: false
Add to your spec/spec_helper.rb:
if ENV['CIRCLECI']
# If running in CircleCI, run with Coveralls too
require 'coveralls'
Coveralls.wear!('rails')
end
Add to your circle.yml (not coveralls.yml as the article states):
notify:
webhooks:
- url: https://coveralls.io/webhook?repo_token=your-repo-token
Add to (or create) a .coveralls.yml in the root folder, and add:
repo_token: your-repo-token
Finally, go into the CircleCI project configuration and add an environment variable: COVERALLS_PARALLEL=true
This worked for us, however we find errors in the proper line count (reported to Coveralls) but it is working and gives us a good idea about our coverage changing over time.
I am looking into using capybara-webkit to do somewhat close-to-reality tests of app. This is absolutely neccessary as the app features a very rich JS-based UI and the Rails part is mostly API calls.
The question is: is there any tools to integrate into testing pipeline which could instrument Javascript code and report its coverage? The key here is the ability to integrate into testing workflow (just like rcov/simplecov) easily – I don't like the idea do it myself with jscoverage or analogue :)
Many thanks in advance.
This has now been added to JSCover (in trunk) - the related thread at JSCover is here.
I managed to get JSCover working in the Rails + Capybara pipeline, but
it did take quite a bit of hacking to get it to work
These changes are now in JSCover's trunk and will be part of version 1.0.5. There's working examples (including a Selenium IDE recorded example) and documentation in there too.
There is some additional work needed to get the branch detection to
work since that uses objects that cannot be easily serialized to JSON
This is a function to do this which is used in the new code.
Anyway, the end result works nicely
I agree. This makes JSCover useable by higher level tools that don't work well with iFrames or multiple windows which are avoided by this approach. It also means code coverage can be added to existing Selenium tests with two adjustments:
Make the tests run through the JSCover proxy
Save the coverage report at the end of the test suite
See JSCover's documentation for more information. Version 1.0.5 containing these changes should be released in a few days.
Update: Starting from JSCover version 1.05 the hacks I outlined in my previous answer are no longer needed. I've updated my answer to reflect this.
I managed to get JSCover working in the Rails + Capybara pipeline, but it did take some hacking to get it to work. I built a little rake task that:
uses the rails asset pipeline to generate the scripts
calls the java jar to instrument all the files and generate an empty report into a temp dir
patches the jscover.js script to operate in "report mode" (simply add jscoverage_isReport=true at the end)
copies the result to /public/assets so the tests pick it up without needing any changes and so the coverage report can be opened automatically in the browser
Then I added a setup task to clear out the browser's localStorage at the start of the tests and a teardown task that writes out the completed report at the end.
def setup
unless $startup_once
$startup_once=true
puts 'Clearing localStorage'
visit('/')
page.execute_script('localStorage.removeItem("jscover");')
end
end
def teardown
out=page.evaluate_script("typeof(_$jscoverage)!='undefined' && jscoverage_serializeCoverageToJSON()")
unless out.blank? then
File.open(File.join(Rails.root,"public/assets/jscoverage.json"), 'w') {|f| f.write(out) }
end
end
Anyway, the end result works nicely, the advantage of doing it this way is that it also works on headless browsers so it can also be included in CI.
*** Update 2: Here is a rake task that automates the steps, drop this in /lib/tasks
# Coverage testing for JavaScript
#
# Usage:
# Download JSCover from: http://tntim96.github.io/JSCover/ and move it to
# ~/Applications/JSCover-1
# First instumentalize the javascript files:
# rake assets:coverage
# Then run browser tests
# rake test
# See the results in the browser
# http://localhost:3000/assets/jscoverage.html
# Don't forget to clean up instrumentalization afterwards:
# rake assets:clobber
# Also don't forget to re-do instrumentalization after changing a JS file
namespace :assets do
desc 'Instrument all the assets named in config.assets.precompile'
task :coverage do
Rake::Task["assets:coverage:primary"].execute
end
namespace :coverage do
def jscoverage_loc;Dir.home+'/Applications/JSCover-1/';end
def internal_instrumentalize
config = Rails.application.config
target=File.join(Rails.public_path,config.assets.prefix)
environment = Sprockets::Environment.new
environment.append_path 'app/assets/javascripts'
`rm -rf #{tmp=File.join(Rails.root,'tmp','jscover')}`
`mkdir #{tmp}`
`rm -rf #{target}`
`mkdir #{target}`
print 'Generating assets'
require File.join(Rails.root,'config','initializers','assets.rb')
(%w{application.js}+config.assets.precompile.select{|f| f.is_a?(String) && f =~ /\.js$/}).each do |f|
print '.';File.open(File.join(target,f), 'w') {|ff| ff.write(environment[f].to_s) }
end
puts "\nInstrumentalizing…"
`java -Dfile.encoding=UTF-8 -jar #{jscoverage_loc}target/dist/JSCover-all.jar -fs #{target} #{tmp} #{'--no-branch' unless ENV['C1']} --local-storage`
puts 'Copying into place…'
`cp -R #{tmp}/ #{target}`
`rm -rf #{tmp}`
File.open("#{target}/jscoverage.js",'a'){|f| f.puts 'jscoverage_isReport = true' }
end
task :primary => %w(assets:environment) do
unless Dir.exist?(jscoverage_loc)
abort "Cannot find JSCover! Download from: http://tntim96.github.io/JSCover/ and put in #{jscoverage_loc}"
end
internal_instrumentalize
end
end
end