Zentest - How to stop automatic testing when red - ruby-on-rails

I am trying to configure autotest so that when I run my test suite and I have a failing test, autotest stops testing and waits for me to make a change before testing again. With my current configuration autotest keeps testing indefinetly when it encounters a failing test making it a bit of a hassle to deal with (having to tab into terminal and stop the autotest server everytime I get a failing test).
I am working on a rails app using RSpec, Zentest and Spork.
Relevant Gem Versions:
autotest (4.4.6)
rspec (2.6.0)
rspec-core (2.6.4)
rspec-expectations (2.6.0)
rspec-mocks (2.6.0)
rspec-rails (2.6.1)
spork (0.9.0.rc8)
ZenTest (4.5.0)
My .autotest file:
module Autotest::Notify
def self.notify title, msg, img, pri='low', time=3000
`notify-send -i #{img} -u #{pri} -t #{time} '#{msg}'`
end
Autotest.add_hook :ran_command do |autotest|
results = [autotest.results].flatten.join("\n")
output = results.slice(/(\d+)\s+examples?,\s*(\d+)\s+failures?(,\s*(\d+)\s+pending)?/)
folder = "~/.autotest_icons/"
if output =~ /[1-9]\d*\sfailures?/
notify "FAIL:", "#{output}", folder+"failed.png", 'critical', 10000
elsif output =~ /[1-9]\d*\spending?/
notify "PENDING:", "#{output}", folder+"pending.png", 'normal', 10000
else
notify "PASS:", "#{output}", folder+"passed.png"
end
end
end
Note: Most of my .autotest file was to get popups working in linux to display if my tests are passing or failing.
I have been searching for an answer to this problem for a while and have had no luck and I have found it very difficult to get my hands on some good documentation for Autotest. I have been staring at the RDoc for Zentest for quite a while now as well and I must just be missing something.
Any help, links to examples, etc, would be greatly appreciated.

I had the same problem and found that my development server was writing to the log file. After I added this to my .autotest file, the problem went away:
Autotest.add_hook :initialize do |at|
at.add_exception(%r{^\./\.git})
at.add_exception(%r{^\./log})
end

I saw a similar problem with ZenTest when I had a gem that was writing data to a directory that ZenTest was monitoring. IIRC, it was a gem that did full-text searching -- the index file the search generated was triggering ZenTest to run again, thereby regenerating the index.
I tracked down the problem by modifying the Growl notifications to tell me which files were triggering the autotest run (I was running on a Mac at the time).
The solution was to add an exception/exclude to the .autotest file, to tell it to ignore the index.
(I've just seen : Spork is repeatedly re-running failing tests in autotest which sounds very similar to your problem)

Related

Problems with Guard, Spork, Rspec & Rails 3

I've followed the spork railscast video and it gives me the following error when I try to run guard:
Guard is now watching at '/Users/m/work/'
Starting Spork for Test::Unit & RSpec
Couldn't find a supported test framework that begins with 'testunit'
Supported test frameworks:
( ) Cucumber
(*) RSpec
Legend: ( ) - not detected in project (*) - detected
Using RSpec
Preloading Rails environment
Loading Spork.prefork block...
Spork is ready and listening on 8989!
--> ERROR: Could not start Spork server for Test::Unit & RSpec. Make sure you can use it manually first.
Guard::RSpec is running, with RSpec 2!
Running all specs
It looks like The spork server starts up fine and then errors and tries to carry on. I've tried making the :wait option 120 seconds and it still has the same issue.
Spork works fine if I execute the tests without guard & guard-spork
Environment:
Mac OSX
rails (3.1.1)
guard (0.8.8)
guard-rspec (0.5.4)
spork (0.9.0.rc9)
guard-spork (0.3.1)
How would I go about debugging this issue? I have no idea where to start.
I had exactly this issue on Ubuntu. My solution was simple enough:
1) I stopped guard
2) I deleted the test folder
3) started guard
This time, instead of seeing Starting Spork for Test::Unit & RSpec, I got Starting Spork for RSpec. So spork automatically recognised that my test folder was no longer needed and everything ran sweetly.
Interestingly enough the
Supported test frameworks:
( ) Cucumber
(*) RSpec
message also disappeared and I got exactly the expected results as per railscast.
The only conclusions I can come up with are:
1) This is not an OS dependant issue.
2) spork, rspec and test unit don't play nicely on Rails > v3.x apps which makes sense. You don't need both. UPDATE - Please see updated answer below if you want both.
Obviously I followed the Railscast instructions very closely but I seriously suggest that you look at deleting the test folder.
UPDATE
It should be noted that it is possible to get both to play nicely together as per the comment below from #yuvilio, a quote of which follows:
I don't think the issue is that they don't get along. I got Cucumber/Rspec/testunit to play nice with each other in spork. In my gemfile, in addition to rspec/cucumber/guard related gems, I added spork-testunit, guard-test, ruby-prof gems and ran bundle install. Then, I bootstrapped testunit: bundle exec spork testunit --bootstrap.Then customized test/test_helper.rb. Then updated the guard file watching for testunit: bundle exec guard init test. When I ran guard, I got output including Spork server for RSpec, Cucumber, Test::Unit successfully started
If you don't want to delete the test folder, you can simply tell guard to ignore it:
guard 'spork', test_unit: false do
# ...
end

Autotest ruby on rails with rspec

I'm using autotest with ruby on rails. I have passing 2 passing tests when I run. rspec spec/; however, when I try to use autotest this is the output:
matt#matt-laptop:~/sample_app$ autotest
loading autotest/rails_rspec2
style: RailsRspec2
matt#matt-laptop:~/sample_app$
I get no output about the results of the tests. The same thing works with bundle exec autotest. I saw a post recommending autospec but that command is deprecated with rspec2. My Gemfile is
source 'http://rubygems.org'
gem 'rails', '3.0.5'
gem 'sqlite3-ruby', '1.3.2', :require => 'sqlite3'
group :development do
gem 'rspec-rails', '2.5.0'
gem 'autotest','4.4.4'
end
group :test do
gem 'rspec', '2.5.0'
gem 'webrat', '0.7.1'
gem 'autotest', '4.4.4'
gem 'redgreen', '1.2.2'
end
I have tried putting the .autotest config file in the root directory of my project as well as the home directory and neither makes a difference on the output. My .autotest file looks like this
#!/bin/ruby
require 'autotest/timestamp'
module Autotest::GnomeNotify
def self.notify title, msg, img
system "notify-send '#{title}' '#{msg}' -i #{img} -t 3000"
end
Autotest.add_hook :ran_command do |at|
image_root = "~/.autotest_images"
results = [at.results].flatten.join("\n")
results.gsub!(/\\e\[\d+m/,'')
output = results.slice(/(\d+)\sexamples?,\s(\d+)\sfailures?(,\s(\d+)\spending?|)/)
full_sentence, green, failures, garbage, pending = $~.to_a.map(&:to_i)
if output
if failures > 0
notify "FAIL", "#{output}", "#{image_root}/fail.png"
elsif pending > 0
notify "Pending", "#{output}", "#{image_root}/pending.png"
else
notify "Pass", "#{output}", "#{image_root}/pass.png"
end
end
end
end
I've also checked that libnotify-bin is installed and functioning.
To get verbose results from rspec, create a .rspec file in your root project folder and write :
--format documentation
If i may, allow me to suggest watchr instead of autotest (with spork as well). Very easy to set up and very effective.
Take a look at
http://www.rubyinside.com/how-to-rails-3-and-rspec-2-4336.html?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+RubyInside+%28Ruby+Inside%29
if you like.
I've had much the same problem with Autotest. I'm not certain, but I believe it may be very finicky about dependency versions. I'm a bit of a Rails noob, but since I so recently shared your troubles, let me show you what I did to fix the problem:
I was able to get it working by getting rid of all the code in the .autotest file and replacing it with:
require 'autotest/growl'
require 'autotest/fsevent'
To be clear, that second line should only apply if you're using OSX. I think you can safely get rid of all the "code" in your .autotest file because it relates to manual "red/green" growl notifications, which is no longer necessary once you have the "redgreen" gem installed. In short, you'll end up with a one or two line .autotest file.
Here are some files I installed via gem (some, like rspec-expectations, should be automatically installed as dependencies). The following files are the ones I believe should be relevant to your Autotest setup, the versions are the latest as of 10 minutes prior to writing this response.
autotest (4.4.6, 4.3.2)
autotest-fsevent (0.2.5, 0.2.2)
autotest-growl (0.2.9, 0.2.4)
autotest-rails-pure (4.1.2, 4.1.0)
redgreen (1.2.2)
rspec-core (2.5.1, 2.0.0.beta.18)
rspec-expectations (2.5.0, 2.0.0.beta.18)
rspec-mocks (2.5.0, 2.0.0.beta.18)
rspec-rails (2.5.0, 2.0.0.beta.18)
spork (0.8.4)
webrat (0.7.3)
ZenTest (4.5.0)
Do a gem list command to see which ones you have installed. It might be a simple matter of updating your versions and simplifying your .autotest file (I have mine in my home directory, if you care about location). Also, don't forget that the autotest-fsevent file only applies to OSX.
P.S. You'll probably STILL end up with unexplained autotest errors unless you put some additional code at the very end of your spec/controllers/spec_helper.rb file:
# Webrat configuration
Webrat.configure do |config|
config.mode = :rails
end

I can't get autotest working with Rails 3.0.4 and MongoID 2.0.0.rc.7

I'm not able to get autotest started. It just hangs after it clears the console screen:
$ autotest
loading autotest/rails_rspec2
--------------------------------------------------------------------------------
there is nothing in test.log. Other searches for this problem on S.O. don't provide an answer.
autotest 4.4.6
ZenTest 4.4.2
Rails 3.0.4
MongoID 2.0.0.rc.7
Maybe there is another place I could be looking for autotest errors?
It appears that an update to autotest (or rspec?) something somewhere made the config info in my ~/.autotest file obsolete. So i commented everything out except for:
require 'autotest/fsevent'
require 'autotest/growl'
and now it appears to be working!
no more: Autotest.add_hook(:initialize) {|autotest| ...

How to combine autotest and spork in Rails testing?

Autotest increases the speed at which tests run by running only the changed tests.
But I want to push it even further by using spork to preload the Rails environment, so that I will get even faster feedback.
Is this possible?
Autotest : https://github.com/grosser/autotest
Spork : http://github.com/timcharper/spork
ARTICLE1 mikbe has you covered! I would attempt to rephrase it here, but the post does such a great job.
If you happen to be on OSX, there are also instructions to utilize Growl for tray notifications.
ARTICLE2 Ruby Inside also has a walkthrough for Rails 3 and RSpec 2.
If you use Ruby 1.9 and want to use spork and autotest together with Test::Unit (actually MiniTest) try this:
Gemfile:
group :test do
# Newer version of test::unit:
gem 'minitest'
# spork preloads a rails instance which is forked every time the tests are
# run, removing test startup time.
gem 'spork'
# Run 'spork minitest' to start drb server (test server). Use 'testdrb' to
# run individual tests via spork.
gem 'spork-minitest'
# Run 'bundle exec autotest' to rerun relevant tests whenever a file/test is
# changed. '.autotest' makes sure the tests are run via test server (spork).
gem 'autotest-standalone'
# -pure gives us autotest without ZenTest gem.
gem 'autotest-rails-pure'
end
.autotest:
class Autotest
# run tests over drb server (spork)
def make_test_cmd files_to_test
if files_to_test.empty?
"" # no tests to run
else
"testdrb #{files_to_test.keys.join(' ')}"
end
end
end
(Note: Instructions says bin/testdrb, but I changed it to testdrb to make it work for me.)
In a terminal:
spork minitest --bootstrap
Edit test/test_helper.rband follow instructions.
After the above setup is done once, you can start the test server:
spork minitest
Finally start autotest in another terminal:
bundle exec autotest
And (hopefully) enjoy really fast autotesting with MiniTest.
I haven't tried it yet, but there's a section in chapter 3 of the RailsTutorial that tells some "hacks" to set up spork. The tutorial currently says:
... as of this writing Spork doesn’t officially support Rails 3
The chapter goes on to tell how to set it up with autotest. One thing to know is that you'll need
--drb
in your .rspec file.

Why is autotest not working?

I changed my .autotest file to use it with a Ruby-based project.
After that, when I wanted to use it for Rails, it is using the .autotest configuration settings I used for the Ruby project.
I uninstalled autotest and reinstalled it with no luck.
I also removed the .autotest file in the root directory but it is not working.
I'm trying to get autotest up and running as well. I just installed the gem. Running autotest or autotest --rails inside my rails app starts autotest, but it doesn't runs a single test. It reports that there aren't any.
UPDATE:
Just discovered I needed to install autotest-rails as well.
You should also install autotest-fsevent to make sure that autotest isn't polling all the time.
I've posted the results of my day of autotest at http://ryanbooker.com/archive/autotest-your-rails-apps.
The Short story:
sudo gem install ZenTest autotest-rails autotest-fsevent autotest-growl redgreen
Edit your ~/.autotest
# Include plugins
require 'autotest/fsevent'
require 'autotest/growl'
require 'redgreen/autotest' # yes this is correct
# Skip some paths
Autotest.add_hook :initialize do |autotest|
%w{.git .DS_Store ._* vendor}.each { |exception| autotest.add_exception(exception) }
false
end
You can launch autotest with:
cd myrailsapp
autotest
how are you launching autotest? If you use autotest --rails it should definitely work.
If you are using Rails 3.1 or higher, I highly recommend using guard for TDD with rspec. It works like magic. https://github.com/guard/guard

Resources