I'm having intermittent build issues on our CI that consistently succeed locally. We had our master build succeed but rebuild fails (without cache clear or any changes). The failures are only on our js-driven feature specs, the others succeed consistently for both CI and dev.
Error: Net::ReadTimeout
Partial Stack:
Rails 4.1.4
RSpec 3.0
Capybara (2.4.1, 2.5, and master)
selenium-webdriver 2.45.0
Failed fix attempts:
Updating capybara and/or selenium
Revert first merge we noticed the errors
Clearing build cache
Updating dev group and/or test group gems
Any help is hugely appreciated!
Ended up solving this one. It's partially due to an error on our part and partially due to a selenium bug.
Steps to reproduce:
visit /path/to/page
alert("whatever") {don't close it}
call visit /path/to/page again
Any subsequent pages selenium loads will still have the alert box open, thus most matchers will break.
Has anyone seen this problem before? Sometimes when running specs for my Rails 3.2.14 project rspec seems to finish as usual:
Finished in 1.27 seconds
6 examples, 2 failures
Failed examples:
rspec ./spec/models/my_spec.rb:123 # Hello world 1
rspec ./spec/models/my_spec.rb:234 # Hello world 2
but then it just hangs there and won't let me continue working in that shell. I can kill -9 the process from another terminal tab, or just start a new shell and run the tests again there, but it makes test driven development a huge pain.
When I restart my computer, the problem goes away for a while, but it always happens again eventually. After it hangs once, it keeps hanging every time I run rspec, even if I run different tests in a different project. The same tests in the same projects pass just fine on my coworkers' computers every time.
I'm not sure what information would help to answer this question so let me know if there is something I should add to this post. I'm running ruby 2.0.0p195 and rails 3.2.14. I've got Mac OS 10.7.5. I use zsh and rbenv.
Thanks for reading!
I had the same issue and solved it by adding the following to my spec_helper.rb:
SimpleCov.start do
use_merging false
end
Note that I'm running on a virtual box vm with a synched directory.
Here's the explanation I found as to why this works:
"When simplecov exists, by default it will try to merge the recorded coverage with what's on disk. To avoid corruption it uses a lock file to guard this merge. Because my virtual box shared fs is not actually posix compliant, the lock file would never acquire, and silently block forever here.
Since I don't care about merging coverage results, the solution for me was to simply disable this behavior with the use_merging flag."
https://gist.github.com/k-yamada/3930916
Ok, figured it out with a friend's help. All it took was removing SimpleCov from my spec_helper. Not sure why SimpleCov was causing the issue, but I'll post an update here if I find out.
I have a rails 2.3 application running on REE. I am using capybara 1.1.2, cucumber 1.1.0, poltergeist (used both 0.6.0 and lastest from github) and phantomJS 1.5.0. My problem is that I believe I have a race condition in phantomJS that is causing my cucumber tests to randomly fail. The cucumber test work perfectly when I run them in Chrome so I am pretty sure the problem is in poltergeist or phantomJS. When I run the cucumber tests in PhantomJS (so that I can run them on the CI server) I will randomly get errors like this:
One or more errors were raised in the Javascript code on the page:
TypeError: 'undefined' is not a function
http://static.ak.fbcdn.net/rsrc.php/v2/yd/r/c9z-2BUoufJ.js:75
http://static.ak.fbcdn.net/rsrc.php/v2/yd/r/c9z-2BUoufJ.js:75
http://static.ak.fbcdn.net/rsrc.php/v2/yd/r/c9z-2BUoufJ.js:9 in bound:(?) (Capybara::Poltergeist::JavascriptError)
./features/step_definitions/site_steps.rb:8:in `/I am on the home page/'
features/account_creation.feature:89:in `Given I am on the home page'
Failing Scenarios:
cucumber features/account_creation.feature:5 # Scenario: I should be able to start creating a resume without signing up
cucumber features/account_creation.feature:79 # Scenario: I should be able to log in with an old username and password
cucumber features/account_creation.feature:88 # Scenario: I should be able to remember my login so that when the browser is closed I am automatically logged in
Based off of the Github page for poltergeist I have added this to my env.rb to solve the rack pre version 1.3 race condition problems:
Capybara.server do |app, port|
require 'rack/handler/thin'
Thin::Logging.silent = true
Thin::Server.new('0.0.0.0', port, app).start
end
But this has not made any difference that I can tell. Any thoughts would be appreciated even if it is just a suggesting that this is a poltergeist problem or a phantomJS problem so I know how I should ask about it next.
I have an application with a bunch of cucumber tests, and I tried using selenium with it today. It took a bit, but I got it running, it starts up selenium, uses the correct database and everything, but everybody has a wrong password.
I can see the user getting created, I can log the password and see that it's the correct password, but every test user just gets redirected back to the login page with the incorrect username/password error.
Is this a Selenium problem, or some other dependent system or am I missing something? It works fine with the standard :rails webrat mode.
I run the task as
rake features
Ubuntu 9.04
machinist 0.3.1
Selenium 1.1.14
Firefox 3.0.10
ruby 1.8.7-p160
cucumber 0.3.7
EDIT: Now, after a restart (office reorg) it won't run tests at all. Firefox just starts, and shuts down, then everything fails.
14:13:25.141 ERROR - Failed to start new browser session, shutdown browser an clear all session data
org.openqa.selenium.server.RemoteCommandException: timed out waiting for window 'null' to appear
I've tried updating the selenium-server.jar from the latest version, and no dice.
Cucumber::Rails.use_transactional_fixtures
Was the culprit. In selenium you can't have transactional fixtures, so writing cucumber steps will be a bit more fun, but whatever. Commenting out the above line fixes the password problem.
Whenever I run rspec tests for my Rails application it takes forever and a day of overhead before it actually starts running tests. Why is rspec so slow? Is there a way to speed up Rails' initial load or single out the part of my Rails app I need (e.g. ActiveRecord stuff only) so it doesn't load absolutely everything to run a few tests?
I definitely suggest checking out spork.
http://spork.rubyforge.org/
The railstutorial specifically addresses this, and gives a workaround to get spork running nicely in rails 3.0 (as of this moment, spork is not rails 3 ready out of the box). Of course, if you're not on rails 3.0, then you should be good to go.
The part of the tutorial showing how to get spork running in rails 3.0
http://railstutorial.org/chapters/static-pages#sec:spork
Checking when spork is rails 3.0 ready
http://www.railsplugins.org/plugins/440-spork
You should be able to to speed up your script/spec calls by running script/spec_server in a separate terminal window, then adding the additional -X parameter to your spec calls.
Why is rspec so slow? because it loads all the environement, loads fixtures and all that jazz.
Is there a way to speed up Rails' initial load you could try using mocks instead of relying on the database, this is actually correct for unit testing and will definitly speed up your unit tests. Additionnaly using the spec server as mentionned by #Scott Matthewman can help, same with the autotest from zentest mentionned by #Marc-Andre Lafortune
Is there a way to single out the part of my Rails app I need (e.g. ActiveRecord stuff only) so it doesn't load absolutely everything to run a few tests? what about this
rake test:recent
I am not sure how the rspec task integrate with this but you could definitely use the test:recent task as a template to do the same with rspec tests if the.
rake test:rspec:recent
doesn't exist yet
because it loads all the environement, loads fixtures and all that jazz.
The real culprit is if you run it using rake spec, it runs the db:test:prepare task.
This task drops your entire test database and re-creates it from scratch. This seems ridiculous to me, but that's what it does (the same thing happens when you run rake:test:units etc).
You can easily work around this using the spec application which rspec installs as part of the rspec gem.
Like this:
cd railsapp
spec spec # run all specs without rebuilding the whole damn database
spec spec/models # run model specs only
cd spec
spec controllers/user* # run specs for controllers that start with user
I think the "zen" experience you're looking for is to run spec_server and autospec in the background, with the result being near-instant tests when you save a file.
However, I'm having problems getting these two programs to communicate.
I found an explanation here:
I've noticed that autotest doesn't send commands to the spec_server.
Instead it reloads the entire Rails environment and your application's
plugins everytime it executes. This causes autotest to run
significantly slower than script server, because when you run the
script/spec command the specs are sent to the spec_server which
already has your Rails environment fired up and ready to go. If you
happen to install a new plugin or something like that, then you'll
have to restart the spec_server.
But, how do we fix this issue? I'm guessing it would involve downloading ZenTest and changing code for the autotest program, but don't have time to try it out right now.
Are you running this over Rails? If so, it's not RSpec's initialization that's slow, it's Rails'. Rails has to initialize the entire codebase and yours before running the specs. Well, it doesn't have to, but it does. RSpec runs pretty fast for me under my small non-rails projects.
Running tests can be really slow because the whole rails environment has to load (try script/console) and only then can all tests run. You should use autotest which keeps the environment loaded and will check which files you edit. When you edit and save a file, only the tests that depend on these will run automatically and quickly.
If you're using a Mac I recommend using Rspactor over autotest as it uses a lot fewer resources for polling changed files than autotest. There is both a full Cocoa version
RSpactor.app
or the gem version that I maintain at Github
sudo gem install pelle-rspactor
While these don't speed up individual rspec tests, they feel much faster as they auto run the affected spec's within a second of you hitting save.
As of rspec-rails-1.2.7, spec_server is deprecated in favor of the spork gem.
The main reason is that require takes forever on windows, for some reason.
Tips for speedup:
spork now works with windows, I believe.
You can try "faster_require" which caches locations:
http://github.com/rdp/faster_require
GL.
-rp
If you are on a Windows environment then there is probably little you can do as Rails seems to startup really slowly under Windows. I had the same experience on Windows and had to move my setup to a Linux VM to make it really zippy (I was also using autotest).