I'm having intermittent build issues on our CI that consistently succeed locally. We had our master build succeed but rebuild fails (without cache clear or any changes). The failures are only on our js-driven feature specs, the others succeed consistently for both CI and dev.
Error: Net::ReadTimeout
Partial Stack:
Rails 4.1.4
RSpec 3.0
Capybara (2.4.1, 2.5, and master)
selenium-webdriver 2.45.0
Failed fix attempts:
Updating capybara and/or selenium
Revert first merge we noticed the errors
Clearing build cache
Updating dev group and/or test group gems
Any help is hugely appreciated!
Ended up solving this one. It's partially due to an error on our part and partially due to a selenium bug.
Steps to reproduce:
visit /path/to/page
alert("whatever") {don't close it}
call visit /path/to/page again
Any subsequent pages selenium loads will still have the alert box open, thus most matchers will break.
Related
I'm having build issues that consistently succeed locally.The failures are only on login feature specs at the first time, the others succeed consistently for both Circle and dev.
Error: "Net::ReadTimeout"
Partial Stack:
- Rails 4.2.4
- RSpec 3.3.0
- Capybara (2.7.1)
- poltergeist ( 1.6.0 )
It happens at the first time that cucumber run on circle ci.
I have tried to use retry gem and add more time for timeout (180s for poltergeist ). But it still happens. It works fine on local.
Any help is hugely appreciated!
Generally this is caused by asset compilation on the first request taking a long time. Try precompiling the test assets before running the tests
Has anyone seen this problem before? Sometimes when running specs for my Rails 3.2.14 project rspec seems to finish as usual:
Finished in 1.27 seconds
6 examples, 2 failures
Failed examples:
rspec ./spec/models/my_spec.rb:123 # Hello world 1
rspec ./spec/models/my_spec.rb:234 # Hello world 2
but then it just hangs there and won't let me continue working in that shell. I can kill -9 the process from another terminal tab, or just start a new shell and run the tests again there, but it makes test driven development a huge pain.
When I restart my computer, the problem goes away for a while, but it always happens again eventually. After it hangs once, it keeps hanging every time I run rspec, even if I run different tests in a different project. The same tests in the same projects pass just fine on my coworkers' computers every time.
I'm not sure what information would help to answer this question so let me know if there is something I should add to this post. I'm running ruby 2.0.0p195 and rails 3.2.14. I've got Mac OS 10.7.5. I use zsh and rbenv.
Thanks for reading!
I had the same issue and solved it by adding the following to my spec_helper.rb:
SimpleCov.start do
use_merging false
end
Note that I'm running on a virtual box vm with a synched directory.
Here's the explanation I found as to why this works:
"When simplecov exists, by default it will try to merge the recorded coverage with what's on disk. To avoid corruption it uses a lock file to guard this merge. Because my virtual box shared fs is not actually posix compliant, the lock file would never acquire, and silently block forever here.
Since I don't care about merging coverage results, the solution for me was to simply disable this behavior with the use_merging flag."
https://gist.github.com/k-yamada/3930916
Ok, figured it out with a friend's help. All it took was removing SimpleCov from my spec_helper. Not sure why SimpleCov was causing the issue, but I'll post an update here if I find out.
I've recently moved mi CI server (Teamcity) to another powerful machine with same configuration and pretty similar OS.
Since then some of my integration specs have started to fail. My setup is pretty standard, Rails 3 + capybara + poltergeist + phantomjs.
Failures are deterministic, they always happen and they are always related to some missing nodes in the DOM. Also, failures happens across different projects with similar setup so it's not something related to project configuration. This is happening with both capybara 1.x and capybara 2.
This is the simplest failing spec. Note that this spec runs with no need of javascript, so the issue is also present in rack only specs.
scenario 'require an unsubscription' do
visit unsubscribe_index_path
within main_content do
choose list.description
fill_in 'Email', :with => subscriber.email
click_button 'Unsubscribe'
end
save_page # <--- Added to debug output
# !!! HERE is the first failing assertion
page.should have_content('You should have received a confirmation message')
# Analytics event recorded
# !!! this also is failing
page.should have_event('Unsubscription', 'Sent', list_name)
# If I comment previous two lines the spec passes on CI machine
# this means that the form is submitted with success since email is triggered
# from controller code
last_email_sent.should have_subject 'Unsubscribe request received'
last_email_sent.should deliver_to subscriber.email
end
What I've tried:
run the specs on different machines, they works on every dev machine and also in a staging server. I can only reproduce the failure on the CI machine even outside of CI environment (i.e. by running the specs via command line)
Increased Capybara.default_wait_time to a ridiculous 20
Tried with a brutal sleep before the page.should have_content line
upgrade RVM, ruby, capybara, poltergeist on their latest versions on the CI machine.
upgrade teamcity to its latest version
The strangest thing I found is when I've added a save_page call just before the failing line. If i run the spec on my machine and then on the CI where the server is failing and comparing those two files the result is this:
$ diff capybara-201309071*.html
26a27,29
> <script type='text/javascript'>
> _gaq.push(["_trackEvent","Unsubscription","Sent","listname"]);
> </script>
90a94,96
> <div class="alert-message message notice">
> <p>You should have received a confirmation message</p>
> </div>
Which are the two missing pieces which make the spec failing, so the form is submitted, controller action is run successfully but there are two missing pieces of dom. How that is possible? And why this is happening only on one machine?
For the records, those two pieces of DOM are added with standard rails tools one with
redirect_to unsubscribe_index_path, notice: ...
and the other with the analytical gem
I've found the issue, the two failing projects I'm using dalli_store as session store and I've put the config.cache_store = :dalli_store line in config/application.rb instead of config/environments/production.rb.
On the old CI server there was a memcached daemon running hence all specs was running fine.
In the new server since it's just a CI server and it doesn't run any production or staging code memcached is not present thus any session write (such as flash messages) was silently discarded and this is the reason why all that kind of specs was failing.
I solved by putting the config.cache line in the appropriate environment file, but I'm still wondering why dalli gem doesn't raise any warning when no memcached is available. While the choice of not failing on missing cache daemon is reasonable since the application should work with no cached data, it could be a performance killer in production code and it might go unnoticed if no warning is given.
I have a rails 2.3 application running on REE. I am using capybara 1.1.2, cucumber 1.1.0, poltergeist (used both 0.6.0 and lastest from github) and phantomJS 1.5.0. My problem is that I believe I have a race condition in phantomJS that is causing my cucumber tests to randomly fail. The cucumber test work perfectly when I run them in Chrome so I am pretty sure the problem is in poltergeist or phantomJS. When I run the cucumber tests in PhantomJS (so that I can run them on the CI server) I will randomly get errors like this:
One or more errors were raised in the Javascript code on the page:
TypeError: 'undefined' is not a function
http://static.ak.fbcdn.net/rsrc.php/v2/yd/r/c9z-2BUoufJ.js:75
http://static.ak.fbcdn.net/rsrc.php/v2/yd/r/c9z-2BUoufJ.js:75
http://static.ak.fbcdn.net/rsrc.php/v2/yd/r/c9z-2BUoufJ.js:9 in bound:(?) (Capybara::Poltergeist::JavascriptError)
./features/step_definitions/site_steps.rb:8:in `/I am on the home page/'
features/account_creation.feature:89:in `Given I am on the home page'
Failing Scenarios:
cucumber features/account_creation.feature:5 # Scenario: I should be able to start creating a resume without signing up
cucumber features/account_creation.feature:79 # Scenario: I should be able to log in with an old username and password
cucumber features/account_creation.feature:88 # Scenario: I should be able to remember my login so that when the browser is closed I am automatically logged in
Based off of the Github page for poltergeist I have added this to my env.rb to solve the rack pre version 1.3 race condition problems:
Capybara.server do |app, port|
require 'rack/handler/thin'
Thin::Logging.silent = true
Thin::Server.new('0.0.0.0', port, app).start
end
But this has not made any difference that I can tell. Any thoughts would be appreciated even if it is just a suggesting that this is a poltergeist problem or a phantomJS problem so I know how I should ask about it next.
I have an application with a bunch of cucumber tests, and I tried using selenium with it today. It took a bit, but I got it running, it starts up selenium, uses the correct database and everything, but everybody has a wrong password.
I can see the user getting created, I can log the password and see that it's the correct password, but every test user just gets redirected back to the login page with the incorrect username/password error.
Is this a Selenium problem, or some other dependent system or am I missing something? It works fine with the standard :rails webrat mode.
I run the task as
rake features
Ubuntu 9.04
machinist 0.3.1
Selenium 1.1.14
Firefox 3.0.10
ruby 1.8.7-p160
cucumber 0.3.7
EDIT: Now, after a restart (office reorg) it won't run tests at all. Firefox just starts, and shuts down, then everything fails.
14:13:25.141 ERROR - Failed to start new browser session, shutdown browser an clear all session data
org.openqa.selenium.server.RemoteCommandException: timed out waiting for window 'null' to appear
I've tried updating the selenium-server.jar from the latest version, and no dice.
Cucumber::Rails.use_transactional_fixtures
Was the culprit. In selenium you can't have transactional fixtures, so writing cucumber steps will be a bit more fun, but whatever. Commenting out the above line fixes the password problem.