About 3 in 10 times, this spec will fail because fill_in fails to fill in all of the characters of the user[username] field.
uuid = SecureRandom.uuid
fill_in "user[username]", :with => uuid
Notes:
Sometimes it succeeds
Sometimes it drops a random number of characters, usually somewhere after index 29
never fails locally on OSX
failures happen remotely on Solano aka tddium
Does anyone know what is going on?
This is my configuration:
capybara (2.4.4)
selenium-webdriver (3.0.0)
ChromeDriver 2.25.426924 (649f9b868f6783ec9de71c123212b908bf3b232e) (very recent)
using Chrome, not headless:
And in spec_helper:
Capybara.register_driver :selenium do |app|
http_client = Selenium::WebDriver::Remote::Http::Default.new
http_client.timeout = 100
Capybara::Selenium::Driver.new(app, :browser => :chrome, :http_client => http_client)
end
#thomas-walpole probably is pointing out the root cause - a change in chromedriver that was not compensated for by a change in capybara as we had pegged that gems version number at 2.4.4.
I upgraded capybara to the latest version, 2.10.1, fixed all the necessary deprecations, and retested.
Somewhere in there, it seems to work now. At least, where as I used to get 3 failures in 10 tries, now it works consistently for at least 20 repetitions.
Related
Specs: Rails 5.2, Ruby 2.5.1, Yarn 1.91.1, React-Rails 2.6.0, selenium-webdriver 3.14, chromedriver 77, capybara gem 3.29 , Ubuntu 18.04
Issue: I created a questionnaire using react components in a Rails app which works when run in development mode. I can click the yes or no button then a callback function renders the new question text. However my selenium integration tests don't pick this up. The page object continues to have the same text for the first question so I receive an error like this
Failure/Error: expect(question.text).to include('last question')
expected "first question" to include "last question"
The test itself looks like this in features/questionnaire.rb
RSpec.feature "Onboarding Questionnaire", type: :feature, js:true do
(...)
it...
question = find('h3.question')
expect(question.text).to include('first question')
yes = find('#yes')
yes.click
sleep 5
question = find('h3.question')
expect(question.text).to include('last question')
The problem arises after yes.click. I thought at first my animations were causing the issue, but I removed those and it is just using setState in my callback function.
Here is my react callback function
saveAnswer(e) {
questionnaire = this
++questionnaire.state.page
questionnaire.setState({page: questionnaire.state.page})
}
Here is my driver config
Capybara.javascript_driver = :selenium_chrome_headless
Now using sleep works as desired. Here are the changes I made.
1) I removed the deprecated gem chromedriver-helper and replaced it with webdrivers
2) Added require 'webdrivers' to the rails_helper.rb file.
3) Found error in my react code (which wasn't showing up in development or being logged in chrome webdriver) of questionnaire = this which I changed to var questionnaire = this.
Integration tests now pass fine using selenium_chrome_headless
I'm noticing a strange issue that I haven't been able to solve for a few days.
I have a Rails 5 API server with system tests using RSpec and Capybara + Selenium-webdriver driving headless Chrome.
I'm using Capybara.app_host = 'http://localhost:4200' to make the tests hit a separate development server which is running an Ember front-end. The Ember front-end looks at the user agent to know to then send requests to the Rails API test database.
All the tests run fine except for ones which use RSpec file fixtures.
Here's one spec that is failing:
describe 'the affiliate program', :vcr, type: :system do
fixtures :all
before do
Capybara.session_name = :affiliate
visit('/')
signup_and_verify_email(signup_intent: :seller)
visit_affiliate_settings
end
it 'can use the affiliate page' do
affiliate_token = page.text[/Your affiliate token is \b(.+?)\b/i, 1]
expect(affiliate_token).to be_present
# When a referral signs up.
Capybara.session_name = :referral
visit("?client=#{affiliate_token}")
signup_and_verify_email(signup_intent: :member)
refresh
# It can track the referral.
Capybara.session_name = :affiliate
refresh
expect(page).to have_selector('.referral-row', count: 1)
# When a referral makes a purchase.
Capybara.session_name = :referral
find('[href="/videos"]').click
find('.price-area .coin-usd-amount', match: :first).click
find('.cart-dropdown-body .checkout-button').click
find('.checkout-button').click
wait_for { find('.countdown-timer') }
order = Order.last
order.force_complete_payment!
Rake::Task['affiliate_referral:update_amounts_earned'].invoke
# It can track the earnings.
Capybara.session_name = :affiliate
refresh
amount = (order.price * AffiliateReferral::COMMISSION_PERCENTAGE).floor.to_f
amount_in_dom = find('.referral-amount-earned', match: :first).text.gsub(/[^\d\.]/, '').to_f * 100
expect(amount).to equal(amount_in_dom)
end
end
This will fail maybe 99% of the time. There is the odd case where it passes. I can get my test suite to eventually pass by running it on a loop for a day.
I ended up upgrading all versions to the latest (Node 10, latest Ember, latest Rails) but the issue persists.
I can post a sample repo that reproduces the issue later. I just wanted to get this posted in case anyone has encountered the issue.
Here's a typical stack trace when the timeout happens:
1.1) Failure/Error: page.evaluate_script('window.location.reload()')
Net::ReadTimeout:
Net::ReadTimeout
# /home/mhluska/.rvm/gems/ruby-2.5.1/gems/webmock-3.3.0/lib/webmock/http_lib_adapters/net_http.rb:97:in `block in request'
# /home/mhluska/.rvm/gems/ruby-2.5.1/gems/webmock-3.3.0/lib/webmock/http_lib_adapters/net_http.rb:110:in `block in request'
# /home/mhluska/.rvm/gems/ruby-2.5.1/gems/webmock-3.3.0/lib/webmock/http_lib_adapters/net_http.rb:109:in `request'
# /home/mhluska/.rvm/gems/ruby-2.5.1/gems/selenium-webdriver-3.14.0/lib/selenium/webdriver/remote/http/default.rb:121:in `response_for'
# /home/mhluska/.rvm/gems/ruby-2.5.1/gems/selenium-webdriver-3.14.0/lib/selenium/webdriver/remote/http/default.rb:76:in `request'
# /home/mhluska/.rvm/gems/ruby-2.5.1/gems/selenium-webdriver-3.14.0/lib/selenium/webdriver/remote/http/common.rb:62:in `call'
# /home/mhluska/.rvm/gems/ruby-2.5.1/gems/selenium-webdriver-3.14.0/lib/selenium/webdriver/remote/bridge.rb:164:in `execute'
# /home/mhluska/.rvm/gems/ruby-2.5.1/gems/selenium-webdriver-3.14.0/lib/selenium/webdriver/remote/oss/bridge.rb:584:in `execute'
# /home/mhluska/.rvm/gems/ruby-2.5.1/gems/selenium-webdriver-3.14.0/lib/selenium/webdriver/remote/oss/bridge.rb:267:in `execute_script'
# /home/mhluska/.rvm/gems/ruby-2.5.1/gems/selenium-webdriver-3.14.0/lib/selenium/webdriver/common/driver.rb:211:in `execute_script'
# /home/mhluska/.rvm/gems/ruby-2.5.1/gems/capybara-3.8.2/lib/capybara/selenium/driver.rb:84:in `execute_script'
# /home/mhluska/.rvm/gems/ruby-2.5.1/gems/capybara-3.8.2/lib/capybara/selenium/driver.rb:88:in `evaluate_script'
# /home/mhluska/.rvm/gems/ruby-2.5.1/gems/capybara-3.8.2/lib/capybara/session.rb:575:in `evaluate_script'
# ./spec/support/selenium.rb:48:in `refresh'
# ./spec/support/pages.rb:70:in `signup_and_verify_email'
# ./spec/system/payment_spec.rb:43:in `block (3 levels) in <top (required)>'
I should point out it doesn't always happen with page.evaluate_script('window.location.reload()'). It can happen with something benign like visit('/').
Edit: I tried disabling Ember FastBoot (server-side rendering) using the DISABLE_FASTBOOT env variable and suddenly all tests pass. I'm thinking that somehow the RSpec fixtures are causing Ember FastBoot to not finish rendering in some cases. This certainly lines up with dropped connections I've occasionally seen in production logs.
I've been experimenting with the client code and it may be due to my use of FastBoot's deferRendering call.
Edit: I'm using the following versions:
ember-cli: 3.1.3
ember-data: 3.0.2
rails: 5.2.1
rspec: 3.8.0
capybara: 3.8.2
selenium-webdriver: 3.14.0
google chrome: 69.0.3497.100 (Official Build) (64-bit)
Edit: I'm using this somewhat flaky Node/Express library fastboot-app-server to do server-side rendering. I've discovered that it sometimes strips important response headers (Content-Type and Content-Encoding). I'm wondering if this is contributing to the issue.
Edit: I added a strict Content Security Policy to make sure there are no external requests running during the test suite that could be causing the Net::ReadTimeout.
I inspect the Chrome network tab at the point when it locks up and it seems to be loading nothing. Manually refreshing the browser allows the tests to pick up and continue running. How strange.
I've spent a couple weeks on this now and it may be time to give up on Selenium tests.
I upgraded to Chrome 70 and chromedriver 2.43. It didn't seem to make a difference.
I tried using the rspec-retry gem to force a refresh when the timeout occurs but the gem seems to fail to catch the timeout exception.
I've inspected the raw request to chromedriver where things hang. It looks like it's always POST http://127.0.0.1/session/<session id>/refresh. I tried refreshing in an alternate way: visit(page.current_path) which seems to fix things!
I finally got my test suite to pass by switching page.driver.browser.navigate.refresh to visit(page.current_path).
I know it's an ugly hack but it's the only thing I could find to get things working (see my various attempts in the question edits).
I looked at the request to chromedriver that was causing the timeouts each time: POST http://127.0.0.1/session/<session id>/refresh. I can only guess that it's some kind of issue with chromedriver. Perhaps incidentally, it only hangs when multiple chromedriver instances are active (which happens when multiple Capybara sessions are being used).
Edit: I needed to account for query params as well:
def refresh
query = URI.parse(page.current_url).query
path = page.current_path
path += "?#{query}" if query.present?
visit(path)
end
I tried just doing visit(page.current_url) but that was giving timeouts as well.
I have a rails project, which I'm testing with rspec/capybara/poltergeist/phantomjs. I know I can increase the general poltergeist timeout with the general settings
Capybara.register_driver :poltergeist do |app|
Capybara::Poltergeist::Driver.new(app, timeout: 2.minutes)
end
But is there a way to increase the timeout for a specific request?
I have a page with a button (id=submit) which kicks off a longish (90-120 seconds) running process, before returning. I'm working on optimizing the back end to shorten the request time, but in the meanwhile, I want to increase the timeout for that specific request when testing, so something along the lines of
click_button 'submit', wait: 180
You can do
Capybara.using_wait_time(180) do
click_button 'submit'
end
Another thing you can do is
# capybara.rb
Capybara.register_driver :poltergeist do |app|
Capybara::Poltergeist::Driver.new(app, timeout: 30)
end
Capybara.register_driver :poltergeist_long do |app|
Capybara::Poltergeist::Driver.new(app, timeout: 180)
end
# wherever.rb
session = Capybara::Session.new(:poltergeist_long)
session.visit("http://thatlongwaittime.com")
Timeuouts for specific requests can be increased by increasing the value of default wait time which is generally configured in you env.rb file.
To understand this well lets take below mention code:
Cucumber file:
When Joe is on abc page
Then Joe clicks submit button
Step definition for clicking submit button:
Then(/^Then Joe clicks submit button$/) do
Capybara.default_wait_time = 120 // increasing the default wait time to 180 seconds
click_button('submit') // performing the action
Capybara.default_wait_time = DEFAULT_WAIT_TIME // reset the wait time to its default value after clicking submit button.
end
Note: The value of DEFAULT_WAIT_TIME can be configured in env.rb file
Hope this helps :)
I'm using Capybara and Poltergeist and cannot for the life of me get all my tests to consistently pass. I have this one issue in particular with a date selector. It should be really simple - user clicks on input, out puts a selection of months (first image). A month is then clicked, and then a day selection appears (second image), on which a day of the month is selected.
Now, my code looks as follows:
all(:css, 'input.from_date').last.click
expect(page).to have_css(".datepicker-months")
within(:css, '.datepicker-months') { find('.month', :text => 'Jun', match: :first).click }
expect(page).not_to have_css(".datepicker-months")
expect(page).to have_css(".datepicker-days")
within(:css, '.datepicker-days') { find('.day', :text => work[:start_date].stamp('31').to_i.to_s, match: :first).click } #.to_i.to_s used to remove leading zeros
page.assert_no_selector('.datepicker-days')
Sometimes it passes, but most of the time it says:
expected to find css ".datepicker-days" but there were no matches
or
expected not to find css ".datepicker-months", found 1 match: "« 2015 » JanFebMarAprMayJunJulAugSepOctNovDec"
If I try and debug this by calling binding.pry, I can run the commands step by step in the console and it works perfectly. My timeout is set to more than enough (i think). Any ideas why this test fails intermittently?
My config:
Capybara.javascript_driver = :poltergeist
Capybara.default_wait_time = 60
Capybara.register_driver :poltergeist do |app|
Capybara::Poltergeist::Driver.new(app, {
timeout: 60,
js_errors: false,
phantomjs_logger: File.open("log/phantomjs.log", "a")})
end
UPDATE:
When adding sleep(0.5) after every step in the process, it passes each and every time. This is bad practice and I'm supposed to be able to write tests without doing this. :/
Whenever I run into these inconsistencies, I try to find other things to assert against so that you don't have to count on sleep.
If there are any sort of animations that start/stop, you can try to assert against those.
You can also try to be more specific in your current expectation, since it seems like you had no trouble clicking on the object you found
expect(page).not_to have_css('.datepicker-months .month', :text => 'Jun')
If any of that gets you any farther let us know.
I have been writing rspec tests using capybara and selenium-webdriver. Almost without fail, whenever I run one of these tests, the console output is gone.
For instance:
~/code/code> bundle exec rspec spec/features/interactions_spec.rb
InteractionsSpec
~/code/code>
Thats just about all I ever see. The browser launches, performs the action's I've coded, but I don't see the usual output.
Sometimes I'll see this as output instead (yay!):
InteractionsSpec
login as admin works
login as user works
Finished in 16.84 seconds (files took 7.9 seconds to load)
2 examples, 0 failures
What could be causing the output from the tests to disappear? This makes it nearly impossible to write tests - as I have no idea what has run, what has passed, what has failed, or why they have failed.
I'm using these Gems, but performing a bundle update does not change the behavior.
capybara (2.4.4)
capybara-screenshot (1.0.3)
rspec (3.0.0)
rspec-activemodel-mocks (1.0.1)
rspec-collection_matchers (1.0.0)
rspec-core (3.0.3)
rspec-expectations (3.0.3)
rspec-mocks (3.0.3)
rspec-rails (3.0.2)
rspec-support (3.0.3)
selenium-webdriver (2.44.0)
shoulda-matchers (2.6.2)
Other involved software:
Firefox 33.1.1 (Chrome 39.0.2171.65 (64-bit) has the same behavior)
OSX 10.9.5
Rails 4.1.4
Update
This seemed to fix the problem for a while, even with a sleep of 1ms. However, that was only a temporary fix and this problem persists.
RSpec.configure do |config|
config.before(:each, :type => :feature) do
sleep(0.5)
end
end
Looks to me like the logger is getting silenced via a call to logger.silence() by some other thread.
When you put the sleep() in the code, then you yielded to some other thread that fixed the logging level - that's why it didn't matter how long the sleep was, it was the context switch that was fixing the issue.
When you upgraded the activesupport gem, this commit here https://github.com/rails/activerecord-session_store/commit/f92d1135fc620cb4d65239ef286b267945bbbbc6 fixes the issue (as you said). Reading that commit, notice a new threadsafe implementation of logger.silence. That's why it fixes your issue.
The old implementation of active_support/logger_silence.rb here isn't threadsafe:
require 'active_support/concern'
module LoggerSilence
extend ActiveSupport::Concern
included do
cattr_accessor :silencer
self.silencer = true
end
# Silences the logger for the duration of the block.
def silence(temporary_level = Logger::ERROR)
if silencer
begin
old_logger_level, self.level = level, temporary_level
yield self
ensure
self.level = old_logger_level
end
else
yield self
end
end
end
There seems to be a number of issues related to thread safety and race conditions in initializing the rails environment under capybara, which uses multiple threads. Rob's answer explains why it starts working (logging is fixed to be thread-save in activerecord-session_store).
It seems that these issues and commits are also related
https://github.com/rails/rails/issues/15089
https://github.com/jnicklas/capybara/issues/1419
https://github.com/jnicklas/capybara/commit/a4fc9d96f705e9ac4732c5fa05ab79d947743599