Originally posted this on the capybara google group, but I think this is a more appropriate place since it may be more poltergeist related.
I am having some performance issues when using poltergeist/phantomjs for headless browser testing.. The specs below log in a user and hit a page w/ very little load, but take a very long time to run. Using poltergeist as the driver more than doubles the time it takes to run the spec compared to using selenium or chrome as the driver. I would expect the opposite to be true: that using poltergeist as the driver would decrease test times since the browser is headless.
Using: phantomjs 1.9.7, capybara 2.3.0, poltergeist 1.5.1, rspec 2.99, rails 4.1.4
I get the following results:
# ~ 13 seconds
scenario "with js - poltergeist", :js => true, driver: :poltergeist do
login_as user
visit work_on_stories_path
expect(true).to be_true
end
# ~ 5 seconds
scenario "with js 2 - selenium", :js => true, driver: :selenium do
login_as user
visit work_on_stories_path
expect(true).to be_true
end
# ~ 5 seconds
scenario "with js 3 - chrome", :js => true, driver: :chrome do
login_as user
visit work_on_stories_path
expect(true).to be_true
end
# ~ < .5 seconds
scenario "without js" do
login_as user
visit work_on_stories_path
expect(true).to be_true
end
Are these normal results? Are there ways to improve them? What are common pitfalls that could result in these types of test run times?
Thanks.
Related
A Rails app that uses remote testing to test a CakePHP app running on a Vagrant Ubuntu VM.
My OS is macOS High Sierra.
'rspec/rails'
'capybara/rails'
'capybara/mechanize'
'capybara/poltergeist'
'phantomjs'
If I run rspec ./spec/features/my_tests_folder/,
the first 2 tests in the folder always pass and the rest always end up with Capybara::Poltergeist::TimeoutError:.
If I run any of the tests in that folder individually, they ALL pass ALWAYS.
There are 7 test files total. They each have 1 feature with 1 scenario. All are js: true.
I have tried increasing :timeout in Capybara.register_driver and increasing default_max_wait_time in Capybara.configure. Neither changed the outcome.
I've also played around with Capybara.reset! after and before each test. It didn't seem to matter, either.
When I ran this with config.order = :random, sometimes 5 out of 7 had the erros, sometimes only 2 out of 7. But, there were always some errors and some passing. Also, every test was in the errors group at least once.
I'm running out of ideas. What could cause something like this?
UPDATE (to include Capybara and Poltergeist configs and example of a failing test):
Configs in rails_helper.rb:
Capybara.register_driver :poltergeist do |app|
options = {
:timeout => 90, # default is 30
:js_errors => false,
:phantomjs => Phantomjs.path,
# :debug => true
}
Capybara::Poltergeist::Driver.new(app, options)
end
Capybara.register_driver :mechanize do |app|
driver = Capybara::Mechanize::Driver.new(app)
driver.configure do |agent|
agent.user_agent_alias = 'Mac Safari'
end
driver
end
Capybara.configure do |config|
config.run_server = false
config.app_host = "http://my_vm_domain.com"
config.javascript_driver = :poltergeist
config.default_driver = :mechanize
config.default_max_wait_time = 10 # default is 2
end
Example of failing test (not failing, but getting Capybara::Poltergeist::TimeoutError:):
require 'rails_helper'
feature 'agente visualiza estoques de um passeio', js: true do
scenario 'com sucesso' do
agente_log_in
visit '/roteiro/show/723'
find(:partial_href, 'new_passeio/723').click
select 'Passeio Teste', from: 'passeio_produto'
fill_in 'passeio_data', with: '11/11/2017'
within('button#estoque_controls_0') do
within('div#estoque_0_hora') do
expect(page).to have_content('07:30')
end
within('span#estoque_0_vagas') do
expect(page).to have_content('3')
end
end
within('button#estoque_controls_1') do
within('div#estoque_1_hora') do
expect(page).to have_content('10:00')
end
within('span#estoque_1_vagas') do
expect(page).to have_content('5')
end
end
end
end
Code from agente_log_in.rb in support folder:
def agente_log_in
Capybara.app_host = "http://my_vm_domain.com"
visit '/usuario/logout'
visit '/'
fill_in 'data[AdmUsuario][usuario]', with: 'agente'
fill_in 'data[AdmUsuario][senha]', with: 'pa$$w0rd'
click_on 'Entrar'
end
Code for that :partial_href find:
module Selectors
Capybara.add_selector(:partial_href) do
xpath {|href| XPath.descendant[XPath.attr(:href).contains(href)] }
end
end
Everything is fine with the other tests that are in the app's other folders. They're also fine if I run the tests in this folder individually. The problem only seems to happen when I run THIS specific folder as a whole.
After including the extra information requested by Thomas Walpole, I continued searching and studying possibilities.
I eventually came across poltergeist's issue #781 on GitHub, which describes a situation very similar to mine, and eventually presents a wait_for_ajax solution.
Before implementing it in my project, I read more about waiting for Ajax and found this post very helpful, too.
In the end, I chose jasonfb's code from GitHub because it seemed more thorough and informative.
It worked like a charm! My tests now pass consistently. I was even able to remove customization for :timeout and default_max_wait_time.
The CakePHP app in question is very js heavy and the specific part that this folder tests is particularly full of Ajax requests.
I'm attempting to set up system tests with Capybara and Selenium on an existing Rails 5.1 (Upgraded from Rails 4) app that already had capybara based feature tests. Here's what I've done so far.
In the gem file under group :development, :test:
gem 'chromedriver-helper'
gem 'selenium-webdriver'
gem 'rack_session_access'
In the environments/development.rb and environments/test.rb:
config.action_mailer.default_url_options = { host: 'localhost:3000' }
In the spec\rails_helper.rb:
Capybara.register_driver :selenium do |app|
Capybara::Selenium::Driver.new(app, browser: :chrome)
end
Capybara.configure do |config|
config.default_max_wait_time = 10 # seconds
config.default_driver = :selenium
config.app_host = 'http://localhost:3000'
config.server_host = 'localhost'
end
The issues I'm having are both with new systems tests and old feature tests.
With the system tests it appears that Capybara isn't creating a page object as I get undefined local variable or method 'page' Additionally when I duplicate the same test under the feature test directory I don't have this issue.
With the old Capybara feature tests, working with the rackTest driver, a Chrome window opens but I get No route matches [GET] "/rack_session/edit"
config.middleware.use RackSessionAccess::Middleware is already present in the environments/test.rb
Example system test:
require 'rails_helper'
describe User do
let(:user) { create :user }
let(:membership) { create :membership, admin: true}
let(:admin) { create :user, memberships: [membership] }
context 'viewing the index' do
it 'directs you to the appropriate page' do
set_current_user(admin)
visit root_url
click_button 'Manage Users'
expect(page.current_url).to end_with users_path
expect(page).to have_selector 'h1', text: 'Users'
end
end
end
Example feature test:
require 'rails_helper'
describe 'edit an assignment' do
let(:roster) { create :roster }
let(:user) { create :user, rosters: [roster] }
before :each do
Timecop.freeze Date.new(2018, 1, 10)
set_current_user(user)
end
after :each do
Timecop.return
end
context 'returns the user to the appropriate index page' do
let(:date_today) { Date.new(2017, 4, 4) }
let(:start_date) { Date.new(2017, 3, 31) }
let(:month_date) { date_today.beginning_of_month }
it 'redirects to the correct URL' do
visit roster_assignments_url(roster, date: date_today)
visit new_roster_assignment_url(roster, date: start_date)
click_button 'Create'
expect(current_url)
.to eq roster_assignments_url(roster,
date: month_date)
end
In the spec_helper:
def set_current_user(user)
page.set_rack_session user_id: user.id
end
You need to have the gem puma installed. New rails 5 projects have it installed by default, but your app was made in Rails 4, which is why it didn't.
Why is this? Well, if you were to do a bundle update (and I'll admit I can't explain why) you'd get this error when trying to run the specs, which is a lot more explanatory:
System test integration requires Rails >= 5.1 and has a hard dependency on a webserver and `capybara`, please add capybara to your Gemfile and configure a webserver (e.g. `Capybara.server = :webrick`) before attempting to use system tests.
Googling this error led me to this page, which explains Capybara needs a server.
After adding puma, I'm able to run system tests on your application.
A number of issues here.
Why are you setting - config.app_host = 'http://localhost:3000' ??? That would run the tests against your dev instance rather than the test instance Capybara starts. app_host really should only ever need to be set if you are doing subdomain based testing. This could be the reason for the no route error (normally rack_session_access would only be included in the test environment), or that could be caused by having not actually included the middleware as specified in the rack_session_access gem readme.
NEVER do expectations against current_path/current_url directly, instead use the provided matchers or you'll have flaky tests
expect(page).to have_current_path(users_path)
page is just an alias for Capybara.current_session and is a member of the module Capybara::DSL. If it's not available in the scope of your tests that it most likely means Capybara::DSL isn't included. That would normally be done by rspec-rails - https://github.com/rspec/rspec-rails/blob/master/lib/rspec/rails/vendor/capybara.rb#L21 - so it's possible you haven't actually set the test type to 'system'. If it's that it's not available in your spec_helper methods, just using Capybara.current_session instead is usually easier.
I have below spec code for testing user login feature.
feature "User login" do
context "using browser", :js => true do
before(:each) do
visit "/"
first(:link, "Login", visible: :any).click
end
scenario "with valid details" do
...
# rest of code.
end
end
end
It is working fine in my local machine with headless chrome. Here is my Capybara javascript_driver config in spec_helper.rb.
Capybara.register_driver :headless_chrome do |app|
options = Selenium::WebDriver::Chrome::Options.new(
args: %w[headless disable-gpu no-sandbox]
)
Capybara::Selenium::Driver.new(app, browser: :chrome, options: options)
end
Capybara.javascript_driver = :headless_chrome
But not working with gitlab-ci, below is error i am getting.
Failure/Error: first(:link, "Login", visible: :any).click
NoMethodError: undefined method 'click' for nil:NilClass
./spec/features/user_login_spec.rb:6:in 'block (3 levels) in '
/usr/local/bundle/gems/rspec-retry-0.5.6/lib/rspec/retry.rb:115:in
'block in run'
/usr/local/bundle/gems/rspec-retry-0.5.6/lib/rspec/retry.rb:104:in
'loop'
/usr/local/bundle/gems/rspec-retry-0.5.6/lib/rspec/retry.rb:104:in
'run'
/usr/local/bundle/gems/rspec-retry-0.5.6/lib/rspec_ext/rspec_ext.rb:12:in
'run_with_retry'
I found this blog to use headless chrome but nothing helped.
Here are the versions of dependencies i am using.
google-chrome-stable (63.0.3239.108-1)
chromedriver (2.34)
By default first doesn't have waiting/retrying behavior, which means if a matching link doesn't exist when it runs it won't wait for one to appear. Additionally passing visible: :any to find an element that you're then going to call click on doesn't make any sense since you can't click on non-visible elements so that would cause an error too. If you do actually need/want to use first (multiple matching links that you can't scope down to one) then you should be using
before(:each) do
visit "/"
first(:link, "Login", minimum: 1).click
end
The minimum: 1 option will trigger first to wait/retry up to Capybara.default_max_wait_time seconds for a matching link to appear on the page. This is one of the reasons using first and all are generally
bad choices when looking for elements to actually interact with (unless you've done a previous find for something that guarantees the page is in a stable state, or use any of the count options to trigger waiting/retrying behavior). If there aren't actually more than one matching link on the page then you should just be using
before(:each) do
visit "/"
click_link('Login') # same as find(:link, 'Login').click
end
After adding selenium-webdriver gem, when started capybara test, browser started to show home page and after it, refreshed with the blank page.
It cause to fail my tests that previously passed. Showing Capybara::ElementNotFound error.
I use capybara 2.2.1; selenium 2.42.0; ruby 2.1.0; rails 4.1.0; rspec 3.0
Here is my test:
require 'spec_helper'
describe "Navigation" do
it "changing active menu element depending on route", js: true do
visit '/'
first(:link, 'WHAT IS DREAMDO?').click
expect(find('.navigation').find('.active').find('a').text).to eq('WHAT IS DREAMDO?')
first(:link, 'DISCOVER DREAMS').click
expect(find('.navigation').find('.active').find('a').text).to eq('DISCOVER DREAMS')
first(:link, 'DREAMDO WEEKLY').click
expect(find('.dreamdo-menu-items').find('.active').find('a').text).to eq('Dreamdo Weekly')
end
end
describe "Search", js: true do
it "checks for the availability and search results" do
search_text = find(:xpath, '//input[#id="search_text"]').set("chocolate")
search_text.native.send_keys :return
page.should have_content "chocolate"
end
end
Could you, please, recommend me what to do?
Actually, it is not good to use selenium, because it can't run multiple instances:
http://robots.thoughtbot.com/capybara-webkit
Try to use capybara-webkit, hope it will solve your problem.
Solution
This worked. The main essence is that I have to set the Capybara.server_port and Capybara.app_host and sign in manually in the sign in form. Capybara.app_host cannot be set with a dynamic subdomain unless its declared in a variable. All urls has to be hard coded.
require 'spec_helper'
feature 'customer' do
let(:user) {FactoryGirl.create(:user)}
let(:firm) {user.firm}
let(:customers) {"http://#{firm.subdomain}.lvh.me:31234/customers"}
let(:root_url) {"http://#{firm.subdomain}.lvh.me:31234/"}
before(:all) do
Capybara.server_port = 31234
sub = firm.subdomain
Capybara.app_host = root_url
end
def sign_in_on_js
visit root_url
fill_in "Email", :with => user.email
fill_in "Password", :with => "password"
click_button "Sign in"
page.should have_content("Signed in successfully.")
end
scenario "make new", js: true do
sign_in_on_js
visit customers
page.should have_content("Add new customer")
find("#dialog_customer").click
page.should have_content("Create new customer")
end
end
Original question
I am making a multitenant app in rails. There is going to be a lot of javascript. But, I cant get the testing to work.
When not running :js = true every thing works. The problem arises in specs like this one
let(:customers) {"http://#{firm.subdomain}.lvh.me:3003/customers"}
scenario "Statistics select", :js => true do
visit customers
page.should have_content("Add new customer")
end
The poltergeist web driver for capybara cannot find the url and returns a blank page
Failure/Error: page.should have_content("Add new customer")
expected there to be text "Add new customer" in ""
I have this in my spec_helper.rb
require 'capybara/rspec'
require 'capybara/poltergeist'
Capybara.javascript_driver = :poltergeist
Capybara.register_driver :poltergeist do |app|
Capybara::Poltergeist::Driver.new(app, :debug => true)
end
Poltergeist and phantomjs try to deliver. I get this output
{"name"=>"set_debug", "args"=>[true]}
{"response"=>true}
{"name"=>"visit", "args"=>["http://subdomain2.lvh.me:3003/statistics"]}
poltergeist [1362522132943] state default -> loading
{"response"=>{"status"=>"fail"}}
Do I need to have a server running during testing to make this work?
I've tried selenium and capybara-webkit, but phantomjs has gotten closest to success.
I have also tried to change the hosts file in different ways( maybe not correct )
Any tips on setup are welcome!
Update
Starting to get desperate. I now start the rails server
rails s -e test -p 3001
and then run my tests.
Now I get redirected to the sign in page. I have this in the specs
before(:each) do
login_as(user, :scope => :user)
end
How can I sign in the test user on the rails test server without going trough the sign in process manually for every spec
Capybara already starts a server for you, to quote the docs:
Some Capybara drivers need to run against an actual HTTP server. Capybara takes care of this and starts one for you in the same process as your test, but on another thread. Selenium is one of those drivers, whereas RackTest is not.
Within your test you can use the visit method with a relative url, for example:
visit("/statistics")
Capybara will direct this request to the server it just started for this test.
When your want to use an absolute url within your test, you can, but you should also specify the port the server is running on. This port is being randomly chosen during the test. Some drivers have a method available to retrieve the port number.
For example when you use the Capybara-Webkit driver:
Capybara.current_session.driver.server_port
To visit an absolute url you can then use:
port_number = Capybara.current_session.driver.server_port
visit("http://127.0.0.1:#{port_number}/statistics")
Within the test specs probably a method login_as won't work. You have to log in with a few simple steps. For example:
before(:each) do
visit "/login"
fill_in "Email", :with => "my#email.com"
fill_in "Password", :with => "secret"
click_button "Login"
end
To test multiple subdomains you can set the Capybara.app_host. Take a look at this question for a detailed explanation.
UPDATE
Capybara 2 includes a nice feature called always_include_port which will automatically add the port number the server is running on.
Capybara.always_include_port = true
So instead of
visit("http://127.0.0.1:#{port_number}/statistics")
you can now use
visit("/statistics")
and it will automatically connect to http://127.0.0.1:#{port_number}/statistics.
If you want to test multiple subdomains with Capybara.app_host, you could use a domain name which always resolves to 127.0.0.1 for example lvh.me.
For example, if you specify Capybara.app_host = "http://example.lvh.me" it will run the tests using the example subdomain.