Erratic behavior of Vue.js application in test mode (rspec) - ruby-on-rails

I have a Rails 6 application which I test with rspec, Capybara and Chrome headless on a remote VM. With the new webdrivers gem, not that ancient poltergeist thing.
It has an user manager mini-app written in Vue 2.something that behaves in some stupefying ways:
Excerpt from Vue application
{
el: "#app",
data: {
initial_load_completed: false,
users: []
},
created: function(){
this.loadUsers();
},
methods: {
loadUsers: function(){ /* straightforward JSON load from server into .users and set .initial_load_completed to true */ }
/* lots of other code */
},
computed: {
hasUsers: function(){
return this.users.length > 0;
}
/* lots of other code */
}
}
View excerpt
<div id="app">
<!-- loads of other code -->
<div v-if="!initial_load_completed && !hasUsers">Loading your users, please wait...</div>
<div v-if="initial_load_completed && !hasUsers">There are no users for your account right now...</div>
<!-- lots of other code -->
</div>
The application works perfectly in prod and dev, on chrome, safari, tablets, iphones, even on my 3 year old smart TV Trashroid, even on IE. But under rspec tests it does things such as this:
This example with those 2 divs showing/hiding based on users loaded is just a small thing that's wrong in this picture, many other controls were supposed to not show with an empty users array. And this is a happy happy joy joy case, about 50% of example runs it just doesn't output anything at all, #app is blank... randomly.
In my test.log I see how the Vue app hits the JSON endpoint of my back-end and how it renders data with a 200.
For the life of me I can't imagine how initial_load_completed can be true and false at the same time.
What I've tried?
Rebooted the machine (heh). Then reinstalled all software to latest versions.
Then spent about 2 days trying to get chrome to work on a "virtual" display to which I would connect to see what's going on... after some 218 iterations fixing various deps/errors and configurations and code and signs and more errors and so on I just gave up.
Driver definition:
Webdrivers.logger.level = :DEBUG
default_chrome_args = [ '--disable-extensions', '--no-sandbox', '--disable-dev-shm-usage', '--remote-debugging-port=9222', '--remote-debugging-address=0.0.0.0' ]
Capybara.register_driver :headless_chrome do |app|
capabilities = Selenium::WebDriver::Remote::Capabilities.chrome( loggingPrefs: { browser: 'ALL' }, chromeOptions: {
'args' => default_chrome_args + ['--headless', '--disable-gpu', '--window-size=1920,1600' ]
})
Capybara::Selenium::Driver.new app, browser: :chrome, desired_capabilities: capabilities
end
CSP's are disabled, tried with and without them anyway.
Yesterday I tried logging JS errors:
config.after(:each, type: [ :feature, :system ], js: true) do
errors = page.driver.browser.manage.logs.get(:browser)
if errors.present?
aggregate_failures 'javascript errrors' do
errors.each do |error|
expect(error.level).not_to eq('SEVERE'), error.message
next unless error.level == 'WARNING'
STDERR.puts 'WARN: javascript warning'
STDERR.puts error.message
end
end
end
end
... no luck.
config.after(:each, type: [ :feature, :system ], js: true) do
errors = page.driver.browser.manage.logs.get(:browser)
errors.each do |error|
STDERR.puts error.message
end
end
... also nada just like several other few variations of this code.
Can't even seem to get the examples to "puts :whatever" to stdout but that's another story.
Can someone kind at heart pretty please help a poor dumb me not lose all hair?

Something that is not clear from the code samples in your question, is whether you are actually applying the driver you are defining.
System tests will use the default driver, so you must set it explicitly:
RSpec.configure do |config|
config.before(:each, type: :system) {
driven_by :headless_chrome
}
end
It can also be applied on a per-scenario basis on feature tests:
RSpec.feature 'Balance' do
scenario 'check the balance', driver: :headless_chrome do
...
end
end

Related

Capybara ends up with inconsistent results on websockets

I am experiencing flaky action cable tests on capybara backed by cuprite(headless mode). Basically, I am creating a post using action cable and setting it on React using Mobx. I ran the test 150 times in a loop and it failed 30 times. What can cause this inconsistent failure?
If I make the driver go to another page or reload the same page the post appears as expected.
The settings are as follow:
spec/rails_helper.rb
require 'rails_helper.rb'
Dir[File.join(__dir__, "system/support/**/*.rb")].sort.each { |file| require file }
module CupriteHelpers
# some test helpers
end
RSpec.configure do |config|
config.include CupriteHelpers, type: :system
end
spec/system/support/capybara.rb
Capybara.default_max_wait_time = 10
Capybara.default_normalize_ws = true
Capybara.save_path = ENV.fetch("CAPYBARA_ARTIFACTS", "./tmp/capybara")
Capybara.singleton_class.prepend(Module.new do
attr_accessor :last_used_session
def using_session(name, &block)
self.last_used_session = name
super
ensure
self.last_used_session = nil
end
end)
spec/system/support/curprite.rb
Capybara.register_driver(:cuprite) do |app|
Capybara::Cuprite::Driver.new(
app,
**{
window_size: [1440, 900],
browser_options: {},
process_timeout: 60,
timeout: 15,
inspector: ENV['INSPECTOR'] == 'true',
headless: !ENV['HEADLESS'].in?(%w[n 0 no false]),
slowmo: (0.2 if ENV['HEADLESS'].in?(%w[n 0 no false])),
js_errors: false,
logger: FerrumLogger.new
}
)
end
Capybara.default_driver = Capybara.javascript_driver = :cuprite
spec/system/post_spec.rb
it 'can create a new post and the creator is the user do
click_button 'Add New Post +'
expect(page).to have_css('#post-2')
# rest of the tests but the line above fails
end

Capybara and Ajax request on document ready

So I have script on page:
$(document).ready ->
$.rails.ajax
type: 'POST'
url: url
data:
request:
'some data': data
success: (response) ->
do something
And I have feature test:
RSpec.describe 'some test', type: :feature, js: true do
it 'tests' do
visit '/'
page.select 'some value', from: 'id'
click_on('Some button')
expect(page).to have_content 'some text'
end
Also some config:
Capybara.register_driver :chrome do |app|
Capybara::Selenium::Driver.new(app, browser: :chrome)
end
Capybara.javascript_driver = :chrome
How to make capybara run those script that should run on document ready? It seems that id doesn't run now.
There is no need to tell Capybara to run a script in the page. If the script is in the page then Chrome will run it (assuming no JS errors, etc).
Since you don't indicate which line the test fails on, or what the exact failure message is, it's tough to tell exactly what the ajax request is loading to the page and whether you're expecting that code to run on the page returned by visit '/' or on the page directed to by clicking on the button. If the former then you need to have an expectation for whatever visible change to the page happens when the ajax request succeeds. If the latter then most likely you just have Capybara.default_max_wait_time set too low for the hardware you're testing on (increase it to 5 or 10).

Rspec with Capybara sometimes not pass tests

I have problem with testing Rails Application. My tests generally work perfectly. But sometimes tests will fail when I type some features test for modal bootstrap window, or notify with success/error [js]. How can I resolve this problem ?
I'm using Rspec, Capybara, Rails4.2, PhantomJs, Poltergeist as JS driver. Tests is running locally and in Wercker. In test mode, every bootstrap animation is disabled. What perhaps I do wrong ?
Test:
scenario 'return deutsch default title' do
find('.f-edit-item', match: :first).click
find('a', :text => 'Lang').click
find('a', :text => t('menu.languages.de')).click
find('.f-reset-button', match: :first).click
expect(page).to have_field('menu_item[title]', with: 'Exhibitions_de')
end
Output:
Objects Restore Language restore title translations exist for deutsch translation return deutsch default title
Failure/Error: expect(page).to have_field('object_item[title]', with: 'Exhibitions_de')
expected to find field "object_item[title]" with value "Exhibitions_de" but there were no matches. Also found "", "", which matched the selector but not all filters.
When I click manually, everything is working. When I run this test, sometimes passed, sometimes not. Form is in bootstrap modal. Curiosity: When I add save_and_open_page before find('.f-reset-button', match: :first).click test is passed always(5x in a row)
Because the tests are to do with a Bootstrap modal, my guess is that the test is searching the page for the matching elements, BEFORE the modal has loaded in the DOM.
Edit: As #TomWalpole pointed out, it should be enough to override Capybara's max wait time like so:
expect(page).to have_field('menu_item[title]', with: 'Exhibitions_de', wait: 1.0)
But if you are loading the contents of your modal via AJAX, you may need to force a wait for AJAX to complete the expect line. Here is a good guide on how to do this.
Specifically you need:
# spec/support/wait_for_ajax.rb
module WaitForAjax
def wait_for_ajax
Timeout.timeout(Capybara.default_wait_time) do
loop until finished_all_ajax_requests?
end
end
def finished_all_ajax_requests?
page.evaluate_script('jQuery.active').zero?
end
end
RSpec.configure do |config|
config.include WaitForAjax, type: :feature
end
And then your test would become:
scenario 'return deutsch default title' do
find('.f-edit-item', match: :first).click
find('a', :text => 'Lang').click
find('a', :text => t('menu.languages.de')).click
find('.f-reset-button', match: :first).click
wait_for_ajax
expect(page).to have_field('menu_item[title]', with: 'Exhibitions_de')
end

rails wisper under test

I have a project which used wisper https://github.com/krisleech/wisper to provide publisher and subscribers functionalities.
The gem works perfectly under development and production modes. However, when I try to add some tests for them (rake test:integration), the newly added tests refused to work. The publisher (maybe also the listener) in the tests mode stopped working anymore.
Core::Request.subscribe(Listener::Studentlistener, async: true)
Core::Request.subscribe(Listener::Tutorlistener, async: true)
I used the sidekiq as a async backend, i used wisper-sidekiq gem to handle the async requests, not sure if this would be the problem?
,puma as the server, MRI ruby 2.0.0
Do I have to a set up something in order for the test to run?
it "Student can get latest status after looking for xxx tutor" do
post api_v1_students_request_look_for_xxx_tutor_path,
{ subject: 'nothing' },
{ "AUTHORIZATION" => "xxx"}
expect(response).to be_success
get api_v1_students_status_path, nil,
{ "AUTHORIZATION" => "xxx"}
expect(response).to be_success
json_response = JSON.parse(response.body)
expect(json_response['state']).to eq('matching')
end
The listener should receive the publishing between these two posts and update the state to be "matching". However, now when I run rspec the test failed because the publisher never publish anything and hence the state is not updated correctly.
Even the authors are relying on some mocking/stubbing in the integrations tests, so that might be the correct way.
class MyCommand
include Wisper::Publisher
def execute(be_successful)
if be_successful
broadcast('success', 'hello')
else
broadcast('failure', 'world')
end
end
end
describe Wisper do
it 'subscribes object to all published events' do
listener = double('listener')
expect(listener).to receive(:success).with('hello')
command = MyCommand.new
command.subscribe(listener)
command.execute(true)
end
https://github.com/krisleech/wisper/blob/master/spec/lib/integration_spec.rb

capybara waiting for ajax without using sleep

I'm using Capybara 2.x for some integration tests for a large Rails/AngularJS app and I've come across a test in which I need to put a sleep to get it working.
My test:
describe "#delete", js: true do
it "deletes a costing" do
costing = Costing.make!
visit "/api#/costings"
page.should have_content("General")
click_link "Delete" # Automatically skips the confirm box when in capybara
sleep 0.4
page.should_not have_content("General")
end
end
The code it tests is using ng-table which takes a split second to update, without that sleep it will fail. Capybara used to have a wait_until method for this but it's been taken out. I found this website: http://www.elabs.se/blog/53-why-wait_until-was-removed-from-capybara but cannot get any of the recommended alternatives working for this problem.
Here is the code I'm testing.
# --------------------------------------------------------------------------------
# Delete
# --------------------------------------------------------------------------------
$scope.destroy = (id) ->
Costing.delete (id: id), (response) -> # Success
$scope.tableParams.reload()
flash("notice", "Costing deleted", 2000)
This updates the ng-table (tableParams variable) which is this code
$scope.tableParams = new ngTableParams({
page: 1,
count: 10,
sorting: {name: 'asc'}
},{
total: 0,
getData: ($defer, params) ->
Costing.query {}, (data) ->
# Once successfully returned from the server with my data process it.
params.total(data.length)
# Filter
filteredData = (if params.filter then $filter('filter')(data, params.filter()) else data)
# Sort
orderedData = (if params.sorting then $filter('orderBy')(filteredData, params.orderBy()) else data)
# Paginate
$defer.resolve(orderedData.slice((params.page() - 1) * params.count(), params.page() * params.count()))
})
Try bumping the Capybara.default_wait_time up to 3 seconds or 4.
If that fails, try changing the spec to look for the flash notice message before it checks to see if the item has been removed from the page. (Assuming the flash message gets rendered in the HTML body)
describe "#delete", js: true do
it "deletes a costing" do
costing = Costing.make!
visit "/api#/costings"
page.should have_content("General")
click_link "Delete"
page.should have_content("Costing deleted")
page.should_not have_content("General")
end
end
Edit - removed explanation because it was incorrect.

Resources