Using Capybara how to disable "wait_for_pending_requests" as it raise an error - capybara

I'm using Capybara with WebMock and a proxy (Sinatra) to test a remote app.
I'm not stubbing the requests but using WebMock to assert requests. I've assigned the proxy to capybara.app and added to the chrome driver so the requests will be forwarded to the proxy.
My problem is that sometimes I have pending requests which will raise the following error:
Failure/Error: raise "Requests did not finish in 60 seconds"
I wonder how can I disable this error?
Also how can I change the hard coded timeout which is 60 (which will block the continuation of the test anyway)
Timeout.timeout(60) { sleep(0.01) while #middleware.pending_requests? }
capybara.rb:
require 'capybara/rspec'
require 'capybara'
require 'capybara/dsl'
require_relative 'sinatra_proxy'
require 'selenium/webdriver'
require 'selenium/webdriver/remote/http/curb' if !isWindows
Capybara.server_port = 9980
Capybara.register_driver :selenium_chrome do |app|
http_client = isWindows ? nil : Selenium::WebDriver::Remote::Http::Curb.new
options = {
http_client: http_client,
browser: :chrome,
switches: [
"--proxy-server=0.0.0.0:9980",
"--disable-web-security",
'--user-agent="Chrome under Selenium for Capybara"',
"--start-maximized",
'--no-sandbox',
]
}
Capybara::Selenium::Driver.new app, options
end
Capybara.default_driver = :selenium_chrome
Capybara.app = SinatraProxy.new
Capybara.server_host = '0.0.0.0'
Capybara.default_max_wait_time = 8
Sinatra proxy:
require "sinatra"
require 'net/http'
require 'json'
file = File.read 'config.json'
config_json = JSON.parse(file)
HOST = 'remote_app'
PORT = '80'
HEADERS_NOT_TO_PROXY = %w(transfer-encoding)
class SinatraProxy < Sinatra::Base
# configure :development do
# register Sinatra::Reloader
# end
def request_headers
request.env.select {|k,v| k.start_with? 'HTTP_'}
.collect {|pair| [pair[0].sub(/^HTTP_/, ''), pair[1]]}
.to_h # Ruby 2.1
.merge('CONTENT-TYPE' => request.env['CONTENT_TYPE'] || 'application/json')
end
proxy = lambda do
# puts "REQUEST HEADERS #{request_headers}"
uri = URI.parse(request.url)
http = Net::HTTP.new(HOST, PORT)
response = http.send_request(
request.request_method.upcase,
uri.request_uri,
request.body.read,
request_headers)
response_headers = {}
response.to_hash.each{|k,v| response_headers[k]=v.join unless HEADERS_NOT_TO_PROXY.include?(k) }
status response.code
headers response_headers
headers 'Access-Control-Allow-Origin' => '*'
# puts "RESPONSE HEADERS #{response_headers}, BODY: #{response.body}"
body response.body
end
get '/*', &proxy
post '/*', &proxy
patch '/*', &proxy
put '/*', &proxy
delete '/*', &proxy
options "*" do
headers 'Access-Control-Allow-Origin' => '*',
'Access-Control-Allow-Methods' => 'HEAD,GET,PUT,DELETE,OPTIONS'
halt 200
end
end

You can't change the timeout for waiting during reset, however you shouldn't be setting Capybara.app to your proxy app to start with. Capybara.app is meant for your AUT (application under test) where any requests that hang for more than 60 seconds after being told to quit is definitely an error. Since your proxy isn't your AUT just run it separately and tell Capybara not to run a server/app.
Capybara.run_server = false
Thread.new do
Rack::Handler::Puma.run(SinatraProxy.new, Host: '0.0.0.0', Port: 9980) # You might need/want other options here or to use thin/webrick/etc
end

Related

Capybara ends up with inconsistent results on websockets

I am experiencing flaky action cable tests on capybara backed by cuprite(headless mode). Basically, I am creating a post using action cable and setting it on React using Mobx. I ran the test 150 times in a loop and it failed 30 times. What can cause this inconsistent failure?
If I make the driver go to another page or reload the same page the post appears as expected.
The settings are as follow:
spec/rails_helper.rb
require 'rails_helper.rb'
Dir[File.join(__dir__, "system/support/**/*.rb")].sort.each { |file| require file }
module CupriteHelpers
# some test helpers
end
RSpec.configure do |config|
config.include CupriteHelpers, type: :system
end
spec/system/support/capybara.rb
Capybara.default_max_wait_time = 10
Capybara.default_normalize_ws = true
Capybara.save_path = ENV.fetch("CAPYBARA_ARTIFACTS", "./tmp/capybara")
Capybara.singleton_class.prepend(Module.new do
attr_accessor :last_used_session
def using_session(name, &block)
self.last_used_session = name
super
ensure
self.last_used_session = nil
end
end)
spec/system/support/curprite.rb
Capybara.register_driver(:cuprite) do |app|
Capybara::Cuprite::Driver.new(
app,
**{
window_size: [1440, 900],
browser_options: {},
process_timeout: 60,
timeout: 15,
inspector: ENV['INSPECTOR'] == 'true',
headless: !ENV['HEADLESS'].in?(%w[n 0 no false]),
slowmo: (0.2 if ENV['HEADLESS'].in?(%w[n 0 no false])),
js_errors: false,
logger: FerrumLogger.new
}
)
end
Capybara.default_driver = Capybara.javascript_driver = :cuprite
spec/system/post_spec.rb
it 'can create a new post and the creator is the user do
click_button 'Add New Post +'
expect(page).to have_css('#post-2')
# rest of the tests but the line above fails
end

Rails + BMP + Capybara + Selenium webdriver + Firefox - Not able to establish connection between proxy and server while launching https site

I am trying to capture the network data from a https site via browsermob proxy.
My BMP - v2.1.4, firefox - v46.0 and selenium server - v2.53.1, rails - v5.1.3.
My code is as follows,
require "rubygems"
require "childprocess"
require 'browsermob/proxy'
require 'selenium-webdriver'
require 'capybara'
#proxy = BrowserMob::Proxy::Client.from("http://#{my_ip}:9090")
client_timeout = Selenium::WebDriver::Remote::Http::Default.new
client_timeout.timeout = 180
client = client_timeout
url = "http://#{my_ip}:4444/wd/hub"
profile = Selenium::WebDriver::Firefox::Profile.new
proxy_sel = #proxy.selenium_proxy(:http, :ssl)
profile['browser.cache.disk.enable'] = false
profile['browser.cache.memory.enable'] = false
profile['browser.cache.offline.enable'] = false
profile['network.http.use-cache'] = false
caps = Selenium::WebDriver::Remote::Capabilities.firefox(:firefox_profile =>
profile, :proxy => proxy_sel)
Capybara.register_driver :grid_firefox_proxy do |app|
Capybara::Selenium::Driver.new(
app,
:url => url,
:browser => :remote,
:http_client => client,
:desired_capabilities => caps
)
end
Capybara.default_driver = :grid_firefox_proxy
session = Capybara::Session.new(:grid_firefox_proxy)
session.driver.browser.manage.window.maximize
#browser = session
#proxy.new_har
#browser.driver.browser.get(#{https_url})
har = #proxy.har
puts har.entries.first.request.url
puts har.entries.first.response.status
har.save_to "mean.har"
#proxy.close
#browser.driver.quit
When I run this code, the browser is getting launched and it says, "Firefox cannot establish connection with the server". And in the BMP server, I am getting an error like,
"[ERROR 2017-10-25T13:18:17,201 org.littleshoot.proxy.impl.
ProxyToServerConnection] (LittleProxy-853-ProxyToServerWorker-2)
(DISCONNECTED) [id: 0xecabc1af, L:0.0.0.0/0.0.0.0:63045 !
R:abcd.com/66.151.240.102:443]: Caught an exception
on ProxyToServerConnection io.netty.handler.codec.DecoderException:
javax.net.ssl.SSLHandshakeException: General SSLEngine problem"
When I created selenium proxy without ssl option (proxy_sel = #proxy.selenium_proxy(:http)), I am able to fetch some irrelevant network entries.
http://ocsp.digicert.com/
http://ocsp.digicert.com/
http://ocsp.digicert.com/
http://ss.symcd.com/
http://clients1.google.com/ocsp
http://ocsp.comodoca.com/
http://ocsp.comodoca.com/
http://ss.symcd.com/
http://clients1.google.com/ocsp
http://ocsp.digicert.com/
http://ocsp.sca1b.amazontrust.com/
http://ocsp.digicert.com/
http://clients1.google.com/ocsp
http://ocsp.digicert.com/
http://ocsp.digicert.com/
http://clients1.google.com/ocsp
http://ocsp.digicert.com/
I want to pull the correct network data. Any help would be appreciated!!
Thanks!!

Webmock and VCR, allow Http Connections if there is no cassette

I have a problem, I can run a test that uses vcr on its own and it works, it creates the cassette and it uses that on the next test. Great.
The problem is when I run all my tests together this particular test fails, because webmock disables http connections, I have seen this example on the Github repo page that explains how to expect real and not stubbed requests
My question is how Do I say: Allow Http connections for requests UNLESS there is a cassette. It should also CREATE the cassette when HTTP connections are allowed.
The VCR Settings
require 'vcr'
VCR.configure do | c |
if !ARGV.first.nil?
c.default_cassette_options = { :record => :new_episodes, :erb => true }
c.filter_sensitive_data('<BLACKBIRD_API_KEY>') {YAML.load(File.read('config/application.yml'))['BLACKBIRD_API_KEY'].to_s}
c.filter_sensitive_data('<BLACKBIRD_API_URL>') {YAML.load(File.read('config/application.yml'))['BLACKBIRD_API_URL'].to_s}
c.debug_logger = File.open(ARGV.first, 'w')
c.cassette_library_dir = 'spec/vcr'
c.hook_into :webmock
end
end
the above if statement exists because not EVERY test creates a cassette. So we want them to run when a cassette isn't needed.
The Test
require 'spec_helper'
describe Xaaron::Publishers::Users do
context "publish created users" do
before(:each) do
Xaaron.configuration.reset
no_user_member_roles_relation
Xaaron.configuration.publish_to_black_bird = true
Xaaron.configuration.black_bird_api_url = YAML.load(File.read('config/application.yml'))['BLACKBIRD_API_URL']
Xaaron.configuration.black_bird_api_key =YAML.load(File.read('config/application.yml'))['BLACKBIRD_API_KEY']
end
it "should publish to blackbird" do
VCR.use_cassette 'publisher/create_user_response' do
expect(
Xaaron::Publishers::Users.publish_new_user({user: {
first_name: 'adsadsad', user_name: 'sasdasdasdsa' ,
email: 'asdassad#sample.com', auth_token: 'asdsadasdasdsa'
}}).code
).to eql 200
end
end
end
end
Runs fine on its own, creates the cassette, fails when run with all other tests due to webmock.
The Failure
Failure/Error: Xaaron::Publishers::Users.publish_new_user({user: {
WebMock::NetConnectNotAllowedError:
Real HTTP connections are disabled. Unregistered request: GET some_site_url_here with headers {'Http-Authorization'=>'api_key_here', 'User-Agent'=>'Typhoeus - https://github.com/typhoeus/typhoeus'}
You can stub this request with the following snippet:
stub_request(:get, "some site url here").
with(:headers => {'Http-Authorization'=>'some api key here', 'User-Agent'=>'Typhoeus - https://github.com/typhoeus/typhoeus'}).
to_return(:status => 200, :body => "", :headers => {})

Sinatra doesn't know this ditty?

I am trying to setup the Spotify IOS API but everytime I run this Ruby file and go to http://localhost:1234/swap I get "Sinatra doesn't know this ditty".
Here is my code:
require 'sinatra'
require 'net/http'
require 'net/https'
require 'base64'
require 'json'
require 'encrypted_strings'
# This is an example token swap service written
# as a Ruby/Sinatra service. This is required by
# the iOS SDK to authenticate a user.
#
# The service requires the Sinatra and
# encrypted_strings gems be installed:
#
# $ gem install sinatra encrypted_strings
#
# To run the service, enter your client ID, client
# secret and client callback URL below and run the
# project.
#
# $ ruby spotify_token_swap.rb
#
# IMPORTANT: The example credentials will work for the
# example apps, you should use your own in your real
# environment. as these might change at any time.
#
# Once the service is running, pass the public URI to
# it (such as http://localhost:1234/swap if you run it
# with default settings on your local machine) to the
# token swap method in the iOS SDK:
#
# NSURL *swapServiceURL = [NSURL urlWithString:#"http://localhost:1234/swap"];
#
# -[SPAuth handleAuthCallbackWithTriggeredAuthURL:url
# tokenSwapServiceEndpointAtURL:swapServiceURL
# callback:callback];
#
print "\e[31m------------------------------------------------------\e[0m\n"
print "\e[31mYou're using example credentials, please replace these\e[0m\n"
print "\e[31mwith your own and remove this silly warning.\e[0m\n"
print "\e[31m------------------------------------------------------\e[0m\n"
print "\7\7"
sleep(2)
CLIENT_ID = ""
CLIENT_SECRET = ""
ENCRYPTION_SECRET = ""
CLIENT_CALLBACK_URL = "dawgone://returnhere"
AUTH_HEADER = "Basic " + Base64.strict_encode64(CLIENT_ID + ":" + CLIENT_SECRET)
SPOTIFY_ACCOUNTS_ENDPOINT = URI.parse("https://accounts.spotify.com")
set :port, 1234 # The port to bind to.
set :bind, '0.0.0.0' # IP address of the interface to listen on (all)
post '/swap' do
# This call takes a single POST parameter, "code", which
# it combines with your client ID, secret and callback
# URL to get an OAuth token from the Spotify Auth Service,
# which it will pass back to the caller in a JSON payload.
auth_code = params[:code]
http = Net::HTTP.new(SPOTIFY_ACCOUNTS_ENDPOINT.host, SPOTIFY_ACCOUNTS_ENDPOINT.port)
http.use_ssl = true
request = Net::HTTP::Post.new("/api/token")
request.add_field("Authorization", AUTH_HEADER)
request.form_data = {
"grant_type" => "authorization_code",
"redirect_uri" => CLIENT_CALLBACK_URL,
"code" => auth_code
}
response = http.request(request)
# encrypt the refresh token before forwarding to the client
if response.code.to_i == 200
token_data = JSON.parse(response.body)
refresh_token = token_data["refresh_token"]
encrypted_token = refresh_token.encrypt(:symmetric, :password => ENCRYPTION_SECRET)
token_data["refresh_token"] = encrypted_token
response.body = JSON.dump(token_data)
end
status response.code.to_i
return response.body
end
post '/refresh' do
# Request a new access token using the POST:ed refresh token
http = Net::HTTP.new(SPOTIFY_ACCOUNTS_ENDPOINT.host, SPOTIFY_ACCOUNTS_ENDPOINT.port)
http.use_ssl = true
request = Net::HTTP::Post.new("/api/token")
request.add_field("Authorization", AUTH_HEADER)
encrypted_token = params[:refresh_token]
refresh_token = encrypted_token.decrypt(:symmetric, :password => ENCRYPTION_SECRET)
request.form_data = {
"grant_type" => "refresh_token",
"refresh_token" => refresh_token
}
response = http.request(request)
status response.code.to_i
return response.body
end
This is because swap is a POST endpoint. When you pull up a URL in your browser you are doing an HTTP GET.
If you want to see that the sinatra service is running and you can at least talk to it you could try hitting it with curl from the command line with the right POST parameters.

check https status code ruby

Is there a way to check for an HTTPS status code in ruby? I know that there are ways to do this in HTTP using require 'net/http', but I'm looking for HTTPS. Maybe there is a different library that I need to use?
You can do this in net/http:
require "net/https"
require "uri"
uri = URI.parse("https://www.secure.com/")
http = Net::HTTP.new(uri.host, uri.port)
http.use_ssl = true
request = Net::HTTP::Get.new(uri.request_uri)
res = http.request(request)
res.code #=> "200"
Refs:
Net::HTTP cheat sheet
How to Cure Net::HTTP’s Risky Default HTTPS Behavior
You can use any wrapper around Net::HTTP(S) to get much easier behavior.
I use Faraday here ( https://github.com/lostisland/faraday ) but HTTParty has almost the same functionality ( https://github.com/jnunemaker/httparty )
require 'faraday'
res = Faraday.get("https://www.example.com/")
res.status # => 200
res = Faraday.get("http://www.example.com/")
res.status # => 200
(as a bonus you get options for parsing responses, raising state exceptions, logging requests....
connection = Faraday.new("https://www.example.com/") do |conn|
# url-encode the body if given as a hash
conn.request :url_encoded
# add an authorization header
conn.request :oauth2, 'TOKEN'
# use JSON to convert the response into a hash
conn.response :json, :content_type => /\bjson$/
# ...
conn.adapter Faraday.default_adapter
end
connection.get("/")
# GET https://www.example.com/some/path?query=string
connection.get("/some/path", :query => "string")
# POST, PUT, DELETE, PATCH....
connection.post("/some/other/path", :these => "fields", :will => "be converted to a request string in the body"}
# add any number of headers. in this example "Accept-Language: en-US"
connection.get("/some/path", nil, :accept_language => "en-US")
require 'uri'
require 'net/http'
res = Net::HTTP.get_response(URI('http://www.example.com/index.html'))
puts res.code # -> '200'
Slightly more readable way:
response.kind_of?(Net::HTTPOK)

Resources