I interacted with a website using Capybara, the driver i chose is capybara-webkit ,but the problem is that the code was stuck at visit and cost about 5 minutes or more to load the page.
It was a terrible thing for me.
And my code is the following:
def crawl(city_en)
#Global Setting
Capybara.current_driver = :webkit
Capybara.default_selector = :xpath
Capybara.app_host = 'http://hotel.qunar.com'
# Set user agent
page.driver.header 'User-Agent',"Mozilla/5.0 (Macintosh; Intel Mac OS X)"
#city = city_en
#uri = "/city/#{#city}"
puts 'Visiting...'
page.visit #uri
#page = Nokogiri::HTML page.driver.browser.body
end
I am not sure if there something wrong with my code, if not ,are there some tricks to faster webkit? Thanks.
Capybara is acceptance test framework. For automating interaction with websites you can use mechanize. Mechanize automatically stores and sends cookies, follows redirects, and can follow links and submit forms.
Related
Ruby 1.9.3, RSpec 2.13.0, WebMock 1.17.4, Rails 3
I am writing tests for a company app. The controller in question displays a table of a customer's placed calls, and allows for sort/filter options.
EDIT The test fails because with my current setup, the path does not render, because the recorder_server is either not running locally, OR not setup correctly. Please help with this, too.
A Errno::ECONNREFUSED occurred in recordings#index:
Connection refused - connect(2)
/usr/local/lib/ruby/1.9.1/net/http.rb:763:in `initialize'
-------------------------------
Request:
-------------------------------
* URL : http://www.recorder.example.com:8080/recorded_calls
* IP address: 127.0.0.1
* Parameters: {"controller"=>"recordings", "action"=>"index"}
* Rails root: /var/www/rails/<repository>
As a call is placed, its data joins an xml file, created by an external API, called Recorder
The RecordingsController takes the xml file, and parses it into a hash.
When you visit the associated path, you see the results of the hash -- a table of placed calls, their attributes, and parameters for sort/filter.
Here is my spec so far.
require 'spec_helper'
include Helpers
feature 'Exercise recordings controller' do
include_context "shared admin context"
background do
canned_xml = File.open("spec/support/assets/canned_response.xml").read
stub_request(:post, "http://recorder.example.com:8080/recorder/index").
with(body: {"durations"=>["1"], "durations_greater_less"=>["gt"], "filter_from_day"=>"29", "filter_from_hour"=>"0", "filter_from_minute"=>"0", "filter_from_month"=>"12", "filter_from_year"=>"2014", "filter_prefix"=>true, "filter_to_day"=>"29", "filter_to_hour"=>"23", "filter_to_minute"=>"59", "filter_to_month"=>"12", "filter_to_year"=>"2014"}, # "shared_session_id"=>"19f9a08807cc70c1bf41885956695bde"},
headers: {'Accept'=>'*/*', 'Content-Type'=>'application/x-www-form-urlencoded', 'User-Agent'=>'Ruby'}).
to_return(status: 200, body: canned_xml, headers: {})
uri = URI.parse("http://recorder.example.com:8080/recorder/index")
visit recorded_calls_path
end
scenario 'show index page with 1 xml result' do
#page.save_and_open_page
expect(title).to eq("Recorded Calls")
end
end
And here is the RecordingsController
class RecordingsController < ApplicationController
# before_filter options
def index
test_session_id = request.session_options[:id]
#Make request to recording app for xml of files
uri = URI.parse("http://#{Rails.application.config.recorder_server}:#{Rails.application.config.recorder_server_port}/recorder/index")
http = Net::HTTP.new(uri.host, uri.port)
xml_request = Net::HTTP::Post.new(uri.request_uri)
xml_request_data = Hash.new
# sorting params
xml_request_data[:shared_session_id] = request.session_options[:id]
xml_request.set_form_data(xml_request_data)
response = http.request(xml_request)
if response.class == Net::HTTPOK
#recordings_xml = XmlSimple.xml_in(response.body)
#recordings_sorted = #recordings_xml["Recording"].sort { |a,b| Time.parse("#{a["date"]} #{a["time"]}") <=> Time.parse("#{b["date"]} #{b["time"]}") } unless #recordings_xml["Recording"].nil?
else #recordings_xml = Hash.new
end
end
# other defs
end
Any and all advice is much appreciated. Thank you.
How I configured WebMock
I am answering my own question, with the help of B-Seven and a string of comments. File by file, I will list the changes made in order to properly use WebMock.
Add WebMock to Gemfile under group :test, :development.
bundle install to resolve dependencies
my current setup included Ruby 1.9.3, Rails 2.13.0, WebMock 1.17.4
Setup spec_helper.rb to disable "Real HTTP connections". (This was a backtrace error received later on in this puzzling process.) This allows, to my understanding, all "real connections" to translate into localhost connections and work offline... Which is great since, ideally, I do not want the external app's server to run simultaneously.
require 'webmock/rspec'
WebMock.disable_net_connect!(allow_localhost: true)
In my test.rb environment file, the configurations for recorder_server and port were commented out... If left uncommented, the controller would raise an exception stating uninitialized constants. I used the test server/port (substituting the company name for example) as my layout for the spec stubbing.
In recordings_controller_spec.rb, I had already figured out how to make a canned XML response. With these changes above, my spec was able to correctly stub a response on an external, secondary app, and use such response to correctly render the view associated with the controller being tested.
require 'spec_helper'
include Helpers
feature "Exercise recordings_controller" do
include_context "shared admin context"
# A background is currently not used, because I have 3 scenario types... No xml
# results, 1 result, and 2 results. I will later DRY this out with a background,
# but the heavy lifting is over, for now.
scenario "show index page with 1 xml result" do
canned_xml_1 = File.open("spec/support/assets/canned_response_1.xml").read
stub_request(:post, "http://recorder.example.com:8080/recorder/index").
with(headers: {'Accept'=>'*/*', 'User-Agent'=>'Ruby'}).
to_return(status: 200, body: canned_xml_1, headers: {})
uri = URI.parse("http://recorder.example.com:8080/recorder/index")
visit recorded_calls_path
title.should == "Recorded Calls"
page.should have_content("Search Results")
page.should have_content("Inbound", "5551230000", "175", "December 24 2014", "12:36:24", "134")
end
end
Advice/Resources that helped
With B-Seven's suggestion to my original question (see revisions), I was initially stubbing localhost:3000. He said this was incorrect. After further research, I agree since stubbing with WebMock is typically reserved for outside http connections.
In comments after his answer, B-Seven listed articles to refer to. I will list the ones that helped me the most.
http://robots.thoughtbot.com/how-to-stub-external-services-in-tests
http://railscasts.com/episodes/275-how-i-test
https://github.com/bblimke/webmock
http://www.agileventures.org/articles/testing-with-rspec-stubs-mocks-factories-what-to-choose
It is very important to read the backtrace generated from an errors. What took me so long to figure out how to mock was mainly reading them incorrectly. As you can see from my question, I was making a :get stub request. A coworker pointed out that the backtrace suggested to use :post. That was the final piece to make my spec pass.
I decided not to input the configuration variables as my stub request, for it would result in long lines of code. Instead, this is why I needed to uncomment out those configurations in test.rb.
Why are you stubbing localhost? I think you want to
stub_request(:get, "http://#{Rails.application.config.recorder_server}:#{Rails.application.config.recorder_server_port}/recorder/index").
I'm new with Rubyonrails and Koala gem and I'm sure I'm doing something wrong. I've been tuning my code to the minimun expression but the problem persist. Then I tried to do the same without koala gem, but the problem persisted.
This is the code:
require 'koala'
require 'open-uri'
puts Time.now
#graph = Koala::Facebook::API.new
resp = #graph.graph_call("cocacola", {}, "get", {})
puts resp
puts Time.now
coke_url = "https://graph.facebook.com/cocacola"
response = open coke_url
response = JSON.parse response.read
puts response.inspect
puts Time.now
I have to wait always 21 seconds the Facebook's response. If I put the https://graph.facebook.com/cocacola on my browser, the response is instantaneous ¿is not the same?
thanks
I'm using Koala in my application and did not have such experience. The only difference is that I don't use it anonymously. Instead I created an app and I'm using an access token to access the Facebook API. This might be the root cause, as I've found this post that also seems relates.
Finally, it was a DNS problem (thanks jpgeek).
When I did just a GET request to any website, the response was after 21 seconds, but using the IP of the same website, the response was instantaneously.
I found on google the solution: http://www.mikeperham.com/2010/02/10/asynchronous-dns-resolution/
I have use this personal solution in Gemfile (I'm not sure if it's the best):
group :development do
require 'resolv'
require 'resolv-replace'
end
Now it's working fine!
i am using cucumber and capybara. in a rails 3.0.9 platform. i am getting this test case fail:
log is:
(::) failed steps (::)
No route matches "/wiki/Baltimore_Ravens" (ActionController::RoutingError)
<internal:prelude>:10:in `synchronize'
./features/step_definitions/web_steps.rb:20:in `/^(?:|I )am on (.+)$/'
features/annotate.feature:7:in `Given I am on a web page'
Failing Scenarios:
cucumber features/annotate.feature:11 # Scenario: launch annotation/ logged in
6 scenarios (1 failed, 5 skipped)
63 steps (1 failed, 62 skipped)
the file web_steps: got this piece of code:
19 Given /^(?:|I )am on (.+)$/ do |page_name|
20 visit path_to(page_name)
21 end
the file annotate.feature got this code:
7 Given I am on a web page
"a web page" is defined in support/paths.rb as:
when /a web page/
'http://en.wikipedia.org/wiki/Baltimore_Ravens'
obviously this is an external url. i want to open it and capybara and cucumber wont allow me to do it. so, help me find a way to open outside url in cucumber test case!
Capybara uses RackTest as the default driver, and this driver doesn't allow to visit external urls (i.e. test remote applications).
If you want to visit external urls (to test, e.g, that your app redirects correctly), you have basically two options:
1/ Use another driver like e.g selenium:
before do
Capybara.current_driver = :selenium
end
Then, in code, you can call the url like so:
visit 'http://en.wikipedia.org/wiki/Baltimore_Ravens'
Or, if you set the default app_host like so:
Capybara.app_host = 'http://en.wikipedia.org'
Capybara.run_server = false # don't start Rack
Then you can call the url:
visit '/wiki/Baltimore_Ravens'
You can configure the driver and app host in your spec_helper.rb to enable them globally across all you specs:
Capybara.configure do |config|
config.current_driver = :selenium
config.run_server = false
config.app_host = 'http://en.wikipedia.org'
end
2/ Use capybara-mechanize
I'm fairly new to Rails and Rack, but this guy had a seemingly straightforward write-up about using Rack to implement dynamic session domain middleware. The code looks good to and I've implemented it here on my local machine, but I'm still not able to transcend top level domains on a single login.
Here's the middleware code:
class SetCookieDomain
def initialize(app, default_domain)
#app = app
#default_domain = default_domain
end
def call(env)
host = env["HTTP_HOST"].split(':').first
env["rack.session.options"][:domain] = custom_domain?(host) ? ".#{host}" : "#{#default_domain}"
#app.call(env)
end
def custom_domain?(host)
domain = #default_domain.sub(/^\./, '')
host !~ Regexp.new("#{domain}$", Regexp::IGNORECASE)
end
end
And then in environment.db:
config.load_paths += %W(#{RAILS_ROOT}/app/middlewares)
Lastly in production.db (and development.db):
config.middleware.use "SetCookieDomain", ".example.org"
Any help is greatly appreciated.
EDIT: I'm running Rails 2.3.3 and Rack 1.0
I had similar problems getting this to work in development mode. When I was trying with localhost, I couldn't get it to work. However, by accessing it via a domain configured in /etc/hosts to point to localhost, for example computer.local, I was able to get it to work.
A client of mine has asked me to integrate a 3rd party API into their Rails app. The only problem is that the API uses SOAP. Ruby has basically dropped SOAP in favor of REST. They provide a Java adapter that apparently works with the Java-Ruby bridge, but we'd like to keep it all in Ruby, if possible. I looked into soap4r, but it seems to have a slightly bad reputation.
So what's the best way to integrate SOAP calls into a Rails app?
I built Savon to make interacting with SOAP webservices via Ruby as easy as possible.
I'd recommend you check it out.
We used the built in soap/wsdlDriver class, which is actually SOAP4R.
It's dog slow, but really simple. The SOAP4R that you get from gems/etc is just an updated version of the same thing.
Example code:
require 'soap/wsdlDriver'
client = SOAP::WSDLDriverFactory.new( 'http://example.com/service.wsdl' ).create_rpc_driver
result = client.doStuff();
That's about it
We switched from Handsoap to Savon.
Here is a series of blog posts comparing the two client libraries.
I also recommend Savon. I spent too many hours trying to deal with Soap4R, without results. Big lack of functionality, no doc.
Savon is the answer for me.
Try SOAP4R
SOAP4R
Getting Started with SOAP4R
And I just heard about this on the Rails Envy Podcast (ep 31):
WS-Deathstar SOAP walkthrough
Just got my stuff working within 3 hours using Savon.
The Getting Started documentation on Savon's homepage was really easy to follow - and actually matched what I was seeing (not always the case)
Kent Sibilev from Datanoise had also ported the Rails ActionWebService library to Rails 2.1 (and above).
This allows you to expose your own Ruby-based SOAP services.
He even has a scaffold/test mode which allows you to test your services using a browser.
I have used HTTP call like below to call a SOAP method,
require 'net/http'
class MyHelper
def initialize(server, port, username, password)
#server = server
#port = port
#username = username
#password = password
puts "Initialised My Helper using #{#server}:#{#port} username=#{#username}"
end
def post_job(job_name)
puts "Posting job #{job_name} to update order service"
job_xml ="<soapenv:Envelope xmlns:soapenv=\"http://schemas.xmlsoap.org/soap/envelope/\" xmlns:ns=\"http://test.com/Test/CreateUpdateOrders/1.0\">
<soapenv:Header/>
<soapenv:Body>
<ns:CreateTestUpdateOrdersReq>
<ContractGroup>ITE2</ContractGroup>
<ProductID>topo</ProductID>
<PublicationReference>#{job_name}</PublicationReference>
</ns:CreateTestUpdateOrdersReq>
</soapenv:Body>
</soapenv:Envelope>"
#http = Net::HTTP.new(#server, #port)
puts "server: " + #server + "port : " + #port
request = Net::HTTP::Post.new(('/XISOAPAdapter/MessageServlet?/Test/CreateUpdateOrders/1.0'), initheader = {'Content-Type' => 'text/xml'})
request.basic_auth(#username, #password)
request.body = job_xml
response = #http.request(request)
puts "request was made to server " + #server
validate_response(response, "post_job_to_pega_updateorder job", '200')
end
private
def validate_response(response, operation, required_code)
if response.code != required_code
raise "#{operation} operation failed. Response was [#{response.inspect} #{response.to_hash.inspect} #{response.body}]"
end
end
end
/*
test = MyHelper.new("mysvr.test.test.com","8102","myusername","mypassword")
test.post_job("test_201601281419")
*/
Hope it helps. Cheers.
I have used SOAP in Ruby when i've had to make a fake SOAP server for my acceptance tests. I don't know if this was the best way to approach the problem, but it worked for me.
I have used Sinatra gem (I wrote about creating mocking endpoints with Sinatra here) for server and also Nokogiri for XML stuff (SOAP is working with XML).
So, for the beginning I have create two files (e.g. config.rb and responses.rb) in which I have put the predefined answers that SOAP server will return.
In config.rb I have put the WSDL file, but as a string.
##wsdl = '<wsdl:definitions name="StockQuote"
targetNamespace="http://example.com/stockquote.wsdl"
xmlns:tns="http://example.com/stockquote.wsdl"
xmlns:xsd1="http://example.com/stockquote.xsd"
xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/"
xmlns="http://schemas.xmlsoap.org/wsdl/">
.......
</wsdl:definitions>'
In responses.rb I have put samples for responses that SOAP server will return for different scenarios.
##login_failure = "<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">
<s:Body>
<LoginResponse xmlns="http://tempuri.org/">
<LoginResult xmlns:a="http://schemas.datacontract.org/2004/07/WEBMethodsObjects" xmlns:i="http://www.w3.org/2001/XMLSchema-instance">
<a:Error>Invalid username and password</a:Error>
<a:ObjectInformation i:nil="true"/>
<a:Response>false</a:Response>
</LoginResult>
</LoginResponse>
</s:Body>
</s:Envelope>"
So now let me show you how I have actually created the server.
require 'sinatra'
require 'json'
require 'nokogiri'
require_relative 'config/config.rb'
require_relative 'config/responses.rb'
after do
# cors
headers({
"Access-Control-Allow-Origin" => "*",
"Access-Control-Allow-Methods" => "POST",
"Access-Control-Allow-Headers" => "content-type",
})
# json
content_type :json
end
#when accessing the /HaWebMethods route the server will return either the WSDL file, either and XSD (I don't know exactly how to explain this but it is a WSDL dependency)
get "/HAWebMethods/" do
case request.query_string
when 'xsd=xsd0'
status 200
body = ##xsd0
when 'wsdl'
status 200
body = ##wsdl
end
end
post '/HAWebMethods/soap' do
request_payload = request.body.read
request_payload = Nokogiri::XML request_payload
request_payload.remove_namespaces!
if request_payload.css('Body').text != ''
if request_payload.css('Login').text != ''
if request_payload.css('email').text == some username && request_payload.css('password').text == some password
status 200
body = ##login_success
else
status 200
body = ##login_failure
end
end
end
end
I hope you'll find this helpful!
I was having the same issue, switched to Savon and then just tested it on an open WSDL (I used http://www.webservicex.net/geoipservice.asmx?WSDL) and so far so good!
https://github.com/savonrb/savon