I'd like to debug the request my Rails app makes with RestClient. The RestClient docs say:
To enable logging you can
set RestClient.log with a ruby Logger
or set an environment variable to avoid modifying the code (in this case you can use a file name, “stdout” or “stderr”):
$ RESTCLIENT_LOG=stdout path/to/my/program
Either produces logs like this:
RestClient.get "http://some/resource"
=> 200 OK | text/html 250 bytes
RestClient.put "http://some/resource", "payload"
=> 401 Unauthorized | application/xml 340 bytes
Note that these logs are valid Ruby, so you can paste them into the restclient shell or a >script to replay your sequence of rest calls.
How do I do get these logs included in my Rails apps log folder?
from: https://gist.github.com/jeremy/1383337
require 'restclient'
# RestClient logs using << which isn't supported by the Rails logger,
# so wrap it up with a little proxy object.
RestClient.log =
Object.new.tap do |proxy|
def proxy.<<(message)
Rails.logger.info message
end
end
Create a file in config/initializers:
RestClient.log = 'log/a_log_file.log'
Or just put last in console
https://github.com/adelevie/rest-client/commit/5a7ed325eaa091809141d3ef6e31087569614e9d
This worked for me, running on RestClient 1.8 and Rails 4.2.1:
::RestClient.log = Rails.logger
You can use this gem:
https://github.com/uswitch/rest-client-logger
It works out of the box just by adding "gem 'rest-client-logger'" to your Gemfile.
may be so: RestClient.log = Rails.logger
Related
Ruby 1.9.3, RSpec 2.13.0, WebMock 1.17.4, Rails 3
I am writing tests for a company app. The controller in question displays a table of a customer's placed calls, and allows for sort/filter options.
EDIT The test fails because with my current setup, the path does not render, because the recorder_server is either not running locally, OR not setup correctly. Please help with this, too.
A Errno::ECONNREFUSED occurred in recordings#index:
Connection refused - connect(2)
/usr/local/lib/ruby/1.9.1/net/http.rb:763:in `initialize'
-------------------------------
Request:
-------------------------------
* URL : http://www.recorder.example.com:8080/recorded_calls
* IP address: 127.0.0.1
* Parameters: {"controller"=>"recordings", "action"=>"index"}
* Rails root: /var/www/rails/<repository>
As a call is placed, its data joins an xml file, created by an external API, called Recorder
The RecordingsController takes the xml file, and parses it into a hash.
When you visit the associated path, you see the results of the hash -- a table of placed calls, their attributes, and parameters for sort/filter.
Here is my spec so far.
require 'spec_helper'
include Helpers
feature 'Exercise recordings controller' do
include_context "shared admin context"
background do
canned_xml = File.open("spec/support/assets/canned_response.xml").read
stub_request(:post, "http://recorder.example.com:8080/recorder/index").
with(body: {"durations"=>["1"], "durations_greater_less"=>["gt"], "filter_from_day"=>"29", "filter_from_hour"=>"0", "filter_from_minute"=>"0", "filter_from_month"=>"12", "filter_from_year"=>"2014", "filter_prefix"=>true, "filter_to_day"=>"29", "filter_to_hour"=>"23", "filter_to_minute"=>"59", "filter_to_month"=>"12", "filter_to_year"=>"2014"}, # "shared_session_id"=>"19f9a08807cc70c1bf41885956695bde"},
headers: {'Accept'=>'*/*', 'Content-Type'=>'application/x-www-form-urlencoded', 'User-Agent'=>'Ruby'}).
to_return(status: 200, body: canned_xml, headers: {})
uri = URI.parse("http://recorder.example.com:8080/recorder/index")
visit recorded_calls_path
end
scenario 'show index page with 1 xml result' do
#page.save_and_open_page
expect(title).to eq("Recorded Calls")
end
end
And here is the RecordingsController
class RecordingsController < ApplicationController
# before_filter options
def index
test_session_id = request.session_options[:id]
#Make request to recording app for xml of files
uri = URI.parse("http://#{Rails.application.config.recorder_server}:#{Rails.application.config.recorder_server_port}/recorder/index")
http = Net::HTTP.new(uri.host, uri.port)
xml_request = Net::HTTP::Post.new(uri.request_uri)
xml_request_data = Hash.new
# sorting params
xml_request_data[:shared_session_id] = request.session_options[:id]
xml_request.set_form_data(xml_request_data)
response = http.request(xml_request)
if response.class == Net::HTTPOK
#recordings_xml = XmlSimple.xml_in(response.body)
#recordings_sorted = #recordings_xml["Recording"].sort { |a,b| Time.parse("#{a["date"]} #{a["time"]}") <=> Time.parse("#{b["date"]} #{b["time"]}") } unless #recordings_xml["Recording"].nil?
else #recordings_xml = Hash.new
end
end
# other defs
end
Any and all advice is much appreciated. Thank you.
How I configured WebMock
I am answering my own question, with the help of B-Seven and a string of comments. File by file, I will list the changes made in order to properly use WebMock.
Add WebMock to Gemfile under group :test, :development.
bundle install to resolve dependencies
my current setup included Ruby 1.9.3, Rails 2.13.0, WebMock 1.17.4
Setup spec_helper.rb to disable "Real HTTP connections". (This was a backtrace error received later on in this puzzling process.) This allows, to my understanding, all "real connections" to translate into localhost connections and work offline... Which is great since, ideally, I do not want the external app's server to run simultaneously.
require 'webmock/rspec'
WebMock.disable_net_connect!(allow_localhost: true)
In my test.rb environment file, the configurations for recorder_server and port were commented out... If left uncommented, the controller would raise an exception stating uninitialized constants. I used the test server/port (substituting the company name for example) as my layout for the spec stubbing.
In recordings_controller_spec.rb, I had already figured out how to make a canned XML response. With these changes above, my spec was able to correctly stub a response on an external, secondary app, and use such response to correctly render the view associated with the controller being tested.
require 'spec_helper'
include Helpers
feature "Exercise recordings_controller" do
include_context "shared admin context"
# A background is currently not used, because I have 3 scenario types... No xml
# results, 1 result, and 2 results. I will later DRY this out with a background,
# but the heavy lifting is over, for now.
scenario "show index page with 1 xml result" do
canned_xml_1 = File.open("spec/support/assets/canned_response_1.xml").read
stub_request(:post, "http://recorder.example.com:8080/recorder/index").
with(headers: {'Accept'=>'*/*', 'User-Agent'=>'Ruby'}).
to_return(status: 200, body: canned_xml_1, headers: {})
uri = URI.parse("http://recorder.example.com:8080/recorder/index")
visit recorded_calls_path
title.should == "Recorded Calls"
page.should have_content("Search Results")
page.should have_content("Inbound", "5551230000", "175", "December 24 2014", "12:36:24", "134")
end
end
Advice/Resources that helped
With B-Seven's suggestion to my original question (see revisions), I was initially stubbing localhost:3000. He said this was incorrect. After further research, I agree since stubbing with WebMock is typically reserved for outside http connections.
In comments after his answer, B-Seven listed articles to refer to. I will list the ones that helped me the most.
http://robots.thoughtbot.com/how-to-stub-external-services-in-tests
http://railscasts.com/episodes/275-how-i-test
https://github.com/bblimke/webmock
http://www.agileventures.org/articles/testing-with-rspec-stubs-mocks-factories-what-to-choose
It is very important to read the backtrace generated from an errors. What took me so long to figure out how to mock was mainly reading them incorrectly. As you can see from my question, I was making a :get stub request. A coworker pointed out that the backtrace suggested to use :post. That was the final piece to make my spec pass.
I decided not to input the configuration variables as my stub request, for it would result in long lines of code. Instead, this is why I needed to uncomment out those configurations in test.rb.
Why are you stubbing localhost? I think you want to
stub_request(:get, "http://#{Rails.application.config.recorder_server}:#{Rails.application.config.recorder_server_port}/recorder/index").
I have an api written in rails which on each request responds with a JSON response.
The response could be huge, so i need to compress the JSON response using gzip.
Wondering how to do this in rails controller?
I have added the line
use Rack::Deflater
in config.ru
Should I also be changing something in the line which renders JSON?
render :json => response.to_json()
Also, how do i check if the response is in gzip format or not..??
I did a curl request from terminal, I see only the normal plain JSON.
My post Content Compression with Rack::Deflater describes a couple of ways to integrate Rack::Deflater. The easiest would be to just update config/application.rb with:
module YourApp
class Application < Rails::Application
config.middleware.use Rack::Deflater
end
end
and you'll automatically compress all controller responses with deflate / gzip if the client explicitly says they can handle it.
For the response to be in gzip format we don't have to change the render method call.
If the request has the header Accept-Encoding: gzip, Rails will automatically compress the JSON response using gzip.
If you don't want the user to send a request with preset header., you can add the header to the request manually in the controller before rendering the response:
request.env['HTTP_ACCEPT_ENCODING'] = 'gzip'
render :json => response.to_json()
You can query Curl by setting a custom header to get gzipped response
$ curl -H "Accept-Encoding: gzip, deflate" localhost:3000/posts.json > posts_json.gz
then, then decompress it to view the actual response json
$ gzip -d posts_json.gz
$ cat posts_json
If it doesn't work. post back with output of rake middlewares to help us troubleshoot further.
In some cases you can consider to write huge response into a file and gzip it:
res = {} # huge data hash
json = res.to_json
Zlib::GzipWriter.open('public/api/huge_data.json.gz') { |gz| gz.write json }
and update this file regularly
Consider not putting Rack middlewares in config.ru when using Rails
Rails has it's own middleware stack manager since Rails 2.
The correct way is:
# config/application.rb or config/environment.rb depends on your Rails version
config.middleware.use Rack::Deflater
Don't use #djcp's solution when using Rack::ETag
Short answer:
module MyApp
class Application < Rails::Application
config.middleware.insert_before Rack::ETag, Rack::Deflater
end
end
The order of Rack::Deflater and Rack::ETag matters because Rack::Deflater uses Zlib::GzipWriter to compress the response body and it would compress with a timestamp by default, which means the compressed response body would change every second even if the original response body is the same.
To reproduce this problem, run the following script:
require 'rack/etag'
require 'rack/deflater'
require 'rack/content_length'
#app = Rack::Builder.new do
use Rack::ETag
use Rack::Deflater
use Rack::ContentLength
run ->(*) { [200, {}, ['hello world']] }
end
def puts_etag
puts #app.call({ 'HTTP_ACCEPT_ENCODING' => 'gzip' })[1]['ETag']
end
puts_etag
sleep 1
puts_etag
One can simply swap the lines of Rack::ETag and Rack::Deflater and get the expected output.
Rails uses Rack::ETag by default and config.middleware.use is just appending. To insert Rack::Deflater before Rack::Etag, use config.middleware.insert_before instead.
🍻
I've setup unicorn in rails 3.1 and http streaming works until I enable Rack::Deflater.
I've tried both with and without use Rack::Chunked. In curl I can see my response while in chrome I get the following errror: ERR_INVALID_CHUNKED_ENCODING
The result is same in other browsers (firefox, safari) and between development (osx) and production (heroku).
config.ru:
require ::File.expand_path('../config/environment', __FILE__)
use Rack::Chunked
use Rack::Deflater
run Site::Application
unicorn.rb:
listen 3001, :tcp_nopush => false
worker_processes 1 # amount of unicorn workers to spin up
timeout 30 # restarts workers that hang for 30 seconds
controller:
render "someview", :stream => true
Thanks for any help.
The problem is that Rails ActionController::Streaming renders directly into a Chunked::Body. This means the content is first chunked and then gzipped by the Rack::Deflater middleware, instead of gzipped and then chunked.
According to the HTTP/1.1 RFC 6.2.1, chunked must be last applied encoding to a transfer.
Since "chunked" is the only transfer-coding required to be understood
by HTTP/1.1 recipients, it plays a crucial role in delimiting messages
on a persistent connection. Whenever a transfer-coding is applied to a
payload body in a request, the final transfer-coding applied must be
"chunked".
I fixed it for us by monkey patching ActionController::Streaming _process_options and _render_template methods in an initializer so it does not wrap the body in a Chunked::Body, and lets the Rack::Chunked middleware do it instead.
module GzipStreaming
def _process_options(options)
stream = options[:stream]
# delete the option to stop original implementation
options.delete(:stream)
super
if stream && env["HTTP_VERSION"] != "HTTP/1.0"
# Same as org implmenation except don't set the transfer-encoding header
# The Rack::Chunked middleware will handle it
headers["Cache-Control"] ||= "no-cache"
headers.delete('Content-Length')
options[:stream] = stream
end
end
def _render_template(options)
if options.delete(:stream)
# Just render, don't wrap in a Chunked::Body, let
# Rack::Chunked middleware handle it
view_renderer.render_body(view_context, options)
else
super
end
end
end
module ActionController
class Base
include GzipStreaming
end
end
And leave your config.ru as
require ::File.expand_path('../config/environment', __FILE__)
use Rack::Chunked
use Rack::Deflater
run Roam7::Application
Not a very nice solution, it will probably break some other middlewares that inspect/modify the body. If someone has a better solution I'd love to hear it.
If you are using new relic, its middleware must also be disabled when streaming.
I'm using Savon library to get some SOAP requests work. And i'm using almost the same code within IRB and Rails application. When i'm running IRB everything works as it should, but Rails generate error "no method 'to_hash' for nil:NilClass" inside savon's do_request -> respond_with methods.
Here's my code (the same when running it within IRB or Rails):
# setup Savon client for SOAP requests
client = Savon::Client.new "http://www.webservicex.net/country.asmx?WSDL"
# test if "webservicex.net" server is up and running
actions = client.wsdl.soap_actions
raise "SOAP server is down" if actions.nil? or actions.length <= 0
# get country list
resp = client.request :get_countries
raise "No response for countries" if resp.nil?
resp = resp[:get_countries_response][:get_countries_result]
None of that exceptions is risen nor code far than 'pinging server' is executed.
What's wrong and how could i fix that?
It's problem with httpi gem - https://github.com/rubiii/savon/issues/163
Just use 0.9.1 version.
A client of mine has asked me to integrate a 3rd party API into their Rails app. The only problem is that the API uses SOAP. Ruby has basically dropped SOAP in favor of REST. They provide a Java adapter that apparently works with the Java-Ruby bridge, but we'd like to keep it all in Ruby, if possible. I looked into soap4r, but it seems to have a slightly bad reputation.
So what's the best way to integrate SOAP calls into a Rails app?
I built Savon to make interacting with SOAP webservices via Ruby as easy as possible.
I'd recommend you check it out.
We used the built in soap/wsdlDriver class, which is actually SOAP4R.
It's dog slow, but really simple. The SOAP4R that you get from gems/etc is just an updated version of the same thing.
Example code:
require 'soap/wsdlDriver'
client = SOAP::WSDLDriverFactory.new( 'http://example.com/service.wsdl' ).create_rpc_driver
result = client.doStuff();
That's about it
We switched from Handsoap to Savon.
Here is a series of blog posts comparing the two client libraries.
I also recommend Savon. I spent too many hours trying to deal with Soap4R, without results. Big lack of functionality, no doc.
Savon is the answer for me.
Try SOAP4R
SOAP4R
Getting Started with SOAP4R
And I just heard about this on the Rails Envy Podcast (ep 31):
WS-Deathstar SOAP walkthrough
Just got my stuff working within 3 hours using Savon.
The Getting Started documentation on Savon's homepage was really easy to follow - and actually matched what I was seeing (not always the case)
Kent Sibilev from Datanoise had also ported the Rails ActionWebService library to Rails 2.1 (and above).
This allows you to expose your own Ruby-based SOAP services.
He even has a scaffold/test mode which allows you to test your services using a browser.
I have used HTTP call like below to call a SOAP method,
require 'net/http'
class MyHelper
def initialize(server, port, username, password)
#server = server
#port = port
#username = username
#password = password
puts "Initialised My Helper using #{#server}:#{#port} username=#{#username}"
end
def post_job(job_name)
puts "Posting job #{job_name} to update order service"
job_xml ="<soapenv:Envelope xmlns:soapenv=\"http://schemas.xmlsoap.org/soap/envelope/\" xmlns:ns=\"http://test.com/Test/CreateUpdateOrders/1.0\">
<soapenv:Header/>
<soapenv:Body>
<ns:CreateTestUpdateOrdersReq>
<ContractGroup>ITE2</ContractGroup>
<ProductID>topo</ProductID>
<PublicationReference>#{job_name}</PublicationReference>
</ns:CreateTestUpdateOrdersReq>
</soapenv:Body>
</soapenv:Envelope>"
#http = Net::HTTP.new(#server, #port)
puts "server: " + #server + "port : " + #port
request = Net::HTTP::Post.new(('/XISOAPAdapter/MessageServlet?/Test/CreateUpdateOrders/1.0'), initheader = {'Content-Type' => 'text/xml'})
request.basic_auth(#username, #password)
request.body = job_xml
response = #http.request(request)
puts "request was made to server " + #server
validate_response(response, "post_job_to_pega_updateorder job", '200')
end
private
def validate_response(response, operation, required_code)
if response.code != required_code
raise "#{operation} operation failed. Response was [#{response.inspect} #{response.to_hash.inspect} #{response.body}]"
end
end
end
/*
test = MyHelper.new("mysvr.test.test.com","8102","myusername","mypassword")
test.post_job("test_201601281419")
*/
Hope it helps. Cheers.
I have used SOAP in Ruby when i've had to make a fake SOAP server for my acceptance tests. I don't know if this was the best way to approach the problem, but it worked for me.
I have used Sinatra gem (I wrote about creating mocking endpoints with Sinatra here) for server and also Nokogiri for XML stuff (SOAP is working with XML).
So, for the beginning I have create two files (e.g. config.rb and responses.rb) in which I have put the predefined answers that SOAP server will return.
In config.rb I have put the WSDL file, but as a string.
##wsdl = '<wsdl:definitions name="StockQuote"
targetNamespace="http://example.com/stockquote.wsdl"
xmlns:tns="http://example.com/stockquote.wsdl"
xmlns:xsd1="http://example.com/stockquote.xsd"
xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/"
xmlns="http://schemas.xmlsoap.org/wsdl/">
.......
</wsdl:definitions>'
In responses.rb I have put samples for responses that SOAP server will return for different scenarios.
##login_failure = "<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">
<s:Body>
<LoginResponse xmlns="http://tempuri.org/">
<LoginResult xmlns:a="http://schemas.datacontract.org/2004/07/WEBMethodsObjects" xmlns:i="http://www.w3.org/2001/XMLSchema-instance">
<a:Error>Invalid username and password</a:Error>
<a:ObjectInformation i:nil="true"/>
<a:Response>false</a:Response>
</LoginResult>
</LoginResponse>
</s:Body>
</s:Envelope>"
So now let me show you how I have actually created the server.
require 'sinatra'
require 'json'
require 'nokogiri'
require_relative 'config/config.rb'
require_relative 'config/responses.rb'
after do
# cors
headers({
"Access-Control-Allow-Origin" => "*",
"Access-Control-Allow-Methods" => "POST",
"Access-Control-Allow-Headers" => "content-type",
})
# json
content_type :json
end
#when accessing the /HaWebMethods route the server will return either the WSDL file, either and XSD (I don't know exactly how to explain this but it is a WSDL dependency)
get "/HAWebMethods/" do
case request.query_string
when 'xsd=xsd0'
status 200
body = ##xsd0
when 'wsdl'
status 200
body = ##wsdl
end
end
post '/HAWebMethods/soap' do
request_payload = request.body.read
request_payload = Nokogiri::XML request_payload
request_payload.remove_namespaces!
if request_payload.css('Body').text != ''
if request_payload.css('Login').text != ''
if request_payload.css('email').text == some username && request_payload.css('password').text == some password
status 200
body = ##login_success
else
status 200
body = ##login_failure
end
end
end
end
I hope you'll find this helpful!
I was having the same issue, switched to Savon and then just tested it on an open WSDL (I used http://www.webservicex.net/geoipservice.asmx?WSDL) and so far so good!
https://github.com/savonrb/savon