Testing RestClient::Request.execute in RSpec - ruby-on-rails

I am trying to test that 'ApiClient.do_request' is sending the correct arguments and calling 'RestClient::Request.execute.' However, my below specs aren't working as intended. RSpec is unable to properly call 'RestClient::Request.execute,' as the method's 'response' variable keeps coming back as nil, which is why I get the below TypeError when JSON.parse(response) is called. What's strange to me is that I have no issues when walking through the process in Rails console. I'm sure there's something obvious I'm missing here... Any ideas why this is happening, or perhaps suggestions on how I can test this more effectively?
Model:
class ApiClient < ActiveRecord::Base
BASE_PATH = "http://api.bandsintown.com/artists/"
APP_ID = ENV["APP_ID"]
def do_request(method:, base_url:BASE_PATH, app_id:APP_ID, url:, format:"json", options: nil)
response = RestClient::Request.execute(method: method.to_sym,
url: "#{base_url}#{url}.#{format}?api_version=2.0&app_id=#{app_id}#{options}",
timeout: 10)
JSON.parse(response)
end
Spec:
describe ApiClient do
describe "do_request" do
context "when all required arguments are present" do
it "RestClient executes the request" do
#test_client = ApiClient.new
expect(RestClient::Request).to receive(:execute).with(:method=>:get, :url=> "http://api.bandsintown.com/artists/Damien%20Jurado/events/search.json?api_version=2.0&app_id=ShowBoatTest&location=San+Diego,CA&radius=15", :timeout=>10)
#test_client.do_request(method:"get", app_id:"ShowBoatTest",url:"Damien%20Jurado/events/search",options:"&location=San+Diego,CA&radius=15")
end
end
end
RSpec Error:
1) ApiClient do_request when all required arguments are present RestClient executes the request
Failure/Error: JSON.parse(response)
TypeError:
no implicit conversion of nil into String
# /Users/slamflipstrom/.rvm/gems/ruby-2.1.5/gems/json-1.8.3/lib/json/common.rb:155:in `initialize'
# /Users/slamflipstrom/.rvm/gems/ruby-2.1.5/gems/json-1.8.3/lib/json/common.rb:155:in `new'
# /Users/slamflipstrom/.rvm/gems/ruby-2.1.5/gems/json-1.8.3/lib/json/common.rb:155:in `parse'
# ./app/models/api_client.rb:12:in `do_request'
# ./spec/models/api_client_spec.rb:13:in `block (4 levels) in <top (required)>'

Not considering minor syntax errors like api_version=2.0&app_id=ShowBoat in your ApiClient class, the problem seems to be caused by your RestClient::Request.execute request returning nil.
Debug or inspect the value of response and make sure it is not nil.
class ApiClient < ActiveRecord::Base
BASE_PATH = "http://api.bandsintown.com/artists/"
APP_ID = ENV["APP_ID"]
def do_request(method:, base_url:BASE_PATH, app_id:APP_ID, url:, format:"json", options: nil)
response = RestClient::Request.execute(method: method.to_sym,
url: "#{base_url}#{url}.#{format}?api_version=2.0&app_id=#{app_id}#{options}",
timeout: 10)
puts response.inspect
response ? JSON.parse(response) : []
end
On the other hand, you are testing ApiClient class's implementation. In other words, you're testing that the class is internally calling some other class in specific manner. If you decide to replace the RestClient with HTTParty for example at some later time, your tests will fail (even though they shouldn't, because your class is still working correctly).
So, testing the class's implementation makes your tests "fragile". Instead, give an input and test for the expected correct output.
describe ApiClient do
context "with all required arguments" do
describe "#do_request" do
it "returns object in correct format" do
response = subject.do_request(method:"get", app_id:"ShowBoatTest",url:"Damien%20Jurado/events/search",options:"&location=San+Diego,CA&radius=15")
expect(response.body).to include(title: "Damien Jurado # The Casbah in San Diego, CA")
# etc.
end
end
end
end

Related

Stub Httparty call: Wrong number of arguments (given 2, expected 1)

I created a simple ruby file (not Rails) and I am trying to test (using Rspec) a method where I am calling an API. In the test I am trying to mock the call via WebMock but it keeps giving me this error:
Requests::FilesManager#display fetches the files from the API
Failure/Error: Requests::FilesManager.new.display
ArgumentError:
wrong number of arguments (given 2, expected 1)
The files are:
#run.rb
module Requests
require "httparty"
require 'json'
class FilesManager
include HTTParty
def initialize
end
def display
response = HTTParty.get('https://api.publicapis.org/entries', format: :json)
parsed_response = JSON.parse(response.body)
puts "The secret message was: #{parsed_response["message"]}"
end
end
end
and the spec file:
require 'spec_helper'
require_relative '../run'
RSpec.describe Requests::FilesManager do
describe "#display" do
it 'fetches the files from the API' do
stub_request(:get, "https://api.publicapis.org/entries").
to_return(status: 200, body: "", headers: {})
Requests::FilesManager.new.display
end
end
end
EDIT:
So the error seems to come from the line:
JSON.parse(response.body)
If I comment it out it disappears. The problem then is that the output of the call is not a json (even with the format: :json when calling the HTTParty). I tried other solutions but nothing seems to work in making the response json. It is just a string.
Change
response = HTTParty.get('https://api.publicapis.org/entries', format: :json)
to
response = HTTParty.get('https://api.publicapis.org/entries').
I think you don't need the format: :json more so when you explicitly format the response to JSON anyway.
You need to return a json object in the body parameter of the stubbed response:
E.g: For an empty response:
stub_request(:get, "https://api.publicapis.org/entries").
to_return(status: 200, body: "".to_json, headers: {})
OR For a valid response: (Note: You may have to require json to convert a hash to json)
require 'json'
...
stub_request(:get, "https://api.publicapis.org/entries").
to_return(status: 200, body: { entries: { '0': { message: "Hello World" } } }.to_json, headers: {})
Solved!
It seems there was an error because the json gem version that HTTParty uses is too old.
Moved on to RestClient gem for the RESTful API calls. It had another conflict in the mime gem versioning.
Finally moved to Faraday and that solved my problems:
JSON.parse(response.body, :quirks_mode => true)
tl;dr Had the same issue and ended up having to upgrade webmock.
Long form:
Webmock inserts middleware into your calls, so when HTTParty makes the calls they end up going through the Webmock interfaces first.
You can verify this by trying the call standalone (withouth all the rspec config):
bundle console
irb> require "httparty"
=> true
irb> httparty.get("https://google.com")
If that standalone call succeeds, the issue is somewhere within Webmock itself.
For me, somewhere along the line of calls through Webmock was an outdated interface that was incompatible and throwing the Wrong Number of Arguments error. And this was also crashing my debugger (RubyMine).
Upgrading Webmock solved this issue (because they had fixed it in newer versions).

Mock GoogleAPI request

I am using the google-maps gem.
I am trying to mock/stub api requests unsuccessfully.
module GoogleMap
class Route
attr_accessor :start_location, :end_location
def initialize(start_location, end_location)
#start_location = start_location
#end_location = end_location
end
def driving_duration_in_seconds
route.duration.value
end
def driving_distance_in_meters
route.distance.value
end
def driving_distance_hash
return unless start_location && end_location
{ distance_in_meters: driving_distance_in_meters, duration_in_seconds: driving_duration_in_seconds }
end
private
def coordinates_as_strings(location)
"#{location.latitude},#{location.longitude}"
end
def route
#route ||= Google::Maps.route(coordinates_as_strings(start_location), coordinates_as_strings(end_location))
end
end
end
I need to stub:
WebMock::NetConnectNotAllowedError:
Real HTTP connections are disabled. Unregistered request: GET https://maps.googleapis.com/maps/api/directions/json?destination=37.19687112,116.43791248&key=<apikey>&language=en&origin=2.15362819,-81.63712649 with headers {'Accept'=>'*/*', 'Date'=>'Sat, 12 Feb 2022 21:35:55 GMT', 'User-Agent'=>'HTTPClient/1.0 (2.8.3, ruby 2.7.2 (2020-10-01))'}
You can stub this request with the following snippet:
stub_request(:get, "https://maps.googleapis.com/maps/api/directions/json?destination=37.19687112,116.43791248&key=<apikey>&language=en&origin=2.15362819,-81.63712649").
with(
headers: {
'Accept'=>'*/*',
'Date'=>'Sat, 12 Feb 2022 21:35:55 GMT',
'User-Agent'=>'HTTPClient/1.0 (2.8.3, ruby 2.7.2 (2020-10-01))'
}).
to_return(status: 200, body: "", headers: {})
If I try a most basic stub I get an error:
stub_request(:any, /maps.googleapis.com/).
to_return(status: 200, body: '', headers: {})
Google::Maps::InvalidResponseException:
unknown error: 783: unexpected token at ''
# .../gems/ruby-2.7.2/gems/google-maps-3.0.7/lib/google_maps/api.rb:64:in `rescue in response'
# .../gems/ruby-2.7.2/gems/google-maps-3.0.7/lib/google_maps/api.rb:60:in `response'
# .../.rvm/gems/ruby-2.7.2/gems/google-maps-3.0.7/lib/google_maps/api.rb:27:in `query'
I think it is erroring out because I am not passing a key in. But I don't see why I should have to pass in a valid api key into a webmock.
I also would not have my route defined by anything. And in order to test that route can return route.distance.value etc, I would need to mock with something.
For other tests I was successful in mocking instances, but to test this lib that it actually works, I feel like mocking instance methods and not that an api was actually called is a waste of a test. Maybe this is just a waste of time, and I should assume it works because I am using a gem.
But I was expecting something like this:
RSpec.describe GoogleMap do
let(:start_location) { create(:location) }
let(:end_location) { create(:location) }
context 'GoogleMaps::Route.new(start_location, end_location)' do
let(:subject) { GoogleMap::Route.new(start_location, end_location) }
# I have not been successful in stubbing this with the correct regex
# stub_request(:get, "https://maps.googleapis.com/maps/api/directions/json?destination=<lat>,<long>key=<key>&language=en&origin=<lat>,<long>").
# with(
# headers: {
# 'Accept'=>'*/*',
# 'Date'=>'Thu, 10 Feb 2022 21:09:02 GMT',
# 'User-Agent'=>'HTTPClient/1.0 (2.8.3, ruby 2.7.2 (2020-10-01))'
# }).
# to_return(status: 200, body: "", headers: {})
# stub_request(:get, %r{https:\/\/maps\.googleapis\.com\/maps\/api\/directions\/json\?destination=.+,.+&key=.+&language=en&origin=.+,.+}).
# stub_request(:any, /maps.googleapis.com/).
# to_return(status: 200, body: '', headers: {})
xit 'gives driving distance in seconds'
xit 'gives driving duration in meters'
end
end
Your WebMock is working fine. Google::Maps::InvalidResponseException is raised after WebMock has replaced the network call. At the point that exception is raised, the Google Maps API client is trying to parse what the network call returned, which is ''.
It's expecting some valid JSON to be returned. If you have your mock return {} is should get past that line. It may well stumble on some other exception later though, as the gem expects a certain schema.
You can dig that out and add in a valid response if you wanted to continue down this path. However, I'd recommend not mocking the network request as that's an implementation detail of a third party piece of code which could change at any time - making your test fail. Instead, I would mock out Google::Maps.route to return what you need it to.

How do I test whether a Sidekiq worker is sending the right data to an external API?

I have a Sidekiq worker that reaches out to an external API to get some data back. I am trying to write tests to make sure that this worker is designed and functioning correctly. The worker grabs a local model instance and examines two fields on the model. If one of the fields is nil, it will send the other field to the remote API.
Here's the worker code:
class TokenizeAndVectorizeWorker
include Sidekiq::Worker
sidekiq_options queue: 'tokenizer_vectorizer', retry: true, backtrace: true
def perform(article_id)
article = Article.find(article_id)
tokenizer_url = ENV['TOKENIZER_URL']
if article.content.nil?
send_content = article.abstract
else
send_content = article.content
end
# configure Faraday
conn = Faraday.new(tokenizer_url) do |c|
c.use Faraday::Response::RaiseError
c.headers['Content-Type'] = 'application/x-www-form-urlencoded'
end
# get the response from the tokenizer
resp = conn.post '/tokenize', "content=#{URI.encode(send_content)}"
# the response's body contains the JSON for the tokenized and vectorized article content
article.token_vector = resp.body
article.save
end
end
I want to write a test to ensure that if the article content is nil that the article abstract is what is sent to be encoded.
My assumption is that the "right" way to do this would be to mock responses with Faraday such that I expect a specific response to a specific input. By creating an article with nil content and an abstract x I can mock a response to sending x to the remote API, and mock a response to sending nil to the remote API. I can also create an article with x as the abstract and z as the content and mock responses for z.
I have written a test that generically mocks Faraday:
it "should fetch the token vector on ingest" do
# don't wait for async sidekiq job
Sidekiq::Testing.inline!
# stub Faraday to return something without making a real request
allow_any_instance_of(Faraday::Connection).to receive(:post).and_return(
double('response', status: 200, body: "some data")
)
# create an attrs to hand to ingest
attrs = {
data_source: #data_source,
title: Faker::Book.title,
url: Faker::Internet.url,
content: Faker::Lorem.paragraphs(number: 5).join("<br>"),
abstract: Faker::Book.genre,
published_on: DateTime.now,
created_at: DateTime.now
}
# ingest an article from the attrs
status = Article.ingest(attrs)
# the ingest occurs roughly simultaneously to the submission to the
# worker so we need to re-fetch the article by the id because at that
# point it will have gotten the vector saved to the DB
#token_vector_article = Article.find(status[1].id)
# we should've saved "some data" as the token_vector
expect(#token_vector_article.token_vector).not_to eq(nil)
expect(#token_vector_article.token_vector).to eq("some data")
end
But this mocks 100% of uses of Faraday with :post. In my particular case, I have no earthly idea how to mock a response of :post with a specific body...
It's also possible that I'm going about testing this all wrong. I could be instead testing that we are sending the right content (the test should check what is being sent with Faraday) and completely ignoring the right response.
What is the correct way to test that this worker does the right thing (sends content, or sends abstract if content is nil)? Is it to test what's being sent, or test what we are getting back as a reflection of what's being sent?
If I should be testing what's coming back as a reflection of what's being sent, how do I mock different responses from Faraday depending on the value of something being sent to it/
** note added later **
I did some more digging and thought, OK, let me test that I'm sending the request I expect, and that I'm processing the response correctly. So, I tried to use webmock.
it "should fetch token vector for article content when content is not nil" do
require 'webmock/rspec'
# don't wait for async sidekiq job
Sidekiq::Testing.inline!
request_url = "#{ENV['TOKENIZER_URL']}/tokenize"
# webmock the expected request and response
stub = stub_request(:post, request_url)
.with(body: 'content=y')
.to_return(body: 'y')
# create an attrs to hand to ingest
attrs = {
data_source: #data_source,
title: Faker::Book.title,
url: Faker::Internet.url,
content: "y",
abstract: Faker::Book.genre,
published_on: DateTime.now,
created_at: DateTime.now
}
# ingest an article from the attrs
status = Article.ingest(attrs)
# the ingest occurs roughly simultaneously to the submission to the
# worker so we need to re-fetch the article by the id because at that
# point it will have gotten the vector saved to the DB
#token_vector_article = Article.find(status[1].id)
# we should have sent a request with content=y
expect(stub).to have_been_requested
# we should've saved "y" as the token_vector
expect(#token_vector_article.token_vector).not_to eq(nil)
expect(#token_vector_article.token_vector).to eq("y")
end
But I think that webmock isn't getting picked up inside the sidekiq job, because I get this:
1) Article tokenization and vectorization should fetch token vector for article content when content is not nil
Failure/Error: expect(stub).to have_been_requested
The request POST https://zzzzz/tokenize with body "content=y" was expected to execute 1 time but it executed 0 times
The following requests were made:
No requests were made.
============================================================
If I try to include webmock/rspec in any of the other places, for example, at the beginning of my file, random things start to explode. For example, if I have these lines in the beginning of this spec file:
require 'spec_helper'
require 'rails_helper'
require 'sidekiq/testing'
require 'webmock/rspec'
Then I get:
root#c18df30d6d22:/usr/src/app# bundle exec rspec spec/models/article_spec.rb:174
database: test
Run options: include {:locations=>{"./spec/models/article_spec.rb"=>[174]}}
There was an error creating the elasticsearch index for Article: #<NameError: uninitialized constant Faraday::Error::ConnectionFailed>
There was an error removing the elasticsearch index for Article: #<NameError: uninitialized constant Faraday::Error::ConnectionFailed>
Which I am guessing is because the test suite is trying to initialize stuff, but webmock is interfering...
I ended up abandoning Faraday and a more complicated test as an approach. I decomposed the worker into both a Service class and a worker. The worker simply invokes the Service class. This allows me to test the service class directly, and then just validate that the worker calls the service class correctly, and that the model calls the worker correctly.
Here's the much simpler service class:
require 'excon'
# this class is used to call out to the tokenizer service to retrieve
# a tokenized and vectorized JSON to store in an article model instance
class TokenizerVectorizerService
def self.tokenize(content)
tokenizer_url = ENV['TOKENIZER_URL']
response = Excon.post("#{tokenizer_url}/tokenize",
body: URI.encode_www_form(content: content),
headers: { 'Content-Type' => 'application/x-www-form-urlencoded' },
expects: [200])
# the response's body contains the JSON for the tokenized and vectorized
# article content
response.body
end
end
Here's the test to see that we are calling the right destination:
require 'rails_helper'
require 'spec_helper'
require 'webmock/rspec'
RSpec.describe TokenizerVectorizerService, type: :service do
describe "tokenize" do
it "should send the content passed in" do
request_url = "#{ENV['TOKENIZER_URL']}/tokenize"
# webmock the expected request and response
stub = stub_request(:post, request_url).
with(
body: {"content"=>"y"},
headers: {
'Content-Type'=>'application/x-www-form-urlencoded',
}).
to_return(status: 200, body: "y", headers: {})
TokenizerVectorizerService.tokenize("y")
expect(stub).to have_been_requested
end
end
end

Cannot bypass Omniauth initializer in test mode

I am trying to test my graphql schema without any need for authentication.
I have added skip_before_action :verify_authenticity_token to the GraphqlController, and when using postman (copied by a curl request from graphiql), and I am seeing a successful query in development mode.
In postman I have the query in the body {"query":"{\n user(id: 1) {\n id\n created_at\n updated_at\n jwt\n}\n}\n","variables":null,"operationName":null}, and Content-Type application/json in the header and this works fine.
Now in test mode, I am hitting the auth initializer for Omniauth:
Rails.application.config.middleware.use OmniAuth::Builder do
provider(
:auth0,
Auth0::Config["app_client_id"],
Auth0::Config["app_client_secret"],
Auth0::Config["domain"],
callback_path: "/auth/auth0/callback"
)
end
Although I don't want to since I don't want any headers required in this post request.
Here is my rspec request:
require 'graphlient'
RSpec.shared_context "GraphQL Client", shared_context: :metadata do
let(:client) do
Graphlient::Client.new('https://api.example.org/graphql') do |client|
client.http do |h|
h.connection do |c|
c.use Faraday::Adapter::Rack, app
end
end
end
end
end
and here's the actual test
it 'retrieves schema' do
expect(client.schema).to be_a GraphQL::Schema
end
with the error:
Failure/Error:
expect { client.schema.status }
.to raise_error(Graphlient::Errors::ServerError)
expected Graphlient::Errors::ServerError, got #<ArgumentError: Received wrong number of arguments. [nil, nil, nil, {:callback_path=>"/auth/auth0/callback"}]> with backtrace:
# /usr/local/bundle/gems/omniauth-auth0-1.4.2/lib/omniauth/strategies/auth0.rb:41:in `initialize'
I think I got it! I forgot to add keys for test in config/auth0.yml.
It was a hidden file.

Is there a default response for Rspec controller tests when no action is passed?

I came accross this controller spec which has no action passed. It's clear for me that it's not testing anything. But the expectation for a response.status == 200 doesn't fail. So I'd like to understand how Rspec builds the controller tests and if there's a default response.status == 200.
describe 'SomeController', type: :controller do
let!(:user){ FactoryGirl.create :admin_user }
before { sign_in user }
describe 'GET#action' do
it 'response is success' do
expect(response.status).to eq 200
end
end
end
Finished in 0.93954 seconds (files took 1.63 seconds to load)
1 example, 0 failures
As the RSpec controller spec documents point out:
A controller spec is an RSpec wrapper for a Rails functional test
(ActionController::TestCase::Behavior).
So, looking at the the in-code documentation for ActionController::TestCase::Behavior, under the Special Instance Variables section, we can see that ActionController::TestCase will automatically provide a #response instance variable (readable as just response in the test), which is "an ActionDispatch::TestResponse object, representing the response of the last HTTP response". So, that would seem to explain why there is a response able to be accessed without needing an explicit request made in a controller spec, but why is it's status 200?
Well, ActionDispatch::TestResponse inherits from ActionDispatch::Response, which when initialized provides 200 as the default status. You can even test this out in your rails console:
> ActionDispatch::TestResponse.new
=> #<ActionDispatch::TestResponse:0x007fc449789b68
#blank=false,
#cache_control={},
#charset=nil,
#committed=false,
#content_type=nil,
#cv=
#<MonitorMixin::ConditionVariable:0x007fc449789848
#cond=#<Thread::ConditionVariable:0x007fc449789820>,
#monitor=#<ActionDispatch::TestResponse:0x007fc449789b68 ...>>,
#etag=nil,
#header={"X-Frame-Options"=>"SAMEORIGIN", "X-XSS-Protection"=>"1; mode=block", "X-Content-Type-Options"=>"nosniff"},
#mon_count=0,
#mon_mutex=#<Thread::Mutex:0x007fc449789a50>,
#mon_owner=nil,
#sending=false,
#sending_file=false,
#sent=false,
#status=200, # <<< Here's your default status.
#stream=#<ActionDispatch::Response::Buffer:0x007fc449789938 #buf=[], #closed=false, #response=#<ActionDispatch::TestResponse:0x007fc449789b68 ...>>>
So, I hope this deep dive assisted in your understanding of the response object in RSpec controller specs, as it certainly did mine.

Resources