debugging activeresource - ruby-on-rails

I'm trying to get activeresource (on Rails 3.2) working with the Freebase API and I haven't had much luck yet. How can I debug rails to see what's going on and make sure the request is well formed?
I think the suffixed .json is causing the failure but I don't know how to check what's going on?
Error:
ActiveResource::ResourceNotFound: Failed. Response code = 404. Response message = Not Found.
Code:
class Freebase < ActiveResource::Base
class << self # also tried without this, same result
def element_path(id, prefix_options = {}, query_options = nil)
prefix_options, query_options = split_options(prefix_options) if query_options.nil?
"#{prefix(prefix_options)}#{collection_name}/#{id}#{query_string(query_options)}"
end
def collection_path(prefix_options = {}, query_options = nil)
prefix_options, query_options = split_options(prefix_options) if query_options.nil?
"#{prefix(prefix_options)}#{collection_name}#{query_string(query_options)}"
end
end
self.site = "https://www.googleapis.com/freebase/v1/"
self.format = :json
#https://www.googleapis.com/freebase/v1/search?query=nirvana&indent=true
#Freebase.get('search', :query => 'nirvana')
end
UPDATE:
Ok so I found two things going wrong...
1) The collection_path I'm trying to supersede doesn't work at all.. it still tacks .json onto every request.
2) https://www.googleapis.com:443/freebase/v1/freebases/search.json?query=nirvana
It tacks freebase on afterwards... any ideas?
I also tried this fix:
Remove .xml extension from ActiveResource request
But it didn't remove the JSON suffix either.
UPDATE UPDATE:
Added the suggest update below which gives the correct PATH but now I'm getting
GET https://www.googleapis.com:443/freebase/v1/search/?query=monkey
--> 200 OK 2732 (693.3ms)
NoMethodError: undefined method `collect!' for #<Hash:0x007f9bde674900>

Add ActiveResource::Base.logger = Logger.new(STDERR) to your config/application.rb ( Rails 3.x ).
You'll get output like :
POST http://localhost:3000/freebase.json
--> 201 Created 0 (15.8ms)
That shows method and response code ...

ActiveResource isn't dead, but relative to ActiveRecord I can understand why you'd think so, it is definitely an unloved and underprivileged stepchild.
You'd probably be better off using something like Faraday_Middleware or HTTPParty. The former is my personal preference. Unless what you're doing is pulling from another Rails app or one that has perfect restful behaviour like Rails (which Freebase doesn't), ActiveResource is usually more trouble than it's worth.
That being said, you can accomplish what you want without overwriting any class methods by doing:
self.site = "https://www.googleapis.com/"
self.format = :json
def self.search(word)
self.find(:all, :from => "/freebase/v1/search/", :params => { :query => word })
end

To get detail login for ActiveResource have to patch the request method inside the gem(method.
place bellow files inside config/initializers you will get http method, path, request body, request hedaers
response body and header is already there if you need. doc
config/initializers/activeresource_patch.rb
module ActiveResource
class Connection
private
def request(method, path, *arguments)
result = ActiveSupport::Notifications.instrument("request.active_resource") do |payload|
payload[:method] = method
payload[:request_uri] = "#{site.scheme}://#{site.host}:#{site.port}#{path}"
payload[:request_path] = path
payload[:request_body] = arguments[0]
payload[:request_headers] = arguments[1]
payload[:result] = http.send(method, path, *arguments)
end
handle_response(result)
rescue Timeout::Error => e
raise TimeoutError.new(e.message)
rescue OpenSSL::SSL::SSLError => e
raise SSLError.new(e.message)
end
end
end
config/initializers/activeresource_logger.rb
Rails.application.configure do
def activeresource_logger
#activeresource_logger ||= Logger.new("#{Rails.root}/log/activeresource_logger.log")
end
ActiveSupport::Notifications.subscribe('request.active_resource') do |name, start, finish, id, payload|
if Rails.env.development?
activeresource_logger.info("====================== #{start} : #{payload[:method].upcase} ======================")
activeresource_logger.info("PATH: #{payload[:request_path]}")
activeresource_logger.info("BODY: #{payload[:request_body]}")
activeresource_logger.info("HEADERS: #{payload[:request_headers]}")
# activeresource_logger.info("STATUS_CODE: #{payload[:result].code}")
# activeresource_logger.info("RESPONSE_BODY: #{payload[:result].body}")
end
end
end

ActiveResource is going to expect that endpoint to return the data in a very specific format.
For the longest time we've been using ActiveResource at my company for inter-application communication. However more recently we've started leaning towards HTTParty because it performs a lot less voodoo magic, and tends to be a much small exercise in hair-pulling.
Here's an example of how we're using HTTParty now:
module CoreResources
class Job
include HTTParty
base_uri Rails.configuration.core_resource_uri
basic_auth Rails.configuration.core_resource_user, Rails.configuration.core_resource_password
def self.search(entity)
get("/api/v1/jobs.json", :query => {:entity_id => entity.id})
end
def self.find(id)
result = get("/api/v1/jobs/#{id}.json")
raise CoreResources::JobNotFound.new if result.response.code == "404"
raise "Unexpected response from resource job find: #{result.response.code} #{result.response.to_s}" if result.response.code =~ /^(?:4|5)..$/
result
end
end
end
The problem with ActiveResource is that it will take the very specifically-crafted json or xml markup, and instantiate ActiveResource objects and nested objects based on it. It's having issues calling collect because something in the json response was supposed to be formatted like an enumerable, and wasn't (likely the parent node should have been an array or something, not sure), which makes it blow up.
With HTTParty you get a json-parsed collection to work with.
It's easy enough for me to do this:
jobs = CoreResources::Job.search(my_entity)
puts jobs.inspect
# [{
# "id" => 4,
# "created_by_id" => 12,
# "description" => "I like pie"
# },
# {
# "id" => 5",
# "created_by_id" => 12,
# "description" => "Mmm, cake"
# }]
Which let's me access jobs via an easy collection array/hash construct jobs[0].fetch("description"), as opposed to ActiveResource: jobs[0].description. ActiveResource is slower to insantiate those collections, needlessly takes up memory with them, and encourages you to duplicate code that should just be served by the endpoint in your ActiveResource model (Then again, if you're using a third-party API you may have no other choice, but I have never successfully gotten ARes to interface with third-party API's).
We've run into a lot of other ActiveResource problems where it does this nonsensical dynamic creation of class names based on nested resources from your endpoint, but half the time does it incorrectly... It's really just a mess.
Moral of the story: Much more of a fan of HTTParty now. That endpoint is probably just not returning data in the right format, and unless it does ActiveResource will be a hackfest to get it to read it right.

ActiveResource has a fairly narrow use-case. You probably want to use a more generic interface for working with Freebase.
Some code from the LinkTV Platform's FreeBase API might be of some help.

The NoMethodError: undefined method 'collect!' for #<Hash:0x007f9bde674900>
error seems to be an issue on rails https://github.com/rails/rails/issues/2318 .
I had a similar problem but the hacks provided didn't work so I had to tweak them a bit , you can find my answer here .

Related

Rails Httparty JSON to params to save

Rails 4.5 Ruby 2.3.1
I am getting json from an API and trying to store the following into a model CLrates
1. timestamp as unix time (date)
2. Currency_code (string)
3. quote (decimal monetary value)
I can use the following in irb after parsing the json and know how to get the elements individually using: response["quotes"]. How can I generate params to be saved in the model above when the body is as follows:
irb(main):036:0> puts response.body
{
"success":true,
"terms":"https:\/\/xxxxx.com\/terms",
"privacy":"https:\/\/xxxxx.com\/privacy",
"timestamp":1504817289,
"source":"USD",
"quotes":{
"USDAED":3.672703,
"USDAFN":68.360001,
"USDCUC":1,
"USDCUP":26.5,
"USDCVE":91.699997,
"USDCZK":21.718701,
............ many more lines removed for brevity
"USDZWL":322.355011
}
I can do this using a separate associated model but have very little idea how to create the params to save to a single table.
The following links got me to this point and well worth a read if you need info on httparty GET (client):
1. http://www.rubydoc.info/github/jnunemaker/httparty/HTTParty/
2. http://eric-price.net/blog/rails-api-wrapper/
3. https://www.driftingruby.com/episodes/playing-with-json
The class and method in lib/clayer.rb:
class clayer
include HTTParty
format :json
read_timeout 10
def self.get_quotes
response = HTTParty.get('http://www.nnnnnnn.net/api/live?
access_key=nnnnnnnnnn&format=1')
end
end
I used irb as I am still learning how to run this through rails c. This will be called in the controller and saved however need to work out how to get the params from the json
Thanks for the help
OK: after digging I think I am on the right track
I get the response["QUOTES"], loop through them and build the params required saving each at the end of the loop
rates = response["QUOTES"]
rates.each do |k,v|
clrate = Realtimerates.new
clrate.date = response["timestamp"]
clrate.countrycode = "#{k}"
clrate.price = "#{v}"
clrate.save
end
Going to give this a whirl
In model
class Realtimerate < ActiveRecord::Base
include HTTParty
format :json
read_timeout 5
def self.get_cl_rates
response = HTTParty.get('http://www.mmmmm.net/api/live?access_key="key"&format=1')
rates = response["quotes"]
rates.each do |k,v|
create!(
date: Time.at(response["timestamp"]),
country_code: "#{k}",
price: "#{v}")
end
end
end
In the controller:
def index
Realtimerate.get_cl_rates
#realtimerates = Realtimerate.all
end
This is working and shows latest GET.
You already have a hash in your response.body. All you need to do now is to assign the relevant key-value to your model's attributes. e.g.
clrate = ClRate.new
res = response.body
clate.time = res["timestamp"]
...
clate.save

ActiveResource choking on freebase json request

I've got ActiveResource setup to consume the freebase api in json, and it should work fine except that the json freebase returns causes ActiveResource to blowup.
NoMethodError: undefined method `collect!' for #<Hash:0x007fd674831dd0>
How can I define a custom json parser to fix whatever is wrong?
class Freebase < ActiveResource::Base
self.site = "https://www.googleapis.com/"
self.format = :json
def self.search(word)
self.find(:all, :from => "/freebase/v1/search/", :params => { :query => word })
end
#https://www.googleapis.com/freebase/v1/search?query=nirvana
#Freebase.get('search', :query => 'nirvana')
end
Json being returned:
https://www.googleapis.com/freebase/v1/search?query=nirvana
{"status":"200 OK","result":[{"mid":"/m/05b3c","name":"Nirvana","notable":{"name":"Belief","id":"/religion/belief"},"lang":"en","score":67.540009},{"mid":"/m/0b1zz","name":"Nirvana","notable":{"name":"Musical Artist","id":"/music/artist"},"lang":"en","score":64.311432},{"mid":"/m/092bf5","name":"Buddhism","notable":{"name":"Religion","id":"/religion/religion"},"lang":"en","score":33.647118},{"mid":"/m/02_6qh","name":"Nirvana","notable":{"name":"Film","id":"/film/film"},"lang":"en","score":30.068491},{"mid":"/m/01h89tx","name":"Nirvana","notable":{"name":"Musical Album","id":"/music/album"},"lang":"en","score":27.799274},{"mid":"/m/01rn9fm","name":"Nirvana","notable":{"name":"Musical Group","id":"/music/musical_group"},"lang":"en","score":27.445602},{"mid":"/m/015k7","name":"Gautama Buddha","notable":{"name":"Deity","id":"/religion/deity"},"lang":"en","score":24.129679},{"mid":"/m/01rkx5","name":"Mahayana Mahaparinirvana Sutra","lang":"en","score":22.359026},{"mid":"/m/03d7q7v","name":"Nirvana","lang":"en","score":21.034473},{"mid":"/m/055ym7w","name":"Nirvana bootleg recordings","notable":{"name":"Musical Album","id":"/music/album"},"lang":"en","score":19.241596},{"mid":"/m/0122_j","name":"Nevermind","notable":{"name":"Musical Album","id":"/music/album"},"lang":"en","score":18.366383},{"mid":"/m/04n7mt","name":"Nirvana fallacy","lang":"en","score":17.212397},{"mid":"/m/0484q","name":"Kurt Cobain","notable":{"name":"Musician","id":"/m/09jwl"},"lang":"en","score":16.594929},{"mid":"/m/027_k8j","name":"Nirvana","lang":"en","score":16.336584},{"mid":"/m/0285c","name":"Dave Grohl","notable":{"name":"Musician","id":"/m/09jwl"},"lang":"en","score":16.115103},{"mid":"/m/068shv","name":"Smells Like Nirvana","notable":{"name":"Musical Album","id":"/music/album"},"lang":"en","score":15.350652},{"mid":"/m/01kq85c","name":"Manic Nirvana","notable":{"name":"Musical Album","id":"/music/album"},"lang":"en","score":15.275189},{"mid":"/m/0437sc","name":"Lithium","notable":{"name":"Composition","id":"/music/song"},"lang":"en","score":14.637386},{"mid":"/m/055kh1","name":"Mechanus","lang":"en","score":14.621847},{"mid":"/m/01f1vf","name":"Lucifer","notable":{"name":"Fictional Character","id":"/fictional_universe/fictional_character"},"lang":"en","score":13.504528}],"cursor":20,"cost":11,"hits":3104}
Using Rails 3.2.1
UPDATE:
Found this issue, but still not sure how to overcome it. https://github.com/rails/rails/issues/2318
The problem is that the JSON returned isn't formed the way ActiveResource expects it to be. ARes doesn't expect all of that metadata that is being returned, it is only expecting what is in the results portion of the response.
To be explicit, you're getting back:
{"status":"200 OK","result":[{"mid":"/m/05b3c","name":"Nirvana","notable":{"name" ...
But ARes wants:
[{"mid":"/m/05b3c","name":"Nirvana","notable":{"name"...
The easiest (and probably dirtiest) solution I can come up with is to overwrite the ActiveResource::Base#find_every private method in your model like so:
class Freebase < ActiveResource::Base
self.site = "https://www.googleapis.com/"
self.format = :json
def self.search(word)
self.find(:all, :from => "/freebase/v1/search/", :params => { :query => word })
end
private
def self.find_every(options)
begin
case from = options[:from]
when Symbol
instantiate_collection(get(from, options[:params]))
when String
path = "#{from}#{query_string(options[:params])}"
instantiate_collection(format.decode(connection.get(path, headers).body['result']) || [])
else
prefix_options, query_options = split_options(options[:params])
path = collection_path(prefix_options, query_options)
instantiate_collection( (format.decode(connection.get(path, headers).body['result']) || []), prefix_options )
end
rescue ActiveResource::ResourceNotFound
# Swallowing ResourceNotFound exceptions and return nil - as per
# ActiveRecord.
nil
end
end
end
The only change I made is the .body method calls are now .body['result']. With this addition, #instantiate_collection will now receive an Array as it expects rather than a Hash.
Ultimately, while this should make the error go away, I don't know that this will solve all of your problems as you might need some of the data that gets lost with this method. My suggestion would be to abandon ActiveResource if you can and use something like RestClient and build your model around that. Another reason for this approach is that the semantics as they are now are broken. You aren't getting back a collection of Freebases. You're using the Freebase API to get search results.
I followed Derek's advice and abandoned ActiveResource...
for anyone else I ended up using Faraday and Faraday_Middleware
class Freebase
def self.search(id)
connection = Faraday.new 'https://www.googleapis.com/freebase/v1' do |conn|
conn.adapter Faraday.default_adapter
conn.use FaradayMiddleware::ParseJson
#conn.use Faraday::Response::Mashify
end
response = connection.get do |req|
req.url('search', :query => id, :limit => 10)#, :filter => '(any namespace:/wikipedia/en_id namespace:/authority/imdb/title)')# #:filter => '(any namespace:/wikipedia/en_id namespace:/authority/imdb/title namespace:/authority/netflix/movie)' , :with => 'commons'
end
end
Supposedly you can use several built in modules to make rails style methods out of the returned hash, but I haven't figured that part out yet. ActiveResource seems only good at other Rails apps or RESTFUL routes that perfectly imitate it.

Rails Builder::XmlMarkup for Web Service - Repetitive Section

I am using Builder to construct XML messages being sent to a WebService. Each of the different methods require different xml but they all have a set of common elements to start of the request (mostly account authentication stuff). Is there any way to do it in a DRY way? Here is my code for constructing a change pass phrase request:
# XML REQUEST SETUP
msg = Builder::XmlMarkup.new(:indent=>2)
query = {}
test_hsh = self.testmode ? {:Test => "YES"} : {}
# BUILD THE REQUEST
query[:changePassPhraseRequestXML] = msg.ChangePassPhraseRequest(test_hsh) do |asr|
asr.RequesterID APP_CONFIG[:endicia_partner_id].to_s
asr.RequestID "1"
asr.CertifiedIntermediary do |ci|
ci.AccountID APP_CONFIG[:endicia_account_number].to_s
ci.PassPhrase APP_CONFIG[:endicia_passphrase].to_s
end
asr.NewPassPhrase APP_CONFIG[:passphrase].to_s
end
Basically all the elements except the NewPassPhrase one are common to all (or most) requests. Right now I copy the same code over and over but I don't like this at all.
Any ideas on Dry'ing it up?
As soon as I posted this. I had an idea, put the first set into their own method. Duh!
def account_status(options = {})
# XML REQUEST SETUP
msg = Builder::XmlMarkup.new(:indent=>2)
query = {}
test_hsh = self.testmode ? {:Test => "YES"} : {}
# BUILD THE REQUEST
query[:changePassPhraseRequestXML] = msg.ChangePassPhraseRequest(test_hsh) do |asr|
self.add_authentication_elements(asr)
asr.NewPassPhrase APP_CONFIG[:new_pass_phrase].to_s
end
end
def add_authentication_elements(parent_node)
parent_node.RequesterID self.endicia_partner_id.to_s
parent_node.RequestID "1"
parent_node.CertifiedIntermediary do |ci|
ci.AccountID self.endicia_account_number.to_s
ci.PassPhrase self.endicia_passphrase.to_s
end
end
Works great! Another option would of course be to extend Builder in some way but this is nice and simple.

SimpleDB to ActiveResource. Rails

Im looking for a way to map an ActiveResource to SimpleDB
I want to avoid plugins/gems as all I have used are outdated/buggy/not mantained
It doesnt seem hard, I wonder if any of you have succesfully implemented a rails app with simpleDB as an Active Resource. How did you do it? Thanks.
I haven't worked with SimpleDB, but I have mapped ActiveResource to Amazon's Flexible Payments Service REST api and just skimming the docs they seem similar so here's basically what I did, maybe you can use this as a starting point.
require 'base64'
require 'openssl'
class AmazonFlexiblePaymentResource < ActiveResource::Base
self.site = AMZ_CONFIG['flexible_api_url']
def self.rest_api(options = {})
params = common_request_params.update(options)
sig = compute_signature(AMZ_CONFIG['secret_access_key'], 'get', site, params)
rest_req = {'Signature' => sig}.update(params)
# make the http get call
connection.get("/#{query_string(rest_req)}", headers)
end
protected
# these are the params are common to all rest api calls
def self.common_request_params
{ 'AWSAccessKeyId' => AMZ_CONFIG['access_key_id'],
'SignatureVersion' => 2,
'SignatureMethod' => 'HmacSHA256',
'Timestamp' => Time.now.utc.iso8601,
'Version' => '2008-09-17'}
end
def self.compute_signature(key, method, end_point_url, params)
query_str = parameters.sort.collect {|k, v| v.to_query(k)}.join '&'
# cannot use plus for space, and tilde needs to be reversed
query_str.gsub!('+', '%20')
query_str.gsub!('%7E', '~')
to_sign = [method.upcase, end_point_uri.host.downcase,
end_point_uri.request_uri, query_str].join "\n"
digest = OpenSSL::Digest::Digest.new('sha256')
hmac = OpenSSL::HMAC.digest(digest, key, to_sign)
Base64.encode64(hmac).chomp
end
end
Then I just make calls like this
res = AmazonFlexiblePaymentResource.rest_api({ 'Action' => 'GetTransactionStatus', 'TransactionId' => '1234567890ABCDEFGHIJ' })
And the response is a hash of the parsed xml. Again this works for Amazon Flexible Payments Service, so you may need to make adjustments to match the SimpleDB REST API.
Does it need to be ActiveResource? If you want it like ActiveRecord, check out SimpleRecord.
http://github.com/appoxy/simple_record
It's very actively maintained.

Disabled/Custom params_parser per action

I have a create action that handles XML requests. Rather than using the built in params hash, I use Nokogiri to validate the XML against an XML schema. If this validation passes, the raw XML is stored for later processing.
As far as I understand, the XML is parsed twice: First the Rails creates the params hash, then the Nokogiri parsing happens. I've been looking for ways to disable the params parsing to speed things up but have found nothing.
ActionController::Base.param_parsers[Mime::XML] = lambda do |body|
# something
end
I know it's possible to customize the XML params parsing in general using something like the above, but I depend on the default behaviour in other controllers.
Is it possible to bypass the params parsing on a per-action basis? What options do I have?
Thank you for your help!
I've managed to solve the problem using Rails Metal. The relevant part looks something like this:
class ReportMetal
def self.call(env)
if env["PATH_INFO"] =~ /^\/reports/
request = Rack::Request.new(env)
if request.post?
report = Report.new(:raw_xml => request.body.string)
if report.save # this triggers the nokogiri validation on raw_xml
return [201, { 'Content-Type' => 'application/xml' }, report.to_xml]
else
return [422, { 'Content-Type' => 'application/xml' }, report.errors.to_xml]
end
end
end
[404, { "Content-Type" => "text/html" }, "Not Found."]
ensure
ActiveRecord::Base.clear_active_connections!
end
end
Thanks!
PS: Naive benchmarking with Apache Bench in development shows 22.62 Requests per second for standard Rails vs. 57.60 Requests per second for the Metal version.

Resources