I've got ActiveResource setup to consume the freebase api in json, and it should work fine except that the json freebase returns causes ActiveResource to blowup.
NoMethodError: undefined method `collect!' for #<Hash:0x007fd674831dd0>
How can I define a custom json parser to fix whatever is wrong?
class Freebase < ActiveResource::Base
self.site = "https://www.googleapis.com/"
self.format = :json
def self.search(word)
self.find(:all, :from => "/freebase/v1/search/", :params => { :query => word })
end
#https://www.googleapis.com/freebase/v1/search?query=nirvana
#Freebase.get('search', :query => 'nirvana')
end
Json being returned:
https://www.googleapis.com/freebase/v1/search?query=nirvana
{"status":"200 OK","result":[{"mid":"/m/05b3c","name":"Nirvana","notable":{"name":"Belief","id":"/religion/belief"},"lang":"en","score":67.540009},{"mid":"/m/0b1zz","name":"Nirvana","notable":{"name":"Musical Artist","id":"/music/artist"},"lang":"en","score":64.311432},{"mid":"/m/092bf5","name":"Buddhism","notable":{"name":"Religion","id":"/religion/religion"},"lang":"en","score":33.647118},{"mid":"/m/02_6qh","name":"Nirvana","notable":{"name":"Film","id":"/film/film"},"lang":"en","score":30.068491},{"mid":"/m/01h89tx","name":"Nirvana","notable":{"name":"Musical Album","id":"/music/album"},"lang":"en","score":27.799274},{"mid":"/m/01rn9fm","name":"Nirvana","notable":{"name":"Musical Group","id":"/music/musical_group"},"lang":"en","score":27.445602},{"mid":"/m/015k7","name":"Gautama Buddha","notable":{"name":"Deity","id":"/religion/deity"},"lang":"en","score":24.129679},{"mid":"/m/01rkx5","name":"Mahayana Mahaparinirvana Sutra","lang":"en","score":22.359026},{"mid":"/m/03d7q7v","name":"Nirvana","lang":"en","score":21.034473},{"mid":"/m/055ym7w","name":"Nirvana bootleg recordings","notable":{"name":"Musical Album","id":"/music/album"},"lang":"en","score":19.241596},{"mid":"/m/0122_j","name":"Nevermind","notable":{"name":"Musical Album","id":"/music/album"},"lang":"en","score":18.366383},{"mid":"/m/04n7mt","name":"Nirvana fallacy","lang":"en","score":17.212397},{"mid":"/m/0484q","name":"Kurt Cobain","notable":{"name":"Musician","id":"/m/09jwl"},"lang":"en","score":16.594929},{"mid":"/m/027_k8j","name":"Nirvana","lang":"en","score":16.336584},{"mid":"/m/0285c","name":"Dave Grohl","notable":{"name":"Musician","id":"/m/09jwl"},"lang":"en","score":16.115103},{"mid":"/m/068shv","name":"Smells Like Nirvana","notable":{"name":"Musical Album","id":"/music/album"},"lang":"en","score":15.350652},{"mid":"/m/01kq85c","name":"Manic Nirvana","notable":{"name":"Musical Album","id":"/music/album"},"lang":"en","score":15.275189},{"mid":"/m/0437sc","name":"Lithium","notable":{"name":"Composition","id":"/music/song"},"lang":"en","score":14.637386},{"mid":"/m/055kh1","name":"Mechanus","lang":"en","score":14.621847},{"mid":"/m/01f1vf","name":"Lucifer","notable":{"name":"Fictional Character","id":"/fictional_universe/fictional_character"},"lang":"en","score":13.504528}],"cursor":20,"cost":11,"hits":3104}
Using Rails 3.2.1
UPDATE:
Found this issue, but still not sure how to overcome it. https://github.com/rails/rails/issues/2318
The problem is that the JSON returned isn't formed the way ActiveResource expects it to be. ARes doesn't expect all of that metadata that is being returned, it is only expecting what is in the results portion of the response.
To be explicit, you're getting back:
{"status":"200 OK","result":[{"mid":"/m/05b3c","name":"Nirvana","notable":{"name" ...
But ARes wants:
[{"mid":"/m/05b3c","name":"Nirvana","notable":{"name"...
The easiest (and probably dirtiest) solution I can come up with is to overwrite the ActiveResource::Base#find_every private method in your model like so:
class Freebase < ActiveResource::Base
self.site = "https://www.googleapis.com/"
self.format = :json
def self.search(word)
self.find(:all, :from => "/freebase/v1/search/", :params => { :query => word })
end
private
def self.find_every(options)
begin
case from = options[:from]
when Symbol
instantiate_collection(get(from, options[:params]))
when String
path = "#{from}#{query_string(options[:params])}"
instantiate_collection(format.decode(connection.get(path, headers).body['result']) || [])
else
prefix_options, query_options = split_options(options[:params])
path = collection_path(prefix_options, query_options)
instantiate_collection( (format.decode(connection.get(path, headers).body['result']) || []), prefix_options )
end
rescue ActiveResource::ResourceNotFound
# Swallowing ResourceNotFound exceptions and return nil - as per
# ActiveRecord.
nil
end
end
end
The only change I made is the .body method calls are now .body['result']. With this addition, #instantiate_collection will now receive an Array as it expects rather than a Hash.
Ultimately, while this should make the error go away, I don't know that this will solve all of your problems as you might need some of the data that gets lost with this method. My suggestion would be to abandon ActiveResource if you can and use something like RestClient and build your model around that. Another reason for this approach is that the semantics as they are now are broken. You aren't getting back a collection of Freebases. You're using the Freebase API to get search results.
I followed Derek's advice and abandoned ActiveResource...
for anyone else I ended up using Faraday and Faraday_Middleware
class Freebase
def self.search(id)
connection = Faraday.new 'https://www.googleapis.com/freebase/v1' do |conn|
conn.adapter Faraday.default_adapter
conn.use FaradayMiddleware::ParseJson
#conn.use Faraday::Response::Mashify
end
response = connection.get do |req|
req.url('search', :query => id, :limit => 10)#, :filter => '(any namespace:/wikipedia/en_id namespace:/authority/imdb/title)')# #:filter => '(any namespace:/wikipedia/en_id namespace:/authority/imdb/title namespace:/authority/netflix/movie)' , :with => 'commons'
end
end
Supposedly you can use several built in modules to make rails style methods out of the returned hash, but I haven't figured that part out yet. ActiveResource seems only good at other Rails apps or RESTFUL routes that perfectly imitate it.
Related
In order to improve my understanding of Rails, I'm converting a Sinatra app that uses data_mapper.
I'm trying to find the best replacements for data mappers 'first' method that searches database and returns first instance of the record sought.
Can anyone comment if this is done right, or if there's a better solution?
Situation #1
Sinatra
url = Url.first(:original => original)
Rails (both of these ok?)
url = Url.find_by_original(original) #this find_by_original
url = Url.where(:first_name => 'original')
situation #2
Sinatra
raise 'Someone has already taken this custom URL, sorry' unless Link.first(:identifier => custom).nil?
My Rails (with find)
raise 'Someone has already taken this custom URL, sorry' unless Link.find(:identifier => custom).nil? #this Link.find
Original context was a method that shortens urls
def self.shorten(original, custom=nil)
url = Url.first(:original => original)
return url.link if url
link = nil
if custom
raise 'Someone has already taken this custom URL, sorry' unless Link.first(:identifier => custom).nil?
raise 'This custom URL is not allowed because of profanity' if DIRTY_WORDS.include? custom
transaction do |txn|
link = Link.new(:identifier => custom)
link.url = Url.create(:original => original)
link.save
end
else
transaction do |txn|
link = create_link(original)
end
end
return link
end
You can just use a validator on the model. Just do a validates_uniqueness_of :first_name in the Url model (app/models/url.rb). There are also other validations available in the Rails guides
EDIT
If you really want to find if such a record exists manually. You can just do Url.find(:first, :conditions => { :first_name => 'original' }) and check for nil
raise 'Someone has already taken this custom URL, sorry' unless Link.find(:first, :conditions => { :identifier => custom }).nil?
Also, if you are new to rails might want to look at the query interface. Personally, I like to use the squeel gem so my queries are strictly ruby instead of mixing ruby and sql statements.
I'm trying to get activeresource (on Rails 3.2) working with the Freebase API and I haven't had much luck yet. How can I debug rails to see what's going on and make sure the request is well formed?
I think the suffixed .json is causing the failure but I don't know how to check what's going on?
Error:
ActiveResource::ResourceNotFound: Failed. Response code = 404. Response message = Not Found.
Code:
class Freebase < ActiveResource::Base
class << self # also tried without this, same result
def element_path(id, prefix_options = {}, query_options = nil)
prefix_options, query_options = split_options(prefix_options) if query_options.nil?
"#{prefix(prefix_options)}#{collection_name}/#{id}#{query_string(query_options)}"
end
def collection_path(prefix_options = {}, query_options = nil)
prefix_options, query_options = split_options(prefix_options) if query_options.nil?
"#{prefix(prefix_options)}#{collection_name}#{query_string(query_options)}"
end
end
self.site = "https://www.googleapis.com/freebase/v1/"
self.format = :json
#https://www.googleapis.com/freebase/v1/search?query=nirvana&indent=true
#Freebase.get('search', :query => 'nirvana')
end
UPDATE:
Ok so I found two things going wrong...
1) The collection_path I'm trying to supersede doesn't work at all.. it still tacks .json onto every request.
2) https://www.googleapis.com:443/freebase/v1/freebases/search.json?query=nirvana
It tacks freebase on afterwards... any ideas?
I also tried this fix:
Remove .xml extension from ActiveResource request
But it didn't remove the JSON suffix either.
UPDATE UPDATE:
Added the suggest update below which gives the correct PATH but now I'm getting
GET https://www.googleapis.com:443/freebase/v1/search/?query=monkey
--> 200 OK 2732 (693.3ms)
NoMethodError: undefined method `collect!' for #<Hash:0x007f9bde674900>
Add ActiveResource::Base.logger = Logger.new(STDERR) to your config/application.rb ( Rails 3.x ).
You'll get output like :
POST http://localhost:3000/freebase.json
--> 201 Created 0 (15.8ms)
That shows method and response code ...
ActiveResource isn't dead, but relative to ActiveRecord I can understand why you'd think so, it is definitely an unloved and underprivileged stepchild.
You'd probably be better off using something like Faraday_Middleware or HTTPParty. The former is my personal preference. Unless what you're doing is pulling from another Rails app or one that has perfect restful behaviour like Rails (which Freebase doesn't), ActiveResource is usually more trouble than it's worth.
That being said, you can accomplish what you want without overwriting any class methods by doing:
self.site = "https://www.googleapis.com/"
self.format = :json
def self.search(word)
self.find(:all, :from => "/freebase/v1/search/", :params => { :query => word })
end
To get detail login for ActiveResource have to patch the request method inside the gem(method.
place bellow files inside config/initializers you will get http method, path, request body, request hedaers
response body and header is already there if you need. doc
config/initializers/activeresource_patch.rb
module ActiveResource
class Connection
private
def request(method, path, *arguments)
result = ActiveSupport::Notifications.instrument("request.active_resource") do |payload|
payload[:method] = method
payload[:request_uri] = "#{site.scheme}://#{site.host}:#{site.port}#{path}"
payload[:request_path] = path
payload[:request_body] = arguments[0]
payload[:request_headers] = arguments[1]
payload[:result] = http.send(method, path, *arguments)
end
handle_response(result)
rescue Timeout::Error => e
raise TimeoutError.new(e.message)
rescue OpenSSL::SSL::SSLError => e
raise SSLError.new(e.message)
end
end
end
config/initializers/activeresource_logger.rb
Rails.application.configure do
def activeresource_logger
#activeresource_logger ||= Logger.new("#{Rails.root}/log/activeresource_logger.log")
end
ActiveSupport::Notifications.subscribe('request.active_resource') do |name, start, finish, id, payload|
if Rails.env.development?
activeresource_logger.info("====================== #{start} : #{payload[:method].upcase} ======================")
activeresource_logger.info("PATH: #{payload[:request_path]}")
activeresource_logger.info("BODY: #{payload[:request_body]}")
activeresource_logger.info("HEADERS: #{payload[:request_headers]}")
# activeresource_logger.info("STATUS_CODE: #{payload[:result].code}")
# activeresource_logger.info("RESPONSE_BODY: #{payload[:result].body}")
end
end
end
ActiveResource is going to expect that endpoint to return the data in a very specific format.
For the longest time we've been using ActiveResource at my company for inter-application communication. However more recently we've started leaning towards HTTParty because it performs a lot less voodoo magic, and tends to be a much small exercise in hair-pulling.
Here's an example of how we're using HTTParty now:
module CoreResources
class Job
include HTTParty
base_uri Rails.configuration.core_resource_uri
basic_auth Rails.configuration.core_resource_user, Rails.configuration.core_resource_password
def self.search(entity)
get("/api/v1/jobs.json", :query => {:entity_id => entity.id})
end
def self.find(id)
result = get("/api/v1/jobs/#{id}.json")
raise CoreResources::JobNotFound.new if result.response.code == "404"
raise "Unexpected response from resource job find: #{result.response.code} #{result.response.to_s}" if result.response.code =~ /^(?:4|5)..$/
result
end
end
end
The problem with ActiveResource is that it will take the very specifically-crafted json or xml markup, and instantiate ActiveResource objects and nested objects based on it. It's having issues calling collect because something in the json response was supposed to be formatted like an enumerable, and wasn't (likely the parent node should have been an array or something, not sure), which makes it blow up.
With HTTParty you get a json-parsed collection to work with.
It's easy enough for me to do this:
jobs = CoreResources::Job.search(my_entity)
puts jobs.inspect
# [{
# "id" => 4,
# "created_by_id" => 12,
# "description" => "I like pie"
# },
# {
# "id" => 5",
# "created_by_id" => 12,
# "description" => "Mmm, cake"
# }]
Which let's me access jobs via an easy collection array/hash construct jobs[0].fetch("description"), as opposed to ActiveResource: jobs[0].description. ActiveResource is slower to insantiate those collections, needlessly takes up memory with them, and encourages you to duplicate code that should just be served by the endpoint in your ActiveResource model (Then again, if you're using a third-party API you may have no other choice, but I have never successfully gotten ARes to interface with third-party API's).
We've run into a lot of other ActiveResource problems where it does this nonsensical dynamic creation of class names based on nested resources from your endpoint, but half the time does it incorrectly... It's really just a mess.
Moral of the story: Much more of a fan of HTTParty now. That endpoint is probably just not returning data in the right format, and unless it does ActiveResource will be a hackfest to get it to read it right.
ActiveResource has a fairly narrow use-case. You probably want to use a more generic interface for working with Freebase.
Some code from the LinkTV Platform's FreeBase API might be of some help.
The NoMethodError: undefined method 'collect!' for #<Hash:0x007f9bde674900>
error seems to be an issue on rails https://github.com/rails/rails/issues/2318 .
I had a similar problem but the hacks provided didn't work so I had to tweak them a bit , you can find my answer here .
I am using the great Tequila-JSON Parser ( http://github.com/inem/tequila ) in an Web-application, to render more or less complex JSON server-replies. More and more the JSON-Templates (.jazz) are growing in somehow real "views". I am trying now, to get an instance-variable from the according controller, into the .jazz template, but this somehow fails.
Here is what I am trying to do.
The controller
def get_userlist
#users = User.find(:all, :order => "value DESC", :limit => 10)
#user = User.find_by_email(params[:user_email])
#userid = #user.id # also tried: #userid = 2
respond_to do |format|
format.json
end
end
The .jazz view:
-#users
:only
.nickname
.level
.user_icon_url
.email
:methods
.isfriend(#userid)
+last_checkin
+last_checkin_place
:only
.name
.city
This all returns a pretty valid JSON server-reply, but unfortunately, there is a problem with the
:methods
.isfriend(#userid)
The Method "isfriend" resides in the model "User", is called successfully and returns in the JSON, what it should. But the value of the instance-variable somehow is wrong. Opposed to the above, this one works fine:
:methods
.isfriend(1)
Now the question: Is Tequila not able, to interpret instance-variables in its own .jazz templates? Does anyone have experience, solutions or workarounds?
For the sake of completeness, here is the isfriend method of the User-Model:
def isfriend(user_id)
"Hi, I am User with the id: " + user_id.to_s
end
Nope. Also it doesn't work on Rails 3. I just spent 6 hours trying to port it and got basically nowhere :-(
Im looking for a way to map an ActiveResource to SimpleDB
I want to avoid plugins/gems as all I have used are outdated/buggy/not mantained
It doesnt seem hard, I wonder if any of you have succesfully implemented a rails app with simpleDB as an Active Resource. How did you do it? Thanks.
I haven't worked with SimpleDB, but I have mapped ActiveResource to Amazon's Flexible Payments Service REST api and just skimming the docs they seem similar so here's basically what I did, maybe you can use this as a starting point.
require 'base64'
require 'openssl'
class AmazonFlexiblePaymentResource < ActiveResource::Base
self.site = AMZ_CONFIG['flexible_api_url']
def self.rest_api(options = {})
params = common_request_params.update(options)
sig = compute_signature(AMZ_CONFIG['secret_access_key'], 'get', site, params)
rest_req = {'Signature' => sig}.update(params)
# make the http get call
connection.get("/#{query_string(rest_req)}", headers)
end
protected
# these are the params are common to all rest api calls
def self.common_request_params
{ 'AWSAccessKeyId' => AMZ_CONFIG['access_key_id'],
'SignatureVersion' => 2,
'SignatureMethod' => 'HmacSHA256',
'Timestamp' => Time.now.utc.iso8601,
'Version' => '2008-09-17'}
end
def self.compute_signature(key, method, end_point_url, params)
query_str = parameters.sort.collect {|k, v| v.to_query(k)}.join '&'
# cannot use plus for space, and tilde needs to be reversed
query_str.gsub!('+', '%20')
query_str.gsub!('%7E', '~')
to_sign = [method.upcase, end_point_uri.host.downcase,
end_point_uri.request_uri, query_str].join "\n"
digest = OpenSSL::Digest::Digest.new('sha256')
hmac = OpenSSL::HMAC.digest(digest, key, to_sign)
Base64.encode64(hmac).chomp
end
end
Then I just make calls like this
res = AmazonFlexiblePaymentResource.rest_api({ 'Action' => 'GetTransactionStatus', 'TransactionId' => '1234567890ABCDEFGHIJ' })
And the response is a hash of the parsed xml. Again this works for Amazon Flexible Payments Service, so you may need to make adjustments to match the SimpleDB REST API.
Does it need to be ActiveResource? If you want it like ActiveRecord, check out SimpleRecord.
http://github.com/appoxy/simple_record
It's very actively maintained.
I have a create action that handles XML requests. Rather than using the built in params hash, I use Nokogiri to validate the XML against an XML schema. If this validation passes, the raw XML is stored for later processing.
As far as I understand, the XML is parsed twice: First the Rails creates the params hash, then the Nokogiri parsing happens. I've been looking for ways to disable the params parsing to speed things up but have found nothing.
ActionController::Base.param_parsers[Mime::XML] = lambda do |body|
# something
end
I know it's possible to customize the XML params parsing in general using something like the above, but I depend on the default behaviour in other controllers.
Is it possible to bypass the params parsing on a per-action basis? What options do I have?
Thank you for your help!
I've managed to solve the problem using Rails Metal. The relevant part looks something like this:
class ReportMetal
def self.call(env)
if env["PATH_INFO"] =~ /^\/reports/
request = Rack::Request.new(env)
if request.post?
report = Report.new(:raw_xml => request.body.string)
if report.save # this triggers the nokogiri validation on raw_xml
return [201, { 'Content-Type' => 'application/xml' }, report.to_xml]
else
return [422, { 'Content-Type' => 'application/xml' }, report.errors.to_xml]
end
end
end
[404, { "Content-Type" => "text/html" }, "Not Found."]
ensure
ActiveRecord::Base.clear_active_connections!
end
end
Thanks!
PS: Naive benchmarking with Apache Bench in development shows 22.62 Requests per second for standard Rails vs. 57.60 Requests per second for the Metal version.