How Can I Make ActiveResource XML Parsing More Consistent? - ruby-on-rails

I'm using ActiveResource to consume a REST webservice provided by Redmine (a bug-tracking tool). That webservice produces XML like the following:
<custom_field name="Issue Owner" id="15">Fred Fake</custom_field>
<custom_field name="Needs Printing" id="16">0</custom_field>
<custom_field name="Review Assignee" id="17">Fran Fraud</custom_field>
<custom_field name="Released On" id="20"></custom_field>
<custom_field name="Client Facing" id="21">0</custom_field>
<custom_field name="Type" id="22">Bug</custom_field>
<custom_field name="QA Assignee" id="23"></custom_field>
<custom_field name="Company Name" id="26"></custom_field>
<custom_field name="QA Notes" id="27"></custom_field>
<custom_field name="Failed QA Attempts" id="28">2</custom_field>
However, when ActiveResource parses that, and I iterate through the results printing them out, I get:
Fred Fake
0
Fran Fraud
#<Redmine::Issue::CustomFields::CustomField:0x5704e95d>
0
Bug
#<Redmine::Issue::CustomFields::CustomField:0x32fd963>
#<Redmine::Issue::CustomFields::CustomField:0x3a68f437>
#<Redmine::Issue::CustomFields::CustomField:0x407964d6>
2
That's right, it throws out all of the attribute info from anything with a value, but keeps the attribute info from the empty elements.
Needless to say, this makes things rather difficult when you're trying to find the value for id 15 (or whatever). Now I can reference things by their position, but that's very brittle, because those elements are likely to change in the future. I assume there has to be some way to make ActiveResource keep the attribute info, but since I'm not doing anything special.
(My ActiveResource extension is just five lines long: it extends ActiveResource, defines the url, username and password of the service, and that's it).
So, does anyone know how I can make ActiveResource not parse this XML so strangely?

This is a known issue with ActiveResource apparently:
https://github.com/rails/rails/issues/588
Unfortunately, nothing appears to be done about it & the issue was closed. If you're feeling up to it, the Rails 3 code for updating ActiveResource and Hash.from_xml to preserve all attributes are all in the gist below and you could create a tailored version in your Redmine module to fix it:
https://gist.github.com/971598
Update:
An alternative, as it appears ActiveResource will not be part of Rails 4 core and will be spun out as a separate gem, would be to use an alternative ORM for REST APIs, like Her.
Her allows you to use a custom parser for your XML. This is an example custom parser called Redmine::ParseXML:
https://gist.github.com/3879418
So then all you need to do is create a file like config/initializers/her.rb:
Her::API.setup :url => "https://api.xxxxx.org" do |connection|
connection.use Faraday::Request::UrlEncoded
connection.use Redmine::ParseXML
connection.use Faraday::Adapter::NetHttp
end
and you get a Hash like the following:
#<Redmine::Issue(issues) issues={:attributes=>{:type=>"array", :count=>1640},
:issue=>{:id=>4326,
:project=>{:attributes=>{:name=>"Redmine", :id=>1}},
:tracker=>{:attributes=>{:name=>"Feature", :id=>2}},
:status=>{:attributes=>{:name=>"New", :id=>1}},
:priority=>{:attributes=>{:name=>"Normal", :id=>4}},
:author=>{:attributes=>{:name=>"John Smith", :id=>10106}},
:category=>{:attributes=>{:name=>"Email notifications", :id=>9}},
:subject=>"\n Aggregate Multiple Issue Changes for Email Notifications\n ",
:description=>"\n This is not to be confused with another useful proposed feature that\n would do digest emails for notifications.\n ",
:start_date=>"2009-12-03",
:due_date=>{},
:done_ratio=>0,
:estimated_hours=>{},
:custom_fields=>{
:custom_field=>[
{:attributes=>{:name=>"Issue Owner", :id=>15}, "value"=>"Fred Fake"},
{:attributes=>{:name=>"Needs Printing", :id=>16}, "value"=>0},
{:attributes=>{:name=>"Review Assignee", :id=>17}, "value"=>"Fran Fraud"},
{:attributes=>{:name=>"Released On", :id=>20}},
{:attributes=>{:name=>"Client Facing", :id=>21}, "value"=>0},
{:attributes=>{:name=>"Type", :id=>22}, "value"=>"Bug"},
{:attributes=>{:name=>"QA Assignee", :id=>23}},
{:attributes=>{:name=>"Company Name", :id=>26}},
{:attributes=>{:name=>"QA Notes", :id=>27}},
{:attributes=>{:name=>"Failed QA Attempts", :id=>28}, "value"=>2}]},
:created_on=>"Thu Dec 03 15:02:12 +0100 2009",
:updated_on=>"Sun Jan 03 12:08:41 +0100 2010"}}>

Related

Base CRM Rails Gem legacy search?

It looks like Base CRM has upgraded their API and replaced all of their endpoints/parameters.
Previously I was able to retrieve "Won" deals using this call:
session = BaseCrm::Session.new("<LEGACY_ACCESS_TOKEN>")
session.deals.all(stage: :won, sort_by: :last_activity, sort_order: :desc, page: 1)
This query recently started ignoring my parameters, yet it continued to respond with unfiltered data (that was fun when I realized that was happening).
The new syntax is:
client = BaseCRM::Client.new(access_token: "<YOUR_PERSONAL_ACCESS_TOKEN>")
client.deals.where(organization_id: google.id, hot: true)
yet this does not work:
client.deals.where(stage_name: :won)
client.deals.where(stage_name: "Won")
client.deals.where(stage_id: 8) # specified ID found in Base Docs for "Won"
etc.
I've looked into the most recent updates to the Base CRM Gem as well as the Base CRM API Docs but have not found a solution to searching by specific deal stage.
Has anyone had any luck with the new API and this kind of query?
Is there a way to use the legacy API?
I've left message with Base but I really need to fix this, you know, yesterday.
Thanks for your help!
ADDITIONAL INFO
The legacy API/gem responded with JSON where the v2 API/gem responds with a BaseCRM::Deal object:
$ session.deals.find(123456)
# <BaseCRM::Deal
dropbox_email="dropbox#67890.deals.futuresimple.com",
name="Cool Deal Name",
owner_id=54321,
creator_id=65432,
value=2500,
estimated_close_date=nil,
last_activity_at="2016-04-21T02:29:43Z",
tags=[],
stage_id=84588,
contact_id=098765432,
custom_fields={:"Event Location"=>"New York, NY", :Source=>"Friend"},
last_stage_change_at="2016-04-21T02:08:20Z",
last_stage_change_by_id=559951,
created_at="2016-04-18T22:16:35Z",
id=123456,
updated_at="2016-04-21T02:08:20Z",
organization_id=nil,
hot=false,
currency="USD",
source_id=1466480,
loss_reason_id=nil
>
Checkout stage_id. Is this a bug? According to the Docs stage_id should return an integer between 1 and 10.

How to get more details out of Capybara at failure time?

My shopping list is...
ruby-2.0.0-p247
rails (4.0.1)
rspec-rails (2.14.0)
capybara (2.0.3)
capybara-selenium-remote (0.0.1)
The point is RSpec feature-tests that drive Capybara, that drive a Firefox web browser hitting our website.
Now ogle this failing ... expectation:
old_date = 'November 16, 2013 04:00 pm'
expect(page).to have_field('somethingDate', with: old_date)
When it fails, it emits this:
Failure/Error: expect(page).to have_field('somethingDate', with: old_date)
expected #has_field?("somethingDate", {:with=>"November 16, 2013 04:00 pm"}) \
to return true, got false
The general question this: How to write an assertion that reveals _everything_ it knows about a situation. Instead of almost NOTHING: "got false".
I would expect, based on my (cough) experience with assert_select and assert_xpath, to at least see a snip of the HTML around the fault point. If not, then maybe at LEAST the value that has_field? DID find in that field. (And keep in mind this is only RAILS FOUR, already...)
So the two sub-questions are:
Would a different assertion tell me more? (If so, why is has_field? supposed to be the industry flagship assertion here?)
Is there some way to configure or tune or plug-in has_field?, to emit more details at fault time?

Ruby/Rails - Parse JSON and split object to convert to datetime

I have the following code:
page = Net::HTTP.get_response(URI.parse('http://www.enhancetv.com.au/tvguide/rss/melbournerss.php')).body rescue nil
show_info = Hash.from_xml(page).to_json
puts show_info
Which outputs the following JSON:
{"rss":{"version":"2.0","channel":{"title":"EnhanceTV Melbourne TV Guide ","description":null,"link":"http://www.enhancetv.com.au","lastBuildDate":"Wed, 21 Dec
2011 10:25:55 1000","generator":"FeedCreator 1.7.2","item":[{"title":"Toasted TV - TEN - 07:00:00 - 21/12/2011","link":"http://www.enhancetv.com.au/tvguide/","d
escription":"Join the team for the latest in gaming, sport, gadgets, pop culture, movies, music and other seriously fun stuff! Featuring a variety of your favou
rite cartoons."},{"title":"Totally Wild - TEN - 08:00:00 - 21/12/2011","link":"http://www.enhancetv.com.au/tvguide/","description":"The Totally Wild team brings
you the latest in action, adventure and wildlife from Australia and around theglobe."},{"title":"Explore: Africa&apos;s Rift Valley - SBS ONE - 19:30:00 - 21/
12/2011","link":"http://www.enhancetv.com.au/tvguide/","description":"Simon Reeve leads a team of journalists on a spectacular journey down East Africa&apos;s R
ift Valley. From the tiny country of Djibouti, which is the centre of America&apos;s military presence in Africa, to the wide open plains of Kenya, the team enc
ounter awe-inspiring landscapes, rich culture and amazing wildlife."}]}}}
First of all I'd like to actually be able to loop through each of the items.
This is what I have so far, however it's throwing an error. I'm not entirely sure how to even loop through the parsed JSON properly:
result = JSON.parse(show_info) # convert JSON data to native ruby
result["channel"].each do |a|
puts a['title']
puts a['description']
puts a['link']
end
This gives the following error:
scraper.rb:118:in `<main>': undefined method `each' for "rss":String (NoMethodEr
ror)
I'd then want to be able to do a split on the "title" so I can convert the time and date into a DateTime object to use later.
Thanks in advance.
I've decided to use Nokogiri to parse the XML instead of converting the XML to JSON.
Thanks for your help.

to_xml giving strange results

When I do that
{"New York"=>33, :Versailles => 3231}.to_xml
I get
<?xml version=\"1.0\" encoding=\"UTF-8\"?>
<hash>
<Versailles type=\"integer\">3231</Versailles>
<New York type=\"integer\">33</New York>
</hash>
I would have expected rails to dasherize "New York", no?
This issue is closed via the merging of pull request 445 : https://github.com/rails/rails/pull/445
Spaces will now be dasherized (and the private _dasherize method is enhanced to handle spaces.)
{"New York"=>33}.to_xml will result in
..<New-York type=\"integer\">33</New-York>..
Thanks for putting up this lighthouse ticket & the stackoverflow question (which added more info in the discussions); with the help of all the information provided, I was able to make my first rails commit!
I'm seeing the same thing too.
According to the docs the :dasherize option to to_xml should do the trick.
Some configuration is available through options. [...]
This behavior can be controlled with :only, :except, :skip_instruct, :skip_types, :dasherize and :camelize [...]
The default is to dasherize all column names, but you can disable this setting :dasherize to false. Setting :camelize to true will camelize all column names - this also overrides :dasherize.
So, it looks like, at a minimum:
asdf.to_xml(:dasherize => true)
should do it, and adding the :camelize option should force it.
=> "<?xml version="1.0" encoding="UTF-8"?>\n<Hash>\n <New York type="integer">33</New York>\n <Versailles type="integer">3231</Versailles>\n</Hash>\n"
So, maybe someone can 'splain the anomoly.
Normally database fields don't have spaces in them, so your example in the context of #to_xml is a garbage-in-garbage-out situation.

Feed Parsing In Rails

I am working on Ubuntu 10.04 and I am using feed-zirra to parse RSS feeds and I have MySQL database.
I am trying to parse RSS feeds from Times of India Top Stories. There seems to be problem with the first link, I am sure TOI guys will correct it soon. But anyway, I dont want to face similar error later so thats why I want to ask you guys how to solve this problem.
Just look at this and especially look for link
<item>
<title>CWG: Abhinav Bindra, Gagan Narang win first Gold for India</title
<description>Abhinav Bindra and Gagan Narang on Tuesday bagged Gold for the men's 10 m air rifle pair's event, getting India its first gold in the 19th Commonwealth Games.</description>
<link>/cwgarticleshow/6688747.cms</link>
<guid>/cwgarticleshow/6688747.cms</guid>
<pubDate>Tue, 05 Oct 2010 04:57:46 GMT</pubDate>
</item>
The link is <link>/cwgarticleshow/6688747.cms</link>
Now, when I click the link, in the view.. its getting routed to http://localhost:3000/cwgarticleshow/6688747.cms instead of http://timesofindia.indiatimes.com/cwgarticleshow/6688747.cms
And the error I am getting is
**Routing Error**
No route matches "/cwgarticleshow/6688747.cms" with {:method=>:get}
How do I correct this type of Error?
Looking forward for your help and support
Thanks
You just need to prepend http://timesofindia.indiatimes.com to the link tag value and you'll be ok.
You can use URI class. You can, for example, define following method
require "uri"
def repair_link(feed_link)
uri = URI.parse(feed_link)
uri.scheme ||= "http"
uri.host ||= "timesofindia.indiatimes.com"
uri.to_s
end
It will set the scheme and host part of the URL if they are not already filled. So if you call it for normal link (like http://foo/bar.cms) then nothing will be changed.
And last thing - you probably should catch exception somewhere as the #parse method raises exception InvalidURIError in case of invalid URI. But it's up to you how you will deal with it.

Resources