Rails nokogiri parse XML file - ruby-on-rails

I'm a little bit confused: could not find in web good examples of parsing xml with nokogiri...
example of my data:
<?xml version="1.0" encoding="UTF-8"?>
<root>
<rows SessionGUID="6448680D1">
<row>
<AnalogueCode>0451103079</AnalogueCode>
<AnalogueCodeAsIs>0451103079</AnalogueCodeAsIs>
<AnalogueManufacturerName>BOSCH</AnalogueManufacturerName>
<AnalogueWeight>0.000</AnalogueWeight>
<CodeAsIs>OC90</CodeAsIs>
<DeliveryVariantPriceAKiloForClientDescription />
<DeliveryVariantPriceAKiloForClientPrice>0.00</DeliveryVariantPriceAKiloForClientPrice>
<DeliveryVariantPriceNote />
<PriceListItemDescription />
<PriceListItemNote />
<IsAvailability>1</IsAvailability>
<IsCross>1</IsCross>
<LotBase>1</LotBase>
<LotType>1</LotType>
<ManufacturerName>KNECHT/MAHLE</ManufacturerName>
<OfferName>MSC-STC-58</OfferName>
<PeriodMin>2</PeriodMin>
<PeriodMax>4</PeriodMax>
<PriceListDiscountCode>31087</PriceListDiscountCode>
<ProductName>Фильтр масляный</ProductName>
<Quantity>41</Quantity>
<SupplierID>30</SupplierID>
<GroupTitle>Замена</GroupTitle>
<Price>203.35</Price>
</row>
<row>
<AnalogueCode>0451103079</AnalogueCode>
<AnalogueCodeAsIs>0451103079</AnalogueCodeAsIs>
<AnalogueManufacturerName>BOSCH</AnalogueManufacturerName>
<AnalogueWeight>0.000</AnalogueWeight>
<CodeAsIs>OC90</CodeAsIs>
<DeliveryVariantPriceAKiloForClientDescription />
<DeliveryVariantPriceAKiloForClientPrice>0.00</DeliveryVariantPriceAKiloForClientPrice>
<DeliveryVariantPriceNote />
<PriceListItemDescription />
<PriceListItemNote>[0451103079] Bosch,MTGC#0451103079</PriceListItemNote>
<IsAvailability>1</IsAvailability>
<IsCross>1</IsCross>
<LotBase>1</LotBase>
<LotType>0</LotType>
<ManufacturerName>KNECHT/MAHLE</ManufacturerName>
<OfferName>MSC-STC-1303</OfferName>
<PeriodMin>3</PeriodMin>
<PeriodMax>5</PeriodMax>
<PriceListDiscountCode>102134</PriceListDiscountCode>
<ProductName>Фильтр масляный</ProductName>
<Quantity>5</Quantity>
<SupplierID>666</SupplierID>
<GroupTitle>Замена</GroupTitle>
<Price>172.99</Price>
</row>
</rows>
</root>
and ruby code:
...
xml_doc = Nokogiri::XML(response.body)
parts = xml_doc.xpath('/root/rows/row')
with the help of xpath i could do this? also how to get this parts object (row)?

You're on the right track. parts = xml_doc.xpath('/root/rows/row') gives you back a NodeSet i.e. a list of the <row> elements.
You can loop through these using each or use row indexes like parts[0], parts[1] to access specific rows. You can then get the values of child nodes using xpath on the individual rows.
e.g. you could build a list of the AnalogueCode for each part with:
codes = []
parts.each do |row|
codes << row.xpath('AnalogueCode').text
end
Looking at the full example of the XML you're processing there are 2 issues preventing your XPath from matching:
the <root> tag isn't actually the root element of the XML so /root/.. doesn't match
The XML is using namespaces so you need to include these in your XPaths
so there are a couple of possible solutions:
use CSS selectors rather than XPaths (i.e. use search) as suggested by the Tin Man
after xml_doc = Nokogiri::XML(response.body) do xml_doc.remove_namespaces! and then use parts = xml_doc.xpath('//root/rows/row') where the double slash is XPath syntax to locate the root node anywhere in the document
specify the namespaces:
e.g.
xml_doc = Nokogiri::XML(response.body)
ns = xml_doc.collect_namespaces
parts = xml_doc.xpath('//xmlns:rows/xmlns:row', ns)
codes = []
parts.each do |row|
codes << xpath('xmlns:AnalogueCode', ns).text
end
I would go with 1. or 2. :-)

First, Nokogiri supports XPath AND CSS. I recommend using CSS because it's more easily read:
doc.search('row')
will return a NodeSet of every <row> in the document.
The equivalent XPath is:
doc.search('//row')
...how to get this parts object (row)?
I'm not sure what that means, but if you want to access individual elements inside a <row>, it's easily done several ways.
If you only want one node inside each of the row nodes:
doc.search('row Price').map(&:to_xml)
# => ["<Price>203.35</Price>", "<Price>172.99</Price>"]
doc.search('//row/Price').map(&:to_xml)
# => ["<Price>203.35</Price>", "<Price>172.99</Price>"]
If you only want the first such occurrence, use at, which is the equivalent of search(...).first:
doc.at('row Price').to_xml
# => "<Price>203.35</Price>"
Typically we want to iterate over a number of blocks and return an array of hashes of the data found:
row_hash = doc.search('row').map{ |row|
{
AnalogueCode: row.at('AnalogueCode').text,
Price: row.at('Price').text,
}
}
row_hash
# => [{:AnalogueCode=>"0451103079", :Price=>"203.35"},
# {:AnalogueCode=>"0451103079", :Price=>"172.99"}]
These are ALL covered in Nokogiri's tutorials and are answered many times here on Stack Overflow, so take the time to read and search.

Related

Removing elements with XPath

So let's say I have an XML file and I want to remove some nodes from it using their XPath. How would I do that and is it possible in the first place with xmerl or erlsom or maybe something else?
And if there is not a simple way with XPath, what is the correct way to remove elements from XML in general?
As stated by W3C,
XPath is a language for addressing parts of an XML document
the above literally means XPath is to query XML, not to modify it. The common approach to modifying XML document, would be to one of those:
using XSLT transformation schema;
reading the content into memory, modifying it and saving it back to the file.
AFAIU, the former is out of the scope of this question. For the latter, one might use Exsom library, which is “an Elixir wrapper around the erlsom XML parsing library.”
Assuming we have the xml and xsd taken from Excom examples:
<?xml version="1.0" encoding="UTF-8"?>
<foo attr="yo">
<bar>1</bar>
<bar>2</bar>
</foo>
One might do something like this to delete second bar node (most of the code is taken as is from Excom tests:
{ :ok, model } = Exsom.XSD.File.parse("complex.xsd")
{ :ok, instance, _ } = Exsom.File.parse("complex.xml", model)
#⇒ {:ok, {:foo_type, [], 'yo', ['1', '2']}}
Modify it according to what you want, e.g. remove bar element with 2
instance = {:foo_type, [], 'yo', ['1']}
{ :ok, binary_xml } = Exsom.compose(instance, model, [{ :output, :binary }])
#⇒ {:ok, "<foo attr=\"yo\"><bar>1</bar></foo>"}
Now you might write the binary_xml to a file.

Xml parsing in rails

I have this XML data:
<?xml version="1.0" encoding="UTF-8"?>
<responseParam>
<RESULT>-1</RESULT>
<ERROR_CODE>509</ERROR_CODE>
</responseParam>
How can I fetch the value of error code only?
I have tried this :
result = Net::HTTP.get(URI.parse(otpUrl))
data = Hash.from_xml(result)
puts "#{data['ERROR_CODE']}"
puts data[:ERROR_CODE]
printing only "data" gives me the whole hash. I am not able to get only the value of ERROR_CODE.
Any help ?
you can use Nokigiri here.
suppose this is your error.xml
<?xml version="1.0" encoding="UTF-8"?>
<responseParam>
<RESULT>-1</RESULT>
<ERROR_CODE>509</ERROR_CODE>
</responseParam>
you can do something like:-
#doc = Nokogiri::XML(File.open("error.xml"))
#doc.xpath("//ERROR_CODE")
will give you something like:-
# => ["<ERROR_CODE>509</ERROR_CODE>]"
The Node methods xpath and css actually return a NodeSet, which acts very much like an array, and contains matching nodes from the document.

Read XML file with Nokogiri

I currently have an XML file that is reading correctly except for one part. It is an item list and sometimes one item has multiple barcodes. In my code it only pulls out the first. How can I iterate over multiple barcodes. Please see code below:
def self.pos_import(xml)
Plu.transaction do
Plu.delete_all
xml.xpath('//Item').each do |xml|
plu_import = Plu.new
plu_import.update_pointer = xml.at('Update_Type').content
plu_import.plu = xml.at('item_no').content
plu_import.dept = xml.at('department').content
plu_import.item_description = xml.at('item_description').content
plu_import.price = xml.at('item_price').content
plu_import.barcodes = xml.at('UPC_Code').content
plu_import.sync_date = Time.now
plu_import.save!
end
end
My test XML file looks like this:
<?xml version="1.0" encoding="UTF-16" standalone="no"?>
<items>
<Item>
<Update_Type>2</Update_Type>
<item_no>0000005110</item_no>
<department>2</department>
<item_description>DISC-ALCOHOL PAD STERIL 200CT</item_description>
<item_price>7.99</item_price>
<taxable>No</taxable>
<Barcode>
<UPC_Code>0000005110</UPC_Code>
<UPC_Code>1234567890</UPC_Code>
</Barcode>
</Item>
</Items>
Any ideas how to pull both UPC_Code fields out and write them to my database?
.at will always return a single element. To get an array of elements use xpath like you do to get the list of Item elements.
plu_import.barcodes = xml.xpath('//UPC_Code').map(&:content)
Thanks for all the great tips. It definitely led me in the right direction. The way that I got it to work was just adding a period before the double //.
plu_import.barcodes = xml.xpath('.//UPC_Code').map(&:content)

How to find text in XML and display its sibling with Nokogiri?

I have the following XML consumed from a REST API:
<dataitems>
<dataitem colour="null">
<value>
<label>Intel</label>
<count>43</count>
</value>
<value>
<label>AMD</label>
<count>39</count>
</value>
<value>
<label>ARM</label>
<count>28</count>
</value>
</dataitem>
</dataitems>
I would like to search for the text in the <label> tag and display the matching value for <count> in a table.
In the controller I have: #post_count = Nokogiri::XML(Post.item_data.to_xml).
In the view I'm not sure whether I need to use #post_count.xpath or #post_count.search.
Can someone please point me in the right direction on the right method and syntax for it?
Thanks in advance.
Although I'm not sure what information you are looking for, I have a few suggestions.
1) If you know the value in the element before you do your searching:
doc = Nokogiri.XML(open(source_xml))
# Assuming there is only one of each label
node = doc.xpath('//label[text()="Intel"]').first
count = node.next_element.text
# or if there are many of each label
nodes = doc.xpath('//label[text()="Intel"]')
nodes.each {|node|
count = node.next_element.text
# do something with count here
}
2) Assuming that you don't know the names within the tag in advance
doc = Nokogiri.XML(open(source_xml))
labels = {}
doc.xpath('//label').each {|node|
labels[node.text] = node.next_element.text
}
# labels => {"Intel"=>"43", "AMD"=>"39", "ARM"=>"28"}
I personally like the second solution better because it gives you a clean hash, but I prefer to work with hashes and arrays as quickly as possible.

Hpricot Element intersection

I want to remove all images from a HTML page (actually tinymce user input) which do not meet certain criteria (class = "int" or class = "ext") and I'm struggeling with the correct approach. That's what I'm doing so far:
hbody = Hpricot(input)
#internal_images = hbody.search("//img[#class='int']")
#external_images = hbody.search("//img[#class='ext']")
But I don't know how to find images where the class has the wrong value (not "int" or "ext").
I also have to loop over the elements to check other attributes which are not standard html (I use them for setting internal values like the DB id, which I set in the attribute dbsrc). Can I access these attributes too and is there a way to remove certain elements (which are in the hpricot search result) when they don't meet my criteria?
Thanks for your help!
>> doc = Hpricot.parse('<html><img src="foo" class="int" /><img src="bar" bar="42" /><img src="foobar" class="int"></html>')
=> #<Hpricot::Doc {elem <html> {emptyelem <img class="int" src="foo">} {emptyelem <img src="bar" bar="42">} {emptyelem <img class="int" src="foobar">} </html>}>
>> doc.search("img")[1][:bar]
=> "42"
>> doc.search("img") - doc.search("img.int")
=> [{emptyelem img src"bar" bar"42"}]
Once you have results from search you can use normal array operations. nonstandard attributes are accessible through [].
Check out the not CSS selector.
(hbody."img:not(.int)")
(hbody."img:not(.ext)")
Unfortunately, it doesn't seem you can concat not expressions. You might want to fetch all img nodes and remove those where the .css selector doesn't include neither .int nor .ext.
Additionally, you could use the difference operator to calculate which elements are not part of both collections.
Use the .remove method to remove nodes or elements: Hpricot Altering documentation.

Resources