Transitioning from Scrubyt to Nokogiri- Write to XML or Hash? - ruby-on-rails

I'm trying to transition this bit of code from scrubyt to nokogiri, and am stuck trying to write my results to either a hash or xml. In scrubyt it looks like the following:
require 'rubygems'
require 'scrubyt'
result_data = Scrubyt::Extractor.define do
fetch "http://www.amazon.com/gp/offer-listing/0061673730"
results "//div[#class='resultsset']" do
item "//tbody/tr" do
condition "//div[#class = 'Condition']"
price "//span[#class = 'price']"
shipping "//span[#class = 'price_shipping']"
end
end
end
#description = result_data.to_xml
return #description
end
With nokogiri I can parse out the information I want, but there doesn't seem to be a quick way to return items in a hash or xml document. Here's all I have in nokogiri.
require 'rubygems'
require 'nokogiri'
require 'open-uri'
doc = Nokogiri::HTML(open('http://www.amazon.com/gp/offer-listing/0061673730'))
doc.css('div.condition, span.price, span.price_shipping ').each do |item|
puts item.content
end
How would one return item information to either xml or a hash?

You can use the Builder to build XML.
builder = Nokogiri::XML::Builder.new do |xml|
xml.root {
xml.items {
doc.css('div.condition, span.price, span.price_shipping').each do |o|
xml.item_content = o
end
}
}
end
puts builder.to_xml

Figured it out...
require 'rubygems'
require 'nokogiri'
require 'open-uri'
doc = Nokogiri::HTML(open('http://www.amazon.com/gp/offer-listing/0061673730'))
builder = Nokogiri::XML::Builder.new do |xml|
xml.root {
doc.xpath('//tbody[#class="result"]').each do |res|
xml.result {
res.css('span.price').each do |p|
xml.price = p.content
end
res.css('span.price_shipping').each do |s|
xml.ship = s.content
end
}
end
}
end
puts builder.to_xml
Results:
<?xml version="1.0"?>
<root>
<result>
<price=>$6.09</price=>
<ship=>+ $3.99</ship=>
</result>
<result>
<price=>$6.48</price=>
<ship=>+ $3.99</ship=>
</result>
<result>
<price=>$7.12</price=>
<ship=>+ $3.99</ship=>
</result>
<result>
<price=>$7.31</price=>
<ship=>+ $3.99</ship=>
</result>
<result>
<price=>$7.52</price=>
<ship=>+ $3.99</ship=>
</result>
<result>
<price=>$7.52</price=>
<ship=>+ $3.99</ship=>
</result>
<result>
<price=>$11.53</price=>
</result>
<result>
<price=>$7.56</price=>
<ship=>+ $3.99</ship=>
</result>
<result>
<price=>$7.61</price=>
<ship=>+ $3.99</ship=>
</result>
<result>
<price=>$7.61</price=>
<ship=>+ $3.99</ship=>
</result>
<result>
<price=>$7.95</price=>
<ship=>+ $3.99</ship=>
</result>
<result>
<price=>$7.95</price=>
<ship=>+ $3.99</ship=>
</result>
<result>
<price=>$8.59</price=>
<ship=>+ $3.99</ship=>
</result>
<result>
<price=>$8.99</price=>
<ship=>+ $3.99</ship=>
</result>
<result>
<price=>$10.05</price=>
<ship=>+ $3.99</ship=>
</result>
<result>
<price=>$10.32</price=>
<ship=>+ $3.99</ship=>
</result>
<result>
<price=>$10.32</price=>
<ship=>+ $3.99</ship=>
</result>
<result>
<price=>$10.55</price=>
<ship=>+ $3.99</ship=>
</result>
<result>
<price=>$10.56</price=>
<ship=>+ $3.99</ship=>
</result>
<result>
<price=>$11.42</price=>
<ship=>+ $3.99</ship=>
</result>
<result>
<price=>$11.59</price=>
<ship=>+ $3.99</ship=>
</result>
<result>
<price=>$11.90</price=>
<ship=>+ $3.99</ship=>
</result>
<result>
<price=>$11.95</price=>
<ship=>+ $3.99</ship=>
</result>
<result>
<price=>$12.07</price=>
<ship=>+ $3.99</ship=>
</result>
<result>
<price=>$12.35</price=>
<ship=>+ $3.99</ship=>
</result>
</root>

Thanks! That's exactly what I need. I'm having trouble looping correctly, though.
require 'rubygems'
require 'nokogiri'
require 'open-uri'
doc = Nokogiri::HTML(open('http://www.amazon.com/gp/offer-listing/0061673730'))
builder = Nokogiri::XML::Builder.new do |xml|
xml.root {
xml.item {
doc.css('span.price').each do |o|
xml.price = o
doc.css('span.price_shipping').each do |o|
end
end
}
}
end
puts builder.to_xml
That returns this:
<?xml version="1.0"?>
<root>
<item>
<price=><span class="price">$6.09</span></price=>
<price=><span class="price">$6.48</span></price=>
<price=><span class="price">$11.95</span></price=>
<ship=><span class="price_shipping">+ $3.99</span></ship=>
<ship=><span class="price_shipping">+ $3.99</span></ship=>
<ship=><span class="price_shipping">+ $3.99</span></ship=>
</item>
</root>
How would I rewrite my code to return something like this:
<?xml version="1.0"?>
<root>
<item>
<price=><span class="price">$6.09</span></price=>
<ship=><span class="price_shipping">+ $3.99</span></ship=>
</item>
<item>
<price=><span class="price">$6.48</span></price=>
<ship=><span class="price_shipping">+ $3.99</span></ship=>
</item>
<item>
<price=><span class="price">$11.95</span></price=>
<ship=><span class="price_shipping">+ $3.99</span></ship=>
</item>
</root>

you may want to omit "=" in
xml.price = p.content

Related

How to match template based on condition?

Here is my source XML which I'm trying to transform based on values of <City>
<?xml version="1.0" encoding="UTF-8"?>
<Workers>
<Worker>
<EmpID>12345</EmpID>
<City>NYC</City>
<Allowance>
<Type>Meal</Type>
<Amount>150</Amount>
</Allowance>
<Allowance>
<Type>Gym</Type>
<Amount>200</Amount>
</Allowance>
</Worker>
<Worker>
<EmpID>56789</EmpID>
<City>SFO</City>
<Base>
<BaseType>General</BaseType>
<BaseAmount>1000</BaseAmount>
</Base>
</Worker>
<Worker>
<EmpID>18978</EmpID>
<City>LAX</City>
<Base>
<BaseType>General</BaseType>
<BaseAmount>3000</BaseAmount>
</Base>
</Worker>
</Workers>
I'm attempting to transform as below. Transformation doesn't need be to applied if the value of <City> is either NYC or SFO
<?xml version="1.0" encoding="UTF-8"?>
<Root>
<SOAP-ENV:Body xmlns:SOAP-ENV="http://www.w3.org/2001/12/soap-envelope">
<m:GetQuotationResponse xmlns:m = "http://www.example.com">
<m:Worker>12345</m:Worker>
<m:Location>NYC</m:Location>
<m:Expense>
<m:ExpenseType>Meal</m:ExpenseType>
<m:Amount>150</m:Amount>
</m:Expense>
<m:Expense>
<m:ExpenseType>Gym</m:ExpenseType>
<m:Amount>200</m:Amount>
</m:Expense>
</m:GetQuotationResponse>
</SOAP-ENV:Body>
<SOAP-ENV:Body xmlns:SOAP-ENV="http://www.w3.org/2001/12/soap-envelope">
<m:GetQuotationResponse xmlns:m = "http://www.example.com">
<m:Worker>56789</m:Worker>
<m:Location>SFO</m:Location>
<m:Expense>
<m:ExpenseType>General</m:ExpenseType>
<m:Amount>1000</m:Amount>
</m:Expense>
</m:GetQuotationResponse>
</SOAP-ENV:Body>
</Root>
This is my attempt
<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
xmlns:xs="http://www.w3.org/2001/XMLSchema"
xmlns:SOAP-ENV = "http://www.w3.org/2001/12/soap-envelope"
xmlns:m = "http://www.example.com"
exclude-result-prefixes="xs m SOAP-ENV"
version="2.0">
<xsl:output method="xml" indent="yes"/>
<xsl:template match="Worker[City='NYC']">
<Root>
<SOAP-ENV:Body xmlns:m = "http://www.example.com">
<m:GetQuotationResponse>
<m:Worker><xsl:value-of select="EmpID"/></m:Worker>
<m:Location><xsl:value-of select="City"/></m:Location>
<xsl:for-each select="Allowance">
<m:Expense>
<m:ExpenseType><xsl:value-of select="Type"/></m:ExpenseType>
<m:Amount><xsl:value-of select="Amount"/></m:Amount>
</m:Expense>
</xsl:for-each>
</m:GetQuotationResponse>
</SOAP-ENV:Body>
</Root>
</xsl:template>
<xsl:template match="Worker[City='SFO']">
<Root>
<SOAP-ENV:Body xmlns:m = "http://www.example.com">
<m:GetQuotationResponse>
<m:Worker><xsl:value-of select="EmpID"/></m:Worker>
<m:Location><xsl:value-of select="City"/></m:Location>
<xsl:for-each select="Base">
<m:Expense>
<m:ExpenseType><xsl:value-of select="BaseType"/></m:ExpenseType>
<m:Amount><xsl:value-of select="BaseAmount"/></m:Amount>
</m:Expense>
</xsl:for-each>
</m:GetQuotationResponse>
</SOAP-ENV:Body>
</Root>
</xsl:template>
</xsl:stylesheet>
I have two issues(or more)
Couldn't get <Root> as the Parent node
Templates which are not matching values of <City> also returned. I wanted to match only <xsl:template match="Worker[City='NYC']"> or <xsl:template match="Worker[City='SFO']">
Current Output
<?xml version="1.0" encoding="UTF-8"?>
<Root>
<SOAP-ENV:Body xmlns:SOAP-ENV="http://www.w3.org/2001/12/soap-envelope">
<m:GetQuotationResponse xmlns:m="http://www.example.com">
<m:Worker>12345</m:Worker>
<m:Location>NYC</m:Location>
<m:Expense>
<m:ExpenseType>Meal</m:ExpenseType>
<m:Amount>150</m:Amount>
</m:Expense>
<m:Expense>
<m:ExpenseType>Gym</m:ExpenseType>
<m:Amount>200</m:Amount>
</m:Expense>
</m:GetQuotationResponse>
</SOAP-ENV:Body>
</Root>
<Root>
<SOAP-ENV:Body xmlns:SOAP-ENV="http://www.w3.org/2001/12/soap-envelope">
<m:GetQuotationResponse xmlns:m="http://www.example.com">
<m:Worker>56789</m:Worker>
<m:Location>SFO</m:Location>
<m:Expense>
<m:ExpenseType>General</m:ExpenseType>
<m:Amount>1000</m:Amount>
</m:Expense>
</m:GetQuotationResponse>
</SOAP-ENV:Body>
</Root>
18978
LAX
General
3000
Any help is appreciated to get this working using xslt 2.0 or xslt 3.0. Thank you
I'm posting the first solution that Martin Honnen has suggested.
<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
xmlns:xs="http://www.w3.org/2001/XMLSchema"
xmlns:SOAP-ENV = "http://www.w3.org/2001/12/soap-envelope"
xmlns:m = "http://www.example.com"
exclude-result-prefixes="xs m SOAP-ENV"
version="2.0">
<xsl:output method="xml" indent="yes"/>
<xsl:template match="Workers">
<Root>
<xsl:apply-templates/>
</Root>
</xsl:template>
<xsl:template match="Worker[City='NYC']">
<SOAP-ENV:Body xmlns:m = "http://www.example.com">
<m:GetQuotationResponse>
<m:Worker><xsl:value-of select="EmpID"/></m:Worker>
<m:Location><xsl:value-of select="City"/></m:Location>
<xsl:for-each select="Allowance">
<m:Expense>
<m:ExpenseType><xsl:value-of select="Type"/></m:ExpenseType>
<m:Amount><xsl:value-of select="Amount"/></m:Amount>
</m:Expense>
</xsl:for-each>
</m:GetQuotationResponse>
</SOAP-ENV:Body>
</xsl:template>
<xsl:template match="Worker[City='SFO']">
<SOAP-ENV:Body xmlns:m = "http://www.example.com">
<m:GetQuotationResponse>
<m:Worker><xsl:value-of select="EmpID"/></m:Worker>
<m:Location><xsl:value-of select="City"/></m:Location>
<xsl:for-each select="Base">
<m:Expense>
<m:ExpenseType><xsl:value-of select="BaseType"/></m:ExpenseType>
<m:Amount><xsl:value-of select="BaseAmount"/></m:Amount>
</m:Expense>
</xsl:for-each>
</m:GetQuotationResponse>
</SOAP-ENV:Body>
</xsl:template>
<xsl:template match="Worker[not(City = ('SFO', 'NYC'))]"/>
</xsl:stylesheet>
Here is what alternate solution returns. I'm not sure how this solution can be modified to get expected output since City='SFO' has different sibling node <Base> than City='NYC'
<?xml version="1.0" encoding="UTF-8"?>
<Root>
<SOAP-ENV:Body xmlns:SOAP-ENV="http://www.w3.org/2001/12/soap-envelope">
<m:GetQuotationResponse xmlns:m="http://www.example.com">
<m:Worker>12345</m:Worker>
<m:Location>NYC</m:Location>
<m:Expense>
<m:ExpenseType>Meal</m:ExpenseType>
<m:Amount>150</m:Amount>
</m:Expense>
<m:Expense>
<m:ExpenseType>Gym</m:ExpenseType>
<m:Amount>200</m:Amount>
</m:Expense>
</m:GetQuotationResponse>
</SOAP-ENV:Body>
<SOAP-ENV:Body xmlns:SOAP-ENV="http://www.w3.org/2001/12/soap-envelope">
<m:GetQuotationResponse xmlns:m="http://www.example.com">
<m:Worker>56789</m:Worker>
<m:Location>SFO</m:Location>
</m:GetQuotationResponse>
</SOAP-ENV:Body>
</Root>
Following part is missing from the output
<m:Expense>
<m:ExpenseType>General</m:ExpenseType>
<m:Amount>1000</m:Amount>
</m:Expense>
Alternate solution suggested by #Martin Honnen with some minor tweaks also provide desired output.
<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
xmlns:xs="http://www.w3.org/2001/XMLSchema"
xmlns:SOAP-ENV = "http://www.w3.org/2001/12/soap-envelope"
xmlns:m = "http://www.example.com"
exclude-result-prefixes="xs m SOAP-ENV"
version="2.0">
<xsl:output method="xml" indent="yes"/>
<xsl:template match="Workers">
<Root>
<xsl:apply-templates select="Worker[City = ('SFO', 'NYC')]"/>
</Root>
</xsl:template>
<xsl:template match="Worker">
<SOAP-ENV:Body xmlns:m = "http://www.example.com">
<m:GetQuotationResponse>
<m:Worker><xsl:value-of select="EmpID"/></m:Worker>
<m:Location><xsl:value-of select="City"/></m:Location>
<xsl:for-each select="Allowance">
<m:Expense>
<m:ExpenseType><xsl:value-of select="Type"/></m:ExpenseType>
<m:Amount><xsl:value-of select="Amount"/></m:Amount>
</m:Expense>
</xsl:for-each>
<xsl:for-each select="Base"> <!-- This part was included to get desired output without having to use more than one templates -->
<m:Expense>
<m:ExpenseType><xsl:value-of select="BaseType"/></m:ExpenseType>
<m:Amount><xsl:value-of select="BaseAmount"/></m:Amount>
</m:Expense>
</xsl:for-each>
</m:GetQuotationResponse>
</SOAP-ENV:Body>
</xsl:template>
</xsl:stylesheet>
Start with a template
<xsl:template match="Workers">
<Root>
<xsl:apply-templates/>
</Root>
</xsl:template>
then map your selected Workers to a SOAP body with a single template using e.g. <xsl:template match="Worker[City = ('SFO', 'NYC')]"> or <xsl:template match="Worker[City = 'SFO'] | Worker[City = 'NYC']">, if you prefer.
For other Workers, set up an empty template e.g. <xsl:template match="Worker[not(City = ('SFO', 'NYC'))]"/>.
As an alternative, you can of course just use a template matching Worker to map to a SOAP body and make your desired selection in the apply-templates of the first template I have show, i.e. change that to <xsl:apply-templates select="Worker[City = ('SFO', 'NYC')]"/>, that way you also ensure that only the wanted Workers are processed e.g.
<xsl:template match="Workers">
<Root>
<xsl:apply-templates select="Worker[City = ('SFO', 'NYC')]"/>
</Root>
</xsl:template>
<xsl:template match="Worker">
<SOAP-ENV:Body xmlns:m = "http://www.example.com">
<m:GetQuotationResponse>
<m:Worker><xsl:value-of select="EmpID"/></m:Worker>
<m:Location><xsl:value-of select="City"/></m:Location>
<xsl:for-each select="Allowance">
<m:Expense>
<m:ExpenseType><xsl:value-of select="Type"/></m:ExpenseType>
<m:Amount><xsl:value-of select="Amount"/></m:Amount>
</m:Expense>
</xsl:for-each>
</m:GetQuotationResponse>
</SOAP-ENV:Body>
</xsl:template>

Grouping and Identity transformation

How to apply identity transformation and grouping at the same time
<items>
<user>
<id>8788989</id>
<firstname>test</firstname>
<lastname>user</lastname>
</user>
<info>test xml</info>
<fromdate><fromdate>
<todate></todate>
<item id="123" name="Java">
<price>1</price>
<description></description>
</item>
<item id="123" name="Java and XML">
<price>2</price>
<description></description>
</item>
<item id="234" name="python">
<price>3</price>
<description></description>
</item>
<item id="234" name="scala">
<price>3</price>
<description></description>
</item>
</items>
I want output as
<items>
<user>
<id>8788989</id>
<firstname>test</firstname>
<lastname>user</lastname>
</user>
<info>test xml</info>
<fromdate><fromdate>
<todate></todate>
<group>
<item id="123" name="Java">
<price>1</price>
<description></description>
</item>
<item id="123" name="Java and XML">
<price>2</price>
<description></description>
</item>
</group>
<group>
<item id="234" name="python">
<price>3</price>
<description></description>
</item>
<item id="234" name="scala">
<price>3</price>
<description></description>
</item>
</group>
</items>
Grouping is done on item/#id
You can group like this:
<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
xmlns:xs="http://www.w3.org/2001/XMLSchema"
exclude-result-prefixes="xs"
version="2.0">
<xsl:output indent="yes"/>
<xsl:template match="#* | node()">
<xsl:copy>
<xsl:apply-templates select="#* | node()"/>
</xsl:copy>
</xsl:template>
<xsl:template match="items">
<items>
<xsl:apply-templates select="* except item"/>
<xsl:for-each-group select="item" group-by="#id">
<group>
<xsl:apply-templates select="../item[#id = current()/#id]"/>
</group>
</xsl:for-each-group>
</items>
</xsl:template>
</xsl:stylesheet>
UPDATED ANSWER:
<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
xmlns:xs="http://www.w3.org/2001/XMLSchema"
exclude-result-prefixes="xs"
version="2.0">
<xsl:output indent="yes"/>
<xsl:template match="#* | node()">
<xsl:copy>
<xsl:apply-templates select="#* | node()"/>
</xsl:copy>
</xsl:template>
<xsl:template match="items">
<items>
<xsl:apply-templates select="* except item"/>
<xsl:for-each-group select="item" group-by="#id">
<group>
<xsl:apply-templates select="current-group()"/>
</group>
</xsl:for-each-group>
</items>
</xsl:template>
</xsl:stylesheet>

Copy matching elements under one group element

I have a flat xml and I need to copy all matching elements under one element.
In input xml there are title elements which can randomly appear in xml. I want to put them under one element. Any help?
Input:
<root>
<element>
<para>Text 11.</para>
<para>Text 22.</para>
</element>
<title number="1">
<title.block>Title1</title.block>
<para>Text 33.</para>
<para>Text 44.</para>
</title>
<title number="2">
<title.block>Title2</title.block>
</title>
<element1>
<para>Some Text</para>
</element1>
<title number="3">
<title.block>Title2</title.block>
<para>Text 55.</para>
</title>
<result>
<para>Some Text</para>
</result>
</root>
desired output is:
<root>
<element>
<para>Text 11.</para>
<para>Text 22.</para>
</element>
<title.group>
<title number="1">
<title.block>Title1</title.block>
<para>Text 33.</para>
<para>Text 44.</para>
</title>
<title number="2">
<title.block>Title2</title.block>
</title>
<title number="3">
<title.block>Title2</title.block>
<para>Text 55.</para>
</title>
</title.group>
<element1>
<para>Some Text</para>
</element1>
<result>
<para>Some Text</para>
</result>
</root>
Simply write a template for the first title that creates the group and copies that title and all following siblings into the group:
<xsl:transform xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="2.0">
<xsl:output indent="yes"/>
<xsl:template match="#*|node()">
<xsl:copy>
<xsl:apply-templates select="#*|node()"/>
</xsl:copy>
</xsl:template>
<xsl:template match="root/title[1]">
<title-group>
<xsl:copy-of select="., following-sibling::title"/>
</title-group>
</xsl:template>
<xsl:template match="root/title[position() gt 1]"/>
</xsl:transform>
http://xsltransform.net/a9Giwu

MyBatis - some records are missing in result

I have a result map like:
<resultMap type="cz.pse.agata.commons.dto.Security" id="detailed_security">
<result property="id" column="VAID"/>
<result property="issueId" column="VAIDVA"/>
<result property="effectiveDate" column="VAEFDT"/>
<result property="bic" column="VABIC"/>
.
.
.
<association property="isin" column="VACVAL" javaType="cz.pse.agata.commons.dto.ISIN">
<result property="isin" column="VACVAL"/>
</association>
<association property="legalPerson" javaType="cz.pse.agata.commons.dto.LegalPerson"
resultMap="legalPerson" columnPrefix="specialist_person_"/>
</association>
<association property="emitent" column="VAEMIT" javaType="cz.pse.agata.commons.dto.LegalPerson"
resultMap="cz.pse.agata.commons.dto.mapper.LegalPersonMapper.legalPerson"/>
<association property="administrator" column="VAIDAD" javaType="cz.pse.agata.commons.dto.LegalPerson"
resultMap="legalPerson"/>
<association property="changeReason" column="VAIDCD" javaType="cz.pse.agata.commons.dto.ChangeReason">
<result property="id" column="CDID"/>
<result property="reason" column="CDTEXT"/>
</association>
<association property="tail" javaType="Object">
<discriminator javaType="String" column="VATVAL">
<case value="000" resultType="cz.pse.agata.commons.dto.SecurityTailImpl">
<result property="issueId" column="VAIDVA"/>
</case>
<case value="520" resultType="cz.pse.agata.commons.dto.SecurityTailImpl">
<result property="issueId" column="VAIDVA"/>
</case>
<case value="100" resultMap="cz.pse.cp.dto.mapper.BondMapper.Bond"/>
<case value="150" resultMap="cz.pse.cp.dto.mapper.BondMapper.Bond"/>
<case value="200" resultMap="cz.pse.cp.dto.mapper.BondMapper.Bond"/>
<case value="500" resultType="cz.pse.agata.commons.dto.SecurityTailImpl">
<result property="issueId" column="VAIDVA"/>
</case>
<case value="510" resultType="cz.pse.agata.commons.dto.SecurityTailImpl">
<result property="issueId" column="VAIDVA"/>
</case>
<case value="610" resultMap="cz.pse.cp.dto.mapper.WarrantMapper.Warrant"/>
<case value="660" resultMap="cz.pse.cp.dto.mapper.WarrantMapper.Warrant">
<result property="id" column="WAID"/>
</case>
<case value="600" resultMap="cz.pse.cp.dto.mapper.InvestementCertificateMapper.InvestementCertificate"/>
<case value="650" resultMap="cz.pse.cp.dto.mapper.InvestementCertificateMapper.InvestementCertificate"/>
<case value="300" resultType="cz.pse.agata.commons.dto.SecurityTailImpl">
<result property="issueId" column="VAIDVA"/>
</case>
<case value="305" resultType="cz.pse.agata.commons.dto.SecurityTailImpl">
<result property="issueId" column="VAIDVA"/>
</case>
<case value="400" resultType="cz.pse.agata.commons.dto.SecurityTailImpl">
<result property="issueId" column="VAIDVA"/>
</case>
</discriminator>
</association>
</resultMap>
And I have a query, that returns 4 rows:
==> Preparing: with dates as ( select vaefdt as dat from ....
==> Parameters: 109735(Long), 109735(Long), ....
<== Total: 4
But the resulting list has 3 items only.
After some debugging I realized, that it happens (probably) because 2 of the 4 rows have all data the same except tail values:
(Method org.apache.ibatis.executor.resultset.DefaultResultSetHandler.handleRowValuesForNestedResultMap(ResultSetWrapper, ResultMap, ResultHandler, RowBounds, ResultMapping))
private void handleRowValuesForNestedResultMap(ResultSetWrapper rsw, ResultMap resultMap, ResultHandler resultHandler, RowBounds rowBounds, ResultMapping parentMapping) throws SQLException {
final DefaultResultContext resultContext = new DefaultResultContext();
skipRows(rsw.getResultSet(), rowBounds);
Object rowValue = null;
while (shouldProcessMoreRows(resultContext, rowBounds) && rsw.getResultSet().next()) {
final ResultMap discriminatedResultMap = resolveDiscriminatedResultMap(rsw.getResultSet(), resultMap, null);
final CacheKey rowKey = createRowKey(discriminatedResultMap, rsw, null);
Object partialObject = nestedResultObjects.get(rowKey);
// issue #577 && #542
if (mappedStatement.isResultOrdered()) {
if (partialObject == null && rowValue != null) {
nestedResultObjects.clear();
storeObject(resultHandler, resultContext, rowValue, parentMapping, rsw.getResultSet());
}
rowValue = getRowValue(rsw, discriminatedResultMap, rowKey, rowKey, null, partialObject);
} else {
rowValue = getRowValue(rsw, discriminatedResultMap, rowKey, rowKey, null, partialObject);
if (partialObject == null) {
storeObject(resultHandler, resultContext, rowValue, parentMapping, rsw.getResultSet());
}
}
}
if (rowValue != null && mappedStatement.isResultOrdered() && shouldProcessMoreRows(resultContext, rowBounds)) {
storeObject(resultHandler, resultContext, rowValue, parentMapping, rsw.getResultSet());
}
}
Note bold text above: for the 2nd iteration proper rowValue is created, but it is not added to the result, because partialObject is not null.
Any help would be appreciated.

collect data from child links by web harvest

Is there any way to collect data from child link for Web Harvest?
Below is a xml segment I use:
<loop item="item" index="i">
<list><var name="products"/></list>
<body>
<xquery>
<xq-param name="item"><var name="item"/></xq-param>
<xq-expression><![CDATA[
declare variable $item as node() external;
for $i in $item//div[1]/p/a[#trace='auction'][1]
let $url := data($i/#href)
How can I grab data base on this new url which now is $url?
Please help me. Thx.
You just need to create another to contain this information. I have created a sample for you to understand it with ease. Please have a look:
SCRIPT:
<?xml version="1.0" encoding="UTF-8"?>
<config>
<var-def name="MainSite">http://www.appszoom.com/android_games/arcade_and_action</var-def>
<loop item="titles" index="i">
<list>
<xpath expression="//li[#class='app captureLinkBox']/div/div/span/a">
<html-to-xml>
<http url="${MainSite}"></http>
</html-to-xml>
</xpath>
</list>
<body>
<var-def name="titleURL">
<xpath expression="data(/a/#href)">
<var name="titles"/>
</xpath>
</var-def>
<file action="append" path="D:\navin.xml">
<xquery>
<xq-param name="titles"><template>${titles}</template></xq-param>
<xq-param name="titleURLContent">
<html-to-xml>
<http url="${titleURL}"></http>
</html-to-xml>
</xq-param>
<xq-expression>
<![CDATA[
declare variable $titles as node() external;
declare variable $titleURLContent as node() external;
<game>
<title>{$titles/a/text()}</title>
<downloads>{$titleURLContent//*[#id="left-bar"]/p[2]/span/text()}</downloads>
</game>
]]>
</xq-expression>
</xquery>
</file>
</body>
</loop>
</config>
OUTPUT:
<game>
<title>Clash of Clans</title>
<downloads>10,000,000 - 50,000,000</downloads>
</game>
<game>
<title>DEER HUNTER 2014</title>
<downloads>10,000,000 - 50,000,000</downloads>
</game>
<game>
<title>Subway Surfers</title>
<downloads>100,000,000 - 500,000,000</downloads>
</game>
<game>
<title>RoboCop™</title>
<downloads>5,000,000 - 10,000,000</downloads>
</game><game>
<title>DragonFlight for Kakao</title>
<downloads>10,000,000 - 50,000,000</downloads>
</game>
<game>
<title>Castle Clash</title>
<downloads>10,000,000 - 50,000,000</downloads>
</game>
<game>
<title>Sonic Dash</title>
<downloads>10,000,000 - 50,000,000</downloads>
</game>
<game>
<title>Injustice: Gods Among Us</title>
<downloads>1,000,000 - 5,000,000</downloads>
</game>
<game>
<title>Banana Kong</title>
<downloads>10,000,000 - 50,000,000</downloads>
</game>
<game>
<title>Temple Run 2</title>
<downloads>100,000,000 - 500,000,000</downloads>
</game>
You do not give full code, for me to check running, but this should get you on your way:
<config>
<loop item="item" index="i">
<list><var name="products"/></list>
<body>
<var-def name="new_url">
<xquery>
<xq-param name="item"><var name="item"/></xq-param>
<xq-expression><![CDATA[
declare variable $item as node() external;
for $i in $item//div[1]/p/a[#trace='auction'][1]
let $url := data($i/#href)
return
{$url}
]]></xq-expression>
</xquery>
</var-def>
<!-- now your new url is saved in webharvest variable new_url and you are free to run a
new webharvest http request using it -->
<var-def name="new_page_content">
<http url="${new_url}"/>
</var-def>
<!-- now the content of the new page has been downloaded and saved in new variable
new_page_content and you are free to query it further should you want to -->
<var-def name="contact">
<xpath expression="//a[contains(., 'contact')]/#href">
<var name="new_page_content"/>
</xpath>
</body>
</loop>
</config>

Resources