YQL two requests for paging/limit - yql

Question

I'm playing around with a XML API where the search doesn't support paging/limit. The recommended way is to just request all the IDs and then in a second request get the data and handle paging on your own.
First request:
http://example.com?search=foobar&columns=ID
<results>
<item><id>1</id></item>
<item><id>2</id></item>
<item><id>3</id></item>
<item><id>4</id></item>
<item><id>5</id></item>
<item><id>6</id></item>
<item><id>7</id></item>
<item><id>8</id></item>
<item><id>9</id></item>
<item><id>10</id></item>
</results>
Second request:
http://example.com?search=1,2,3,4,5&columns=ID,title,description
<results>
<item><id>1</id><title>foobar</title><description /></item>
<item><id>2</id><title>foobar</title><description /></item>
<item><id>3</id><title>foobar</title><description /></item>
<item><id>4</id><title>foobar</title><description /></item>
<item><id>5</id><title>foobar</title><description /></item>
</results>
Is it possible with YQL to combine this into a single request with a search result count and paging support?

Answer 1

I don't have a straight forward way from the documentation but you could do this:
1) Create a YQL table A which queries http://example.com?search=foobar&columns=ID
2) Create a YQL table B which queries http://example.com?search=1,2,3,4,5&columns=ID,title,description
3) Now, create a YQL table C which does a y.query on join of A and B like so:
select * from B where search in (select ids from A where search="foobar")
Ofcourse the query syntax will change based on table name and keys defined in it. For more information on YQL join refer here
Hope this is clear and if you find something better in this case, do let me know :)

Answer 2

Create a YQL table with paging:
<paging model="offset" matrix="true">
<start id="internalIndex" default="0" />
<pagesize id="internalPerPage" max="250" />
</paging>
Use Javascript to handle the two fetches and the paging:
var internalIndex = parseInt( request.matrixParams['internalIndex'] );
var internalPerPage = parseInt( request.matrixParams['internalPerPage'] );
var interimURL = 'http://example.com?columns=ID';
interimURL += '&search=' + request.queryParams['search'];
var interimQueryParameter = {url:interimURL};
var interimQuery = y.query("SELECT * FROM xml WHERE url=#url", interimQueryParameter);
var rows = interimQuery.results.*;
// get subset
var xml = rows;//this.copy(); // clone XML
var from = internalIndex;
var to = ((from + internalPerPage) < xml.length()) ? (from + internalPerPage) : xml.length();
var sliced = [];
for (; from < to; from++) {
sliced.push(xml[from].#["ID"]);
}
var finalURL = 'http://example.com?';
finalURL += '&search=' + sliced.join(",");
finalURL += "&columns=ID,title,description"
var finalQueryParameter = {url:finalURL};
var finalQuery = y.query("SELECT * FROM xml WHERE url=#url", finalQueryParameter);
var finalResults = finalQuery.results.response;
finalResults.node += <internalCurserPositon>{internalIndex}</internalCurserPositon>
finalResults.node += <internalCount>{rows.length()}</internalCount>
response.object = finalResults;