Is it possible to get the next page of results for a duck duck go search on the html version using a url query param?
I found that with the right parameters, it is possible to get different pages from the /html/ version of duckduckgo.com.
You need to specify:
q for the search
s for the offset of the current page (nothing for the first page)
dc for the offset of the current page + 1 (start of the new page, nothing for the first page)
v to the arbitrary value l
o to the format json
api to the endpoint /d.js
Which gives, given the search $term and the page $n:
https://duckduckgo.com/html/?q=$term&s=$(($n*30))&dc=$(($n*30+1))&v=l&o=json&api=/d.js
And for the first page:
https://duckduckgo.com/html/?q=$term&s=&dc=&v=l&o=json&api=/d.js
Note 1: This can be used with both GET and POST HTML methods.
Note 2: According to my tests, it appears that the s and dc parameters need to respectively be a multiple of 30, and a multiple of 30, plus 1.
HTH
Related
On third page not getting next page token google place api?
I am using Google place text search API in my Ruby on rails application, everything is working fine but after third page I am not getting any next page token so for every text search I am getting only 60 result. Is I am missing something please suggest any help would be appreciable. This happen for every text.
My request for first page :-
https://maps.googleapis.com/maps/api/place/textsearch/json?key=#{my_key}&query=#{my_query}
My request for other page with token:-
https://maps.googleapis.com/maps/api/place/textsearch/json?key=#{my_key}&pagetoken=#{next_page_token}
And usually I am searching on google it show 100's of result for same place text. How can I get result more than 60.
This is intended behavior for Google's Places API, as you can only get up to 60 places, split across 3 pages (3 queries). This is why there is no next_page_token on the third page.
By default, each Nearby Search or Text Search returns up to 20
establishment results per query; however, each search can return as
many as 60 results, split across three pages. If your search will
return more than 20, then the search response will include an
additional value — next_page_token.
Reference here.
After a lot of debugging, it finally occured to me that seemingly Youtube is only issueing the first 100 comments when using the v2 YouTube-API for getting comments. I finally tried using:
curl -Lk -X GET "http://gdata.youtube.com/feeds/api/videos/MShbP3OpASA/comments?alt=json&start-index=100&max-results=50"
And all I get is a response without an entry parameter. That is to say, I do not receive an error response or something like that - I get a perfectly good response, but without the entry parameter.
Digging a little deeper, in my response the value for openSearch$totalResults is 100, so in accordance to this resource this seems to be the expected result (although it tells about some kind of error message which I don't get?).
But here comes the kicker: When I use
curl -Lk -X GET "http://gdata.youtube.com/feeds/api/videos/MShbP3OpASA/comments?alt=json&start-index=1&max-results=50&orderby=published"
openSearch$totalResults equals 3141, the actual count of the comments.
Now here is my question: Since the v2 API is officially been deprecated about a week ago, is it possible that Google just set up a limit on the comments? So only the first 100 comments are accessible? Since the v3 API does not allow for comment retrieval, that would be a pretty bummer for me.
Does anyone have any ideas?
I've figured out how to retrieve all the comments using the navigation links embedded in the json response.
Suppose you retrieve the first using a link like (python here, but you get the point):
r'https://gdata.youtube.com/feeds/api/videos/' + aVideoID + r'/comments?alt=json&start-index=1&max-results=50&prettyprint=true&orderby=published'
Embedded in the json under "feed" (and before the comments) will be a four element array called "link". The fourth element will be called "rel": "next" and under "href" there will be a link you can use to get the next 50 comments. The link will look something like:
https://gdata.youtube.com/feeds/api/videos/fH0cEP0mvlU/comments?alt=json&orderby=published&alt=json&start-token=EgkI2NqyoZDRvgIosK%2FPosPRvgIw653cmsXRvgI4AUAC&max-results=50&orderby=published
for an original URL of:
https://gdata.youtube.com/feeds/api/videos/fH0cEP0mvlU/comments?alt=json&start-index=1&max-results=50&prettyprint=true&orderby=published
If you follow the next link it will return similar json to the original link, with another 50 comments. Continue this process over and over until you get all the comments (in my code I check for both the absence of this item in the json or zero comments in the json to determine when to stop).
You need the "&orderby=published" in the original URL because otherwise the "next" links eventually grow to be too large and cause an error (something in the token the API uses to track which comments you've seen in the default orderby takes a lot of space). Something about the published orderby keeps the "start-token" small, whereas after about 500 comments with the default orderby you will start getting 414 Request URI too long errors.
Hope this helps.
My problem is i need a bit of code to reference a tracking number eg 1193849302, and insert it into a url http://www.example.com/track=[tracking number], so that the [tracking number] is replaced by 1193849302
then to have showing only the tracking number which is hyperlinked to said URL, i.e. 1193849302 (hyperlinked)
Either in html or javascript I think most of the site is CSS based though
Is this possible?
Thanks in advance
Edit maybe i should explain what i want a bit better
I have a customer area on a retail website , which returns the customers parcels tracking number when despatched.
What i'm after is a way to turn this tracking number into a working hyperlink which when clicked opens a new window into the couriers website automatically pushing the tracking number into the url and tracking the parcel for them.
Does this sound possible??
Most languages have a String.replace function.
For example, in Javascript, use replace() for this. Note that it takes a regex as an argument, so the [] need to be escaped:
var str = "http://www.example.com/track=[tracking number]";
str = str.replace("\[tracking number\]", "1193849302");
use String.replace method
The replace() method searches a string for a specified value, or a regular expression, and returns a new string where the specified values are replaced.
var str="http://www.example.com/track=[tracking number]";
var n=str.replace("tracking number","1193849302");
You can find more informations at http://www.w3schools.com/jsref/jsref_replace.asp
I'm planning out how to track internal search data in Omniture/SiteCatalyst.
It's a fairly straight-forward plan for a standard "enter a term and get a page of results" model: set sProps and eVars with the terms, the count of results, and the page searched from, then fire a success event for searching and another for clicking a search result.
For a type-ahead search--where the user is given search results as they type in a search bar--what's a good strategy for handling the timing of event submissions so that you don't end up with different events/entries for letters 4, 5, 6, and 7 of a search term's entry?
Our solution was to leverage a delay on the autocomplete to reduce the number of calls. From a tracking standpoint, if someone pauses for 1 second (or 500 ms, whatever), then they're probably actually waiting for the autocomplete results, and that constitutes a valid search.
From a technical standpoint, we leveraged the delay option on the jQuery UI widget.
Strategy I've always used is to not track the "auto-complete" search features..put the tracking on the search results page same as normal. Or are you saying the whole search results page is being output as the user types? If that is the case...one thing you could do is write some code to pop the Omniture code when the search field loses focus.
Another thing you can do is as the visitor is typing in the search bar, on each keypress, write the current value to a cookie. Then have some code that runs on page load to look for that cookie and if it exists, pop the Omniture search variables and erase the cookie. Alternatively you can keep track of current value w/ a server-side session variable since I assume this thing is ajax driven, and output the omn code w/ server-side code if session var exists. These methods would mean that the search events and vars would not pop on the search results page...this probably isn't that big a deal, unless you have supporting variables you pop, like an "internal search referrer" prop/eVar that keeps track of the previous page the visitor was on (or the page the visitor was on when they performed the search). So you'll have to keep that in mind and carry that over as well.
Whenever you do a search you might be aware of the concept that query string parameter get added at the end of URL.
Suppose www.stackoverfow.com is website and when are you performing a search on it then it will be like www.stackoverflow.com?q=yourname , yourname is the searchkeyword.This keyword we can capture in sitecatalyst.
you can see google.com while searching on internet for sitecatalyst is ---
www.google.co.in/search?q=sitecatalyst
In the same way we can use query string parameter as q = something.
after doing all this thing we can use the plugin getQueryParam in plugin section of the s_code library file to fetch that variable and store that in sitecatalyst variable...
example:-
function s_doPlugins(s) {
var one = s.getQueryParam("q");
if(one)
s.eVar1=one;
}
s.doPlugins=s_doPlugins
insert this below code outside plugin section
/*
* Returns the value of a specified query string parameter, if found in the current page URL.
*/
s.getQueryParam=new Function("p","d","u",""
+"var s=this,v='',i,t;d=d?d:'';u=u?u:(s.pageURL?s.pageURL:s.wd.locati"
+"on);if(u=='f')u=s.gtfs().location;while(p){i=p.indexOf(',');i=i<0?p"
+".length:i;t=s.p_gpv(p.substring(0,i),u+'');if(t){t=t.indexOf('#')>-"
+"1?t.substring(0,t.indexOf('#')):t;}if(t)v+=v?d+t:t;p=p.substring(i="
+"=p.length?i:i+1)}return v");
s.p_gpv=new Function("k","u",""
+"var s=this,v='',i=u.indexOf('?'),q;if(k&&i>-1){q=u.substring(i+1);v"
+"=s.pt(q,'&','p_gvf',k)}return v");
s.p_gvf=new Function("t","k",""
+"if(t){var s=this,i=t.indexOf('='),p=i<0?t:t.substring(0,i),v=i<0?'T"
+"rue':t.substring(i+1);if(p.toLowerCase()==k.toLowerCase())return s."
+"epa(v)}return ''");
you will find that it will capture your search results
please let me know in case of more clarifications
As there are 2 response format(Atom and Json) from Twitter search api, i am trying to fetch the twits(Json format) by using the URL:[http://search.twitter.com/search.json?q=pareshmayani], but you know it is returning only 1 page with 15 twits at a time(and yes its as per the twitter search API documentation) same as the below image:
1
But what if i want to fetch 2nd page of twits, i know i can use page querystring value such as http://search.twitter.com/search.json?q=pareshmayani&page=2, but when i use it, it returns me the below output:
Problem: So As you have noticed the above response, it should return 2nd page of 15 twits in response. Please give me suggestions if i am doing something wrong to fetch 2nd page of twits.
Thanx,
Paresh
There is only one page of results.
If you search for a popular term, like cheryl cole, you will see a 'next_page' field returned in the JSON. This field contains a query string to retrieve the next page.