We are calling the /contacts API for SurveyMonkey. Docs are here. We need to get all active contacts, and I'm wondering about the paging feature. If we don't specify a per_page parameter, will we truly get all active contacts? Or will we get the first page only? There is no default listed for per_page, unfortunately, so I don't know how it will act.
A quick follow-up to this is wondering if there is a max for this field. If there is a default per_page, and I need to mess with paging, what is the max I can set per_page to?
We currently don't have a ton of contacts so it's tough to test the defaults/max.
By default, all list endpoints unless specified otherwise can accept a max page size of 1000 resources as specified under Request and Response Limits. Perhaps that should (also?) be under Pagination. Some bulk endpoints for example, have a max page size of 100.
I don't see in the docs what the page size is by default, but it is 50. So if you don't specify the number of resources to return, you will get up to 50 back.
Related
Through a script I can collect a sequence of videos that search list returns. The maxresults variable was set to 50. The total number items are big in number but the number of next page tokens are not enough to retrieve all the desired results. Is there any way to take all the returned items or it is YouTube restricted?
Thank you.
No, retrieving the results of a search is limited in size.
The total results that you are allowed to retrieve seems to have been reduced to 500 (in the past it was limited to 1000). The api does not allow you to retrieve more from a query. To try to get more, try using a number of queries with different parameters, like: publishedAfter, publishedBefore, order, type, videoCategoryId, or vary the query tags and keep track of getting different video id's returned.
See for a reference:
https://code.google.com/p/gdata-issues/issues/detail?id=4282
BTW. "totalResults" is an estimation and its value can change on the next page call.
See: YouTube API v3 totalResults field is returning 1 000 000 when it shoudn't
I'm trying to load the first 100 videos of a YouTube channel and I can only load 50 videos. I always get this error message
Max-results value is too high. Only up to 50 results can be returned per query.
and I'm using this URL.
http://gdata.youtube.com/feeds/api/users/shaytards/uploads?&max-results=100
I'm not sure if I need any kind of developer key to load 100 results. Even if you know a way to load videos 75-100 would be great but any help would be appreciated.
The Youtube data API requires you to set up your own pagination if you need to get more than 50 results. In this case, then, you would set max-results to 50 and do the query, then on your page have some sort of "next" button (or however you want to implement your pagination) that would trigger a new call (either through ajax, a new page, or whatever your own workflow is). This new call would be to the same data feed, but you'd append the parameter &start-index=51 so that you'd get 50 more results, numbered 51-100. You can continue on like this up to 1000 results.
See https://developers.google.com/youtube/2.0/reference#Paging_through_Results for more details.
In the YouTube Data API V3 it is a little different. Each search query returns a "nextPageToken" that can be used in a subsequent query for more results. Documentation: https://developers.google.com/youtube/v3/guides/implementation/pagination
I am currently trying to pull data about videos from a YouTube user upload feed. This feed contains all of the videos uploaded by a certain user, and is accessed from the API by a request to:
http://gdata.youtube.com/feeds/api/users/USERNAME/uploads
Where USERNAME is the name of the YouTube user who owns the feed.
However, I have encountered problems when trying to access feeds which are longer than 1000 videos. Since each request to the API can return 50 items, I am iterating through the feed using max_length and start_index as follows:
http://gdata.youtube.com/feeds/api/users/USERNAME/uploads?start-index=1&max-results=50&orderby=published
http://gdata.youtube.com/feeds/api/users/USERNAME/uploads?start-index=51&max-results=50&orderby=published
And so on, incrementing start_index by 50 on each call. This works perfectly up until:
http://gdata.youtube.com/feeds/api/users/USERNAME/uploads?start-index=1001&max-results=50&orderby=published
At which point I receive a 400 error informing me that 'You cannot request beyond item 1000.' This confused me as I assumed that the query would have only returned 50 videos: 1001-1051 in the order of most recently published. Having looked through the documentation, I discovered this:
Limits on result counts and accessible results
...
For any given query, you will not be able to retrieve more than 1,000
results even if there are more than that. The API will return an error
if you try to retrieve greater than 1,000 results. Thus, the API will
return an error if you set the start-index query parameter to a value
of 1001 or greater. It will also return an error if the sum of the
start-index and max-results parameters is greater than 1,001.
For example, if you set the start-index parameter value to 1000, then
you must set the max-results parameter value to 1, and if you set the
start-index parameter value to 980, then you must set the max-results
parameter value to 21 or less.
I am at a loss about how to access a generic user's 1001st last uploaded video and beyond in a consistent fashion, since they cannot be indexed using only max-results and start-index. Does anyone have any useful suggestions for how to avoid this problem? I hope that I've outlined the difficulty clearly!
Getting all the videos for a given account is supported, but you need to make sure that your request for the uploads feed is going against the backend database and not the search index. Because you're including orderby=published in your request URL, you're going against the search index. Search index feeds are limited to 1000 entries.
Get rid of the orderby=published and you'll get the data you're looking for. The default ordering of the uploads feed is reverse-chronological anyway.
This is a particularly easy mistake to make, and we have a blog post up explaining it in more detail:
http://apiblog.youtube.com/2012/03/keeping-things-fresh.html
The nice thing is that this is something that will no longer be a problem in version 3 of the API.
I'm calling the lists/statuses method as follows, but the results are inconsistent with the per_page parameter.
http://api.twitter.com/1/lists/statuses.xml?slug=wp1906ultras&owner_screen_name=enloes&per_page=20&page=3
In my example above where the per_page = 20, when I get page=1 I get 11 results. When I get page=2, I get 9 results and page=3 I get 12 results. If I set per_page to 11, I believe I get 5 results for page=1.
Does this make sense to anyone? Sure doesn't to me...
One of the reasons we've recently announced that paging will be deprecated in favor of the usage of since_id and max_id is because it's difficult to guarantee accurate paging metaphors from a service that doesn't & can't organize tweets that way.
All count and per_page parameters in the API are really "up to" parameters -- you'll get no more than the amount you specify.
This blog post outlines the upcoming deprecation: https://dev.twitter.com/blog/api-housekeeping and https://dev.twitter.com/docs/working-with-timelines outlines best practices for working with timelines like these.
I recommend moving away from using the paging metaphor to the usage of since_id and max_id for better reliability and long-term viability.
I'm using the Twitter API atom format
http://search.twitter.com/search.atom?q=Name&:)&since:year-month-date&rpp=1500
but it's only returning 100 tweets, I tried using the JSON format as well, but it only returned 100 results. Is there anything that I'm doing wrong to only get 100 results?
Yes, you're limited on the number of results per page. In order to get more results, you have to use the page parameter like so:
http://search.twitter.com/search.atom?q=Name&:)&since:year-month-date&rpp=1500&page=2
EDIT
rpp: the number of tweets to return
per page, up to a max of 100. E.g.,
http://search.twitter.com/search.atom?lang=en&q=devo&rpp=15
page: the page number to return, up to
a max of roughly 1500 results (based
on rpp * page)
Source: http://search.twitter.com/api/
In other words your rpp won't work as you expect because the max is 100.
My sugestion.
Make a request to your API and retrieve 100 results by time.
Use a loop to check if your result count is set to 100.
if true, do a new request to page 2.
test again and check the number of itens until the resultset is lower than 100.
The Twitter Search API has changed, including in the naming of the parameters: for instance, rpp is now count and the page parameter was removed in favor of max_id, a parameter based on a timeline concept:
"To use max_id correctly, an application’s first request to a
timeline endpoint should only specify a count. When processing this
and subsequent responses, keep track of the lowest ID received. This
ID should be passed as the value of the max_id parameter for the next
request, which will only return Tweets with IDs lower than or equal to
the value of the max_id parameter."
https://developer.twitter.com/en/docs/tweets/timelines/guides/working-with-timelines
The updated link to the Twitter search api is:
https://developer.twitter.com/en/docs/tweets/search/api-reference/get-search-tweets.html
Remember that not all tweets are indexed and if you are using the non-commercial version, you are limited to a 7-day search.