Like the title.
I do a request like this :
https://www.googleapis.com/youtube/v3/search?key=AIzaSyDuxczhyyvHWfxKuF3ygW9p0GWmKlvWLYc&part=id,snippet&publishedAfter=2014-12-09T00:00:00Z&publishedBefore=2014-12-11T00:00:00Z&videoCategoryId=GCSG93LXRvICYgRElZ&type=video&maxResults=50&pageToken=
Total result is 1000000. But I just can get 500 results maximum (10 page, 50 results/page).
At 10th page, I don't see nextPageToken property to go to the next page. ???
I don't know why.
How can I get all of result.
YouTube imposes a soft limit of about 500. There is no direct way to get more than that through the API.
Full details: https://code.google.com/p/gdata-issues/issues/detail?id=4282
Relevant Excerpt:
"We can't provide more than ~500 search results for any arbitrary YouTube query via the API without the quality of the search results severely degrading (duplicates, etc.).
The v1/v2 GData API was updated back in November to limit the number of search results returned to 500. If you specify a start-index of 500 or more, you won't get back any results.
This was supposed to have also gone into effect for the v3 API (which uses a different method of paging through results) but it apparently was not pushed out, so it is still possible to retrieve up to 1000 search results in v3—the last 500 of which are usually of bad quality.
The change to limit v3 to 500 search results will be pushed out sometime in the near future. There will no longer be nextPageTokens returned once you hit 500 results.
I understand that the totalResults that are returned is much higher than 500 in all of these cases, but that is not the same thing as saying that we can effectively return all X million possible results. It's meant as an estimate of the total size of the set of videos that match a query and normally isn't very useful."
Updated - How to get around the 500 result max soft limit
Use the filters 'publishedAfter' and 'publishedBefore' to break up your query into loops of queries by day/week/month until no more results are returned. Each periodic query should return less than 500 results each, but you'll get them all.
There's documentation for channelId (still not for videoCategoryId) by the way
https://developers.google.com/youtube/v3/docs/search/list#channelId
Note: Search results are constrained to a maximum of 500 videos if your request specifies a value
for the channelId parameter and sets the type parameter value to video, ...
Related
Good day. I'm trying to get the top videos for gaming within the specified publish dates ordered by view count. I'm having a problem with the number of items that it is returning in the JSON response because it's less than 50 items even though I have set the maxResults parameter to 50 and the JSON response returns pageInfo.resultsPerPage equal to 50. The pageInfo.totalResults also returns a lot more than 50 so it should be displaying 50 items in the result. As an example, I'm using the followng URL
https://www.googleapis.com/youtube/v3/search?key={API_Key}&part=snippet&maxResults=50&order=viewCount&publishedAfter=2017-02-01T00%3A00%3A00Z&publishedBefore=2017-02-01T00%3A01%3A59Z&type=video&videoCategoryId=20
The returned result here the last time I ran this query has 20 items even though there are 161,307 total results. I would like to ask a solution that will make sure that I will always get the number of items equal to the maxResults if possible. Hope someone can help me on this. Thank you very much.
EDIT: I know how to use the page token, but I'm not aiming for getting the items that I need for more than one request (as the Data API have limited credits per day). The issue that I'm trying to resolve is to make sure that I always get 50 items everytime I invoke the request.
Although there are 160k+ total results, your publishedAfter/publishedBefore filters cut the number returned down to the videos published in that time range, which is 20.
Well, I got 50 results with the URL request that you provided. Just make sure that you place right value on the parameter that you use, especially the value in publishedAfter. You can also use the parameter pageToken to get the next page or next 50 results.
For more information, check this SO question on how to use pageToken
Through a script I can collect a sequence of videos that search list returns. The maxresults variable was set to 50. The total number items are big in number but the number of next page tokens are not enough to retrieve all the desired results. Is there any way to take all the returned items or it is YouTube restricted?
Thank you.
No, retrieving the results of a search is limited in size.
The total results that you are allowed to retrieve seems to have been reduced to 500 (in the past it was limited to 1000). The api does not allow you to retrieve more from a query. To try to get more, try using a number of queries with different parameters, like: publishedAfter, publishedBefore, order, type, videoCategoryId, or vary the query tags and keep track of getting different video id's returned.
See for a reference:
https://code.google.com/p/gdata-issues/issues/detail?id=4282
BTW. "totalResults" is an estimation and its value can change on the next page call.
See: YouTube API v3 totalResults field is returning 1 000 000 when it shoudn't
I'm trying to load the first 100 videos of a YouTube channel and I can only load 50 videos. I always get this error message
Max-results value is too high. Only up to 50 results can be returned per query.
and I'm using this URL.
http://gdata.youtube.com/feeds/api/users/shaytards/uploads?&max-results=100
I'm not sure if I need any kind of developer key to load 100 results. Even if you know a way to load videos 75-100 would be great but any help would be appreciated.
The Youtube data API requires you to set up your own pagination if you need to get more than 50 results. In this case, then, you would set max-results to 50 and do the query, then on your page have some sort of "next" button (or however you want to implement your pagination) that would trigger a new call (either through ajax, a new page, or whatever your own workflow is). This new call would be to the same data feed, but you'd append the parameter &start-index=51 so that you'd get 50 more results, numbered 51-100. You can continue on like this up to 1000 results.
See https://developers.google.com/youtube/2.0/reference#Paging_through_Results for more details.
In the YouTube Data API V3 it is a little different. Each search query returns a "nextPageToken" that can be used in a subsequent query for more results. Documentation: https://developers.google.com/youtube/v3/guides/implementation/pagination
Is there some way I can retrieve more than 100 results using YouTube API?
I can retrive upto 100 results using this query string:
https://gdata.youtube.com/feeds/api/videos?q=football&orderby=relevance&start-index=1&max-results=10&v=2
But when I try
https://gdata.youtube.com/feeds/api/videos?q=football&orderby=relevance&start-index=101&max-results=10&v=2
I get:
http://www.w3.org/2005/Atom' xmlns:openSearch='http://a9.com/-/spec/opensearch/1.1/' xmlns:gd='http://schemas.google.com/g/2005' gd:etag='W/"C0YFQHg9fyp7I2A9WhBVGUk."'>tag:youtube.com,2008:videos2013-04-26T01:51:51.667Zhttp://schemas.google.com/g/2005#kind' term='http://gdata.youtube.com/schemas/2007#video'/>Videos matching: footballhttp://www.youtube.com/img/pic_youtubelogo_123x63.gifhttps://www.youtube.com'/>http://schemas.google.com/g/2005#feed' type='application/atom+xml' href='https://gdata.youtube.com/feeds/api/videos?v=2'/>http://schemas.google.com/g/2005#batch' type='application/atom+xml' href='https://gdata.youtube.com/feeds/api/videos/batch?v=2'/>https://gdata.youtube.com/feeds/api/videos?q=football&start-index=101&max-results=10&orderby=relevance&v=2'/>https://gdata.youtube.com/feeds/api/videos?alt=atom-service&v=2'/>https://gdata.youtube.com/feeds/api/videos?q=football&start-index=91&max-results=10&orderby=relevance&v=2'/>YouTubehttp://www.youtube.com/http://gdata.youtube.com'>YouTube data API10010110
From what I understand, you can get 999 results using YouTube search API. Why is this limited to 100 here? Is there a workaround for me to get more than 100 results?
Thanks!
I can't speak to whether or not something has changed internally that is now capping totalResults at 100, but it seems to be a pretty common problem across the board today. However, if you were to try using v3 of the API, this wouldn't be an issue: for example, this query:
https://www.googleapis.com/youtube/v3/search?q=football&part=id&key=YOUR_DEVELOPER_KEY
Will give you many, many more total results. With v3, you can set the maxResults parameter up to 50 videos returned for each response, and then paginate through the responses by setting the pageToken parameter to whatever the "nextPageToken" attribute of a given response is.
Try like this https://gdata.youtube.com/feeds/api/videos?q=football&orderby=relevance&v=2
It will take the default size. When you use the start index and size you are using the pagination mechanism. I guess thats the problem here. And misfortune is that Google youtube API supports only max of 50 videos per query. Its there in the docs. 999 videos are the total videos that can be in a video feed. Not the number of videos it would query out. Dont mistakenly take it as it would return you all the videos. You have to use pagination with start index and size inorder to go through all the videos.
you can use this url
https://www.googleapis.com/youtube/v3/search?q=football&part=id&key=YOUR_DEVELOPER_KEY&max-results=150
and you easily get more than 100 results
I'm confused by this same issue - video search queries with start-index > 100 do not return any results:
http://gdata.youtube.com/feeds/api/videos?v=2&q=skateboarding+dog&max-results=50&start-index=101
and queries with a start-index < 100 return only the number of videos up to 100:
http://gdata.youtube.com/feeds/api/videos?v=2&q=skateboarding+dog&max-results=50&start-index=90
gives you 10 videos.
Shouldn't you be able to access results beyond the first 100 returned by the search?
There's a caching/escalating issue for pagination.
Check this answer here.
The correct way to page through a feed is to make the first request
for the feed without a start-index, and then check to see whether
there's a <link rel="next'...> and if so, use that URL to retrieve the
next page of a feed.
I discovered that if you put a small pause between each pagination call to API, it returns 400 results, it's mean 1 call per page putting 2 second between each call. It works for my. If you wuould like get more it, you have to close the connection to API and try again one or two minutes later.
I am currently trying to pull data about videos from a YouTube user upload feed. This feed contains all of the videos uploaded by a certain user, and is accessed from the API by a request to:
http://gdata.youtube.com/feeds/api/users/USERNAME/uploads
Where USERNAME is the name of the YouTube user who owns the feed.
However, I have encountered problems when trying to access feeds which are longer than 1000 videos. Since each request to the API can return 50 items, I am iterating through the feed using max_length and start_index as follows:
http://gdata.youtube.com/feeds/api/users/USERNAME/uploads?start-index=1&max-results=50&orderby=published
http://gdata.youtube.com/feeds/api/users/USERNAME/uploads?start-index=51&max-results=50&orderby=published
And so on, incrementing start_index by 50 on each call. This works perfectly up until:
http://gdata.youtube.com/feeds/api/users/USERNAME/uploads?start-index=1001&max-results=50&orderby=published
At which point I receive a 400 error informing me that 'You cannot request beyond item 1000.' This confused me as I assumed that the query would have only returned 50 videos: 1001-1051 in the order of most recently published. Having looked through the documentation, I discovered this:
Limits on result counts and accessible results
...
For any given query, you will not be able to retrieve more than 1,000
results even if there are more than that. The API will return an error
if you try to retrieve greater than 1,000 results. Thus, the API will
return an error if you set the start-index query parameter to a value
of 1001 or greater. It will also return an error if the sum of the
start-index and max-results parameters is greater than 1,001.
For example, if you set the start-index parameter value to 1000, then
you must set the max-results parameter value to 1, and if you set the
start-index parameter value to 980, then you must set the max-results
parameter value to 21 or less.
I am at a loss about how to access a generic user's 1001st last uploaded video and beyond in a consistent fashion, since they cannot be indexed using only max-results and start-index. Does anyone have any useful suggestions for how to avoid this problem? I hope that I've outlined the difficulty clearly!
Getting all the videos for a given account is supported, but you need to make sure that your request for the uploads feed is going against the backend database and not the search index. Because you're including orderby=published in your request URL, you're going against the search index. Search index feeds are limited to 1000 entries.
Get rid of the orderby=published and you'll get the data you're looking for. The default ordering of the uploads feed is reverse-chronological anyway.
This is a particularly easy mistake to make, and we have a blog post up explaining it in more detail:
http://apiblog.youtube.com/2012/03/keeping-things-fresh.html
The nice thing is that this is something that will no longer be a problem in version 3 of the API.