I'm writing a PHP script that retrive all videos of a channel using Google APIs Client Library for PHP and youtube.search.list with params:
part = snippet
maxResults = 50
channelId = ...
type = video
order = rating
pageToken = [nextPageToken]
Scrolling all the results of pagination I see that sometimes the search returns videos that were present in the previous pages.
I have tested thoroughly and the calls to the next pages are correct with the right nextPageToke.
Related
Assumption / What I want to achieve
I want to use YouTube Data API V3 to get the video ID without any omissions, and find out if the cause of the trouble is in the code or in the video settings of YouTube (API side).
Problem
The following code is used to get the video information from YouTube Data API, but the number of IDs I got did not match the number of videos that are actually posted.
from apiclient.discovery
import build
id = "UCD-miitqNY3nyukJ4Fnf4_A" #sampleID
token_check = None
nextPageToken = None
id_info = []
while True:
if token_check != None:
nextPageToken = token_check
Search_Video = youtube.search().list(
part = "id",
channelId = id,
maxResults = 50,
order = 'date',
safeSearch = "none",
pageToken = nextPageToken
).execute()
for ID_check in Search_Video.get("items", []):
if ID_check["id"]["kind"] == "youtube#video":
id_info.append(ID_check["id"]["videoId"])
try:
token_check = Search_Video["nextPageToken"]
except:
print(len(id_info)) #check number of IDs
break
I also used the YouTube Data API function to get the videoCount information of the channel, and noticed that the value of videoCount did not match the number of IDs obtained by the code above, which is why I posted this.
According to channels() API, this channel have 440 videos, but the above code gets only 412 videos (at 10:30 a.m. JST).
Supplemental Information
・Python 3.9.0
・YouTube Data API v3
You have to acknowledge that the Search.list API endpoint does not have a crisp behavior. That means you should not expect precise results from it. Google does not document this behavior as such, but this forum has many posts from users experiencing that.
If you want to obtain all the IDs of videos uploaded by a given channel then you should employ the following two-step procedure:
Step 1: Obtain the ID of the Uploads Playlist of a Channel.
Invoke the Channels.list API endpoint, queried with its request parameter id set to the ID of the channel of your interest (or, otherwise, with its request parameter mine set to true) for to obtain that channel's uploads playlist ID, contentDetails.relatedPlaylists.uploads.
def get_channel_uploads_playlist_id(youtube, channel_id):
response = youtube.channels().list(
fields = 'items/contentDetails/relatedPlaylists/uploads',
part = 'contentDetails',
id = channel_id,
maxResults = 1
).execute()
items = response.get('items')
if items:
return items[0] \
['contentDetails'] \
['relatedPlaylists'] \
.get('uploads')
else:
return None
Do note that the function get_channel_uploads_playlist_id should only be called once for to obtain the uploads playlist
ID of a given channel; subsequently use that ID as many times as needed.
Step 2: Retrieve All IDs of Videos of a Playlist.
Invoke the PlaylistItems.list API endpoint, queried with its request parameter playlistId set to the ID obtained from get_channel_uploads_playlist_id:
def get_playlist_video_ids(youtube, playlist_id):
request = youtube.playlistItems().list(
fields = 'nextPageToken,items/snippet/resourceId',
playlistId = playlist_id,
part = 'snippet',
maxResults = 50
)
videos = []
is_video = lambda item: \
item['snippet']['resourceId']['kind'] == 'youtube#video'
video_id = lambda item: \
item['snippet']['resourceId']['videoId']
while request:
response = request.execute()
items = response.get('items', [])
assert len(items) <= 50
videos.extend(map(video_id, filter(is_video, items)))
request = youtube.playlistItems().list_next(
request, response)
return videos
Do note that, when using the Google's APIs Client Library for Python (as you do), API result set pagination is trivially simple: just use the list_next method of the Python API object corresponding to the respective paginated API endpoint (as was shown above):
request = API_OBJECT.list(...)
while request:
response = request.execute()
...
request = API_OBJECT.list_next(
request, response)
Also note that above I used twice the fields request parameter. This is good practice: ask from the API only the info that is of actual use.
Yet an important note: the PlaylistItems.list endpoint would not return items that correspond to private videos of a channel when invoked with an API key. This happens when your youtube object was constructed by calling the function apiclient.discovery.build upon passing to it the parameter developerKey.
PlaylistItems.list returns items corresponding to private videos only to the channel owner. This happens when the youtube object is constructed by calling the function apiclient.discovery.build upon passing to it the parameter credentials and if credentials refer to the channel that owns the respective playlist.
An additional important note: according to Google staff, there's an upper 20000 limit set by design for the number of items returned via PlaylistItems.list endpoint when queried for a given channel's uploads playlist. This is unfortunate, but a fact.
We got to build our own reporting database for our Youtube channel to measure the channel and video performance.
To support this, we implemented an ETL job to extract using Youtube Analytics API and used below python code to get the data.
def GetAnalyticsData(extractDate,accessToken, channelId):
channelId = 'channel%3D%3D{0}'.format(channelId)
headers = {'Authorization': 'Bearer {}'.format(accessToken),
'accept': 'application/json'}
url = 'https://youtubeanalytics.googleapis.com/v2/reports?dimensions={dimensions}&endDate={enddate}&ids={ids}&maxResults={maxresults}&metrics={metrics}&startDate={startdate}&alt={alt}&sort={sort}'.format(
dimensions='video',
ids=channelId,
enddate= extractDate,
startdate=extractDate,
metrics = 'views%2Ccomments%2Clikes%2Cdislikes%2Cshares%2CestimatedMinutesWatched%2CsubscribersGained%2CsubscribersLost%2CannotationClicks%2CannotationClickThroughRate%2CaverageViewDuration%2CaverageViewPercentage%2CannotationCloseRate%2CannotationImpressions%2CannotationClickableImpressions%2CannotationClosableImpressions%2CannotationCloses',
maxresults = 200,
alt ='json',
sort='-views'
)
return requests.get(url,headers=headers)
We hit this API everyday and get all the video metric and sorted by views in descending order.
This solved our need partially and it returns only 200 videos, if we specify maxResults more than 200, its return 400 error code.
The challenge is, how to get all videos for the given date and given channel?
Thanks in advance.
Regards,
Guna
I am not keen on YouTube analytics API, but it seems that you are looking for startIndex.
startIndex
integer
The 1-based index of the first entity to retrieve. (The default value is 1.) Use this parameter as a pagination mechanism along with the max-results parameter.
I am trying to get a list of all titles from YouTube channel machinima. Machinima currently has (I'm guessing here) tens of thousands of videos.
The following Python script is supposed to walk the chain of pages containing playlist items (the playlist being user's uploads).
channels_response = youtube.channels().list(
id=DEFAULT_CHANNEL,
part="contentDetails"
).execute()
if len(channels_response["items"]) == 0:
channels_response = youtube.channels().list(
forUsername=DEFAULT_CHANNEL,
part="contentDetails"
).execute()
for channel in channels_response["items"]:
# From the API response, extract the playlist ID that identifies the list
# of videos uploaded to the authenticated user's channel.
uploads_list_id = channel["contentDetails"]["relatedPlaylists"]["uploads"]
print "Videos in list %s" % uploads_list_id
# Retrieve the list of videos uploaded to the authenticated user's channel.
playlistitems_list_request = youtube.playlistItems().list(
playlistId=uploads_list_id,
part="snippet",
maxResults=50
)
while playlistitems_list_request:
playlistitems_list_response = playlistitems_list_request.execute()
# Print information about each video.
for playlist_item in playlistitems_list_response["items"]:
title = playlist_item["snippet"]["title"]
video_id = playlist_item["snippet"]["resourceId"]["videoId"]
print video_id
videos.append(video_id)
titles.append(title)
playlistitems_list_request = youtube.playlistItems().list_next(
playlistitems_list_request, playlistitems_list_response)
However, for some reason, I always end up getting no more than 5,000 videos.
I checked the API documentation, but there is no reference of limit of such sort. The code I'm using is from the official documentation too (slightly modified) and should work.
Does anyone know how could I get ALL the videos from Machinima?
EDIT: Update on October 20th. The limit seems to have disappeared.
I've been trying to get the view count on videos that I query through the following method:
function search() {
var request = gapi.client.youtube.search.list({
part: 'snippet',
channelId: 'IRRELEVANT',
order: 'date',
maxResults: '25'
});
request.execute(function(response){
YoutubeResponse(response);
});
While the documentation tells me that there's a statistics portion to every video, after the snippet I have __proto__ which I guess means there was an error somewhere? or did the API change? Essentially I need the view count of those 25 most recent videos...
I tried changing part: 'snippet' to part: 'statistics' but got back a code: -32602...
Thanks for the help,
Cheers!
EDIT: Apparently the search.list doesn't have the "statistics" but rather I need to search every video individually... The thing is, when using googles "Try It" feature (https://developers.google.com/youtube/v3/docs/videos/list#try-it) when you ask for the statistics in the "Fields" part at the bottom, it doesn't do anything... So I am VERY confused as to how the heck can I get the view counts & length of all 25 videos (if individually or all at once - preferably-)
The link you gave https://developers.google.com/youtube/v3/docs/videos/list#try-it is working for me.
To get duration and viewCount: Fill in for part: contentDetails,statistics and for id: a comma-separated-list of video-id's like: TruIq5IxuiU,-VoFbH8jTzE,RPNDXrAvAMg,gmQmYc9-zcg
This will create a request as:
GET https://www.googleapis.com/youtube/v3/videos?part=contentDetails,statistics&id=TruIq5IxuiU,-VoFbH8jTzE,RPNDXrAvAMg,gmQmYc9-zcg&key={YOUR_API_KEY}
Agree with the answer provided by #Als.
But I found a code snippet which might be more convenient for some of you:
function youtube_view_count_shortcode($params)
{
$videoID = $params['id']; // view id here
$json = file_get_contents("https://www.googleapis.com/youtube/v3/videos?
part=statistics&id=" . $videoID . "&key=xxxxxxxxxxxxxxxxxxxxxxxx");
$jsonData = json_decode($json);
$views = $jsonData->items[0]->statistics->viewCount;
return number_format($views);
}
Replace the key value with the google api key for youtube data API and the video id with the youtube video id and Voila you get the total number of views for the youtube video.
Source: https://www.codementor.io/rajharajesuwari/how-to-get-youtube-views-count-aftojpxhj
i am trying with youtube api v3 php search...
first time i'm using this api for this i am beginner...
i have 3 question;
1) how can below search list showing pagination numbers? (per page 50 result)
2) how can video duration show in list? (3:20 min:second)
3) how can order viewCount
if ($_GET['q']) {
require_once 'src/Google_Client.php';
require_once 'src/contrib/Google_YoutubeService.php';
$DEVELOPER_KEY = 'my key';
$client = new Google_Client();
$client->setDeveloperKey($DEVELOPER_KEY);
$youtube = new Google_YoutubeService($client);
try {
$searchResponse = $youtube->search->listSearch('id,snippet', array(
'q' => $_GET['q'],
'maxResults' => 50,
'type' => "video",
));
foreach ($searchResponse['items'] as $searchResult) {
$videos .= '<li style="clear:left"><img src="'.$searchResult['snippet']['thumbnails']['default']['url'].'" style="float:left; margin-right:18px" alt="" /><span style="float:left">'.$searchResult['snippet']['title'].'<br />'.$searchResult['id']['videoId'].'<br />'.$searchResult['snippet']['publishedAt'].'<br />'.$item['contentDetails']['duration'].'</span></li>';
}
$htmlBody .= <<<END
<ul>$videos</ul>
END;
} catch (Google_ServiceException $e) {
$htmlBody .= sprintf('<p>A service error occurred: <code>%s</code></p>',
htmlspecialchars($e->getMessage()));
} catch (Google_Exception $e) {
$htmlBody .= sprintf('<p>An client error occurred: <code>%s</code></p>',
htmlspecialchars($e->getMessage()));
}
}
1) how can below search list showing pagination numbers? (per page 50 result)
You need to write your own cacheing logic to implement this feature because with every result you get two tokens "NextPageToken" and "PreviousPageToken" and subsequent query must contain that token number to get next page or previous page token like below.
So whenever results are not available at cache then you should send either nextpagetoken or previous page token.
https://www.googleapis.com/youtube/v3/search?key=API_KEY&part=snippet&q=japan&maxResults=10&order=date&pageToken=NEXT_or_PREVIOUS_PAGE_TOKEN
In particular your case where you need 50 pages per page and you are showing 3 pagination like (1,2,NEXT) then you need to fetch results two times. Both the results you will keep in cache so for page 1 and 2 results will be retrieved from cache. For next you make it sure that you are making query google again by sending nextPageToken.
Thus to show pagination 1-n and every page 50 results then you need to make n-1 queries to google api. But if you are showing 10 results per page then you cane make single query of 50 results using which you can show first 5 pages (1-5) with the help of retrieved results and at next you should again send next page token like above.
NOTE- Google youtube api provide 50 results max.
2) how can video duration show in list? (3:20 min:second)
Youtube API v3 do not return video duration at simple first search response. To get video duration we need to make one extra call to youtube api like below.
https://www.googleapis.com/youtube/v3/videos?id=VIDEO_ID1%2CVIDEO_ID2&part=contentDetails&key=API_KEY (max 50 IDs)
This issue is highlighted in "http://code.google.com/p/gdata-issues/issues/detail?id=4294".I posted my answer here too.
Hence if we want to display video duration then we need to make two calls every time.
3) how can order viewCount
Trigger below query it will provide results ordered by view count.
https://www.googleapis.com/youtube/v3/search?key=KEY&part=snippet&q=japan&maxResults=5&order=viewCount
For detail please refer this - https://developers.google.com/youtube/v3/docs/search/list#order
The youTube API V3 is somehow complicated compare to API V2.
To the question above, my approach is not for search result rather is to retrieve user uploaded videos. I believe this can be useful
References
The way you create pagination in v3 is not the same as in v2 where you can make your call simply like
$youtube = "http://gdata.youtube.com/feeds/api/users/Qtube247/uploads?v=2&alt=jsonc&start-index=1&max-results=50";
In v3 you need to make two or three calls the first one will be to get the channel detail and second call will be to retrieve playlist from where we will get the channel playlist Id and finally retrieve individual video data.
I am using Php CURL
$youtube = “https://www.googleapis.com/youtube/v3/channels?part=snippet%2CcontentDetails%2Cstatistics&id=yourChannelIdgoeshere&key=yourApiKey”;
Here we retrieve user playlist ID
$result = json_decode($return, true);
$playlistId=$result['items'][0]['contentDetails']['relatedPlaylists']['uploads'];
we define pagetoken
$pageToken=’’;
Each time user click control button we retrieve pagetoken from session[] and feed the curl url, and in turn will produce nextpagetoken or prevpagetoken. Whatever you feed the url the Api know what set of list to populate.
if(isset($_REQUEST['ptk']) && $_REQUEST['ptk’]!==''){
$pageToken=$_REQUEST['ptk’];
}
Here we retrieve user playlist
$ playlistItems =”https://www.googleapis.com/youtube/v3/playlistItems?part=snippet&pageToken=”.$pageToken.”&maxResults=50&playlistId=$playlistId&key= yourApiKey”;
If user has more than maxResult, we should have nextPageToken, take for an example user has 200 uploaded videos,the first pagetoken may be CDIQAA and next pagetoken may be CGQQAA while previous may be CDIQAQ , something like that so is not a number.
Here we save the pagetoken
if(isset($result['nextPageToken'])) { $_SESSION[nextToken]=$result['nextPageToken'];
}
if(isset($result['prevPageToken'])) { $_SESSION[prevToken]=$result['prevPageToken'];
}
we can then create our control button <>
$next=$_SESSION[nextToken];
$prev=$_SESSION[prevToken];
The control button here
<a href=”?ptk=<?php echo $prev?>” ><<prev</a>
<a href=”?ptk=<?php echo $next?>” >next>></a>
From here when user click link it set either next or prev page in session variable (go to up to see how this work)
To get video duration we use same Php curl
$videoDetails="https://www.googleapis.com/youtube/v3/videos?part=id,snippet,contentDetails,statistics,status&id=videoIdHere&key=yourApiKey";
$videoData = json_decode($return, true);
$duration = $videoData['items'][0]['contentDetails']['duration'];
$viewCount = $videoData['items'][0]['statistics']['viewCount'];
you may get something like this ('PT2H34M25S')
I have answer a question Here which show you how to convert the duration data
See Working Demo Here