In YouTube's Data API, data are returned in fields, and fields are grouped in parts. In the YouTube documentation, YouTube Data API Overview, the section, How to use the part parameter, gives a list of the parts available for videos:
snippet, contentDetails, fileDetails, player, processingDetails, recordingDetails, statistics, status, suggestions, topicDetails
There are two things I have not found in the documentation:
The list of parts available for channels and playlists.
The list of fields available in each part for videos, channels, and playlists.
Where can I get that information?
Also, some fields are available to the public, while others are only available to the owner of the subject resource. So in the list of fields, I need to know which I can query and which are restricted.
Found the answer:
Channels
Parts: docs/channels/list#parameters
Fields: docs/channels#properties
Restrictions: docs/channels/list#auth
Playlists
Parts: docs/playlists/list#parameters
Fields: docs/playlists#properties
Restrictions: If there are any, they don't seem to be documented.
Playlist items
Parts: docs/playlistItems/list#parameters
Fields: docs/playlistItems#properties
Restrictions: If there are any, they don't seem to be documented.
Videos
Parts: docs/videos/list#parameters (The list at that link gives three additional parts that are not in the list at How to use the part parameter: id, liveStreamingDetails, and localizations.)
Fields: docs/videos#properties
Restrictions: Stated in the descriptions of the fileDetails, processingDetails, and suggestions fields.
===================================
Fields that are missing from my previous answer are
contentDetails
regionRestriction
hasCustomThumbnail
status
failureReason
rejectionReason
publishAt (But this should never appear in a public listing because it only applies to videos with privacyStatus = 'private`.)
selfDeclaredMadeForKids (But only returned if authorized.)
statistics
dislikeCount (But only returned if authorized.)
player
embedHeight
embedWidth
recordingDetails
location
recordingDate
liveStreamingDetails
6 fields
localizations
key (language)
I have used a method to get a partial list of the public fields available for videos. This is a hack and I hope someone will post a better answer, preferably a link to the answer in official YouTube documentation.
Until we get that answer, here is the hack I used:
First, I took the list of the ten parts given for videos and submitted it in an API call for three arbitrary videos:
https://www.googleapis.com/youtube/v3/videos?part=snippet,contentDetails,fileDetails,player,processingDetails,recordingDetails,statistics,status,suggestions,topicDetails&id=XqZsoesa55w,kJQP7kiw5Fk,F4tHL8reNCs&key=<key>
The result was this error message:
The request is not properly authorized to access video file or
processing information. Note that the fileDetails,
processingDetails, and suggestions parts are
only available to that video's owner.
That is information. It tells me that those three parts are not publicly available, so I can't query them on videos I don't own.
So next I submitted the same request with those three parts removed:
https://www.googleapis.com/youtube/v3/videos?part=snippet,contentDetails,player,recordingDetails,statistics,status,topicDetails&id=XqZsoesa55w,kJQP7kiw5Fk,F4tHL8reNCs&key=<key>
I then took the response from that and parsed the JSON to extract the labels at the third and fourth level of nesting. These are mostly the part and field names. The first three rows of the following table are fields; the rest are parts with their constituent fields.
Field or part
Field
kind
etag
id
snippet
publishedAt
channelId
title
description
thumbnails
channelTitle
tags
categoryId
liveBroadcastContent
defaultLanguage
localized
defaultAudioLanguage
contentDetails
duration
dimension
definition
caption
licensedContent
contentRating
projection
status
uploadStatus
privacyStatus
license
embeddable
publicStatsViewable
madeForKids
statistics
viewCount
likeCount
favoriteCount
commentCount
player
embedHtml
topicDetails
topicCategories
recordingDetails
Notice that part recordingDetails has no fields listed. That doesn't mean it has no fields. It means that none of the videos for the IDs submitted have recordingDetails in their data. That is why I said at the beginning that what I've got is a partial list. It is only based on the data retrieved for those three videos. Another example of this issue is that only two of the three videos submitted have a commentCount in their data. If none of them had data for that field, then I would not know about the existence of that field from this exercise.
So there's my partial list of public fields for videos. Can someone give us a better answer pointing to documentation of all the fields for videos, channels, and playlists?
Related
I have tested YouTube's PlaylistItems.list API to fetch a channel's default playlist.
In the response payload, a video -- that was published on 2020-10-14T20:22:24Z -- is in the first object, and then the next video -- that was published on 2020-10-21T17:54:05Z -- is in the second object.
In this answer it is mentioned that:
The publishedAt getting returned from a Playlist query is the date the
video was added to the playlist, rather than the date the video was
published on YouTube
What might be the reason for my case?
Is there any specific sorting order maintained for this API?
The reason you see the ordering you've exemplified (though you did not mentioned your playlist ID such that others to verify your claim) is two fold.
1. A video's publication time may well be different than its upload time
First thing is the following: according to the official docs of the Video resource's publishedAt property (the emphasis below is mine):
snippet.publishedAt (datetime)
The date and time that the video was published. Note that this time might be different than the time that the video was uploaded. For example, if a video is uploaded as a private video and then made public at a later time, this property will specify the time that the video was made public.
There are a couple of special cases:
If a video is uploaded as a private video and the video metadata is retrieved by the channel owner, then the property value specifies the date and time that the video was uploaded.
If a video is uploaded as an unlisted video, the property value also specifies the date and time that the video was uploaded. In this case, anyone who knows the video's unique video ID can retrieve the video metadata.
The value is specified in ISO 8601 format.
Therefore, in cases, it may very well be that a given video has different upload time and time of publication.
Furthermore, the official docs of PlaylistItems resource says the following w.r.t. two related properties:
snippet.publishedAt (datetime)
The date and time that the item was added to the playlist. The value is specified in ISO 8601 format.
contentDetails.videoPublishedAt (datetime)
The date and time that the video was published to YouTube. The value is specified in ISO 8601 format.
From the spec of snippet.publishedAt, it follows that, in the case of a playlist being the uploads playlist of a given channel (that is a playlist of which ID is of form UU...; note that, usually, a channel ID and its corresponding uploads playlist ID are related by s/^UC([0-9a-zA-Z_-]{22})$/UU\1/, though not documented officially), the value of snippet.publishedAt is the upload date of that video.
The second spec, indicates that the value of contentDetails.videoPublishedAt is that of the corresponding Video resource's snippet.publishedAt, the time of publication of that video.
2. For the uploads playlist of a channel, the items returned by PlaylistItems.list API endpoint are (have to be) ordered in reverse chronological order by contentDetails.videoPublishedAt
Indeed, the ordering condition seems to be true. This feature is not documented officially, but, in my experience, the statement above holds true for every result set obtained from PlaylistItems.list endpoint.
I could make here the following argument justifying the necessity that the items resulted upon the invocation of PlaylistItems.list endpoint be ordered in reverse chronological order (newest first) by contentDetails.videoPublishedAt:
This argument is of a pragmatic kind: if the result set of PlaylistItems.list is not ordered as mentioned, then this endpoint becomes useless.
This is so, since, in this case, for one to obtain the most recently published video would have to fetch locally all the uploaded items (the number of which is limited by design to 20000), for to then scan that result set for the most recent one. Being compelled to fetch all uploaded items only for to obtain the newest one is pragmatically a nonsense. If the number of uploads of a given channel exceeds the limit of 20000, then the most recent video could possibly fall outside this boundary; thus, for such kind of channels, the most recently published video could not be obtained from the API at all.
Therefore, by way of contradiction, the result set has to be ordered in reverse chronological order by contentDetails.videoPublishedAt.
i would like to crawl Youtube for videos of a specific language that contains subtitles/closed-captions(CC).
For example,
I want to crawl for 200 random English videos with English subtitles/(CC).
I want to crawl for 300 random Chinese videos with Chinese subtitles/(CC).
I want to crawl for 550 random Malay videos with Malay subtitles/(CC).
There's an api here that helps to extract transcripts, but the main bottleneck right now is that i have to go youtube to search for these videos and watch one by one to find out if they are indeed in the correct language, and if they really contain subtitles/CC.
An option is:
Use YouTube Data API - search request for search videos that contains subtitles; for that, use videoCaption parameter with value: closedCaption.
You might need use another parameters for reduce the search terms to specific topics or get certain desired results; for example, for the q parameter, use a search term that retrieves the desired results; also all parameters like: videoDuration, type = video, relevanceLanguage.
Once you got such results, copy/paste the videoId you got from the results of the request and use your web-crawler for get more videos and the related ones.
For anyone still struggling with this, and as per the YouTube Data API for videoCaption to work, you need to also set the type parameter's value to video:
If you specify a value for this parameter, you must also set the type
parameter's value to video.
I'm trying to use Youtube Data API for a project, the goal is seeking for a keyword in a channel and show the response to the user. I used the Search for "snippet" part and launched it querying for a specific keyword and specifing the channel id but the response didn't include all the videos that I was expecting. For instance, let's say that the channel has 10 videos with "c" charachter in the title, setting the q field with "c" value will return only one video.
On the other side, if I search for a whole word it returns some videos with that word on title and some other videos that doesn't have it, neither in description. The order criteria in this case seems to be ok (from the strongest match to the most weak), but I don't know if all of this is working fine.
Is this a normal behaviour or am I doing something wrong?
Setting an order on the search, the issue seems to be solved.
Is there any way to have updated lists of top 100 videos of youtube by genre and/or by country!
Any kind of resources like json files or xml.
You'll want to use the chart parameter of the videos.list endpoint. Set the chart to mostPopular, and then include a regionCode parameter and videoCategoryId parameter for further narrowing down. For example,
GET https://www.googleapis.com/youtube/v3/videos?part=snippet&chart=mostPopular®ionCode=UA&key={YOUR_API_KEY}
Will retrieve the 5 most popular videos in the Ukraine.
GET https://www.googleapis.com/youtube/v3/videos?part=snippet&chart=mostPopular&maxResults=25®ionCode=DE&videoCategoryId=1&key={YOUR_API_KEY}
Will retrieve the 25 most popular videos in Germany that relate to Film/Animation. And so on.
Note that if you don't include a videoCategoryId parameter, it will return results from all categories. If you don't include a regionCode, it returns the most popular videos across all regions. You can only set videoCategoryId to a value that's valid in the region you're searching in (you can use the videoCategories.list endpoint to find valid categories for regions, languages, etc.)
I am trying to search for "Food+Show" from two youtube channels. ABCNetwork and FoxBroadcasting. The query I gave is
http://gdata.youtube.com/feeds/api/videos?v=2&alt=jsonc&q=Food+Show&max-results=3&authors=ABCNetwork,FoxBroadcasting&prettyprint=true
The first result I got was id UKfLsIgJB1g where uploader is wafelsanddinges and not ABC or Fox. Please tell me why my query is not retuning correct result.
The parameter for the v2 data API is "author," not "authors." Unfortunately, fixing that won't solve the problem, as the retrieval of videos from a particular channel can only accept one author at a time. This is also true for v3 of the API.
The reason behind this is that the comma is treated as a concatenator, looking for a video that was published on FoxBroadcasting AND ABCNetwork (the use case for having multiple authors in that parameter is if you are retrieving activity feeds, in which case you want both feeds so having the comma serve as an AND is correct).
So for now, the only solution is two separate calls.