While processing youtube videos, I'm working with the youtube's API v3. The cycle goes like this:
1) Get video
2) Request API for video's topic Id
3) Find the tuple(s) on my local Freebase dump
4) Do things.
For efficiency reasons I'd like to avoid the online request on step (2), and instead do it offline. Is there any datasets/dumps I can get, that store information like:
Youtube video's ID -> video's topic ID ?
Thanks in advance,
Orestes.
YouTube doesn't provide data dumps of video topicDetails (or any other data server by the YouTube Data API as far as I know).
There is not a store I know. But there are few things to make it more efficient. Instead of calling videos->list per each video, make all video Ids into a comma separated string and do one call. Also specify filters like items/contentdetails/topicId, so response would only return topicIds, instead of all other information.
That way you can get all your information at once, with less bandwidth.
Related
Does anyone know how this website is able to get historical views data for YouTube playlists?
From my research, YouTube API doesn't even let you get the current playlist views (I might be wrong).
I am talking about the playlist views, not the views from videos that were added to the playlist (you can see the view count here for example, on the left side - on desktop).
How can we get access to this type of data?
One more time YouTube Data API v3 doesn't provide a basic feature.
I would recommend you to use my open-source YouTube operational API. Indeed by requesting the JSON document https://yt.lemnoslife.com/playlists?part=statistics&id=PLAYLIST_ID containing item["statistics"]["viewCount"] integer, your problem is solved.
You can request up to maxResults (currently 50) playlists separating playlist ids by a comma in id, example: https://yt.lemnoslife.com/playlists?part=statistics&id=PLAYLIST_ID_0,PLAYLIST_ID_1,...,PLAYLIST_ID_49
You can also request the title of the playlist by adding snippet to part, example: https://yt.lemnoslife.com/playlists?part=snippet,statistics&id=PLAYLIST_ID
we are using the youtube api to keep track of youtube videos. currently we "watch" around 30k videos. now we have the issue that we want to figure out which videos have been deleted. is there some way to get e.g. a list of all deleted vids of a channel or some other way?
sending a single request (and handling 404) for each video doesn't seem to be a good idea.
I guess you can not do it with only youtube api.
You can create your own database of all your videos and channels and periodically sync it with youtube to check what video status changed or not exists in list anymore.
Youtube API has method Videos.list, it returns list of all your videos.
Another tricky way: load every video url and check the size of content instead of parsing html.
If video exists then content size about 50kb, if not - 15kb. It seems youtube do not pass Content-Length header. That's why you need to load content to check size.
But I prefer the first way.
I'm looking to programmatically discover the YouTube Preferred channels (Examples: https://www.youtube.com/channel/UCiq_aAkP44mMpRx8VvQiF6w/channels?shelf_id=5204494150831773367&view=49 and https://www.youtube.com/channel/UCpXBkrx5YWAe8I9nbmLK_fA/channels?) but the channel pages with their AJAX lists is all I've found. Obviously it would be much easier to crawl an API or an RSS feed (or even a static HTML page!) and I've yet to find anything like that with each of the YouTube Preferred channel ids.
Am I missing something? Do these preferred channel lists really only exist in this one form?
Edit #1: I have tried to use the YouTube Data API to pull these "Preferred" channels' playlists via /channel/lists, but both efforts were a bust as the /channel/list only returned the single top-level channel and the playlists were empty.
Edit #2: I tried the solution from this: Youtube api get the channels from the Featured channels module and it did not work (despite this question being tagged as a duplicate of it).
You can use the channelSections.list endpoint with the Google Preferred channel id (UCiq_aAkP44mMpRx8VvQiF6w), the preferred channel ids are in the contentDetails separated in 1% and 5% groups
Example: https://developers.google.com/apis-explorer/#p/youtube/v3/youtube.channelSections.list?part=contentDetails&channelId=UCiq_aAkP44mMpRx8VvQiF6w&_h=1&
When using the YouTube api to query and retrieve videos, is there a way to limit results only official content? I want to retrieve the new music videos for certain artists but I want to make sure I am only receiving real videos and not UGC stuff. Can this be accomplished? Thank you.
There is no search parameter that ensures that you only get back "real" music videos. (And the line between a "real" video and user-generated content isn't always clear.)
I've explored most of this topic using the previous question How do I get a list of uploaded videos for a certain channel with the new YouTube Data API (V3)?, but I wanted some specifics that build upon the answer given.
Is it possible to use the channel's username without calling a search query to get the channel ID? I have an https request that can get a search list shown below, but I'd rather not waste API quotas when I know the exact channel username.
https://www.googleapis.com/youtube/v3/search?q={channel_username}&key={my_api_key}&part=snippet
My hack for this is just take a videoid from the channel and extract a uploader ID from there, but its not elegant and won't work if the upload is removed.
Also, once I've worked my way down to getting the uploads key, how can I get a json that shows all of the uploads and not just a list of 25? If I do have to recursively call pages of uploads, how can I do that?
Thanks.
I think that to get a channel, using the channel's username, the only option is to use the search, and the best strategy to minimize the use of quota is to use a cache, or save the channel ID.
For a full list of videos uploaded, as JSON, you can use contentDetails.relatedPlaylists.uploads (the ID of the playlist that contains the channel's uploaded videos) and retrieve the PlaylistItems, using pages with 50 items per page, and using the nextPageToken value to retrieve each page in the result set.
I've found the answer. The correct way to go about it is to query the site using the search, but directly state the type of list to respond with, in effect searching only channels. This is done by
https://www.googleapis.com/youtube/v3/search?q={search_term}&key={API_key}&type=channel&part=snippet
As you can see the type is channel. By searching with a search term of the exact channel username, you'll get a return list with the first item object as the channel, and you can store the channel ID and upload ID from there.
To update this answer, you need to recognise the "quota limits".
As such, to use the "search?q=[search]" will incur a 100 quota usage.
Better to use "channels?part=id&forUsername={username]" as is only a quota of 1.
Then you can use "channels?part=id,snippet ....... &id={channel_ID]" to grab all your data