I am curious to know what would be the most efficient way to walk the youtube website. My goal is to eventually index all videos on youtube (hypothetically) and the only way I can think of is to go channel by channel indexing all of the videos. I am not very familiar with the v3 APi, so if there is a better way to accomplish this, please let me know. This gives rise to a few problems I can think of:
Where to begin? Channels and videos are accessed using random string IDs, so if I simply start with IDs beginning with 'A' I am going to run into a lot of null values. Not sure how IDs are assigned, but this also may keep the indexing in a certain segment/section of video types if it is based on the ID alphanumerics.
I am hoping to move methodically through the youtube directory, trying to avoid accidently indexing the same channel/video.
Should I somehow seperate the videos into groups and request them based on other parameters? A grouped scheme may be easier to work with, update, etc.
I won't know if the video has anything I am interested in indexing before accessing it.
First you need to understand that there are way too many videos for you to do this without having access to the stack directly, which you do not have and will not get.
As to automate the selection of video's, you can try to use the video ID's.
They are 11 characters long, consisting of only "a-z A-Z _ and - " . So that would reduce (still is 54 to the power of 11) the indexing/scanning if a video exists. Then save that ID (with related info) and move on.
Not a perfect option, but best I can see with your options and requirements.
Related
I am looking for a way to find all of the 360° videos from Youtube using Youtube APIs for my research. Currently I haven't been able to find this feature from the Youtube APIs, or hack a work-around using available API functions.
Search results on Yuotube webpage can be filtered by featureto just show the 360° videos among the search results. Even replicating this functionality using available API methods would be a good starting point to my needs. Maybe this could be accomplished by using snippet's tags, but I haven't been able to make it work.
(This question is quite similar ;-) to :
Searching 360 degree videos using YouTube Data API?
. But that question remains unanswered after 8 moths, even though Google Youtube Data API support page states: "We support the YouTube Data API on Stack Overflow. Google engineers monitor and answer questions with the youtube-api, youtube-data-api, and youtube-v3-api tags." As I just came across this problem, I hope the question gets some attention.)
I have been struggling with this problem as well and have used a combination of approaches with limited results:
First, we use this method and add a publishedBefore filter for subsequent fresh searches (no pageToken) of the oldest result we have found. This enables us to get more than the 1000 results youtube apparently limits you to.
Second we found good lots of good results for the query #360Video, and we perform 2 sets of searches for that string: videoDimension: '2d' and videoDimension: '3d'. For all results, we query their contentDetails and make filter out those who's contentDetails.projection does not equal '360'. For good measure, we also do a '3d' dimension search without the query string.
Finally we combine all of the sets of results we retrieved. Unfortunately we do still seem to miss quite a few videos we spot test by hand. So far we have only been to query ~1000 videos. It seems like finding all 360 videos could require lots of manual work.
update: We have employed a manual step where we regularly scrape a list of user accounts, channels, and playlists with many 360 videos. This has yielded several hundred more videos at the least with, arguably, better quality. There might be a further step where we check all of the "recommended" videos for each of those videos and so on.
Well, I found in this thread that YouTube rolls out support for 360-degree live streams and spatial audio. And if you check the YouTube API documentation, there is no guide there that shows you how to use the 360 videos in YouTube.
So, YouTube 360 videos are not fully supported in YouTube API as of the moment. It is currently filed as a feature request that you can find here.
Hope it helps you.
I need to get a list of current live broadcasts in a given category from the YouTube Data API, ordered by number of viewers currently watching.
It looks like the liveStreamingDetails (which contains the concurrentViewers property) are only exposed on the individual video level. However, grabbing all the current streams, then making separate calls for each stream to get the number of viewers, then ordering that list is woefully inefficient and costly.
Feels like there must be a better way! Any ideas?
Currently there is no better way. Please file a feature request at https://code.google.com/p/gdata-issues/issues/list?q=label:APi-YouTube to keep track.
Suppose I have a video on YouTube that gets the URL https://www.youtube.com/watch?v=vWSyMuKkXXX (not a real video/ID, fwiw). If I delete that video, what are the chances that "vWSyMuKkXXX" will get reassigned to another video that somebody else puts up? 62^11 (is that right?) is a pretty large space from which to be assigning symbols, but YouTube must be doing some uniqueness test to avoid duplicates. The question, I guess, would then be whether they're including deleted IDs in that test (at least, given the way I'm guessing what they're doing internally).
This question is all about how much work I have to do to figure out whether the video corresponding to an ID exists and that it is the video that I think it is -- whether I can get away with using a simple call to http://www.youtube.com/oembed?... , or whether I need to get authentication and the APIs involved (which might still not resolve the question). Any thoughts? Thanks!
If YouTube were to ever reuse IDs, it would cause problems such as old links now pointing to new (possibly unlisted) videos. There is no advantage in reusing IDs, only problems including privacy problems. It would be an ugly bug.
To support unlisted videos, the IDs cannot be sequential. They must come from a large space of possible values.
how much work I have to do to figure out whether the video corresponding to an ID exists
You must send a query to YouTube.
and that it is the video that I think it is
How do you define "the video that I think it is"? By ID? By watching the transcoded video at your current resolution -- where the individual pixels might not match the uploaded pixels?
I query the YouTube v2 api for category=music videos with safeSearch=strict option set. Previous to 10/03/2013 many music videos were coming through on the safeSearch=strict option, but starting on 10/03/2013 it seems these restrictions became much stricter filtering out many results that it did not yesterday. Is this an error? I don't know where to reach out to YouTube on this. Two example queries:
https://gdata.youtube.com/feeds/api/videos?q=katy+perry+roar&v=2&safeSearch=strict
versus
https://gdata.youtube.com/feeds/api/videos?q=katy+perry+roar&v=2
return different results than it did previous to 10/03/2013. Yesterday the official video was the first result using safeSearch=strict and today it is not. The video is still the same identifier and the content did not change. This is happening for about 85% of content that my query strings have hit today. In my opinion, the content I'm querying is no where meeting their safeSearch thresholds. Can anyone shed some light or maybe point me to some resources I can read up on why this large change happened?
After struggling with getting Ytd to work for a couple of days I'm about to dive into Youtube Direct Lite which looks much friendlier to set up.
My first question is about the playlist size limit. Once a playlist is full (200 videos?) what would happen with further video submissions? Would the oldest be dropped or is it just impossible to add any more, effectively breaking the widget for that playlist?
I expect I would need to use multiple playlists and manually make new playlists and widgets if there's a lot of videos, but is there a best practice kinda way to do this for a large number of videoslso?
Also, would it be possible to automate the submission approvals programmatically if there's a lot of videos or is this beyond the scope of ytd-lite.
Thought it's better to ask these questions now before starting the process of setting this up for my site. Ytd-lite looks like a great project.
thanks.
from the Doc:
https://developers.google.com/youtube/2.0/developers_guide_protocol_playlists#Adding_a_playlist
Note: Playlists contain a maximum of 200 videos. As such, you will not be able to add a video to a playlist that already contains that many videos.
I dont'n try to force this situation but I expect an error.
I believe that to automate the submission approvals programmatically you can modify the source code of YouTube Direct Lite, and with a little logic in the server side of your app you can do what you want.