Most efficient way to get all comments for youTube channel - youtube-api

I am trying to write an app that will get all of the comments for a youTube channel.
I have spent a while looking at the docs and trying out examples but I am still unclear of the best way of doing this.
The comment Threads List call seems to do what I need but it doesn't return any of the actual text of the reply. It only contains details about the comment. It tells me who made it, their youTube channel details and so on but to actually get the comment text I then need to use the comments list call to get the content. I will also have to make 2 calls to that as one only gives me the main comment, to get the replies I have to make another call with the parentId set.
So, the calls I need to make:
Get list of channel Ids
For each Channel get a list of videos
For each video load comment threads
For each comment thread load the comment
for each comment thread with replies load the replies
This seems like a needlessly complicated and resource intensive way of doing this. Why don't they just include the comment content in the comment Threads call?
Am I missing something??

Related

How to get comments for Youtube video via API

I have an abstract question: How can I get comments (Not count of comments!) from Youtube via API fast? I mean in API side - I have a powerful server and fine code. But via API it's really painful long process.
I see only one way. There are steps:
Make API request for get first page of comments;
Save comments from step 1 and get nextPageToken from response;
Make API request with pageToken option from step 2;
So... Loop steps 2 and 3 while have a nextPageToken in response.
If we have 10-20-50 pages of comments it's painful... But if we have 1k+ pages of comments it's a HELL! And we have a limit for API requests... So for 2-3 popular videos we can spend the limit.
It looks like I can't find something :) It can't be so hard... Isn't it?
How do you get a comments from Youtube?
Well, the max comment retrieval limit is 100 each time right? So i guess you will have to go that way anyway. Have you tried any non-API solution? I found this but not sure if working better (or at all)?
Use CommentThreads.list to get all the comments in a certain video. The required parameters you need to provide are part, videoId and fields.
And to test this, go to Youtube API explorer Commenthreads.list and fill in the parameter for videoId (added the others for you). Then, click Authorize and Execute. All the comments of your video will be listed under textOriginal.
It's now up to you to implement this.

Youtube Data API v3: commentThread call doesn't give replies for some comment threads

I have a problem with commentThread API call. Here is one specific case: comment id: z13ocxipdz3hwxqqe04cgbuadtmnhhmybyc0k
https://www.googleapis.com/youtube/v3/commentThreads?id=z13ocxipdz3hwxqqe04cgbuadtmnhhmybyc0k&part=snippet%2C+replies&key={YOUR_DEVELOPER_KEY}&alt=json&order=time
There are 44 replies for now.
But if you use video_id (not comment id like we did above. Link: https://www.googleapis.com/youtube/v3/commentThreads?videoId=Ui-ulCWmpOU&maxResults=100&pageToken=ChYQp7fipbfsxgIYkNGqiobjxQIgACgDEhQIABCQ0aqKhuPFAhiMqqKive-5AhgCIKwC&part=snippet%2C+replies&key={YOUR_DEVELOPER_KEY}&alt=json&order=time) and come across the comment (you may have to use pageToken to iterate over pages) you will see this:
(Screenshot: https://www.dropbox.com/s/d4bf9tk51eaw7dr/Screenshot%202015-07-21%2021.08.31.png?dl=0)
As you noticed; there is 39 replies (comments) which is not true. And even if we have replies; there is no replies section. Am I missing something?
Two questions: Why number of replies don't match? Why that commentThread doesn't have replies even it has 39 replies?
Notes: As part I am using 'snippet, replies'. I don't get any API error (like throttle or limit violation)
From the commentThreads documentation:
The commentThread resource does not necessarily contain all replies to
a comment, and you need to use the comments.list method if you want to
retrieve all replies for a particular comment.
Use the comments.list call instead and specify the commentThread's ID for the parentId.
When I do the following call:
https://www.googleapis.com/youtube/v3/comments?part=snippet,id&parentId=z13ocxipdz3hwxqqe04cgbuadtmnhhmybyc0k&key=[API_KEY_HERE]&order=time&maxResults=50
I get all (44) of the responses to the comment thread you mentioned.
To address your questions:
Why number of replies don't match?
If I had to guess, I'd say it might be due to how the API differentiates between comments and comment threads (where it considers replies to other replies in a thread as opposed to them being associated with the video itself). Since the documentation says that the commentThread resource might not contain all replies to a comment, I think this is still working as intended. At any rate, in the future it's generally better to just go with the higher number of replies reported, since it's more likely that the API is missing some results than reporting extra results. Also, remember that you should be using comments.list if you want all of the replies to a certain comment thread.
Why that commentThread doesn't have replies even it has 39 replies?
To be honest, I'm not sure if it's a YouTube thing or a G+ thing. When I looked at some of the other comment threads that had replies, I was able to see them via the API call when video ID is specified. However, as Daniel mentioned, you can't actually see the replies for the thread you specified on the video itself (but you can see them when you go to that user's G+ page). It's possible that user did something to their G+ settings regarding comments on their shared posts, which is why they aren't showing up on the video itself and thus not showing up in the commentThreads list request. It's hard to say unless you talked to someone who's familiar with G+/YouTube integration.
While I wasn't able to answer your questions definitely, I was able to find some useful info that may give you some clarity and direction.
With regard to the specific case outlined in your question: the user "authorDisplayName": "Tammy L." commented on the video "videoId": "Ui-ulCWmpOU" on 11/16/2013, and received replies between 11/16/2013 and 11/18/2013. According to Tammy L.'s record on Google+, there are 44 replies to her initial comment from this date:
Although, according to the record on YouTube, it appears that there are 42 replies to her comment, which doesn't match either value returned from your API calls. Even more bizarre, Google won't display the comments when I click on "View all 42 replies":
It seems that the commentId call provides the correct number of replies made to Tammy L.'s initial comment, which is consistent with the number of replies shown on her Google+ account. Perhaps the issues with the videoId call (including the fact that this call doesn't provide replies) are attributable to the changes Google implemented around the time that Tammy L. made her initial comment (November 2013), although I'm not totally sure.

What is the easiest way to get the latest comment on any of my videos, using v2 of the YouTube API?

As pointed out in the question How do I fetch comments in version 3 of the YouTube API?, there is currently no way of fetching video comments using version 3 of the YouTube API. Now I'm trying to figure it out using version 2 instead.
What I want is the latest comment on any of my uploaded videos, in other words the latest comment in the "aggregated" comment feed of all my videos.
It seems like the only way to do this is too fetch all videos, and then make a call for each of them to get the comments. With a few hundred uploaded videos, this becomes very expensive in terms of number of API calls and total time for completion.
Is there a simpler and/or better way?
There's no way to be "notified" via the API when a video gets a new comment, so you're going to have to do some polling. The comments feed for a given video id, e.g. https://gdata.youtube.com/feeds/api/videos/fhWaJi1Hsfo/comments?v=2 for video id fhWaJi1Hsfo, is sorted in reverse-chronological order by default, so the last comment added should always be first in the list of entries.
Making a request for the comments feed of several hundred videos, even if you do that a couple of times a day, doesn't sound like an unreasonable amount of traffic. You should follow the best practices outlined in this blog post if you do run into any quota issues, though.

How to get a list of all retweeters in Twitter?

I have seen numerous companies doing like Twitter lotteries where users got to retweet their tweet and then one of retweeters will get the prize or whatever.
I was wondering now how do they get the list of all retweeters or pick the winner?
I checked Twitter API and found only this thing: GET statuses/retweets/:id but it returns maximum of 100 retweeters. Is that the only way?
It looks likes there's a couple services out there doing almost exactly this. A quick google pulls up http://onekontest.com/ and there's a few other Twitter contest services, but they all seem to be different levels of broken since they haven't kept up with changes to the API.
As far as the Twitter API itself is concerned, if you were expecting more than 100 responses, I think using GET statuses/mentions makes the most sense. That API call returns any mentions of a user, and you can pass the flag include_rts to include any retweets of your tweets. Then, if you wanted to list RTs of a specific tweet, you could check the in_reply_to_status_id field in the returned data to see if it matches the original tweet ID. This API call only returns the last 800 status, 200 at a time, so if you expect a bunch of data, you would need to poll the API repeatedly over time to get all the tweets. I imagine services like favstar are doing exactly this, just on a larger scale.
If you're actually looking for code to do something like this, I wrote a sinatra app called twitter-rss-digest which handles querying Twitter over time to track different sorts of queries. It's pretty rough, and doesn't quite handle this specifically, but it might point you in the right direction if you want to code something.
The Twitter API has an endpoint that returns up to 100 retweeter IDs for a given tweet.
Note that for historical reasons, that endpoint only returns up to 100 results and the documentation about the cursor is useless. Twitter refused to fix the situation.
2013 note
That said, the first thread on the Developers site that surfaced in a quick google has #episod, a Twitter employee saying:
You can't likely get to all of them. The data is spread out. Your best bet is to use the REST API to determine many of the users who performed the retweet, but the data will still be constrained.
It's easiest to track retweets as they happen rather than try to find them from the past. Use the Streaming API for that.
I like muffinista's method, but I think if you want a 100% complete list of retweets, simply enable the retweet email notifications and write a script that polls the email box for those matching the subject "retweeted one of your Tweets!" and put the data into a table. Do this right from the start.
The site https://twren.ch/ enlists all the retweeters for a given tweet (note that it only enlists retweeters who are direct followers of the source tweeter.) Nevertheless its probably the only public source available.

YouTube API: videoEntry context in playlist

Is it possible to find the context of a video in respect to any playlist that it is included in? I would like to be able to check if the videoEntry is included in a playlist and, if so, figure out the next video in the playlist.
Flickr has a lot of great photo context methods, I hope YouTube does too, I just can't find anything on it.
Well, obviously you want to know the VideoId of the video in question first, right? So say it's "4eUibFQKJqI" and it's in a playlist whose Id is "6C0464B5CC81A607" (You can easily get those Ids from your 'My Account' section in YouTube).
If I get the feed for that playlist, then the resulting xml file gives you all the information you need to check if that Video is present, and the order that you have set in 'My Account' is preserved.
So, you have two ways of analysing the results that YouTube sends you. One is to make a request to that playlist address and parsing the results using Linq-to-Sql, if you use Asp.Net, for example. Whatever language you use, it will have Xml-parsing capabilities, so it's just a case of iterating through the entries in the feed.
If you use a library for your language, it may well have a specific method to determine whether a given video is in a given playlist, but if it doesn't, it's pretty easy to query the raw xml file and get that information, as described above.
Hope that helps - it's hard to tell from your question what level you're at with this stuff... Anyway, it's all at the Developers' Guide.
http://gdata.youtube.com/feeds/api/playlists/PLAYLIST_ID?v=2
from http://code.google.com/apis/youtube/2.0/reference.html

Resources