I have an abstract question: How can I get comments (Not count of comments!) from Youtube via API fast? I mean in API side - I have a powerful server and fine code. But via API it's really painful long process.
I see only one way. There are steps:
Make API request for get first page of comments;
Save comments from step 1 and get nextPageToken from response;
Make API request with pageToken option from step 2;
So... Loop steps 2 and 3 while have a nextPageToken in response.
If we have 10-20-50 pages of comments it's painful... But if we have 1k+ pages of comments it's a HELL! And we have a limit for API requests... So for 2-3 popular videos we can spend the limit.
It looks like I can't find something :) It can't be so hard... Isn't it?
How do you get a comments from Youtube?
Well, the max comment retrieval limit is 100 each time right? So i guess you will have to go that way anyway. Have you tried any non-API solution? I found this but not sure if working better (or at all)?
Use CommentThreads.list to get all the comments in a certain video. The required parameters you need to provide are part, videoId and fields.
And to test this, go to Youtube API explorer Commenthreads.list and fill in the parameter for videoId (added the others for you). Then, click Authorize and Execute. All the comments of your video will be listed under textOriginal.
It's now up to you to implement this.
Related
It seems that the YouTube API doesn't give updated results for mostPopular videos in my country since few days.
Example:
This request (https://www.googleapis.com/youtube/v3/videos?part=snippet,contentDetails&chart=mostpopular®ionCode=FR) doesn't give me the same videos results than the ones displayed directly on YouTube for the French most popular channel (https://www.youtube.com/channel/UCmzy72gDEpfXoFV9Xdtd0DQ). It seems that the results of this request is not updated since the 1th of february. Results was real time updated before this.
Does someone know if something is wrong with my API request, or if there are some issues with the YouTube API at this moment?
There is nothing wrong with your request. This is a known issue with the YouTube API reported here for Saudi Arabia (but also applicable to multiple regions), and another related issue here with regard to content from France.
Your best bet would be to follow up with the YouTube team on one of those defects, or potentially (and dangerously) scrape the YouTube site for the correct results.
Problem seems to be solved since 13th of february (maybe someone from Google have seen my post..)
YouTube Channels and chart=mostPopular parameter data are separate data entities, aka you will get different results. They may be related but there is no guarantee you will get the same data. To get the data that you want you may need to query for channel itself and its videos.
I got this information from the thread #Jal linked, there was an update by matthewc...#google.com a few days ago:
The most popular channel for Saudia
Arabia and
the mostPopular chart parameter in the video.list
call
are separate and distinct entities. If you'd like to get the content
of the most popular channel for Saudia
Arabia
please use the Data API video.list call to list the videos with the
channel ID (in this case "UCWY-_j1MCth6yf24m58Bh_Q") by setting the
items/snippet/channelId parameter.
My current concern right now is that there is supposedly a way to get video information from the videos.list endpoint using a channelId, which does not seem the case in the API Explorer. I will update my answer once I figure out what this person meant exactly.
I'm writing a script that does stuff with the youtube advertisment videos that people have to watch before the actual videos. (These ad-vids at the start are just simple youtube videos from the brand's channel)
I've searched the whole source code and scripts, but I can't find the video id of those ads anywhere. It must be somewhere, but it seems to be hidden well.
Anybody got an idea where to look?
I did some research on your case.
The video id of the ad video is definitively not part of the initial source code, as you already figured out. Youtube makes an ajax-Request to the http://googleads.g.doubleclick.net/ API to get information about related ad videos.
If you take a look into the source code, you can see a lot of javascript that is related to the google ads part. By looking into the code, you can find URL routes to the API. See screenshot, it is just an excerpt:
But unfortunately you cannot simply copy the url and make a remote-call to it. By doing so, you get a 400 Bad Request response.
As i figured out, there are missing params, which are dynamically added by youtube's javascript.
If you compare the request, that is actually made by youtube, you can see that there are more params sent:
compared to the request, that is directly copied from source code:
The result of the working request looks like this:
I tried several ways to make the invalid API request work, but have not found a way. Debugging the javascript is not that easy, because its obfuscated and minified. But additionally the variables are scoped within the function. So anyways, you would not have access to them.
If you make a javascript breakpoint right before the XHR request, you can see the actual API request:
But it is within the Local scope, without access to it.
Later on, there is even a second request to the youtube data API to fetch information about the ad video
In my opinion, there are only two ways to get the video, both require javascript to run.
Look into the source code AFTER Javascript has run. Then you can get the id directly from your markup:
Hook onto the Ajax Requests and grab the data directly from your network traffic.
I've been using the old url api(v1) to get the count of a given url, lately I needed to get also the re-tweets and started searching about that.
this is the exact url I'm using right now:
http://urls.api.twitter.com/1/urls/count.json?url=http://google.com
As I viewed with some reading the v1 api is deprecated but at least it's still working.
I found some questions on the dev page of twitter:
https://dev.twitter.com/discussions/12643
those are a little old questions and have no specific solving to the problem. I mean, the most near solution was using the search api(search/tweets) which could be good but not a exactly replacement for the urls/count method.
Please note that Twitter's search service and, by extension, the
Search API is not meant to be an exhaustive source of Tweets. Not all
Tweets will be indexed or made available via the search interface.
also it has a limit for 100 results at maximum per 'page', even it throws the link to get the next set of objects, thats good but when the search reaches 1 million of results I'll need to get page over page to now how much tweets I got and having to do to much request to the api...
I sought some question over the dev page on twitter suggested using the stream api, I've tried using (statuses/filter) but that don't work very well given a URL as track param(which they said that is the keyword to track).
So, anyone who's been using the old urls/count has found a reliable alternative with the new apiv1.1, especiffically to get the tweets and re-tweets for a given url ?
The official suggestion by Twitter staff is that either the search/tweets endpoint (having just the last 7 days data) or the Streaming API be used (handling yourself the counters, making everything just too complicated for a d*mn counter).
As an extra warning, the old endpoint (http://urls.api.twitter.com/1/urls/count.json?url=YOUR_URL) will stop working on November 20th, and according to this blog post from Twitter there are no plans to replace it with anything in the short term and they are even removing the count from their own buttons.
I am using YouTube API in one of my projects. In some cases I find that the "gd$rating" is not present in the response I receive . What can be the possible reasons for this ? Is there a work around to get the same ?
This sounds like a classic example of "don't always trust the API return values to be wholly complete or accurate". You will most likely have to anticipate this sort of thing happening in your code. Even if it is defined in the API.
For example: https://developers.google.com/youtube/2.0/reference#youtube_data_api_tag_gd:rating
From the docs: gd$rating This element is only returned if the video has been rated.
What I'm trying to do is pull some search results, and sort them by users.
Right now I'm using
$to->OAuthRequest('http://search.twitter.com/search.json', array('q' => 'search-term'), 'GET');
to get the results, but I can't actually get information about the user like follower_count, etc. so I though I'd take each result's user_id, and run it through http://twitter.com/users/show.xml to get the rest of the information.
The problem is, if I get 50 results that way, that's 50 API calls, which is 1/3 of the hourly limit...
Is there a better way to do this?
Update:
I have requested whitelisting, and the guys at Twitter told me I should instead use the new Streaming API.
This kinda sucks since it's in Alpha testing, and the documentation is unclear, but the concept looks cool, and the limits are different than the regular API. Gonna dive into it later.
Only reliable user information you can get from twitter search results are username (from_user) and profile image url. If you want more information, you will have to call the users/show method for each user. There is no workaround or better way(that I'm aware of).
You will have to Cache user information and request whitelisting.