I am using https://developers.google.com/youtube/v3/guides/push_notifications to continually monitor ~ 2,000 channels for new videos etc.
Up until last week, I wasn't having any problems, with pubnubsubbub, however, for some reason, I am getting 403 when I am trying to access (via curl or requests) https://pubsubhubbub.appspot.com/subscribe saying nothing more than
<p>
<b>403.</b> <ins>That’s an error.</ins>
<p>Your client does not have permission to get URL <code>/subscribe</code> from this server. <ins>That’s all we know.</ins>
</p>
So my questions are:
is there any limit (either IP or time or quota) that is documented somewhere and I am somehow hitting? As xml calls were "unlimited" and i do not see any difference in the official YouTube Data Api documentation
have somebody found a better pub/sub channel (either paid or not) with YouTube DataApi 3
if somebody had a similar situation, how long my "ban" will stay
As suggested by Marco Aurelio, I am adding my solution.
My VPS provider had some bans from Google on their IPv6 network in the data center where I had my server running. Thus by contacting them and fixing my network settings the pubsubhubbub continued working without issues.
So the tip for future debuggers stumbling upon here is trying to curl any other Google API (e.g. a function) that you are 100% sure it works and if you are still getting 403, then your IP address range is blocked for some reason (like data center ban in my case).
If you continue to have the problem and getting the "notification" with a maximum delay of 6 minutes isn't a problem, here is a workaround.
I don't use pubsubhubbub but you can simulate its advantages. Indeed you have 10 000 quota a day, using PlaylistItems: list with the uploads playlist of the channel found with contentDetails in part by using Channels: list.
In that way you can check for new videos for 50 YouTube channels for 1 quota so if you want to check your 2 000 channels you have to spend 40 of quota.
If you want to uniformly check during the day you can so make 250 global checks a day, so every 346 seconds (almost 6 minutes).
Of course you have to store your last videos found for a YouTube channel in order to check if there is any difference during your new global check.
If you just want a boolean response whether or not your current request response is different from the previous one, you can check the etag.
Related
I'm providing management of live streaming to the International Sporting Competition. At the request of the client we're using YouTube as a delivery platform. I've built an interface so the events and stream keys can be easily managed outside of the Youtube API. Ideally I would like to insert all events, so end users can have a direct link to the YouTube event.
After inserting 30 events (inserting at a rate of 1 per second) I seem to have hit an undocumented limit. I get a 403 error "request not authorised". I can still use the credentials for other actions so I have not crossed the global quota for the user or the account.
Unfortunately the Youtube API increase form is broken so I cannot contact Youtube there and request a rate increase.
Looks like there's a limit of 50 liveBroadcast.inserts(). This may be per 24 hour period I will confirm tomorrow. The web gui confirms that it is denying my requests because I have made too many live events and to try again "later".
Is there a way to get data (especially subscriptions.list and superChatEvents.list) in a more realtime manner?
What we do currently: we send request to endpoints every X seconds, and filter received data to get only the latest. We have X seconds delay, which isn't even close to realtime. Shortening X costs us too much quota, because we need to support a lot of users (lets say 50 000)
What we would like to do is:
send only ore request which gets data from more than one youtube channel (not something that YT API supports)
or
have WebSocket connection (Push notifications doesn't support what we're interested in)
or
?
There is a service online doing that (ex. you connect YT channel with them, channel gets subscription, and they know it immediately, and they support far more than 50 000 users), but they aren't open-source. All I managed to peep out is an enigmatic xhr request to https://www.youtube.com/youtubei/v1/log_interaction?alt=json&key=[devKey] but google doesn't help when I'm searching for details about that, like it wouldn't exists.
I'm a developer of a program for streamers. It receives and displays various information, such as chat, super chat, sponsors. The problem is that at the moment the program is close to the limit. And this despite the fact that I increased the delay between requests, but it did not help, because every day there are new users.
How can I increase the quota for Youtube v3?
Check this quota request page for Youtube API. You'll be required to answer several questions:
This application also requires you to submit screenshots and design
documents relating to your API Client(s) and your use of YouTube
API(s). If you do not have these ready, please apply once these are
available.
I have an application for sending out say around 100+ tweets every day. I am using OAuth for authentication. The twitter API says that post messages are not rate limited. However I am receiving the following error:
403:The request is understood, but it has been refused. An accompanying error message will explain why. This code is used when requests are being denied due to update limits (http://support.twitter.com/forums/10711/entries/15364).
error - User is over daily status update limit.
request - /1/statuses/update.json
Relevant discussions can be on the Internet at:
http://www.google.co.jp/search?q=15bb6564 or
http://www.google.co.jp/search?q=010f3e5b
TwitterException{exceptionCode=[15bb6564-010f3e5b], statusCode=403, retryAfter=0, rateLimitStatus=null, version=2.1.11}
Does this mean that status update API calls are also rate limited?
Thanks
Yes Status updates are rate limited. but 100 per day won't be an issue.
Current Twitter Limits
The current technical limits for accounts are:
Updates: 1,000 per day. The daily update limit is further broken down into smaller limits for semi-hourly intervals. Retweets are counted as updates.
From : http://support.twitter.com/forums/10711/entries/15364
Are you using some kind of shared hosting? the twitter API is limited by IP address and if there are other apps on your IP address also using the twitter API then you will be sharing rate limits with them. I ran into this issue with a tiny app on google app engine, despite only making about 10 requests in an hour.
The update limit is 1000 per day.But these are divided into intervals in which different amount of tweets are allowed per window. You can follow this thread.
Does this mean that status update API calls are also rate limited?
Yes,but not directly. As "The twitter API says that post messages are not rate limited" is true but not all POST requests are status update calls.
We switched over to YouTube from a previously developed internal video system, and right now I've got it going on public upload feeds for users. The problem comes when we first got only about a dozen out of 150 or so videos available from the primary user's account. Over time of the day that number rose to about 50, and its stayed there. I found one post on the YouTube API forums about delay, but no word on how long I can expect that.
Anyone have experience about this to tell me what to expect?
Expected latencies can vary, and are documented here (http://code.google.com/apis/youtube/2.0/reference.html) - ways you can minimize it:
1) Upload the video as public. If you upload the video as private first, this will leave the video out of the fast-track indexing
2) Make an authenticated request to the user feed, this will guarantee the freshest data that the API has.
ie. http://gdata.youtube.com/api/feeds/users/username/uploads (with a dev key and auth token for 'username') instead of http://gdata.youtube.com/api/feeds/videos?author=username