I'm providing management of live streaming to the International Sporting Competition. At the request of the client we're using YouTube as a delivery platform. I've built an interface so the events and stream keys can be easily managed outside of the Youtube API. Ideally I would like to insert all events, so end users can have a direct link to the YouTube event.
After inserting 30 events (inserting at a rate of 1 per second) I seem to have hit an undocumented limit. I get a 403 error "request not authorised". I can still use the credentials for other actions so I have not crossed the global quota for the user or the account.
Unfortunately the Youtube API increase form is broken so I cannot contact Youtube there and request a rate increase.
Looks like there's a limit of 50 liveBroadcast.inserts(). This may be per 24 hour period I will confirm tomorrow. The web gui confirms that it is denying my requests because I have made too many live events and to try again "later".
Related
I am using https://developers.google.com/youtube/v3/guides/push_notifications to continually monitor ~ 2,000 channels for new videos etc.
Up until last week, I wasn't having any problems, with pubnubsubbub, however, for some reason, I am getting 403 when I am trying to access (via curl or requests) https://pubsubhubbub.appspot.com/subscribe saying nothing more than
<p>
<b>403.</b> <ins>That’s an error.</ins>
<p>Your client does not have permission to get URL <code>/subscribe</code> from this server. <ins>That’s all we know.</ins>
</p>
So my questions are:
is there any limit (either IP or time or quota) that is documented somewhere and I am somehow hitting? As xml calls were "unlimited" and i do not see any difference in the official YouTube Data Api documentation
have somebody found a better pub/sub channel (either paid or not) with YouTube DataApi 3
if somebody had a similar situation, how long my "ban" will stay
As suggested by Marco Aurelio, I am adding my solution.
My VPS provider had some bans from Google on their IPv6 network in the data center where I had my server running. Thus by contacting them and fixing my network settings the pubsubhubbub continued working without issues.
So the tip for future debuggers stumbling upon here is trying to curl any other Google API (e.g. a function) that you are 100% sure it works and if you are still getting 403, then your IP address range is blocked for some reason (like data center ban in my case).
If you continue to have the problem and getting the "notification" with a maximum delay of 6 minutes isn't a problem, here is a workaround.
I don't use pubsubhubbub but you can simulate its advantages. Indeed you have 10 000 quota a day, using PlaylistItems: list with the uploads playlist of the channel found with contentDetails in part by using Channels: list.
In that way you can check for new videos for 50 YouTube channels for 1 quota so if you want to check your 2 000 channels you have to spend 40 of quota.
If you want to uniformly check during the day you can so make 250 global checks a day, so every 346 seconds (almost 6 minutes).
Of course you have to store your last videos found for a YouTube channel in order to check if there is any difference during your new global check.
If you just want a boolean response whether or not your current request response is different from the previous one, you can check the etag.
I have finished creating a bot and got blocked 1 minute after I finally got it to work. The account is now restricted.
this is what I get on my terminal if I try to run the bot now:
tweepy.error.TweepError: [{'message': 'Application cannot perform write actions. Contact Twitter Platform Operations through https://help.twitter.com/forms/platform.', 'code': 261}]
I understand Twitter may restrict new bots for several reasons. I have summited my report to try and get my account unrestricted.
What are good do and don'ts for bots?
In my original bot I was replying every 10s was this too much ? Should I had set a different time frame?
https://help.twitter.com/en/rules-and-policies/twitter-limits
Relevant parts:
Tweets: 2,400 per day. The daily update limit is further broken down into smaller limits for semi-hourly intervals. Retweets are counted as Tweets.
These limits include actions from all devices, including web, mobile, phone, API, etc. API requests from all third-party applications are tracked against the hourly API limit. People who use multiple third-party applications with their account will therefore reach the API limit more quickly.
What happens if I hit a limit?
If you do reach a limit, we'll let you know with an error message telling you which limit you've hit. For limits that are time-based (like the Direct Messages, Tweets, changes to account email, and API request limits), you'll be able to try again after the time limit has elapsed.
10 seconds is too much.
Is there a way to get data (especially subscriptions.list and superChatEvents.list) in a more realtime manner?
What we do currently: we send request to endpoints every X seconds, and filter received data to get only the latest. We have X seconds delay, which isn't even close to realtime. Shortening X costs us too much quota, because we need to support a lot of users (lets say 50 000)
What we would like to do is:
send only ore request which gets data from more than one youtube channel (not something that YT API supports)
or
have WebSocket connection (Push notifications doesn't support what we're interested in)
or
?
There is a service online doing that (ex. you connect YT channel with them, channel gets subscription, and they know it immediately, and they support far more than 50 000 users), but they aren't open-source. All I managed to peep out is an enigmatic xhr request to https://www.youtube.com/youtubei/v1/log_interaction?alt=json&key=[devKey] but google doesn't help when I'm searching for details about that, like it wouldn't exists.
I'm a developer of a program for streamers. It receives and displays various information, such as chat, super chat, sponsors. The problem is that at the moment the program is close to the limit. And this despite the fact that I increased the delay between requests, but it did not help, because every day there are new users.
How can I increase the quota for Youtube v3?
Check this quota request page for Youtube API. You'll be required to answer several questions:
This application also requires you to submit screenshots and design
documents relating to your API Client(s) and your use of YouTube
API(s). If you do not have these ready, please apply once these are
available.
I have an application for sending out say around 100+ tweets every day. I am using OAuth for authentication. The twitter API says that post messages are not rate limited. However I am receiving the following error:
403:The request is understood, but it has been refused. An accompanying error message will explain why. This code is used when requests are being denied due to update limits (http://support.twitter.com/forums/10711/entries/15364).
error - User is over daily status update limit.
request - /1/statuses/update.json
Relevant discussions can be on the Internet at:
http://www.google.co.jp/search?q=15bb6564 or
http://www.google.co.jp/search?q=010f3e5b
TwitterException{exceptionCode=[15bb6564-010f3e5b], statusCode=403, retryAfter=0, rateLimitStatus=null, version=2.1.11}
Does this mean that status update API calls are also rate limited?
Thanks
Yes Status updates are rate limited. but 100 per day won't be an issue.
Current Twitter Limits
The current technical limits for accounts are:
Updates: 1,000 per day. The daily update limit is further broken down into smaller limits for semi-hourly intervals. Retweets are counted as updates.
From : http://support.twitter.com/forums/10711/entries/15364
Are you using some kind of shared hosting? the twitter API is limited by IP address and if there are other apps on your IP address also using the twitter API then you will be sharing rate limits with them. I ran into this issue with a tiny app on google app engine, despite only making about 10 requests in an hour.
The update limit is 1000 per day.But these are divided into intervals in which different amount of tweets are allowed per window. You can follow this thread.
Does this mean that status update API calls are also rate limited?
Yes,but not directly. As "The twitter API says that post messages are not rate limited" is true but not all POST requests are status update calls.