According to the docs:
Functions are limited to 30 concurrent invocations - meaning that if
you have more than 30 Functions being invoked at the same time, you
will begin to see new Function invocations return with a 429 status
code. To keep below the threshold optimize Functions to return as fast
as possible - avoid artificial timeouts and use asynchronous calls to
external systems rather than waiting on a large number of API calls to
complete.
-- https://www.twilio.com/docs/runtime/functions/faq
How can I obtain the execution time metrics of my Twilio functions? In order to ensure my functions stay under the 30 concurrent invocations limit, I'll need to compute the number of concurrent invocations, based on execution time and number of requests. I need to know the execution times (and ideally the number of invocations, but I can get that elsewhere). Does Twilio provide any metrics for Twilio Functions?
Twilio developer evangelist here.
I don't believe that information is available either in the response headers or via the API right now.
If you are concerned about breaching those limits, I would recommend you raise this with your account executive at Twilio. If you don't have an account executive, that sort of usage sounds as though you should have a relationship, so get in touch with sales or failing that support.
Related
I have a system as shown in the diagram. Multiple clients request to the web pod, web pod pushes the request to a shared queue.
Messages from the queue are de-queued based on the rate at which process_task_worker processes the request.
Once the processing is done, it sends the message to the shared queue. These messages are then dequeued by the webhook_worker and webhook (API call) is fired to respective clients.
I would like to add rate limiter to rate limit the number of webhook sent by the webhook_worker per client. rate limiting factor could be a generic count across all the clients.
But if rate limit is breached, that message should be re-enqueued to the queue instead of discarding it.
What all techniques can i employ to solve this? What is the term called for such thing since we are not discarding it but rather re-enqueuing it?
We are building an outbound IVR that will receive a payload, in bulk, via REST API, and place an outbound call to the recipient. We are attempting to limit the number of concurrent flow executions, or calls placed at a time, to prevent transfers by the called parties from flooding the shared inbound queue. Is there any way to accomplish this internally to Twilio?
If my assumptions are correct, the limiting factors when placing outbound API calls via Twilio Studio are the inbound API queue, the number of concurrent flow executions, and the number of Calls Per Second (CPS).
My understanding is that the queued API requests are executed 30 at a time, on a FIFO basis- as one execution is completed, another begins.
Each execution can then place a call at a rate of no more than 1 CPS, so 30 seconds for all calls to be sent.
Is this correct?
Is there any means of throttling these executions, or outbound calls?
A CPS limitation would be ideal, however the minimum number is 1 CPS, which is still 3600 Calls Per Hour, far too many for this call center to handle. Can this be lowered to less than 1 CPS?
Your assumptions are correct, as far as I can tell, but Twilio is built to, within those limits, make the calls you want to make as fast as it can.
If you don't want to Twilio to place the outbound calls, you need to not send the API request to start the call until you are ready. You can do this by knowing the concurrency that you can handle in your own application and by subscribing to call status webhooks so that you know when a call is complete and you can then place a new outbound call.
I got this error:
The request cannot be completed because you have exceeded your quota.
and I can not understand YouTube limits the number of requests? That is, I cannot create my project by taking API from my channel? If this is so, what is the point of YouTube Data API, if at the development stage I was already limited, what will happen when users come in, then my project will fall within 5 minutes?
and I cannot understand how I was able to make 10,000 requests per day, given that I worked on the localhost for about 3 hours, is this possible?
Indeed the Google's Developers Console shows text like Queries per day, but that's very much misleading (and may well be reported as an Web UI bug to Google).
You have to acknowledge that YouTube Data API's quota system is not accounting for the number of endpoints calls you made during a day long, but it accounts for the cumulated number of quota units corresponding to each of your endpoint calls.
For example, if you have 10000 units of quota allocated for daily usage, you may very easily exceed this upper bound after only 100 calls to the Search.list API endpoint.
Many API users find the default amount of quota allocated -- 10000 units -- to be quite constraining -- that even during the development stage of their apps. For tackling this issue, I recommend two things:
Develop your app such that to cache API responses it received from the endpoints it calls; this way, during the development stage of your app (afterwards, even during production, but albeit functioning with a different logic), repeated calls to endpoints would not result in actual API requests, but would get served from the app's local cache.
Apply for a quota extension, using Google's official form; be aware that, as per the experience of users of this forum, Google's answer, usually, does not arrive shortly.
I know this question is wired, But I am not 100% sure if it is possible or not. Need expert advise.
I am using this architecture (see Fig 1), there is a MVC WebAPI which puts data in Azure Queue and then Queue will call Azure Function to perform small tasks but very large in number e.g Queue is sending 5k - 10k requests to Azure Function in 1 minute.
Fig 1
We want to remove Azure Function because it cost us a lot. We want to go for alternate of it.
For this, someone share an idea to remove Azure function with another MVC WebAPI. (see Fig 2)
Fig 2
Is above architecture is possible ? If yes then How and If no then can anyone please suggest anything?
When using Azure Functions with Storage Queue trigger, Azure Functions will scale out based on the load on the queue. By default, batchSize is set to 16. The setting can be configured via host.json
The number of queue messages that the Functions runtime retrieves simultaneously and processes in parallel. When the number being processed gets down to the newBatchThreshold, the runtime gets another batch and starts processing those messages. So the maximum number of concurrent messages being processed per function is batchSize plus newBatchThreshold. This limit applies separately to each queue-triggered function.
This setting alone might not be sufficient when the number of messages is substantial. In that case, you want to restrict the scale-out behaviour associated with the number of VMs used to execute the Function App. The setting is an App Setting WEBSITE_MAX_DYNAMIC_APPLICATION_SCALE_OUT. Setting it to 1 would prevent any scale-out to new VMs, but according to the documentation
This setting is a preview feature - and only reliable if set to a value <= 5
While your focus is on the cost of processing, take into consideration time as well. Unless it's OK to wait for the messages to get processed for a long time, you're likely to have other alternatives to Functions. But the trade-off between the cost and the time to process will always be there.
Is there a specific number of requests/minute (specific to a tenant) that an application can make to Microsoft Graph APIs before requests start getting throttled?
No, not specific to a tenant (at least not for the Outlook-related parts of the Graph). Throttling is done per user per app. The threshold is 10000 requests every 10 minutes.
https://blogs.msdn.microsoft.com/exchangedev/2017/04/07/throttling-coming-to-outlook-api-and-microsoft-graph/
For non-Outlook stuff, I'm not sure what the limits are. All Graph has to say about it is here:
https://developer.microsoft.com/en-us/graph/docs/concepts/throttling
The takeaway here is you should not depend on a specific threshold since we can always change it if we need to in order to protect the integrity of the service. Ensure that your app can gracefully handle being throttled by handling the 429 error response properly.