I have a system as shown in the diagram. Multiple clients request to the web pod, web pod pushes the request to a shared queue.
Messages from the queue are de-queued based on the rate at which process_task_worker processes the request.
Once the processing is done, it sends the message to the shared queue. These messages are then dequeued by the webhook_worker and webhook (API call) is fired to respective clients.
I would like to add rate limiter to rate limit the number of webhook sent by the webhook_worker per client. rate limiting factor could be a generic count across all the clients.
But if rate limit is breached, that message should be re-enqueued to the queue instead of discarding it.
What all techniques can i employ to solve this? What is the term called for such thing since we are not discarding it but rather re-enqueuing it?
Related
We are building an outbound IVR that will receive a payload, in bulk, via REST API, and place an outbound call to the recipient. We are attempting to limit the number of concurrent flow executions, or calls placed at a time, to prevent transfers by the called parties from flooding the shared inbound queue. Is there any way to accomplish this internally to Twilio?
If my assumptions are correct, the limiting factors when placing outbound API calls via Twilio Studio are the inbound API queue, the number of concurrent flow executions, and the number of Calls Per Second (CPS).
My understanding is that the queued API requests are executed 30 at a time, on a FIFO basis- as one execution is completed, another begins.
Each execution can then place a call at a rate of no more than 1 CPS, so 30 seconds for all calls to be sent.
Is this correct?
Is there any means of throttling these executions, or outbound calls?
A CPS limitation would be ideal, however the minimum number is 1 CPS, which is still 3600 Calls Per Hour, far too many for this call center to handle. Can this be lowered to less than 1 CPS?
Your assumptions are correct, as far as I can tell, but Twilio is built to, within those limits, make the calls you want to make as fast as it can.
If you don't want to Twilio to place the outbound calls, you need to not send the API request to start the call until you are ready. You can do this by knowing the concurrency that you can handle in your own application and by subscribing to call status webhooks so that you know when a call is complete and you can then place a new outbound call.
Let's say on twitter, a celebrity update her status, and it's pushed to all of her followers.
If we set up queue, where the publisher is service that fetches the celebrity's status, and the consumer are the individual followers. But with a million followers, whoever receives that message first will see the update, while others will not ? What is a common pattern to use here so that every one of her followers will see the update and not 'compete' with each other to consume the message first?
I guess you are thinking of queueing system as only having the ability to make point-to-point communication, ie, producer to queue to consumer. This is partly correct. Most queueing systems have atleast two patterns:
Producer-consumer: In this scenario, a message is delivered to just one consumer, ie, if there are multiple consumers, they are competing against each other to get the message from the "queue".
Publish-subscribe: Here, publishers push the message on a "topic" and consumers subscribe to the topic to get messages. The consumers are not competing - each consumer gets all the messages once they have subscribed.
In your example, it's publish-subscribe pattern in action. Underlying implementation may be different, but the basic pattern is that of publish-subscribe.
Refer: https://stackoverflow.com/a/42477769/6886283
I tried using parallel requests but the due to retention by AWS, it does not allow to poll back the same queue unless previously polled messages are deleted.
I however achieved doing the same using the FIFO, but not the standard queue.
Thanks in Advance!
:)
When you say "it does not allow to poll back the same queue unless previously polled messages are deleted", I assume you're talking about the inflight messages per queue limit, which is pretty high at 120,000:
For most standard queues (depending on queue traffic and message backlog), there can be a maximum of approximately 120,000 inflight messages (received from a queue by a consumer, but not yet deleted from the queue). If you reach this limit, Amazon SQS returns the OverLimit error message. To avoid reaching the limit, you should delete messages from the queue after they're processed. You can also increase the number of queues you use to process your messages. To request a limit increase, file a support request.
The expected use case of SQS is to have workers that receive a message, do some work, then delete the message. If you're not following this pattern, I'd strongly recommend reevaluating whether SQS is the right tool for what you're trying to do.
However, if you really have a valid use case for having more than 120K messages inflight at once, you'll need to describe your use case to AWS and get their approval to increase that limit.
I'm using an Amazon SQS queue to send notifications to an external system.
If the HTTP request fails when using SQS' SendMessage, I don't know whether the message has been queued or not. My default policy would be to retry posting the message to the queue, but there's a risk to post the message twice, which might not be acceptable depending on the use case.
Is there a way to have SQS refuse the message if there is a duplicate on the message body (or some kind of message metadata, such as a unique ID we could provide) so that we could retry until the message is accepted, and be confident that there won't be a duplicate if the first request had been already queued, but the response had been lost?
No, there's no such mechanism in SQS. Going further, it is also possible that a message will be delivered twice or more (at-least-once delivery semantics). So even if such a mechanism existed, you wouldn't be able to guarantee that the message isn't delivered multiple times.
See: http://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/DistributedQueues.html
For exactly-once deliveries, you need some form of transactions (and HTTP isn't a transactional protocol) both on the sending and receiving end.
AFAIK, right now SQS does support what was asked!
Please see the "What's new" post entitled Amazon SQS Introduces FIFO Queues with Exactly-Once Processing and Lower Prices for Standard Queues
According to SQS FAQ:
FIFO queues provide exactly-once processing, which means that each message is delivered once and remains available until a consumer processes it and deletes it. Duplicates are not introduced into the queue.
There's also an AWS Blog post with a bit more insight on the subject:
These queues are designed to guarantee that messages are processed exactly once, in the order that they are sent, and without duplicates.
......
Exactly-once processing applies to both single-consumer and multiple-consumer scenarios. If you use FIFO queues in a multiple-consumer environment, you can configure your queue to make messages visible to other consumers only after the current message has been deleted or the visibility timeout expires. In this scenario, at most one consumer will actively process messages; the other consumers will be waiting until the first consumer finishes or fails.
Duplicate messages can sometimes occur when a networking issue outside of SQS prevents the message sender from learning the status of an action and causes the sender to retry the call. FIFO queues use multiple strategies to detect and eliminate duplicate messages. In addition to content-based deduplication, you can include a MessageDeduplicationId when you call SendMessage for a FIFO queue. The ID can be up to 128 characters long, and, if present, takes higher precedence than content-based deduplication.
I am going to make the rails application which integrates the Amazon's cloud services.
I have explore amazon's SNS service which gives the facility of public subscription which i don't want to do. I want to notify only particular subscriber.
For example if I have 5 subscriber in one topic then the notification should be goes to particular subscriber.
I have also explored amazon's SQS in which i have to write a poller which monitor the queue for message. SQS has also a lock mechanism but the problem is that it is distributed so there would be a chance of getting same message from another copy of queue for process.
I want to know that what would be the possible approach to go.
SQS sounds like what you want.
You can run multiple "worker" processes that compete over messages in the queue. Each message is only consumed once. The logic behind the "lock" / timeout that you mention is as follows: if one of your workers were to die after downloading a message, but before processing it, then you want that message to eventually time out and be re-downloaded for processing on another node.
Yes, SQS is built on a polling model. For example, I have a number of use cases in which I use a minutely cron job to poll for new messages in the queue and take action on any messages found. This pattern is stupid simple to build and works wonders for a bunch of use cases -- a handy little "client" script that pushes a message into the queue, and the cron activated script that will process that message within a minute or so.
If your message pattern is extremely sparse -- eg, only a few messages a day -- it may seem wasteful to poll constantly while the queue is empty. It hardly matters.
My original calculation was that a minutely cron job would cost $0.04 (now $0.02) per month. Since then, SQS added a "Long-Polling" feature that lets you achieve sub-second latency on processing new messages by sending 1 "long-poll" message every 20 seconds to poll an idle queue. Plus, they dropped the price 50%. So per month, that's 131k messages (~$0.06), a little bit more expensive, but with near realtime request processing.
Keep in mind that a minutely cron job I described only costs ~$0.04 / month in request load (30d*24h*60m * 1c / 10k msgs). So at a minutely clip, cost shouldn't really be a concern here. Even polling every second, the price rises only to $2.59 / mo, not exactly a bank buster.
However, it is possible to avoid frequent polling using a webservice that takes an SNS HTTP message. Such an architecture would work as follows: client pushes message to SNS, which pushes message to SQS and routes an HTTP request to your webservice, triggering it to drain the queue. You'd still want to poll the queue hourly or daily, just in case an HTTP request was dropped. In the end though, I'm not sure I can think of any scenario which really justifies such complexity. I'd much rather pay $0.04 a month to have a dirt simple cron job polling my queue.