I am trying to prevent Recurly Webhooks from being executed multiple times if they are retried incorrectly (i.e.: when builds go out and servers time out occasionally). I see in the dashboard that webhooks have unique IDs, but these don't seem to be accessible in the message hash, am I missing something? I'm using Rails for what it's worth.
<?xml version="1.0" encoding="UTF-8"?>
<updated_subscription_notification>
<account>
<account_code>1</account_code>
<username nil="true"></username>
<email>verena#example.com</email>
<first_name>Verena</first_name>
<last_name>Example</last_name>
<company_name nil="true"></company_name>
</account>
<subscription>
<plan>
<plan_code>1dpt</plan_code>
<name>Subscription One</name>
</plan>
<uuid>292332928954ca62fa48048be5ac98ec</uuid>
<state>active</state>
<quantity type="integer">1</quantity>
<total_amount_in_cents type="integer">200</total_amount_in_cents>
<activated_at type="datetime">2010-09-23T22:12:39Z</activated_at>
<canceled_at nil="true" type="datetime"></canceled_at>
<expires_at nil="true" type="datetime"></expires_at>
<current_period_started_at type="datetime">2010-09-23T22:03:30Z</current_period_started_at>
<current_period_ends_at type="datetime">2010-09-24T22:03:30Z</current_period_ends_at>
<trial_started_at nil="true" type="datetime">
</trial_started_at>
<trial_ends_at nil="true" type="datetime">
</trial_ends_at>
<collection_method>automatic</collection_method>
</subscription>
</updated_subscription_notification>
Unfortunately the only way to stop webhooks from being resent is to reply with a 200 status code with 5 seconds.(https://docs.recurly.com/docs/webhooks)
It was 30 seconds but that got changed November 2015 I think.
Anyway I solved it by throwing the message off to a queue and immediately responding with 200 response.
A webhook may be retired several times, however once a successful status code is received by your server, Recurly will stop attempting to resend that notification. Hence, only one webhook will be delivered per event. Also, webhook IDs aren't included in the message hash, they're only found in the UI.
Related
I am building a slack application that will schedule a message when someone posts a specific type of workflow in a channel.
It will schedule a message, and if someone from a specific group of users replies before it has sent, it will delete the scheduled message.
Unfortuantely these messages are still sending, even though the list of scheduled messages is empty and the response when deleting the message is a successful one. I am also deleting the message within the 60 second limit that is noted on the API.
Scheduling the message gives me a success response, and if I use the list scheduled messages I get:
[
{
id: 'MESSAGE_ID',
channel_id: 'CHANNEL_ID',
post_at: 1620428096, // 2 minutes in the future for testing
date_created: 1620428026,
text: 'thread_ts: 1620428024.001300'
}
]
Canceling the message:
async function cancelScheduledMessage(scheduled_message_id) {
const response = await slackApi.post("/chat.deleteScheduledMessage", {
channel: SLACK_CHANNEL,
scheduled_message_id
})
return response.data
}
response.data returns { "ok": true }
If I use the list scheduled message API to retrieve what is scheduled I get an empty array []
However, the message will still send to the thread.
Is there something I am missing? I have the proper scopes set up and the API calls appear to be working.
If it helps, I am using AWS Lambda, and DynamoDB to store/retrieve the thread_ts and message IDs.
Thanks all.
For messages due in 5 minutes or less, chat.deleteScheduleMessage has a bug (as of November 2021) [1]. Although this API call may return OK, the actual message will still be delivered due to the bug.
Note that for messages within 60 seconds, this API does return an proper error code, as described in the documentation [2]. For the range (60 seconds, ~5 minutes), the API call returns OK but fails behind the scenes.
Before this bug is fixed, the only thing one can do is to only delete messages scheduled 5 minutes (the exact threshold may vary, according to Slack) or more (yes not very ideal and may not be feasible for some applications).
[1] Private communication with Slack support.
[2] https://api.slack.com/methods/chat.deleteScheduledMessage
We are looking at Event Bridge to give us a scheduled task added to our SQS once per minute.
We are looking at Event Bridge to make it happen. So far it properly puts messages into the queue, but we are trying to schedule it for once per minute and noticing that the queue only gets messages once per five minutes sometimes six minutes.
The metrics seem to state invocation is happening; however, the queue isn't receiving them in the time frame specified.
Considerations
SQS FIFO Queue - Deduplication
Constant JSON String
The "duh" of not seeing messages at prescribed interval is because of this in the AWS documentation:
The token used for deduplication of sent messages. If a message with a
particular message deduplication ID is sent successfully, any messages
sent with the same message deduplication ID are accepted successfully
but aren’t delivered during the 5-minute deduplication interval
Open to suggestions and will be looking for a workaround.
Update
I tried using Input Transformer to fix by adding the time as uniquely changing item in the queue message; however, still not getting below 5 minutes.
Variable Input
{"addedOn":"$.time"}
Message
{"AddedOn":<addedOn>}
The queue polling built into SQS just wasn't polling for my updated count of greater than 10 messages. Once I deleted out the old messages the timing was correct and it was updating 1/min.
The answer is if you are going to use a constant string it'll have to be for scheduled jobs that are greater than 5 minutes.
Adding info here despite redundancy from question for linked Google searches:
The token used for deduplication of sent messages. If a message with a particular message deduplication ID is sent successfully, any messages sent with the same message deduplication ID are accepted successfully but aren’t delivered during the 5-minute deduplication interval
Despite the messages being a unique event once per minute the Constant (JSON text) not being unique still saw it as a duplicate to remove.
To solve I switched to Input transformer
Example event and what you other fields you can add as variables:
{
"version": "0",
"id": "7bf73129-1428-4cd3-a780-95db273d1602",
"detail-type": "EC2 Instance State-change Notification",
"source": "aws.ec2",
"account": "123456789012",
"time": "2015-11-11T21:29:54Z",
"region": "us-east-1",
"resources": [
"arn:aws:ec2:us-east-1:123456789012:instance/i-abcd1111"
],
"detail": {
"instance-id": "i-0123456789",
"state": "RUNNING"
}
}
I needed a unique variable so time was an obvious choice.
Input for Input Transformer
Input Path:
{"addedOn":"$.time"}
Template:
{"AddedOn":<addedOn>}
Documentation
Also, found that moving over to not using FIFO queues is a potential solution if that's an easy option for future SQS developers as well.
Background: Using webhooks to direct incoming voice call to our application handler which connects the caller to an available reps cell phone.
The first part of the app checks to see if the incoming Caller telephone number is on an internal blacklist (to ignore robo dialers etc) -- if there is a match, our app will reject the call so we do not get billed. This rejection is being done echoing a Reject TWIML verb as follows:
echo '<Response><Reject reason="rejected"/></Response>';
exit;
The code works fine, send the xml response and exits the PHP script, however, the incoming caller doesn't get a "busy" signal or "Not In Service" message -- instead it just hangs in silence.
Additionally, the Twilio webhooks handler keeps firing off calls with the CallStatus = no-answer every approximately 2-seconds until the caller hangs up.
Any idea as to what I am doing wrong?
I determined that problem was with the formatting of the xml response being echo out. It turns out that formatting of the XML response needs to preserve format and whitespace, etc of XML (???):
This did not work (twilio will hang in silence until caller hangs up):
echo '<Response><Reject reason="rejected"/></Response>';
exit;
This does work (caller will receive busy signal):
echo '<?xml version="1.0" encoding="UTF-8"?>
<Response>
<Reject reason="busy" />
</Response>';
exit;
I have an IMAP connection to fetch emails using Mule. I'm running into an issue.
Here are my 2 simple requirements:
I want to fetch emails in reverse order. (latest first)
Ignore SEEN messages but don't delete them.
I was looking at the code that mule (3.3.1) uses:
org.mule.transport.email.RetrieveMessageReceiver.poll().
The code seems to be fetching messages from message 1.
348: Message[] messages = folder.getMessages(1, batchSize);
The messages fetched here are processed in a loop in :
org.mule.transport.email.RetrieveMessageReceiver.messagesAdded(MessageCountEvent)
142: if (!messages[i].getFlags().contains(Flags.Flag.DELETED)
143: && !messages[i].getFlags().contains(Flags.Flag.SEEN))
What this whole logic is doing is that it is trying to read OLD unread messages. The code comes back to line 348 and executes
folder.getMessages(1, batchSize);
again, and gets the same messages and it keeps on waiting. How can i change the order of fetch.
FYI: Using MS Exchange for IMAP
Not sure why you say that Mule tried to read "OLD unread messages"? It actually just tries to read unread messages, ie not DELETED nor SEEN.
Anyway, theoretically the Mulesque way of sorting the messages would be to use resequencer. Unfortunately the mail message receivers do not set any of the required control properties to let Mule process the received messages as a single batch so that won't work.
So the only solution I can think of is to extend org.mule.transport.email.RetrieveMessageReceiver and register your custom version on the IMAP connector with a <service-overrides /> child element.
To developers/users of LMAX Disruptor http://code.google.com/p/disruptor/ :
My question:
Can anyone suggest an approach to how apply a timeout function to Disruptor e.g. using EventHandler?
Here is one scenario that came up in my line of work:
Outbox - messages sent to the Server over a network
Inbox - ACK messages received from the Server
ACK Handler - marks outbox messages as ACKed
Timeout Handler - marks outbox message as NACKed (much needed, but where can it fit into the Disruptor design?)
Is there anyone who share the same opinion?
Or can anyone point out why it is unnecessary.
I hope the ensuing debate would be brief.
Thank you.
To clarify the timeout-handler would "fire" after a certain period of time when a message could not be delivered?
The way it works with disruptor is you have a ringbuffer for inbound and a ringbuffer for outbound messges... so email comes in, place it into the inbound ring buffer using an appropriate event. then process the message (i.e. decode, analye, log, store) and send it along to another sytem by placing it into the outbound ringbuffer... another handler takes the message and stores it into a database or sends it to another server using smtp... if a error / timeout etc. occurs, your create an event in the inbound ringbuffer signaling the error (NACK) and process this message. does that make sense?!?