How to fix 403 error what Twilio calls my webhook? - twilio

We're using Twilio. We have webhooks set up so that when Twilio receives a call, it forwards it to a URL on our site.
This appears to have been working fine. But now I made a change to the code, and suddenly Twilio is having problems calling the webhook. We don't receive the message, and if I look in the Twilio log, it says it got a 403 error. (I can't swear that this has never happened before. I've never noticed a message to be lost, but maybe I missed it while debugging other errors, attributed a lost message to something else.)
The truly strange part is, about 2/3 of the message that come in are received and processed fine, but about 1/3 get the 403 error. This is on a test server where we don't have any load balancing, so all requests are going to the same instance of our app. The tests I've been doing today are all from the same cell phone to the same Twilio number.
We do have authorization on the app, but the authorization is all on sub-directories, not the top level, and the sub-directory with our web hook has no authorization set up.
The first thing my our web hook now does when it gets called is send me an email with the content of the message from Twilio. (For debugging purposes.) I'm not getting that email, so I'm very confidant it's not getting called. And as I say, I can look at Twilio's log and it says that it received the text message and got a 403 error trying to forward it to my webhook.
The fact that it's only like 1/3 of the time is particularly puzzling. It's from the same number, to the same number, hitting exactly the same URL on the same site. Why would it work sometimes and not other times?
I tried to reproduce the problem on my desktop by calling the URL directly, not going through Twilio, and that does not give the same error. (It occurs to me as I type that the next logical test is to hit the page on the server without going through Twilio.)
Oh, the server is ASP.NET. The code is in VB but I doubt that matters as we're not getting as far as executing any of our code when it fails. When it doesn't get the 403, the code is working fine.

Check your firewall configuration as it might block the requests.
If it does then whitelist requests originating from Twilio.
We're using AWS WAF and ran into a similar situation: We saw the requests erroring out with a 403 in the Twilio Debugger but the requests never hit our endpoints. Once we adjusted the whitelist the problem was gone.

Related

send request to a bot with start parameter

I wonder if there is a way I can request a bot by the link like 'https://t.me/bot_name?start=XXX', and get the bot's answer? I've searched for it for a long time, but found no way.to solve it.
I've tried sending to the bot a '/start=XXX' message, but it returns a different answer from the manual operation.

Sendgrid Inbound Webhook - Undelivered notification

As per sendgrid documentation, the way Inbound parser webhook responds to failures are that it will retry to post the email to the configured endpoint for 3 days and will drop the email if undelivered.
The Parse API will POST the parsed email to a URL that you specify. If a POST is unsuccessful, SendGrid automatically queues and retries any POSTs that respond with a 5XX status. This prevents data loss for customers who have misconfigured their website or POST URL.
Respond with a 2xx status to the POST request to stop the email from retrying.
To avoid returning an error your link must return a 2xx HTTP code when the email is received. This response lets our system know that your link has received the email. It is then removed from our send queue. If we do not get a valid 2xx HTTP response, our servers will believe they have failed to deliver your message. Messages that cannot be delivered after 3 days will be dropped.
I would like to know if before dropping the undelivered emails, will sendgrid notify the sender that their email could not be delivered?
Based on a few tests I ran, it does not appear SendGrid will ever send a notification to the sender, nor is there any easy way to ascertain whether SendGrid has dropped any inbound emails.
Although the documentation wording is slightly ambiguous, my understanding based on testing (which aligns with the documentation wording) is this:
Server returns 2xx: email is deemed accepted, no further attempts made.
Server returns 5xx: server has error, attempts made until 2xx is returned (see below for timing of subsequent attempts), or 72 hours lapses
Server returns any other response, or DNS records do not exist: email is deemed failed, no further attempts made.
My conclusions are based on a number of tests I ran over the course of a week, from which I ascertained the following:
Case 1. Parse server returns 400 or 403 error
The email is dropped after one attempt.
No further attempts are made to POST the email.
No notification is sent to the sender or to the SendGrid account.
(Test method: Configure SendGrid to a URL that returns one of the above error codes. Checked server logs over 1 week, and noted only one attempt was made.)
Case 2. The parse hook URL does not have a DNS record
The email is dropped, presumably after one attempt.
No notification is sent to the sender or to the SendGrid account.
(Test method: configure the hook URL to a subdomain that does not have any DNS records. Run DNS search and try opening the hook URL to confirm the DNS records do not point anywhere. Send an email. Then, after 12 hours, add an appropriate record to the subdomain to point it to a script. Checked server logs and confirmed no attempts were made to POST the email. A subsequent email attempt successfully POSTed.)
Case 3. Parse server returns 500 error
SendGrid attempts to POST to the hook URL at the following intervals:
+0 +5m +10m +15m +20m
+25m +35m +50m +1h20m +2h20m
...then every 3 hours
The last attempt occurs +71h20m
The 20 min offset is a bit unusual, but it falls on the hour, so it may be because the message is queued to attempt POSTing on the hour.
No further attempts are made to POST the email after 72 hours.
No notification is sent to the sender or the SendGrid account.
Statistics
The documentation refers to the availability of statistics, however, I have found the numbers to be inaccurate.
For example, during my testing, I had a number of emails go through the API, including some that were intended to be successful (and did indeed POST to my server) and a few that were intended to fail (as part of the above testing), however, the numbers returned did not align.
Takeaways
Care should be exercised in the following scenarios:
Server downtime: this would ultimately depend on how the server is configured. If the server returns a 5xx response code, SendGrid will keep trying to POST the email. I also tried to test a timeout scenario, however, SendGrid seems to be very patient (e.g. I made my script pause for 10 minutes, SendGrid kept the connection for 10 minutes, though interestingly, due to the second attempt occurring after 5 mins, the email POSTed twice).
Server misconfiguration that returns 4xx error codes. SendGrid will drop the email.
It should also be noted if an email is dropped, there doesn't appear to be any reliable method of finding this out.
SendGrid won't send a notification before the email is dropped. However, the message will be dropped only after receiving an error from your Webhook URL and retrying to post the message for three days.
We got a confirmation from the Sendgrid support team, that the emails that are dropped in the Inbound Webhook Parser feature, will not have any notification or being saved in Sendgrid. So, we won't be able to get any info regarding the emails which are dropped.

500 response from sms.botframework.com/api/sms through Twilio

I've got a bot up and running, most of the time it is working ok, but Twilio throws some 500 and 502 errors on every inbound SMS.
An attempt to retrieve content from https://sms.botframework.com/api/sms returned the HTTP status code 500.
or
An attempt to retrieve content from https://sms.botframework.com/api/sms returned the HTTP status code 502.
The 502 requests continue fine and responses are sent, but the 500s do not - the user doesn't get a response.
How can i get logging from the sms.botframework.com service to see what's going on? No errors are showing on the azure side for my bot.
Thank you
Right now the Bot Framework isn't set up for you to be able to get to the logs inside our SMS connector; the 502's are benign, but the 500's aren't something I normally see. If you wish, you can contact me with your botid at bf-reports#microsoft.com and I'll take a look, or DM me on twitter (#jameslew).
#Philnash happy to fill you in more on the BotFramework and how developers can use it with Twilio; just let us know how to contact you.

Twilio gets a 502 Bad Gateway for a URL that responds properly to me

This problem was an internal networking issue on our side. It was not an issue with Twilio or their API
When my app contacts Twilio it passes a call back URL so Twilio can get additional info. I never see the request from Twilio but I get a call telling me that an application error has occurred.
When I check the Twilio debugging log it says it got a "502 Bad Gateway" error for the URL I passed it. However, when I try the same URL, I get a response just fine.
My app is running on a non-standard port but I'm pretty sure Twilio doesn't care about that.
What can cause Twilio to get a 502 when I don't
Here's the response I get when I hit the URL
<Response>
<Gather action="http://dev1.onshift.com/twilio/user_response_handler?sendID=38297795" method="POST" numDigits="1">
<Play>http://dev1.onshift.com/wavs/254_preamble.wav</Play>
<Play>http://dev1.onshift.com/wavs/8144924.wav</Play>
<Play>http://dev1.onshift.com/static/messages/handle_human_footer.wav</Play>
<Play>http://dev1.onshift.com/wavs/254_preamble.wav</Play>
<Play>http://dev1.onshift.com/wavs/8144924.wav</Play>
<Play>http://dev1.onshift.com/static/messages/handle_voicemail_footer_v2.wav</Play>
<Play>http://dev1.onshift.com/wavs/8144924.wav</Play>
<Play>http://dev1.onshift.com/static/messages/handle_voicemail_footer_v2.wav</Play>
<Play>http://dev1.onshift.com/static/messages/goodbye.wav</Play>
</Gather>
</Response>
From OP's comments above:
In my case there was a change on our network that blocked the server I
was working on from outside of our network. Your network people may be
wrong or confused. If you know the URL you're passing to Twilio you
can prove them right or wrong by seeing if you can hit that URL from
outside your network. You can try from home, or, what I did was turn
off wireless on my phone and tried to hit the URL from it. I couldn't
do it although it worked from my laptop on the company wifi. If you
can't hit the URL from outside your network neither can Twilio
Follow these suggested steps will help others begin to debug as they receive this error.

Twilio client error when accepting client calls

I am using twilio client to make & accept calls from a browser. But when I run the accept page, I get a javascript error on the browser -
Uncaught Twilio.Exception: 401: Unable to determine account.
# line - twilio.js:2683
Here's the code from my controlled thats gets executed when showing the accept call page/view -
def prepare_accept_call
# create the twilio capability object
capability = Twilio::Util::Capability.new TWILIO_ACCOUNT_SID, TWILIO_AUTH_TOKEN
# guessing this is not needed for accepting calls, but added to see if it fixes my issue and it doesnt :(
capability.allow_client_outgoing TWILIO_APP_SID
capability.allow_client_incoming "some-client-identifier"
# use this token for client JS - Twilio.Device.setup("#{#token}");
#token = capability.generate
end
NOTE -: I am successfully able to show the make call page, there are no javascript error - when i click the call button I get the flash microphone permission popup. There are no 401. So guessing my key, tokens etc are ok.
Any idea what could be happening here?
I also had this error with similar conditions where all my credentials were correct but the Twilio client debugger was spitting out "Application not found 31001" - which is a Twilio client error code by the way. I realized that I had not entered the Voice request url that is in the applications section. Apparently this is a necessary prerequisite to access the Twilio Client service.
I feel they should have been more clear with their message but regardless I solved it by entering a publicly reachable callback url using a SSH tunneler like Localtunnel or free server like Heroku.
looking at the code it should work, because it looks good. Can you verify where TWILIO_ACCOUNT_SID and TWILIO_AUTH_TOKEN are ultimately being defined? maybe that could have something to do with it?

Resources