I wanted an Autopilot bot to send its memory data to a webhook file of mine (which is in PHP) and in return (as a callback) just print a message saying "Received".
But every time, I get an error saying "It does not match with Action Schema."
So I was wondering if anyone could give me a nice example as to how to do this?
I've tried reading the ActionSchema.json file provided in the Debugger, but I couldn't understand it at all.
I have tried like this
echo '"definitions": {
"say": {
"anyOf": [
{
"type": "strdfghing"
}';
I want Twilio Autopilot to reply "Got it!" after it sends its memory events to a webhook file.
Twilio developer evangelist here.
Welcome to StackOverflow!
So you probably have your Collect flow, where you can use the Say verb to respond with "got it!" at the end in the on_complete of a Redirect. Your Autopilot task may look something like this:
"actions": [
{
"collect": {
"name": "collect_clothes_order",
"questions": [
{
"question": "What is your first name?",
"name": "first_name",
"type": "Twilio.FIRST_NAME"
},
],
"on_complete": {
"redirect": "your-web-server-ie-a-twilio-function-maybe"
}
}
}
]
Then, in your Twilio Function (or another web server in the language of your choosing), you could receive the memory data (in this case, the first name that the user responded with), and return the Say containing "got it!":
exports.handler = function(context, event, callback) {
let responseObject = {};
let memory = JSON.parse(event.Memory);
console.log(memory.twilio.collected_data);
let first_name = memory.twilio.collected_data.collect_clothes_order.answers.first_name.answer;
console.log(first_name); //collected data from memory
responseObject = {"actions":[
{ "say": { "speech": "Got it!" } }
]};
callback(null, responseObject);
};
Let me know if this helps at all!
Related
Current we send one-way SMS via MS Flow and Twilio which works fine. I have been exploring how to handle incoming SMS, so I followed a guide and managed to utilise Sendgrid to forward incoming SMS to my e-mail address which also works.
However, I am looking to have the original sender receive the SMS via e-mail. I can give each staff member their own phone number which would define each individual but I need a way of Twilio or Sendgrid doing a lookup prior to sending the reply e-mail so it knows where to send it i.e.
User 1 = 01234455678,
User 2 = 01234543245,
User 3 = 06546546445,...etc.
I guess I could re-create the same process for each number but it would require lots of Twilio Functions and Variables which doesn't seem like a great way to accomplish this?
Sorry, I a not much of a coder and try to use on-line guides and forums where I can.
Many Thanks,
JP
You can try something like this, hosting the mapping on Twilio Assets as a Private asset but you could also pull this information into Studio via the HTTP Request Widget if you hosted it on an external server (a bit more advanced). In my case I called my file mapping.json which has the format:
[
{
"name": "John Doe",
"phone": "+14075551212",
"email": "jdoe#example.com"
},
{
"name": "Susan Doe",
"phone": "+19545551212",
"email": "sdoe#example.com"
},
{
"name": "Nadia Doe",
"phone": "+14705551212",
"email": "ndoe#example.com"
},
{
"name": "Carl Doe",
"phone": "+18025551212",
"email": "cdoe#example.com"
}
]
Then you would use the Run Function widget and send in 3 key:value pairs (Function Parameters):
From - {{trigger.message.From}}
To - {{trigger.message.To}}
Body - {{trigger.message.Body}}
Your Twilio Function would then consume these parameters and the contents of the private asset to handle the mapping. Make sure to configure your Twilio Functions environment with the Sendgrid NPM package, #sendgrid/mail version 7.0.1 and you configure the two Sendgrid specific environmental variables below with their respective values (accessed via the context object in the JavaScript):
SENDGRID_API_KEY
FROM_EMAIL_ADDRESS
const fs = require('fs');
const sgMail = require('#sendgrid/mail');
exports.handler = function(context, event, callback) {
let from = event.From;
let to = event.To;
let body = event.Body;
let fileName = 'mapping.json';
let file = Runtime.getAssets()[fileName].path;
let text = fs.readFileSync(file);
let mappings = JSON.parse(text);
// Filter array to match to number
let result = mappings.filter(record => record.phone === to);
if (result.length) {
sgMail.setApiKey(context.SENDGRID_API_KEY);
// Define message params
const msg = {
to: result[0].email,
from: context.FROM_EMAIL_ADDRESS,
text: body,
subject: `New SMS from: ${from}`,
};
// Send message
sgMail.send(msg)
.then(response => {
console.log("Success.");
callback();
})
.catch(err => {
console.log("Not Success.");
callback(err);
});
} else {
console.log("** NO MATCH **");
callback();
}
};
Let me know how it goes.
I'm not a programmer so please forgive me. But I've spent hours and hours of research on the topic of collecting information with Twilio AutoPilot and posted that data to Airtable, which I will then have Zapier do some things with that data. I finally had a breakthrough today and am now able to post data from a call or text to Airtable. The only way I got the ending to work was to send the call or text to Studio to finish up the call. Everything seems to work from the end user standpoint, but I'm getting an error 90100 from Twilio. I'm sure I'm just missing one line of code for this to work, and I'm at the end of my rope.
{
"actions": [
{
"say": "Okay lets get you a new appointment. I just need you to answer a few questions."
},
{
"collect": {
"name": "member",
"questions": [
{
"question": "Please tell me your first name.",
"name": "name",
"type": "Twilio.FIRST_NAME"
},
{
"question": "Thanks, and what is your email address?",
"name": "email",
"type": "Twilio.EMAIL"
}
],
"on_complete": {
"redirect": "task://complete_booking"
}
}
}
]
}
Then i have another task setup to redirect to the Twilio Function. This is probably overkill, but it's what I found in research.
{
"actions": [
{
"redirect": {
"method": "POST",
"uri": "https://TWILIO_FUNCTION_URL/atable_post"
}
}
]
}
Then the function is as follows. Mind you, this is posting correctly to airtable.
exports.handler = function(context, event, callback) {
let memory = JSON.parse(event.Memory);
let name = memory.twilio.collected_data.member.answers.name.answer;
let email = memory.twilio.collected_data.member.answers.email.answer;
console.log(memory);
let member = {
name : memory.twilio.collected_data.member.answers.name.answer,
email : memory.twilio.collected_data.member.answers.email.answer,
date : Date.now()
};
var Airtable = require("airtable");
var base = new Airtable({apikey: context.AIRTABLE_API_KEY}).base("AIRTABLE_ID");
base("Members1").create(member, function(err, record) {
if (err) { console.error(err); return; }
console.log(record.getId());
callback(null, member);
});
};
The call hung up at this point, so I redirected it to a Studio Flow, which does work and the call finishes with the response I'm give it before ending the call. Again, everything is working fine, but I get the following error from twilio, and I have no idea how to resolve it.
Invalid Autopilot Actions JSON: Invalid Autopilot Action
Any help would be greatly appreciated. Thanks!
Nice work James! It looks the the issue is the redirect to your Twilio Function is not returning the expected JSON Action response to execute.
Autopilot - Redirect
https://www.twilio.com/docs/autopilot/actions/redirect
Redirecting to URLs When redirecting to a URL, Redirect will make an
HTTP callback to your application and will expect an Autopilot Actions
JSON as a response. The request will contain all the dialogue
information. This is an example of a dynamic Action since the JSON is
rendered dynamically with a URL or your own endpoint.
Can you modify the Twilio Function to return valid Action JSON to Autopilot which sets the returned data, if needed via the Remember action which you can access from Studio?
I'm building a chatbot with Twilio Autopilot and I want to get the images that I send to the bot, how can I get it? How can I get locations as well?
Thanks a lot.
Twilio developer evangelist here.
There isn't a simple way to do this, but there are a few potential work-arounds. One is to have a webhook endpoint that will get the input and if the payload contains elements of an image, then do whatever you want with it, otherwise if it is just text, then maybe send to Autopilot. That is gone over in this blog post on Autopilot enhancements in Node.js.
Another is to make a Twilio Function that would point to a Twilio Studio flow or Assets if it is media in the first message.
Another is to use Twilio Functions or a similar server. You should have an Autopilot task that redirects to that Function with JSON like this:
{
"actions": [
{
"redirect": {
"uri": "https://REPLACE-WITH-YOUR-FUNCTION-URL.twil.io/auso",
"method": "POST"
}
}
]
}
Then your Twilio Function could get the image URL with something like this in Node.js:
const bodyContent = event.MediaUrl0;
const filename = event.MessageSid + '.png';
Now in a Collect Action you can also specify the Twilio.MEDIA as the type for questions expecting Media and currently we support all media format which are supported by Twilio Messaging.
{
"question": "Please a take a picture of insurance card?",
"name": "insurance_card",
"type": "Twilio.MEDIA",
"validate": {
"allowed_types": {
"list": [
"image/jpeg",
"image/gif",
"image/png",
"image/bmp"
]
},
Lastly, you may be interested in this blog post on building an image classifier with Autopilot and TensorFlow.
Let me know if this helps at all! :D
Regarding images - As seen in this example of an autopilot task program, specify the input type to be an image
{
"actions": [
{
"collect": {
"name": "contact",
"questions": [
{
"question": "Please upload a cool picture of yourself",
"name": "contact_image",
"type": "Twilio.MEDIA"
}
],
"on_complete": {
"redirect": {
"method": "POST",
"uri": "https://url.twil.io/image-processing"
}
}
}
}
]
}
Then you can access the image as seen have done in the following function
exports.handler = function(context, event, callback) {
//we get the Memory from the answered questions.
let memory = JSON.parse(event.Memory);
//set up an array of object "actions" for the autopilot to continue.
let actions = [];
let responseItem;
//print the url of the image
let image_url = memory.twilio.collected_data.contact.answers.contact_image.media.url;
console.log(image_url);
responseItem = {
"redirect": {
"method": "POST",
"uri": "task://next_task"
}
};
actions.push(responseItem);
let respObj = {
"actions": actions
};
callback(null, respObj);
};
Autopilot Troublehooting
Unable to receive picture messages
Autopilot is currently unable to receive messages with pictures or other media types supported by Twilio on any messaging channel and will throw error with code 11200.
I haven't tried WhatsApp location data but there is a blog on the functionality that may help?
New Rich Features Support Deeper Customer Engagement on WhatsApp
I've done plenty and this is one of the last things to get the chatbot working. I'm trying to receive an image (a selfie) from the user on my WhatsApp chatbot with no success. I'm using Python with Django.
#twilio_view
def visita_selfie(request):
print('VISITA SELFIE')
message_sid = request.POST.get('MessageSid', '')
from_number = request.POST.get('From', '')
num_media = int(request.POST.get('NumMedia', 0))
media_files = [(request.POST.get("MediaUrl{}".format(i), ''),
request.POST.get("MediaContentType{}".format(i), ''))
for i in range(0, num_media)]
print('media files')
print(media_files)
memory = json.loads(request.POST.get('Memory'))
print('Memory= ',memory)
return HttpResponse(status=200)
At first I tried something like:
{
"actions": [
{
"say": "Please send me the selfie."
},
{
"listen": true
},
{
"redirect" : "my webhook"
}
]
}
but that didn't work.
I got nothing on my logs and the Twilio Debugger showed me:
Request Inspector
- Collapse All
POST
https://channels.autopilot.twilio.com/v1/AC0de44f5c63d50ab672ae85bef67ff3bc/UA313a4235eb1ee7cc39ce2ec1781d911a/twilio-messaging/whatsapp
2019-11-12 17:54:05 UTC
400
Request
URL
https://channels.autopilot.twilio.com/v1/AC0de44f5c63d50ab672ae85bef67ff3bc/UA313a4235eb1ee7cc39ce2ec1781d911a/twilio-messaging/whatsapp
Parameters
Show Raw
ApiVersion
"2010-04-01"
MediaUrl0
"https://api.twilio.com/2010-04-01/Accounts/AC0de44f5c63d50ab672ae85bef67ff3bc/Messages/MMd9aa12480e0bebd90ace8a8d5e46cca9/Media/MEaf6ae55ca8c5b27defdef15698a182f8"
SmsSid
"MMd9aa12480e0bebd90ace8a8d5e46cca9"
SmsStatus
"received"
SmsMessageSid
"MMd9aa12480e0bebd90ace8a8d5e46cca9"
NumSegments
"1"
From
"whatsapp:+553584311599"
To
"whatsapp:+14155238886"
MessageSid
"MMd9aa12480e0bebd90ace8a8d5e46cca9"
Body
""
AccountSid
"AC0de44f5c63d50ab672ae85bef67ff3bc"
NumMedia
"1"
MediaContentType0
"image/jpeg"
Message Text
Show Raw
Msg
"Bad Request"
sourceComponent
"14100"
ErrorCode
"11200"
EmailNotification
"false"
httpResponse
"400"
LogLevel
"ERROR"
url
"https://channels.autopilot.twilio.com/v1/AC0de44f5c63d50ab672ae85bef67ff3bc/UA313a4235eb1ee7cc39ce2ec1781d911a/twilio-messaging/whatsapp"
Response
Headers
X-Cache MISS from Twilio-Cache
X-Cache-Lookup MISS from Twilio-Cache:3128
X-Twilio-WebhookAttempt 1
X-Twilio-WebhookRetriable false
Content-Length 66
Date Tue, 12 Nov 2019 17:54:06 GMT
Content-Type application/json
Body
Show Raw
{
"code": 20001,
"status": 400,
"message": "Missing/Invalid Parameters"
}
Then I tried with Collect:
{
"actions": [
{
"collect": {
"name": "collect-selfie",
"questions": [
{
"question": "Para que possamos identificá-lo, poderia nos enviar uma foto do rosto? (selfie)\n",
"name": "selfie"
}
],
"on_complete": {
"redirect": {
"method": "POST",
"uri": "https://dimep.herokuapp.com/selfie/"
}
}
}
}
]
}
I didn't work either, my log only showed me the 'VISITA SELFIE' (meaning it got to the function) when I typed in text with the collect, but the media_files was empty [].
What am I doing wrong?
Twilio developer evangelist here.
I'm afraid Twilio Autopilot doesn't support media right now, so you won't be able to pick up the image from the webhook like this.
I've managed to create a simple interactive button Slack app using a Google Apps Script (GAS).
I know how to replace the original message w/ the response, but I would like to replace only the buttons, as demonstrated (but not clearly explained) multiple places in the Slack Interactive Button documentation:
https://api.slack.com/docs/message-buttons#crafting_your_message
I'd like to do what's demonstrated here:
https://a.slack-edge.com/dcb1/img/api/message_guidelines/Example_6.gif
Is this an update of the original message, a replacement of the original message with identical text but different attachment, ...?
My current interactive buttons message code looks like this:
function sendMsgWithButton() {
// slack channel url (where to send the message)
var slackUrl = "https://hooks.slack.com/services/...";
// message text
var messageData = {
"text": "Here's your interactive buttons message.",
"attachments": [
{
"text": "Can you click the button?",
"fallback": "Sorry, no support for buttons.",
"callback_id": "ptNotificationButtonResponse",
"color": "#3AA3E3",
"attachment_type": "default",
"actions": [
{
"name": "userResponse",
"text": "OK",
"style": "primary",
"type": "button",
"value": "ok"
}
]
}
]
}
// format for Slack
var options = {
'method' : 'post',
'contentType': 'application/json',
// Convert the JavaScript object to a JSON string.
'payload' : JSON.stringify(messageData)
};
// post to Slack
UrlFetchApp.fetch(slackUrl, options);
}
My current action URL code right now looks like this:
function doPost() {
var replyMessage = {"replace_original": true,
"response_type": "in_channel",
"text": "I see you clicked the button."
};
return ContentService.createTextOutput(JSON.stringify(replyMessage)).setMimeType(ContentService.MimeType.JSON);
}
Instead of replacing the entire original message, I'd like to replace just the buttons with something like a checkbox and confirmation message as demonstrated in the gif above.
Thanks!
You can only replace the complete message, not just a part.
There are two options to update the original message:
Respond to the Slack request with {"replace_original": true}
Use chat.update
If your original message was not of type ephemeral you will get a copy of the original message as part of the payload from Slack in the original_message property, which can be helpful to update the exchange the original message.
See this page in the Slack documentation as reference.