Receiving and handling media messages in Twilio-dialogflow WhatsApp integration - twilio

I have deployed a server on GCP to receive message traffic from twilio via webhook and integrated it with Google's dialogflow. You can see the original project here "https://github.com/GoogleCloudPlatform/dialogflow-integrations#readme".
The function works fine for receiving and responding via intent detection but it can't handle any media inputs from a user as dialogflow can't interpret it. I've been trying to code a simple IF statement that converts any input media into URL's prior to processing by dialogflow. The full code server.js file is given below:
const express = require('express');
const request = require('request');
const app = express();
const dialogflowSessionClient = require('../botlib/dialogflow_session_client.js');
const bodyParser = require('body-parser');
app.use(bodyParser.json());
app.use(bodyParser.urlencoded({ extended: true }));
const projectId = 'PROJECT-ID';
const phoneNumber = "+1##########";
const accountSid = '*********************';
const authToken = '*******************';
const client = require('twilio')(accountSid, authToken);
const MessagingResponse = require('twilio').twiml.MessagingResponse;
const sessionClient = new dialogflowSessionClient(projectId);
const listener = app.listen(process.env.PORT, function() {
console.log('listner marker');
console.log('Your Twilio01 integration server is listening on port '+ listener.address().port);
});
app.post('/', async function(req, res) {
const body = req.body;
const text = body.Body;
const id = body.From;
console.log('body marker');
const dialogflowResponse = (await sessionClient.detectIntent(text, id, body)).fulfillmentText;
const twiml = new MessagingResponse();
const message = twiml.message(dialogflowResponse);
res.send(twiml.toString());});
process.on('SIGTERM', () => {
listener.close(() => {
console.log('Closing http server.');
process.exit(0);
});
});
I have tried to add my IF statement like below but it fails to execute when placed into the main file.
if (MessagingResponse.NumMedia != "0") {
console.log(MessagingResponse.MediaUrl0);
MessagingResponse = MessagingResponse.MediaUrl0;
console.log(response.toString());
}

you cannot change the const variable value on the next statement so
change your
const text;
to
var text;
and add
if(body.MediaUrl0){
text = body.MediaUrl0;
}
before
const dialogflowResponse = (await sessionClient.detectIntent(
text, id, body)).fulfillmentText;

Twilio developer evangelist here.
You're trying to read the NumMedia and other properties from the MessagingResponse class. You should be trying to read it from the body of the incoming request: req.body.NumMedia.
Edit
Checked your code from the repo. The const still doesn't seem right. Try this:
app.post("/", async function (req, res) {
const body = req.body;
let text;
if (body.NumMedia != "0") {
text = body.MediaUrl0;
} else {
text = body.Body;
}
const id = body.From;
const dialogflowResponse = (await sessionClient.detectIntent(text, id, body))
.fulfillmentText;
const twiml = new MessagingResponse();
twiml.message(dialogflowResponse);
res.send(twiml.toString());
});
In this case we define text with let instead of const and outside of the conditional. That way we can reassign to text and use it later in the function too.

Related

Solana keeps showing "baseAccount not provided error" Whats missing?

I'm trying to call my solana contract from my frontend app.
Even though I added the baseAccount. Solana complains it's not added. Whats going on?
const baseAccount = Keypair.generate()
async function createNFT({ nftPrice, nftRoyalty, nftInfo }) {
// console.log('image', image)
// const added = await client.add(image)
// console.log(
// '`https://ipfs.infura.io/ipfs/${added.path}`',
// `https://ipfs.infura.io/ipfs/${added.path}`
// )
const provider = await getProvider()
const program = new Program(idl, programID, provider)
const price = new BN(nftPrice * 10 ** 9)
const result = await program.rpc.mintNft(
wallet.publicKey,
price,
nftRoyalty,
wallet.publicKey,
nftInfo,
{
accounts: {
baseAccount: baseAccount.publicKey,
user: provider.wallet.publicKey,
systemProgram: SystemProgram.programId,
},
signers: [baseAccount],
}
)
console.log('result', result)
}

How to Receive WhatsApp Audios Messages Using Twilio for Dialogflow

I am doing a chatbot with dialogflow-es and I connected it with twilio for whatsapp, but the problem now is that I want that dialogflow receive the audio´s user, but I don´t know how....
In that example, there is a server that receives the incoming webhook request from Twilio. It looks like this:
app.post('/', async function(req, res) {
const body = req.body;
const text = body.Body;
const id = body.From;
const dialogflowResponse = (await sessionClient.detectIntent(
text, id, body)).fulfillmentText;
const twiml = new MessagingResponse();
const message = twiml.message(dialogflowResponse);
res.send(twiml.toString());
});
You can see that the application gets the text from the message from req.body.Body and the user's WhatsApp phone number from req.body.From. When Twilio sends webhooks to this endpoint for messages it sends all the information in the body of the request. You can see all the parameters that Twilio will send in the documentation here.
In this case you are looking for media that comes with the message, so you would first look for the NumMedia parameter. If it is greater than 0 then some media has been sent with the message.
Once you know how many media items have been sent you can then access them in parameters MediaUrl{N} where N is an integer. The media is delivered as a URL, so if you need to access the file you will need to download it.
So, you might deal with this like:
app.post('/', async function(req, res) {
const body = req.body;
const text = body.Body;
const id = body.From;
const numMedia = parseInt(body.NumMedia, 10);
if (numMedia > 0) {
for (let i = 0; i < numMedia; i++) {
console.log(body[`MediaUrl${i}`]);
}
}
const dialogflowResponse = (await sessionClient.detectIntent(
text, id, body)).fulfillmentText;
const twiml = new MessagingResponse();
const message = twiml.message(dialogflowResponse);
res.send(twiml.toString());
});
I'm not sure how you would deal with the audio with respect to Dialogflow, but that's how you access it.

how can i save received image or document file from sender whatsapp using twilio?

I have a scenario like user will send the image or pdf file to twilio whatsapp number so i need to save that image/pdf in folder which will be processed to next level.
How can i save the files? I am using Node SDK.
Thanks in advance.
Assuming you've already configured your webhook on your sandbox page, so that messages containing media from Whatsapp are getting to your app.
As documentation says, you'll receive MediaContentType{N} and MediaUrl{N} as long as body and other parameters. The following snippet was translated-ish from a Python example from official documentation:
const Fs = require('fs')
const Path = require('path')
const Axios = require('axios')
const num_media = req.body.NumMedia;
const media_files = []
for (let i = 0; i <= num_media; i++) {
const id = req.body.MessageSid
const media_url = req.body[`MediaUrl{i}`];
const mime_type = req.body[`MediaContentType{i}`);
media_files.push({'media_url': media_url, 'mime_type': mime_type});
download(media_url, id);
}
async function download(url, name) {
const path = Path.resolve(__dirname, 'files', name)
const writer = Fs.createWriteStream(path)
const response = await Axios({
url,
method: 'GET',
responseType: 'stream'
})
response.data.pipe(writer)
return new Promise((resolve, reject) => {
writer.on('finish', resolve)
writer.on('error', reject)
})
}

Run firebase cloud function when a key is a specific value

const functions = require('firebase-functions');
var IAPVerifier = require('iap_verifier');
const admin = require('firebase-admin');
admin.initializeApp(functions.config().firebase);
exports.verifyReceipt = functions.database.ref('/Customers/{uid}/updateReceipt')
.onWrite(event => {
const uid = event.params.uid;
var receipt = event.data.val();
(strReceipt).toString('base64');
var client = new IAPVerifier('IAP_secretkey')
client.verifyAutoRenewReceipt(receipt, true,function(valid, msg, data){
console.log(' RECEIPT');
if(valid) {
console.log('VALID RECEIPT');
console.log('msg:' + msg);
var strData = JSON.stringify(data);
console.log('data"' + strData);
const newReceiptRef = admin.database().ref('/Customers/{uid}/');
newReceiptRef.update({'receiptData1': data});
const recVerRef = admin.database().ref('/Customers/{uid}/');
newReceiptRef.update({'updateReceipt': 0});
// update status of payment in your system
}else{
console.log('INVALID RECEIPT');
console.log('msg:' + msg);
var strData = JSON.stringify(data);
console.log('data"' + strData);
}
});
});
This is my node js cloud function. The possible values for 'updateReceipt' are 0 and 1. Is it possible to run the cloud function only when the value is 1?
Thanks.
There is no way to only trigger the function when a specific value is present.
I can think of two options:
Write the nodes to a different branch depending on the updateReceipt value.
Add an if to your code.
The second options is definitely the simplest:
exports.verifyReceipt =
functions.database.ref('/Customers/{uid}/updateReceipt')
.onWrite(event => {
const uid = event.params.uid;
var receipt = event.data.val();
if (receipt.updateReceipt === 0) {
var client = new IAPVerifier('IAP_secretkey')
...
Alternatively, you can keep the updated receipt in a separate branch from the new receipts. That way you can trigger a function separately for just the new receipts.

How do I get request parameters URLs with Node.js [duplicate]

Can we get the variables in the query string in Node.js just like we get them in $_GET in PHP?
I know that in Node.js we can get the URL in the request. Is there a method to get the query string parameters?
Since you've mentioned Express.js in your tags, here is an Express-specific answer: use req.query. E.g.
var express = require('express');
var app = express();
app.get('/', function(req, res){
res.send('id: ' + req.query.id);
});
app.listen(3000);
In Express it's already done for you and you can simply use req.query for that:
var id = req.query.id; // $_GET["id"]
Otherwise, in NodeJS, you can access req.url and the builtin url module to url.parse it manually:
var url = require('url');
var url_parts = url.parse(request.url, true);
var query = url_parts.query;
In Express, use req.query.
req.params only gets the route parameters, not the query string parameters. See the express or sails documentation:
(req.params) Checks route params, ex: /user/:id
(req.query) Checks query string params, ex: ?id=12 Checks urlencoded body params
(req.body), ex: id=12 To utilize urlencoded request bodies, req.body should be an object. This can be done by using the _express.bodyParser middleware.
That said, most of the time, you want to get the value of a parameter irrespective of its source. In that case, use req.param('foo'). Note that this has been deprecated as of Express 4: http://expressjs.com/en/4x/api.html#req.param
The value of the parameter will be returned whether the variable was in the route parameters, query string, or the encoded request body.
Side note- if you're aiming to get the intersection of all three types of request parameters (similar to PHP's $_REQUEST), you just need to merge the parameters together-- here's how I set it up in Sails. Keep in mind that the path/route parameters object (req.params) has array properties, so order matters (although this may change in Express 4)
For Express.js you want to do req.params:
app.get('/user/:id', function(req, res) {
res.send('user' + req.params.id);
});
I learned from the other answers and decided to use this code throughout my site:
var query = require('url').parse(req.url,true).query;
Then you can just call
var id = query.id;
var option = query.option;
where the URL for get should be
/path/filename?id=123&option=456
//get query&params in express
//etc. example.com/user/000000?sex=female
app.get('/user/:id', function(req, res) {
const query = req.query;// query = {sex:"female"}
const params = req.params; //params = {id:"000000"}
})
If you are using ES6 and Express, try this destructuring approach:
const {id, since, fields, anotherField} = request.query;
In context:
const express = require('express');
const app = express();
app.get('/', function(req, res){
const {id, since, fields, anotherField} = req.query;
});
app.listen(3000);
You can use default values with destructuring too:
// sample request for testing
const req = {
query: {
id: '123',
fields: ['a', 'b', 'c']
}
}
const {
id,
since = new Date().toString(),
fields = ['x'],
anotherField = 'default'
} = req.query;
console.log(id, since, fields, anotherField)
There are 2 ways to pass parameters via GET method
Method 1 :
The MVC approach where you pass the parameters like /routename/:paramname
In this case you can use req.params.paramname to get the parameter value For Example refer below code where I am expecting Id as a param
link could be like : http://myhost.com/items/23
var express = require('express');
var app = express();
app.get("items/:id", function(req, res) {
var id = req.params.id;
//further operations to perform
});
app.listen(3000);
Method 2 :
General Approach : Passing variables as query string using '?' operator
For Example refer below code where I am expecting Id as a query parameter
link could be like : http://myhost.com/items?id=23
var express = require('express');
var app = express();
app.get("/items", function(req, res) {
var id = req.query.id;
//further operations to perform
});
app.listen(3000);
You should be able to do something like this:
var http = require('http');
var url = require('url');
http.createServer(function(req,res){
var url_parts = url.parse(req.url, true);
var query = url_parts.query;
console.log(query); //{Object}
res.end("End")
})
UPDATE 4 May 2014
Old answer preserved here: https://gist.github.com/stefek99/b10ed037d2a4a323d638
1) Install express: npm install express
app.js
var express = require('express');
var app = express();
app.get('/endpoint', function(request, response) {
var id = request.query.id;
response.end("I have received the ID: " + id);
});
app.listen(3000);
console.log("node express app started at http://localhost:3000");
2) Run the app: node app.js
3) Visit in the browser: http://localhost:3000/endpoint?id=something
I have received the ID: something
(many things have changed since my answer and I believe it is worth keeping things up to date)
Express specific simple ways to fetch
query strings(after ?) such as https://...?user=abc&id=123
var express = require('express');
var app = express();
app.get('/', function(req, res){
res.send('id: ' + req.query.id);
});
app.listen(3000);
query params such as https://.../get/users/:id
var express = require('express');
var app = express();
app.get('/get/users/:id', function(req, res){
res.send('id: ' + req.params.id);
});
app.listen(3000);
A small Node.js HTTP server listening on port 9080, parsing GET or POST data and sending it back to the client as part of the response is:
var sys = require('sys'),
url = require('url'),
http = require('http'),
qs = require('querystring');
var server = http.createServer(
function (request, response) {
if (request.method == 'POST') {
var body = '';
request.on('data', function (data) {
body += data;
});
request.on('end',function() {
var POST = qs.parse(body);
//console.log(POST);
response.writeHead( 200 );
response.write( JSON.stringify( POST ) );
response.end();
});
}
else if(request.method == 'GET') {
var url_parts = url.parse(request.url,true);
//console.log(url_parts.query);
response.writeHead( 200 );
response.write( JSON.stringify( url_parts.query ) );
response.end();
}
}
);
server.listen(9080);
Save it as parse.js, and run it on the console by entering "node parse.js".
Whitequark responded nicely. But with the current versions of Node.js and Express.js it requires one more line. Make sure to add the 'require http' (second line). I've posted a fuller example here that shows how this call can work. Once running, type http://localhost:8080/?name=abel&fruit=apple in your browser, and you will get a cool response based on the code.
var express = require('express');
var http = require('http');
var app = express();
app.configure(function(){
app.set('port', 8080);
});
app.get('/', function(req, res){
res.writeHead(200, {'content-type': 'text/plain'});
res.write('name: ' + req.query.name + '\n');
res.write('fruit: ' + req.query.fruit + '\n');
res.write('query: ' + req.query + '\n');
queryStuff = JSON.stringify(req.query);
res.end('That\'s all folks' + '\n' + queryStuff);
});
http.createServer(app).listen(app.get('port'), function(){
console.log("Express server listening on port " + app.get('port'));
})
It is so simple:
Example URL:
http://stackoverflow.com:3000/activate_accountid=3&activatekey=$2a$08$jvGevXUOvYxKsiBt.PpMs.zgzD4C/wwTsvjzfUrqLrgS3zXJVfVRK
You can print all the values of query string by using:
console.log("All query strings: " + JSON.stringify(req.query));
Output
All query strings : { "id":"3","activatekey":"$2a$08$jvGevXUOvYxKsiBt.PpMs.zgzD4C/wwTsvjz
fUrqLrgS3zXJVfVRK"}
To print specific:
console.log("activatekey: " + req.query.activatekey);
Output
activatekey: $2a$08$jvGevXUOvYxKsiBt.PpMs.zgzD4C/wwTsvjzfUrqLrgS3zXJVfVRK
You can use
request.query.<varible-name>;
You can use with express ^4.15.4:
var express = require('express'),
router = express.Router();
router.get('/', function (req, res, next) {
console.log(req.query);
});
Hope this helps.
In express.js you can get it pretty easy, all you need to do in your controller function is:
app.get('/', (req, res, next) => {
const {id} = req.query;
// rest of your code here...
})
And that's all, assuming you are using es6 syntax.
PD. {id} stands for Object destructuring, a new es6 feature.
app.get('/user/:id', function(req, res) {
res.send('user' + req.params.id);
});
You can use this or you can try body-parser for parsing special element from the request parameters.
consider this url -> /api/endpoint/:id?name=sahil
here id is param where as name is query. You can get this value in nodejs like this
app.get('/api/endpoint/:id', (req, res) => {
const name = req.query.name; // query
const id = req.params.id //params
});
There are many answers here regarding accessing the query using request.query however, none have mentioned its type quirk. The query string type can be either a string or an array, and this type is controlled by the user.
For instance using the following code:
const express = require("express");
const app = express();
app.get("/", function (req, res) {
res.send(`Your name is ${(req.query.name || "").length} characters long`);
});
app.listen(3000);
Requesting /?name=bob will return Your name is 3 characters long but requesting /?name=bob&name=jane will return Your name is 2 characters long because the parameter is now an array ['bob', 'jane'].
Express offers 2 query parsers: simple and extended, both will give you either a string or an array. Rather than checking a method for possible side effects or validating types, I personally think you should override the parser to have a consistent type: all arrays or all strings.
const express = require("express");
const app = express();
const querystring = require("querystring");
// if asArray=false only the first item with the same name will be returned
// if asArray=true all items will be returned as an array (even if they are a single item)
const asArray = false;
app.set("query parser", (qs) => {
const parsed = querystring.parse(qs);
return Object.entries(parsed).reduce((previous, [key, value]) => {
const isArray = Array.isArray(value);
if (!asArray && isArray) {
value = value[0];
} else if (asArray && !isArray) {
value = [value];
}
previous[key] = value;
return previous;
}, {});
});
app.get("/", function (req, res) {
res.send(`Your name is ${(req.query.name || "").length} characters long`);
});
app.listen(3000);
So, there are two ways in which this "id" can be received:
1) using params: the code params will look something like :
Say we have an array,
const courses = [{
id: 1,
name: 'Mathematics'
},
{
id: 2,
name: 'History'
}
];
Then for params we can do something like:
app.get('/api/posts/:id',(req,res)=>{
const course = courses.find(o=>o.id == (req.params.id))
res.send(course);
});
2) Another method is to use query parameters.
so the url will look something like ".....\api\xyz?id=1" where "?id=1" is the query part. In this case we can do something like:
app.get('/api/posts',(req,res)=>{
const course = courses.find(o=>o.id == (req.query.id))
res.send(course);
});
In case you want to avoid express, use this example:
var http = require('http');
const url = require('url');
function func111(req, res) {
res.writeHead(200, {'Content-Type': 'text/plain'});
var q = url.parse(req.url, true);
res.end("9999999>>> " + q.query['user_name']);
}
http.createServer(func111).listen(3000);
usage:
curl http://localhost:3000?user_name=user1
by yl
you can use url module to collect parameters by using url.parse
var url = require('url');
var url_data = url.parse(request.url, true);
var query = url_data.query;
In expressjs it's done by,
var id = req.query.id;
Eg:
var express = require('express');
var app = express();
app.get('/login', function (req, res, next) {
console.log(req.query);
console.log(req.query.id); //Give parameter id
});
If you ever need to send GET request to an IP as well as a Domain (Other answers did not mention you can specify a port variable), you can make use of this function:
function getCode(host, port, path, queryString) {
console.log("(" + host + ":" + port + path + ")" + "Running httpHelper.getCode()")
// Construct url and query string
const requestUrl = url.parse(url.format({
protocol: 'http',
hostname: host,
pathname: path,
port: port,
query: queryString
}));
console.log("(" + host + path + ")" + "Sending GET request")
// Send request
console.log(url.format(requestUrl))
http.get(url.format(requestUrl), (resp) => {
let data = '';
// A chunk of data has been received.
resp.on('data', (chunk) => {
console.log("GET chunk: " + chunk);
data += chunk;
});
// The whole response has been received. Print out the result.
resp.on('end', () => {
console.log("GET end of response: " + data);
});
}).on("error", (err) => {
console.log("GET Error: " + err);
});
}
Don't miss requiring modules at the top of your file:
http = require("http");
url = require('url')
Also bare in mind that you may use https module for communicating over secured domains and ssl. so these two lines would change:
https = require("https");
...
https.get(url.format(requestUrl), (resp) => { ......
do like me
npm query-string
import queryString from "query-string";
export interface QueryUrl {
limit?: number;
range?: string;
page?: number;
filed?: string;
embody?: string;
q?: string | object;
order?: number;
sort?: string;
}
let parseUri: QueryUrl = queryString.parse(uri.query);
I am using MEANJS 0.6.0 with express#4.16, it's good
Client:
Controller:
var input = { keyword: vm.keyword };
ProductAPi.getOrder(input)
services:
this.getOrder = function (input) {return $http.get('/api/order', { params: input });};
Server
routes
app.route('/api/order').get(products.order);
controller
exports.order = function (req, res) {
var keyword = req.query.keyword
...

Resources