I'm not able by any methods to stop the streamingRecognize. I tried to unpipe the stream, tried to unset client but I still get a 'Audio Timeout Error:'. Is there any method to stop recognizing?
When we call streamingRecognize() what is returned to us is an input stream. We then push data records through the stream for as long as we have input. When we have sent all the data we wish to send, we must instruct the speech to text processor that there is no more data to process. We do this by calling the end() method of the stream.
In your example fragment:
let recognizeStream = client
.streamingRecognize(request)
.on('error', console.error)
.on('data', data =>
{
console.log(data.results[0].alternatives[0].transcript);
recognizeStream.end();
}
);
I was able to stop it without errors in case someone has the same problem this way:
let recognizeStream = client
.streamingRecognize(request)
.on('error', console.error)
.on('data', data =>
{
console.log(data.results[0].alternatives[0].transcript);
// first stop the microphone stream
micInstance.stop();
//call again to client.streamingRecognize with request set to null
client.streamingRecognize(null);
}
);
But now I get a Error Response: [4] DEADLINE_EXCEEDED at node_modules/grpc/src/common.js:91:15)
Related
I'm using an incoming call flow that starts call recording, asks a bunch of questions, gathers responses, at the end of the call: stops recording and sends a sms to caller using send/wait for reply widget. A response is expected and based on what's in the body of the incoming call, it calls a function.
All this works, except, I am not receiving a response back from the caller.
My concurrent call setting =off
incoming trigger = incoming call
the flow is tied to a phone number (voice)
I'm not sure how to get a reply back into the same flow. Do I need to attach something to the message section of the phone number?
Any guidance would be appreciated
A Studio flow execution represents one call, one SMS, or the incoming call trigger from the REST Trigger. As the call initiates your flow, it will terminate when the call ends.
But you can work around this by using a function that gets invoked when the recording is done. This function can then use the Twilio APIs to fetch contextual information from the call and trigger the REST API interface of the same flow (but with a different trigger).
I created a small example that does something similar:
The flow is triggered by a call, starts a recording, and gathers data
There is a recording callback URL that points to my function
// This is your new function. To start, set the name and path on the left.
exports.handler = function (context, event, callback) {
console.log(`Recording ${event.RecordingSid} state changed to ${event.RecordingStatus}.`)
if (event.RecordingStatus === "completed") {
const client = context.getTwilioClient();
return client.calls(event.CallSid).fetch()
.then(call => {
client.studio.v2.flows(<Flow ID>)
.executions
.create({
to: call.from,
from: <YOUR NUMBER>,
parameters: {
RecordingUrl: event.RecordingUrl,
}
})
.then(execution => {
console.log(`Triggered execution ${execution.sid}`)
return callback(null, "OK");
});
})
.catch(callback)
}
return callback(null, "OK");
};
You can find the ID of your flow in the console (or when you click on the root element and check the Flow Configuration):
The REST API triggers a second flow execution that reads the parameter and uses them to send a text message:
Business Impact: Not being able to use the Pub/Sub job update notifications feature. Issue Summary: We (Recall) are trying to use the Pub/Sub notifications for job updates (https://cloud.google.com/transcoder/docs/how-to/create-pub-sub).
However, since the PubsubMessage data is serialized into bytes [1], without knowing the proto type, we cannot deserialize it [2]. (Unless the intention is to treat the data as a string and parse from there?)
Based off the message we see published in the Pub/Sub topic:
data: "{\"job\":{\"name\":\"projects/PROJECT_NUMBER/locations/us-central1/jobs/JOB_ID\",\"state\":\"SUCCEEDED\",\"failureReason\":null}}"
message_id: "2356400951506061"
publish_time {
seconds: 1620063162
nanos: 430000000
}
We were guessing that the type is google.cloud.video.transcoder.v1beta1.Job [3], but parsing the message into that type throws InvalidProtocolBufferException.
Is this the type we should be using to deserialize the message? Any tips to help parse the published message would be helpful.
Thanks!
Pubsub message in v1beta1 doesn't refer to any protos. The data is in json form so I think you could "treat the data as a string and parse from there".
The message in transcoder v1 will be in proto format. The doc has not been updated yet. Please check later
https://cloud.google.com/pubsub/docs/pull
Here is documentation, they have mentioned we need to convert the buffer to string and then we can parse the data.
// Create an event handler to handle messages
let messageCount = 0;
const messageHandler = message => {
console.log(`Received message ${message.id}:`);
console.log(`\tData: ${message.data}`);
console.log(`\tAttributes: ${message.attributes}`);
messageCount += 1;
// "Ack" (acknowledge receipt of) the message
message.ack();
};
// Listen for new messages until timeout is hit
subscription.on('message', messageHandler);
I have tried the same
converting buffer to a string
parsing the string
let stringData = message.data.toString();
console.log(stringData);
let jsonData = JSON.parse(stringData);
console.log(jsonData);
This worked for me in javascript
I'm using Twilio Programmable Voice to process phone calls.
I want to use bi-directional stream feature to send some raw audio data to play by twilio, the initialization code looks like,
from twilio.twiml.voice_response import Connect, VoiceResponse, Stream
response = VoiceResponse()
connect = Connect()
connect.stream(url='wss://mystream.ngrok.io/audiostream')
response.append(connect)
Then when got wss connection from twilio, I start to send raw audio data to twilio, like this
async def send_raw_audio(self, ws, stream_sid):
print('send raw audio')
import base64
import json
with open('test.wav', 'rb') as wav:
while True:
frame_data = wav.read(1024)
if len(frame_data) == 0:
print('no more data')
break
base64_data = base64.b64encode(frame_data).decode('utf-8')
print('send base64 data')
media_data = {
"event": "media",
"streamSid": stream_sid,
"media": {
"playload": base64_data
}
}
media = json.dumps(media_data)
print(f"media: {media}")
await ws.send(media)
print('finished sending')
test.wav is a wav file encoded audio/x-mulaw with a sample rate of 8000.
But when run, I can't hear anything, and on twilio console, it said
31951 - Stream - Protocol - Invalid Message
Possible Causes
- Message does not have JSON format
- Unknown message type
- Missing or extra field in message
- Wrong Stream SID used in message
I have no idea which part is wrong. Does anyone know what's my problem? I can't find an example about this scenario, just follow instructions here, really appreciate it if someone knows there is an example about this, thanks.
Not sure if this will fix it but I use .decode("ascii"), not "utf-8"
Question is probably not relevant anymore, but I came across this while debugging my bi-directional stream, so it might be useful for someone:
Main reason why were you receiving this error because of the typo in json content. You are sending "playload" instead of "payload".
Another issue when sending data to twilio stream is that you should send mark message at the end of data stream to notify twilio that complete payload was sent. https://www.twilio.com/docs/voice/twiml/stream#message-mark-to-twilio
When sending data back to twilio stream, be aware that payload should not contain audio file type header bytes, so make sure you remove them from your recording or alternatively skip them while sending data to twilio.
On Swift, I use
socket.on("test") {data, ack in
print(data)
}
In order to subscribe to a room (socket) on my Sails.js API.
When I broadcast a message from the server, with
sails.sockets.broadcast('test', { text : 'ok' })
the socket.on handler is never called.
However, if I set "log" TRUE to config when connecting my socket.io client from swift, in Socket-IO logs the message arrives.
What's wrong?
Eventually, I found my mistake:
The whole process I did is right:
(The request to join the room is done by the server, with sails.sockets.join)
Wrong thing was using socket.on with the ROOM NAME parameter.
I will explain it better, for others having same problem:
From Swift you should subscribe by making a websocket request to an endpoint on the server that accepts websocket requests (GET, POST, PUT). For example, you can make a POST request, passing in the room name into the body.
socket.emitWithAck("post", [
"room": "testroom",
"url": "/api/v1.0/roomsubscribing"
]).timingOut(after: 0) {data in
print("Server responded with \(data)")
}
On server side, inside the room-subscribing endpoint, you should have the following code:
roomSubscribing: function(req, res) {
if (!req.isSocket) {
return res.badRequest();
}
sails.sockets.join(req, req.params('room'), function(err) {
if (err) {
return res.serverError(err);
}
});
}
When the server want to broadcast some data to subscribers of the "testroom" room, the following code must be used:
sails.sockets.broadcast('testroom', { message: 'testmessage' }
Now on the swift's side you must use:
socket.on("message") { data, ack in
print(data)
}
in order to get the message handler to work. I thought you should use room name, instead you should use the KEY of the KEY/VALUE entry you used in your server when you broadcasted the data (in this case, "message").
I only have a small amount of experience with sockets, but in case nobody else answers...
I think you are missing step one of the three step socket process:
A client sends a message to the server asking to subscribe to a particular room.
The client sets up a socket.on to handle particular events from that room.
The server broadcasts an event in a particular room. All subscribers/clients with a .on for that particular event will react.
I could be wrong, but it sounds from your description like you missed step one. Your client has to send a message with io.socket, something like here, then your server has to use the socket request to have them join the room, something like in the example here.
(the presence of log data without the socket.on firing would seem to confirm that the event was broadcast in the room, but that client was not subscribed)
Good luck!
I have a messaging app and I'm using firebase Offline, When I updatechildvalues, it goes to the local cache and then to the server. However, when i turn off my internet, it only goes to the local cache, and if i close the app, the data is lost.
is there a way to determine if something was saved to the server instead of the local cache?
if not, what is the most effective way, to determine if something was successfully sent to the server.
I tried an observe value, but it also observes offline updates.
A completion listener will fire only when the data is written to the server. See an example:
let message = ["name": "puf", "text": "Hello from iOS"]
ref!.childByAutoId().setValue(message) { (error) in
print("Error while writing message \(error)")
}
But if there's been an app restart in between the write operation and the commit to server, that callback won't be fired. There is no workaround for that at the moment.
I had the same problem, and it can be solved using cloud function. Basically, just create a cloud function called when your message is created, and set a "persisted" field to true to this message. It will propagate to your client, who will treat any message without the "persisted" property as a message not sent yet.
exports.confirmMessageSave = functions.database
.ref('/chat/messages/{channelId}/{messageId}')
.onCreate(function(event) {
var message = event.data.val();
var updateFanOut = {};
updateFanOut['/messages/' + event.params.channelId + '/' + event.params.messageId + '/persisted'] = true;
return admin.database().ref('/chat').update(updateFanOut);
});