I'm trying to send a second source of audio (streamed from a bluetooth connected device as Int16 data flow) to a room using twilio-video on iOS.
I already looked at TVIAudioController but found nothing about audioInput.
Checked-out also TVIAudioSink but I don't know if it can do the job.
Anyone can help ?
Twilio developer evangelist here.
I'm not an expert in this at all, but since no-one else has helped yet, I can try to point you in the right direction. It sounds like you're trying to hack around the limitation of using one microphone on an iOS device at a time. Good luck!
In order to send another audio track, you need to create a TVILocalAudioTrack. Normally this is initialised using the track method, which picks the audio input itself. It's not recommended to use the regular initializer, but that is probably worth exploring.
Once you have your TVILocalAudioTrack you need to add it to your room Participant using addAudioTrack.
Let me know if that helps at all.
Related
I am trying to build an iOS application that streams audio coming directly from the input (or mic) of a device. What I am thinking is that every certain period of time, I'd have to send the audio buffer to the server, so that the server sends it to another client that might want to listen. I am planning to use WebSockets for the server-side implementation.
Is there a way to grab just a specific stream of buffer from the input (mic) of the iOS device and send it to the server while the user speaks another bit and so on and so forth? I am thinking that if I could start an AVAudioRecorder perhaps with AVAudioEngine and record every 1 second or half a second, but I think that that would create too much of a delay and possibly lost streams in the transition process.
Is there a better way to accomplish this? I am really interested in understanding the science behind it. If this is not the best approach please tell me which one it is and maybe a basic idea for its implementation or something that could point me in the right direction.
I found the answer to my own question!! The answer lies in the AVFoundation framework, specifically AVCaptureAudioDataOutput and its delegate that will send you a buffer as soon as the input source captures it.
I am using julius speech recognition for my application. I have one doubt regarding julius:
I have downloaded the latest version and was successful in using its lib and making it work. the problem I am facing is..once the app starts and I call the voice recognition function in my application...it takes the input from mic and displays whatever is said in the mic, but the function still continues to do so again and again. The control will never come out of that function. Here I am facing problem since the control is not returning back I am not able to proceed further. What I want is once the engine gets input from mic it should recognize and stop there.. which I tried to do by deleting the callback function but was unsuccessful.
Can anyone please guide me in this matter, what I need to do to get the desired output. It will be helpful for me.
As discussed in the same post on VoxForge:
You have a couple of choices: first to use the Julius -input control to get the sound data from a list of files (see the .jconf sample file), so that when the list (even if only length one) is exhausted then Julius stops. It is quite easy to record the voice input to a file and then feed the file into Julius. Second you can put a dialog manager in control. If you need more information on what a dialog manager does there are many posts on this forum on that subject accessible by a search.
The basic function of Julius is to start up and then keep on decoding input. When you get more experience you can run Julius as a server, and then tell the server to respond, not respond or shut down as required. It's more efficient than having Julius start and stop all the time.
When an avenue exists for a complex application to yield the required result by using an effective combination of options at run time, editing the application, while possible, might involve a lot of unnecessary work. The emphasis then shifts to passing the options correctly in whatever script is being used to access Julius.
I believe the answer is no, but does Twilio provide ability to pause/resume a recording? Use case is recording a call, but pausing recording when collecting sensitive information. From the REST documentation, it doesn't appear to be a supported capability. Thought someone might have found some options for this requirement.
This is possible, though it's not wholly obvious how from the documentation.
You can modify call state using the REST API, as per https://www.twilio.com/docs/api/rest/change-call-state , and we basically use it to tell the call to re-dial to the same agent (presumably this is a call centre?) but with no-record, and then again with record re-activated once we're done.
You end up with two separate recordings for the call, which in our case we download, stitch together, and store back to our storage platform.
Edit:
Having discussed this issue with Twilio support, there's another possibility which allows you to just have a single recording.
Instead of dialling the two ends of the call together, you instead put them both into a conference that's recorded when you initially connect the call. When you want to pause it, using the REST API, you add a new "hold" leg into the conference, then move the two real ends of the call onto a new conference that isn't recorded. When you're done, you move them back again and it's "unpaused". You then only have one recording from the original conference.
None of these is ideal, and apparently they are working on a proper support setup for this (fairly obvious!) requirement, but this should solve it for now.
The Recording Pause & Resume feature is now supported in the Twilio API. Here's a link that gets you started:
https://support.twilio.com/hc/en-us/articles/360010199074-Getting-Started-with-Call-Recording-Controls#pause_resume
I would like to know if it is posible to cast the audio taken directly from the microphone iOS device to the receiver. (in a live way)
I´ve downloaded all the git example projects, and in all of them use a "loadMedia" method to start the casting. Example of one of those:
- (NSInteger)loadMedia:(GCKMediaInformation *)mediaInfo
autoplay:(BOOL)autoplay
playPosition:(NSTimeInterval)playPosition;
Can I follow this approach to do what I want? If so, what´s the expected delay?
Thanks a lot
Echo is likely if the device (iOS, Android, or Chrome) is in range of the speakers. That said:
Pick a fast codec that is supported, such as CELT/Opus or Vorbis
I haven't tried either of these, but they should be possible.
Implement your own protocol using CastChannel that passes the binary data. You'll want to do some simple conversion of the stream from Binary to something a bit more friendly. Take a look at Intro to Web Audio for using AudioContext.
or, 2. Setup a trivial server to stream from on your device, then tell the Receiver to just access that local server.
I've used Twilio to detect whether an outbound call gets a human or answering machine response, but now I'm trying to determine when I'm on hold and when the rep eventually picks up. The goal is to have a flag to know when I'm no longer on hold with the party I called.
Is this simply impossible with Twilio? Do you know of any work-arounds or other services that I might explore?
There's nothing built into the Twilio API specific to this. You could maybe get away with playing a repeating prompt for someone to press a digit. The on hold time would never complete the prompt, but when a human picks up they would be able to and it would indicate their presence.