I am creating an ios application using webRTC for video conferencing. I want to detect who is talking in the peer connection.
To be more specific,I want to detect the audio activity of the remote peer I am connected to so that I can detect the person who is currently speaking.
This can be implemented by measuring the value of "audioOutputLevel" in peer-connection stats reports. The Function that will you should study is
- (void)peerConnection:(RTCPeerConnection*)peerConnection didGetStats:(NSArray*)stats
Check out this guide for building a sample WebRTC iOS application.
Check the section WebRTC Stats reporting
For example, audioSendInputLevel property indicates mic input level even while audio track disabled, so you can check if user is currently speaking/talking.
Related
I am using iOS SDK Skype for Business following are my concerns.
Latency issue - It takes lots of time to connect the call even at good network also I am keeping video service on demand default connection is only for Audio feeds.
After call connected audio feed default set to muted, didChangeIsMuted delegate returns true (Mute). User has to manually press the button to unmute it.
Latest SDK Demo at url https://github.com/OfficeDev/skype-ios-app-sdk-samples/tree/master/BankingAppSwift is not compiling successfully. Few resource files are missing (Helper files).
The Meeting join only connects to IM and Audio. It does not enable Video by default. You have to explicitly start the video service one the meeting join in complete (Conversation moves to established state).
For Latency, can you give an estimate.. how long did it take to join the meeting?
We will look at this one and reply back.
Please use the Helper files from the SDK Zip package. They are not provided by default in the GitHub samples.
I want to develop a native iOS app using WebRTC that have almost similar functionality to skype. I have few queries :
Q 1) I know we can implement Audio and Video call using WebRTC but is
it possible to implement Text Chat and Screen Sharing in iOS app
using native WebRTC ?
Q 2) I need to provide time estimation of **Audio Call,Video Call,
Group Audio call, Group Video call, single user text(share media as well) chatting, multi user text (share media as well) chatting and
Screen Sharing.
Please let me know approx. development time of individual functionality if single developer works on it, who have good hands in iOS development.
Thank you....:)
I apologise for the possibility of the title of my question would lead to confusion of the problem. For that I will explain my purpose in detail.
We are currently developing our own wifi speaker which is built with MIPS. The speaker comes with an app that will be used to manage it. One of the features that would we would like to include in the app is accessing contents of Spotify and be able to play them on the speakers.
Unfortunately, after going through the iOS SDK Documentation, and did some tests on Web API Console provided by the official of Spotify, I noticed that Spotify does not allow developers to directly get URL of a song, except for preview purposes. I also wasn't able to find any way to get the data bytes of the music streamed from the server. Every content comes with a corresponding URI which is used for a request.
For the device(WiFi Speaker) part, we recently tried to contact Spotify and ask for an SDK that can be used for development. However, one problem is that Spotify told us that they have SDK for x86, and ARMs architecture only. They don't have MIPS.
Now, here are my questions:
Is there any way for me to push music from an app to the WiFi Speakers without having to use SDK (for backend device)?
If Spotify can provide an SDK for our device, then how can we integrate the SDK with our platform?
I'll explain my 2nd question for clarity. Like for instance, in Android and iOS, these are popular platforms and are widely used by mobile devices. So if they provide SDKs for the two OS, then they can use default system frameworks to access the player for playing the content. (In iOS, it's the AVFoundation Framework). However, if Spotify were able to provide the SDK that we need, how would we able to integrate that with our own platform?
I will answer your question no 1:
You should be able to push music from an app using a buffer that you can read from using Core Audio and also forward to a device of your choice. I think what you are looking for can be found at CocoaLibSpotify
I have made an app that uses Openears framework to readout some text. But I haven't used any of Openears' speech recognition/speech synthesis features, just the talk to speech feature. My app got rejected by apple telling that the app asks for permission to use microphone while the app doesn't have any features of that kind. The following is the rejection message from apple:
During review we were prompted to provide consent to use the microphone, however, we were not able to find any features or functionality that use the microphone for audio recording.
The microphone consent request is generated by the use of either AVAudioSessionCategoryRecord or AVAudioSessionCategoryPlayAndRecord audio categories.
If you do not intend to record audio with your application, it would be appropriate to choose the AVAudioSession session category that fits your application's needs or modify your app to include audio-recording features.
For more information, please refer to the Security section of the iOS SDK Release Notes for iOS 7 GM Seed.
I have searched the app for AVAudioSessionCategoryRecord or AVAudioSessionCategoryPlayAndRecord audio categories as mentioned in the message but couldn't find any. How can I disable the prompting for permission to use microphone?
Your application got rejected because you don't need the microphone feature, openears by default interface with the use of the microphone feature hence why the user permissions came up. These user permissions are not dismissible as apple increased the security features for users so that they can be in more control of what they want their applications to be able to do. If you have to use OpenEars audio management feature for speech recognition see Update 1 otherwise continue on for a different solution using Apples Siri's Speech Synthesizer on iOS 7.
In your case, if all you want to do is read out some text, then you can use iOS7 Speech Synthesizer, which is the same synthesizer used to create Siri's voice.
It's SO easy to setup and I am currently using it for one of my projects to interact with the user via voice. Here's a quick tutorial on how to get it all setup:
Speech synthesizer tutorial
UPDATE 1
After #halle's comment, I decided to update the post for those that have to use the OpenEars framework who will be using only the FliteController Text To Speech feature without any sort of OpenEars speech recognition.
You can set the FliteController property noAudioSessionOverrides to TRUE so that you ensure that OpenEars wont interface with the Audio recording stream and this will stop the Microphone permissions alert from popping up.
[self.fliteController setNoAudioSessionOverrides:TRUE]
UPDATE 2
Based on #Halle's comment, you no longer need to do update 1:
Just an update that starting with today's update 1.65, FliteController won't ever make audio session calls on its own, so there is no further rejection danger here and it isn't necessary to set noAudioSessionOverrides.
I'm sorry your app was rejected. To use TTS only without any of the audio session management related to speech recognition in OpenEars, set FliteController's property noAudioSessionOverrides to TRUE. This will result in no audio session changes/no use of the mic stream.
I'll see if I can make the documentation for this setting a bit more prominent for developers doing TTS with OpenEars' FliteController only.
For completeness' sake, the documentation on how to greatly reduce your app binary size when using OpenEars, since that was also an issue for you:
http://www.politepix.com/forums/topic/slimming-down-your-app/
http://www.politepix.com/openears/support/#Q_How_can_I_trim_down_the_size_of_the_final_binary_for_distribution
Edit: starting with today's version 1.65 of OpenEars and its plugins, if you just use FliteController there is no danger of rejection because the TTS classes no longer make any calls to the audio session by themselves. Thanks for the heads-up about this and, again, sorry you had a rejection due to this.
I'm working in app where I need to implement a simple voice functionality between two iOS devices in same network.
My doubt is about how to get audio units from master device and send over wifi or bluetooth network directly to slave device in realtime.
Some things about network communication I have done, I can transfer any NSData between devices using TCP.
Is very important to not use GK framework, because I need to connect two clients without any notification. For example, when I use GK to connect two devices, the iOS display a alert with a connection request, I need to avoid this connection request.
Currently I can do this with video, is more simple than audio.
Any ideias about the implementation of this are welcome.
Thanks.
First of all, you need learn how to capture the audio using Audio Queue Services.
Then transmit the audio data it over wifi and bluetooth and play it at the other side.
The most obvious choice for you is to use bonjour.
GameKit framework is built on top of this. You dont have to build too much on top of this for your application though. Yours is a straightforward application of bonjour.
Please refer the code for chatty if needed. Incase you need some background theory, please refer to http://mobileorchard.com/tutorial-networking-and-bonjour-on-iphone/.
I have used Audio Queue Services for the same sort of project.
For Networking I am using Bonjour and it has really solved the problem of transmitting text and video.
Did a lot of workarounds to make a voice chat in wifi using Audio Queue Services but have not succeeded. I will update once I find the solution other than this.