Native video in Air on iOS stopping sound and microphone - ios

I'm trying to play a native video in a StageWebView in an Air for iPad app that plays sound and requires user interaction via the microphone.
Everything seems to work, but when I start playing the video, sound and microphone stop.
If I dispose the StageWebView, sound and mic get back but only after 15 seconds (and I need sound and mic to work at least straight after the StageWebView is released).
I tried to get this work on an iOS5 iPad1, and on an iOS6 iPad2, using Air 3.4, 3.5 and 3.6 beta. I tried to switch the mute button of the iPad, and I also tried to change the SoundMixer.audioPlaybackMode to Media and Ambient.
But it didn't worked and I'm stucked.
Here is my code that deals with the microphone :
var microphone:Microphone = Microphone.getMicrophone();
microphone.addEventListener(SampleDataEvent.SAMPLE_DATA, __micHandler);
private function micHandler(event : SampleDataEvent) : void {
trace("mic is working !");
}
For the audio :
_snd = new Sound();
_snd.load(new URLRequest(path));
_sndChannel = _snd.play();
private function soundStopHandler(event : MouseEvent) : void {
if(_sndChannel) _sndChannel.stop();
}
And for the video player :
_webview = new StageWebView();
_webview.stage = stage;
_webview.viewPort = new Rectangle(10, 120, 480, 300);
_webview.loadURL(path);
private function videoStopHandler(event : MouseEvent) : void {
if(_webview) {
_webview.dispose();
_webview = null;
}
}
Did anyone faced this problem before me ? Is there anything I forgot or did in a wrong way ?

Maybe this problem is related to the iOS System.
In iOS perspective, as far as I know, officially it is not possible. When you start the recording sessions, except for recording does not occupy it. Once you playback and recording sessions in the session must be solved. recording and playback session, the session can not be occupied simultaneously.
refer a apple documentation: Audio Session
AVAudioSessionCategoryPlayAndRecord or the equivalent kAudioSessionCategory_PlayAndRecord—Use this category for an application that inputs and outputs audio. The input and output need not occur simultaneously, but can if needed. This is the category to use for audio chat applications.
In AIR, if you code no problem. most likely this reason.

Related

Mono audio output in iOS app when using a webRTC powered video call

The app i'm writing contains 2 parts:
An audio player that plays stereo MP3 files
Video conferencing using webRTC
Each part works perfectly in isolation, but the moment i try them together, one of two things happens:
The video conference audio fades out and we just hear the audio files (in stereo)
We get audio output from both, but the audio files are played in mono, coming out of both ears equally
My digging had taken me down a few routes:
https://developer.apple.com/forums/thread/90503
&
https://github.com/twilio/twilio-video-ios/issues/77
Which suggest that the issue could be with the audio session category, mode or options.
However i've tried lots of the combos and am struggling to get anything working as intended.
Does anyone have a better understanding of the audio options to point in the right direction?
My most recent combination
class BBAudioClass {
static private var audioCategory : AVAudioSession.Category = AVAudioSession.Category.playAndRecord
static private var audioCategoryOptions : AVAudioSession.CategoryOptions = [
AVAudioSession.CategoryOptions.mixWithOthers,
AVAudioSession.CategoryOptions.allowBluetooth,
AVAudioSession.CategoryOptions.allowAirPlay,
AVAudioSession.CategoryOptions.allowBluetoothA2DP
]
static private var audioMode = AVAudioSession.Mode.default
static func setCategory() -> Void {
do {
let audioSession: AVAudioSession = AVAudioSession.sharedInstance()
try audioSession.setCategory(
BBAudioClass.audioCategory,
mode: BBAudioClass.audioMode,
options: BBAudioClass.audioCategoryOptions
)
} catch {
};
}
}
Update
I managed to get everything working as i wanted by:
Starting the audio session
Connecting to the video conference (at this point all audio is mono)
Forcing all output to the speaker
Forcing output back to the headphones
Obviously this is a crazy thing to have to do, but does prove that it should work.
But it would be great if anyone knew WHY this works, in order that i can actually get things to work properly first time without going through all these hacky steps

Switch to default audio input device when it's connected in js

I'm using twilio sdk to implement web calling app. Let's say I'm making a call to someone with my laptop devices (mic and speakers). During the call I plugged-in my headset. In the system both audio input and output devices are changed. The call audio output signal is transferred fine (I can here my counterpart through my headphones). But the audio input device stays the same - app doesn't start to use the mic on my headset.
It there any way to update audio input track to switch to a headset once it's connected?
First of all, get local praticipant tracks and filter audio track.
const publications = Array.from(this.participant.tracks.values());
const audioPublication = publications.find(item => item.kind === 'audio');
Then set the deviceId like this.
const constraints = { deviceId: { exact: deviceId } };
audioPublication.track.restart(constraints);

Unity/Videoplayer - Game sometimes crash if I load next video

I use Video Player which attached 3D game object. This game object contain "Mesh Renderer" component too. Video Player Renderer-> this game object.
I try many method, but always crash game.
I try use in Update: Videoplayer().isplaying if false, then I load from Resources next video and play, but not working. (Unity Editor working fine, but in iPhone sometimes crash game.)
Then I try other method..
But sometimes crash too..
Okey, I think maybe loading and start playing too fast, maybe this cause the crash.
Then I try use Ienumerator waitforsecond every step, for debugging I set it 5 seconds.
I try first set video player.clip = null, maybe this is the problem.. But not..
Now the code:
void Start()
{
index = 0;
videoPlayer = gameObject.GetComponent<VideoPlayer>();
videoPlayer.loopPointReached += CheckOver;
}
IEnumerator Waitforseconds()
{
Debug.Log("Waitforseconds start");
videoPlayer.clip = null;
Debug.Log("Videoplayer clip null);
yield return new WaitForSeconds(5);
videoPlayer.clip = theVideo;
Debug.Log("Videoplayer set the video");
yield return new WaitForSeconds(5);
Debug.Log("Next play video player");
videoPlayer.Play();
Debug.Log("Play video player");
isPlaying = true;
}
void CheckOver(UnityEngine.Video.VideoPlayer vp)
{
Debug.Log("Checkover");
index += 1;
if (index >= videos.Length)
{
index = 0;
}
theVideo = videos[index];
Debug.Log("Set the video");
StartCoroutine(Waitforseconds());
}
The videos size 5-10 MB and 10-60 seconds.
The result: Unity Editor working good, I build to iOS and sometimes crash the game if I play next video (iPhone 12 Pro Max, supposedly the fastest smartphone chip, so if that doesn't work, there's a big problem..)
Debug:
I get last Debug.log before the game crash:
"Videoplayer clip null"
So I think the game crash here: videoPlayer.clip = theVideo;
But only iOS! Unity editor working good.
Xcode error:
I try find solution in google, but I can't find..
Maybe memory error? should I somehow delete the previous video from memory?
Or how can I solve this problem? Please help me.
Thank you very much!
EDIT: I see the problem is Audio. (Audio Output mode: Audio Source, this component attached to same gameobject. So same game object contain audio source and video player.) If I set Audio Output mode to "none", then game not crash, workin good. So the problem when "PrepareAudioTap". How can I solve this problem?
Solved!
I change Video Player component "Audio Output mode": Audio Source to Direct.
Then not crash game.
But I would like to use Audio Source, because I need 3D Sound. (not important, I'm glad it works, but if you have idea how to work with 3D sound, I would be glad.)
Thanks!

IOS WebAudio only works on headphones

I've been running into an issue now for a while where on some ios devices my webaudio system only seems to work with headphones where as other devices (exact same os, model, etc) the audio plays perfectly fine through the speakers or headphones. I've searched for a solution to this but haven't found anything on this exact issue. The only thing I can think of is that maybe it's an audio channel issue or something.
How can I fix this?
#Alastair is correct, the mute toggle switch does mute WebAudio, but it does not mute HTML5 tags. Thanks to his work I managed to find a work around for the web which enables WebAudio to play even when the mute toggle switch is on. I'd post this as a comment on his reply, but I don't have the reputation.
In order to play WebAudio you must also play at least one WebAudio sound source node and one HTML5 tag during a user action. It is fine if these sounds are short bits of silence. I found that this self contained code works without any extra files needed:
EDIT 11/29/19:
Removed vestigial typescript typedefs. Thanks #Joep. I also realized the code below is woefully out of date and janky. Just consider it an example. Editing this post prompted me to create an open source solution for this. You can see a demo of it here: https://spencer-evans.com/share/github/unmute/ and check out the repo here: https://github.com/swevans/unmute
/**
* PLEASE DONT USE THIS AS IT IS, THIS IS JUST EXAMPLE CODE.
* If you want a drop in solution I have a script on git hub
* Demo:
* #see https://spencer-evans.com/share/github/unmute/
* Github Repo:
* #see https://github.com/swevans/unmute
*/
var isWebAudioUnlocked = false;
var isHTMLAudioUnlocked = false;
function unlock() {
if (isWebAudioUnlocked && isHTMLAudioUnlocked) return;
// Unlock WebAudio - create short silent buffer and play it
// This will allow us to play web audio at any time in the app
var buffer = myContext.createBuffer(1, 1, 22050); // 1/10th of a second of silence
var source = myContext.createBufferSource();
source.buffer = buffer;
source.connect(myContext.destination);
source.onended = function()
{
console.log("WebAudio unlocked!");
isWebAudioUnlocked = true;
if (isWebAudioUnlocked && isHTMLAudioUnlocked)
{
console.log("WebAudio unlocked and playable w/ mute toggled on!");
window.removeEventListener("mousedown", unlock);
}
};
source.start();
// Unlock HTML5 Audio - load a data url of short silence and play it
// This will allow us to play web audio when the mute toggle is on
var silenceDataURL = "data:audio/mp3;base64,//MkxAAHiAICWABElBeKPL/RANb2w+yiT1g/gTok//lP/W/l3h8QO/OCdCqCW2Cw//MkxAQHkAIWUAhEmAQXWUOFW2dxPu//9mr60ElY5sseQ+xxesmHKtZr7bsqqX2L//MkxAgFwAYiQAhEAC2hq22d3///9FTV6tA36JdgBJoOGgc+7qvqej5Zu7/7uI9l//MkxBQHAAYi8AhEAO193vt9KGOq+6qcT7hhfN5FTInmwk8RkqKImTM55pRQHQSq//MkxBsGkgoIAABHhTACIJLf99nVI///yuW1uBqWfEu7CgNPWGpUadBmZ////4sL//MkxCMHMAH9iABEmAsKioqKigsLCwtVTEFNRTMuOTkuNVVVVVVVVVVVVVVVVVVV//MkxCkECAUYCAAAAFVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVV";
var tag = document.createElement("audio");
tag.controls = false;
tag.preload = "auto";
tag.loop = false;
tag.src = silenceDataURL;
tag.onended = function()
{
console.log("HTMLAudio unlocked!");
isHTMLAudioUnlocked = true;
if (isWebAudioUnlocked && isHTMLAudioUnlocked)
{
console.log("WebAudio unlocked and playable w/ mute toggled on!");
window.removeEventListener("mousedown", unlock);
}
};
var p = tag.play();
if (p) p.then(function(){console.log("play success")}, function(reason){console.log("play failed", reason)});
}
window.addEventListener("mousedown", unlock);
This is likely because the iPhone's side switch is on "mute". It's very confusing - HTML5 <audio> tags still play fine when the phone is muted, but WebAudio does not. Why? Who knows. But it's a restriction I currently haven't found a way around.
If the iPhone mute button is down, meaning that the iPhone is muted, what is played through Web Audio Api will be muted.
Unfortunately there is no way to check if that physical button (located on the left edge towards the top of the iPhone) is on or off through Javascript.
This issue is completely independent from the fact that in iOS Safari the audio has to be started by a user action for it to be unmuted. There are some tricks that can be done to overcome that fact, including the one suggested by here Spencer, were you use "any action or a specific action" started by the user to "play" a silent audio file to allow subsequently playing audio files to play unmuted.
had same issue, and finally understood problem.
indeed WebView don't play sound on internal speakers if phone is in mute.
when i dig deeper i found a workaround :)
original post => https://stackoverflow.com/a/37874619/8064246
do {
try AVAudioSession.sharedInstance().setCategory(AVAudioSessionCategoryPlayback)
//print("AVAudioSession Category Playback OK")
do {
try AVAudioSession.sharedInstance().setActive(true)
//print("AVAudioSession is Active")
} catch _ as NSError {
//print(error.localizedDescription)
}
} catch _ as NSError {
//print(error.localizedDescription)
}

Audio player work on iOS simulator in Background but does not on iOS device (same version)

I am trying to play an audio streaming until application is in background.
Here is the code :
var service;
if(isiOS4Plus()){
Ti.App.addEventListener('resumed',function(e){
if(service!=null){
if(winPlayer.controlStreamer)// if my controller (containing the audio player object) exists
{
//function that redeclare intervals and event listeners
winPlayer.controlStreamer.foregroundStreamer();
}
service.stop();
service.unregister();
}
});
Ti.App.addEventListener('pause',function(e){
if(winPlayer.controlStreamer)// if my controller (containing the audio player object) exists
{
//function that stop and remove intervals and event listeners
winPlayer.controlStreamer.backgroundStreamer();
}
service = Titanium.App.iOS.registerBackgroundService({url:'/player/sound.js'});
Ti.API.info("registered background service = "+service);
});
}
Audio player object is intitialized before in this method :
this.streamer=Titanium.Media.createAudioPlayer({url:this.url_stream_stetienne,bufferSize:1000000});
Here is my problem : when i running app on iOS Simulator (iOS 5.0), stream player continue playing (what I want it does), but when I test it on iOS Device (iOS 5.0) sound volume decrease and player stop.
Note : the file info.plist contains the following lines :
<key>UIBackgroundModes</key>
<array>
<string>audio</string>
</array>
And I now with alert boxes that events are correctly triggered and handled.
Ok, it is not a bug :
If you want to play an audio streaming (like a webradio stream) on iOS, you have to set the "audioSessionMode" parameter to "Titanium.Media.AUDIO_SESSION_MODE_PLAYBACK" (maybe others values are correct ?) in your audioPlayer instance.
If you don't, streamer will work fine in background mode, but only if tested with iPhone simulator, not with device.
Hope this helps somebody else.

Resources