I've developed an app using Cordova and the Web Audio API, that allows the user to plug in headphones, press the phone against their heart, and hear their own heartbeat.
It does this by using audio filter nodes.
//Setup userMedia
context = new (window.AudioContext||window.webkitAudioContext);
navigator.getUserMedia = (navigator.getUserMedia ||
navigator.webkitGetUserMedia ||
navigator.mozGetUserMedia ||
navigator.msGetUserMedia);
navigator.getUserMedia(
{audio:true},
userMediaSuccess,
function(e) {
alert("error2 " + e.message);
});
function userMediaSuccess(stream)
{
//set microphone as input
input = context.createMediaStreamSource(stream);
//amplify the incoming sounds
volume = context.createGain();
volume.gain.value = 10;
//filter out sounds below 25Hz
lowPass = context.createBiquadFilter();
lowPass.type = 'lowpass';
lowPass.frequency.value = 25;
//filter out sounds above 425Hz
highPass = context.createBiquadFilter();
highPass.type = 'highpass';
highPass.frequency.value = 425;
//apply the filters and amplification to microphone input
input.connect(lowPass);
input.connect(highPass);
input.connect(volume);
//send the result of these filters to the phones speakers
highPass.connect(context.destination);
lowPass.connect(context.destination);
volume.connect(context.destination);
}
It runs fine when I deploy to Android, but it seems most of these features aren't available on iOS mobile browsers.
I managed to make getUserMedia function using the iosRTC plugin, but createMediaStreamSource is still "not a function."
So, I'm looking for an alternative to the Web Audio API that can filter out frequencies, or if there are any plugins I could use, that would be perfect.
There's no way to do this on ios web. You'd need a native app, since Apple doesn't support audio input in safari.
Did you try to use
document.addEventListener('deviceready', function () {
// Just for iOS devices.
if (window.device.platform === 'iOS') {
cordova.plugins.iosrtc.registerGlobals();
}
});
You asked this question quite a while ago, but sadly createMediaStreamSource is still not supported in Safari Mobile (will it ever be?).
As previously said, a plugin is the only way to achieve this, and there is actually a Cordova/Phonegap plugin that does exactly that. cordova-plugin-audioinput gives you access to the sound from the microphone using either the Web Audio API or by callbacks that delivers raw audio data chunks, and it supports iOS as well as Android.
Since I don't want to post the same answer twice, I'll instead point you to the following answer here on stackoverflow, where you'll also find a code example: https://stackoverflow.com/a/38464815/6609803
I'm the creator of the plugin and any feedback is appreciated.
Good news, full support for ios safari
https://developer.mozilla.org/en-US/docs/Web/API/AudioContext/createMediaStreamSource
Related
We would like to enable a feature that allows a model to be viewed using a deep link to our ARKit app from a web page.
Has anyone discovered a way to discover if a device is ARKit compatible using the user agent string or any other browser-based mechanism?
Thanks!
Apple seems to use the following code to show/hide the "Visit this page on iOS 12 to try AR Quick Look" on https://developer.apple.com/arkit/gallery/
(function () {
var isRelAR = false;
var a = document.createElement('a');
if (a.relList.supports('ar')) {
isRelAR = true;
}
document.documentElement.classList.add(isRelAR ? 'relar' : 'no-relar');
})();
The interesting part of course being
var isRelAR = false;
var a = document.createElement('a');
if (a.relList.supports('ar')) {
isRelAR = true;
}
Make your actions accordingly based on the value of isRelAR.
Safari doesn’t expose any of the required hardware information for that.
If you already have a companion iOS app for your website, another option might be to still provide some non-AR experience for your content, so that the website has something to link to in all cases.
For example, AR furniture catalogs seem to be a thing now. But if the device isn’t ARKit capable, you could still provide a 3D model of each furniture piece linked from your website, letting the user spin it around and zoom in on it with touch gestures instead of placing it in AR.
I'm working on an iOS text to speech app and trying to add an option to use the Alex voice, which is new for iOS 9. I need to determine whether or not the user has downloaded the Alex voice in Settings -> Accessibility. I can't seem to find out how to do this.
if ([AVSpeechSynthesisVoice voiceWithIdentifier:AVSpeechSynthesisVoiceIdentifierAlex] == "Not Found" ) {
// Do something...
}
The reason is the other language voices that are standard, play back at a certain rate, different from the Alex voice. So I have a working app, but if the user hasn't downloaded the voice, iOS automatically defaults to a basic voice, but it plays back at the incorrect rate. If I can detect the voice hasn't been downloaded, I can compensate for the difference and / or advise the user.
OK, so I guess I was overthinking this and thought it was more complicated. The solution was simple.
if (![AVSpeechSynthesisVoice voiceWithIdentifier:AVSpeechSynthesisVoiceIdentifierAlex]) {
// Normalize the speech rate since the user hasn't downloaded the voice and/or trigger a notification that they need to go into settings and download the voice.
}
Thanks to everyone who looked at this and to #CeceXX for the edit. Hope this helps someone else.
Here's one way to do it. Let's stick with Alex as an example:
- (void)checkForAlex {
// is Alex installed?
BOOL alexInstalled = NO;
NSArray *voices = [AVSpeechSynthesisVoice speechVoices];
for (id voiceName in voices) {
if ([[voiceName valueForKey:#"name"] isEqualToString:#"Alex"]) {
alexInstalled = YES;
}
}
// react accordingly
if (alexInstalled) {
NSLog(#"Alex is installed on this device.");
} else {
NSLog(#"Alex is not installed on this device.");
}
}
This method loops through all installed voices and queries each voice's name. If Alex is among them, he's installed.
Other values you can query are "language" (returns a language code like en-US) and quality (1 = standard, 2 = enhanced).
I'm trying to implement speech recognition on Chrome on the iPad without any luck. Just to cut to the chase and remove any dependencies on my implementation of the webkitSpeechRecognition api, Glenn Shire's excellent sample code does not run on Chrome v27 on an iPad 1 running iOS 5.1.1 or Chrome v31 on an iPad3 running iOS 7.0.4, at least as far as I can tell. It fails at this line:
if (!('webkitSpeechRecognition' in window)) {
r.onState('upgrade');
return;
}
I can't figure out a workaround, and I've not seen any online postings that say anything about speech recognition not working in the iOS version of Chrome. Anyone else run into this?
Chrome on iOS doesn't support Speech Recognition at the moment.
Google have to use iOS UIWebView that mean there is no special web interpretation feature that are not supported on Safari.
You may have a look to this link.
In case you want to recognize few simple commands you can look on Pocketsphinx.js
The code to recognize speech is simple:
var id = 0;
recognizer.postMessage({command: 'initialize', callbackId: id});
var keyphrase = "HELLO WORLD";
recognizer.postMessage({command: 'addKeyword', data: keyphrase, callbackId: id});
recognizer.postMessage({command: 'start', data: id});
recognizer.postMessage({command: 'process', data: array});
recognizer.postMessage({command: 'stop'});
recognizer.onmessage = function(e) {
if (e.data.hasOwnProperty('hyp')) {
alert(e.data.hyp);
}
};
For more details see also the full example here
I've been running into an issue now for a while where on some ios devices my webaudio system only seems to work with headphones where as other devices (exact same os, model, etc) the audio plays perfectly fine through the speakers or headphones. I've searched for a solution to this but haven't found anything on this exact issue. The only thing I can think of is that maybe it's an audio channel issue or something.
How can I fix this?
#Alastair is correct, the mute toggle switch does mute WebAudio, but it does not mute HTML5 tags. Thanks to his work I managed to find a work around for the web which enables WebAudio to play even when the mute toggle switch is on. I'd post this as a comment on his reply, but I don't have the reputation.
In order to play WebAudio you must also play at least one WebAudio sound source node and one HTML5 tag during a user action. It is fine if these sounds are short bits of silence. I found that this self contained code works without any extra files needed:
EDIT 11/29/19:
Removed vestigial typescript typedefs. Thanks #Joep. I also realized the code below is woefully out of date and janky. Just consider it an example. Editing this post prompted me to create an open source solution for this. You can see a demo of it here: https://spencer-evans.com/share/github/unmute/ and check out the repo here: https://github.com/swevans/unmute
/**
* PLEASE DONT USE THIS AS IT IS, THIS IS JUST EXAMPLE CODE.
* If you want a drop in solution I have a script on git hub
* Demo:
* #see https://spencer-evans.com/share/github/unmute/
* Github Repo:
* #see https://github.com/swevans/unmute
*/
var isWebAudioUnlocked = false;
var isHTMLAudioUnlocked = false;
function unlock() {
if (isWebAudioUnlocked && isHTMLAudioUnlocked) return;
// Unlock WebAudio - create short silent buffer and play it
// This will allow us to play web audio at any time in the app
var buffer = myContext.createBuffer(1, 1, 22050); // 1/10th of a second of silence
var source = myContext.createBufferSource();
source.buffer = buffer;
source.connect(myContext.destination);
source.onended = function()
{
console.log("WebAudio unlocked!");
isWebAudioUnlocked = true;
if (isWebAudioUnlocked && isHTMLAudioUnlocked)
{
console.log("WebAudio unlocked and playable w/ mute toggled on!");
window.removeEventListener("mousedown", unlock);
}
};
source.start();
// Unlock HTML5 Audio - load a data url of short silence and play it
// This will allow us to play web audio when the mute toggle is on
var silenceDataURL = "data:audio/mp3;base64,//MkxAAHiAICWABElBeKPL/RANb2w+yiT1g/gTok//lP/W/l3h8QO/OCdCqCW2Cw//MkxAQHkAIWUAhEmAQXWUOFW2dxPu//9mr60ElY5sseQ+xxesmHKtZr7bsqqX2L//MkxAgFwAYiQAhEAC2hq22d3///9FTV6tA36JdgBJoOGgc+7qvqej5Zu7/7uI9l//MkxBQHAAYi8AhEAO193vt9KGOq+6qcT7hhfN5FTInmwk8RkqKImTM55pRQHQSq//MkxBsGkgoIAABHhTACIJLf99nVI///yuW1uBqWfEu7CgNPWGpUadBmZ////4sL//MkxCMHMAH9iABEmAsKioqKigsLCwtVTEFNRTMuOTkuNVVVVVVVVVVVVVVVVVVV//MkxCkECAUYCAAAAFVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVV";
var tag = document.createElement("audio");
tag.controls = false;
tag.preload = "auto";
tag.loop = false;
tag.src = silenceDataURL;
tag.onended = function()
{
console.log("HTMLAudio unlocked!");
isHTMLAudioUnlocked = true;
if (isWebAudioUnlocked && isHTMLAudioUnlocked)
{
console.log("WebAudio unlocked and playable w/ mute toggled on!");
window.removeEventListener("mousedown", unlock);
}
};
var p = tag.play();
if (p) p.then(function(){console.log("play success")}, function(reason){console.log("play failed", reason)});
}
window.addEventListener("mousedown", unlock);
This is likely because the iPhone's side switch is on "mute". It's very confusing - HTML5 <audio> tags still play fine when the phone is muted, but WebAudio does not. Why? Who knows. But it's a restriction I currently haven't found a way around.
If the iPhone mute button is down, meaning that the iPhone is muted, what is played through Web Audio Api will be muted.
Unfortunately there is no way to check if that physical button (located on the left edge towards the top of the iPhone) is on or off through Javascript.
This issue is completely independent from the fact that in iOS Safari the audio has to be started by a user action for it to be unmuted. There are some tricks that can be done to overcome that fact, including the one suggested by here Spencer, were you use "any action or a specific action" started by the user to "play" a silent audio file to allow subsequently playing audio files to play unmuted.
had same issue, and finally understood problem.
indeed WebView don't play sound on internal speakers if phone is in mute.
when i dig deeper i found a workaround :)
original post => https://stackoverflow.com/a/37874619/8064246
do {
try AVAudioSession.sharedInstance().setCategory(AVAudioSessionCategoryPlayback)
//print("AVAudioSession Category Playback OK")
do {
try AVAudioSession.sharedInstance().setActive(true)
//print("AVAudioSession is Active")
} catch _ as NSError {
//print(error.localizedDescription)
}
} catch _ as NSError {
//print(error.localizedDescription)
}
I want to preload multiple audio files. To do this, I tried to create multiple Audio elements in JavaScript.
function loadAudio(){
audio1 = new Audio();
audio1.addEventListener('canplaythrough', isLoaded, false);
audio1.src = 'assets/audio/Maid with the Flaxen Hair.mp3';
audio1.load();
}
function isLoaded(){
audio1.removeEventListener('canplaythrough', isAppLoaded);
alert('maid');
alert('start audio 2');
audio2 = new Audio();
audio2.addEventListener('canplaythrough', isLoaded2, false);
audio2.src = 'assets/audio/Kalimba.mp3';
audio2.load();
}
function isLoaded2(){
alert('kalimba');
}
I only get the first alert, the second one never works.
I found that I can only play one sound at a time, but can I also only load one? Does the script need another user input for every new Audio object I create? Or does anyone have another way to create a preloader for audio?
this could have to do with the limitations on autoplaying (and, as far as i know autoloading) audiofiles on ios-devices (see here: Limitations of HTML5 Audio on iOS 4?).
In short: You cannot programmatically start audio-playback on ios devices, this is only allowed from within event-handlers for trusted (=initiated by the user) events.
Yo can play multiple audio on IOS device
using following way
$("#btn").on("click",function(){
alert("audio clicked");
var aud=new Audio();
aud.src="music.mp3";
aud.play();
var aud1=new Audio();
aud1.src="intro.mp3";
aud1.play();
})
The above code play multiple audio on device