I am writing an extension for Firefox Mobile (Fennec) which should play a sound. Using nsISound, i.e.
var sound = ios.newURI(pathToSoundFile, null, null);
var player = Components.classes["#mozilla.org/sound;1"].createInstance(Components.interfaces.nsISound);
player.play(sound);
works fine on the Desktop version of Firefox, it but wouldn't produce any sound on FF Mobile. I also tried appending an audio element to the website,
var audioElement = win.content.document.createElement('audio');
audioElement.setAttribute('id','audio_test');
audioElement.setAttribute('src',pathToSoundFile);
win.content.document.body.appendChild(audioElement);
win.content.document.getElementById('audio_test').play();
which works, but only if the website is stored locally on my computer (due to security
restrictions, I assume).
I tested this on an Asus Transformer Tablet with Android 4.2. Any suggestions?
nsISound is not implemented on Fennec.
HTML5 audio should work with data URis. Read your audio file, base64-encode it and pass it as src e.g. "data:audio/ogg;base64," + base64encodeddata
Related
I am trying to use the HTML5 video - https://developer.mozilla.org/en-US/docs/Web/HTML/Element/video to access camera on the device.
I am following this example - https://developer.mozilla.org/en-US/docs/Web/API/MediaStream_Recording_API/Recording_a_media_element to record the video.
It works perfectly in Chrome browser but does not seem to be working on Safari and iOS mobile app.
On debugging both captureStream() and mozCaptureStream() functions are undefined.
Any suggestions on how to fix this?
PS: Is this a known issue - https://bugs.webkit.org/show_bug.cgi?id=181663. Are there any workarounds? Thanks!
I don't think recording webcam as video is an available option on Apple devices.
It might be a licensing issue rather than a technical limitation.
It works perfectly in Chrome browser but does not seem to be working
on Safari and iOS mobile app. On debugging both captureStream() and
mozCaptureStream() functions are undefined.
Your link is recording into Google's own webM video which is a format not supported by Apple. The Safari browser cannot encode pixels as VP8 or VP9 to use inside webM container. Apple has an MPEG video license so for Safari, I expect MP4 to be their expected output (but is there a free H264 encoder inside Safari? Nope).
mozCaptureStream() is specific to Mozilla Firefox. Safari won't know/accept it.
captureStream() is not fully suppported on Safari. The missing part is the recording part.
Possible workarounds:
Try enabling MediaRecorder API in Safari settings.
Research (double-check) for known issues, like as mentioned at bottom of this read this article: https://blog.addpipe.com/safari-support-on-macos-beta/
If you can draw webcam to Canvas then consider bringing in your own (Javascript-based) H264 encoder (for MP4 video) or VP8/VP9 encoder (for webM video).
You can try: https://github.com/TrevorSundberg/h264-mp4-encoder.
Wait for webCodecs API to be added to Safari. Note in Chrome/Edge you can just use the built-in webCodecs API to encode video for MP4 or WebM formats. Safari has no free encoders and thus has no webCodecs API.
I've been banging my head on getting captureStream to work on iOS (Safari and Chrome, same results) to no avail. This reply by #VC.One helped me stop trying to use webM format, which was used by the sample code I was following from mdn which is here... https://developer.mozilla.org/en-US/docs/Web/API/MediaStream_Recording_API/Recording_a_media_element
So I simply switched to mp4 instead of webm, using .srcObject instead of .captureStream, and now I have things working on iOS. I still have a lot of clean-up to do, but when my solution is finished I will make a nice write-up and post back.
Also, someone pointed out that I needed to use the onloadedmetadata with the play method, in order to get a video element to show a real-time feed of bytes from getUserMedia, and that worked. Then I just needed to store the bytes somewhere that iOS will not freak out about (it does not support captureStream) and then it all came together. It definitely seems to be a licensing issue at the heart of it all, which is why this works using mp4 instead of webm.
I am trying to stream hls on safari iOS with Aframe that has three.js under the hood. But the video shows a black screen with just the audio playing. The video src is of type .m3u8. I tried to read through a lot of related posts but none seem to have a proper solution. Is it some kind of a wishful thinking getting HLS & WebGL to play on iOS? If not, can some one please help me with a solution.
A couple of discussions on the issues that are available on github:
HLS m3u8 video streaming
HLS on Safari
To your question:
Is it some kind of a wishful thinking getting HLS & WebGL to play on iOS?
Yes, wishful thinking :-) The problem/issue/bug is with Apple, not any library. No matter what the JS library, A-Frame, Three, etc, this will always be an issue on any browser in iOS (all browsers on iOS are basically wrappers for Safari), and OSX Safari.
The issue is this (from my understanding):
At some point in the history of WebGL, there were restrictions on cross-origin content (videos, images, etc). I can't find a source for this, but I remember reading it somewhere, so this might not be 100% accurate.
Recently (a couple years ago? 2015?) all major browsers came to the conclusion that cross-origin media for use in WebGL was safe. Except Apple/Safari.
For most browsers, the crossorigin attribute on a <video> element could signal that this content came from another source. In Safari, for whatever reason, this attribute is ignored or not implemented. In fact, it looks like WebKit, which Safari is based on, fixed this as far back as 2015, but Apple still does not implement it. Even Apple refuses to comment on any progress.
Possible workarounds:
WebGL on Safari works with progressive (not a stream like HLS/Dash) .mp4 videos. Check out any 360 video on Facebook (website, not app) in iOS/Safari, and you'll note the source is an .mp4.
Use HLS (or Dash), but play the video flat, without WebGL. Check out any 360 video on YouTube (website, not app), and I think they are using HLS or Dash, but the point is they stream the video, whereas Facebook doesn't.
Here's a good starting point to the real issue: link.
Here's another detailed thread: link.
https://github.com/video-dev/hls.js#compatibility
Please note: iOS Safari "Mobile" does not support the MediaSource API.
Safari browsers have however built-in HLS support through the plain
video "tag" source URL. See the example above (Getting Started) to run
appropriate feature detection and choose between using Hls.js or
natively built-in HLS support.
When a platform has neither MediaSource nor native HLS support, you
will not be able to play HLS.
<script src="https://cdn.jsdelivr.net/npm/hls.js#latest"></script>
<!-- Or if you want a more recent canary version -->
<!-- <script src="https://cdn.jsdelivr.net/npm/hls.js#canary"></script> -->
<video id="video"></video>
<script>
var video = document.getElementById('video');
if (Hls.isSupported()) {
var hls = new Hls();
hls.loadSource('https://video-dev.github.io/streams/x36xhzz/x36xhzz.m3u8');
hls.attachMedia(video);
hls.on(Hls.Events.MANIFEST_PARSED, function() {
video.play();
});
}
// hls.js is not supported on platforms that do not have Media Source Extensions (MSE) enabled.
// When the browser has built-in HLS support (check using `canPlayType`), we can provide an HLS manifest (i.e. .m3u8 URL) directly to the video element through the `src` property.
// This is using the built-in support of the plain video element, without using hls.js.
// Note: it would be more normal to wait on the 'canplay' event below however on Safari (where you are most likely to find built-in HLS support) the video.src URL must be on the user-driven
// white-list before a 'canplay' event will be emitted; the last video event that can be reliably listened-for when the URL is not on the white-list is 'loadedmetadata'.
else if (video.canPlayType('application/vnd.apple.mpegurl')) {
video.src = 'https://video-dev.github.io/streams/x36xhzz/x36xhzz.m3u8';
video.addEventListener('loadedmetadata', function() {
video.play();
});
}
</script>
I am working with a school group that has a Django website with a page that allows the users to press a button, record audio, and save this audio to their user pages. This website was built to work on computers and Samsung tablets with Dolphin browsers using the HTML Media Capture to capture the audio by launching the microphone app on the tablets. The group wants to now switch to iPads. I have been researching ways to launch the microphone app from iPads but it seems this is not possible using HTML5. I have done some research and seen the links below, but I don't know what the best strategy is to modify the existing website to make it work on the iPad.
What is the best way forward to allow this group to record audio (not video) from their iPads using a browser-based website?
Research--
http://caniuse.com/#search=getUser - shows getUserMedia will not work on iPads
http://www.html5audio.org/2012/11/capturing-audio-on-ios-6-via-the-videotag.html - looks pretty hacky, does it work?
How to record and play voice in ios6 using HTML5 - using PhoneGap might work, but how would the recording app be launched from the browser and then save back the audio to the users account?
Thanks,
Lee
After having the same problem for a year, I found a new article with a solution: How to record audio from a mobile web page on iOS and Android
One of the newer APIs available is the MediaRecorder API. My first attempt at building this application started with this class. I implemented the entire application and it worked great on my desktop. It was easy to capture audio and the data was already compressed into .ogg format and ready to ship to my server. Then I tried it on iOS. It turns out that the MediaRecorder API is not supported and wouldn’t meet my needs. After I stopped cursing Apple, I began again from scratch.
... That was the last piece of the puzzle that allowed me to construct a working demo and it revolves around three steps:
1. Capture the microphone so we can begin recording
2. Accumulate captured audio data into a series of byte array chunks
3. Combine the chunks into one large array and massage the array into the format of a .wav file
There is a bunch of code on the website. And here is the demo file: https://www.gmass.co/blog/wp-content/uploads/2018/03/wzrecorder.zip
I have tested it and it works on iPad.
Demo: https://q2apro.github.io/wzrecorder/
Safari iOS does not (yet) support the accept="audio/*" part of the <input> element even though it does support taking videos and images through HTML Media Capture.
<input type="file" accept="audio/*" > on iOS 10 will prompt the user to select between:
Take Photo or Video
Photo Library
iCloud Drive
More...
The same code works just fine in Chrome on Android.
Safari in iOS6 was the 1st mobile Safari to support the <input ... > element and I've tested all the way up to iOS10. Chrome on iOS uses the same rendering engine so the same limits apply.
I'm struggling to get simple audio playback. I've got a list of tracks, each at unique url's that I would like to play when a user presses the associated play button. I've attempted soundmanager2 and jplayer but couldn't get it to work for my use case (desktop browser and ios). I have fallen back to straight up html5 with the code as follows
<audio controls>
<source src="/path/to/file" type="audio/wav">
Your browser does not support the audio element.
</audio>
This works perfectly in desktop chrome and desktop safari. In ios chrome and ios safari (latest) the player isn't presented, but instead a message within a grey box saying "Cannot play audio file" is presented.
Am I using this tag correctly? How can I overcome these errors?
Update 1
I am sending the wav file from google appengine (as a blob). I have found that safari cannot play unless I add .wav to the end of the src - despite the src just being an indirect link to the file. The actual file returned does end in .wav but Safari isn't smart enough to recognise this.
Update 2
The following works in all browsers (mentioned above) - so it is not specific to wav files.
<audio controls preload="metadata">
<source src="http://www-mmsp.ece.mcgill.ca/Documents/AudioFormats/WAVE/Samples/AFsp/M1F1-Alaw-AFsp.wav" type="audio/wav">
Your browser does not support the audio element.
</audio>
I've sent the file from google app engine as both a MIME attachment and a raw response but it makes no difference.
Update 3
I've swapped in a longer wav file (http://www.villagegeek.com/downloads/webwavs/ever_again.wav) and this too is unable to play (on ios). It isn't clear if it's because of the length, size or some other variable.
Update 4
I've ruled out size being the issue because this 24s wav file works
<audio controls preload="auto">
<source src="http://www.dailywav.com/sites/default/files/wavs/dontlikelaughing.wav" type="audio/wav">
Your browser does not support the audio element.
</audio>
Update 5
So I'm serving the file from a google cloud storage bucket. When saving the file I'm not specifying the MIME type and as a result it is being returned as binary/octet-stream. The desktop browsers are smart enough to overcome this, but the mobile browsers are not.
So I'm serving the file from a google cloud storage bucket. When saving the file I'm not specifying the MIME type and as a result it is being returned as binary/octet-stream. The desktop browsers are smart enough to overcome this, but the mobile browsers are not. The answer to the above question is therefore to set the MIME type for the relevant file upon writing to GCS.
I created simple html page with content like:
...<body><video width="848" height="352" class="shown" id="videoShowcase" controls="controls" ><source src="http://myportal/PublishingImages/apple-html5-demo-tron-us_848x352.m4v" type='video/mp4; codecs="avc1.42E01E, mp4a.40.2"' />
and its NOT working on ipad(ios4), while it works in chrome. If I change src attribute to "http://movies.apple.com/media/us/html5/showcase/2011/demos/" for example, it works on both(even IE in 9.0 mode).
It makes me think that the probles is 1) SP2010 access to "PublishingImages" list, but i break role inheritence, and granted anonimous access to this
2)the way SP2010 stream videos, my guess is SP2010 handlers over IIS
Any thoughts?
The file extension is irrelevant - mp4, m4v, mov etc. are all just container format extensions. The real format of the file is declared internally, and it's most likely the internal format that's the problem.
I believe that iOS will only play .h264 encoded videos. I'm not sure what avc1.42E01E is, but I'm guessing iPad doesn't support it.
I suggest you try ripping the video to an iPad compatible .h264 video using Quicktime Pro. If you find a copy of Quicktime 7 and pay the $30 license fee for pro, it comes with a bunch of easy-to-use export options for videos that will work correctly on iPad.
There are other free video rippers such as Handbrake if $30 for QTP seems a bit steep - YMMV.
Mobile Safari on Apple i-devices uses hardware acceleration to play videos. So instead of making request to video in the current(authenticated) context, it makes new request to the video.That request has authentication failed. So, the only "solution" to this is to store public videos allowing anonimous login.