Navigator.mediaDevices.getUserMedia not working on iOS 12 Safari - ios

As of iOS 12, navigator.mediaDevices.getUserMedia() is returning an error in Safari.
To recreate this, open iPhone Web Inspector, then run this snippet in the console:
var constraints = { audio: true, video: { width: 1280, height: 720 } };
navigator.mediaDevices.getUserMedia(constraints)
.then(function() {
console.log('getUserMedia completed successfully.');
})
.catch(function(error) {
console.log(error.name + ": " + error.message);
});
You'll see that this runs successfully in desktop browsers, and in iOS 11 Safari, but fails in iOS 12 Safari.
NotAllowedError: The request is not allowed by the user agent or the platform in the current context, possibly because the user denied permission.
Any idea why?
note: This is happening prior to the user being asked if their camera can be accessed, ruling out the possibility of it being because the user denied permission.

Setting these three attributes before calling getUserMedia solved the problem for me:
video.setAttribute('autoplay', '');
video.setAttribute('muted', '');
video.setAttribute('playsinline', '');
For some reason video.setAttribute() works but trying to assign the value directly to the video object for example video.muted = '' does not.
Also it appears that it's not necessary to call video.play().
Simply setting the video.srcObject to the stream returned by getUserMedia worked for me.
This medium post has a link to a working demo & source code.

There are two possible reasons for immediate NotAllowedError at the moment:
1. getUserMedia requries https
Safari seems to require https for camera and mic access, both in iOS and OSX.
With an https link, iOS Safari 12 works for me; same link in http gets NotAllowedError.
Chrome has the same requirement. This is consistent with the direction of the specification, which recently has restricted getUserMedia to secure contexts. Browsers who have yet to update, still expose navigator.mediaDevices in http, but getUserMedia always rejects with NotAllowedError.
In the future, expect browsers to remove mediaDevices entirely in http, to comply with the spec.
2. getUserMedia requires feature policy in cross-origin iframes.
This appears new with Safari 12. In iframes, getUserMedia's feature policy is off by default for cross-origin content.
This works for me:
<iframe
allow="camera;microphone"
src="https://webrtc.github.io/samples/src/content/getusermedia/gum/">
</iframe>
This doesn't work:
<iframe src="https://webrtc.github.io/samples/src/content/getusermedia/gum/">
</iframe>
...and in addition to failing with NotAllowedError, Safari warns in web console:
The top-level frame has prevented a document with a different security origin to
call getUserMedia.
This is also a recent update to the spec.

Turns out my particular issue was a bug in 12.01. Every device with that version had the issue, and as soon as I updated them to a newer version it went away.

Related

Twitch Embedded Player on Ionic iOS not working

I create a App with Ionic which embed the twitch player for streams and clips.
var url = https://player.twitch.tv/?video=v${videoId}&time=${time}&parent=localhost
<iframe [src]='url' frameborder='0' allow='autoplay; encrypted-media' allowfullscreen></iframe>
On Android its working perfect but in the iOS App the iframe will not be shown and no player is available. Just a white screen.
Does anybody know what the problem can be related to ios?
I have the same issue. Security limitations are the culprit indeed: I opened the Safari console while the iframe was trying to load and saw this error:
Refused to load https://player.twitch.tv/?channel=...&controls=false&parent=localhost&muted=true because it does not appear in the frame-ancestors directive of the Content Security Policy.
Looking at this security header that Twitch returns, it looks like this:
Content-Security-Policy: frame-ancestors http://localhost:* https://localhost:*
However, when using the Ionic WebView, according to its documentation:
iosScheme
<preference name="iosScheme" value="httpsionic" />
Default value is ionic
Configures the Scheme the app uses to load the content.
Values like http, https or file are not valid and will use default value instead.
So your WKWebView will have ionic://locahost as its application URL by default, which doesn't match the http://localhost and https://localhost prefixes specified by Twitch.
I tried using &parent=ionic://localhost in the Twitch iframe URL. The WKWebView doesn't refuse to open the iframe, but Twitch itself freaks out about the special characters in that parameter and refuses to render its player page.
To me it seems that currently, iOS Ionic/Cordova Apps and Twitch iframes are mutually exclusive.
Edit:
I found a bunch of discussions in the Twitch forum talking about this subject (1, 2), but none of them went technically in-depth to the point where a working solution or workaround could be found. Unfortunately, threads are closed after 1 month of inactivity, so I had to open yet another thread in the Twitch forum explaining the technical details from my point of view. I also added a feature request in their user-voice feedback portal. I'm afraid trying to make Twitch realise / accept that they should do something about mobile embeds is all we can do at this point. Here's the link to it:
https://twitch.uservoice.com/forums/310213-developers/suggestions/44682766-make-embeds-work-on-ios-web-based-apps-like-ionic

The embedded player shows "Please click here to watch this video on YouTube" to some users under unclearly circumstances

We use the YouTube embedded player on a hybrid app based on Cordova. The app has a lot of traffic worldwide.
The player displays the message "Please click here to watch this video on YouTube" to some users when they try to play any video under unclearly circumstances.
We do not see a clear pattern, it doesn't seem to depend on the user's country nor the restrictions of the video.
We have seen in the player code (https://www.youtube.com/yts/jsbin/player_ias-vflrnurMS/en_US/base.js) that the message is assigned to a constant called "TOO_MANY_REQUESTS_WITH_LINK" but there is no high volume of requests per user and the quota cannot be configured as in other Google APIs.
We follow the iFrame API reference:
https://developers.google.com/youtube/iframe_api_reference?hl=en
<iframe id="player" src="https://www.youtube.com/embed/M7lc1UVf-VE?autoplay=1&cc_load_policy=0&controls=0&disablekb=1&enablejsapi=1&fs=0&iv_load_policy=3&loop=0&modestbranding=1&playsinline=1&rel=0&showinfo=0&wmode=transparent&origin=XXX" frameborder="0"></iframe>
<script type="text/javascript">
var player,
scriptTag = document.createElement('script'),
firstScriptTag = document.getElementsByTagName('script')[0];
scriptTag.src = "https://www.youtube.com/iframe_api";
firstScriptTag.parentNode.insertBefore(scriptTag, firstScriptTag);
function onYouTubeIframeAPIReady() {
var options = JSON.parse('{"autoplay":1,"cc_load_policy":0,"controls":0,"disablekb":1,"enablejsapi":1,"fs":0,"iv_load_policy":3,"loop":0,"modestbranding":1,"playsinline":1,"rel":0,"showinfo":0,"wmode":"transparent","origin":"XXX"}');
if (!options.origin) {
options.origin = location.origin ||
location.protocol + '//' + location.hostname + (location.port ? ':' + location.port : '');
}
player = new YTRemoteVPlayer('player', '<%-videoId%>', options);
}
</script>
Does anyone know what may be happening and how to fix it?
Me and my teammates managed to reproduce that behaviour locally.
To reproduce that behaviour, all you need to do is, under the same IP Address, to load the youtube JS multiple times and for different videos. Although I cannot clarify the exact amount of times it must be done (we left some devices aside doing it), the behavior certainly happened.
As an attempt of a workaround for that problem, we accessed the https://web.archive.org/web website and fetched an old YouTube JS script (https://www.youtube.com/iframe_api). We tried to use that old script, but with no success. The message continued to appear =/.
I really do not know if there is much we, developers, can do about it, but wait for an YouTube statement and/or revision of recent changes on their end.
Update
There is an issue opened on Google's issue tracker website:
https://issuetracker.google.com/issues/143091693
Update
It looks like that YouTube have resolved that problem on their end. We'll keep monitoring :)
Update (November 5th 2019)
The problem is happening again.
Update (November 7th 2019)
I have opened a new issue on issuetracker:
- https://issuetracker.google.com/issues/144057800
I don't have a fix for it, but I can add more information about the problem, this way maybe someone can help us with this issue.
I am getting the same error with some users, and like you said there is no a clear pattern. My devices are working properly, but I did some tests with the help of a user of my app that is getting this error. Following the results:
Demo used in my tests: https://developers.google.com/youtube/youtube_player_demo
Test 1 - Open the iframe player api demo on device's browser - WORKED
Test 2 - Open the iframe player api demo into a webview on my app - FAIL
Test 3 - Open the iframe player api demo into a webview on my app using a VPN connection - WORKED
So, it appears to be a combination of APP + IP is blocked. I don't know the reason, and it will be very helpful if someone repeat these tests. I am trying to confirm it with another user, but it's a little hard to find someone willing to help.
UPDATE: a second user tried to use a VPN and it worked to him too.
UPDATE 2: a user with this problem told me that the videos are working properly again now. It appears to be a temporary block by Youtube on server side.

Ionic 4 - Inappbrowser failed to load in IOS

I am working on project where we need to get a JWT token from a external source to the mobile app. In Android it works without any issue. However in IOS i am not able to get the token from the Iframe.
I have used both Ionic and Cordova inappbrowsers, the authentication works fine and i am able to execute script inside the iframe in IOS, but while using the callback function in executescript() function, gap-iab:// was blocked by content security policy.
Did anyone faced a issue like this in IOS and solved it. Please let me know. Thanks in advance.
My Code:
const bro = this.iab.create(legacyUrl, '_blank', 'location=no,toolbar=no');
bro.on('loadstop').subscribe( (e) => {
                bro.executeScript({
                  code: 'if (document.getElementById("Token") != null) { document.getElementById("Token").
getAttribute("value");}'
                }).then((value) => {
console.log(value);
});
});
And i am getting below error from the application while running in IOS simulator,
Refused to load gap-iab://InAppBrowser627412330/%5Bnull%5D because it appears in neither the child-src directive nor the default-src directive of the Content Security Policy.
"gap-iab:" needs to be child-src in your content security policy. That is usually set on your server side and tells your in-app-browser which content it can load and from where.

Opening apps with custom protocols (twitter://, instagram://) in Cordova/PhoneGap on iOS

I can't seem to use custom URL protocols to open the apps that are installed on an iPhone; even with InAppBrowser installed.
For example:
// Instagram
window.open('instagram://user?username=jgillick', '_system', 'location=no');
// Twitter
window.open('twitter://user?screen_name=jgillick', '_system', 'location=no');
Both of these things do absolutely nothing. They don't even open the InAppBrower window.
Facebook is a really interesting one. Instead of using a custom protocol, if you try to access https://www.facebook.com/<facebook ID number> on your phone, the Facebook app will automatically take over (you can use this tool to find your facebook ID). This works when I put it in Safari on my iPhone, but doesn't in the InAppBrowser, it just opens the app browser and directs me to the logged-out version of that page.
// Facebook
window.open('https://www.facebook.com/763639132', '_system', 'location=no');
I'm at a complete loss for getting this working.
For some reason the window.open('...', '_system') method suddenly just started working. I'm not sure why it wasn't before.
For me that wasn't giving anything useful
window.open('...', '_system')
what worked was:
window.location = 'instagram://media?id=' + id;
window.location = 'fb://post/' + id;
of course those native apps have to be installed, otherwise no action will happen.
this might come handy https://github.com/ohh2ahh/AppAvailability

Approaches for debugging HTML5 video on an iPad

I'm writing a video encoder/server to serve video to a web application intended to run on an iPad. The iPad uses an HTML5 video tag to retrieve and decode the video, and I'm running into issues where the encoded video isn't being decoded correctly.
Is there anything like a system log on the iPad where I can find any information about what the video decoder finds objectionable in my bitstream, or any other way of getting some visibility into the decoding process?
Older versions of IOS allowed you to turn on the Safari debug console (settings ->safari->advanced -> debug console). This was handy for logging errors etc. If you are a mac user apparently there is a nice interface for doing so.
If you have desktop Safari you can also fake the user agent see: http://www.dummies.com/how-to/content/how-to-activate-user-agent-switcher-in-safari.html this will allow you to use web debug tools to see whats going on.
Alternatively you can create a 'Debug' panel in your webapp and hijack the console.log function so you can see errors etc.
Example:
<div id="debug-info"></div>
<script>
(function(){
var oldLog = console.log;
console.log = function (message) {
// DO MESSAGE HERE.
oldLog.apply(console, arguments);
$('#debug-info').prepend('<p>'+message+'</p>')
};
})();
</script>

Resources