iOS Safari does not support canvas.drawImage() from video element hence i cannot access the pixels for computation. It does work from time to time but not always and not supported. How to get around it?
"The image object can be an img element, a canvas element, or a video element. Use of the video element is not supported in Safari on iOS, however."
https://developer.apple.com/documentation/webkitjs/canvasrenderingcontext2d/1630282-drawimage
I did try to draw the stream onto a canvas and it works from time to time but breaks mostly.
I also tried to use ImageCapture but it is not supported on iOS safari.
Related
I' trying to capture video by AVAssetWriter and AVCaptureOutput
You can find sample project here.
The video should be in portrait mode with any resolution. The main problem that it should be in portrait mode.
I'm trying to set different setting, but in the end, video is rotated and scaled to size (1920x1080) on iPhone SE.
Is it possible to control this resolution? Or at least orientation?
Video resolution is determined by the AVCaptureSession sessionPreset. You're setting that to medium, so you're getting the resolution that comes with that. If you want a different resolution, pass a different session preset, or use AVCaptureDevice to set a specific capture format. (For a good overview of capture session presets vs device formats, go back to this WWDC13 video.)
Per this Apple Developer Q&A, you need to set an orientation on the capture connection after you start the capture session in order to get "physically" rotated frame buffers (at a capture performance cost), or set the transform property on your asset writer (so that buffers are recorded in the sensor's native orientation, but clients display it in your intended orientation).
I'm using openGL ES to display CVPixelBuffers on iOS. The openGL pipeline uses the fast texture upload APIs (CVOpenGLESTextureCache*). When running my app on the actual device the display is great but on the simulator it's not the same (I understand that those APIs don't work on the simulator).
I noticed that, when using the simulator, the pixel format is kCVPixelFormatType_422YpCbCr8 and I'm trying to extract the Y and UV components and use the glTexImage2D to upload but, I'm getting some incorrect results. For now I'm concentrating on the Y component only, and the result looks like the image is half of the expected width and is duplicated - if it makes sense.
I would like to know from some one that has successfully displayed YUV422 video frames on iOS simulator if I'm on the right track and/or if I can get some pointers on how to solve my problem.
Thanks!
I have an embedded youtube video on a web page. On the desktop it works fine. However, on an ipad (running iOS7) the video quality is rather poor, probably 480p. Inside the iframe it looks fine, but when you make the video full screen it is very noticeable. I have tried using hd=1&vq=hd720 in the querystring and using cueVideoById with a suggested quality of 'hd720' but neither worked. Are there any work arounds either to get higher resolution on the ipad when embedded youtube videos are made fullscreen, or to force higher resolution for embedded youtube videos on the ipad?
I found a workaround that works, although it is hardly elegant.
Basically, I create the iframe with a dimensions large enough to get 720p quality. (Such as making the iframe 720x440 pixels.) And then, I use CSS transforms to scale the iframe back down to the size I want. The CSS looks something like:
-webkit-transform-origin: left top;
-webkit-transform: scale(0.5);
Where 0.5 could be replaced with whatever scale you had to increase the size of the iframe by.
The main downside to this from the user's perspective is that the video controls are scaled down. However, in full screen mode the controls are the normal size, and the user can pinch out to full screen, so for my team at least the trade-off in increased video quality was worth it.
I have one HTML file control using this i am trying to upload file on server.
Click on file control
It will automatically open dialog for file/camera selection.
I select Camera
I captured photo and save
Now submit form
the Capture photo save successfully on server but it is rotated 90
degree.
This problem is only with iPhone or IOS.
<input id="image" name="image" type="file" accept="image/*" capture style="width: 58px" />
Please let me know, how to stop automatic rotation of image if captured from camera.
Thanks,
Imdadhusen
This could have something to do with EXIF/TIFF metatags in the images.
One of these tags indicates the orientation of the image. Some photos you upload to a server (eg from the iPhone) may have these tags, others (from a different source / workflow) may not. The server may or may not retain these tags, and may or may not read the tags in order to attempt correct orientation of the image. Similarly a web browser may or may not pay attention to these tags. If you upload images containing tags, the results will certainly be unpredictable.
A good way to find out if this is the issue would be to download a 'problem' image and a 'good' image from the server, and compare them. Open them in preview and check the second tab of the Inspector. Any meta-info will be displayed there. See if one image has orientation info and the other not. If neither display meta-info then it is fairly likely the server is stripping it all out and not using it, and you can forget about this answer.
Also you should compare in different browsers. Some browsers read this orientation info and rotate the image accordingly, others do not.
For example, an iPhone photo of mine which was take in portrait mode has the following metatags:
Pixel Height: 2,448
Pixel Width: 3,264
Orientation: 6 (Rotated 90° CCW)
The native bitmap orientation is landscape.
The 'orientation' metatag indicates to image reading software to rotate to portrait for display.
The metatag is used by (some) image software to 'correctly' rotate the image to portrait but ignored by other software which will display the image as landscape.
The situation is confusing and there is no established standard for this (the tags are just a hint), so when preparing images for an online publication the safest thing is to strip out these tags and physically rotate the image to the correct BITMAP orientation before uploading. If you are writing the server side of an app you could do this on the server. But it is vital to deliver images to a web browser in the correct BITMAP orientation with no orientation metatags.
Why is this? Because even today different browsers take different approaches to metatags.
The iPhone image I just mentioned displays as follows:
Google Chrome 24 portrait
Safari 6 portrait
Firefox 17 landscape
Opera 12 landscape
Not a great state of affairs!
I have an app that I would like to have video capture for the front-facing camera only. That's no problem. But I would like the video capture to always be in landscape, even when the phone is being held in portrait.
I have a working implementation based on the AVCamDemo code that Apple published. And borrowing from the information in this tech note, I am able to specify the orientation. There's just one trick: while the video frame is oriented correctly, the contents still appear as though shot in portrait:
I'm wondering if I'm just getting boned by the physical constraints of the hardware: is the image sensor just oriented this way? The referenced tech note above makes this note:
Important: Setting the orientation on a still image output and movie
file output doesn't physically rotate the buffers. For the movie file
output, it applies a track transform (matrix) to the video track so
that the movie is rotated on playback, and for the still image output
it inserts exif metadata that image viewers use to rotate the image
properly when viewing later.
But my playback of that video suggests otherwise. Any insight or suggestions would be appreciated!
Thanks,
Aaron.
To answer your question, yes, the image sensor is just oriented that way. The video camera is an approx 1-megapixel "1080p" camera that has a fixed orientation. The 5MP (or 8MP for 4S, etc) still camera also has a fixed orientation. The lenses themselves don't rotate nor do any of the other camera bits, and hence the feed itself has a fixed orientation.
"But wait!", you say, "pictures I take with the camera app (or API) get rotated correctly. Why is that?" That's cuz iOS takes a look at the orientation of the phone when a picture is taken and stores that information with the picture (as an Exif attachment). Yet video isn't so flagged -- and each frame would have to be individually flagged, and then there's issues about what to do when the user rotates the phone during video....
So, no, you can't ask a video stream or a still image what orientation the phone was in when the video was captured. You can, however, directly ask the phone what orientation it is in now:
UIDeviceOrientation currentOrientation = [UIDevice currentDevice].orientation;
If you do that at the start of video capture (or when you grab a still image from a video feed) you can then use that information to do your own rotation of playback.