I'm building an app whose core functionality is centered around 1-10 second videos. Currently, I'm recording video using PBJVision with the preset set to AVCaptureSessionPresetMedium. A 10 second video is around ~3-5MB. Considering each user could theoretically download hundreds or even thousands of videos a day, I was wondering if there was a more bandwidth efficient way of packing these videos up.
Could WebM be a more suitable container format?
I searched across the web, but couldn't find any articles pertaining to this specific question.
Edit: this looks promising
Modern video codecs (include WebM VP8) usually has compression ratio around 1/50. By adjusting codec parameters we can archive ~ 1/100 (IMHO), but very difficult and horrible picture quality.
Roughly, we can think of 1 camera pixel consist of 1.5 bytes (YUV 12 or 16 bits).
If the resolution is 720x480 and the frame rate is 30/sec,
720 x 480 x 1.5 x 30 = 15,552,000
x 10 sec = 155,520,000
/ 50 = 3,110,400
~= 3MB
It seems PBJVision doing good.
Reducing resolution, or lowering frame rate would be the first consideration, I think.
ios wont playback webm unless you use a software decoder. A software decoder will take more CPU/battery and produce more heat. And webm will not even solve your problem. What you want is to reduce the bitrate, but this will also reduce the quality. So its a trade off.
Related
I'm working on the image compression techniques and analyzing the best algorithms that could produce smaller output within 100 ms or lesser for images of resolution 1920 * 1080 on a laptop with octa core processor for transmission through the network.
I had been using GDI+ and CxImage libraries for image compression, through JPG or PNG compression techniques that gives me the output image within around 30 ms for JPG and around 70 ms using PNG for colorful images, time taken is pretty good but the compressed data size is much higher in size if I go for better quality.
Then I came across google's WebP format. I tried it using libWebP with VC++. The quality and the compression rate is really awesome but the time taken is is much higher than I expected. It takes more than 300 ms and sime times even more than 1 second if I set true for alpha filtering and alpha compression.
Here are my WebpConfig settings
m_webp_config.quality = 50;
m_webp_config.alpha_quality = 0;
m_webp_config.lossless = false;
m_webp_config.method = 3;
m_webp_config.alpha_compression = false;
m_webp_config.alpha_filtering = false;
m_webp_config.autofilter = false;
m_webp_config.filter_sharpness = false;
m_webp_config.filter_strength = 0;
m_webp_config.filter_type = 0;
m_webp_config.use_sharp_yuv = false;
And sometimes I get black images whenever I capture command prompt or notepad++ (I suspect that the problem is with those images with lot of text data but the same is not true with web pages that has huge amount of text)
Am I doing anything wrong with the WebPConfig ? Is there a way to optimize it?.
I didn't find much documentation and any forums that can give me some idea with these problems.
Any help would be appreciated. Thanks in advance.
Try lossless compression? There can be modes of lossless that are faster than the lossy.
If you are in control of the compression and decompression, split the image into 256x256 squares and compress and send them separately. That way you can not only parallelize the computation, but also interleave some of the transmission to happen during the compression, which may simplify your time budgeting for the compression computation.
If you reduce the 'method' value, you will generally find a faster WebP compression. For lossless, you need to reduce BOTH method and quality, they control the same thing in a complicated manner. For lossless, try quality 20 and method 1 (or perhaps there is a 0 method, too, don't remember).
As title states and as searching on google can give, on iOS there is a limit for what the devices can handle for jpeg images.
As per Apple docs (Know iOS Resource Limits):
Because of the memory available on iOS, there are limits on the number
of resources it can process:
The maximum size for decoded GIF, PNG, and TIFF images is 3 megapixels
for devices with less than 256 MB RAM and 5 megapixels for devices
with greater or equal than 256 MB RAM. That is, ensure that width *
height ≤ 3 * 1024 * 1024 for devices with less than 256 MB RAM. Note
that the decoded size is far larger than the encoded size of an image.
The maximum decoded image size for JPEG is 32MP megapixels using
subsampling. JPEG images can be up to 32 megapixels due to
subsampling, which allows JPEG images to decode to a size that has one
sixteenth the number of pixels. JPEG images larger than 2 megapixels
are subsampled—that is, decoded to a reduced size. JPEG subsampling
allows the user to view images from the latest digital cameras.
I added the enfasis on the point that's bugging me mostly. I'm trying to display a fairly big image, but still largely in the above 32MP mentioned limit, specifically its a 3995px * 2138px for a total of 8.5MP and 396kb weight (jpeg quality/compression set to 25 via PS).
Still whenever I call for that image as ex. source of an <img> tag, nothing is displayed on any iOS device I've been able to test, on emulators and couple real devices (iphone4, ipad2, 3, mini...).
Is there anything I'am missing blatantly or maybe I've not understand from the docs above?
What can I do apart replace it with a reduced file size? If forced to replace it, what is the highest width I can reach without breaking? How can I ensure iOS honor the 32MP limit mentioned?
I'm speaking in a website perspective, not a native app on the device.
It doesn't fix your current problem but if you look at image handling in IOS8 there are no longer any image size limits (CoreImage can automatically tile) - perhaps you could target that?
You can split up images and tile them.I routinely display images 180,000 x 120,000 pixels on IOS devices by chopping them up and using a CATiledLayer.
I've searched quite a lot and it seems that couldn't find a definite answer to what is the maximum render size of a video on iOS using AVFoundation.
I need to stitch two or more videos side by side or above each and render them in one new video with a final size larger than 1920 x 1080. So for example if I have two full hd videos (1920 x 1080) side by side the final composition would be 3840 x 1080.
I've tried with AVAssetExportSession and it always shrinks the final video proportionally to max 1920 in width or 1080 in height. It's quite understandable because of all possible AVAssetExportSession settings like preset, file type etc.
I tried also using AVAssetReader and AVAssetWriter but the results are the same. I only have more control over the quality, bitrate etc.
So.. is there a way this can be achieved on iOS or we have to stick to max Full HD?
Thanks
Well... Actually the answer should be YES and also NO. At least of what I've found until now.
H.264 allows higher resolutions only using a higher level profile which is fine. However on iOS the max profile that can be used is AVVideoProfileLevelH264High41 which according the specs, permits a max resolution of 1,920×1,080#30.1 fps or 2,048×1,024#30.0 fps.
So encoding with H.264 won't do the job and the answer should be NO.
The other option is to use other compression/codec. I've tried AVVideoCodecJPEG and was able to render such a video. So the answer should be YES.
But.. the problem is that this video is not playable on iOS which again changes the answer to NO.
To summarise I'd say: it is possible if that video is meant to be used out of the device otherwise the video will simply not be useable.
Hope it will help other people as well and if someone else gives a better, even different answer I'll be glad.
Two parts:
Correct me if I'm wrong, but there isn't a standard video file format that holds 2048 x 1536 frames? (i.e. recording the full resolution of the iPad retina is impossible?)
My app uses a glReadPixels call to record the screen, and appends the pixel buffers to an AVAssetWriterInputPixelBufferAdaptor. If the video needs to be resized to export, what's the best way to do this? I'm trying right now with AVMutableVideoCompositionLayerInstructions and CGAffineTransforms, but it's not working. Any ideas?
Thanks
Sam
Yes , it is possible. My app is also taking big frame video .
Don't use glReadpixels it causes a lot of delay especially if you record big frames as 2048 x 1536
Since iOS 5.0 you can use a faster way using texture cash (link)
Currently i am working on an image app where i am able to capture an image by using AVFoundation framework. But what i am looking is , to capture an image with certain resolution and DPI (may be 300 DPI or greater).
How to do this ?
There have been numerous posts on here about trying to do OCR on camera generated images. The problem is not that the resolution is too low, but that its too high. I cannot find a link right now, but there was a question a year or so ago where in the end, if the image size was reduced by a factor of four or so, the OCR engine worked better. If you examine the image in Preview, what you want is the number of pixels per character to be say 16x16 or 32x32, not 256x256. Frankly I don't know the exact number but I'm sure you can research this and find posts from actual framework users telling you the best size.
Here is a nice response on how to best scale a large image (with a link to code).