I have a use case where I need to downscale a 716x1280 mp4 video to 358x640 (half of the original). Command that I used is
ffmpeg -i ./input.mp4 -vf "scale=640:640:force_original_aspect_ratio=decrease,pad=ceil(iw/2)*2:ceil(ih/2)*2" ./output.mp4
Out of 10 sample videos, 2 of the them suffered impact on colors. Below I have attached a comparison from the one which was impacted the most.
NOTE: The one on the right is a frame from the original video and the frame on the left is the one from the processed (down scaled) video. Notice the colors red and green in the image (even the skin color and hair color were changed).
What I am looking for is
Is there any way I can prevent changes like these happening? Probably some flag on saturation, brightness, contrast or any other parameter.
I am assuming that ffmpeg uses some default settings while downscaling a video. What made ffmpeg change colors only for these two videos? If it made similar changes for the rest of the videos as well, how to predict this behaviour before hand?
EDIT:
What I already have Tried?
-crf with values 0 and 18.
-preset veryslow as mentioned here
None helped
Mediainfo input V/S output
param
input
output
color range
Limited
NA (attribute not in description)
color primaries
BT.2020
NA (attribute not in description)
transfer characteristics
HLG
NA (attribute not in description)
matrix coefficients
BT.2020 non-constant
NA (attribute not in description)
bit deapth
8
8
Logs of the ffmpeg command
ffmpeg -i ./input.mp4 -vf "scale=640:640:force_original_aspect_ratio=decrease,pad=ceil(iw/2)*2:ceil(ih/2)*2" -movflags +faststart ./output.mp4
ffmpeg version 4.3.1 Copyright (c) 2000-2020 the FFmpeg developers
built with Apple clang version 12.0.0 (clang-1200.0.32.28)
configuration: --prefix=/usr/local/Cellar/ffmpeg/4.3.1_9 --enable-shared --enable-pthreads --enable-version3 --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libdav1d --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librtmp --enable-libspeex --enable-libsoxr --enable-videotoolbox --enable-libzmq --enable-libzimg --disable-libjack --disable-indev=jack
libavutil 56. 51.100 / 56. 51.100
libavcodec 58. 91.100 / 58. 91.100
libavformat 58. 45.100 / 58. 45.100
libavdevice 58. 10.100 / 58. 10.100
libavfilter 7. 85.100 / 7. 85.100
libavresample 4. 0. 0 / 4. 0. 0
libswscale 5. 7.100 / 5. 7.100
libswresample 3. 7.100 / 3. 7.100
libpostproc 55. 7.100 / 55. 7.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from './input.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf58.45.100
Duration: 00:00:30.05, start: 0.000000, bitrate: 10366 kb/s
Stream #0:0(und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(tv, bt2020nc/bt2020/arib-std-b67), 716x1280, 10116 kb/s, 30 fps, 30 tbr, 19200 tbn, 38400 tbc (default)
Metadata:
handler_name : Core Media Video
Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 245 kb/s (default)
Metadata:
handler_name : Core Media Audio
Stream mapping:
Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
Stream #0:1 -> #0:1 (aac (native) -> aac (native))
Press [q] to stop, [?] for help
[libx264 # 0x7faab4808800] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 # 0x7faab4808800] profile High, level 3.0, 4:2:0, 8-bit
[libx264 # 0x7faab4808800] 264 - core 161 r3027 4121277 - H.264/MPEG-4 AVC codec - Copyleft 2003-2020 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=12 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to './output.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf58.45.100
Stream #0:0(und): Video: h264 (libx264) (avc1 / 0x31637661), yuv420p, 358x640, q=-1--1, 30 fps, 15360 tbn, 30 tbc (default)
Metadata:
handler_name : Core Media Video
encoder : Lavc58.91.100 libx264
Side data:
cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 128 kb/s (default)
Metadata:
handler_name : Core Media Audio
encoder : Lavc58.91.100 aac
[mp4 # 0x7faab5808800] Starting second pass: moving the moov atom to the beginning of the file
frame= 901 fps=210 q=-1.0 Lsize= 3438kB time=00:00:30.02 bitrate= 938.0kbits/s speed=7.01x
video:2933kB audio:472kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.974633%
[libx264 # 0x7faab4808800] frame I:6 Avg QP:22.60 size: 20769
[libx264 # 0x7faab4808800] frame P:228 Avg QP:24.84 size: 7657
[libx264 # 0x7faab4808800] frame B:667 Avg QP:27.59 size: 1697
[libx264 # 0x7faab4808800] consecutive B-frames: 0.9% 0.9% 1.0% 97.2%
[libx264 # 0x7faab4808800] mb I I16..4: 9.5% 64.6% 26.0%
[libx264 # 0x7faab4808800] mb P I16..4: 2.5% 12.2% 2.5% P16..4: 37.2% 20.6% 11.2% 0.0% 0.0% skip:13.7%
[libx264 # 0x7faab4808800] mb B I16..4: 0.4% 2.1% 0.2% B16..8: 42.2% 7.1% 1.2% direct: 1.8% skip:44.9% L0:39.4% L1:52.8% BI: 7.8%
[libx264 # 0x7faab4808800] 8x8 transform intra:72.2% inter:74.2%
[libx264 # 0x7faab4808800] coded y,uvDC,uvAC intra: 61.8% 67.2% 20.2% inter: 16.7% 13.9% 1.3%
[libx264 # 0x7faab4808800] i16 v,h,dc,p: 24% 19% 7% 50%
[libx264 # 0x7faab4808800] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 21% 16% 15% 6% 9% 11% 7% 10% 6%
[libx264 # 0x7faab4808800] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 25% 16% 13% 7% 9% 10% 7% 9% 4%
[libx264 # 0x7faab4808800] i8c dc,h,v,p: 53% 16% 26% 5%
[libx264 # 0x7faab4808800] Weighted P-Frames: Y:3.9% UV:1.8%
[libx264 # 0x7faab4808800] ref P L0: 57.8% 19.5% 14.8% 7.8% 0.1%
[libx264 # 0x7faab4808800] ref B L0: 90.7% 7.2% 2.1%
[libx264 # 0x7faab4808800] ref B L1: 95.3% 4.7%
[libx264 # 0x7faab4808800] kb/s:799.80
[aac # 0x7faab2036a00] Qavg: 189.523
We may use Bit Stream Video Filter for setting h264 metadata.
When a video player plays a video file, it looks for metadata that attached to the video stream (h264 metadata for example).
The H.264 metadata parameters that affects the colors and brightness are: video_full_range_flag, colour_primaries, transfer_characteris and matrix_coefficients.
If the parameters are not set, there are defaults.
The defaults for low resolution video are "Limited Range" BT.601 (in most player - I am not sure about MAC OS).
The default gamma curve (affects the brightness) is sRGB gamma curve.
The player converts the pixels from YUV color space to RGB (for displaying the video). The conversion formula is done according to the metadata.
Your input video file input.mp4 has H.264 metadata parameters that are far from the default.
We can assume that scale video filter does not change the color characteristics (the filter applies the YUV elements without converting to RGB).
The characteristics of input.mp4 applies BT.2020, and HLG gamma curve, but converted as if they were default (BT.601 and sRGB gamma), so the colors and brightness are very different from what they should have been.
When FFmpeg encodes a video stream, it does not copy the metadata parameters from the input to the output - we need to set the parameters explicitly.
The solution is using a Bit Stream Video Filter for setting the metadata parameters.
Try using the following command:
ffmpeg -i ./input.mp4 -vf "scale=640:640:force_original_aspect_ratio=decrease,pad=ceil(iw/2)*2:ceil(ih/2)*2" -vcodec libx264 -crf 17 -pix_fmt yuv420p -bsf:v h264_metadata=video_full_range_flag=0:colour_primaries=9:transfer_characteristics=18:matrix_coefficients=9 ./output.mp4
video_full_range_flag=0 applies "limited color range".
colour_primaries=9 applies BT.2020 colour primaries.
transfer_characteristics=18 applies HLG gamma (see
ITU-T Rec. Series H)
matrix_coefficients=9 applies BT.2020 matrix coefficients.
Most of the parameters are documented in ITU-T Rec. H.264 (section E.2.1).
Checking the parameters of output.mp4 using MediaInfo tool:
Color range : Limited
Color primaries : BT.2020
Transfer characteristics : HLG
Matrix coefficients : BT.2020 non-constant
Related
I tried below but it is only showing the background video.
background_video = ffmpeg.input( "input.mp4")
overlay_video = ffmpeg.input(f'{frames_folder}*.png', pattern_type='glob', framerate=25)
subprocess = ffmpeg.overlay(
background_video,
overlay_video,
).filter("setsar", sar=1)
I also tried to assemble sequence of frames into .webm/.mov video but transparency is lost. video is taking black as background.
P.s - frame size is same as background video size. So no scaling needed.
Edit
I tried #Rotem suggestions
Try using single PNG image first
overlay_video = ffmpeg.input('test-frame.png')
It's not working for frames generated by OpenCV but working for any other png image. This is weird, when I'm manually viewing these frames folder it's showing blank images(Link to my frames folder).
But If I convert these frames into the video(see below) it is showing correctly what I draw on each frame.
output_options = {
'crf': 20,
'preset': 'slower',
'movflags': 'faststart',
'pix_fmt': 'yuv420p'
}
ffmpeg.input(f'{frames_folder}*.png', pattern_type='glob', framerate=25 , reinit_filter=0).output(
'movie.avi',
**output_options
).global_args('-report').run()
try creating a video from all the PNG images without overlay
It's working as expected only issue is transparency. Is there is way to create a transparent background video? I tried .webm/.mov/.avi but no luck.
Add .global_args('-report') and check the log file
Report written to "ffmpeg-20221119-110731.log"
Log level: 48
ffmpeg version 5.1 Copyright (c) 2000-2022 the FFmpeg developers
built with Apple clang version 13.1.6 (clang-1316.0.21.2.5)
configuration: --prefix=/opt/homebrew/Cellar/ffmpeg/5.1 --enable-shared --enable-pthreads --enable-version3 --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libdav1d --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librist --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libspeex --enable-libsoxr --enable-libzmq --enable-libzimg --disable-libjack --disable-indev=jack --enable-videotoolbox --enable-neon
libavutil 57. 28.100 / 57. 28.100
libavcodec 59. 37.100 / 59. 37.100
libavformat 59. 27.100 / 59. 27.100
libavdevice 59. 7.100 / 59. 7.100
libavfilter 8. 44.100 / 8. 44.100
libswscale 6. 7.100 / 6. 7.100
libswresample 4. 7.100 / 4. 7.100
libpostproc 56. 6.100 / 56. 6.100
Input #0, image2, from './frames/*.png':
Duration: 00:00:05.00, start: 0.000000, bitrate: N/A
Stream #0:0: Video: png, rgba(pc), 1920x1080, 25 fps, 25 tbr, 25 tbn
Codec AVOption crf (Select the quality for constant quality mode) specified for output file #0 (movie.avi) has not been used for any stream. The most likely reason is either wrong type (e.g. a video option with no video streams) or that it is a private option of some encoder which was not actually used for any stream.
Codec AVOption preset (Configuration preset) specified for output file #0 (movie.avi) has not been used for any stream. The most likely reason is either wrong type (e.g. a video option with no video streams) or that it is a private option of some encoder which was not actually used for any stream.
Stream mapping:
Stream #0:0 -> #0:0 (png (native) -> mpeg4 (native))
Press [q] to stop, [?] for help
Output #0, avi, to 'movie.avi':
Metadata:
ISFT : Lavf59.27.100
Stream #0:0: Video: mpeg4 (FMP4 / 0x34504D46), yuv420p(tv, progressive), 1920x1080, q=2-31, 200 kb/s, 25 fps, 25 tbn
Metadata:
encoder : Lavc59.37.100 mpeg4
Side data:
cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: N/A
frame= 125 fps= 85 q=31.0 Lsize= 491kB time=00:00:05.00 bitrate= 804.3kbits/s speed=3.39x
video:482kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 1.772174%
To draw frame I used below.
for i in range(num_frames):
transparent_img = np.zeros((height, width, 4), dtype=np.uint8)
cv2.line(transparent_img, (x1,y1), (x2,y2) ,(255, 255, 255), thickness=1, lineType=cv2.LINE_AA)
self.frames.append(transparent_img)
## To Save each frame of the video in the given folder
for i, f in enumerate(frames):
cv2.imwrite("{}/{:0{n}d}.png".format(path_to_frames, i, n=num_digits), f)
Here are answers to your two questions:
For drawing a white line on BGRA image, use (255, 255, 255, 255) color instead of (255, 255, 255).
The last 255 applies alpha (transparency) channel value, and 255 makes the line fully opaque.
For creating video with transparent background try: .webm file type, use libvpx-vp9 video codec, and use -pix_fmt yuva420p - the a of yuva applies alpha (transparency) channel.
Here is a "self contained" code sample (please read the comments):
import cv2
import numpy as np
import ffmpeg
# Create synthetic MP4 video file from testing
ffmpeg.input('testsrc=size=192x108:rate=1:duration=10', f='lavfi').output('tmp.mp4').overwrite_output().run()
transparent_img = np.zeros((108, 192, 4), np.uint8)
width, height, fps = 192, 108, 1
def make_sample_image(i):
p = width//60
img = np.zeros((height, width, 4), np.uint8) # Fully transparent
cv2.putText(img, str(i), (width//2-p*10*len(str(i)), height//2+p*10), cv2.FONT_HERSHEY_DUPLEX, p, (255, 255, 255, 255), p*2) # White number
return img
# Create 10 PNG files with transparent background an white number (counter).
for i in range(1, 11):
transparent_img = make_sample_image(i)
cv2.imwrite(f'{i:03d}.png', transparent_img)
output_options = { 'vcodec' : 'libvpx-vp9', # libvpx-vp9 supports transparency.
'crf': 20,
#'preset': 'slower', # Not supported by libvpx-vp9
#'movflags': 'faststart', # Not supported by WebM
'pix_fmt': 'yuva420p' # yuva420p includes transparency.
}
frames_folder = './'
# Create video with transparency:
# reinit_filter=0 is required, only if the PNG images have different characteristics (example: some are RGB and some RGBA).
# Use %03d.png instead of glob pattern, becuase my Windows version of FFmpeg doesn't support glob pattern.
ffmpeg.input(f'{frames_folder}%03d.png', framerate=fps, reinit_filter=0).output(
'movie.webm', # WebM container supports transparency
**output_options
).global_args('-report').overwrite_output().run()
# Overlay the PNG on top of tmp.mp4
background_video = ffmpeg.input( "tmp.mp4")
overlay_video = ffmpeg.input(f'{frames_folder}%03d.png', framerate=fps)
subprocess = ffmpeg.overlay(
background_video,
overlay_video,
).filter("setsar", sar=1)
subprocess.output('overlay_video.webm', **output_options).global_args('-report').overwrite_output().run()
I have created a screencast, it was saved as webm. Using ffmpeg I converted it to a mp4:
That kind of file isn't supported yet. Try uploading a gif, video, or picture instead.
ffmpeg -f input.webm -pix_fmt yuv420p -strict -2 output.mp4
I use yuv420p as that is what this topic says for twitter - https://twittercommunity.com/t/unable-to-upload-video-to-twitter/61721/3
However when I open a new tab, go to twitter, click the photo icon, clicking upload instantly gives me error "That kind of file isn't supported yet. Try uploading a gif, video, or picture instead.". It cant check the pix format that fast. Anyonek now what can be going on here?
Here is ffmpeg console output:
ffmpeg version 2.2.1 Copyright (c) 2000-2014 the FFmpeg developers
built on Jun 9 2014 20:01:41 with emcc (Emscripten GCC-like replacement) 1.12.0 (commit 6960d2296299e96d43e694806f5d35799ef8d39c)
configuration: --cc=emcc --prefix=/Users/bgrinstead/Sites/videoconverter.js/build/ffmpeg/../dist --enable-cross-compile --target-os=none --arch=x86_32 --cpu=generic --disable-ffplay --disable-ffprobe --disable-ffserver --disable-asm --disable-doc --disable-devices --disable-pthreads --disable-w32threads --disable-network --disable-hwaccels --disable-parsers --disable-bsfs --disable-debug --disable-protocols --disable-indevs --disable-outdevs --enable-protocol=file
libavutil 52. 66.100 / 52. 66.100
libavcodec 55. 52.102 / 55. 52.102
libavformat 55. 33.100 / 55. 33.100
libavdevice 55. 10.100 / 55. 10.100
libavfilter 4. 2.100 / 4. 2.100
libswscale 2. 5.102 / 2. 5.102
libswresample 0. 18.100 / 0. 18.100
[vp8 # 0xdde410] Warning: not compiled with thread support, using thread emulation
Input #0, matroska,webm, from 'input.webm':
Metadata:
encoder : QTmuxingAppLibWebM-0.0.1
Duration: N/A, start: 0.000000, bitrate: N/A
Stream #0:0(eng): Video: vp8, yuv420p, 1920x1200, SAR 1:1 DAR 8:5, 30 fps, 30 tbr, 1k tbn, 1k tbc (default)
Incompatible pixel format 'yuv420p' for codec 'gif', auto-selecting format 'pal8'
[swscaler # 0xdd7f40] No accelerated colorspace conversion found from yuv420p to bgr8.
[gif # 0xde5290] Warning: not compiled with thread support, using thread emulation
[vp8 # 0xdde410] Warning: not compiled with thread support, using thread emulation
Output #0, gif, to 'output/output.gif':
Metadata:
encoder : Lavf55.33.100
Stream #0:0(eng): Video: gif, pal8, 1920x1200 [SAR 1:1 DAR 8:5], q=2-31, 200 kb/s, 100 tbn, 30 tbc (default)
Stream mapping:
Stream #0:0 -> #0:0 (vp8 -> gif)
Press [q] to stop, [?] for help
frame= 1 fps=0.0 q=0.0 size= 0kB time=00:00:00.03 bitrate= 8.5kbits/s
frame= 6 fps=0.0 q=0.0 size= 640kB time=00:00:00.33 bitrate=15877.0kbits/s
frame= 12 fps= 11 q=0.0 size= 1452kB time=00:00:00.63 bitrate=18878.9kbits/s
frame= 17 fps= 10 q=0.0 size= 2080kB time=00:00:00.91 bitrate=18723.7kbits/s
frame= 22 fps=9.6 q=0.0 size= 2698kB time=00:00:01.16 bitrate=19052.0kbits/s
frame= 27 fps=9.7 q=0.0 size= 3294kB time=00:00:01.43 bitrate=18869.4kbits/s
frame= 33 fps=9.6 q=0.0 size= 4001kB time=00:00:01.76 bitrate=18622.0kbits/s
frame= 38 fps=9.5 q=0.0 size= 4567kB time=00:00:02.04 bitrate=18339.7kbits/s
frame= 44 fps=9.5 q=0.0 size= 5236kB time=00:00:02.37 bitrate=18097.4kbits/s
frame= 50 fps=9.7 q=0.0 size= 5902kB time=00:00:02.71 bitrate=17839.9kbits/s
frame= 55 fps=9.6 q=0.0 size= 6457kB time=00:00:02.96 bitrate=17868.9kbits/s
frame= 55 fps=9.6 q=0.0 Lsize= 6567kB time=00:00:02.96 bitrate=18175.9kbits/s
video:6623kB audio:0kB subtitle:0 data:0 global headers:0kB muxing overhead -0.834044%
Here is a screenshot:
I am using the gem streamio-ffmpeg to convert an .mp4 file to another .mp4 file that conforms to iOS standards. I want to put that file in an HTML5 video tag. The video plays well on desktop and Android devices. However, no matter what I do, the iPad Mini 2 that I'm using will not play the video.
My options hash looks like this:
options = {video_codec: "libx264", resolution: "480x320", frame_rate: 30, video_bitrate: 500,
x264_vprofile: "baseline", x264_preset: "slow", audio_codec: "aac", audio_bitrate: 56, aspect: 1.333333,
audio_sample_rate: 44100, audio_channels: 1, custom: "-strict -2"}
When I load the new .mp4 file back into streamio-ffmpeg, this is the object (and metadata) returned:
#<FFMPEG::Movie:0x007fb37b663ec0
#path="/Users/ryan/workspace/videotest/app/assets/videos/video1-i.mp4",
#container="mov,mp4,m4a,3gp,3g2,mj2", #duration=5.34, #time=0.02322,
#creation_time=nil,
#bitrate=1352,
#rotation=nil,
#video_stream="h264 (Constrained Baseline) (avc1 / 0x31637661), yuv420p, 480x320 [SAR 1:1 DAR 3:2], 1231 kb/s, 30 fps, 30 tbr, 15360 tbn, 60 tbc (default)",
#audio_stream="aac (LC) (mp4a / 0x6134706D), 44100 Hz, mono, fltp, 128 kb/s (default)",
#video_codec="h264 (Constrained Baseline) (avc1 / 0x31637661)",
#colorspace="yuv420p",
#video_bitrate=1231,
#resolution="480x320",
#dar="3:2",
#audio_codec="aac (LC) (mp4a / 0x6134706D)",
#audio_channels="mono",
#audio_bitrate=nil,
#audio_sample_rate=44100>
I've tried just about everything I could find, with different bit rates, codecs, resolutions, etc. What are the correct options to play an .mp4 on an iOS device?
BTW, streamio-ffmpeg allows pure ffmpeg options, so I'm ok with that.
I have some H.264-encoded videos which render in HTML5 correctly in the web browser, but do not render correctly on the iPad. When I use a H.264 video I downloaded off the internet, my video renders correctly on the iPad, so it is not an HTML problem.
Here is the ffmpeg info about my videos --
My original .mov video:
Seems stream 1 codec frame rate differs from container frame rate: 6000.00 (6000/1) -> 30.00 (30/1)
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'a_video.mp4':
Metadata:
major_brand : qt
minor_version : 537199360
compatible_brands: qt
Duration: 00:00:42.74, start: 0.000000, bitrate: 220 kb/s
Stream #0.0(eng): Audio: aac, 44100 Hz, stereo, s16, 94 kb/s
Stream #0.1(eng): Video: h264, yuv420p, 762x464, 122 kb/s, 30 fps, 30 tbr, 3k tbn, 6k tbc
After using Handbrake to convert my .mov to a mp4, yet doesn't render on the iPad:
Seems stream 0 codec frame rate differs from container frame rate: 180000.00 (180000/1) -> 29.97 (30000/1001)
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'a_video.m4v':
Metadata:
major_brand : mp42
minor_version : 0
compatible_brands: mp42isomavc1
encoder: HandBrake 0.9.5 2011010300
Duration: 00:00:42.77, start: 0.000000, bitrate: 169 kb/s
Stream #0.0(und): Video: h264, yuv420p, 752x464 [PAR 381:376 DAR 381:232], 35 kb/s, PAR 145161:141376 DAR 145161:87232, 29.97 fps, 29.97 tbr, 90k tbn, 180k tbc
Stream #0.1(eng): Audio: aac, 44100 Hz, stereo, s16, 128 kb/s
Here is a .mp4 I found online which does render on the iPad:
Seems stream 1 codec frame rate differs from container frame rate: 180000.00 (180000/1) -> 25.00 (25/1)
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'a_video_3_emu.mp4':
Metadata:
major_brand : M4VP
minor_version : 1
compatible_brands: M4VPM4A mp42isom
encoder : CoreMediaAuthoring 677, CoreMedia 420.17, i386
Duration: 00:01:38.01, start: 0.000000, bitrate: 1023 kb/s
Stream #0.0(und): Audio: aac, 32000 Hz, mono, s16, 97 kb/s
Stream #0.1(und): Video: h264, yuv420p, 480x360 [PAR 1:1 DAR 4:3], 914 kb/s, 25 fps, 25 tbr, 90k tbn, 180k tbc
Does anyone see something wrong with the way I am encoded my videos?
Edit
At first my theory was that the iPad was sensitive to different container formats; but that appears not to be the case. I took a video which does render correctly on the iPad and converted it to a .mov, and it still played correctly on the iPad. So there must be a problem with how the iPad deals with the underlying H.264 stream.
If you have a H.264 video stream -- regardless of the container (mov, m4v, mp4) -- and your HTML5 video renders in a web browser but doesn't render on the iPad, there are two possible fixes:
The first solution is to convert the H.264 video stream to mpeg4.
ffmpeg -i video_h264_not_working.mov -acodec copy -vcodec mpeg4 video_mpeg.mov
(Alternatively, you can select MPEG4 instead of H.264 in Handbrake.)
The second solution is to re-process the H.264 video with the following parameters:
ffmpeg -i video_h264_not_working.mov -vcodec libx264 -r 25 -b 516k -bt 516k -crf 22 -vpre normal video_h264.mov
The second solution came from: http://houseoflaudanum.com/navigate/howtos/html5-video-no-webm/
I'm guessing in the former case, the mpeg4 codec is more relaxed on the iPad; and in the latter case, the iPad didn't like some of the stream parameters from the original H.264 encoding, so a "cleaning" was necessary.
To determine what your video stream actually is, just do ffmpeg -i myvideo.mov.
I've had great success encoding and transcoding HTML5 video using the free Miro Video Converter (OS X, Windows) as recommended in the Video on the Web section of Dive Into HTML5 by Mark Pilgrim.
Miro Video Converter can convert virtually any video file to MP4, Theora, or MP3 (the audio only). It has presets that will convert video to the correct sizes and formats for popular phones, iPods, and other media players. Just convert your video and copy it to your device.
It's a super simple way to convert almost any video to MP4, WebM (vp8), Ogg Theora, or for Android, iPhone, and more. You'll feel good that it's 100% Free and open-source, too.
An another veritable video conversion utility is Video Monkey, a free video encoding application exclusively for Mac. It was created after the demise of the great tool Visual Hub. Video Monkey borrows heavily from the Visual Hub video conversion tool, both conceptually and from the original code dump posted to SourceForge as TranscoderRedux.
sudo port install ffmpeg and try this secret sauce:
#!/bin/bash
BR=512k
WIDTH=640
HEIGHT=272
input=${1}
# strip off the file extension
output=$(echo ${input} | sed 's/\..*//' )
# works for most videos
ffmpeg -y -i ${input} -f mpegts -acodec libmp3lame -ar 48000 -ab 64k -s ${WIDTH}x${HEIGHT} -vcodec libx264 -b ${BR} -flags +loop -cmp +chroma -partitions +parti4x4+partp8x8+partb8x8 -subq 7 -trellis 0 -refs 0 -coder 0 -me_range 16 -keyint_min 25 -sc_threshold 40 -i_qfactor 0.71 -bt 200k -maxrate ${BR} -bufsize ${BR} -rc_eq 'blurCplx^(1-qComp)' -qcomp 0.6 -qmin 30 -qmax 51 -qdiff 4 -level 30 -aspect ${WIDTH}:${HEIGHT} -g 30 -async 2 ${output}-iphone.ts
Adjust width and height to the original video and set the bitrate as you see fit. Pass the original video as parameter to the script. Works with almost any source format. You need macports for the port command.
I'm wondering weather this is an issue with iOS devices or just the iPad (I've actually only had an iPad to play on).
If I were to write a HTML link that linked to an MP4 on a server (over HTTP), the iPad launches it in Safari but doesn't recognise that it's an MP4 and just serves a blank page.
I've had a look on other devices, such as Android for eg, and this fires up the media player and streams correctly.
Is there a specific way to get it to play in the default media player on iPad/iOS?
The problem is more than likely the particular encoding characteristics of the MP4. h264 and MPEG-4 have loads of little options and tweakable parameters. iOS devices are fairly specific about what they do and do not like. (To be fair, this is not a problem specific to iOS devices.)
Have a look at Brightcove's encoding recommendations and re-encode your media.
That failing, and assuming you have FFmpeg installed, post the results of ffmpeg -i problemfile.mp4 and comment below so I can have a look.
I'm experiencing a very similar problem with the iPad in playing an mp4 file, which in fact was encoded by Brightcove.
The video won't start inside an html5 video tag using the iPad, while it works perfectly in Chrome, Safari and IE9.
You can grab the video here: http://tinyurl.com/796c5ub and this is the output of $ffmpeg -i video1.mp4:
ffmpeg version 0.7.1, Copyright (c) 2000-2011 the FFmpeg developers
built on Jul 31 2011 18:31:40 with clang 2.1 (tags/Apple/clang-163.7.1)
configuration: --prefix=/opt/local --enable-gpl --enable-postproc --enable-swscale --enable-avfilter --enable-libmp3lame --enable-libvorbis --enable-libtheora --enable-libdirac --enable-libschroedinger --enable-libfaac --enable-libxvid --enable-libx264 --enable-libvpx --enable-libspeex --enable-nonfree --mandir=/opt/local/share/man --enable-shared --enable-pthreads --disable-indevs --cc=/Developer/usr/bin/clang --arch=x86_64
libavutil 50. 43. 0 / 50. 43. 0
libavcodec 52.122. 0 / 52.122. 0
libavformat 52.110. 0 / 52.110. 0
libavdevice 52. 5. 0 / 52. 5. 0
libavfilter 1. 80. 0 / 1. 80. 0
libswscale 0. 14. 1 / 0. 14. 1
libpostproc 51. 2. 0 / 51. 2. 0
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'video1.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
creation_time : 2011-12-06 08:00:00
encoder : Lavf53.17.0
Duration: 00:00:45.60, start: 0.000000, bitrate: 1818 kb/s
Stream #0.0(eng): Video: h264 (High), yuv420p, 1280x852, 1687 kb/s, 25 fps, 25 tbr, 25 tbn, 50 tbc
Metadata:
creation_time : 2011-12-06 08:00:00
Stream #0.1(eng): Audio: aac, 44100 Hz, stereo, s16, 128 kb/s
Metadata:
creation_time : 2011-12-06 08:00:00
At least one output file must be specified
Found this while looking for a solution for iOS-related MP4-audio problems: Some of my MP4 audio files would start to play, but at a certain point in the file, playback on iOS would hang, while non-iOS players played the same MP4 files flawlessly till the end.
MP4 is a container format. Said to also contain non-audio/non-video streams and metadata headers that describe audio/video. iOS seems to dislike some stream types and if some headers are at unexpected positions in the file.
mp4box and ffmpeg can fix these headers and remove unwanted streams.
I only fiddled with mp4box, that did it for me to fix the audio problems:
mp4box -single 1 input.mp4 -out outputfixed.mp4
The "1" is the stream number containing the audio in my case (may vary, mp4box can also list the contained streams).
Later on I found that the GUI tool "MP4tag", that is primarily meant to edit tags, has a menu function called "MP4 optimize" that is said to be able to fix the same problem.
mp4box should also be able to fix vid+audio combinations, by retrieving more than one stream from the input file.