ffmpeg conversion: Twitter rejects video with "Not valid video" - twitter

I have an app that uploads via twitter api chunked upload, and it finally works with photos.
However, I am trying to get it to work with videos.
Uploading didnt work out-of-the-box even though the video format is mp4. The twitter guidelines for uploads are these
With that in mind, I have this ffmpeg command so far:
ffmpeg -i in.mp4 -vf \"scale=1280:720\" -b:v 5000K -minrate 5000K -maxrate 5000K -b:a 128K -r 30 -f mp4 -vcodec libx264 -profile:v high -pix_fmt yuv420p -strict -2 -ac 2 -acodec aac out.mp4
I still get "Not valid video", and I don't know why.
Here's my mediainfo output for out.mp4:
General
Count : 328
Count of stream of this kind : 1
Kind of stream : General
Kind of stream : General
Stream identifier : 0
Count of video streams : 1
Count of audio streams : 1
Video_Format_List : AVC
Video_Format_WithHint_List : AVC
Codecs Video : AVC
Video_Language_List : English
Audio_Format_List : AAC
Audio_Format_WithHint_List : AAC
Audio codecs : AAC LC
Audio_Language_List : English
Complete name : video-5e4405cd4348a5e4405cd434d2.mp4
File name : video-5e4405cd4348a5e4405cd434d2
File extension : mp4
Format : MPEG-4
Format : MPEG-4
Format/Extensions usually used : mov mp4 m4v m4a m4b m4p 3ga 3gpa 3gpp 3gp 3gpp2 3g2 k3g jpm jpx mqv ismv isma ismt f4a f4b f4v
Commercial name : MPEG-4
Format profile : Base Media
Internet media type : video/mp4
Codec ID : isom
Codec ID : isom (isom/iso2/avc1/mp41)
Codec ID/Url : http://www.apple.com/quicktime/download/standalone.html
CodecID_Compatible : isom/iso2/avc1/mp41
Codec : MPEG-4
Codec : MPEG-4
Codec/Extensions usually used : mov mp4 m4v m4a m4b m4p 3ga 3gpa 3gpp 3gp 3gpp2 3g2 k3g jpm jpx mqv ismv isma ismt f4a f4b f4v
File size : 52664272
File size : 50.2 MiB
File size : 50 MiB
File size : 50 MiB
File size : 50.2 MiB
File size : 50.22 MiB
Duration : 79744
Duration : 1 min 19 s
Duration : 1 min 19 s 744 ms
Duration : 1 min 19 s
Duration : 00:01:19.744
Duration : 00:01:19:20
Duration : 00:01:19.744 (00:01:19:20)
Overall bit rate : 5283334
Overall bit rate : 5 283 kb/s
Frame rate : 30.000
Frame rate : 30.000 FPS
Frame count : 2390
Stream size : 88780
Stream size : 86.7 KiB (0%)
Stream size : 87 KiB
Stream size : 87 KiB
Stream size : 86.7 KiB
Stream size : 86.70 KiB
Stream size : 86.7 KiB (0%)
Proportion of this stream : 0.00169
HeaderSize : 40
DataSize : 52575500
FooterSize : 88732
IsStreamable : No
File last modification date : UTC 2020-02-12 14:05:37
File last modification date (local) : 2020-02-12 15:05:37
Writing application : Lavf57.83.100
Writing application : Lavf57.83.100
Video
Count : 342
Count of stream of this kind : 1
Kind of stream : Video
Kind of stream : Video
Stream identifier : 0
StreamOrder : 0
ID : 1
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format/Url : http://developers.videolan.org/x264.html
Commercial name : AVC
Format profile : High#L3.1
Format settings : CABAC / 4 Ref Frames
Format settings, CABAC : Yes
Format settings, CABAC : Yes
Format settings, ReFrames : 4
Format settings, ReFrames : 4 frames
Internet media type : video/H264
Codec ID : avc1
Codec ID/Info : Advanced Video Coding
Codec : AVC
Codec : AVC
Codec/Family : AVC
Codec/Info : Advanced Video Codec
Codec/Url : http://developers.videolan.org/x264.html
Codec/CC : avc1
Codec profile : High#L3.1
Codec settings : CABAC / 4 Ref Frames
Codec settings, CABAC : Yes
Codec_Settings_RefFrames : 4
Duration : 79667
Duration : 1 min 19 s
Duration : 1 min 19 s 667 ms
Duration : 1 min 19 s
Duration : 00:01:19.667
Duration : 00:01:19:20
Duration : 00:01:19.667 (00:01:19:20)
Bit rate : 5000000
Bit rate : 5 000 kb/s
Width : 1280
Width : 1 280 pixels
Height : 720
Height : 720 pixels
Sampled_Width : 1280
Sampled_Height : 720
Pixel aspect ratio : 1.000
Display aspect ratio : 1.778
Display aspect ratio : 16:9
Rotation : 0.000
Frame rate mode : CFR
Frame rate mode : Constant
FrameRate_Mode_Original : VFR
Frame rate : 30.000
Frame rate : 30.000 FPS
Frame count : 2390
Resolution : 8
Resolution : 8 bits
Colorimetry : 4:2:0
Color space : YUV
Chroma subsampling : 4:2:0
Chroma subsampling : 4:2:0
Bit depth : 8
Bit depth : 8 bits
Scan type : Progressive
Scan type : Progressive
Interlacement : PPF
Interlacement : Progressive
Bits/(Pixel*Frame) : 0.181
Stream size : 51297022
Stream size : 48.9 MiB (97%)
Stream size : 49 MiB
Stream size : 49 MiB
Stream size : 48.9 MiB
Stream size : 48.92 MiB
Stream size : 48.9 MiB (97%)
Proportion of this stream : 0.97404
Writing library : x264 - core 152 r2854 e9a5903
Writing library : x264 core 152 r2854 e9a5903
Encoded_Library_Name : x264
Encoded_Library_Version : core 152 r2854 e9a5903
Encoding settings : cabac=1 / ref=3 / deblock=1:0:0 / analyse=0x3:0x113 / me=hex / subme=7 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=12 / lookahead_threads=2 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=25 / scenecut=40 / intra_refresh=0 / rc_lookahead=40 / rc=abr / mbtree=1 / bitrate=5000 / ratetol=1.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00
Language : en
Language : English
Language : English
Language : en
Language : eng
Language : en
Audio
Count : 275
Count of stream of this kind : 1
Kind of stream : Audio
Kind of stream : Audio
Stream identifier : 0
StreamOrder : 1
ID : 2
ID : 2
Format : AAC
Format/Info : Advanced Audio Codec
Commercial name : AAC
Format profile : LC
Format settings, SBR : No (Explicit)
Format settings, SBR : No (Explicit)
Codec ID : mp4a-40-2
Codec : AAC LC
Codec : AAC LC
Codec/Family : AAC
Codec/CC : 40
Duration : 79744
Duration : 1 min 19 s
Duration : 1 min 19 s 744 ms
Duration : 1 min 19 s
Duration : 00:01:19.744
Duration : 00:01:19:25
Duration : 00:01:19.744 (00:01:19:25)
Bit rate mode : CBR
Bit rate mode : Constant
Bit rate : 128257
Bit rate : 128 kb/s
Channel(s) : 2
Channel(s) : 2 channels
Channel positions : Front: L R
Channel positions : 2/0/0
ChannelLayout : L R
Samples per frame : 1024
Sampling rate : 48000
Sampling rate : 48.0 kHz
Samples count : 3827712
Frame rate : 46.875
Frame rate : 46.875 FPS (1024 SPF)
Frame count : 3738
Compression mode : Lossy
Compression mode : Lossy
Stream size : 1278470
Stream size : 1.22 MiB (2%)
Stream size : 1 MiB
Stream size : 1.2 MiB
Stream size : 1.22 MiB
Stream size : 1.219 MiB
Stream size : 1.22 MiB (2%)
Proportion of this stream : 0.02428
Language : en
Language : English
Language : English
Language : en
Language : eng
Language : en
Default : Yes
Default : Yes
Alternate group : 1
Alternate group : 1
Edit: Guess Ill add my PHP code too (This is chopped in a sensible manner because the actual file is very large):
// Set up Codebird
\Codebird\Codebird::setConsumerKey($consumer_key, $consumer_secret); // static, see README
$cb = \Codebird\Codebird::getInstance();
$cb->setToken($token, $token_secret);
$cb->setTimeout(60 * 1000); // 60 second request timeout
$video = new Video($path);
// Convert to the parameter required by twitter.
$converted = $video->convert();
$path = realpath('videos/' . $converted);
$file = fopen($path, 'rb');
$size = fstat($file)['size'];
$mime_type = mime_content_type($path);
$media = $cb->media_upload([
'command' => 'INIT',
'media_type' => $mime_type,
'media_category' => 'tweet_video',
'total_bytes' => $size,
]);
$success = $media->httpstatus >= 200 && $media->httpstatus < 300; # 2xx
if (!$success) {
throw new TwitterException("Failed to INIT upload for $path...");
}
// APPEND chunks to upload.
$mediaId = $media->media_id_string;
$segmentId = 0;
while (!feof($file)) {
echo "chunk #$segmentId....";
$chunk = fread($file, 512 * 1024); // caps out at 512 MB
echo "chunk size: ". strlen($chunk);
$media = $cb->media_upload([
'command' => 'APPEND',
'media_id' => $mediaId,
'segment_index' => $segmentId,
'media' => $chunk,
]);
$success = $media->httpstatus >= 200 && $media->httpstatus < 300; # 2xx
if (!$success) {
throw new TwitterException("Failed to APPEND to upload for $path, chunk $segmentId...");
}
$segmentId++;
}
// Close file and FINALIZE upload.
fclose($file);
echo "FINALIZING id $mediaId...";
$media = $cb->media_upload([
'command' => 'FINALIZE',
'media_id' => $mediaId,
]);
$success = $media->httpstatus >= 200 && $media->httpstatus < 300; # 2xx
if (!$success) {
var_dump($media);
throw new TwitterException("Failed to FINALIZE upload for $path...");
}
return $mediaId;
Video is my ffmpeg class, which I'll paste below, and $path is a URL leading to a perfectly valid mp4 video.
Video.php:
<?php
class VideoConversionException extends \Exception {}
class Video {
public $name;
public $converted;
public function __construct($name) {
self::clear();
$this->name = $name;
}
public function convert() {
$tmpVideo = 'video-' . uniqid() . uniqid() . '.mp4';
$videoPath = 'videos/' . $tmpVideo;
//$ffmpeg = "ffmpeg -i https://tvcanarias.acfipress.com/BC_190907_gc_teror.mp4 -vf "scale=1280:720" -b:v 5000K -b:a 128K -r 30 -f mp4 -vcodec libx264 -acodec aac output_video.mp4";
$ffmpeg = "ffmpeg -i {$this->name} -vf \"scale=1280:720\" -b:v 5000K -minrate 5000K -maxrate 5000K -b:a 128K -r 30 -f mp4 -vcodec libx264 -profile:v high -pix_fmt yuv420p -strict -2 -ac 2 -acodec aac $videoPath";
//$ffmpeg = "ffmpeg -i {$this->name} -pix_fmt yuv420p -vcodec libx264 -vf scale=640:-1 -acodec aac -vb 1024k -minrate 1024k -maxrate 1024k -bufsize 1024k -ar 44100 -ac 2 -strict experimental -r 30 $videoPath";
$output = [];
exec($ffmpeg, $output, $status);
if ($status != 0) {
//die("Couldnt run ffmpeg. (Error code: #$status)");
throw new VideoConversionException("Couldn't run ffmpeg. (Error code: #$status)");
}
$this->converted = $tmpVideo;
return $tmpVideo;
}
public function shredConverted() {
// delete video.
#unlink("videos/{$this->$converted}");
}
public static function clear() {
// We can't really shred videos right away as they might be still uploading.
// Therefore, every time this library is used, we will just delete videos older
// than, say, an hour.
$files = scandir('videos');
$curtime = time();
foreach ($files as $file) {
if ($file == '.gitignore' || $file == '.' || $file == '..') {
continue;
}
$mtime = filemtime("videos/$file");
$diff = $curtime - $mtime;
$overAnHour = $diff > (60 * 60);
if ($overAnHour) {
#unlink("videos/$file");
}
}
}
}

Related

Make a monocolor jpeg file encoded one bit per pixel?

For example, I see the following output. I don't quite understand its complete meaning. But when I look at the figure, it is just black and white, without gray or other color.
$ file 0447.jpg
0447.jpg: JPEG image data, JFIF standard 1.01, aspect ratio, density 1x1, segment length 16, baseline, precision 8, 1725x2751, components 1
The file size is 652708. To encode, 1725x2751 bits, the file size should be no more than 1725 * 2751/8 = 593184.4. But the file size is bigger than this.
So it seems that a pixel is not encoded as one bit. Is there a way to shrink the size of the file (without loss of info) so that one pixel is one bit (maybe using other formats)?
Don't use IM identify to find the actual bits-per pixel. Use EXIFTOOL.
Input JPG:
Exiftool shows BitsPerSample of 8
exiftool -s -ee -g1 -u -n -D cow.jpg
---- ExifTool ----
- ExifToolVersion : 12.44
---- System ----
- FileName : cow.jpg
- Directory : .
- FileSize : 16214
- FileModifyDate : 2022:10:14 11:31:44-07:00
- FileAccessDate : 2022:10:14 11:31:44-07:00
- FileInodeChangeDate : 2022:10:14 11:31:44-07:00
- FilePermissions : 100644
---- File ----
- FileType : JPEG
- FileTypeExtension : JPG
- MIMEType : image/jpeg
- ImageWidth : 300
- ImageHeight : 300
- EncodingProcess : 0
- BitsPerSample : 8 <-------
- ColorComponents : 1
---- JFIF ----
0 JFIFVersion : 1 1
2 ResolutionUnit : 2
3 XResolution : 28
5 YResolution : 28
---- Composite ----
- ImageSize : 300 300
- Megapixels : 0.09
Threshold to GIF:
convert cow.jpg -threshold 50% cow_bw1.gif
Exiftool shows BitsPerPixel = 1
exiftool -s -ee -g1 -u -n -D cow_bw1.gif
---- ExifTool ----
- ExifToolVersion : 12.44
---- System ----
- FileName : cow_bw1.gif
- Directory : .
- FileSize : 2698
- FileModifyDate : 2022:10:14 11:44:17-07:00
- FileAccessDate : 2022:10:14 11:44:19-07:00
- FileInodeChangeDate : 2022:10:14 11:44:17-07:00
- FilePermissions : 100644
---- File ----
- FileType : GIF
- FileTypeExtension : GIF
- MIMEType : image/gif
---- GIF ----
- GIFVersion : 89a
0 ImageWidth : 300
2 ImageHeight : 300
4.1 HasColorMap : 1
4.2 ColorResolutionDepth : 8
4.3 BitsPerPixel : 1 <-------
5 BackgroundColor : 1
---- Composite ----
- ImageSize : 300 300
- Megapixels : 0.09
Threshold to PNG:
convert cow.jpg -threshold 50% cow_bw1.png
Exiftool shows BitDepth of 1
exiftool -s -ee -g1 -u -n -D cow_bw1.png
---- ExifTool ----
- ExifToolVersion : 12.44
- Warning : [minor] Text/EXIF chunk(s) found after PNG IDAT (may be ignored by some readers)
---- System ----
- FileName : cow_bw1.png
- Directory : .
- FileSize : 2251
- FileModifyDate : 2022:10:14 11:49:20-07:00
- FileAccessDate : 2022:10:14 11:49:21-07:00
- FileInodeChangeDate : 2022:10:14 11:49:20-07:00
- FilePermissions : 100644
---- File ----
- FileType : PNG
- FileTypeExtension : PNG
- MIMEType : image/png
---- PNG ----
0 ImageWidth : 300
4 ImageHeight : 300
8 BitDepth : 1 <-------
9 ColorType : 0
10 Compression : 0
11 Filter : 0
12 Interlace : 0
- Gamma : 2.2
- BackgroundColor : 1
- ModifyDate : 2022:10:14 18:49:20
- Datecreate : 2022-10-14T18:31:44+00:00
- Datemodify : 2022-10-14T18:31:44+00:00
- Datetimestamp : 2022-10-14T18:49:19+00:00
---- PNG-pHYs ----
0 PixelsPerUnitX : 2800
4 PixelsPerUnitY : 2800
8 PixelUnits : 1
---- Composite ----
- ImageSize : 300 300
- Megapixels : 0.09
ADDITION
Here is how to make a 2-bit TIFF file.
convert lena.jpg -colorspace gray -depth 2 -type palette lena_d2.tif
exiftool -s -ee -g1 -u -n -D lena_d2.tif
---- ExifTool ----
- ExifToolVersion : 12.44
---- System ----
- FileName : lena_d2.tif
- Directory : .
- FileSize : 16642
- FileModifyDate : 2022:10:14 16:42:42-07:00
- FileAccessDate : 2022:10:14 16:42:43-07:00
- FileInodeChangeDate : 2022:10:14 16:42:42-07:00
- FilePermissions : 100644
---- File ----
- FileType : TIFF
- FileTypeExtension : TIF
- MIMEType : image/tiff
- ExifByteOrder : II
---- IFD0 ----
256 ImageWidth : 256
257 ImageHeight : 256
258 BitsPerSample : 2 <------
259 Compression : 1
262 PhotometricInterpretation : 3
266 FillOrder : 1
273 StripOffsets : 8
274 Orientation : 1
277 SamplesPerPixel : 1
278 RowsPerStrip : 256
279 StripByteCounts : 16384
282 XResolution : 72
283 YResolution : 72
284 PlanarConfiguration : 1
296 ResolutionUnit : 2
297 PageNumber : 0 1
320 ColorMap : (Binary data 24 bytes, use -b option to extract)
---- Composite ----
- ImageSize : 256 256
- Megapixels : 0.065536

MediaRecorder iOS Chrome does not obey the bitrate

I'm using MediaRecorder to record video from webcam and in case of safari (or iOS Chrome) output is mp4 file. The issue is that video from iOS chrome has huge bitrate.
Using recordrtc library, this code:
this.recorder = new RecordRTCPromisesHandler(new MediaStream([
this.ownStream.getVideoTracks()[0],
this.ownStream.getAudioTracks()[0]
]), {
type: 'video',
mimeType: 'video/mp4',
recorderType: MediaStreamRecorder,
audioBitsPerSecond: 48 * 1024,
videoBitsPerSecond: 384 * 1024,
});
await this.recorder.startRecording();
creates mp4 successfully, but result bitrate of output video is 92643 kb/s for 10 FPS video!
Full ffprobe output:
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'input.mp4': Metadata: major_brand : iso5 minor_version : 1 compatible_brands: isomiso5hlsf creation_time : 2021-12-14T14:52:28.000000Z Duration: 00:00:00.59, start: 0.000000, bitrate: 92643 kb/s Stream #0:0(und): Video: h264 (Baseline) (avc1 / 0x31637661), yuvj420p(pc), 640x480, 97341 kb/s, 10.14 fps, 20 tbr, 600 tbn, 1200 tbc (default) Metadata: rotate : 90 creation_time : 2021-12-14T14:52:28.000000Z handler_name : Core Media Video Side data: displaymatrix: rotation of -90.00 degrees Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, mono, fltp, 2552 kb/s (default) Metadata: creation_time : 2021-12-14T14:52:28.000000Z handler_name : Core Media Audio
This is only a bug in RecordRTC. Working with vanilla mediarecorder and videoBitsPerSecond works on iOS.

How to keep video quality same as it is after merge intro image beggining to video using ffmpeg

I have selected high resolution video but once i run command the video resolution quality was changed and too poor quality's video i get as output video, but i don't would like to loose my video quality. let me share full command and complete log below:
Note: libx264 encoder is not supporting iOS, I'm getting failure error so i use h264_videotoolbox so i would like to get supported command with h264_videotoolbox encoder
Command:
ffmpeg -i test.MOV -loop 1 -t 5 -i 2.jpg -f lavfi -t 5 -i anullsrc
-filter_complex "[0:v]trim=0:5,drawbox=t=fill[base];[1][base]scale2ref=iw:ih:force_original_aspect_ratio=decrease:flags=spline[2nd][base2];[base2][2nd]overlay='(W-w)/2':'(H-h)/2'[padded];[padded][2:a][0:v][0:a]concat=n=2:v=1:a=1[v][a]"
-c:v h264_videotoolbox -c:a aac -map "[v]" -map "[a]" output.mp4
Complete log
ffmpeg version 4.4 Copyright (c) 2000-2021 the FFmpeg developers
built with Apple clang version 12.0.0 (clang-1200.0.32.29)
configuration: --prefix=/usr/local/Cellar/ffmpeg/4.4_1 --enable-shared --enable-pthreads --enable-version3 --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libdav1d --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libspeex --enable-libsoxr --enable-libzmq --enable-libzimg --disable-libjack --disable-indev=jack --enable-videotoolbox
libavutil 56. 70.100 / 56. 70.100
libavcodec 58.134.100 / 58.134.100
libavformat 58. 76.100 / 58. 76.100
libavdevice 58. 13.100 / 58. 13.100
libavfilter 7.110.100 / 7.110.100
libavresample 4. 0. 0 / 4. 0. 0
libswscale 5. 9.100 / 5. 9.100
libswresample 3. 9.100 / 3. 9.100
libpostproc 55. 9.100 / 55. 9.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'test.MOV':
Metadata:
major_brand : qt
minor_version : 0
compatible_brands: qt
creation_time : 2021-03-07T06:36:17.000000Z
com.apple.quicktime.location.accuracy.horizontal: 30.000000
com.apple.quicktime.location.ISO6709: +23.1141+072.5768+061.729/
com.apple.quicktime.make: Apple
com.apple.quicktime.model: iPhone 6s
com.apple.quicktime.software: 14.3
com.apple.quicktime.creationdate: 2021-03-07T12:06:17+0530
Duration: 00:00:29.79, start: 0.000000, bitrate: 15778 kb/s
Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709), 1920x1080, 15643 kb/s, 29.98 fps, 29.97 tbr, 600 tbn, 1200 tbc (default)
Metadata:
creation_time : 2021-03-07T06:36:17.000000Z
handler_name : Core Media Video
vendor_id : [0][0][0][0]
encoder : H.264
Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, mono, fltp, 89 kb/s (default)
Metadata:
creation_time : 2021-03-07T06:36:17.000000Z
handler_name : Core Media Audio
vendor_id : [0][0][0][0]
Stream #0:2(und): Data: none (mebx / 0x7862656D), 0 kb/s (default)
Metadata:
creation_time : 2021-03-07T06:36:17.000000Z
handler_name : Core Media Metadata
Stream #0:3(und): Data: none (mebx / 0x7862656D), 0 kb/s (default)
Metadata:
creation_time : 2021-03-07T06:36:17.000000Z
handler_name : Core Media Metadata
Stream #0:4(und): Data: none (mebx / 0x7862656D), 34 kb/s (default)
Metadata:
creation_time : 2021-03-07T06:36:17.000000Z
handler_name : Core Media Metadata
Input #1, image2, from '2.jpg':
Duration: 00:00:00.04, start: 0.000000, bitrate: 17347 kb/s
Stream #1:0: Video: mjpeg (Baseline), yuvj444p(pc, bt470bg/unknown/unknown), 360x360 [SAR 72:72 DAR 1:1], 25 fps, 25 tbr, 25 tbn, 25 tbc
Input #2, lavfi, from 'anullsrc':
Duration: N/A, start: 0.000000, bitrate: 705 kb/s
Stream #2:0: Audio: pcm_u8, 44100 Hz, stereo, u8, 705 kb/s
Stream mapping:
Stream #0:0 (h264) -> trim
Stream #0:0 (h264) -> concat:in1:v0
Stream #0:1 (aac) -> concat:in1:a0
Stream #1:0 (mjpeg) -> scale2ref:default
Stream #2:0 (pcm_u8) -> concat:in0:a0
concat:out:v0 -> Stream #0:0 (h264_videotoolbox)
concat:out:a0 -> Stream #0:1 (aac)
Press [q] to stop, [?] for help
[swscaler # 0x7f976242b000] deprecated pixel format used, make sure you did set range correctly
Output #0, mp4, to 'output.mp4':
Metadata:
major_brand : qt
minor_version : 0
compatible_brands: qt
com.apple.quicktime.creationdate: 2021-03-07T12:06:17+0530
com.apple.quicktime.location.accuracy.horizontal: 30.000000
com.apple.quicktime.location.ISO6709: +23.1141+072.5768+061.729/
com.apple.quicktime.make: Apple
com.apple.quicktime.model: iPhone 6s
com.apple.quicktime.software: 14.3
encoder : Lavf58.76.100
Stream #0:0: Video: h264 (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 1920x1080, q=2-31, 200 kb/s, 29.97 fps, 30k tbn (default)
Metadata:
encoder : Lavc58.134.100 h264_videotoolbox
Stream #0:1: Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, mono, fltp, 69 kb/s (default)
Metadata:
encoder : Lavc58.134.100 aac
frame= 1 fps=0.0 q=0.0 size= 0kB time=00:00:00.00 bitrate=N/A speed= frame= 12 fps=0.0 q=-0.0 size= 256kB time=00:00:00.32 bitrate=6452.4kbits/frame= 34 fps= 32 q=-0.0 size= 256kB time=00:00:01.02 bitrate=2053.0kbits/frame= 55 fps= 35 q=-0.0 size= 256kB time=00:00:01.74 bitrate=1204.4kbits/frame= 76 fps= 37 q=-0.0 size= 256kB time=00:00:02.46 bitrate= 852.2kbits/frame= 98 fps= 38 q=-0.0 size= 256kB time=00:00:03.18 bitrate= 659.4kbits/frame= 120 fps= 39 q=-0.0 size= 256kB time=00:00:03.90 bitrate= 537.7kbits/frame= 141 fps= 39 q=-0.0 size= 512kB time=00:00:04.62 bitrate= 907.8kbits/[out_0_0 # 0x7f975de0ae80] 100 buffers queued in out_0_0, something may be wrong.
[out_0_1 # 0x7f975de0a5c0] 100 buffers queued in out_0_1, something may be wrong.
frame= 301 fps= 67 q=-0.0 size= 768kB time=00:00:11.09 bitrate= 566.9kbits/frame= 406 fps= 81 q=-0.0 size= 1024kB time=00:00:14.60 bitrate= 574.4kbits/frame= 509 fps= 92 q=-0.0 size= 1280kB time=00:00:18.04 bitrate= 581.2kbits/frame= 604 fps=100 q=-0.0 size= 1792kB time=00:00:21.19 bitrate= 692.5kbits/frame= 705 fps=108 q=-0.0 size= 2048kB time=00:00:24.56 bitrate= 682.9kbits/frame= 809 fps=115 q=-0.0 size= 2304kB time=00:00:28.04 bitrate= 672.9kbits/frame= 909 fps=121 q=-0.0 size= 2816kB time=00:00:31.37 bitrate= 735.4kbits/frame= 1012 fps=126 q=-0.0 size= 3072kB time=00:00:34.71 bitrate= 725.0kbits/frame= 1043 fps=127 q=-0.0 Lsize= 3288kB time=00:00:34.78 bitrate= 774.3kbits/s speed=4.23x
video:2995kB audio:255kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 1.168448%
[aac # 0x7f9760024200] Qavg: 9569.656
The main video have a bitrate aprox of 15778K, and this is a signal for quality hight...
Duration: 00:00:29.79, start: 0.000000, bitrate: 15778 kb/s
You have to put bitrate parameter -b:v {bitrate, in this case 15778K} in your code to keep hight qualiy, some like
ffmpeg -i test.MOV -loop 1 -t 5 -i 2.jpg -f lavfi -t 5 -i anullsrc -filter_complex "[0:v]trim=0:5,drawbox=t=fill[base];[1][base]scale2ref=iw:ih:force_original_aspect_ratio=decrease:flags=spline[2nd][base2];[base2][2nd]overlay='(W-w)/2':'(H-h)/2'[padded];[padded][2:a][0:v][0:a]concat=n=2:v=1:a=1[v][a]" -c:v h264_videotoolbox -b:v 15778K -c:a aac -map "[v]" -map "[a]" output.mp4
NOTE: if have an error with "-c:a aac" parameter, remove it, because in Apple`s OS aac is for default because is an Apple codec
King Regards

FFMPEG making requests for each frame when decoding a stream, slow performance

I am having an issue playing MOV camera captured files from an iPhone. My FFMPEG implementation has no problem playing most file formats, this issue is exclusive only for camera captured MOV.
When trying to open the file, I can see in the logs that many requests are made, each requests decoding only one frame, before making a new request which results the video being buffered extremely slowly.
It takes roughly a minute to buffer about a few seconds of the video.
Another thing to mention is that the very same problematic file is played without an issue locally. The problem is when trying to decode while streaming.
I compiled my code on Xcode 11, iOS SDK 13, with cocoapods mobile-ffmpeg-https 4.2.
Here is a rough representation of my code, its pretty standard:
Here is how I open AVFormatContext:
AVFormatContext *context = avformat_alloc_context();
context->interrupt_callback.callback = FFMPEGFormatContextIOHandler_IO_CALLBACK;
context->interrupt_callback.opaque = (__bridge void *)(handler);
av_log_set_level(AV_LOG_TRACE);
int result = avformat_open_input(&context, [request.urlAsString UTF8String], NULL, NULL);
if (result != 0) {
if (context != NULL) {
avformat_free_context(context);
}
return nil;
}
result = avformat_find_stream_info(context, NULL);
if (result < 0) {
avformat_close_input(&context);
return nil;
}
Video decoder is opened like so, audio decoder is nearly identical
AVCodecParameters *params = context->streams[streamIndex]->codecpar;
AVCodec *codec = avcodec_find_decoder(params->codec_id);
if (codec == NULL) {
return NULL;
}
AVCodecContext *codecContext = avcodec_alloc_context3(codec);
if (codecContext == NULL) {
return NULL;
}
codecContext->thread_count = 6;
int result = avcodec_parameters_to_context(codecContext, params);
if (result < 0) {
avcodec_free_context(&codecContext);
return NULL;
}
result = avcodec_open2(codecContext, codec, NULL);
if (result < 0) {
avcodec_free_context(&codecContext);
return NULL;
}
I read the data from the server like so:
AVPacket packet;
int result = av_read_frame(formatContext, &avPacket);
if (result == 0) {
avcodec_send_packet(codecContext, &avPacket);
// .... decode ....
}
Logs after opening the decoders:
// [tls] Request is made here
// [tls] Request response headers are here
Probing mov,mp4,m4a,3gp,3g2,mj2 score:100 size:2048
Probing mp3 score:1 size:2048
[mov,mp4,m4a,3gp,3g2,mj2 # 0x115918e00] Format mov,mp4,m4a,3gp,3g2,mj2 probed with size=2048 and score=100
[mov,mp4,m4a,3gp,3g2,mj2 # 0x115918e00] type:'ftyp' parent:'root' sz: 20 8 23077123
[mov,mp4,m4a,3gp,3g2,mj2 # 0x115918e00] ISO: File Type Major Brand: qt
[mov,mp4,m4a,3gp,3g2,mj2 # 0x115918e00] type:'wide' parent:'root' sz: 8 28 23077123
[mov,mp4,m4a,3gp,3g2,mj2 # 0x115918e00] type:'mdat' parent:'root' sz: 23066642 36 23077123
// [tls] Request is made here
// [tls] Request response headers are here
[mov,mp4,m4a,3gp,3g2,mj2 # 0x115918e00] stream 0, sample 4, dts 133333
[mov,mp4,m4a,3gp,3g2,mj2 # 0x115918e00] stream 1, sample 48, dts 1114558
[mov,mp4,m4a,3gp,3g2,mj2 # 0x115918e00] stream 2, sample 1, dts 2666667
[h264 # 0x116080200] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 1
// [tls] Request is made here
// [tls] Request response headers are here
[mov,mp4,m4a,3gp,3g2,mj2 # 0x115918e00] stream 0, sample 4, dts 133333
[mov,mp4,m4a,3gp,3g2,mj2 # 0x115918e00] stream 1, sample 48, dts 1114558
[mov,mp4,m4a,3gp,3g2,mj2 # 0x115918e00] stream 2, sample 1, dts 2666667
[h264 # 0x116080200] nal_unit_type: 1(Coded slice of a non-IDR picture), nal_ref_idc: 1
// [tls] Request is made here
// [tls] Request response headers are here
// ...
These are some warnings I found in the log
[mov,mp4,m4a,3gp,3g2,mj2 # 0x11c030800] interrupted
[mov,mp4,m4a,3gp,3g2,mj2 # 0x11c030800] stream 0: start_time: 0.000 duration: 11.833
[mov,mp4,m4a,3gp,3g2,mj2 # 0x11c030800] stream 1: start_time: 0.000 duration: 11.832
[mov,mp4,m4a,3gp,3g2,mj2 # 0x11c030800] stream 2: start_time: 0.000 duration: 11.833
[mov,mp4,m4a,3gp,3g2,mj2 # 0x11c030800] stream 3: start_time: 0.000 duration: 11.833
[mov,mp4,m4a,3gp,3g2,mj2 # 0x11c030800] format: start_time: 0.000 duration: 11.833 bitrate=15601 kb/s
[mov,mp4,m4a,3gp,3g2,mj2 # 0x11c030800] Could not find codec parameters for stream 0 (Video: h264, 1 reference frame (avc1 / 0x31637661), none(bt709, left), 1920x1080, 1/1200, 15495 kb/s): unspecified pixel format
Consider increasing the value for the 'analyzeduration' and 'probesize' options
[mov,mp4,m4a,3gp,3g2,mj2 # 0x11c030800] After avformat_find_stream_info() pos: 23077123 bytes read:16293 seeks:1 frames:0
Also when calling avformat_open_input(...), 2 GET requests are made, before the returning.
Notice the "Probing mp3 score:1", that is not shown for other MOV files or any other files.
I have tried different versions of ffmpeg, I have tried messing around with the delays of the stream, I tried removing my custom interrupt callback, nothing have worked.
Code works fine with any other videos I have tested (mp4, mkv, avi).
Metadata of the test file:
Metadata:
major_brand : qt
minor_version : 0
compatible_brands: qt
creation_time : 2019-04-14T08:17:03.000000Z
com.apple.quicktime.make: Apple
com.apple.quicktime.model: iPhone 7
com.apple.quicktime.software: 12.2
com.apple.quicktime.creationdate: 2019-04-14T11:17:03+0300
Duration: 00:00:16.83, bitrate: N/A
Stream #0:0(und), 0, 1/600: Video: h264, 1 reference frame (avc1 / 0x31637661), none(bt709), 1920x1080 (0x0), 0/1, 15301 kb/s, 30 fps, 30 tbr, 600 tbn (default)
Metadata:
creation_time : 2019-04-14T08:17:03.000000Z
handler_name : Core Media Video
encoder : H.264
Stream #0:1(und), 0, 1/44100: Audio: aac (mp4a / 0x6134706D), 44100 Hz, mono, 100 kb/s (default)
Metadata:
creation_time : 2019-04-14T08:17:03.000000Z
handler_name : Core Media Audio
Stream #0:2(und), 0, 1/600: Data: none (mebx / 0x7862656D), 0/1, 0 kb/s (default)
Metadata:
creation_time : 2019-04-14T08:17:03.000000Z
handler_name : Core Media Metadata
Stream #0:3(und), 0, 1/600: Data: none (mebx / 0x7862656D), 0/1, 0 kb/s (default)
Metadata:
creation_time : 2019-04-14T08:17:03.000000Z
handler_name : Core Media Metadata
I found a fix for this (sorta of):
Set the io_open callback of the AVFormatContext to your own function, and then when that is called, after you call the default io_open you change the buffer size.
static int (*IO_OPEN_DEFAULT)(struct AVFormatContext *s, AVIOContext **pb, const char *url, int flags, AVDictionary **options);
int IO_OPEN_OVERRIDE(struct AVFormatContext *s, AVIOContext **pb, const char *url, int flags, AVDictionary **options) {
int result = IO_OPEN_DEFAULT(s, pb, url, flags, options);
pb[0]->buffer_size = 41239179;
return result;
}
This fixes the issue. The value you set it to is usually very large (20 to 40MB). You can get that value from the second network request bytes range when you are opening the format context (first network request is made with bytes range 0-* and then the second network request is made with bytes range XXXX-* where XXXX should be the buffer size).
The reason why this fixes the issue, is because by buffering all of that data, aviocontext no longer needs to make a new network request to get the audio data. The audio data is already buffered (or at least the first position is).
There might be a better way to fix this issue, it seems like apple MOV files separate their video and audio data with these massive chunks for some reason and that causes ffmpeg to make a million network requests, for each frame.

Why the video's start time output by AVAssetWriter is no zero?

When i use PBJVsion to capture a video ,why is the video's start time output by AVAssetWriter is not zero?
the video output imformation use some video tools as below:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf55.19.104
Duration: 00:00:08.12, start: 0.023220, bitrate: 1050 kb/s
Stream #0.0(und): Video: h264 (High), yuv420p, 480x480, 976 kb/s, 30 fps, 30 tbr, 15360 tbn, 60 tbc
Stream #0.1(und): Audio: aac, 44100 Hz, mono, fltp, 75 kb/s
[17:39:56] scan: decoding previews for title 1

Resources