I am new to OpenCV, and I want write Mat images into video using VideoWriter on Ubuntu 12.04. But when constructing VideoWriter, errors came out.
It seems that OpenCV invoke ffmpeg API using default parameters and ffmpeg invoke x264 using its default parameters. Then these setting is broken for libx264. Thus the "Could not open codec 'libx264'" error.
Anyone has ideas to solve this problem?
More specifically:
anyone knows where and how OpenCV invoke ffmpeg API?
how to change ffmpeg default settings using code, hopefull, can be easily embeded into OpenCV?
will changes of default in ffmpeg be carried to libx264?
Errors:
1. Uising CV_FOURCC('H', '2', '6', '4')
[libx264 # 0x255de40] broken ffmpeg default settings detected
[libx264 # 0x255de40] use an encoding preset (e.g. -vpre medium)
[libx264 # 0x255de40] preset usage: -vpre <speed> -vpre <profile>
[libx264 # 0x255de40] speed presets are listed in x264 --help
[libx264 # 0x255de40] profile is optional; x264 defaults to high
Could not open codec 'libx264': Unspecified error
2. Using FOURCC = -1 to invoke user customized codec
OpenCV Error: Unsupported format or combination of formats (Gstreamer Opencv
backend doesn't support this codec acutally.) in CvVideoWriter_GStreamer::open,
file /home/XXX/Downloads/opencv-2.4.8/modules/highgui/src/cap_gstreamer.cpp,
line 505 terminate called after throwing an instance of 'cv::Exception'
what(): /home/XXX/Downloads/opencv-2.4.8/modules/highgui/src/cap_gstreamer.cpp:
505: error: (-210) Gstreamer Opencv backend doesn't support this codec acutally.
in function CvVideoWriter_GStreamer::open
Codes:
int main(int argc, char *argv[])
{
VideoWriter outputVideo;
bool fourcc_on = true; //switch on / off different error
if (fourcc_on)
outputVideo.open("outVideo.avi", CV_FOURCC('H', '2', '6', '4'), 25, Size(100, 100), true);
else
outputVideo.open("outVideo.avi", -1, 25, Size(100, 100), true);
if (!outputVideo.isOpened())
{
cout << "Could not open the output video for write" << endl;
return -1;
}
return 0;
}
OpenCV Configuration:
-- Detected version of GNU GCC: 46 (406)
-- Found OpenEXR: /usr/lib/libIlmImf.so
-- Looking for linux/videodev.h
-- Looking for linux/videodev.h - not found
-- Looking for linux/videodev2.h
-- Looking for linux/videodev2.h - found
-- Looking for sys/videoio.h
-- Looking for sys/videoio.h - not found
-- Looking for libavformat/avformat.h
-- Looking for libavformat/avformat.h - found
-- Looking for ffmpeg/avformat.h
-- Looking for ffmpeg/avformat.h - not found
-- Could NOT find JNI (missing: JAVA_INCLUDE_PATH JAVA_INCLUDE_PATH2 JAVA_AWT_INCLUDE_PATH)
--
-- General configuration for OpenCV 2.4.8 =====================================
-- Version control: unknown
--
-- Platform:
-- Host: Linux 3.8.0-38-generic x86_64
-- CMake: 2.8.7
-- CMake generator: Unix Makefiles
-- CMake build tool: /usr/bin/make
-- Configuration: RELEASE
--
-- C/C++:
-- Built as dynamic libs?: YES
-- C++ Compiler: /usr/bin/c++ (ver 4.6)
-- C++ flags (Release): -fsigned-char -W -Wall -Werror=return-type -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -msse -msse2 -msse3 -ffunction-sections -O3 -DNDEBUG -DNDEBUG
-- C++ flags (Debug): -fsigned-char -W -Wall -Werror=return-type -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -msse -msse2 -msse3 -ffunction-sections -g -O0 -DDEBUG -D_DEBUG
-- C Compiler: /usr/bin/gcc
-- C flags (Release): -fsigned-char -W -Wall -Werror=return-type -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -msse -msse2 -msse3 -ffunction-sections -O3 -DNDEBUG -DNDEBUG
-- C flags (Debug): -fsigned-char -W -Wall -Werror=return-type -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -msse -msse2 -msse3 -ffunction-sections -g -O0 -DDEBUG -D_DEBUG
-- Linker flags (Release):
-- Linker flags (Debug):
-- Precompiled headers: YES
--
-- OpenCV modules:
-- To be built: core flann imgproc highgui features2d calib3d ml video legacy objdetect photo gpu ocl nonfree contrib python stitching superres ts videostab
-- Disabled: world
-- Disabled by dependency: -
-- Unavailable: androidcamera dynamicuda java
--
-- GUI:
-- QT: NO
-- GTK+ 2.x: YES (ver 2.24.10)
-- GThread : YES (ver 2.32.4)
-- GtkGlExt: NO
-- OpenGL support: NO
--
-- Media I/O:
-- ZLib: /usr/lib/x86_64-linux-gnu/libz.so (ver 1.2.3.4)
-- JPEG: /usr/lib/x86_64-linux-gnu/libjpeg.so (ver )
-- PNG: /usr/lib/x86_64-linux-gnu/libpng.so (ver 1.2.46)
-- TIFF: /usr/lib/x86_64-linux-gnu/libtiff.so (ver 42 - 3.9.5)
-- JPEG 2000: /usr/lib/x86_64-linux-gnu/libjasper.so (ver 1.900.1)
-- OpenEXR: /usr/lib/libImath.so /usr/lib/libIlmImf.so /usr/lib/libIex.so /usr/lib/libHalf.so /usr/lib/libIlmThread.so (ver 1.6.1)
--
-- Video I/O:
-- DC1394 1.x: NO
-- DC1394 2.x: YES (ver 2.2.0)
-- FFMPEG: YES
-- codec: YES (ver 55.58.105)
-- format: YES (ver 55.37.101)
-- util: YES (ver 52.78.100)
-- swscale: YES (ver 2.6.100)
-- gentoo-style: YES
-- GStreamer:
-- base: YES (ver 0.10.36)
-- app: YES (ver 0.10.36)
-- video: YES (ver 0.10.36)
-- OpenNI: NO
-- OpenNI PrimeSensor Modules: NO
-- PvAPI: NO
-- GigEVisionSDK: NO
-- UniCap: NO
-- UniCap ucil: NO
-- V4L/V4L2: Using libv4l (ver 1.0.1)
-- XIMEA: NO
-- Xine: NO
--
-- Other third-party libraries:
-- Use IPP: NO
-- Use Eigen: NO
-- Use TBB: NO
-- Use OpenMP: NO
-- Use GCD NO
-- Use Concurrency NO
-- Use C=: NO
-- Use Cuda: NO
-- Use OpenCL: YES
--
-- OpenCL:
-- Version: dynamic
-- Include path: /home/shixudongleo/Downloads/opencv-2.4.8/3rdparty/include/opencl/1.2
-- Use AMD FFT: NO
-- Use AMD BLAS: NO
--
-- Python:
-- Interpreter: /usr/bin/python (ver 2.7.3)
-- Libraries: /usr/lib/libpython2.7.so
-- numpy: /usr/lib/python2.7/dist-packages/numpy/core/include (ver 1.6.1)
-- packages path: lib/python2.7/dist-packages
--
-- Java:
-- ant: NO
-- JNI: NO
-- Java tests: NO
--
-- Documentation:
-- Build Documentation: NO
-- Sphinx: NO
-- PdfLaTeX compiler: /usr/bin/pdflatex
--
-- Tests and samples:
-- Tests: YES
-- Performance tests: YES
-- C/C++ Examples: NO
--
-- Install path: /usr/local
--
-- cvconfig.h is in: /home/shixudongleo/Downloads/opencv-2.4.8/build
-- -----------------------------------------------------------------
--
-- Configuring done
-- Generating done
-- Build files have been written to: /home/XXX/Downloads/opencv-2.4.8/build
FFMPEG
ffmpeg is enable to support OpenCV and libx264 is enabled when compiling ffmpeg.
By using ffmpeg command line, libx264 is running normally.
$ ffmpeg -i test.avi -vcodec libx264 test.mp4
ffmpeg -i test.avi -vcodec libx264 test.mp4 > ~/Downloads/ffmpeg_log.txt
ffmpeg version 2.2.git Copyright (c) 2000-2014 the FFmpeg developers
built on Apr 24 2014 16:39:51 with gcc 4.6 (Ubuntu/Linaro 4.6.3-1ubuntu5)
configuration: --enable-gpl --enable-libfaac --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libtheora --enable-libvorbis --enable-libx264 --enable-libxvid --enable-nonfree --enable-postproc --enable-version3 --enable-x11grab --enable-shared --enable-pic
libavutil 52. 78.100 / 52. 78.100
libavcodec 55. 58.105 / 55. 58.105
libavformat 55. 37.101 / 55. 37.101
libavdevice 55. 13.100 / 55. 13.100
libavfilter 4. 4.100 / 4. 4.100
libswscale 2. 6.100 / 2. 6.100
libswresample 0. 18.100 / 0. 18.100
libpostproc 52. 3.100 / 52. 3.100
Input #0, avi, from 'test.avi':
Duration: 00:00:03.73, start: 0.000000, bitrate: 1757 kb/s
Stream #0:0: Video: msvideo1 (CRAM / 0x4D415243), rgb555le, 320x240, 1781 kb/s, 15 tbr, 15 tbn, 15 tbc
Metadata:
title : julius.avi Video #1
File 'test.mp4' already exists. Overwrite ? [y/N] y
No pixel format specified, yuv444p for H.264 encoding chosen.
Use -pix_fmt yuv420p for compatibility with outdated media players.
[libx264 # 0x25d08e0] using cpu capabilities: none!
[libx264 # 0x25d08e0] profile High 4:4:4 Predictive, level 1.2, 4:4:4 8-bit
[libx264 # 0x25d08e0] 264 - core 142 - H.264/MPEG-4 AVC codec - Copyleft 2003-2014 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=4 threads=12 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=15 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'test.mp4':
Metadata:
encoder : Lavf55.37.101
Stream #0:0: Video: h264 (libx264) ([33][0][0][0] / 0x0021), yuv444p, 320x240, q=-1--1, 15360 tbn, 15 tbc
Metadata:
title : julius.avi Video #1
Stream mapping:
Stream #0:0 -> #0:0 (msvideo1 -> libx264)
Press [q] to stop, [?] for help
frame= 56 fps=0.0 q=-1.0 Lsize= 321kB time=00:00:03.60 bitrate= 731.0kbits/s
video:320kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.409949%
[libx264 # 0x25d08e0] frame I:3 Avg QP:15.36 size: 7975
[libx264 # 0x25d08e0] frame P:38 Avg QP:26.05 size: 6230
[libx264 # 0x25d08e0] frame B:15 Avg QP:28.25 size: 4418
[libx264 # 0x25d08e0] consecutive B-frames: 46.4% 53.6% 0.0% 0.0%
[libx264 # 0x25d08e0] mb I I16..4: 1.4% 72.8% 25.8%
[libx264 # 0x25d08e0] mb P I16..4: 1.6% 5.7% 15.1% P16..4: 7.6% 6.3% 7.4% 0.0% 0.0% skip:56.3%
[libx264 # 0x25d08e0] mb B I16..4: 0.2% 1.0% 2.0% B16..8: 13.3% 7.8% 8.7% direct: 8.3% skip:58.8% L0:34.9% L1:36.6% BI:28.5%
[libx264 # 0x25d08e0] 8x8 transform intra:37.7% inter:2.3%
[libx264 # 0x25d08e0] coded y,u,v intra: 52.1% 42.1% 30.1% inter: 19.6% 9.2% 5.2%
[libx264 # 0x25d08e0] i16 v,h,dc,p: 56% 17% 24% 2%
[libx264 # 0x25d08e0] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 10% 16% 68% 1% 1% 1% 1% 1% 1%
[libx264 # 0x25d08e0] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 19% 18% 28% 5% 6% 5% 7% 5% 6%
[libx264 # 0x25d08e0] Weighted P-Frames: Y:31.6% UV:21.1%
[libx264 # 0x25d08e0] ref P L0: 70.5% 9.0% 12.1% 6.5% 2.0%
[libx264 # 0x25d08e0] ref B L0: 91.3% 8.7%
[libx264 # 0x25d08e0] kb/s:700.56
Have you tried changing the fourcc parameters ?
VideoWriter outp;
outp.open("aboutput.wmv",CV_FOURCC('M','P','4','2'),10,Size(160,160),false);
this works on my Ubuntu 14.04
Related
I'm trying to use mp4 format coding (fourcc=0x21) on OpenCV 3.2 without success. Probably I have a problem with my Ubuntu 18.04 setup but currently I'm running out of ideas how to fix it. First, I recompiled FFMPEG with x264 support. Than, recompiled OpenCV (with contrib and OPENCV_ENABLE_NONFREE=ON, WITH_FFMPEG=ON). It is still not working.
I cannot use fourcc different than 0x21. I mean.. I cannot use another codec.
Example:
>>> import cv2
>>> out = cv2.VideoWriter('output.mp4', 0x21, 20.0, (640,480))
OpenCV: FFMPEG: tag 0x00000021/'!???' is not found (format 'mp4 / (null)')'
However, this is working fine (other fourcc, codec):
>>> out = cv2.VideoWriter('output.mp4',0x31637661 , 20.0, (640,480))
OpenCV compiled with FFMPEG support (version 3.2), cmake config:
Video I/O:
DC1394 1.x: NO
DC1394 2.x: YES (ver 2.2.5)
FFMPEG: YES
avcodec: YES (ver 57.107.100)
avformat: YES (ver 57.83.100)
avutil: YES (ver 55.78.100)
swscale: YES (ver 4.8.100)
avresample: YES (ver 3.7.0)
FFMPEG build with x264 support:
ffmpeg -version
ffmpeg version N-96891-g60b1f85 Copyright (c) 2000-2020 the FFmpeg developers
built with gcc 7 (Ubuntu 7.4.0-1ubuntu1~18.04.1)
configuration: --prefix=/home/user/ffmpeg_build --pkg-config-flags=--static --extra-cflags=-I/home/user/ffmpeg_build/include --extra-ldflags=-L/home/user/ffmpeg_build/lib --extra-libs='-lpthread -lm' --bindir=/home/user/bin --enable-gpl --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-nonfree
libavutil 56. 42.100 / 56. 42.100
libavcodec 58. 73.102 / 58. 73.102
libavformat 58. 39.101 / 58. 39.101
libavdevice 58. 9.103 / 58. 9.103
libavfilter 7. 77.100 / 7. 77.100
libswscale 5. 6.100 / 5. 6.100
libswresample 3. 6.100 / 3. 6.100
libpostproc 55. 6.100 / 55. 6.100
FFMPEG x264 simple conversion test is working:
ffmpeg -i in.mp4 -vcodec libx264 -f mp4 out.mp4y
ffmpeg version N-96891-g60b1f85 Copyright (c) 2000-2020 the FFmpeg developers
built with gcc 7 (Ubuntu 7.4.0-1ubuntu1~18.04.1)
configuration: --prefix=/home/user/ffmpeg_build --pkg-config-flags=--static --extra-cflags=-I/home/user/ffmpeg_build/include --extra-ldflags=-L/home/user/ffmpeg_build/lib --extra-libs='-lpthread -lm' --bindir=/home/user/bin --enable-gpl --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-nonfree
libavutil 56. 42.100 / 56. 42.100
libavcodec 58. 73.102 / 58. 73.102
libavformat 58. 39.101 / 58. 39.101
libavdevice 58. 9.103 / 58. 9.103
libavfilter 7. 77.100 / 7. 77.100
libswscale 5. 6.100 / 5. 6.100
libswresample 3. 6.100 / 3. 6.100
libpostproc 55. 6.100 / 55. 6.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'in.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf57.83.100
Duration: 00:00:10.00, start: 0.000000, bitrate: 3826 kb/s
Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1280x960, 3825 kb/s, 10 fps, 10 tbr, 10240 tbn, 20 tbc (default)
Metadata:
handler_name : VideoHandler
File 'out.mp4y' already exists. Overwrite? [y/N] y
Stream mapping:
Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
I just find my opencv cannot find .png format images, but works fine with loading other format like .jpg images. I built opencv4.01 with vs2015. Here below are the CMake configurations I saved when the opencv was built:
-- Selecting Windows SDK version to target Windows 10.0.17763.
-- AVX_512F is not supported by C++ compiler
-- AVX512_SKX is not supported by C++ compiler
-- Dispatch optimization AVX512_SKX is not available, skipped
-- libjpeg-turbo: VERSION = 1.5.3, BUILD = opencv-4.0.1-libjpeg-turbo
-- Looking for Mfapi.h
-- Looking for Mfapi.h - found
-- found Intel IPP (ICV version): 2019.0.0 [2019.0.0 Gold]
-- at: C:/opencv-4.0/cmake-build/3rdparty/ippicv/ippicv_win/icv
-- found Intel IPP Integration Wrappers sources: 2019.0.0
-- at: C:/opencv-4.0/cmake-build/3rdparty/ippicv/ippicv_win/iw
-- CUDA detected: 9.0
-- CUDA NVCC target flags: -gencode;arch=compute_30,code=sm_30;-gencode;arch=compute_35,code=sm_35;-gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_52,code=sm_52;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-D_FORCE_INLINES
-- Could not find OpenBLAS include. Turning OpenBLAS_FOUND off
-- Could not find OpenBLAS lib. Turning OpenBLAS_FOUND off
-- Could NOT find BLAS (missing: BLAS_LIBRARIES)
-- LAPACK requires BLAS
-- A library with LAPACK API not found. Please specify library location.
-- VTK is not found. Please set -DVTK_DIR in CMake to VTK build directory, or to VTK install subdirectory with VTKConfig.cmake file
-- OpenCV Python: during development append to PYTHONPATH: C:/opencv-4.0/cmake-build/python_loader
-- Module opencv_cudaoptflow disabled because opencv_optflow dependency can't be resolved!
-- Module opencv_superres disabled because opencv_optflow dependency can't be resolved!
-- Excluding from source files list: <BUILD>/modules/dnn/layers/layers_common.avx512_skx.cpp
--
-- General configuration for OpenCV 4.0.1 =====================================
-- Version control: unknown
--
-- Extra modules:
-- Location (extra): D:/Software/Programming/OpenCV/opencv_contrib-4.0.1/modules
-- Version control (extra): unknown
--
-- Platform:
-- Timestamp: 2019-03-23T15:52:48Z
-- Host: Windows 10.0.17763 AMD64
-- CMake: 3.13.4
-- CMake generator: Visual Studio 14 2015 Win64
-- CMake build tool: C:/Program Files (x86)/MSBuild/14.0/bin/MSBuild.exe
-- MSVC: 1900
--
-- CPU/HW features:
-- Baseline: SSE SSE2 SSE3
-- requested: SSE3
-- Dispatched code generation: SSE4_1 SSE4_2 FP16 AVX AVX2
-- requested: SSE4_1 SSE4_2 AVX FP16 AVX2 AVX512_SKX
-- SSE4_1 (7 files): + SSSE3 SSE4_1
-- SSE4_2 (2 files): + SSSE3 SSE4_1 POPCNT SSE4_2
-- FP16 (1 files): + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 AVX
-- AVX (5 files): + SSSE3 SSE4_1 POPCNT SSE4_2 AVX
-- AVX2 (13 files): + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 FMA3 AVX AVX2
--
-- C/C++:
-- Built as dynamic libs?: YES
-- C++ Compiler: C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/bin/x86_amd64/cl.exe (ver 19.0.24215.1)
-- C++ flags (Release): /DWIN32 /D_WINDOWS /W4 /GR /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi /EHa /wd4127 /wd4251 /wd4324 /wd4275 /wd4512 /wd4589 /MP4 /MD /O2 /Ob2 /DNDEBUG
-- C++ flags (Debug): /DWIN32 /D_WINDOWS /W4 /GR /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi /EHa /wd4127 /wd4251 /wd4324 /wd4275 /wd4512 /wd4589 /MP4 /MDd /Zi /Ob0 /Od /RTC1
-- C Compiler: C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/bin/x86_amd64/cl.exe
-- C flags (Release): /DWIN32 /D_WINDOWS /W3 /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi /MP4 /MD /O2 /Ob2 /DNDEBUG
-- C flags (Debug): /DWIN32 /D_WINDOWS /W3 /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi /MP4 /MDd /Zi /Ob0 /Od /RTC1
-- Linker flags (Release): /machine:x64 /INCREMENTAL:NO
-- Linker flags (Debug): /machine:x64 /debug /INCREMENTAL
-- ccache: NO
-- Precompiled headers: YES
-- Extra dependencies: cudart.lib nppc.lib nppial.lib nppicc.lib nppicom.lib nppidei.lib nppif.lib nppig.lib nppim.lib nppist.lib nppisu.lib nppitc.lib npps.lib cublas.lib cufft.lib -LIBPATH:C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v9.0/lib/x64
-- 3rdparty dependencies:
--
-- OpenCV modules:
-- To be built: bioinspired calib3d core cudaarithm cudabgsegm cudacodec cudafeatures2d cudafilters cudaimgproc cudalegacy cudaobjdetect cudastereo cudawarping cudev dnn dnn_objdetect features2d flann gapi highgui imgcodecs imgproc java_bindings_generator ml objdetect phase_unwrapping photo plot python_bindings_generator reg saliency shape stereo stitching structured_light surface_matching tracking ts video videoio xfeatures2d ximgproc xobjdetect xphoto
-- Disabled: world
-- Disabled by dependency: cudaoptflow superres
-- Unavailable: java js python2 python3
-- Applications: tests apps
-- Documentation: NO
-- Non-free algorithms: NO
--
-- Windows RT support: NO
--
-- GUI:
-- Win32 UI: YES
-- VTK support: NO
--
-- Media I/O:
-- ZLib: build (ver 1.2.11)
-- JPEG: build-libjpeg-turbo (ver 1.5.3-62)
-- WEBP: build (ver encoder: 0x020e)
-- TIFF: build (ver 42 - 4.0.9)
-- JPEG 2000: build (ver 1.900.1)
-- OpenEXR: build (ver 1.7.1)
-- HDR: YES
-- SUNRASTER: YES
-- PXM: YES
-- PFM: YES
--
-- Video I/O:
-- DC1394: NO
-- FFMPEG: YES (prebuilt binaries)
-- avcodec: YES (ver 58.35.100)
-- avformat: YES (ver 58.20.100)
-- avutil: YES (ver 56.22.100)
-- swscale: YES (ver 5.3.100)
-- avresample: YES (ver 4.0.0)
-- GStreamer: NO
-- DirectShow: YES
-- Media Foundation: YES
-- DXVA: NO
--
-- Parallel framework: Concurrency
--
-- Trace: YES (with Intel ITT)
--
-- Other third-party libraries:
-- Intel IPP: 2019.0.0 Gold [2019.0.0]
-- at: C:/opencv-4.0/cmake-build/3rdparty/ippicv/ippicv_win/icv
-- Intel IPP IW: sources (2019.0.0)
-- at: C:/opencv-4.0/cmake-build/3rdparty/ippicv/ippicv_win/iw
-- Lapack: NO
-- Eigen: NO
-- Custom HAL: NO
-- Protobuf: build (3.5.1)
--
-- NVIDIA CUDA: YES (ver 9.0, CUFFT CUBLAS NVCUVID)
-- NVIDIA GPU arch: 30 35 37 50 52 60 61 70
-- NVIDIA PTX archs:
--
-- OpenCL: YES (no extra features)
-- Include path: D:/Software/Programming/opencv-4.0.1/opencv-4.0.1/3rdparty/include/opencl/1.2
-- Link libraries: Dynamic load
--
-- Python (for build): C:/Python36/python.exe
--
-- Java:
-- ant: NO
-- JNI: C:/Java/include C:/Java/include/win32 C:/Java/include
-- Java wrappers: NO
-- Java tests: NO
--
-- Install to: C:/opencv-4.0/mybuild
-- -----------------------------------------------------------------
--
-- Configuring done
-- Generating done
I wonder if there are any modules I missed for loading .png images, or other reasons for that failure.
i have a rails application and change to cookie_store to active_record_store, but when i used a converter video in ffmpeg i getting this error
Maybe someone can help me to solve it?
error while running command ffmpeg -i "/tmp/9ce6ffe844cfb53c6adb76fef1f3330020160930-13071-hutiuw.mp4" -acodec aac -strict experimental -vcodec libx264 -s 720x405 -y "/tmp/9ce6ffe844cfb53c6adb76fef1f3330020160930-13071-hutiuw20160930-13071-1xp6lwi.mp4": Command 'PATH=/usr/local/bin/:$PATH; ffmpeg -i "/tmp/9ce6ffe844cfb53c6adb76fef1f3330020160930-13071-hutiuw.mp4" -acodec aac -strict experimental -vcodec libx264 -s 720x405 -y "/tmp/9ce6ffe844cfb53c6adb76fef1f3330020160930-13071-hutiuw20160930-13071-1xp6lwi.mp4"' returned 1. Expected 0 Here is the command output: STDOUT: STDERR: ffmpeg version 2.8.6-1ubuntu2 Copyright (c) 2000-2016 the FFmpeg developers built with gcc 5.3.1 (Ubuntu 5.3.1-11ubuntu1) 20160311 configuration: --prefix=/usr --extra-version=1ubuntu2 --build-suffix=-ffmpeg --toolchain=hardened --libdir=/usr/lib/i386-linux-gnu --incdir=/usr/include/i386-linux-gnu --cc=cc --cxx=g++ --enable-gpl --enable-shared --disable-stripping --disable-decoder=libopenjpeg --disable-decoder=libschroedinger --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librtmp --enable-libschroedinger --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzvbi --enable-openal --enable-opengl --enable-x11grab --enable-libdc1394 --enable-libiec61883 --enable-libzmq --enable-frei0r --enable-libx264 --enable-libopencv --disable-i686 libavutil 54. 31.100 / 54. 31.100 libavcodec 56. 60.100 / 56. 60.100 libavformat 56. 40.101 / 56. 40.101 libavdevice 56. 4.100 / 56. 4.100 libavfilter 5. 40.101 / 5. 40.101 libavresample 2. 1. 0 / 2. 1. 0 libswscale 3. 1.101 / 3. 1.101 libswresample 1. 2.101 / 1. 2.101 libpostproc 53. 3.100 / 53. 3.100 Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/tmp/9ce6ffe844cfb53c6adb76fef1f3330020160930-13071-hutiuw.mp4': Metadata: major_brand : mp42 minor_version : 0 compatible_brands: isommp42 creation_time : 2016-07-12 16:30:41 Duration: 00:04:38.96, start: 0.000000, bitrate: 1520 kb/s Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709), 1280x720 [SAR 1:1 DAR 16:9], 1325 kb/s, 29.97 fps, 29.97 tbr, 30k tbn, 59.94 tbc (default) Metadata: handler_name : VideoHandler Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 191 kb/s (default) Metadata: creation_time : 2016-07-12 16:30:42 handler_name : IsoMedia File Produced by Google, 5-11-2011 [libx264 # 0xa1105e0] height not divisible by 2 (720x405) Output #0, mp4, to '/tmp/9ce6ffe844cfb53c6adb76fef1f3330020160930-13071-hutiuw20160930-13071-1xp6lwi.mp4': Metadata: major_brand : mp42 minor_version : 0 compatible_brands: isommp42 Stream #0:0(und): Video: h264, none, q=2-31, 128 kb/s, SAR 1:1 DAR 0:0, 29.97 fps (default) Metadata: handler_name : VideoHandler encoder : Lavc56.60.100 libx264 Stream #0:1(eng): Audio: aac, 0 channels, 128 kb/s (default) Metadata: creation_time : 2016-07-12 16:30:42 handler_name : IsoMedia File Produced by Google, 5-11-2011 encoder : Lavc56.60.100 aac Stream mapping: Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264)) Stream #0:1 -> #0:1 (aac (native) -> aac (native)) Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
I don't think your choice of cookie store had anything to do with that. This happened because you chose a video file with this particular input size (1280x720).
Error is:
[libx264 # 0xa1105e0] height not divisible by 2 (720x405)
There are plenty questions on this site about this error but in the end it boils down to a simple answer: "make sure encoded picture has even width and height". In your case output picture size (720x405) is passed down from your site code so you need to fix it there.
I need to compute the frame differences between a source video and a compressed one.
For now I'm using OpenCV with Java, by extracting each frame and doing a simple difference, but it's quite slow (working a 0.5 fps, meaning that a 500 frames video will take more than 15 mins) so I was thinking to move to FFMPEG.
FFMPEG feels a lot faster (everything's done under 1 minute) but it has one big issue that makes the results useless: when compressing the source file, done with FFMPEG too, an extra gray frame is added at the beginning and this fakes the results because different frames are compared.
This is what I'm doing now (knowing that the extra frame messes it all):
ffmpeg -y -i src.avi -i compressed.avi -filter_complex "blend=all_mode=difference,hue=s=0" -c:v libx264 -crf 18 -c:a copy difference.avi
To fix the frame issue I was trying to remove the first frame by re-encoding the compressed video with this command
ffmpeg -y -ss 0.02 -i compressed.mpg -an -f mpeg2video compressed-cut.mpg"
(Note that -ss is 0.02 because it's a 50 fps video, so I did 1/FPS as suggested here)
But I get this response
Output file is empty, nothing was encoded (check -ss / -t / -frames parameters if used)
So, finally, the question is: since extracting all the frames and then compute differences with OpenCV is really slow, how can I use FFMPEG to produce a video containing the difference between two sources while keeping in mind that one of them has an extra frame at the beginning?
EDIT: I wanted to avoid posting endless console outputs but since you asked for it, here we go.
1) Encoding
Input
ffmpeg -i "720p50_mobcal_ter.avi" -an -f mpeg2video -y "720p50_mobcal_ter.mpg"
Output
ffmpeg version N-76684-g1fe82ab Copyright (c) 2000-2015 the FFmpeg developers
built with gcc 5.2.0 (GCC)
configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libdcadec --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinger --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libzimg --enable-lzma --enable-decklink --enable-zlib
libavutil 55. 6.100 / 55. 6.100
libavcodec 57. 15.100 / 57. 15.100
libavformat 57. 14.100 / 57. 14.100
libavdevice 57. 0.100 / 57. 0.100
libavfilter 6. 15.100 / 6. 15.100
libswscale 4. 0.100 / 4. 0.100
libswresample 2. 0.101 / 2. 0.101
libpostproc 54. 0.100 / 54. 0.100
Input #0, avi, from '720p50_mobcal_ter.avi':
Metadata:
encoder : Lavf57.14.100
Duration: 00:00:10.08, start: 0.000000, bitrate: 552974 kb/s
Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 1280x720, 554059 kb/s, SAR 1:1 DAR 16:9, 50 fps, 50 tbr, 50 tbn, 50 tbc
Output #0, mpeg2video, to '720p50_mobcal_ter.mpg':
Metadata:
encoder : Lavf57.14.100
Stream #0:0: Video: mpeg2video, yuv420p, 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 50 fps, 50 tbn, 50 tbc
Metadata:
encoder : Lavc57.15.100 mpeg2video
Stream mapping:
Stream #0:0 -> #0:0 (rawvideo (native) -> mpeg2video (native))
Press [q] to stop, [?] for help
frame= 41 fps=0.0 q=31.0 size= 984kB time=00:00:00.78 bitrate=10330.5kbits/frame= 80 fps= 78 q=31.0 size= 1323kB time=00:00:01.56 bitrate=6948.1kbits/frame= 124 fps= 80 q=31.0 size= 1725kB time=00:00:02.44 bitrate=5790.0kbits/frame= 168 fps= 81 q=31.0 size= 2084kB time=00:00:03.32 bitrate=5142.8kbits/frame= 212 fps= 81 q=31.0 size= 2482kB time=00:00:04.20 bitrate=4841.4kbits/frame= 255 fps= 82 q=31.0 size= 2840kB time=00:00:05.06 bitrate=4597.2kbits/frame= 296 fps= 82 q=31.0 size= 3133kB time=00:00:05.88 bitrate=4364.5kbits/frame= 338 fps= 82 q=24.8 size= 3453kB time=00:00:06.72 bitrate=4209.2kbits/frame= 382 fps= 82 q=31.0 size= 3723kB time=00:00:07.60 bitrate=4013.4kbits/frame= 426 fps= 83 q=31.0 size= 4005kB time=00:00:08.48 bitrate=3869.1kbits/frame= 470 fps= 83 q=24.8 size= 4276kB time=00:00:09.36 bitrate=3742.5kbits/frame= 504 fps= 83 q=31.0 Lsize= 4469kB time=00:00:10.06 bitrate=3639.3kbits/s
video:4469kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000000%
This adds the extra grey frame at the beginning, it just duplicates the first one
2) Removing first frame
Input
ffmpeg -y -i "720p50_mobcal_ter.mpg" -an -f mpeg2video -vf select=gte(n\,1) "CUT-720p50_mobcal_ter.mpg"
Output
ffmpeg version N-76684-g1fe82ab Copyright (c) 2000-2015 the FFmpeg developers
built with gcc 5.2.0 (GCC)
configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libdcadec --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinger --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libzimg --enable-lzma --enable-decklink --enable-zlib
libavutil 55. 6.100 / 55. 6.100
libavcodec 57. 15.100 / 57. 15.100
libavformat 57. 14.100 / 57. 14.100
libavdevice 57. 0.100 / 57. 0.100
libavfilter 6. 15.100 / 6. 15.100
libswscale 4. 0.100 / 4. 0.100
libswresample 2. 0.101 / 2. 0.101
libpostproc 54. 0.100 / 54. 0.100
Input #0, mpegvideo, from '720p50_mobcal_ter.mpg':
Duration: N/A, bitrate: N/A
Stream #0:0: Video: mpeg2video (Main), yuv420p(tv), 1280x720 [SAR 1:1 DAR 16:9], max. 104857 kb/s, 50 fps, 50 tbr, 1200k tbn, 100 tbc
Output #0, mpeg2video, to 'CUT-720p50_mobcal_ter.mpg':
Metadata:
encoder : Lavf57.14.100
Stream #0:0: Video: mpeg2video, yuv420p, 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 50 fps, 50 tbn, 50 tbc
Metadata:
encoder : Lavc57.15.100 mpeg2video
Stream mapping:
Stream #0:0 -> #0:0 (mpeg2video (native) -> mpeg2video (native))
Press [q] to stop, [?] for help
frame= 255 fps=0.0 q=31.0 size= 2781kB time=00:00:05.10 bitrate=4467.3kbits/frame= 503 fps=0.0 q=31.0 Lsize= 4415kB time=00:00:10.08 bitrate=3588.5kbits/s
video:4415kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000000%
3) Frame difference
Input
ffmpeg -y -i "720p50_mobcal_ter.avi" -i "CUT-720p50_mobcal_ter.mpg" -filter_complex "blend=all_mode=difference,hue=s=0" -c:v libx264 -crf 18 -c:a copy "DIFF-720p50_mobcal_ter.mpg"
Output
ffmpeg version N-76684-g1fe82ab Copyright (c) 2000-2015 the FFmpeg developers
built with gcc 5.2.0 (GCC)
configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libdcadec --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinger --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libzimg --enable-lzma --enable-decklink --enable-zlib
libavutil 55. 6.100 / 55. 6.100
libavcodec 57. 15.100 / 57. 15.100
libavformat 57. 14.100 / 57. 14.100
libavdevice 57. 0.100 / 57. 0.100
libavfilter 6. 15.100 / 6. 15.100
libswscale 4. 0.100 / 4. 0.100
libswresample 2. 0.101 / 2. 0.101
libpostproc 54. 0.100 / 54. 0.100
Input #0, avi, from '720p50_mobcal_ter.avi':
Metadata:
encoder : Lavf57.14.100
Duration: 00:00:10.08, start: 0.000000, bitrate: 552974 kb/s
Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 1280x720, 554059 kb/s, SAR 1:1 DAR 16:9, 50 fps, 50 tbr, 50 tbn, 50 tbc
Input #1, mpegvideo, from 'CUT-720p50_mobcal_ter.mpg':
Duration: N/A, bitrate: N/A
Stream #1:0: Video: mpeg2video (Main), yuv420p(tv), 1280x720 [SAR 1:1 DAR 16:9], max. 104857 kb/s, 50 fps, 50 tbr, 1200k tbn, 100 tbc
[libx264 # 000002784dbeb980] using SAR=1/1
[libx264 # 000002784dbeb980] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 AVX2 LZCNT BMI2
[libx264 # 000002784dbeb980] profile High, level 3.2
[mpeg # 000002784dbeaf20] VBV buffer size not set, using default size of 130KB
If you want the mpeg file to be compliant to some specification
Like DVD, VCD or others, make sure you set the correct buffer size
Output #0, mpeg, to 'D:\DOWNLOADS\TMP\Video TDI\AVI\DIFF-720p50_mobcal_ter.mpg':
Metadata:
encoder : Lavf57.14.100
Stream #0:0: Video: h264 (libx264), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], q=-1--1, 50 fps, 90k tbn, 50 tbc (default)
Metadata:
encoder : Lavc57.15.100 libx264
Stream mapping:
Stream #0:0 (rawvideo) -> blend:top
Stream #1:0 (mpeg2video) -> blend:bottom
hue -> Stream #0:0 (libx264)
Press [q] to stop, [?] for help
frame= 504 fps= 39 q=-1.0 Lsize= 32182kB time=00:00:10.04 bitrate=26258.5kbits/s
video:32061kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.377054%
[libx264 # 000002784dbeb980] frame I:30 Avg QP:19.69 size:149974
[libx264 # 000002784dbeb980] frame P:299 Avg QP:23.28 size: 69423
[libx264 # 000002784dbeb980] frame B:175 Avg QP:24.48 size: 43280
[libx264 # 000002784dbeb980] consecutive B-frames: 30.6% 69.4% 0.0% 0.0%
[libx264 # 000002784dbeb980] mb I I16..4: 18.3% 51.4% 30.4%
[libx264 # 000002784dbeb980] mb P I16..4: 0.6% 5.6% 2.4% P16..4: 35.9% 22.9% 15.6% 0.0% 0.0% skip:17.0%
[libx264 # 000002784dbeb980] mb B I16..4: 0.2% 0.5% 0.3% B16..8: 49.5% 12.4% 5.6% direct:15.5% skip:16.1% L0:47.8% L1:42.1% BI:10.1%
[libx264 # 000002784dbeb980] 8x8 transform intra:57.5% inter:38.5%
[libx264 # 000002784dbeb980] coded y,uvDC,uvAC intra: 90.7% 0.0% 0.0% inter: 50.3% 0.0% 0.0%
[libx264 # 000002784dbeb980] i16 v,h,dc,p: 32% 23% 35% 10%
[libx264 # 000002784dbeb980] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 11% 11% 41% 7% 5% 6% 5% 6% 8%
[libx264 # 000002784dbeb980] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 18% 14% 23% 8% 7% 7% 7% 7% 8%
[libx264 # 000002784dbeb980] i8c dc,h,v,p: 100% 0% 0% 0%
[libx264 # 000002784dbeb980] Weighted P-Frames: Y:33.8% UV:0.0%
[libx264 # 000002784dbeb980] ref P L0: 58.1% 16.3% 14.2% 9.4% 2.0%
[libx264 # 000002784dbeb980] ref B L0: 79.3% 20.7%
[libx264 # 000002784dbeb980] kb/s:26056.02
The second command made everything work while the second one in the first part didn't (the one with the -ss option), so I could be quite happy about it but I'm not that sure if FFMPEG duplicates the first frame for every video or if it's just related to the one I'm using now, so it could be better to start off with a compressed video that has the same frame count of the original one.
So let's get to one final question: why does FFMPEG add a duplicated first frame at the beginning of the compressed video and how can I avoid that?
Encoding with
ffmpeg -i "720p50_mobcal_ter.avi" -c:v mpeg2video "720p50_mobcal_ter.mpg"
instead of
ffmpeg -i "720p50_mobcal_ter.avi" -an -f mpeg2video -y "720p50_mobcal_ter.mpg"
prevents the creation of the duplicate frame and makes everything work correctly.
Does anyone have any experience in this area?
I've tried so far with ffmpeg (using libx264 and libaacplus) muxing into mpegts over udp, but that mpegts muxer is obviously broken (confirmed from several different sources).
I've also tried with vlc, but it can only encode AAC-LC and not HE-AAC v2 (aacplus).
Anyway, the problem I need to solve is to have several different geographic locations, covered with webcams and I need udp to be able to have incoming streams all the time and not to worry about networks ups and downs (udp will simply continue sending packets when the network is up again). So, did any of you have any experience using streaming live media using h264 with aacplus over udp protocol and if yes, can you please give me any links or directions how to accomplish it.
Thanks a lot in advance.
I'm developing a system which is kind of a DVR that must periodically record a h264 file from a video device and at the same time provide a local preview to allow for adjusting the video parameters and camera view. Although I am no expert at all in this field, I had a relative success streaming h264 over UDP, so I'll try to share what didn't and what did work out for me, which was all based on ffmpeg as a server (no audio in my case).
Initially I had set up my application to simultaneously record the video to a file and feed it to a .ffm file for ffserver to stream it up using RTP/UDP for the camera preview. The problem with that approach was exactly that when the feeding ffmpeg process stopped to change the video file, the preview would stop and never resume although the subsequent ffmpeg process would have already started feeding ffserver again. With RTP ffserver apparently complains the frames timestamp started back at 0 instead of continuing where they have stopped. Then I realized that if I were able to send the h264 packets over pure UDP I would get exactly the same effect you described, with preview resuming as soon as the next ffmpeg process would take over.
While trying to understand ffmpeg documentation I also tried using the mpegts format, but doing that I was getting MPEG2 video at the player on the other end (this shows a multicast address, but it also worked for a specific target):
$ ffmpeg -y -f video4linux2 -i /dev/video2 -vcodec libx264 -preset ultrafast /mnt/hd/video.mp4 -an -f mpegts udp://224.124.0.1:5000
ffmpeg version N-35860-g62adc60, Copyright (c) 2000-2011 the FFmpeg developers
built on Dec 16 2011 09:47:41 with gcc 4.5.3
configuration: --prefix=/usr --libdir=/usr/lib --shlibdir=/usr/lib --mandir=/usr/man --disable-debug --enable-shared --disable-static --enab
le-pthreads --enable-libtheora --enable-libvorbis --enable-gpl --enable-version3 --enable-postproc --enable-swscale --enable-avfilter --enable
-libx264 --enable-libvpx --enable-librtmp --disable-indev='v4l,dv1394'
libavutil 51. 32. 0 / 51. 32. 0
libavcodec 53. 46. 0 / 53. 46. 0
libavformat 53. 26. 0 / 53. 26. 0
libavdevice 53. 4. 0 / 53. 4. 0
libavfilter 2. 53. 0 / 2. 53. 0
libswscale 2. 1. 0 / 2. 1. 0
libpostproc 51. 2. 0 / 51. 2. 0
[video4linux2,v4l2 # 0x8a96b00] Estimating duration from bitrate, this may be inaccurate
Input #0, video4linux2,v4l2, from '/dev/video2':
Duration: N/A, start: 1325538250.366878, bitrate: 27620 kb/s
Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 320x240, 27620 kb/s, 29.97 tbr, 1000k tbn, 29.97 tbc
[buffer # 0x8a9d8c0] w:320 h:240 pixfmt:yuv420p tb:1/1000000 sar:0/1 sws_param:
[buffer # 0x8a9c860] w:320 h:240 pixfmt:yuv420p tb:1/1000000 sar:0/1 sws_param:
[libx264 # 0x8a97780] using cpu capabilities: MMX2 Cache64
[libx264 # 0x8a97780] profile Constrained Baseline, level 1.3
[libx264 # 0x8a97780] 264 - core 120 - H.264/MPEG-4 AVC codec - Copyleft 2003-2011 - http://www.videolan.org/x264.html - options: cabac=0 ref=
1 deblock=0:0:0 analyse=0:0 me=dia subme=0 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=0 cqm=0 deadzone=21,11
fast_pskip=1 chroma_qp_offset=0 threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=
0 keyint=250 keyint_min=25 scenecut=0 intra_refresh=0 rc=crf mbtree=0 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=0
[mpegts # 0x8a98100] muxrate VBR, pcr every 2 pkts, sdt every 200, pat/pmt every 40 pkts
Output #0, mp4, to '/mnt/hd/video.mp4':
Metadata:
encoder : Lavf53.26.0
Stream #0:0: Video: h264 (![0][0][0] / 0x0021), yuv420p, 320x240, q=-1--1, 30k tbn, 29.97 tbc
Output #1, mpegts, to 'udp://224.124.0.1:5000':
Metadata:
encoder : Lavf53.26.0
Stream #1:0: Video: mpeg2video, yuv420p, 320x240, q=2-31, 200 kb/s, 90k tbn, 29.97 tbc
Stream mapping:
Stream #0:0 -> #0:0 (rawvideo -> libx264)
Stream #0:0 -> #1:0 (rawvideo -> mpeg2video)
Press [q] to stop, [?] for help
On the client PC I was able to watch the video with ffplay, and it was indeed in MPEG2 format:
$ ffplay -f mpegts udp://224.124.0.1:5000
ffplay version N-35860-g62adc60, Copyright (c) 2003-2011 the FFmpeg developers
built on Dec 16 2011 09:47:41 with gcc 4.5.3
configuration: --prefix=/usr --libdir=/usr/lib --shlibdir=/usr/lib --mandir=/usr/man --disable-debug --enable-shared --disable-static --enable-pthreads --enable-libtheora --enable-libvorbis --enable-gpl --enable-version3 --enable-postproc --enable-swscale --enable-avfilter --enable-libx264 --enable-libvpx --enable-librtmp --disable-indev='v4l,dv1394'
libavutil 51. 32. 0 / 51. 32. 0
libavcodec 53. 46. 0 / 53. 46. 0
libavformat 53. 26. 0 / 53. 26. 0
libavdevice 53. 4. 0 / 53. 4. 0
libavfilter 2. 53. 0 / 2. 53. 0
libswscale 2. 1. 0 / 2. 1. 0
libpostproc 51. 2. 0 / 51. 2. 0
[mpegts # 0x80f02e0] Unable to seek back to the start
[mpeg2video # 0x8111a00] mpeg_decode_postinit() failure
Last message repeated 6 times
[mpegts # 0x80f02e0] max_analyze_duration 5000000 reached at 5005000
[mpegts # 0x80f02e0] Estimating duration from bitrate, this may be inaccurate
Input #0, mpegts, from 'udp://224.124.0.1:5000':
Duration: N/A, start: 255.420433, bitrate: 104857 kb/s
Program 1
Metadata:
service_name : Service01
service_provider: FFmpeg
Stream #0:0[0x100]: Video: mpeg2video (Main) ([2][0][0][0] / 0x0002), yuv420p, 320x240 [SAR 1:1 DAR 4:3], 104857 kb/s, 30.97 fps, 29.97 tbr, 90k tbn, 59.94 tbc
VLC also worked to play the stream, but only after specifying the switch --demux ffmpeg (thanks to this):
vlc -vv --demux ffmpeg udp://#224.124.0.1:5000
Since I also wanted the preview to be in h264 but I already had it encoding the file, I tried to use the copy codec for the UDP streaming, but ffmpeg failed with a segfault (versions included for reference):
ffmpeg -y -f video4linux2 -i /dev/video2 -vcodec libx264 -preset ultrafast /mnt/hd/video.mp4 -an -vcodec cop
y -f h264 udp://224.124.0.1:5000
ffmpeg version N-35860-g62adc60, Copyright (c) 2000-2011 the FFmpeg developers
built on Dec 16 2011 09:47:41 with gcc 4.5.3
configuration: --prefix=/usr --libdir=/usr/lib --shlibdir=/usr/lib --mandir=/usr/man --disable-debug --enable-shared --disable-static --enab
le-pthreads --enable-libtheora --enable-libvorbis --enable-gpl --enable-version3 --enable-postproc --enable-swscale --enable-avfilter --enable
-libx264 --enable-libvpx --enable-librtmp --disable-indev='v4l,dv1394'
libavutil 51. 32. 0 / 51. 32. 0
libavcodec 53. 46. 0 / 53. 46. 0
libavformat 53. 26. 0 / 53. 26. 0
libavdevice 53. 4. 0 / 53. 4. 0
libavfilter 2. 53. 0 / 2. 53. 0
libswscale 2. 1. 0 / 2. 1. 0
libpostproc 51. 2. 0 / 51. 2. 0
[video4linux2,v4l2 # 0x92c7b00] Estimating duration from bitrate, this may be inaccurate
Input #0, video4linux2,v4l2, from '/dev/video2':
Duration: N/A, start: 1325539132.411691, bitrate: 27620 kb/s
Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 320x240, 27620 kb/s, 29.97 tbr, 1000k tbn, 29.97 tbc
[buffer # 0x92ce860] w:320 h:240 pixfmt:yuv420p tb:1/1000000 sar:0/1 sws_param:
[libx264 # 0x92c8780] using cpu capabilities: MMX2 Cache64
[libx264 # 0x92c8780] profile Constrained Baseline, level 1.3
[libx264 # 0x92c8780] 264 - core 120 - H.264/MPEG-4 AVC codec - Copyleft 2003-2011 - http://www.videolan.org/x264.html - options: cabac=0 ref=
1 deblock=0:0:0 analyse=0:0 me=dia subme=0 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=0 cqm=0 deadzone=21,11
fast_pskip=1 chroma_qp_offset=0 threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=
0 keyint=250 keyint_min=25 scenecut=0 intra_refresh=0 rc=crf mbtree=0 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=0
Output #0, mp4, to '/mnt/hd/video.mp4':
Metadata:
encoder : Lavf53.26.0
Stream #0:0: Video: h264 (![0][0][0] / 0x0021), yuv420p, 320x240, q=-1--1, 30k tbn, 29.97 tbc
Output #1, h264, to 'udp://224.124.0.1:5000':
Metadata:
encoder : Lavf53.26.0
Stream #1:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 320x240, q=2-31, 27620 kb/s, 90k tbn, 29.97 tbc
Stream mapping:
Stream #0:0 -> #0:0 (rawvideo -> libx264)
Stream #0:0 -> #1:0 (copy)
Press [q] to stop, [?] for help
Segmentation fault
Although less than ideal, specifying the format h264 for the UDP streaming part led to a second concurrent h264 conversion, but it worked:
$ ffmpeg -y -f video4linux2 -i /dev/video2 -vcodec libx264 -preset ultrafast /mnt/hd/video.mp4 -an -f h264 -preset ultrafast udp://224.124.0.1:5000
ffmpeg version N-35860-g62adc60, Copyright (c) 2000-2011 the FFmpeg developers
built on Dec 16 2011 09:47:41 with gcc 4.5.3
configuration: --prefix=/usr --libdir=/usr/lib --shlibdir=/usr/lib --mandir=/usr/man --disable-debug --enable-shared --disable-static --enab
le-pthreads --enable-libtheora --enable-libvorbis --enable-gpl --enable-version3 --enable-postproc --enable-swscale --enable-avfilter --enable
-libx264 --enable-libvpx --enable-librtmp --disable-indev='v4l,dv1394'
libavutil 51. 32. 0 / 51. 32. 0
libavcodec 53. 46. 0 / 53. 46. 0
libavformat 53. 26. 0 / 53. 26. 0
libavdevice 53. 4. 0 / 53. 4. 0
libavfilter 2. 53. 0 / 2. 53. 0
libswscale 2. 1. 0 / 2. 1. 0
libpostproc 51. 2. 0 / 51. 2. 0
[video4linux2,v4l2 # 0x913ab00] Estimating duration from bitrate, this may be inaccurate
Input #0, video4linux2,v4l2, from '/dev/video2':
Duration: N/A, start: 1325539689.729735, bitrate: 27620 kb/s
Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 320x240, 27620 kb/s, 29.97 tbr, 1000k tbn, 29.97 tbc
[buffer # 0x9141840] w:320 h:240 pixfmt:yuv420p tb:1/1000000 sar:0/1 sws_param:
[buffer # 0x913e480] w:320 h:240 pixfmt:yuv420p tb:1/1000000 sar:0/1 sws_param:
[libx264 # 0x913b780] using cpu capabilities: MMX2 Cache64
[libx264 # 0x913b780] profile Constrained Baseline, level 1.3
[libx264 # 0x913b780] 264 - core 120 - H.264/MPEG-4 AVC codec - Copyleft 2003-2011 - http://www.videolan.org/x264.html - options: cabac=0 ref=
1 deblock=0:0:0 analyse=0:0 me=dia subme=0 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=0 cqm=0 deadzone=21,11
fast_pskip=1 chroma_qp_offset=0 threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=
0 keyint=250 keyint_min=25 scenecut=0 intra_refresh=0 rc=crf mbtree=0 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=0
[libx264 # 0x913c820] using cpu capabilities: MMX2 Cache64
[libx264 # 0x913c820] profile Constrained Baseline, level 1.3
Output #0, mp4, to '/mnt/hd/video.mp4':
Metadata:
encoder : Lavf53.26.0
Stream #0:0: Video: h264 (![0][0][0] / 0x0021), yuv420p, 320x240, q=-1--1, 30k tbn, 29.97 tbc
Output #1, h264, to 'udp://224.124.0.1:5000':
Metadata:
encoder : Lavf53.26.0
Stream #1:0: Video: h264, yuv420p, 320x240, q=-1--1, 90k tbn, 29.97 tbc
Stream mapping:
Stream #0:0 -> #0:0 (rawvideo -> libx264)
Stream #0:0 -> #1:0 (rawvideo -> libx264)
Press [q] to stop, [?] for help
On the client side ffplay repeatedly complained about some missing information, but after some seconds it finally showed up the video, which was ok but a bit choppy:
$ ffplay -f h264 udp://224.124.0.1:5000
[h264 # 0xa0be740] non-existing PPS referenced
[h264 # 0xa0be740] non-existing PPS 0 referenced
[h264 # 0xa0be740] decode_slice_header error
[h264 # 0xa0be740] no frame!
(...)
[h264 # 0xa0be740] non-existing PPS referenced
[h264 # 0xa0be740] non-existing PPS 0 referenced
[h264 # 0xa0be740] decode_slice_header error
[h264 # 0xa0be740] no frame!
[h264 # 0xa0e72e0] max_analyze_duration 5000000 reached at 5013967
[h264 # 0xa0e72e0] Estimating duration from bitrate, this may be inaccurate
Input #0, h264, from 'udp://224.124.0.1:5000':
Duration: N/A, bitrate: N/A
Stream #0:0: Video: h264 (Constrained Baseline), yuv420p, 320x240, 47.27 fps, 29.97 tbr, 1200k tbn, 59.94 tbc
[h264 # 0xa0be740] Missing reference picture
[h264 # 0xa0be740] decode_slice_header error
[h264 # 0xa0be740] concealing 300 DC, 300 AC, 300 MV errors
[h264 # 0xa0be740] Missing reference picture 0KB sq= 0B f=0/0 0/0
[h264 # 0xa0be740] decode_slice_header error
[h264 # 0xa0be740] mmco: unref short failure
[h264 # 0xa0be740] concealing 300 DC, 300 AC, 300 MV errors
10.78 A-V: 0.000 fd= 0 aq= 0KB vq= 0KB sq= 0B f=0/0
I then tried VLC with the same ''--demux ffmpeg'' switch; it also complained about SPS/PPS (don't know what it is about yet) but in the end it played the video really smoothly:
$ vlc -v --demux ffmpeg udp/h264://#224.124.0.1:5000
VLC media player 1.1.12 The Luggage (revision exported)
Blocked: call to unsetenv("DBUS_ACTIVATION_ADDRESS")
Blocked: call to unsetenv("DBUS_ACTIVATION_BUS_TYPE")
[0x943346c] main libvlc: Running vlc with the default interface. Use 'cvlc' to use vlc without interface.
Blocked: call to setlocale(6, "")
Blocked: call to setlocale(6, "")
[0x94cde8c] qt4 interface error: Unable to load extensions module
[0x96720e4] h264 demux error: this doesn't look like a H264 ES stream, continuing anyway
[0x963989c] access_udp access warning: unimplemented query in control
[0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS
[0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS
[0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS
[0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS
[0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS
[0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS
[0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS
[0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS
[0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS
[0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS
[0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS
[0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS
[0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS
[0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS
[0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS
[0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS
[0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS
[0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS
[0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS
[0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS
[0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS
[0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS
[0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS
[0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS
[0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS
[0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS
[0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS
[0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS
[0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS
[0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS
[0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS
[0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS
[0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS
[0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS
[0x967328c] packetizer_h264 demux packetizer warning: waiting for SPS/PPS
[0x94be644] main input error: Invalid PCR value in ES_OUT_SET_(GROUP_)PCR !
[0xb2a325f4] avcodec decoder warning: disabling direct rendering
[0x96dfac4] main video output warning: vlc_object_find_name(postproc) is not safe!
[0x94c2474] signals interface warning: signal 17 overridden (0xb6f31030)
[0x94c2474] signals interface warning: /usr/lib/qt/lib/libQtCore.so.4(?)[(nil)]
[0x96dfac4] main video output warning: late picture skipped (32703 > -4)
I'm still trying to figure out what is missing, but the results are pleasing already. After VLC is playing I can stop the feeding ffmpeg that playback pauses and resumes right away when I restart ffmpeg on the other end.
Hope this can be of any help, and please let me know if you have any additional info on using h264 as the chosen format in place of mpegts - I suspect this missing SPS/PPS info might have something to do with it.