I am trying to write video using opencv. It is important for me to do this precisely - so it has to be a lossless codec. I am working with OpenCV 2.4.1 on Ubuntu 12.04
Previously, I was using the fourcc code 0. This gave me the exact result I wanted, and I was able to recover the images perfectly.
I am not sure what happened, but as of a recent update (around Jul 20th 2012), something went wrong and I am no longer able to write files with this fourcc code. I really don't remember what it was, but it could have come from doing an update, removing some software from my software center, and some other things I did during general cleaning...
When I check an older file with mediainfo (http://www.fourcc.org/identifier/) I see the following result:
Complete name : oldsample.avi
Format : AVI
Format/Info : Audio Video Interleave
Format profile : OpenDML
File size : 1.07 GiB
Duration : 41s 467ms
Overall bit rate : 221 Mbps
Writing application : Lavf53.5.0
Video
ID : 0
Format : RGB
Codec ID : 0x00000000
Codec ID/Info : Basic Windows bitmap format. 1, 4 and 8 bpp versions are palettised. 16, 24 and 32bpp contain raw RGB samples
Duration : 41s 467ms
Bit rate : 221 Mbps
Width : 640 pixels
Height : 4294966 816 pixels
Display aspect ratio : 0.000
Frame rate : 30.000 fps
Bit depth : 8 bits
Stream size : 1.07 GiB (100%)
Now, I see that when I write using the 0 fourcc codec, the program actually defaults to the i420 codec. Here is the output from one of the files I try to write now:
Complete name : newsample.avi
Format : AVI
Format/Info : Audio Video Interleave
File size : 73.0 MiB
Duration : 5s 533ms
Overall bit rate : 111 Mbps
Writing application : Lavf54.6.100
Video
ID : 0
Format : YUV
Codec ID : I420
Codec ID/Info : 8 bit Y plane followed by 8 bit 2x2 subsampled U and V planes.
Duration : 5s 533ms
Bit rate : 111 Mbps
Width : 640 pixels
Height : 480 pixels
Display aspect ratio : 4:3
Frame rate : 30.000 fps
Compression mode : Lossless
Bits/(Pixel*Frame) : 12.000
Stream size : 72.9 MiB (100%)
This format, and other formats I try to use (like huffyuv HFYU), do not work for me because I end up with effects like this http://imgur.com/a/0OC4y - you see the bright artifacts coming in due to what I assume is either lossy compression or chroma subsampling in the case of HFYU which is supposed to be lossless. What you are looking at is the red channel from one of my videos. The perceptual effect is negligible when you look at all 3 channels simultaneously but it is essential that I reconstruct the images exactly.
Furthermore, while I am able to play my old files in media players like vlc, I suddenly find them to be completely incompatible with opencv. When I try to open the older files with a videocapture, the open step works fine, but trying to do a read operation results in a segfault. Furthermore, When I try to write with either:
CV_FOURCC(0,0,0,0)
0
Opencv defaults to I420 for some reason.
Next, I tried using some alternate codecs. 'DIB ' seems like something that should work for me, and on the opencv website (http://opencv.willowgarage.com/wiki/VideoCodecs) it is listed as a 'recommended' codec. However, trying to use this results in the following message:
OpenCV-2.4.1/modules/highgui/src/cap_gstreamer.cpp:483: error: (-210) Gstreamer Opencv backend doesn't support this codec acutally. in function CvVideoWriter_GStreamer::open
Aborted (core dumped)
I checked the opencv source for this codec, and stumbled across the following:
cd OpenCV-2.4.1/modules
grep -i -r "CV_FOURCC" ./*
...
./highgui/src/cap_qt.cpp: /*if( fourcc == CV_FOURCC( 'D', 'I', 'B', ' ' ))
./highgui/include/opencv2/highgui/highgui_c.h:#define CV_FOURCC_DEFAULT CV_FOURCC('I', 'Y', 'U', 'V') /* Use default codec for specified filename (Linux only) */
I tried installing qt4 and reconfiguring with the WITH_QT flag, but that did not change anything. I also tried uncommenting that part of the code and reinstalling opencv, but that also did not work.
My ultimate goal is for any way to efficiently store and retrieve a video stream with 16 bits for every pixel (like 32float would work fine, and then it wouldn't need to be perfect). Right now I am unpacking the 16 bits into the red and green channels, which is why I need it to be perfect - since an error of 1 in the red channel is multiplied by 256 in the final result. I am not having success with any of the fourcc codes available to me.
Most probably you uninstalled or updated a codec. Try install a new codec pack, or update your ffmpeg, or gstreamer
I ended up figuring this out a little while ago, and finally got a chance to write it up for everyone. You can see my (rather hacky) solution here:
http://denislantsman.com/?p=111
Edit: As the website is down the following summarizes what can be found from the Wayback Machine:
Save frames as individual PNG images
Run ffmpeg to generate a file which can be opened by OpenCV:
ffmpeg -i ./outimg/depth%d.png -vcodec png depth.mov
The following Python snippet may be useful for saving the individual frames
std::ostringstream out_depth;
...
expand_depth(playback.pDepthMap, expanded_depth, playback.rows, playback.cols);
out_depth << root << "/outimg/depth" << framecount << ".png";
cv::imwrite(out_depth.str(), expanded_depth);
framecount++;
...
Why not use FFV1? Its compression rate is way better then dibs and it is widely available.
VideoWriter video("lossless.mkv", VideoWriter::fourcc('F','F','V','1'),FPS, Size(WIDTH,HEIGHT));
Related
I am working with a video stream from an ip camera on Ubuntu 14.04. Everything was going great with a camera that has these parameters (from FFMPEG):
Stream #0:0: Video: h264 (Main), yuv420p(progressive), 352x192, 29.97 tbr, 90k tbn, 180k tbc
But then i changed to a newer camera, which has these parameters:
Stream #0:0: Video: h264 (High), yuvj420p(pc, bt709, progressive), 1280x720, 25 fps, 25 tbr, 90k tbn, 50 tbc
My C++ program uses OpenCV3 to process the stream. By default OpenCV uses ffmpeg to decode and display the stream with function VideoCapture.
VideoCapture vc;
vc.open(input_stream);
while ((vc >> frame), !frame.empty()) {
*do work*
}
With the new camera stream i get errors like these (from ffmpeg):
[h264 # 0x7c6980] cabac decode of qscale diff failed at 41 38
[h264 # 0x7c6980] error while decoding MB 41 38, bytestream (3572)
[h264 # 0x7c6980] left block unavailable for requested intra mode at 0 44
[h264 # 0x7bc2c0] SEI type 25 truncated at 208
The image sometimes is glitched, sometimes completely frozen. However on vlc it plays perfectly. I installed the newest version (3.2.2) of ffmpeg player with
./configure --enable-gpl --enable-libx264
Now playing directly with ffplay (instead of launching from source code with OpenCV function VideoCapture), the stream plays better, but sometimes still displays warnings:
[NULL # 0x7f834c008c00] SEI type 25 size 896 truncated at 320=1/1
[h264 # 0x7f834c0d5d20] SEI type 25 size 896 truncated at 319=1/1
[rtsp # 0x7f834c0008c0] max delay reached. need to consume packet
[rtsp # 0x7f834c0008c0] RTP: missed 1 packets
[h264 # 0x7f834c094740] concealing 675 DC, 675 AC, 675 MV errors in P frame
Changing the camera hardware is not an option. The camera can be set to encode to h265 or mjpeg. When encoding to mjpeg it can output 5 fps, which is not enough. Decoding to a static video is not an option either, because i need to display real time results about the stream. Here is a list of API backends that can be used in function VideoCapture. Maybe i should switch to some other decoder and player?
From my research i conclude that i have these options:
Somehow get OpenCV to use libVlc instead of ffmpeg
One example of switching to vlc is here, but i don't understand it well enough to say if that is what i need. Or maybe i should be parsing the stream in code?
Use vlc to preprocess the stream, as suggested here.
This is probably slow, which again is bad for real time results.
Any suggestions and coments will be appreciated.
The errors are caused by packet loss since it uses RTP by default. You are seeing more errors now because you have switched to a higher bitrate input.
Append ?tcp to your input, eg. rtsp://*private*/media/video2?tcp in OpenCV's open function to force TCP, assuming it's supported by your hardware and/or usage scenario.
I have mostly solved this issue.
First i recompiled OpenCV with the newest version of ffmpeg manually installed on my device with the necessary settings.
I checked the stream with VLC tools > codec information to check if the stream has corrupted frames, which it didn't.
I lowered the stream resolution. This gave the biggest improvement.
The last error that i'm still left with is
[NULL # 0x7f834c008c00] SEI type 25 size 896 truncated at 320=1/1
[h264 # 0x7f834c0d5d20] SEI type 25 size 896 truncated at 319=1/1
but it doesn't visibly damage the frames. I have not resolved the stream freezing issue however, but it has something to do with my code, not the software. If i can help anyone with similar problems, feel free to ask additional info.
I have same problem after 4 days research finaly i solved the problem easy by this code:
for(;;) {
if(!vcap.read(image)) {
std::cout << "No frame" << std::endl;
cv::waitKey();
}
cv::imshow("Output Window", image);
if(cv::waitKey(1) >= 0) break;
}
OpenCV 2.4 VideoWriter couldn't save video files larger than 2GB, since it only accepts .avi files, I am wondering if this is still the case in OpenCV 3.0, or if it can save other kind of video files that doesn't have this limitations.
I tried to find any documentations pointing to a limit of 2GB or a release note saying it's capable to handle larger files, but I can't find none.
Even though the OpenCV 3.0-beta documentation states otherwise, OpenCV 3.0's VideoWriter seems to handle other file formats, such as mkv, as shown in this issue.
I adapted the code from the above issue to generate a 4GB mkv video (4096 frames of random 2048x2048).
The things to be aware is that the image size should be passed as width then height in the VideoWriter whereas the numpy array should be initialized with height then width. VideoWriter will fail silently otherwise.
You will also require a recent OpenCV 3.0 source to handle uncompressed streams.
This is not OpenCV limitation. AVI file size cannot be larger than 2 GB due to format limitations (4-byte size signed integer has max value 2,147,483,647).
Is it possible to pack video in another container with OpenCV (mkv etc)?
the RIFF header has the following form:
'RIFF' fileSize fileType (data)
where 'RIFF' is the literal FOURCC code 'RIFF',
fileSize is a 4-byte value giving the size of the data in the file,
and fileType is a FOURCC that identifies the specific file type.
I have a bare h.264 file (from a raspberry pi camera), and I'd like to wrap it as an mp4. I don't need to play it, edit it, add or remove anything, or access the pixels.
Lots of people have asked about compiling ffmpeg for iOS, or streaming live data. But given the lack of easy translation between the ffmpeg command line and its iOS build, it's very difficult for me to figure out how to implement this simple command:
ffmpeg -i input.h264 -vcodec copy out.mp4
I don't specifically care whether this happens via ffmpeg, avconv, or AVFoundation (or something else). It just seems like it should be not-this-hard to do on a device.
It is not hard but requires some work and attention to detail.
Here is my best guess:
read PPS/SPS from your input.h264
extract height & width from SPS
generate avcC header from PPS/SPS
create an AVAssetWriter with file type AVFileTypeQuickTimeMovie
create an AVAssetWriterInput
add the AVAssetWriterInput as AVMediaTypeVideo with your height & width to the AVAssetWriter
read from your input.h264 (likely in Annex B format) one NALs at a time
convert your NALs from your input.h264 from start code prefixed (0 0 1; Annex B) to size prefixed (mp4 format)
drop NALs of type AU, PPS, SPS
create a CMSampleBuffer for each NAL and add a CMFormatDescription with the avcC header
regenerate timestamps starting a zero using the known frame rate (watch out if your frames are reordered)
append your CMSampleBuffer to your AVAssetWriterInput
goto 7 until EOF
Basically the audio cape is working. Except for one strange phenomena that mistifies me. I will try to explain.
When I play a .wav file for example speaker-test -t vaw -> if lucky I hear Front Left - Front right as one expects. But 9 out of 10, I hear white noise with the audio front left front right very faint in the background or at another time the sound is simply distorted. The same happens when I play a file with aplay or mplayer.
So when I am lucky, or timing with respect to system clock is in sync I hear the audio clearly, if out of sync it might me white noise or distorted playback.
I have google extensively and have not found any solution. So I hope one of you guys knows whats happening here. It has to be something low level.
I'm quite a newby in this matter but according to this: Troubleshooting Linux Sound all seams to work ok.
These are my system parameters and settings: root#beaglebone:~# lsb_release -a Distributor ID: Angstrom Description: Angstrom GNU/Linux v2012.12 (Core edition) Release: v2012.12 Codename: Core edition
root#beaglebone:~# cat /sys/devices/bone_capemgr*/slots 0: 54:PF---
1: 55:PF---
2: 56:P---L CBB-Relay,00A0,Logic_Supply,CBB-Relay
3: 57:PF---
4: ff:P-O-L Bone-LT-eMMC-2G,00A0,Texas Instrument,BB-BONE-EMMC-2G
5: ff:P-O-- Bone-Black-HDMI,00A0,Texas Instrument,BB-BONELT-HDMI
6: ff:P-O-L Bone-Black-HDMIN,00A0,Texas Instrument,BB-BONELT-HDMIN
7: ff:P-O-L Override Board Name,00A0,Override Manuf,BB-BONE-AUDI-02
root#beaglebone:~# speaker-test -t wav
speaker-test 1.0.25
Playback device is default Stream parameters are 48000Hz, S16_LE, 1 channels WAV file(s) Rate set to 48000Hz (requested 48000Hz) Buffer size range from 128 to 32768 Period size range from 8 to 2048 Using max buffer size 32768 Periods = 4 was set period_size = 2048 was set buffer_size = 32768
0 - Front Left
Time per period = 0.641097
0 - Front Left
root#beaglebone:~# mplayer AxelF.wav MPlayer2 2.0-379-ge3f5043 (C) 2000-2011 MPlayer Team 162 audio & 361 video codecs
Playing AxelF.wav. Detected file format: WAV format (libavformat) [wav # 0xb6082780]max_analyze_duration reached [lavf] stream 0: audio (pcm_s16le), -aid 0 Load subtitles in .
==============================================================[edit]
Forced audio codec: mad Opening audio decoder: [pcm] Uncompressed PCM audio decoder AUDIO: 44100 Hz, 2 ch, s16le, 1411.2 kbit/100.00% (ratio: 176400->176400) Selected audio codec: [pcm] afm: pcm (Uncompressed PCM)
==============================================================[edit]
AO: [alsa] 44100Hz 2ch s16le (2 bytes per sample) Video: no video Starting playback... A: 1.6 (01.6) of 15.9 (15.8) 0.3%
MPlayer interrupted by signal 2 in module: unknown
Exiting... (Quit)
I can shed some light on what is causing the artifacts that you experience. I am sorry I do not yet have a countermeasure - I am struggling with the same problem. You describe the perceptible consequences pretty accurately.
Sound data travels from the ARM System on Chip to the Audio Codec on the audio cape using the I2S bus. I2S is a serial protocol, it sends one bit at a time, starting each sample with the most significant bit, then sending all bits down to the least significant bit. After the least significant bit of one sample is sent, the most significant bit of the sample on the next audio channel is sent. To be able to interpret the bit stream, the receiving audio codec needs to know when a new sound sample starts with its most significant bit, and also, to which channel each sound sample belongs. For this purpose, the "Word Select" (WS) signal is part of I2S and changes its value to indicate the start of the sound sample and also identifies the channel, see this I2S timing diagram for a better understanding of the concept.
What you and I perceive on our not-quite-working audio capes can be fully explained by the bit stream being interpreted out-of-step by the audio codec:
When you hear loud noise and the target signal soft in the background, then one or more of the least significant bits of the preceding sample are interpreted as the most significant bits of the current sample. The more bits are shifted, the softer the target signal, until you might only perceive noise when (this is a guess!) about 4 bits are shifted.
When the shift is in the other direction, i.e. most significant bit of the current sample was interpreted as the least significant bit of the preceding sample, then what you hear will sound correct for soft parts of the signal, i.e. when the most significant bit is not actually used (this is a simplification, see below). For louder parts of the signal, e.g. drum beats, you will perceive the missing most significant bit as distortion. Of course, the distortion gets worse and starts at softer levels as more bits are shifted in this direction.
In the above paragraph, the most significant will change with the sign of the data, so the statement that it is not actually used is valid only insofar as the most significant bit will have the same value as the next most significant bit for soft sounds. See Two's Complement for an introduction how negative integers are represented in computers.
I am not sure, where the corruption occurs. It could be that the WS signal is not correctly interpreted by the Audio Codec on the cape, or the WS signal is not correctly sent by the ARM System-on-Chip, or the bit shift might happen already inside the ARM CPU, e.g. in the Alsa driver.
I have a problem with a RAW YUB video load in OpenCV. I can play it in mplayer with the following command:
mplayer myvideo.raw -rawvideo w=1280:h=1024:fps=30:y8 -demuxer rawvideo
My code for load in OpenCV is:
CvCapture* capture=cvCaptureFromFile("C:\\myvideo.raw");
cvCaptureFromFile always return NULL. But if I try with a normal avi file, the code runs normally (capture is not null).
I'm working with the lastest version of OpenCV under Windows 7.
EDIT: Output messages are
[IMGUTILS # 0036f724] Picture size 0x0 is invalid
[image2 # 009f3300] Could not find codec parameters (Video: rawvideo, yuv420p)
Thanks
OpenCV uses ffmpeg as back-end, however, it includes only a subset of ffmpeg functions. What you can try is to install some codecs. (K-lite helped me some time ago)
But, if your aim is to obtain raw YUV in OpenCV, the answer is "not possible".
OpenCV is hardcoded to convert every input format to BGR, so even if you will be able to open the raw input, it will automatically convery it to BGR before passing it. No chance to solve that, the only way is to use a different capture library or hack into OpenCV.
What you can do (to simulate YUV input) is to capture the avi, convert to YUV
cvtColor(...,CV_BGR2YCBCR /* or CV_BGR2YUV */ );
and then process it