Video Morph Between Two Images, FFMPEG/Minterpolate - image-processing

I am trying to make a quick and easy morph video using two frames (png images) with ffmpeg's minterpolate filter, in a bash script on Ubuntu Linux. The intent is to use the morphs as transitions between similar video in a different video editor later.
It will work on 3+ frames/images, but fails using just 2 frames/images.
First the code that works: 3 frames
This is using three 1080p png files:
test01_01.png
test01_02.png
test01_03.png
input01="test01_%02d.png"
ffmpeg -y -fflags +genpts -r 30 -i $input01 -vf "setpts=100*PTS,minterpolate=fps=24:scd=none" -pix_fmt yuv420p "test01.mp4"
This takes a bit of processing time, then creates a 414kb, roughly three second mp4 video of a morph starting with the first frame, morphing to the second, then morphing to the third.
The code that fails: 2 frames
This is using just two of the same 1080p png files:
test02_01.png
test02_02.png
input01="test02_%02d.png"
ffmpeg -y -fflags +genpts -r 30 -i $input01 -vf "setpts=100*PTS,minterpolate=fps=24:scd=none" -pix_fmt yuv420p "test02.mp4"
This almost immediately creates a 262 byte corrupt mp4 file. There are no differences except the number of frames.
Things I've tried:
I have tried this with the Ubuntu default repo version of ffmpeg, and the static 64bit 5.0 and git-20220108-amd64 versions, all with the same result.
I have also tried with a 2-frame mp4 file as the input, with the same result.
Thoughts?
Is this a bug in ffmpeg or am I doing something wrong?
I am also open to any suggestions for creating a morph like this using other Linux-compatible software.
Thank you for any insight!

It is not documented, but it looks like minterpolate filter requires at least 3 input frames.
We may create a longer video using 5 input frames, and keep the relevant part.
For getting the same output as applying Minterpolate filter with only two input images, we may use the following solution:
Define two input streams:
Set test02_01.png as the first input and test02_02.png as the second input.
Loop each image at least twice, using -stream_loop
(test02_01.png is repeated twice and test02_02.png is repeated 3 times).
Set the input frame rate to 0.3 fps (it is equivalent to -r 30 and setpts=100*PTS).
The input arguments are as follows: -r 0.3 -stream_loop 1 -i test02_01.png -r 0.3 -stream_loop 2 -i test02_02.png.
Concatenate the two input streams using concat filter.
Apply minterpolate filer to the concatenated output.
The output of the above stage is a video with few redundant seconds at the beginning, and few redundant seconds at the end.
Apply trim filter for keeping the relevant part.
Add setpts=PTS-STARTPTS at the end (as recommended when using trim filter).
Suggested command:
ffmpeg -y -r 0.3 -stream_loop 1 -i test02_01.png -r 0.3 -stream_loop 2 -i test02_02.png -filter_complex "[0][1]concat=n=2:v=1:a=0[v];[v]minterpolate=fps=24:scd=none,trim=3:7,setpts=PTS-STARTPTS" -pix_fmt yuv420p test02.mp4
Sample output (as animate GIF):
test02_01.png:
test02_02.png:

Related

Convert a series of jpg into an mov file in Ruby (or using any language)

I am making a site in Ruby in which I have a series of images, (almost like a powerpoint) and I need to automatically convert those images into one continuous video file (mov, mpeg) that shows each image for 5 seconds or so. Any one have any clues where to start.
I'm also open to using another language if there are tools to get the job done.
You could probably use FFmpeg to do this. Here's an example from the FFmpeg Wiki on the subject:
ffmpeg -framerate 1/5 -i img%03d.jpg -c:v libx264 -r 30 -pix_fmt yuv420p -movflags +faststart out.mp4
What this would do is...
-i img%03d.jpg
Read input from a series of JPEG files named img001.jpg, img002.jpg and so on
-framerate 1/5
...at one frame per five seconds...
-c:v libx264
...then turn it into H.264/MPEG-4 AVC...
-r 30
...at thirty frames per second...
-pix_fmt yuv420p
...with YUV420 pixel format (really, all the FFmpeg flags work here)...
-movflags +faststart
...after encoding completes, relocate some data to the beginning of the file so playback can begin before the file is completely downloaded...
out.mp4
...and store it into out.mp4.
If you were using this from Ruby you'd likely launch a subprocess. The flags would be similar if you really want a (QuickTime) .mov file instead of H.264 MPEG-4.

Something wrong with my m3u8 bandwidth value

I use ffmpeg to encode my sample videos following the recommanded bitrates in Technical Note TN2224, then use HLS tools to segment it and create playlists, finally create the variant plist file "all.m3u8"
I used the validation tool to validate my HLS content, it ended up showing except for the 64k audio only bandwidth is low, others are stay in the same bandwidth, I opened "all.m3u8" using text editor and seeing that all other bitrate contents are using the same bandwidth. No matter how I change parameters in the ffmpeg command, I still can't correct them. The following command is the one I used to encode contents:
ffmpeg -i input.m4v -acodec libfaac -vcodec libx264 -s 480x360 -b 350k -r 29.97 -vpre medium output.mp4
The following command is for generating the segments and plists:mediafilesegmenter -b http://www.example.com/stream/ -I -f ~/Documents/sample/ output.mp4
The following command is for generating the all.m3u8:variantplaylistcreator -o all.m3u8 http://www.example.com/stream/110/prog_index.m3u8 ~/Documents/sample/110/prog_index.m3u8 -iframe-url http://www.freeyourteam.com/stream/110/iframe_index.m3u8 http://www.example.com/stream/200/prog_index.m3u8 ~/Documents/sample/200/prog_index.m3u8 -iframe-url http://www.freeyourteam.com/stream/200/iframe_index.m3u8 http://www.example.com/stream/350/prog_index.m3u8 ~/Documents/sample/350/prog_index.m3u8 -iframe-url http://www.freeyourteam.com/stream/350/iframe_index.m3u8 http://www.example.com/stream/550/prog_index.m3u8 ~/Documents/sample/550/prog_index.m3u8 -iframe-url http://www.freeyourteam.com/stream/550/iframe_index.m3u8 http://www.example.com/stream/64/prog_index.m3u8 ~/Documents/sample/64/prog_index.m3u8
and in my "all.m3u8", the bandwidths are all 523894:
Please allow me to ask two more basic questions:
In the tech note, recommanded bitrates are 64 Kbps, 110 Kbps, 200 Kbps, 350 Kbps, 550 Kbps, I wonder if this value includes the audio bitrate or exclude the audio.
How do you insert keyframe to segment? Because in the document it says:"You must include at least one keyframe per segment, preferably more. If you only include one, put it at the beginning of the segment." I don't quite get how you can do it.
Thank you very much for your help and I do appreciate your time.
Jason,
To create all.m3u8 should it not be given multiple m3u8 files each corresponding to a different bitrate?
I am guessing you run ffmpeg say 4 times to create for 4 bitrate files. Then you run the segmenter 4 times to create 4 set of segments and its individual m3u8 files.
Finally you have to tell the variantplaylistcreator where the location of the various m3u8 files per bitrate to create a single master m3u8 file.
Eg.
variantplaylistcreator -o mymedia_all.m3u8 http://mywebserver/mymedia_lo/prog_index.m3u8 mymedia_lo.plist http://mywebserver/mymedia_med/prog_index.m3u8 mymedia_med.plist http://mywebserver/mymedia_hi/prog_index.m3u8 mymedia_hi.plist
I don't see you providing the various filese seperately. I hope you get the picture.
EDIT: To answer your other questions:
Bitrates include audio. What you need to do is ensure you have a fixed key frame interval in your encoding. This will allow the segmenter to segment the files at regular intervals. you don't insert anything anywhere.
Out of curiosity why not directly use ffmpeg to give you the output segmented files? It supports it.
Thanks for everybody's attention and suggestions. I finally figured it out. The reason why the bandwidth stayed the same for different bitrate is that my ffmpeg command missed couple settings. I ended up using the following command:ffmpeg -i inputVideo.m4v -f mpegts -acodec libfaac -ar 44100 -ab 64k -vcodec libx264 -b 350k -s 480x360 -r 29.97 -flags +loop -cmp +chroma -partitions +parti4x4+partp8x8+partb8x8 -subq 5 -trellis 1 -refs 1 -coder 0 -me_range 16 -keyint_min 25 -sc_threshold 40 -i_qfactor 0.71 -bt 200k -maxrate 350k -bufsize 350k -rc_eq 'blurCplx^(1-qComp)' -qcomp 0.6 -qmin 10 -qmax 51 -qdiff 4 -level 30 -aspect 4:3 -g 30 -async 2 output.ts
I put it here so that other people who have the same problem as me will have a reference.
It sounds like you may have uncovered a bug in variantplaylistcreator. I recommend to verify that the sub-streams really are the bitrate you expect, and if it's really putting the wrong value, to report it to apple.
It might have something to do with using multiple -iframe-url. I can't understand why it would be necessary to specify it more than once. Adaptive streaming won't work if the substreams have different I-frame positions -- at least all of the segment boundaries must be aligned.
If you need to fix the playlist up programmatically, I recommend to use ffprobe (from ffmpeg suite) to extract the bitrate of each substream, and replace the bandwidth number with the extracted value.

What is the fastest way to convert PostScript to GIF?

I am using the ImageMagick convert utility right now. I have a PostScript file that takes about 90 seconds to convert to GIF.
I am looking for a faster way to do this perferably by modifying the options to "convert".
When I say "fast", ideally a few seconds but I'll take any significant speed up. Something suitable for an interactive GUI.
I only need this in black and white or greyscale (specifically it is is an image of seismic data "wiggle traces" so B&W is fine.)
Other acceptable formats are BMP, GIF, JPEG, JPG, PCX, PGM, PNG, PNM, PPM, RAS, TGA, TIF, or TIFF.
Trying to stick with ImageMagick as that is already installed and trying to avoid selling my boss on anything new. Still happy to hear other suggestions.
My suggestion is: Use Ghostscript.
Since you have a working ImageMagick already installed, that means Ghostscript is also there: because ImageMagick cannot convert PDF or PostScript to raster images all by its own -- it has to call Ghostscript as its delegate to do this anyway.
Ghostscript can directly convert PDF/PostScript input to TIFF/TIF/TIFFg4, JPEG, PBM, PCX, PNG, PNM, PPM, BMP raster image output.
The advantages are: you don't need to have ImageMagick involved. So it's faster and also gives you more direct control over the conversion parameters. If you run Ghostscript via ImageMagick that's a level of indirection which isn't always required. (Sometimes it may be required to add some fine-tuning and post-processing manipulations to the raster image data that Ghostscript generated -- but that doesn't seem to be the case for you.)
The only disadvantage is: Ghostscript cannot produce GIF. If you required GIF (which you don't seem to), you need ImageMagick for post-processing the raster output of Ghostscript to GIF.
You can see how ImageMagick calls Ghostscript (and which parameters it uses for the call -- look for a printed line on stderr containing gs, gsx or gswin32c or gswin64c) by running for example:
convert -verbose some.pdf[0] some.gif
Update
I did run a very, very un-scientific 'benchmark', running the following two commands 100 time each, which convert the randomly picked page 333 of the official PDF specification (ISO version for PDF-1.7) to GIF, measuring the time consumed. I run these commands in concurrently parallel, so both should have had to deal with the same overall system load, making the results better comparable:
'Comfortably' using ImageMagick's convert to directly produce GIF:
time for i in $(seq -w 1 100); do
convert \
PDF32000_2008.pdf[333] \
p333-im-no_${i}.gif ;
done
Using Ghostscript to create from the same page grayscale PNGs, piping Ghostscript's output to ImageMagick's convert in order to get GIFs:
time for i in $(seq -w 1 100); do
gs \
-q \
-o - \
-dFirstPage=333 \
-dLastPage=333 \
-sDEVICE=pnggray \
PDF32000_2008.pdf \
| \
convert \
- \
p333-gs-no_${i}.gif ;
done
Timing esults for the first command (running the 'comfortable' convert to achieve the PDF->GIF transformation, which uses Ghostscript only 'behind our backs'):
real 2m29.282s
user 2m22.526s
sys 0m5.647s
Timing results for the second command (running gs directly + openly, piping it's output to convert:
real 1m27.370s
user 1m23.447s
sys 0m3.435s
One more thing:
The total size of the 100 'Ghostscript'-GIFs was 1,6 MByte -- but they were 8-bit grayscale.
The total size of the 100 'ImageMagic-direct'-GIFs was 1,2 MByte -- but they were 2-bit black+white.
I don't have the motivation currently to tweak the test commandline parameters more for even closer comparability of the resulting files.
This result (149 seconds vs. 87 seconds) gives me enough confidence into my guess that you can gain significant performance improvements when you follow my recommendation. :-)
I am using the ImageMagick convert utility right now. I have a
PostScript file that takes about 90 seconds to convert to GIF.
I am looking for a faster way to do this perferably by modifying the
options to "convert".
When I say "fast", ideally a few seconds but I'll take any significant
speed up. Something suitable for an interactive GUI.
I only need this in black and white or greyscale (specifically it is
is an image of seismic data "wiggle traces" so B&W is fine.)
You can start with GhostScript:
gs -dSAFER -dBATCH -dNOPAUSE \
-sDEVICE=pnggray -r300 -sOutputFile=seismic.png seismic.pdf
A very longer but interesting way would be to analyze exactly what is in those PDFs.
I had to do something similar with the PDF output of an EKG workflow. The original data were unavailable, we only had the PDF, but I discovered that the PDF was vector based and not raster. After a little hacking it was very easy to decode the labels, the legend and the single elementary lines making up the EKG diagram, and I came up with an option to recolor the tracks starting from what appeared a grayscale image. It did take several days, though.
It is possible that your PDF is generated in a similar way, and the data could be decoded (at first I had to use pdftk to get me a non-compressed PDF, then I found a library that I could use - it implemented the Deflate algorithm). It would be really cool to have output in SVG format :-)

Is it possible to retrieve frames as images out of FMS live stream?

Has anyone tried this ?
What's the best practice for this?
FMS live streams are using the RTMP protocol:
ffmpeg -i rtmp://***server/path* **-acodec copy -vcodec copy -y *captured***.flv**
Here, we are saving the whole stream to an FLV file, which is Flash's static movie file format and so can always preserve all RTMP audio and video codecs without conversion.
You can then extract any frames you want, e.g.
ffmpeg -i *captured***.flv -s** starttime -vframes 1 -f image2 -vcodec mjpeg *captured***.jpg**
If you are ambitious and know exactly what time offsets and intervals you want to capture in advance, you can do both steps at once, e.g. one frame every second:
ffmpeg -i rtmp://***server/path* **-r 1 -f image2 -vcodec mjpeg *captured***%d.jpg**
All commandlines have not been tested, will need fixing but give you a good impression

Using Delphi or FFMpeg to create a movie from image sequence

My Delphi app has created a squence called frame_001.png to frame_100.png.
I need that to be compiled into a movie clip. I think perhaps the easiest is to call ffmpeg from the command line, according to their documentation:
For creating a video from many images:
ffmpeg -f image2 -i foo-%03d.jpeg -r 12 -s WxH foo.avi
The syntax foo-%03d.jpeg specifies to use a decimal number composed of three digits padded with zeroes to express the sequence number. It is the same syntax supported by the C printf function, but only formats accepting a normal integer are suitable.
From: http://ffmpeg.org/ffmpeg-doc.html#SEC5
However my files are (lossless) png format, so I have to convert using imagemagick first.
My command line is now:
ffmpeg.exe -f image2 -i c:\temp\wentelreader\frame_%05d.jpg -r 12 foo.avi
But then I get the error:
[image2 # 0x133a7d0]Could not find codec parameters (Video: mjpeg)
c:\temp\wentelreader\Frame_C:\VID2EVA\Tools\Mencoder\wentel.bat5d.jpg: could not
find codec parameters
What am I doing wrong?
Alternatively can this be done easily with Delphi?
Not sure if you are interested but there are delphi headers for this # http://ultrastardx.svn.sourceforge.net/viewvc/ultrastardx/trunk/src/lib/ffmpeg/
So you can use the DLL vs command line.
-Brad
Look at the file name in the error message. That can't possibly be right. The percent sign needs to get all the way to the program you're running, but it's being expanded by the batch file instead, where %0 expands to the full name and path of the file. Double the percent sign in the batch file:
ffmpeg.exe -f image2 -i c:\temp\wentelreader\frame_%%05d.jpg -r 12 foo.avi
Also, why do you want five digits when you've already said your files are named like frame_001.png, which has only three digits?
ffmpeg can create a movie from png images, why do you think you have to convert them to jpeg?
Guys in DelphiFFMpeg have been produced a component wrapper for FFMpeg. It's very expensive but it's worth to test it. However what you want to do is very simple and command-line is more than enough for you.

Resources