Embedding a video on another video using Opencv - opencv

Question

I wish to run another video in the window of the main video. Here is the attempted code for it :
#include <cv.h>
#include <highgui.h>
#include <iostream>
using namespace std;
void OverlayImage(IplImage* src, IplImage* overlay, CvScalar S, CvScalar D) {
CvPoint location;
//location.x = (0.5*(src->width))-50;
//location.y = src->height-110;
//cout << location.x << " " << location.y << endl;
location.x = 100;
location.y = 100;
for (int i = location.y; i < (location.y + overlay->height); i++) {
for (int j = location.x; j < (location.x + overlay->width); j++) {
CvScalar source = cvGet2D(src, i, j);
CvScalar over = cvGet2D(overlay, i-location.y, j-location.x);
CvScalar merged;
for(int i = 0; i < 4; i++)
merged.val[i] = (S.val[i] * source.val[i] + D.val[i] * over.val[i]);
cvSet2D(src, i + location.y, j + location.x, merged);
}
}
}
int main (int argc, char* argv[]) {
CvCapture* capture = NULL;
CvCapture* ad = NULL;
capture = cvCaptureFromAVI("Cricketc11.avi");
ad = cvCaptureFromAVI("Cricketc1.avi");
assert(ad);
assert(capture);
cvNamedWindow("Video", 0);
int fps = ( int )cvGetCaptureProperty( capture, CV_CAP_PROP_FPS );
int noOfFrames = ( int )cvGetCaptureProperty( capture, CV_CAP_PROP_FRAME_COUNT );
int height = ( int )cvGetCaptureProperty( capture, CV_CAP_PROP_FRAME_HEIGHT );
int width = ( int )cvGetCaptureProperty( capture, CV_CAP_PROP_FRAME_WIDTH );
cout << height << " " << width << endl;
int fpsad = ( int )cvGetCaptureProperty( ad, CV_CAP_PROP_FPS );
int noOfFramesad = ( int )cvGetCaptureProperty( ad, CV_CAP_PROP_FRAME_COUNT );
int heightad = ( int )cvGetCaptureProperty( ad, CV_CAP_PROP_FRAME_HEIGHT );
int widthad = ( int )cvGetCaptureProperty( ad, CV_CAP_PROP_FRAME_WIDTH );
IplImage* tempimg = NULL;
IplImage* tempad = NULL;
while(capture) {
tempimg = cvQueryFrame(capture);
assert(tempimg);
if (ad) {
tempad = cvQueryFrame(ad);
assert(tempad);
IplImage* newimg = cvCreateImage(cvSize(100,100), IPL_DEPTH_8U, tempad->nChannels);
cvResize(tempad, newimg, 1);
OverlayImage(tempimg, newimg, cvScalar(0,0,0,0), cvScalar(1,1,1,1));
}
else
cvReleaseCapture(&ad);
cvWaitKey(1000/fps);
cvShowImage("Video", tempimg);
}
cvReleaseCapture(&capture);
cvDestroyAllWindows();
return 0;
}
This code runs fine only when the input videos are the same. If the videos are of different lengths or fps, it gives an error after the embedded video finishes.
How to correct that ?

Answer 1

What happens
Each time you call cvQueryFrame(source) the inner frame counter of the source is incremented.
This is why your second movie should be as long (speaking in frames) as the main movie.
As a workaround, I would suggest you to use an ad movie that has a number of frames (length * fps) equal to an integer ratio of the master movie and use temporary image buffers to hold the data you need.
An ideal solution would be to first interpolate the shortest (in frames) movie to the size of the longest, then merge them as you do, but temporal upsampling can be challenging to implement if you're not willing to use nearest neighbour or linear interpolation.
If the ad vido is smaller
You can choose among several solutions:
detect that you have reached the end and stop sending an image
detect that you have reached the end and re-open the ad movie from the beginning
use a temporary image to always keep in memory the last valid frame from the ad movie and send this image if there is no new one
etc.