OpenCV MultiBandBlender doesn't work - opencv

I try to blend my images into pano with MultiBandBlender, but it return black pano. But FeatherBlender works fine. What I doing wrong?
blendImages(const std::vector<cv::Point> &corners, std::vector<cv::Mat> images)
{
std::vector<cv::Size> sizes;
for(int i = 0; i < images.size(); i++)
sizes.push_back(images[i].size());
float blend_strength = 5;
cv::Size dst_sz = cv::detail::resultRoi(corners, sizes).size();
float blend_width = sqrt(static_cast<float>(dst_sz.area())) * blend_strength / 100.f;
cv::Ptr<cv::detail::Blender> blender = cv::detail::Blender::createDefault(cv::detail::Blender::MULTI_BAND);
//cv::detail::FeatherBlender* fb = dynamic_cast<cv::detail::FeatherBlender*>(blender.get());
//fb->setSharpness(1.f/blend_width);
cv::detail::MultiBandBlender* mb = dynamic_cast<cv::detail::MultiBandBlender*>(blender.get());
mb->setNumBands(static_cast<int>(ceil(log(blend_width)/log(2.)) - 1.));
blender->prepare(corners, sizes);
for(int i = 0; i < images.size(); i++)
{
cv::Mat image_s;
images[i].convertTo(image_s, CV_16SC3);
blender->feed(image_s, cv::Mat::ones(image_s.size(), CV_8UC1), corners[i]);
}
cv::Mat pano;
cv::Mat panoMask = cv::Mat::ones(dst_sz, CV_8UC1);
blender->blend(pano, panoMask);
return pano;
}

Three possible causes:
Try keeping all image_s and masks in a vector, and feed with the following structure:
for (int i = 0; i < images_s.size(); ++i)
blender->feed(images_s[i], masks[i], corners[i]);
Don't initialize panoMask to ones before blending.
Make sure corners are well defined

Actually, I can't compile your code with OpenCV 2.4, because of blender.get function. There is no such a function in my build of OpenCV 2.4.
Anyway, if you wish to make a panorama, you'd better not use resultRoi function. You need boundingRect. I suppose, it is really hard to get all horizontally aligned images for one panorama.
Also, look at my answer here. It demonstrates how to use MultiBandBlender.

Hey I was getting the same black pano while using MultiBand blender in opencv. Actually the issue was resolved by changing
cv::Mat::ones(image_s.size(), CV_8UC1)
to
cv::Mat::ones(image_s.size(), CV_8UC1)*255
This is because Mat::ones initialize all the pixels to a value of numerical 1, Thus, we need to muliply it with 255 in order to get a pure black & white mask.
And, thanks, your issue solved my problem :)

Related

Matchingproblems when using OpenCVs matchShapes function

I´m trying to find a objekt in a larger Picture with the findContour/matchShape functions (the object can vary so it´s not possible to look after the color or something similar, Featuredetectors like SIFT also doesn´t work because the object could be symetric)
I have written following code:
Mat scene = imread...
Mat Template = imread...
Mat imagegray1, imagegray2, imageresult1, imageresult2;
int thresh=80;
double ans=0, result=0;
// Preprocess pictures
cvtColor(scene, imagegray1,CV_BGR2GRAY);
cvtColor(Template,imagegray2,CV_BGR2GRAY);
GaussianBlur(imagegray1,imagegray1, Size(5,5),2);
GaussianBlur(imagegray2,imagegray2, Size(5,5),2);
Canny(imagegray1, imageresult1,thresh, thresh*2);
Canny(imagegray2, imageresult2,thresh, thresh*2);
vector<vector <Point> > contours1;
vector<vector <Point> > contours2;
vector<Vec4i>hierarchy1, hierarchy2;
// Template
findContours(imageresult2,contours2,hierarchy2,CV_RETR_EXTERNAL,CV_CHAIN_APPROX_SIMPLE,cvPoint(0,0));
// Szene
findContours(imageresult1,contours1,hierarchy1,CV_RETR_EXTERNAL,CV_CHAIN_APPROX_SIMPLE,cvPoint(0,0));
imshow("template", Template);
double helper = INT_MAX;
int idx_i = 0, idx_j = 0;
// Match all contours with eachother
for(int i = 0; i < contours1.size(); i++)
{
for(int j = 0; j < contours2.size(); j++)
{
ans=matchShapes(contours1[i],contours2[j],CV_CONTOURS_MATCH_I1 ,0);
// find the best matching contour
if((ans < helper) )
{
idx_i = i;
helper = ans;
}
}
}
// draw the best contour
drawContours(scene, contours1, idx_i,
Scalar(255,255,0),3,8,hierarchy1,0,Point());
When I'm using a scene where only the Template is located in, i get a good matching result:
But when there are more objects in the pictures i have trouble detecting the object:
Hope someone can tell me whats the problem with the code i´m using. Thanks
You have a huge amount of contours in the second image (almost each letter).
As the matchShape checks for scale-invariant Hu-moments (http://docs.opencv.org/3.1.0/d3/dc0/group__imgproc__shape.html#gab001db45c1f1af6cbdbe64df04c4e944) also a very small contours may fit the shape you are looking for.
Furthermore, the original shape is not distinguished properly like can be seen when excluding all contours with an area smaller 50.
if(contourArea(contours1[i]) > 50)
drawContours(scene, contours1, i, Scalar(255, 255, 0), 1);
To say it with other words, there is no problem with your code. The contour can simply not be detected very well. I would suggest to have a look at approxCurve and convexHull and try to close the contour this way. Or improve the use of Canny in some way.
Then you could use a priori knowledge to restrict the size (and maybe rotation?) of the contour you are looking for.

Segmentation of perspectively distorted barcodes

There are images with perspectively distorted barcodes in them.
They are located and decoded using ZBar.
Now I do not only need the rough location, but the four real corner points of the barcode, that define the enclosing 4-point polygon.
I tried different approaches, but did not yet get the desired result.
One of them was:
convert image to grayscale
threshold image
erode image
floodFill beginning with a pixel known to be part of barcode
obtain the contour around the floodFill result
But around this contour I now would need to find the minimum best fitting 4-point polygon, which seems to be not that easy.
Do you have ideas for better approaches?
You could use the following code and try to reduce your contour to 4-point polygon via approxPoly
vector approx;
for (size_t i = 0; i < contours.size(); i++)
{
approxPolyDP(Mat(contours[i]), approx,
arcLength(Mat(contours[i]), true)*0.02, true);
if (approx.size() == 4 &&
fabs(contourArea(Mat(approx))) > 1000 &&
isContourConvex(Mat(approx)))
{
double maxCosine = 0;
for( int j = 2; j < 5; j++ )
{
double cosine = fabs(angle(approx[j%4], approx[j-2], approx[j-1]));
maxCosine = MAX(maxCosine, cosine);
}
if( maxCosine < 0.3 )
squares.push_back(approx);
}
}
http://opencv-code.com/tutorials/detecting-simple-shapes-in-an-image/
You can also try the following methods, maybe they will produce good enough results for you:
http://docs.opencv.org/modules/imgproc/doc/structural_analysis_and_shape_descriptors.html?highlight=minarearect#minarearect
or
http://docs.opencv.org/modules/imgproc/doc/structural_analysis_and_shape_descriptors.html?highlight=convexhull#convexhull
OK, I found a solution that works good enough for my use case.
First a scanline is generated from the ZBar result.
Now the first and the last black pixels are found in verion of the image resulting from cv::adaptivethreshold with a large enough blockSize.
From there on the first and the last bar are segmented using cv::findContours.
Now for both end bars the two contour points with the most distance to each others are searched.
They finally define the enclosing 4-point-polygon.
Which is not exactly what I posted in my question, but the additional size due to the elongated guard patterns does not matter in my case.

Template Matching on various sizes

Right now I am working on an OCR algorithm with Template Matching, using the opencv library. I am comparing pixel by pixel, and till now I have obtained good results. The problem comes when the area I want to match is of different size.
Ex: Template size = 70x100 while ROI = 140x200.
Is there any function that I can use in order adapt the required size and end up with the same amount of rows and columns?
Thanks
Robert Grech
Usually one makes an image scale pyramid and then only scans with the 70x100 windows across all scales i.e. as in opencv HOGDescriptor:
double scale = 1.;
double scale0 = 1.05;
int maxLevels = 64;
int nLevels;
Size templateSize(70,100);
cv::Mat testImage = cv::imread("test1.jpg");
vector<double> levelScale;
for( nLevels = 0; nLevels < maxLevels; nLevels++ )
{
levelScale.push_back(scale);
if( cvRound(testImage.cols/scale) < templateSize.width ||
cvRound(testImage.rows/scale) < templateSize.height ||
scale0 <= 1 )
break;
scale *= scale0;
}
nLevels = std::max(nLevels, 1);
levelScale.resize(nLevels);
int level;
for(level =0; level<nLevels; level++)
{
cv::Mat testAtScale;
Size sz(cvRound(testImage.cols/levelScale[level]),
cvRound(testImage.rows/levelScale[level]));
resize(testImage,testAtScale,sz);
//result = match(template,testAtScale);
//cv::imshow("sclale",testAtScale);
//cv::waitKey();
}
you would then need to post-process your results back to the original scale, this is simple with a box, but if you have a heat map / response map / probability map, then re-sizing it back up maybe somewhat hacky.

Openframeworks Colour Tracking

I'm working with OpenCV within Openframeworks in order to track a certain colour. My question may be difficult if you are not familiar with the colour tracking code but I'll try to explain the best I can.
What the code does now is follow a certain colour with a red circle and I am working to create a line that does basically the same thing but each point will be stored so that a squiggly type of drawing application is created. Right now it's a straight line that you can pull.
I'll post more code if necessary. Any advice would really help. Thanks!
tespApp.cpp
void testApp::draw(){
ofSetColor(255,255,255);
//draw coloured cv image
rgb.draw(0,0);
contours.draw(0,480);
// draw line that follows the blobs
for (int i=0; i<contours.nBlobs; i++) {
ofSetColor(0);
ofLine( contours.blobs[i].pos.x, contours.blobs[i].pos.y, contours.blobs[i].lastpos.x, contours.blobs[i].lastpos.y );
}
}
ofxCvContourFinder.cpp
for( int i = 0; i < MIN(nConsidered, (int)cvSeqBlobs.size()); i++ ) {
blobs.push_back( ofxCvBlob() );
float area = cvContourArea( cvSeqBlobs[i], CV_WHOLE_SEQ, bFindHoles ); // oriented=true for holes
CvRect rect = cvBoundingRect( cvSeqBlobs[i], 0 );
cvMoments( cvSeqBlobs[i], myMoments );
blobs[i].area = bFindHoles ? fabs(area) : area; // only return positive areas
blobs[i].length = cvArcLength(cvSeqBlobs[i]);
blobs[i].boundingRect.x = rect.x;
blobs[i].boundingRect.y = rect.y;
blobs[i].boundingRect.width = rect.width;
blobs[i].boundingRect.height = rect.height;
blobs[i].centroid.x = (myMoments->m10 / myMoments->m00);
blobs[i].centroid.y = (myMoments->m01 / myMoments->m00);
blobs[i].pos.x =0;
blobs[i].pos.y =0;
blobs[i].lastpos.x = blobs[i].pos.x;
blobs[i].lastpos.y = blobs[i].pos.y;
blobs[i].pos.x =(myMoments->m10 / myMoments->m00);
blobs[i].pos.y = (myMoments ->m01 / myMoments->m00);
A complete example of what you are trying to achieve can be found in here:
https://github.com/kylemcdonald/ofxCv/tree/master/example-contours-color
It uses ofxCv, an add-on that let you integrate openframeworks with openCv in a clean way. Moreover you can use native OpenCV calls, without the need of any kind of wrapper. The internet is full of similar examples that uses pure OpenCV code and with this add ons you can use those snippets as it is, without any kind of problem and it gives some easy way to go from OpenCV data structures to Openframeworks ones and vice-versa. It is great!

Confusion analyzing blobs using contours

I want to analyze the blobs received using contours. However, I came across with a slight problems where is there any difference analyzing the blobs before and after using the following code?
for(unsigned int i = 0; i < rects3.size(); i++) {
Scalar color = Scalar(255,255,255);
drawContours( drawing3, contours3, i, color, CV_FILLED, 8);
}
before using the above, there are only some boundaries line and after using the code we can see the white blobs. As attached are the example of it.
You want to iterate through the possible blobs and then analyze it (area, perimeter, etc).
Your contours are in vector called rects3.
// iterating trough
for(unsigned int i = 0; i < rects3.size(); i++) {
// get the bounding box of one contour
Rect rect = boundingRect(rects3[i]);
//area
double area = contourArea(rects3[i]);
//perimiter
double perimiter = arcLength(rects3[i], true);
}
see http://docs.opencv.org/modules/imgproc/doc/structural_analysis_and_shape_descriptors.html

Resources