OpenCV floodfill with mask - opencv

The documentation for OpenCV's floodfill function states:
The function uses and updates the mask, so you take responsibility of
initializing the mask content. Flood-filling cannot go across non-zero
pixels in the mask. For example, an edge detector output can be used
as a mask to stop filling at edges. It is possible to use the same
mask in multiple calls to the function to make sure the filled area
does not overlap.
How does the function update the mask? Does it set all the pixels within the floodfill to some non-zero value?

All zero-valued pixels in the same connected component as the seed point of the mask are replaced by the value you specify. This value must be added to the flags parameter, left-shifted by 8 bits:
uchar fillValue = 128;
cv::floodFill(img, mask, seed, cv::Scalar(255) ,0, cv::Scalar(), cv::Scalar(), 4 | cv::FLOODFILL_MASK_ONLY | (fillValue << 8));
A simple, but perhaps enlightening example follows. Creating an image like so:
//Create simple input image
cv::Point seed(4,4);
cv::Mat img = cv::Mat::zeros(100,100,CV_8UC1);
cv::circle(img, seed, 20, cv::Scalar(128),3);
Results in this image:
Then, creating a mask and flood-filling it:
//Create a mask from edges in the original image
cv::Mat mask;
cv::Canny(img, mask, 100, 200);
cv::copyMakeBorder(mask, mask, 1, 1, 1, 1, cv::BORDER_REPLICATE);
//Fill mask with value 128
uchar fillValue = 128;
cv::floodFill(img, mask, seed, cv::Scalar(255) ,0, cv::Scalar(), cv::Scalar(), 4 | cv::FLOODFILL_MASK_ONLY | (fillValue << 8));
Gives this result:
The white pixels in the mask are the result of edge detection, while the grey pixels are the result of the flood-fill.
UPDATE:
In response to the comment, flag value 4 specifies the pixel neighborhood with which to compare the color value difference. From the documentation:
Lower bits contain a connectivity value, 4 (default) or 8, used within the function. Connectivity determines which neighbors of a pixel are considered.
When the cv::FLOODFILL_MASK_ONLY flag is not passed, both the image and the mask are updated, but the flood filling will stop at at any nonzero mask values.

And a python version
im = cv2.imread("seagull.jpg")
h,w,chn = im.shape
seed = (w/2,h/2)
mask = np.zeros((h+2,w+2),np.uint8)
floodflags = 4
floodflags |= cv2.FLOODFILL_MASK_ONLY
floodflags |= (255 << 8)
num,im,mask,rect = cv2.floodFill(im, mask, seed, (255,0,0), (10,)*3, (10,)*3, floodflags)
cv2.imwrite("seagull_flood.png", mask)
(Seagull image from Wikimedia: https://commons.wikimedia.org/wiki/Commons:Quality_images#/media/File:Gull_portrait_ca_usa.jpg)
Result:

Per Aurelius' answer, the mask needs to be zeroed.
Checking comment in the source, it stated that
Since this is both an input and output parameter, you must take responsibility
of initializing it. Flood-filling cannot go across non-zero pixels in the input mask.
The mask will impact the result so need to be zeroed before use:
cv::Mat mask;
mask = cv::Mat::zeros(img.rows + 2, img.cols + 2, CV_8UC1);

Related

Extracting transparent background of an image with opencv

I have got a mask calculated in grab_cut(which calculates the foreground). I want to extract only the background leaving the foreground transparent. I manage to do so using the following code in order to extract foreground(background transparent). How is it possible to do the opposite?
int border = 20;
int border2 = border + border;
cv::Rect rectangle(border,border,image.cols-border2,image.rows-border2);
cv::Mat result; // segmentation result (4 possible values)
cv::Mat bgModel,fgModel; /
cv::grabCut(image, // input image
result, // segmentation result
rectangle,// rectangle containing foreground
bgModel,fgModel, // models
1, // number of iterations
cv::GC_INIT_WITH_RECT); // use rectangle
cv::compare(result,cv::GC_PR_FGD,result,cv::CMP_EQ);
cv::Mat foreground(image.size(),CV_8UC3,cv::Scalar(255,255,255));
image.copyTo(foreground,result); // bg pixels not copied
cv::rectangle(image, rectangle, cv::Scalar(255,255,255),1);
cv::imwrite(argv[2], foreground);
cv::imwrite(argv[3], image);
Mat dst;//(src.rows,src.cols,CV_8UC4);
Mat tmp,alpha;
cvtColor(foreground,tmp,CV_BGR2GRAY);
threshold(tmp,alpha,100,255,THRESH_BINARY);
Mat rgb[3];
split(foreground,rgb);
Mat rgba[4]={rgb[0],rgb[1],rgb[2],alpha};
merge(rgba,4,dst);
imwrite("dst.png",dst);
Basically i think I ve got to change those lines:
cv::Mat foreground(image.size(),CV_8UC3,cv::Scalar(255,255,255));
image.copyTo(foreground,result); // bg pixels not copied
How is is possible to select the rest of the image the opposite of result?
Just invert your mask as in:
cv::Mat background(image.size(),CV_8UC3,cv::Scalar(255,255,255));
image.copyTo(background, ~result); // fg pixels not copied

Detect caps on bottles using opencv and python

I know that there are a hundred topics about my question in all over the web, but i would like to ask specific for my problem because I tried almost all solutions without any success.
I am trying to count circles in an image (yes i have already tried hough circles but due to light reflections, i think, on my object is not very robust).
Then I tried to create a classifier (no success i think there is no enough features so the detection is not good)
I have also tried HSV conversation and tried to find my object with color (again I had some problems because of the light and the variations of colors)
As you can see on image, there are 8 caps and i would like to be able to count them.
Using all of this methods, i was able to detect the objects on an image (because I was optimizing all the parameters of functions for the specific image) but as soon as I load a new, similar, image the results was disappointing.
Please follow this link to see the Image
Bellow you can find parts of everything i have tried:
1. Hough circles
img = cv2.imread('frame71.jpg',1)
img = cv2.medianBlur(img,5)
cimg = cv2.cvtColor(img,cv2.COLOR_GRAY2BGR)
if img == None:
print "There is no image file. Quiting..."
quit()
circles = cv2.HoughCircles(img,cv.CV_HOUGH_GRADIENT,3,50,
param1=55,param2=125,minRadius=25,maxRadius=45)
circles = np.uint16(np.around(circles))
for i in circles[0,:]:
# draw the outer circle
cv2.circle(cimg,(i[0],i[1]),i[2],(0,255,0),2)
# draw the center of the circle
cv2.circle(cimg,(i[0],i[1]),2,(0,0,255),3)
print len(circles[0,:])
cv2.imshow('detected circles',cimg)
cv2.waitKey(0)
cv2.destroyAllWindows()
2. HSV Transform, color detection
def image_process(frame, h_low, s_low, v_low, h_up, s_up, v_up, ksize):
temp = ksize
if(temp%2==1):
ksize = temp
else:
ksize = temp+1
#if(True):
# return frame
#thresh = frame
#try:
#TODO: optimize as much as possiblle this part of code
try:
hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)
lower = np.array([h_low, s_low, v_low],np.uint8)
upper = np.array([h_up,s_up,h_up],np.uint8)
mask = cv2.inRange(hsv, lower, upper)
res = cv2.bitwise_and(hsv,hsv, mask= mask)
thresh = cv2.cvtColor(res, cv2.COLOR_BGR2GRAY)
#thresh = cv2.threshold(res, 50, 255, cv2.THRESH_BINARY)[1]
thresh = cv2.threshold(thresh, 50, 255, cv2.THRESH_BINARY)[1]
thresh = cv2.medianBlur(thresh,ksize)
except Exception as inst:
print type(inst)
#cv2.imshow('thresh', thresh)
return thresh
3. Cascade classifier
img = cv2.imread('frame405.jpg', 1)
cap_cascade = cv2.CascadeClassifier('haar_30_17_16_stage.xml')
caps = cap_cascade.detectMultiScale(img, 1.3, 5)
#print caps
for (x,y,w,h) in caps:
cv2.rectangle(img, (x,y), (x+w,y+h), (255,0,0),2)
#cv2.rectangle(img, (10,10),(100,100),(0,255,255),4)
cv2.imshow('image', img)
cv2.waitKey(0)
cv2.destroyAllWindows()
quit()
About training the classifier I really used a lot of variations of images, samples, negatives and positives, number of stages, w and h but the results was not very accurate.
Finally I would like to know from your experience which is the best method I should follow and I will stick on that in order to optimize my detection. Keep in mind that all images are similiar but NOT identical. There are some differences due to light, movement etc
Than you in advance,
I did some experiment with the sample image. I'm posting my results, and if you find it useful, you can improve it further and optimize. Here are the steps:
downsample the image
perform morphological opening
find Hough circles
cluster the circles by radii (bottle circles should get the same label)
filter the circles by a radius threshold
you can also cluster circles by their center x and y coordinates (I haven't done this)
prepare a mask from the filtered circles and extract the possible bottles region
cluster this region by color
Code is in C++. I'm attaching my results.
Mat im = imread(INPUT_FOLDER_PATH + string("frame71.jpg"));
Mat small;
int kernelSize = 9; // try with different kernel sizes. 5 onwards gives good results
pyrDown(im, small); // downsample the image
Mat morph;
Mat kernel = getStructuringElement(MORPH_ELLIPSE, Size(kernelSize, kernelSize));
morphologyEx(small, morph, MORPH_OPEN, kernel); // open
Mat gray;
cvtColor(morph, gray, CV_BGR2GRAY);
vector<Vec3f> circles;
HoughCircles(gray, circles, CV_HOUGH_GRADIENT, 2, gray.rows/8.0); // find circles
// -------------------------------------------------------
// cluster the circles by radii. similarly you can cluster them by center x and y for further filtering
Mat circ = Mat(circles);
Mat data[3];
split(circ, data);
Mat labels, centers;
kmeans(data[2], 2, labels, TermCriteria(CV_TERMCRIT_EPS+CV_TERMCRIT_ITER, 10, 1.0), 2, KMEANS_PP_CENTERS, centers);
// -------------------------------------------------------
Mat rgb;
small.copyTo(rgb);
//cvtColor(gray, rgb, CV_GRAY2BGR);
Mat mask = Mat::zeros(Size(gray.cols, gray.rows), CV_8U);
for(size_t i = 0; i < circles.size(); i++)
{
Point center(cvRound(circles[i][0]), cvRound(circles[i][1]));
int radius = cvRound(circles[i][2]);
float r = centers.at<float>(labels.at<int>(i));
if (r > 30.0f && r < 45.0f) // filter circles by radius (values are based on the sample image)
{
// just for display
circle(rgb, center, 3, Scalar(0,255,0), -1, 8, 0);
circle(rgb, center, radius, Scalar(0,0,255), 3, 8, 0);
// prepare a mask
circle(mask, center, radius, Scalar(255,255,255), -1, 8, 0);
}
}
// use each filtered circle as a mask and extract the region from original downsampled image
Mat rgb2;
small.copyTo(rgb2, mask);
// cluster the masked region by color
Mat rgb32fc3, lbl;
rgb2.convertTo(rgb32fc3, CV_32FC3);
int imsize[] = {rgb32fc3.rows, rgb32fc3.cols};
Mat color = rgb32fc3.reshape(1, rgb32fc3.rows*rgb32fc3.cols);
kmeans(color, 4, lbl, TermCriteria(CV_TERMCRIT_EPS+CV_TERMCRIT_ITER, 10, 1.0), 2, KMEANS_PP_CENTERS);
Mat lbl2d = lbl.reshape(1, 2, imsize);
Mat lbldisp;
lbl2d.convertTo(lbldisp, CV_8U, 50);
Mat lblColor;
applyColorMap(lbldisp, lblColor, COLORMAP_JET);
Results:
Filtered circles:
Masked:
Segmented:
Hello finally i think I found a way to count caps on bottles.
Read image
Teach (find correct values for HSV up/low limits)
Select desire color (using HSV and mask)
Find contours on the masked image
Find the minCircles for contours
Reject all circles beyond thresholds
I have also order a polarized filter which I think it will reduce glares a lot. I am open to suggestions for further improvement (robustness and speed). Both robustness and speed is crucial for my application.
Thank you.

OpenCV - Floodfill onto new Mat

Given a point on an image, I'd like to floodfill all points connected to that point - but onto a new image. A naive way to do this would be to floodfill the original image to a special magic colour value. Then, visit each pixel, and copy all pixels with this magic colour value to the new image. There must be a better way!
Why don't you use the second variant of cv::floodFill to create a mask?
int floodFill(InputOutputArray image, InputOutputArray mask, Point
seedPoint, Scalar newVal, Rect* rect=0, Scalar loDiff=Scalar(), Scalar
upDiff=Scalar(), int flags=4 )
Original image
cv::Mat img = cv::imread("squares.png");
First variant
cv::floodFill(img, cv::Point(150,150), cv::Scalar(255.0, 255.0, 255.0));
This is the img
Second variant
cv::Mat mask = cv::Mat::zeros(img.rows + 2, img.cols + 2, CV_8U);
cv::floodFill(img, mask, cv::Point(150,150), 255, 0, cv::Scalar(), cv::Scalar(),
4 + (255 << 8) + cv::FLOODFILL_MASK_ONLY);
This is the mask. img doesn't change
If you go with this though, note that:
Since the mask is larger than the filled image, a pixel (x,y) in image corresponds to the pixel (x+1, y+1) in the mask.

color a grayscale image with opencv

i'm using openNI for some project with kinect sensor. i'd like to color the users pixels given with the depth map. now i have pixels that goes from white to black, but i want from red to black. i've tried with alpha blending, but my result is only that i have pixels from pink to black because i add (with addWeight) red+white = pink.
this is my actual code:
layers = device.getDepth().clone();
cvtColor(layers, layers, CV_GRAY2BGR);
Mat red = Mat(240,320, CV_8UC3, Scalar(255,0,0));
Mat red_body; // = Mat::zeros(240,320, CV_8UC3);
red.copyTo(red_body, device.getUserMask());
addWeighted(red_body, 0.8, layers, 0.5, 0.0, layers);
where device.getDepth() returns a cv::Mat with depth map and device.getUserMask() returns a cv::Mat with user pixels (only white pixels)
some advice?
EDIT:
one more thing:
thanks to sammy answer i've done it. but actually i don't have values exactly from 0 to 255, but from (for example) 123-220.
i'm going to find minimum and maximum via a simple for loop (are there better way?), and how can i map my values from min-max to 0-255 ?
First, OpenCV's default color format is BGR not RGB. So, your code for creating the red image should be
Mat red = Mat(240,320, CV_8UC3, Scalar(0,0,255));
For red to black color map, you can use element wise multiplication instead of alpha blending
Mat out = red_body.mul(layers, 1.0/255);
You can find the min and max values of a matrix M using
double minVal, maxVal;
minMaxLoc(M, &minVal, &maxVal, 0, 0);
You can then subtract the minValue and scale with a factor
double factor = 255.0/(maxVal - minVal);
M = factor*(M -minValue)
Kinda clumsy and slow, but maybe split layers, copy red_body (make it a one channel Mat, not 3) to the red channel, merge them back into layers?
Get the same effect, but much faster (in place) with reshape:
layers = device.getDepth().clone();
cvtColor(layers, layers, CV_GRAY2BGR);
Mat red = Mat(240,320, CV_8UC1, Scalar(255)); // One channel
Mat red_body;
red.copyTo(red_body, device.getUserMask());
Mat flatLayer = layers.reshape(1,240*320); // presumed dimensions of layer
red_body.reshape(0,240*320).copyTo(flatLayer.col(0));
// layers now has the red from red_body

Thresholding for a colour in opencv

I am trying to set up my programme to threshold for a colour (in BGR format). I have not fully decided which colour I will be looking for yet. I would also like the program to record how many pixels it has detected of that colour. My code so far is below but it is not working.
#include "cv.h"
#include "highgui.h"
int main()
{
// Initialize capturing live feed from the camera
CvCapture* capture = 0;
capture = cvCaptureFromCAM(0);
// Couldn't get a device? Throw an error and quit
if(!capture)
{
printf("Could not initialize capturing...\n");
return -1;
}
// The two windows we'll be using
cvNamedWindow("video");
cvNamedWindow("thresh");
// An infinite loop
while(true)
{
// Will hold a frame captured from the camera
IplImage* frame = 0;
frame = cvQueryFrame(capture);
// If we couldn't grab a frame... quit
if(!frame)
break;
//create image where threshloded image will be stored
IplImage* imgThreshed = cvCreateImage(cvGetSize(frame), 8, 1);
//i want to keep it BGR format. Im not sure what colour i will be looking for yet. this can be easily changed
cvInRangeS(frame, cvScalar(20, 100, 100), cvScalar(30, 255, 255), imgThreshed);
//show the original feed and thresholded feed
cvShowImage("thresh", imgThreshed);
cvShowImage("video", frame);
// Wait for a keypress
int c = cvWaitKey(10);
if(c!=-1)
{
// If pressed, break out of the loop
break;
}
cvReleaseImage(&imgThreshed);
}
cvReleaseCapture(&capture);
return 0;
}
To threshold for a color,
1) convert the image to HSV
2) Then apply cvInrangeS
3) Once you got threshold image, you can count number of white pixels in it.
Try this tutorial to track yellow color: Tracking colored objects in OpenCV
I can tell how to do it in both Python and C++ and both with and without converting to HSV.
C++ Version (Converting to HSV)
Convert the image into an HSV image:
// Convert the image into an HSV image
IplImage* imgHSV = cvCreateImage(cvGetSize(img), 8, 3);
cvCvtColor(img, imgHSV, CV_BGR2HSV);
Create a new image that will hold the threholded image:
IplImage* imgThreshed = cvCreateImage(cvGetSize(img), 8, 1);
Do the actual thresholding using cvInRangeS
cvInRangeS(imgHSV, cvScalar(20, 100, 100), cvScalar(30, 255, 255), imgThreshed);
Here, imgHSV is the reference image. And the two cvScalars represent the lower and upper bound of values that are yellowish in colour. (These bounds should work in almost all conditions. If they don't, try experimenting with the last two values).
Consider any pixel. If all three values of that pixel (H, S and V, in that order) lie within the stated ranges, imgThreshed gets a value of 255 at that corresponding pixel. This is repeated for all pixels. So what you finally get is a thresholded image.
Use countNonZero to count the number of white pixels in the thresholded image.
Python Version (Without converting to HSV):
Create the lower and upper boundaries of the range you are interested in, in Numpy array format (Note: You need to use import numpy as np)
lower = np.array((a,b,c), dtype = "uint8")
upper = np.array((x,y,z), dtype = "uint8")
In the above (a,b,c) is the lower bound and (x,y,z) is the upper bound.
2.Get the mask for the pixels that satisfy the range:
mask = cv2.inRange(image, lower, upper)
In the above, image is the image on which you want to work.
Count the number of white pixels that are present in the mask using countNonZero:
yellowpixels = cv2.countNonZero(mask)
print "Number of Yellow pixels are %d" % (yellowpixels)
Sources:
http://srikanthvidyasagar.blogspot.com/2016/01/tracking-colored-objects-in-opencv.html
http://www.pyimagesearch.com/2014/08/04/opencv-python-color-detection/
count number of black pixels in an image in Python with OpenCV

Resources