OpenCV - iterate over each blob in a binary image and use it as mask - opencv

Question

I have a binary image and a color image of the same size. I need to iterate each blob (white pixel blocks) of the binary image and use it as a mask and find the mean color of this blob region from the color image.
I have tried:
HierarchyIndex[] hierarchy;
Point[][] contours;
binaryImage.FindContours(out contours, out hierarchy, RetrievalModes.List, ContourApproximationModes.ApproxNone);
using (Mat mask = Mat.Zeros(matColor.Size(), MatType.CV_8UC1))
foreach (var bl in contours)
if (Cv2.ContourArea(bl) > 5)
{
mask.DrawContour(bl, Scalar.White, -1);
Rect rect = Cv2.BoundingRect(bl);
Scalar mean = Cv2.Mean(colorImage[rect], mask[rect]);
mask.DrawContour(bl, Scalar.Black, -1);
}
which works for the blobs not having holes. However in my case I have many blob regions having huge holes that affects the mean calculation.
I couldn't figure it out how to solve it using the hierarchy info; or with another approach.
(My code is for OpenCVSharp but answer in any other wrapper or language is wellcome.)
Edit: I've added an example image. The traffic signs part is the problem.
Actually I think I have solved this problem with this method:
using PLine = List<Point>;
using Shape = List<List<Point>>;
internal static IEnumerable<Tuple<PLine, Shape>> FindContoursWithHoles(this Mat mat)
{
Point[][] contours;
HierarchyIndex[] hierarchy;
mat.FindContours(out contours, out hierarchy, RetrievalModes.Tree, ContourApproximationModes.ApproxNone);
Dictionary<int, bool> dic = new Dictionary<int, bool>();
for (int i = 0; i < contours.Length; i++)
if (hierarchy[i].Parent < 0)
dic[i] = true;
bool ok = false;
while (!ok)
{
ok = true;
for (int i = 0; i < contours.Length; i++)
if (dic.ContainsKey(i))
{
bool isParent = dic[i];
var hi = hierarchy[i];
if (hi.Parent >= 0) dic[hi.Parent] = (!isParent);
if (hi.Child >= 0) dic[hi.Child] = (!isParent);
while (hi.Next >= 0)
{
dic[hi.Next] = isParent;
hi = hierarchy[hi.Next];
if (hi.Parent >= 0) dic[hi.Parent] = (!isParent);
if (hi.Child >= 0) dic[hi.Child] = (!isParent);
}
hi = hierarchy[i];
while (hi.Previous >= 0)
{
dic[hi.Previous] = isParent;
hi = hierarchy[hi.Previous];
if (hi.Parent >= 0) dic[hi.Parent] = (!isParent);
if (hi.Child >= 0) dic[hi.Child] = (!isParent);
}
}
else
ok = false;
}
foreach (int i in dic.Keys.Where(a => dic[a]))
{
PLine pl = contours[i].ToList();
Shape childs = new Shape();
var hiParent = hierarchy[i];
if (hiParent.Child >= 0)
{
childs.Add(contours[hiParent.Child].ToList());
var hi = hierarchy[hiParent.Child];
while (hi.Next >= 0)
{
childs.Add(contours[hi.Next].ToList());
hi = hierarchy[hi.Next];
}
hi = hierarchy[hiParent.Child];
while (hi.Previous >= 0)
{
childs.Add(contours[hi.Previous].ToList());
hi = hierarchy[hi.Previous];
}
}
yield return Tuple.Create(pl, childs);
}
}
By drawing the holes as black, we can use each blob as a single mask:
var blobContours = blobs.FindContoursWithHoles().ToList();
using (Mat mask = Mat.Zeros(mat0.Size(), MatType.CV_8UC1))
for (int i = 0; i < blobContours.Count; i++)
{
var tu = blobContours[i];
var bl = tu.Item1;
if (Cv2.ContourArea(bl) > 100)
{
mask.DrawContour(bl, Scalar.White, -1);
foreach (var child in tu.Item2)
mask.DrawContour(child, Scalar.Black, -1);
Rect rect = Cv2.BoundingRect(bl);
Scalar mean = Cv2.Mean(mat0[rect], mask[rect]);
}
}
I think there should be an easier way.
And yet there is another problem. In some cases, an individual red part of the sign (which is a seperate white blob) does not found as a parent outside circle and a child inside circle, but a large parent contour outside with two circles as children (ie. hole inside another hole, makes a seperate blob which is not found as a parent). Yes it is hierarchically correct but does not help me. I hope I could make my self clear, sorry for my English.

Answer 1

#Miki thank you very much. I was able to achieve what I want using ConnectedComponents. Its simple and fast:
var cc = Cv2.ConnectedComponentsEx(binaryImage, PixelConnectivity.Connectivity8);
foreach (var bl in cc.Blobs)
using (Mat mask = new Mat())
{
cc.FilterByBlob(binaryImage, mask, bl);
Rect rect = bl.Rect;
Scalar mean = Cv2.Mean(colorImage[rect], mask[rect]);
}