opencv: Please help me to find out the grid of carton - opencv

Question

This question has been annoying me over 2 weeks.
My goal is to analyze a set of products stored in cartons on a shelf.
Right now, I have tried using the following methods from OpenCV Python module: findContours, canny,HoughLines,cv2.HoughLinesP, but I can't find the result grid.
My goal is to check if the products been filled up in carton.
Here is the original image: http://postimg.org/image/hyz1jpd7p/7a4dd87c/
My first step is to use closing transformation:
closing = cv2.morphologyEx(opening, cv2.MORPH_CLOSE, kernel, iterations=1)]
This gives me the contours (I have not enough reputation to post this url, this image is similar with the last image below, but without red lines!).
Finally, the question is, how could I find the carton grid (i.e., the products in it one by one).
I have added the red lines in the image below.
Please give me the hints, thank you very much!
Red lines: http://postimg.org/image/6i0di4gsx/

Answer 1

I've played a little bit with the input and found a way to extract basically the grid with HoughLinesP after thresholding the Hue channel.
edit: I'm using C++, but similar python methods should be available I guess.
cv::Mat image = cv::imread("box1.png");
cv::Mat output; image.copyTo(output);
cv::Mat hsv;
cv::cvtColor(image, hsv, CV_BGR2HSV);
std::vector<cv::Mat> hsv_channels;
cv::split(hsv, hsv_channels);
// thresholding here is a little sloppy, maybe you have to use some smarter way
cv::Mat h_thres = hsv_channels[0] < 50;
// unfortunately, HoughLinesP couldnt detect all the lines if they were too wide
// to make this part more robust I would suggest a ridge detection on the distance transformed image instead of 'some erodes after a dilate'
cv::dilate(h_thres, h_thres, cv::Mat());
cv::erode(h_thres, h_thres, cv::Mat());
cv::erode(h_thres, h_thres, cv::Mat());
cv::erode(h_thres, h_thres, cv::Mat());
std::vector<cv::Vec4i> lines;
cv::HoughLinesP( h_thres, lines, 1, CV_PI/(4*180.0), 50, image.cols/4, 10 );
for( size_t i = 0; i < lines.size(); i++ )
{
cv::line( output, cv::Point(lines[i][0], lines[i][1]),
cv::Point(lines[i][2], lines[i][3]), cv::Scalar(155,255,155), 1, 8 );
}
here are the images:
hue channel after hsv convert:
threshholded hue channel:
output:
maybe someone else has an idea how to improve the HoughLinesP without those erode steps...
Hope this method helps you a bit and you can improve it further to use it for your needs.