OpenCV background subtraction from a still image - opencv

I'm working on an application that will work with an inside mounted camera on the ceiling. The purpose is for it to keep track of objects on a surface.
I need to remove the background, so that I can get the contours of the "diff" that's there, but using BackgroundSubtractorMOG gets frustrating, as I find that its only application is for video.
What I need is to provide a single image that will be the background, and then calculate on each frame from a stream what has changed.
Here's what I have:
#include <libfreenect/libfreenect_sync.h>
#include <opencv2/opencv.hpp>
#include <iostream>
#include <vector>
const char *kBackgroundWindow = "Background";
const char *kForegroundWindow = "Foreground";
const char *kDiffWindow = "Diff";
const cv::Size kCameraSize(cv::Size(640, 480));
int main(int argc, char **argv) {
uint8_t *raw_frame = (uint8_t *)malloc(640 * 480 * 3);
uint32_t timestamp;
// First, we show the background window. A key press will set the background
// and move on to object detection.
cvNamedWindow(kBackgroundWindow);
cv::Mat background(kCameraSize, CV_8UC3, cv::Scalar(0));
for(;;) {
freenect_sync_get_video((void **)&raw_frame, &timestamp, 0, FREENECT_VIDEO_RGB);
background.data = raw_frame;
cv::cvtColor(background, background, CV_BGR2RGB);
cv::imshow(kBackgroundWindow, background);
if(cv::waitKey(10) > 0)
break;
}
// Create two windows, one to show the current feed and one to show the difference between
// background and feed.
cvNamedWindow(kForegroundWindow);
// Canny threshold values for the track bars
int cannyThresh1 = 20;
int cannyThresh2 = 50;
cvNamedWindow(kDiffWindow);
cv::createTrackbar("Canny Thresh 1", kDiffWindow, &cannyThresh1, 5000, NULL);
cv::createTrackbar("Canny THresh 2", kDiffWindow, &cannyThresh2, 5000, NULL);
// Start capturing frames.
cv::Mat foreground(kCameraSize, CV_8UC3, cv::Scalar(0));
cv::Mat diff(kCameraSize, CV_8UC3, cv::Scalar(0));
cv::BackgroundSubtractorMOG2 bg_subtractor(101, 100.0, false);
bg_subtractor(background, diff, 1);
for(;;) {
freenect_sync_get_video((void **)&raw_frame, &timestamp, 0, FREENECT_VIDEO_RGB);
foreground.data = raw_frame;
cv::cvtColor(foreground, foreground, CV_BGR2RGB);
// Calculate the difference between the background
// and the foreground into diff.
bg_subtractor(foreground, diff, 0.01);
// Run the Canny edge detector in the resulting diff
cv::Canny(diff, diff, cannyThresh1, cannyThresh2);
cv::imshow(kForegroundWindow, foreground);
cv::imshow(kDiffWindow, diff);
cv::waitKey(10);
}
}
How can I change this so that it doesn't "learn" about the new background, but just uses the static image stored in background?
Thanks!

If you truly only want a static image as the background, you can simply subtract the background image from the foreground image:
cv::Mat diff;
cv::absdiff(foreground, background, diff);
As a side note, I think your calls to cv::cvtColor() are unnecessary. OpenCV's native image format is BGR, so imshow() will show the red and blue channels swapped if you convert to RGB beforehand.

Related

Estimate white background

I have image with white uneven background (due to lighting). I'm trying to estimate background color and transform image into image with true white background. For this I estimated white color for each 15x15 pixels block based on its luminosity. So I've got the following map (on the right):
Now I want to interpolate color so it will be more smooth transition from 15x15 block to neighboring block, plus I want it to eliminate outliers (pink dots on left hand side). Could anyone suggest good technique/algorithm for this? (Ideally within OpenCV library, but not necessary)
Starting from this image:
You could find the text on the whiteboard as the parts of your images that have a high gradient, and apply a little dilation to deal with thick parts of the text. You'll get a mask that separates background from foreground pretty well:
Background:
Foreground:
You can then apply inpainting using the computed mask on the original image (you need OpenCV contrib module photo):
Just to show that this works independently of the text color, I tried on a different image:
Resulting in:
Code:
#include <opencv2/opencv.hpp>
#include <opencv2/photo.hpp>
using namespace cv;
void findText(const Mat3b& src, Mat1b& mask)
{
// Convert to grayscale
Mat1b gray;
cvtColor(src, gray, COLOR_BGR2GRAY);
// Compute gradient magnitude
Mat1f dx, dy, mag;
Sobel(gray, dx, CV_32F, 1, 0);
Sobel(gray, dy, CV_32F, 0, 1);
magnitude(dx, dy, mag);
// Remove low magnitude, keep only text
mask = mag > 10;
// Apply a dilation to deal with thick text
Mat1b K = getStructuringElement(MORPH_ELLIPSE, Size(3, 3));
dilate(mask, mask, K);
}
int main(int argc, const char * argv[])
{
Mat3b img = imread("path_to_image");
// Segment white
Mat1b mask;
findText(img, mask);
// Show intermediate images
Mat3b background = img.clone();
background.setTo(0, mask);
Mat3b foreground = img.clone();
foreground.setTo(0, ~mask);
// Apply inpainting
Mat3b inpainted;
inpaint(img, mask, inpainted, 21, CV_INPAINT_TELEA);
imshow("Original", img);
imshow("Foreground", foreground);
imshow("Background", background);
imshow("Inpainted", inpainted);
waitKey();
return 0;
}

Inpainting depth map, still a black image border

I'm trying to inpaint missing depth values of a depth map using the method described here. To summarize the method:
Downsize depth map to 20% of the original size
Inpaint all black (unknown) pixels in the downsized image
Upsize to original size
Replace all black pixels in the original image with corresponding values from the upsized image
Super simple and everything works well. A video showing the results can be found here.
However, I wonder why the left and top image border are still black although they should be inpainted (can be seen in the video). My first thought was that this could have to do something with the border interpolation (black pixels outside the image boundary), but than I would expect this also to happen on the other image borders. My second thought was that it is something specific to the used inpainting method (method by Alexandru Telea), but changing it to the Navier-Stokes based method didn't change the results.
Can somebody explain to me why this happens and how to tell OpenCV to also inpaint these regions, if possible?
Thanks in advance.
After asked by #theodore in http://answers.opencv.org/question/86569/inpainting-depth-map-still-black-image-borders/?comment=86587#comment-86587 I've used the sample images to test the inpaint behavious. It looks like it does not handle the border correctly, so creating a border with cv::copyMakeBorder can be used.
Here's the extended version with some kind of unit testing:
int main(int argc, char* argv[])
{
cv::Mat input = cv::imread("C:/StackOverflow/Input/depthInpaint.png");
cv::Mat img;
cv::cvtColor(input, img, CV_BGR2GRAY);
cv::Mat inpainted;
const unsigned char noDepth = 0; // change to 255, if values no depth uses max value or use the mask image
//cv::inpaint(img, (img == noDepth), depth, 5.0, cv::INPAINT_TELEA); // img is the 8-bit input image (depth map with blank spots)
double inpaintRadius = 5;
int makeBorder = 1;
cv::Mat borderimg;
cv::copyMakeBorder(img, borderimg, makeBorder, makeBorder, makeBorder, makeBorder, cv::BORDER_REPLICATE);
cv::imshow("border", borderimg);
cv::inpaint(borderimg, (borderimg == noDepth), inpainted, inpaintRadius, cv::INPAINT_TELEA); // img is the 8-bit input image (depth map with blank spots)
cv::Mat originalEmbedded = borderimg(cv::Rect(makeBorder, makeBorder, img.cols, img.rows));
cv::Mat inpaintedEmbedded = inpainted(cv::Rect(makeBorder, makeBorder, img.cols, img.rows));
cv::Mat diffImage;
cv::absdiff(img, originalEmbedded, diffImage);
cv::imshow("embedding correct?", diffImage > 0);
cv::Mat mask = img == noDepth;
cv::imshow("mask", mask);
cv::imshow("input", input);
cv::imshow("inpainted", inpainted);
cv::imshow("inpainted from border", inpaintedEmbedded);
cv::waitKey(0);
return 0;
}
Here's the reduced version if you believe it to be correct:
int main(int argc, char* argv[])
{
cv::Mat input = cv::imread("C:/StackOverflow/Input/depthInpaint.png");
cv::Mat img;
cv::cvtColor(input, img, CV_BGR2GRAY);
cv::Mat inpainted;
const unsigned char noDepth = 0; // change to 255, if values no depth uses max value or use the mask image
//cv::inpaint(img, (img == noDepth), depth, 5.0, cv::INPAINT_TELEA); // img is the 8-bit input image (depth map with blank spots)
double inpaintRadius = 5;
int makeBorderSize = 1;
cv::Mat borderimg;
//cv::copyMakeBorder(img, borderimg, borderSize, borderSize, borderSize, borderSize, cv::BORDER_REPLICATE);
cv::copyMakeBorder(img, borderimg, makeBorderSize, makeBorderSize, makeBorderSize, makeBorderSize, cv::BORDER_REPLICATE);
//cv::imshow("border", borderimg);
cv::inpaint(borderimg, (borderimg == noDepth), inpainted, inpaintRadius, cv::INPAINT_TELEA); // img is the 8-bit input image (depth map with blank spots)
// extract the original area without border:
cv::Mat inpaintedEmbedded = inpainted(cv::Rect(makeBorderSize, makeBorderSize, img.cols, img.rows));
cv::imshow("input", input);
cv::imshow("inpainted from border", inpaintedEmbedded);
cv::waitKey(0);
return 0;
}
Here's Input:
Here's the input with border (bordersize 5 to visualize the effect better):
Here's the output:

How to black out everything outside a circle in Open CV

I am currently trying to black out everything outside a circle.
I am drawing the circle using the following lines of code:
cv::Point center(cvRound(circles[i][0]), cvRound(circles[i][1])); // CVRound converts floating numbers to integer
int radius = cvRound(circles[i][2]); // Radius is the third parameter [i][0] = x [i][1]= y [i][2] = radius
circle( image, center, 3, cv::Scalar(0,255,0), -1, 8, 0 ); // Drawing little circle to Image Center , next Line of Code draws the real circle
circle( image, center, radius, cv::Scalar(0,0,255), 3, 8, 0 ); // Circle(img, center, radius, color, thickness=1, lineType=8, shift=0)
What is the best approach of painting everything of the circly black, if I have a radius and the center of my circle?
Does OpenCV provide an easy mechanism of doing this or should I iterate through all the pixels of my image and depending on the position color them black or not?
Thanks to Abid for the hint, i ended up with this approach. Everything works fine:
cv::Mat src = someMethodThatReturnsSrcImage(); // src Image
cv::Mat maskedImage; // stores masked Image
std::vector<cv::Vec3f> circles = someMethodThatReturnsCircles(src);
cv::Mat mask(srcImageForDimensions.size(),srcImageForDimensions.type()); // create an Mat that has same Dimensons as src
mask.setTo(cv::Scalar(0,0,0)); // creates black-Image
// Add all found circles to mask
for( size_t i = 0; i < circles.size(); i++ ) // iterate through all detected Circles
{
cv::Point center(cvRound(circles[i][0]), cvRound(circles[i][1])); // CVRound converts floating numbers to integer
int radius = cvRound(circles[i][2]); // Radius is the third parameter [i][0] = x [i][1]= y [i][2] = radius
cv::circle( mask, center, radius, cv::Scalar(255,255,255),-1, 8, 0 ); // Circle(img, center, radius, color, thickness=1, lineType=8, shift=0)
}
src.copyTo(maskedImage,mask); // creates masked Image and copies it to maskedImage
you can make the background the color you want
image=cv::Scalar(red_value, green_value, blue_value);
then draw your circles
I think that comment just under your question is the best solution.
I made a modified version of your code for 5M image from fisheye camera. This image also needs to make black all points outside circle.
#include <Windows.h>
#include <Vfw.h>
#include <string>
#include <iostream>
#include "opencv2\core\core.hpp"
#include "opencv2\imgproc\imgproc.hpp"
#include "opencv2\imgcodecs\imgcodecs.hpp"
#include "opencv2\highgui\highgui.hpp"
using namespace std;
using namespace cv;
int _tmain(int argc, _TCHAR* argv[])
{
cv::Mat im_source_non_square = cv::imread("D:/FishLib/sample_02.bmp", CV_LOAD_IMAGE_COLOR);
cv::namedWindow("Image",CV_WINDOW_FREERATIO);
cv::imshow("Image", im_source_non_square);
Mat im_source_square;
int m_nCenterX=1280;
int m_nCenterY=960;
int m_nRadius=916;
Mat im_mask=im_source_non_square.clone();
im_mask.setTo(cv::Scalar(0,0,0));
circle( im_mask, cv::Point(m_nCenterX,m_nCenterY), m_nRadius, cv::Scalar(255,255,255), -3, 8, 0 );
cv::namedWindow("Mask image",CV_WINDOW_FREERATIO);
cv::imshow("Mask image", im_mask);
Mat im_source_circle;
cv::bitwise_and(im_source_non_square,im_mask,im_source_circle);
cv::namedWindow("Combined image",CV_WINDOW_FREERATIO);
cv::imshow("Combined image", im_source_circle);
cv::waitKey(0);
return 0;
}
Just tried your code snippet and it works.
Also if you want to change the background color instead of black, according to opencv docs here, before copyTo the destination mat will be initialized if needed, so just add code below:
cv::Mat maskedImage(srcImageForDimensions.size(), srcImageForDimensions.type()); // stores masked Image
maskedImage.setTo(cv::Scalar(0,0,255)); // set background color to red

how to reduce light intensity of an image

Took an example image from opencv (cat.jpg).To reduce brightness at particular area. here is the link for the image
http://tinypic.com/view.php?pic=2lnfx46&s=5
Here is one possible solution. The bright spots are detected using a simple threshold operation. Then the bright spots are darkened using a gamma transformation. The result looks slightly better, but unfortunately, if the pixels in the image are exactly white, all the pixel information is lost and you will not be able to recover this information.
#include <opencv2/opencv.hpp>
#include <iostream>
#include <cfloat>
int threshold = 200;
double gammav = 3;
int main(int argc, char** argv )
{
cv::Mat image,gray_image,bin_image;
// read image
cv::imread(argv[1]).convertTo(image,CV_32FC3);
// find bright spots with thresholding
cv::cvtColor(image, gray_image, CV_RGB2GRAY);
cv::threshold( gray_image, bin_image, threshold, 255,0 );
// blur mask to smooth transitions
cv::GaussianBlur(bin_image, bin_image, cv::Size(21,21), 5 );
// create 3 channel mask
std::vector<cv::Mat> channels;
channels.push_back(bin_image);
channels.push_back(bin_image);
channels.push_back(bin_image);
cv::Mat bin_image3;
cv::merge(channels,bin_image3);
// create darker version of the image using gamma correction
cv::Mat dark_image = image.clone();
for(int y=0; y<dark_image.rows; y++)
for(int x=0; x<dark_image.cols; x++)
for(int c=0;c<3;c++)
dark_image.at<cv::Vec3f>(y,x)[c] = 255.0 * pow(dark_image.at<cv::Vec3f>(y,x)[c]/255.0,gammav);
// create final image
cv::Mat res_image = image.mul((255-bin_image3)/255.0) + dark_image.mul((bin_image3)/255.0);
cv::imshow("orig",image/255);
cv::imshow("dark",dark_image/255);
cv::imshow("bin",bin_image/255);
cv::imshow("res",res_image/255);
cv::waitKey(0);
}

crop and Save ROI as new image in OpenCV 2.4.2 using cv::Mat

Working on Face Detection and Recognition, and after successfully detecting a face, I just want to crop the face and save it somewhere in the drive to give it for the recognition code. I am having hard time doing the saving the Region of Interest as a new image. I have got some codes online but it is written in the previous version of OpenCV that uses IplImage*. I am using OpenCV 2.4.2 that uses cv::Mat.Heeeelp!!!
I will post my codes(Face detection and Recognition per se) if you guys want it.
#include <cv.h>
#include <highgui.h>
#include <math.h>
// alphablend <imageA> <image B> <x> <y> <width> <height>
// <alpha> <beta>
IplImage* crop( IplImage* src, CvRect roi)
{
// Must have dimensions of output image
IplImage* cropped = cvCreateImage( cvSize(roi.width,roi.height), src->depth, src->nChannels );
// Say what the source region is
cvSetImageROI( src, roi );
// Do the copy
cvCopy( src, cropped );
cvResetImageROI( src );
cvNamedWindow( "check", 1 );
cvShowImage( "check", cropped );
cvSaveImage ("style.jpg" , cropped);
return cropped;
}
int main(int argc, char** argv)
{
IplImage *src1, *src2;
CvRect myRect;
// IplImage* cropped ;
src1=cvLoadImage(argv[1],1);
src2=cvLoadImage(argv[2],1);
{
int x = atoi(argv[3]);
int y = atoi(argv[4]);
int width = atoi(argv[5]);
int height = atoi(argv[6]);
double alpha = (double)atof(argv[7]);
double beta = (double)atof(argv[8]);
cvSetImageROI(src1, cvRect(x,y,width,height));
cvSetImageROI(src2, cvRect(100,200,width,height));
myRect = cvRect(x,y,width,height) ;
cvAddWeighted(src1, alpha, src2, beta,0.0,src1);
cvResetImageROI(src1);
crop (src1 , myRect);
cvNamedWindow( "Alpha_blend", 1 );
cvShowImage( "Alpha_blend", src1 );
cvWaitKey(0);
}
return 0;
}
Thanks. Peace
Using cv::Mat objects will make your code substantially simpler. Assuming the detected face lies in a rectangle called faceRect of type cv::Rect, all you have to type to get a cropped version is:
cv::Mat originalImage;
cv::Rect faceRect;
cv::Mat croppedFaceImage;
croppedFaceImage = originalImage(faceRect).clone();
Or alternatively:
originalImage(faceRect).copyTo(croppedImage);
This creates a temporary cv::Matobject (without copying the data) from the rectangle that you provide. Then, the real data is copied to your new object via the clone or copy method.
For cropping the region, the ROI(Region of interest) is used. The opencv2 does the job quite easily. You can check the link:
http://life2coding.blogspot.com/search/label/cropping%20of%20image

Resources