I created three dimension matrix for computing of histogram as follows:
// Histogram of HSV image
int const hue_bins = 180; //
int const sat_bins = 256; //
int const val_bins = 4; // Only four bins for V channel!
float const hue_range[2] = {0, 180};
float const sat_range[2] = {0, 256};
float const val_range[2] = {0, 256};
int const hsv_sizes[] = {hue_bins, sat_bins, val_bins};
cv::Mat1f m_tone_frequences(3, hsv_sizes, 0.);
Then I'm using
cv::calcHist
( &image, 1, channels, mask, histogram
, num_channels, hsv_sizes, ranges);
...
cv::calcBackProject
( &image_f, 1, channels, histogram
, backproject, hsv_sizes, 1.0);
and seems it works fine (code is simplified).
Since the histograms are sampled from a single image, it is possible to run into sampling problems (object of interest has narrow color distribution). So I want to apply Gaussian smoothing to "Value" histogram planes.
I'm tried get histogram rows, but it gives me anothed 3D Mat:
cv::Mat1f hrow = histogram.row(0);
// hrow.dims ==3 && hrow.rows == -1 && hrow.cols == -1
and I don't have ideas about processing of it.
I am at a loss to solve this issue because this action should be very simple to do.
Any advice is greatly appreciated.
Related
I'm trying to reproduce Photoshop's multiply blend mode in OpenCV. Equivalents to this would be what you find in GIMP, or when you use the CIMultiplyBlendMode in Apple's CoreImage framework.
Everything I read online suggests that multiply blending is accomplished simply by multiplying the channels of the two input images (i.e., Blend = AxB). And, this works, except for the case(s) where alpha is < 1.0.
You can test this very simply in GIMP/PhotoShop/CoreImage by creating two layers/images, filling each with a different solid color, and then modifying the opacity of the first layer. (BTW, when you modify alpha, the operation is no longer commutative in GIMP for some reason.)
A simple example: if A = (0,0,0,0) and B = (0.4,0,0,1.0), and C = AxB, then I would expect C to be (0,0,0,0). This is simple multiplication. But this is not how this blend is implemented in practice. In practice, C = (0.4,0,0,1.0), or C = B.
The bottom line is this: I need to figure out the formula for the multiply blend mode (which is clearly more than AxB) and then implement it in OpenCV (which should be trivial once I have the formula).
Would appreciate any insights.
Also, for reference, here are some links which show multiply blend as being simply AxB:
How does photoshop blend two images together
Wikipedia - Blend Modes
Photoshop Blend Modes
Here is an OpenCV solution based the source code of GIMP, specifically the function gimp_operation_multiply_mode_process_pixels.
NOTE
Instead of looping on all pixels it can be vectorized, but I followed the steps of GIMP.
Input images must be of type CV_8UC3 or CV_8UC4.
it supports also the opacity value, that must be in [0, 255]
in the original GIMP implementation there is also the support for a mask. It can be trivially added to the code, eventually.
This implementation is in fact not symmetrical, and reproduce your strange behaviour.
Code:
#include <opencv2\opencv.hpp>
using namespace cv;
Mat blend_multiply(const Mat& level1, const Mat& level2, uchar opacity)
{
CV_Assert(level1.size() == level2.size());
CV_Assert(level1.type() == level2.type());
CV_Assert(level1.channels() == level2.channels());
// Get 4 channel float images
Mat4f src1, src2;
if (level1.channels() == 3)
{
Mat4b tmp1, tmp2;
cvtColor(level1, tmp1, COLOR_BGR2BGRA);
cvtColor(level2, tmp2, COLOR_BGR2BGRA);
tmp1.convertTo(src1, CV_32F, 1. / 255.);
tmp2.convertTo(src2, CV_32F, 1. / 255.);
}
else
{
level1.convertTo(src1, CV_32F, 1. / 255.);
level2.convertTo(src2, CV_32F, 1. / 255.);
}
Mat4f dst(src1.rows, src1.cols, Vec4f(0., 0., 0., 0.));
// Loop on every pixel
float fopacity = opacity / 255.f;
float comp_alpha, new_alpha;
for (int r = 0; r < src1.rows; ++r)
{
for (int c = 0; c < src2.cols; ++c)
{
const Vec4f& v1 = src1(r, c);
const Vec4f& v2 = src2(r, c);
Vec4f& out = dst(r, c);
comp_alpha = min(v1[3], v2[3]) * fopacity;
new_alpha = v1[3] + (1.f - v1[3]) * comp_alpha;
if ((comp_alpha > 0.) && (new_alpha > 0.))
{
float ratio = comp_alpha / new_alpha;
out[0] = max(0.f, min(v1[0] * v2[0], 1.f)) * ratio + (v1[0] * (1.f - ratio));
out[1] = max(0.f, min(v1[1] * v2[1], 1.f)) * ratio + (v1[1] * (1.f - ratio));
out[2] = max(0.f, min(v1[2] * v2[2], 1.f)) * ratio + (v1[2] * (1.f - ratio));
}
else
{
out[0] = v1[0];
out[1] = v1[1];
out[2] = v1[2];
}
out[3] = v1[3];
}
}
Mat3b dst3b;
Mat4b dst4b;
dst.convertTo(dst4b, CV_8U, 255.);
cvtColor(dst4b, dst3b, COLOR_BGRA2BGR);
return dst3b;
}
int main()
{
Mat3b layer1 = imread("path_to_image_1");
Mat3b layer2 = imread("path_to_image_2");
Mat blend = blend_multiply(layer1, layer2, 255);
return 0;
}
I managed to sort this out. Feel free to comment with any suggested improvements.
First, I found a clue as to how to implement the multiply function in this post:
multiply blending
And here's a quick OpenCV implementation in C++.
Mat MultiplyBlend(const Mat& cvSource, const Mat& cvBackground) {
// assumption: cvSource and cvBackground are of type CV_8UC4
// formula: (cvSource.rgb * cvBackground.rgb * cvSource.a) + (cvBackground.rgb * (1-cvSource.a))
Mat cvAlpha(cvSource.size(), CV_8UC3, Scalar::all(0));
Mat input[] = { cvSource };
int from_to[] = { 3,0, 3,1, 3,2 };
mixChannels(input, 1, &cvAlpha, 1, from_to, 3);
Mat cvBackgroundCopy;
Mat cvSourceCopy;
cvtColor(cvSource, cvSourceCopy, CV_RGBA2RGB);
cvtColor(cvBackground, cvBackgroundCopy, CV_RGBA2RGB);
// A = cvSource.rgb * cvBackground.rgb * cvSource.a
Mat cvBlendResultLeft;
multiply(cvSourceCopy, cvBackgroundCopy, cvBlendResultLeft, 1.0 / 255.0);
multiply(cvBlendResultLeft, cvAlpha, cvBlendResultLeft, 1.0 / 255.0);
delete(cvSourceCopy);
// invert alpha
bitwise_not(cvAlpha, cvAlpha);
// B = cvBackground.rgb * (1-cvSource.a)
Mat cvBlendResultRight;
multiply(cvBackgroundCopy, cvAlpha, cvBlendResultRight, 1.0 / 255.0);
delete(cvBackgroundCopy, cvAlpha);
// A + B
Mat cvBlendResult;
add(cvBlendResultLeft, cvBlendResultRight, cvBlendResult);
delete(cvBlendResultLeft, cvBlendResultRight);
cvtColor(cvBlendResult, cvBlendResult, CV_RGB2RGBA);
return cvBlendResult;
}
I am struggling with finding the appropriate contour algorithm for a low quality image. The example image shows a rock scene:
What I am trying to achieve is to find contours arround features such as:
light areas
dark areas
grey1 areas
grey2 areas
etc. until grey-n areas
(The number of areas shall be a parameter of choice)
I do not want to take a simple binary-threshold but rather use some sort of contour-finding (for example watershed or other). The major feature-lines shall be kept, noise within a feature-are can be flattened.
The result of my code can be seen on the images to the right.
Unfortunately, as you can easily tell, the colors do not really represent the original large-scale image features! For example: check out the two areas that I circled with red - these features are almost completely flooded with another color. What I imagine is that at least the very light and the very dark areas are covered by its own color.
cv::Mat cv_src = cv::imread(argv[1]);
cv::Mat output;
cv::Mat cv_src_gray;
cv::cvtColor(cv_src, cv_src_gray, cv::COLOR_RGB2GRAY);
double clipLimit = 0.1;
cv::Size titleGridSize = cv::Size(8,8);
cv::Ptr<cv::CLAHE> clahe = cv::createCLAHE(clipLimit, titleGridSize);
clahe->apply(cv_src_gray, output);
cv::equalizeHist(output, output);
cv::cvtColor(output, cv_src, cv::COLOR_GRAY2RGB);
// Create binary image from source image
cv::Mat bw;
cv::cvtColor(cv_src, bw, cv::COLOR_BGR2GRAY);
cv::threshold(bw, bw, 180, 255, cv::THRESH_BINARY);
// Perform the distance transform algorithm
cv::Mat dist;
cv::distanceTransform(bw, dist, cv::DIST_L2, CV_32F);
// Normalize the distance image for range = {0.0, 1.0}
cv::normalize(dist, dist, 0, 1., cv::NORM_MINMAX);
// Threshold to obtain the peaks
cv::threshold(dist, dist, .2, 1., cv::THRESH_BINARY);
// Create the CV_8U version of the distance image
cv::Mat dist_8u;
dist.convertTo(dist_8u, CV_8U);
// Find total markers
std::vector<std::vector<cv::Point> > contours;
cv::findContours(dist_8u, contours, cv::RETR_EXTERNAL, cv::CHAIN_APPROX_SIMPLE);
int ncomp = contours.size();
// Create the marker image for the watershed algorithm
cv::Mat markers = cv::Mat::zeros(dist.size(), CV_32S);
// Draw the foreground markers
for (int i = 0; i < ncomp; i++)
cv::drawContours(markers, contours, i, cv::Scalar::all(i+1), -1);
// Draw the background marker
cv::circle(markers, cv::Point(5,5), 3, CV_RGB(255,255,255), -1);
// Perform the watershed algorithm
cv::watershed(cv_src, markers);
// Generate random colors
std::vector<cv::Vec3b> colors;
for (int i = 0; i < ncomp; i++)
{
int b = cv::theRNG().uniform(0, 255);
int g = cv::theRNG().uniform(0, 255);
int r = cv::theRNG().uniform(0, 255);
colors.push_back(cv::Vec3b((uchar)b, (uchar)g, (uchar)r));
}
// Create the result image
cv::Mat dst = cv::Mat::zeros(markers.size(), CV_8UC3);
// Fill labeled objects with random colors
for (int i = 0; i < markers.rows; i++)
{
for (int j = 0; j < markers.cols; j++)
{
int index = markers.at<int>(i,j);
if (index > 0 && index <= ncomp)
dst.at<cv::Vec3b>(i,j) = colors[index-1];
else
dst.at<cv::Vec3b>(i,j) = cv::Vec3b(0,0,0);
}
}
// Show me what you got
imshow("final_result", dst);
I think you can use a simple clustering such as k-means for this, then examine the cluster centers (or the mean and standard deviations of each cluster). I quickly tried it in matlab.
im = imread('tvBqt.jpg');
gr = rgb2gray(im);
x = double(gr(:));
idx = kmeans(x, 4);
cl = reshape(idx, 600, 472);
figure,
subplot(1, 2, 1), imshow(gr, []), title('original')
subplot(1, 2, 2), imshow(label2rgb(cl), []), title('clustered')
The result:
You could try using SLIC Superpixels. I tried it and showed some good results. You could vary the parameters to get better clustering.
SLIC Superpixels
SLIC Superpixels with OpenCV C++
SLIC Superpixels with OpenCV Python
I have what should be a simple exercise in OpenCV, but can't seem to get it working. I'm trying to determine the density of edges in a section of an image. This is the process I follow:
1. pull subimage from image
2. use Canny to find edges in subImage
3. threshold to create binary image
4. create histogram for binary image
5. get number of pixels in binary image that are "on" (255)
6. calculate "edge density" as numPixelsOn/totalPixels
I've checked the results of 1,2,and 3 above, and results seem ok. Steps 4 and 5 seem to be giving me trouble.
Here's my code for calculating the histogram:
int histSize = 256; // bin size
float range[] = { 0, 256} ;
const float* histRange = { range };
bool uniform = true;
bool accumulate = false;
Mat hist;
/// Compute the histograms:
calcHist( &gray, 1, 0, Mat(), hist, 1, &histSize, &histRange, uniform, accumulate );
This doesn't seem to be working. When I check hist after calling calcHist, it has no data (i.e. data == 0)... or maybe I don't understand what I'm looking at.
Now for accessing the "bins" in the histogram, I've tried a number of things. First I tried this:
uchar* p;
p = hist.ptr<uchar>(0);
double edgePixels = p[255];
I also tried to use:
cvQueryHistValue_1D(hist,255); // #include <opencv2/legacy/compat.hpp>
This wouldn't compile. Gave 2 errors: 'cv::Mat' does not have an overloaded member 'operator ->', and 'bins': is not a member of 'cv::Mat'
I guess I need some help on this.
There is an error in your 3rd param - channels, that should be an array so you should call it like this
int histSize = 256; // bin size
float range[] = { 0, 256} ;
const float* histRange = { range };
bool uniform = true;
bool accumulate = false;
Mat hist;
int channels[] = {0};
/// Compute the histograms:
calcHist( &gray, 1, channels, Mat(), hist, 1, &histSize, &histRange, uniform, accumulate );
You should also call:
hist.at<float>(0);
to get your value, OpenCV stores them as floats, this is the reason you're getting 0 when using uchar as uchar is smaller than float and the numbers stores as small enough to not fill the first bites.
I've built a 3D Histogram in OpenCV from H-S-V samples from an (CV_8UC3) image.
I need to normalize this histogram so that all the values sum to 1.0 (preferrably in a float representation), since it will be used as a probability mass function (pmf) for a lookup table.
I've tried various permutations of built-in OpenCV functions, but none seem to give the desired result.
int histSize[] = {hBins, sBins, vBins};
float hRange[] = {0.0f, (float)H_RANGE};
float sRange[] = {0.0f, (float)S_RANGE};
float vRange[] = {0.0f, (float)V_RANGE};
const float* ranges[] = {hRange, sRange, vRange};
const int channels[] = {0, 1, 2}; // we compute the 3D histogram on all 2 channels (H-S-V)
cv::calcHist(&newBGSamples, 1, channels, cv::Mat(), currentBGColourHist, 3, histSize, ranges, true, false);
// currentBGColourHist /= cv::sum(bgHistoricalColourHist)(0);
cv::normalize(currentBGColourHist, currentBGColourHist, 1.0, 1.0, cv::NORM_L1, CV_32FC3);
// cv::normalize(currentBGColourHist, currentBGColourHist, 1.0, 0, cv::NORM_L2, -1, cv::Mat());
// cv::norm(currentBGColourHist, )
// cv::divide((double)1.0/cv::sum(bgHistoricalColourHist)(0), currentBGColourHist, currentBGColourHist, CV_32FC3);
The commented lines show my rough ideas for the normalisation.
I am trying to create a histogram of the depth videos (converted to grayscale first) in order to apply a threshold to keep only highest values, and then do some dilation in order to extract contours. Apparently I am stuck, and besides that i don't know if what I am thinking is the right way to extract contours from depth videos.
In the following code I got stuck in the point of applying the threshold. I think that iam applying it in the wrong way. Which is the correct to apply a threshold in this situation in order to obtain a black and white image?
Any suggestions or links of tutorials would be awesome!!!
Thank you very much!
int bins = 256;
int hsize[] = {bins};
//max and min value of the histogram
float max_value = 0, min_value = 0;
float value;
int normalized;
//ranges - grayscale 0 to 256
float xranges[] = { 0, 256 };
float* ranges[] = { xranges };
//image is the actual source from input depth video
gray = cvCreateImage( cvGetSize(image), 8, 1 );
cvCvtColor( image, gray, CV_BGR2GRAY );
cvNamedWindow("original",1);
cvNamedWindow("gray",1);
cvNamedWindow("histogram",1);
cvNamedWindow("black & white",1);
IplImage* planes[] = { gray };
//get the histogram and some info about it
hist = cvCreateHist( 1, hsize, CV_HIST_ARRAY, ranges,1);
cvCalcHist( planes, hist, 0, NULL);
cvGetMinMaxHistValue( hist, &min_value, &max_value);
printf("min: %f, max: %f\n", min_value, max_value);
imgHistogram = cvCreateImage(cvSize(bins, image->height),8,1);
cvRectangle(imgHistogram, cvPoint(0,0), cvPoint(256,image->height), CV_RGB(255,255,255),-1);
//I think that here i have messed up things :( Any suggestions ???
bw_img = cvCreateImage(cvGetSize(imgHistogram), IPL_DEPTH_8U, 1);
cvThreshold(imgHistogram, bw_img, 150, 255, CV_THRESH_BINARY);
//draw the histogram
for(int i=0; i < bins; i++){
value = cvQueryHistValue_1D( hist, i);
normalized = cvRound(value*image->height/max_value);
cvLine(imgHistogram,cvPoint(i,image->height), cvPoint(i,image->height-normalized), CV_RGB(0,0,0));
}
//show the image results
cvShowImage( "original", image );
cvShowImage( "gray", gray );
cvShowImage( "histogram", imgHistogram );
cvShowImage( "balck & white", bw_img);