Related
I am trying to segment an image of rocks and I get a decent result. But now I need to count the pixels in the largest colored object.
The picture above shows a segmented image of a rock pile and I want to count the number of green pixels which denote the largest rock in the image. And then also count the 2nd largest,i.e, the yellow one. After counting I would like to compare it with the ground truth to compare my results.
The code to get the segmented image is referred from Watershed segmentation opencv. A part of my code is also given below :
cv::findContours(peaks_8u, contours, CV_RETR_EXTERNAL, CV_CHAIN_APPROX_SIMPLE);
// Create the marker image for the watershed algorithm
// CV_32S - 32-bit signed integers ( -2147483648..2147483647 )
cv::Mat markers = cv::Mat::zeros(input_image.size(), CV_32S);
// Draw the foreground markers
for (size_t i = 0; i < contours.size(); i++)
{
cv::drawContours(markers, contours, static_cast<int>(i), cv::Scalar(static_cast<int>(i) + 1), -1);
}
// Draw the background marker
cv::circle(markers, cv::Point(5, 5), 3, cv::Scalar(255), -1);
cv::watershed(in_sharpened_image, markers);
// Generate random colors; result of watershed
std::vector<cv::Vec3b> colors;
for (size_t i = 0; i < contours.size(); i++)
{
int b = cv::theRNG().uniform(0, 256); //0,256
int g = cv::theRNG().uniform(0, 256);
int r = cv::theRNG().uniform(0, 256);
colors.push_back(cv::Vec3b((uchar)b, (uchar)g, (uchar)r));
}
// Create the result image
cv::Mat dst = cv::Mat::zeros(markers.size(), CV_8UC3);
// Fill labeled objects with random colors
for (int i = 0; i < markers.rows; i++)
{
for (int j = 0; j < markers.cols; j++)
{
int index = markers.at<int>(i, j);
if (index > 0 && index <= static_cast<int>(contours.size()))
{
dst.at<cv::Vec3b>(i, j) = colors[index - 1];
}
}
}
Question: Is there an efficient way to count the pixels inside the largest/marker in opencv?
You can calculate a histogram of markers using cv::calcHist with range from 0 to contours.size() + 1 and find the largest value in it starting from the index 1.
Instead of counting pixels you could use contourArea() for your largest contour. This will work much faster.
Something like this.
cv::Mat mask;
// numOfSegments - number of your labels (colors)
for (int i = 0; i < numOfSegments; i++) {
std::vector<cv::Vec4i> hierarchy;
// this "i + 2" may be different for you
// depends on your labels allocation.
// This is thresholding to get mask with
// contour of your #i label (color)
cv::inRange(markers, i + 2, i + 2, mask);
contours.clear();
findContours(mask, contours, hierarchy, cv::RETR_EXTERNAL, cv::CHAIN_APPROX_NONE);
double area = cv::contourArea(contours[0]);
}
Having contours in hands is also good because after watershed() they will be quite "noisy" with lots of small peaks and not suitable for most of using in the "raw" form. Having contour you may smooth it with gauss or approxPoly, etc., as well as check for some important properties or contour shape if you need it.
So I am new to OpenCV, what I want to do is to copy elements of an bit16 matrix.
src.create(h, w, CV_16UC(channels));
dst.create(hr, wr, CV_16UC(channels));
finalDst.create(h, w, CV_16UC(channels));
memcpy(src.data, data_in, w*h*sizeof(raw_t_ubit16));
for (i = 0; i < h; i++)
{
for (j = 0; j < w; j++)
{
finalDst.data[j + i*w] = src.data[j + i*w];
}
}
memcpy(data_out, finalDst.data, h*w*sizeof(raw_t_ubit16));
However this only copies one half of the image, Ironically if I put 2*h instead of h then everything gets back to normal. But that shouldn't be so, since I am defining h to be the exact height of my image just like w is the width.
src.data - gives you (*uchar), but not (*raw_t_ubit16).
(*uchar) 8 bit.
(*raw_t_ubit16) 16 bit.
((*uchar)data)[2] - will point to 3-rd byte in array (assumes array of uchar).
((*raw_t_ubit16)data)[2] - will point to 5-th byte in array (assumes array of raw_t_ubit16).
That's why you got only a half copied.
You can write:
((raw_t_ubit16*)finalDst.data)[j + i*w] = ((raw_t_ubit16*)src.data)[j + i*w];
It should work, but better use clone() method of Mat class:
m1=m2.clone();
I've implemented a back-propagating neural network and trained it on my data. The data alternates between sentences in English & Africaans. The neural network is supposed to identify the language of the input.
The structure of the Network is 27 *16 * 2
The input layer has 26 inputs for each letter of the alphabet plus a bias unit.
My problem is that the error is thrown violently in opposite directions as each new training example is encountered. As I mentioned, the training examples are read in, in alternating fashion (English, Africaans, English....)
I can train the network to identify all English, or all Africaans, but not to identify either one (both) in the same pass.
The y-axis below is the Output signal error for each of the two output nodes (English and Africaans), and the x axis is the number of the training example. In a way, it does exactly what I programmed it to do; when the example is English it the changes the weights to identify English better. However, in doing so, it makes the network worse at predicting Africaans. This is why the error goes between positive and negative values.
Clearly this isn't how it should work, but I'm stuck.
I feel as though the error is conceptual on my part, but here is the relevant code:
public void train() throws NumberFormatException, IOException{
// Training Accuracy
double at = 0;
//epoch
int epoch = 0;
int tNum = 0;
for(; epoch < epochMax; epoch++){
// Reads stock files from TestPackage package in existing project
BufferedReader br = new BufferedReader(new InputStreamReader(this.getClass().
getResourceAsStream("/TrainingData/" + trainingData.getName())));
while ((line = br.readLine()) != null) {
Boolean classified = false;
tNum++;
// Set the correct classification Tk
t[0] = Integer.parseInt(line.split("\t")[0]); //Africaans
t[1] = (t[0] == 0) ? 1 : 0; // English
// Convert training string to char array
char trainingLine[] = line.split("\t")[1].toLowerCase().toCharArray();
// Increment idx of input layer z, that matches
// the position of the char in the alphabet
// a == 0, b == 2, etc.....
for(int l = 0; l < trainingLine.length; l++){
if((int)trainingLine[l] >= 97 && (int)trainingLine[l] <= 122)
z[(int)trainingLine[l] % 97]++;
}
/*System.out.println("Z " + Arrays.toString(z));
System.out.println();*/
// Scale Z
for(int i = 0; i < z.length-1; i++){
z[i] = scale(z[i], 0, trainingLine.length, -Math.sqrt(3),Math.sqrt(3));
}
/*----------------------------------------------------------------
* SET NET HIDDEN LAYER
* Each ith unit of the hidden Layer =
* each ith unit of the input layer
* multiplied by every j in the ith level of the weights matrix ij*/
for(int j = 0; j < ij.length; j++){ // 3
double[] dotProduct = multiplyVectors(z, ij[j]);
y[j] = sumVector(dotProduct);
}
/*----------------------------------------------------------------
* SET ACTIVATION HIDDEN LAYER
*/
for(int j = 0; j < y.length-1; j++){
y[j] = sigmoid(y[j], .3, .7);
}
/*----------------------------------------------------------------
* SET NET OUTPUT LAYER
* Each jth unit of the hidden Layer =
* each jth unit of the input layer
* multiplied by every k in the jth level of the weights matrix jk*/
for(int k = 0; k < jk.length; k++){ // 3
double[] dotProduct = multiplyVectors(y, jk[k]);
o[k] = sumVector(dotProduct);
}
/*----------------------------------------------------------------
* SET ACTIVATION OUTPUT LAYER
*/
for(int k = 0; k < o.length; k++){
o[k] = sigmoid(o[k], .3, .7);
}
/*----------------------------------------------------------------
* SET OUTPUT ERROR
* For each traing example, evalute the error.
* Error is defined as (Tk - Ok)
* Correct classifications will result in zero error:
* (1 - 1) = 0
* (0 - 0) = 0
*/
for(int k = 0; k < o.length; k++){
oError[k] = t[k] - o[k];
}
/*----------------------------------------------------------------
* SET TRAINING ACCURACY
* If error is 0, then a 1 indicates a succesful prediction.
* If error is 1, then a 0 indicates an unsucessful prediction.
*/
if(quantize(o[0],.3, .7) == t[0] && quantize(o[1], .3, .7) == t[1]){
classified = true;
at += 1;
}
// Only compute errors and change weiths for classification errors
if(classified){
continue;
}
/*----------------------------------------------------------------
* CALCULATE OUTPUT SIGNAL ERROR
* Error of ok = -(tk - ok)(1 - ok)ok
*/
for(int k = 0; k < o.length; k++){
oError[k] = outputError(t[k], o[k]);
}
/*----------------------------------------------------------------
* CALCULATE HIDDEN LAYER SIGNAL ERROR
*
*/
// The term (1-yk)yk is expanded to yk - yk squared
// For each k-th output unit, multiply it by the
// summed dot product of the two terms (1-yk)yk and jk[k]
for(int j = 0; j < y.length; j++){
for(int k = 0; k < o.length; k++){
/*System.out.println(j+"-"+k);*/
yError[j] += oError[k] * jk[k][j] * (1 - y[j]) * y[j];
}
}
/*----------------------------------------------------------------
* CALCULATE NEW WIGHTS FOR HIDDEN-JK-OUTPUT
*
*/
for(int k = 0; k < o.length; k++){
for(int j = 0; j < y.length; j++){
djk[k][j] = (-1*learningRate)*oError[k]*y[j] + momentum*djk[k][j];
// Old weights = themselves + new delta weight
jk[k][j] += djk[k][j];
}
}
/*----------------------------------------------------------------
* CALCULATE NEW WIGHTS FOR INPUT-IJ-HIDDEN
*
*/
for(int j = 0; j < y.length-1; j++){
for(int i = 0; i < z.length; i++){
dij[j][i] = (-1*learningRate)*yError[j]*z[i] + momentum*dij[j][i];
// Old weights = themselves + new delta weight
ij[j][i] += dij[j][i];
}
}
}
}
// Accuracy Percentage
double at_prec = (at/tNum) * 100;
System.out.println("Training Accuracy: " + at_prec);
}
I agree with the comments that this model is probably not the best for your classification problem but if you are interested in trying to get this to work I will give you the reason I think this oscillates and the way that I would try and tackle this problem.
From my understanding of your question and comments, I cannot understand what the network actually “learns” in this instance. You feed letters in (is this the number of times that the letter occurs in the sentence?) and you force it to map to an output. Let’s say you just use English now and English corresponds to an output of 1. So you “train” it on one sentence and for argument’s sake it chooses the letter “a” as the determining input, which is quite a common letter. It sets the network weight such that when it sees “a” the output is 1, and all other letter inputs get weighted down such that they don’t influence the output. It might not be so black and white, but it could be doing something very similar. Now, every time you feed another English sentence in, it only has to see an “a” to give a correct output. Doing the same for just Africaans as an output of zero, it maps “a” to zero. So, every time you alternate between the two languages, it completely reassigns the weightings... you’re not building on a structure. The back-propagation of error is basically always a fixed value because there are no degrees of rightness or wrongness, it’s one or the other. So I would expect it to oscillate exactly as you are seeing.
EDIT: I think this boils down to something like the presence of letters being used to classify the language category and expecting one of two polar outputs, rather than anything about the relationship between the letters that defines the language.
On a conceptual level, I would have a full pre-processing stage to get some statistics. Off the top of my head, I might calculate (I don’t know the language):
- The ratio of letter “a” to “c” occuring in sentence
- The ratio of letter “d” to “p” occuring in sentence
- The average length of word in the sentence
Do this for 50 sentences of each language. Feed all the data in at once and train on the whole set (70% for training, 15% for validation, 15% for testing). You cannot train a network on a single value each time (as I think you are doing?), it needs to see the whole picture. Now your output is not so black and white, it has the flexibility to map to a value that is between 0 and 1, not absolutes each time. Anything above 0.5 is English, below 0.5 is Africaans. Start with, say, 10 statistical parameters for the languages, 5 neurons in hidden layer, 1 neuron in the output layer.
I have the below image after some conversions.
How can I find a distance between these two lines?
A simple way to do this would be
- Scan across a row until you find a pixel above a threshold.
- Keep scanning until you find a pixel below the threshold.
- Count the pixels until the next pixel above the threshold.
- Take the average across a number of rows sampled from the image (or all rows)
- You'll need to know the image resolution (e.g. dpos per inch) to convert the count to an actual distance
An efficient method to scan across rows can be found in the OpenCV documentation
A more complicated approach would use Houghlines to extract lines. It will give you two points on each line (hopefully you only have two). From that it is possible to work out a distance formula, assuming the lines are parallel.
A skeleton code (not efficient, just readable so that you know how to do it) would be,
cv::Mat source = cv::imread("source.jpg", CV_LOAD_IMAGE_GRAYSCALE);
std::vector<int> output;
int threshold = 35, temp_var; // Change in accordance with data
int DPI = 30; // Digital Pixels per Inch
for (int i=0; i<source.cols; ++i)
{
for (int j=0; j<source.rows; ++j)
{
if (source.at<unsigned char>(i,j) > threshold)
{
temp_var = j;
for (; j<source.rows; ++j)
if (source.at<unsigned char>(i,j) > threshold)
output.push_back( (j-temp_var)/DPI ); // Results are stored in Inch
}
}
}
Afterwards, you could take an average of all the elements in output, etc.
HTH
Assumptions:
You have only two continuous lines without any break in between.
No other pixels (noise) apart from the lines
My proposed solution: Almost same as given above
Mark leftmost line as line 1. Mark rightmost line as line 2.
Scan the image (Mat in OpenCV) from the leftmost column and make a list of points matching the pixel value of line 1
Scan the image (Mat in OpenCV) from the rightmost column and make a list of points matching the pixel value of line 2
Calculate the distance between points from that list using the code below.
public double euclideanDistance(Point a, Point b){
double distance = 0.0;
try{
if(a != null && b != null){
double xDiff = a.x - b.x;
double yDiff = a.y - b.y;
distance = Math.sqrt(Math.pow(xDiff,2) + Math.pow(yDiff, 2));
}
}catch(Exception e){
System.err.println("Something went wrong in euclideanDistance function in "+Utility.class+" "+e.getMessage());
}
return distance;
}
Which kind of filter is being used by CImg library's get_convolve() function(written in C)? Median or Gaussian or bilateral or some other?
I tried to understand the function so that I can use the similar functionality in PIL openCV. In the header file CImg.h of the library, it says:
/**
Compute the convolution of the image by a mask.
The result \p res of the convolution of an image \p img by a mask \p mask is defined to be :
res(x,y,z) = sum_{i,j,k} img(x-i,y-j,z-k)*mask(i,j,k)
param mask = the correlation kernel.
param cond = the border condition type (0=zero, 1=dirichlet)
param weighted_convol = enable local normalization.
**/
Declaration is like this:
template<typename t> CImg<typename cimg::superset2<T,t,float>::type>
get_convolve(const CImg<t>& mask, const unsigned int cond=1, const bool weighted_convol=false) const {}
Here is a code sniplet:
for (int z = mz1; z<mze; ++z)
for (int y = my1; y<mye; ++y)
for (int x = mx1; x<mxe; ++x) {// For each pixel
Ttfloat val = 0;
for (int zm = -mz1; zm<=mz2; ++zm)
for (int ym = -my1; ym<=my2; ++ym)
for (int xm = -mx1; xm<=mx2; ++xm)
val+=(*this)(x+xm,y+ym,z+zm,v)*mask(mx1+xm,my1+ym,mz1+zm);
dest(x,y,z,v) = (Ttfloat)val;
}
if (cond)
cimg_forYZV(*this,y,z,v)
for (int x = 0; x<dimx(); (y<my1 || y>=mye || z<mz1 || z>=mze)?++x:((x<mx1-1 || x>=mxe)?++x:(x=mxe))) {
Ttfloat val = 0;
for (int zm = -mz1; zm<=mz2; ++zm) for (int ym = -my1; ym<=my2; ++ym) for (int xm = -mx1; xm<=mx2; ++xm)
val+=at3(x+xm,y+ym,z+zm,v)*mask(mx1+xm,my1+ym,mz1+zm);
dest(x,y,z,v) = (Ttfloat)val;
}else
cimg_forYZV(*this,y,z,v)
I am using the mask of 7 x 7 and each of the values inside it is '1'.
What I got from the function was that for each pixel, it is taking a 7 by 7 window around it, with the pixel at center and then multiplying with he Identity matrix. It feels like some kind of smoothing filter but which one is it? Which equivalent filter can I use in openCV?
I can post the whole function, but its too long and I don't see the point. I would be really thankful for your help.
So, I found the answer in the thesis of the person who implemented pHash. It said:
During the process of calculating pHash, a mean filter is applied to the image. A ker-
nel with dimension 7x7 is used. To apply this kernel, the get_convolve()
function of the CImg library is used. It is then highlighted as:
For an image I and a mask M it is:
R(x,y,z) = SIGMA(i,j,k) I(x − i, y − j, z − k)M (i, j, k)
Then when I looked at the type of filtering functions offered by openCV here, it matched with the box filter function.