how to use float number as opencv image pixel coordinates - opencv

I have a texture image and some uv coordinators (some float 2D vectors).
Is there any method in OpenCV can automatically interpolate the image and then I can directly use these float number as the pixel coordinator and get the correct pixel value?
I think it should be possible because in some computer vision algorithms like optic flow we will always have some sub pixel value...

Remap is what you want, here is an example:
import cv2
import numpy as np
img = cv2.imread('myimage.png')
interpolated_pixel = cv2.remap(img, np.array([[2.4]], np.float32), np.array([[5.4]], np.float32), cv2.INTER_LINEAR)
print(interpolated_pixel)
You can play with different interpolation schemes, see interpolation flags
Of course you can also batch your request by providing multiple uv coordinates.

Related

watershed segmentation always return black image

I've been recently working at a segmentation process for corneal
endothelial cells, and I've found a pretty decent paper that describes ways to perform it with nice results. I have been trying to follow that paper and implement it all using scikit-image and openCV, but I've gotten stucked at the watershed segmentation.
I will briefly describe how is the process supposed to be:
First of all, you have the original endothelial cells image
original image
Then, they instruct you to perform a morphological grayscale reconstruction, in order to level a little bit the grayscale of the image (however, they do not explain how to get the markers for the grayscale, so I've been fooling around and tried to get some on my own way)
This is what the reconstructed image was supposed to look like:
desired reconstruction
This is what my reconstructed image (lets label it as r) looks like:
my reconstruction
The purpose is to use the reconstructed image to get the markers for the watershed segmentation, how do we do that?! We get the original image (lets label it as f), and perform a threshold in (f - r) to extract the h-domes of the cell, i.e., our markers.
This is what the hdomes image was supposed to look like:
desired hdomes
This is what my hdomes image looks like:
my hdomes
I believe that the hdomes I've got are as good as theirs, so, the final step is to finally perform the watershed segmentation on the original image, using the hdomes we've been working so hard to get!
As input image, we will use the inverted original image, and as markers, our markers.
This is the derised output:
desired output
However, I am only getting a black image, EVERY PIXEL IS BLACK and I have no idea of what's happening... I've also tried using their markers and inverted image, however, also getting black image. The paper I've been using is Luc M. Vincent, Barry R. Masters, "Morphological image processing and network analysis of cornea endothelial cell images", Proc. SPIE 1769
I apologize for the long text, however I really wanted to explain everything in detail of what is my understanding so far, btw, I've tried watershed segmentation from both scikit-image and opencv, both gave me the black image.
Here is the following code that I have been using
img = cv2.imread('input.png',0)
mask = img
marker = cv2.erode(mask, cv2.getStructuringElement(cv2.MORPH_ERODE,(3,3)), iterations = 3)
reconstructedImage = reconstruction(marker, mask)
hdomes = img - reconstructedImage
cell_markers = cv2.threshold(hdomes, 0, 255, cv2.THRESH_BINARY)[1]
inverted = (255 - img)
labels = watershed(inverted, cell_markers)
cv2.imwrite('test.png', labels)
plt.figure()
plt.imshow(labels)
plt.show()
Thank you!
Here's a rough example for the watershed segmentation of your image with scikit-image.
What is missing in your script is calculating the Euclidean distance (see here and here) and extracting the local maxima from it.
Note that the watershed algorithm outputs a piece-wise constant image where pixels in the same regions are assigned the same value. What is shown in your 'desired output' panel (e) are the edges between the regions instead.
import numpy as np
import cv2
import matplotlib.pyplot as plt
from skimage.morphology import watershed
from scipy import ndimage as ndi
from skimage.feature import peak_local_max
from skimage.filters import threshold_local
img = cv2.imread('input.jpg',0)
'''Adaptive thersholding
calculates thresholds in regions of size block_size surrounding each pixel
to handle the non-uniform background'''
block_size = 41
adaptive_thresh = threshold_local(img, block_size)#, offset=10)
binary_adaptive = img > adaptive_thresh
# Calculate Euclidean distance
distance = ndi.distance_transform_edt(binary_adaptive)
# Find local maxima of the distance map
local_maxi = peak_local_max(distance, labels=binary_adaptive, footprint=np.ones((3, 3)), indices=False)
# Label the maxima
markers = ndi.label(local_maxi)[0]
''' Watershed algorithm
The option watershed_line=True leave a one-pixel wide line
with label 0 separating the regions obtained by the watershed algorithm '''
labels = watershed(-distance, markers, watershed_line=True)
# Plot the result
plt.imshow(img, cmap='gray')
plt.imshow(labels==0,alpha=.3, cmap='Reds')
plt.show()

Find orientation of object using PCA

I want to find the orientation of the bright object in the images attached. For this purpose, I used Principal Component Analysis(PCA).
In case of image 1, PCA finds correct orientation as the first principal component is alligned in that direction. However, in case of image 2, the principal components are disoriented.
Can anyone please explain why the PCA is showing different results in the two images? Also, please suggest if there is some other method to find the orientation of the object.
import os
import gdal
import matplotlib
import matplotlib.pyplot as plt
import numpy as np
import skimage
from skimage.filters import threshold_otsu
from skimage.filters import try_all_threshold
import cv2
import math
from skimage import img_as_ubyte
from skimage.morphology import convex_hull_image
import pandas as pd
file="path to image file"
(fileRoot, fileExt)= os.path.splitext(file)
ds = gdal.Open(file)
band = ds.GetRasterBand(1)
arr = band.ReadAsArray()
geotransform = ds.GetGeoTransform()
[cols, rows] = arr.shape
thresh = threshold_otsu(arr)
binary = arr > thresh
points = binary>0
y,x = np.nonzero(points)
x = x - np.mean(x)
y = y - np.mean(y)
coords = np.vstack([x, y])
cov = np.cov(coords)
evals, evecs = np.linalg.eig(cov)
sort_indices = np.argsort(evals)[::-1]
evec1, evec2 = evecs[:, sort_indices]
x_v1, y_v1 = evec1
x_v2, y_v2 = evec2
scale = 40
plt.plot([x_v1*-scale*2, x_v1*scale*2],
[y_v1*-scale*2, y_v1*scale*2], color='red')
plt.plot([x_v2*-scale, x_v2*scale],
[y_v2*-scale, y_v2*scale], color='blue')
plt.plot(x,y, 'k.')
plt.axis('equal')
plt.gca().invert_yaxis()
plt.show()
theta = np.tanh((x_v1)/(y_v1)) * 180 /(math.pi)
You claim you are using just white pixels. Did you check which ones are selected by some overlay render? Anyway I do not think it is enough especially for your second image as it does not contain any fully saturated white pixels. I would use more processing before the PCA.
enhance dynamic range
your current images does not need this step as they contain both black and almost fully saturated white. This step allow to unify threshold values among more sample input images. For more info see:
Enhancing dynamic range and normalizing illumination
smooth a bit
this step will significantly lover the intensity of noise points and smooth the edges of bigger objects (but shrink them a bit). This can be done by any FIR filter or convolution or Gaussian filtering. Some also use morphology operators for this.
threshold by intensity
this will remove darker pixels (clear to black) so noise is fully removed
enlarge remaining objects by morphology operators back to former size
You can avoid this by enlarging the resulting OBB by few pixels (number is bound to smooth strength from #2).
now apply OBB search
You are using PCA so use it. I am using this instead:
How to Compute OBB of Multiple Curves?
When I tried your images with above approach (without the #4) I got these results:
Another problem I noticed with your second image is that there are not many white pixels in it. That may bias the PCA significantly especially without preprocessing. I would try to enlarge the image by bicubic filtering and use that as input. May be that is the only problem you got with it.

Converting matches from 8-bit 4 channels to 64-bit 1 channel in OpenCV

I have a vector of Point2f which have color space CV_8UC4 and need to convert them to CV_64F, is the following code correct?
points1.convertTo(points1, CV_64F);
More details:
I am trying to use this function to calculate the essential matrix (rotation/translation) through the 5-point algorithm, instead of using the findFundamentalMath included in OpenCV, which is based on the 8-point algorithm:
https://github.com/prclibo/relative-pose-estimation/blob/master/five-point-nister/five-point.cpp#L69
As you can see it first converts the image to CV_64F. My input image is a CV_8UC4, BGRA image. When I tested the function, both BGRA and greyscale images produce valid matrices from the mathematical point of view, but if I pass a greyscale image instead of color, it takes way more to calculate. Which makes me think I'm not doing something correctly in one of the two cases.
I read around that when the change in color space is not linear (which I suppose is the case when you go from 4 channels to 1 like in this case), you should normalize the intensity value. Is that correct? Which input should I give to this function?
Another note, the function is called like this in my code:
vector<Point2f>imgpts1, imgpts2;
for (vector<DMatch>::const_iterator it = matches.begin(); it!= matches.end(); ++it)
{
imgpts1.push_back(firstViewFeatures.second[it->queryIdx].pt);
imgpts2.push_back(secondViewFeatures.second[it->trainIdx].pt);
}
Mat mask;
Mat E = findEssentialMat(imgpts1, imgpts2, [camera focal], [camera principal_point], CV_RANSAC, 0.999, 1, mask);
The fact I'm not passing a Mat, but a vector of Point2f instead, seems to create no problems, as it compiles and executes properly.
Is it the case I should store the matches in a Mat?
I am no sure do you mean by vector of Point2f in some color space, but if you want to convert vector of points into vector of points of another type you can use any standard C++/STL function like copy(), assign() or insert(). For example:
copy(floatPoints.begin(), floatPoints.end(), doublePoints.begin());
or
doublePoints.insert(doublePoints.end(), floatPoints.begin(), floatPoints.end());
No, it is not. A std::vector<cv::Pointf2f> cannot make use of the OpenCV convertTo function.
I think you really mean that you have a cv::Mat points1 of type CV_8UC4. Note that those are RxCx4 values (being R and C the number of rows and columns), and that in a CV_64F matrix you will have RxC values only. So, you need to be more clear on how you want to transform those values.
You can do points1.convertTo(points1, CV_64FC4) to get a RxCx4 matrix.
Update:
Some remarks after you updated the question:
Note that a vector<cv::Point2f> is a vector of 2D points that is not associated to any particular color space, they are just coordinates in the image axes. So, they represent the same 2D points in a grey, rgb or hsv image. Then, the execution time of findEssentialMat doesn't depend on the image color space. Getting the points may, though.
That said, I think your input for findEssentialMat is ok (the function takes care of the vectors and convert them into their internal representation). In this cases, it is very useful to draw the points in your image to debug the code.

Pixel color inside a contour using numpy

I am trying to constitute a numpy array containing the color hue of each pixel within a contour, using opencv 2.4. I have extracted the coordinates of all point included inside the contour using pixelpoints = np.transpose(np.nonzero(mask)) (format: N x 2, where N is the number of pixels inside the contour) just as here, and I extracted the hue of all pixels within the image using cv2.split(image) (format: 480 , 640). So I need to obtain an array containing elements (xcoord, ycoord, hue) to 3D plot the hue colormap. Unfortunately, I am not a numpy expert, and do not find what I need in the documentation. Could someone please help? Please find below an example of what I wish to achieve.
import numpy as np
a=range(35,135)
hue=np.reshape(a,(10,10))
pixelpoints=np.array([[5,5],[5,6],[5,7],[6,5],[6,6],[6,7],[7,5],[7,6],[7,7]])
print hue
print pixelpoints
result=np.array([[5,5,90],[5,6,91],[5,7,92],[6,5,100],[6,6,101],[6,7,102],[7,5,110],[7,6,111],[7,7,112]])
print result
For all who wonder, the answer has been given here.
np.c_[pixelpoints, hue[tuple(pixelpoints.T)]]

image transformations

So I've been using gnu-gsl and CImg to implement some of the fundamental projective space techniques for affine and metric rectification.
I've completed computing an affine rectification but, I'm having a hard time figuring out how to apply the affine rectification matrix to the original (input) image.
My current thought process is to iterate across the input image for each pixel coordinate. Then multiply the original pixel coordinate (converted to a homogeneous coordinate) by the affine rectification matrix to get the output pixel coordinate.
Then access the output image using the output pixel coordinate and conduct a blend (addition) operation on the output image's pixel location with the pixel color from the original image.
Does that sound right? I'm getting a lot of really weird values after multiplying the original pixel coordinate by the affine rectification matrix.
No, your values should not be weird. Why don't you make a simple example, a small scale with a small translation; e.g.
x' = 1.01*x + 0.0*y + 5;
y' = 0.0*x + 0.98*y + 10;
Now the pixel at (10,10) should map to (15.1,19.8), right ?
If you want to make a nice output image, you should find the forward projection and then back project to the input image and interpolate there rather than try to blend into the output image. Otherwise you will end up with gaps in the output.
You need to be careful with your terminology here; it sounds to me like you are doing projections, sometimes called warping in the computer graphics community. Rectification is something else, but it depends on what you are doing.

Resources