Related
I have been working on a code where an image is given as shown
I have to place this knife onto some other image. The condition is that I have to crop the knife along its outline and not in a rectangular box.
import numpy as np
import cv2
from matplotlib import pyplot as plt
img = cv2.imread('2.jpg')
img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
plt.imshow(img)
img_blur = cv2.bilateralFilter(img, d = 7,
sigmaSpace = 75, sigmaColor =75)
img_gray = cv2.cvtColor(img_blur, cv2.COLOR_RGB2GRAY)
a = img_gray.max()
_, thresh = cv2.threshold(img_gray, a/2+60, a,cv2.THRESH_BINARY_INV)
plt.imshow(thresh, cmap = 'gray')
contours, hierarchy = cv2.findContours(
image = thresh,
mode = cv2.RETR_TREE,
method = cv2.CHAIN_APPROX_SIMPLE)
contours = sorted(contours, key = cv2.contourArea, reverse = True)
img_copy = img.copy()
final = cv2.drawContours(img_copy, contours, contourIdx = -1,
color = (255, 0, 0), thickness = 2)
plt.imshow(img_copy)
This is what I have tried but it doesn't seem to work well.
Input
Output
You can do it starting with bounding box using snake algorithm with balloon force added.
Snake's algo is defined such that it minimizes 3 energies - Continuity, Curvature and Gradient. The first two (together called internal energy) get minimized when points (on curve) are pulled closer and closer i.e. contract. If they expand then energy increases which is not allowed by snake algorithm.
But this initial algo proposed in 1987 has a few problems. One of the problem is that in flat areas (where gradient is zero) algo fails to converge and does nothing. There are several modifications proposed to solve this problem. The solution of interest here is - Balloon Force proposed by LD Cohen in 1989.
Balloon force guides the contour in non-informative areas of the image, i.e., areas where the gradient of the image is too small to push the contour towards a border. A negative value will shrink the contour, while a positive value will expand the contour in these areas. Setting this to zero will disable the balloon force.
Another improvement is - Morphological Snakes which use morphological operators (such as dilation or erosion) over a binary array instead of solving PDEs over a floating point array, which is the standard approach for active contours. This makes Morphological Snakes faster and numerically more stable than their traditional counterpart.
Scikit-image's implementation using the above two improvements is morphological_geodesic_active_contour. It has a parameter balloon
Using your image
import numpy as np
import matplotlib.pyplot as plt
from skimage.segmentation import morphological_geodesic_active_contour, inverse_gaussian_gradient
from skimage.color import rgb2gray
from skimage.util import img_as_float
from PIL import Image, ImageDraw
im = Image.open('knife.jpg')
im = np.array(im)
im = rgb2gray(im)
im = img_as_float(im)
plt.imshow(im, cmap='gray')
Now let us create a function which will help us to store iterations:
def store_evolution_in(lst):
"""Returns a callback function to store the evolution of the level sets in
the given list.
"""
def _store(x):
lst.append(np.copy(x))
return _store
This method needs image to be preprocessed to highlight the contours. This can be done using the function inverse_gaussian_gradient, although the user might want to define their own version. The quality of the MorphGAC segmentation depends greatly on this preprocessing step.
gimage = inverse_gaussian_gradient(im)
Below we define our starting point - a bounding box.
init_ls = np.zeros(im.shape, dtype=np.int8)
init_ls[200:-400, 20:-30] = 1
List with intermediate results for plotting the evolution
evolution = []
callback = store_evolution_in(evolution)
Now required magic line for morphological_geodesic_active_contour with balloon contraction is below:
ls = morphological_geodesic_active_contour(gimage, 200, init_ls,
smoothing=1, balloon=-0.75,
threshold=0.7,
iter_callback=callback)
Now let us plot the results:
fig, axes = plt.subplots(1, 2, figsize=(8, 8))
ax = axes.flatten()
ax[0].imshow(im, cmap="gray")
ax[0].set_axis_off()
ax[0].contour(ls, [0.5], colors='b')
ax[0].set_title("Morphological GAC segmentation", fontsize=12)
ax[1].imshow(ls, cmap="gray")
ax[1].set_axis_off()
contour = ax[1].contour(evolution[0], [0.5], colors='r')
contour.collections[0].set_label("Starting Contour")
contour = ax[1].contour(evolution[25], [0.5], colors='g')
contour.collections[0].set_label("Iteration 25")
contour = ax[1].contour(evolution[-1], [0.5], colors='b')
contour.collections[0].set_label("Last Iteration")
ax[1].legend(loc="upper right")
title = "Morphological GAC Curve evolution"
ax[1].set_title(title, fontsize=12)
plt.show()
With more balloon force you can get only the blade of knife as well.
ls = morphological_geodesic_active_contour(gimage, 100, init_ls,
smoothing=1, balloon=-1,
threshold=0.7,
iter_callback=callback)
Play with these parameters - smoothing, balloon, threshold to get your perfect curve
How to locate the pupil(the small circle in the eyeball) in the following picture and calculate the area of the pupil. I tried Hough circle detection and ellipse fit on contours with various threshold but none of these naive approaches work very well.
Specifically, HoughCircle detection is totally lost in many noises, and Ellipse detection with pruning often end up giving the larger circle.
And I'm unsure how to determine a functional threshold without manual adjustment on a trackbar. Can somebody give me some guidance on how to accurately do this?
Eyeball_sample
A simple image processing process should help you achieve your goal.
First load the image in grayscale. I think Otsu threshold method is robust enough to extract the pupil region of the eye. Extra morphological process is required to remove noise and unfilled regions
Then using a connected component analysis, we can isolate the pupil region for further processing.
With this region, we can get the edge by subtracting a dilated region with the original region as shown below.
Finally, we can run a circle fitting or a ellipse fitting algorithm to obtain the corresponding shape,
Circle fitting is shown in red wherelse ellipse is shown in green. Both return the same center position albeit slightly different shape.
Here is the code used. I shrink the image to speed up the process but it will work the same using the original size.
import cv2
import numpy as np
img = cv2.imread('eye.jpg',0)
small_img = cv2.resize(img,(0,0),fx = 0.25, fy = 0.25)
r,c = small_img.shape
# Threshold objs
_, thresh = cv2.threshold(small_img,0,255,cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)
# Morphological close process to cluster nearby objects
bin_img = cv2.dilate(thresh, None,iterations = 5)
bin_img = cv2.erode(bin_img, None,iterations = 5)
# Analyse connected components
num_labels, labels, stats, centroids = cv2.connectedComponentsWithStats(bin_img)
# Find circle center and radius
# Radius calculated by averaging the height and width of bounding box
bin_eye = np.zeros((r,c))
cnt_label = labels[r/2,c/2]
bin_eye[labels == cnt_label] = 255
area = stats[cnt_label][4]
radius = np.sqrt(area / np.pi)
cnt_pt = ((centroids[cnt_label][0]),(centroids[cnt_label][1]))
# fit ellipse
bin_eye_large = cv2.dilate(bin_eye, None,iterations = 1)
# Get ellipse edge
edge_eye = bin_eye_large - bin_eye
# extract location points for fitting
ellip_pts = np.where(edge_eye > 0)
ellip_pts = np.transpose(ellip_pts)
temp = np.copy(ellip_pts[:,0])
ellip_pts[:,0] = ellip_pts[:,1]
ellip_pts[:,1] = temp
# fit ellipse
ellip = cv2.fitEllipse(ellip_pts)
# Display final result
edges_color = cv2.cvtColor(small_img,cv2.COLOR_GRAY2BGR)
cv2.circle(edges_color,(int(cnt_pt[0]),int(cnt_pt[1])),int(radius),(0,0,255),1)
cv2.circle(edges_color,(int(cnt_pt[0]),int(cnt_pt[1])),5,(0,0,255),1)
cv2.ellipse(edges_color,ellip, (0,255,0))
cv2.circle(edges_color,(int(ellip[0][0]),int(ellip[0][1])),5,(0,255,0),1)
cv2.imshow('edges_color',edges_color)
cv2.imshow('bin_img',bin_img)
cv2.imshow('eye_label',bin_eye)
cv2.imshow('eye_edge',edge_eye)
cv2.waitKey(0)
I want to measure the distance between two points using scikit-image. Here is the image:
In the above photo I want to measure the distance between the red point and black point. The unit of measurement does not matter to me as I want to normalize the distance by the end of the day. Any idea how I can do it?
Thanks
You could get the job done through the following stepwise procedure:
Compute distance from each pixel to red and black colors.
Binarize using an appropriate threshold.
Perform morphological closing.
Determine the centroid coordinates of the resulting blobs.
Calculate the distance between centroids.
Hopefully the code below will put you on the right track:
import numpy as np
from skimage import io
from skimage.morphology import closing
from skimage.measure import regionprops
import matplotlib.pyplot as plt
from matplotlib.patches import ConnectionPatch
img = io.imread('https://i.stack.imgur.com/vbOmy.jpg')
red = [255, 0, 0]
black = [0, 0, 0]
threshold = 10
dist_from_red = np.linalg.norm(img - red, axis=-1)
dist_from_black = np.linalg.norm(img - black, axis=-1)
red_blob = closing(dist_from_red < threshold)
black_blob = closing(dist_from_black < threshold)
labels = np.zeros(shape=img.shape[:2], dtype=np.ubyte)
labels[black_blob] = 1
labels[red_blob] = 2
blobs = regionprops(labels)
center_0 = np.asarray(blobs[0].centroid[::-1])
center_1 = np.asarray(blobs[1].centroid[::-1])
dist = np.linalg.norm(center_0 - center_1)
Demo
fig, ax = plt.subplots(1, 1)
ax.imshow(img)
con = ConnectionPatch(xyA=center_0, xyB=center_1,
coordsA='data', arrowstyle="-|>", ec='yellow')
ax.add_artist(con)
plt.annotate('Distance = {:.2f}'.format(dist),
xy=(center_0 + center_1)/2, xycoords='data',
xytext=(0.5, 0.7), textcoords='figure fraction', color='blue',
arrowprops=dict(arrowstyle="->", color='blue'))
plt.show(fig)
How can I calculate distance from camera to a point on a ground plane from an image?
I have the intrinsic parameters of the camera and the position (height, pitch).
Is there any OpenCV function that can estimate that distance?
You can use undistortPoints to compute the rays backprojecting the pixels, but that API is rather hard to use for your purpose. It may be easier to do the calculation "by hand" in your code. Doing it at least once will also help you understand what exactly that API is doing.
Express your "position (height, pitch)" of the camera as a rotation matrix R and a translation vector t, representing the coordinate transform from the origin of the ground plane to the camera. That is, given a point in ground plane coordinates Pg = [Xg, Yg, Zg], its coordinates in camera frame are given by
Pc = R * Pg + t
The camera center is Cc = [0, 0, 0] in camera coordinates. In ground coordinates it is then:
Cg = inv(R) * (-t) = -R' * t
where inv(R) is the inverse of R, R' is its transpose, and the last equality is due to R being an orthogonal matrix.
Let's assume, for simplicity, that the the ground plane is Zg = 0.
Let K be the matrix of intrinsic parameters. Given a pixel q = [u, v], write it in homogeneous image coordinates Q = [u, v, 1]. Its location in camera coordinates is
Qc = Ki * Q
where Ki = inv(K) is the inverse of the intrinsic parameters matrix. The same point in world coordinates is then
Qg = R' * Qc + Cg
All the points Pg = [Xg, Yg, Zg] that belong to the ray from the camera center through that pixel, expressed in ground coordinates, are then on the line
Pg = Cg + lambda * (Qg - Cg)
for lambda going from 0 to positive infinity. This last formula represents three equations in ground XYZ coordinates, and you want to find the values of X, Y, Z and lambda where the ray intersects the ground plane. But that means Zg=0, so you have only 3 unknowns. Solve them (you recover lambda from the 3rd equation, then substitute in the first two), and you get Xg and Yg of the solution to your problem.
Hello to everyone. The above image is sum of two images in which i did feature matching and draw all matching points. I also found the contours of the pcb parts in the first image (half left image-3 contours). The question is, how could i draw only the matching points that is inside those contours in the first image instead this blue mess? I'm using python 2.7 and opencv 2.4.12.
I wrote a function for draw matches cause in opencv 2.4.12 there isn't any implemented method for that. If i didn't include something please tell me. Thank you in advance!
import numpy as np
import cv2
def drawMatches(img1, kp1, img2, kp2, matches):
# Create a new output image that concatenates the two images
# (a.k.a) a montage
rows1 = img1.shape[0]
cols1 = img1.shape[1]
rows2 = img2.shape[0]
cols2 = img2.shape[1]
# Create the output image
# The rows of the output are the largest between the two images
# and the columns are simply the sum of the two together
# The intent is to make this a colour image, so make this 3 channels
out = np.zeros((max([rows1,rows2]),cols1+cols2,3), dtype='uint8')
# Place the first image to the left
out[:rows1,:cols1] = np.dstack([img1, img1, img1])
# Place the next image to the right of it
out[:rows2,cols1:] = np.dstack([img2, img2, img2])
# For each pair of points we have between both images
# draw circles, then connect a line between them
for mat in matches:
# Get the matching keypoints for each of the images
img1_idx = mat.queryIdx
img2_idx = mat.trainIdx
# x - columns
# y - rows
(x1,y1) = kp1[img1_idx].pt
(x2,y2) = kp2[img2_idx].pt
# Draw a small circle at both co-ordinates
# radius 4
# colour blue
# thickness = 1
cv2.circle(out, (int(x1),int(y1)), 4, (255, 0, 0), 1)
cv2.circle(out, (int(x2)+cols1,int(y2)), 4, (255, 0, 0), 1)
# Draw a line in between the two points
# thickness = 1
# colour blue
cv2.line(out, (int(x1),int(y1)), (int(x2)+cols1,int(y2)), (255,0,0), 1)
# Show the image
cv2.imshow('Matched Features', out)
cv2.imwrite("shift_points.png", out)
cv2.waitKey(0)
cv2.destroyWindow('Matched Features')
# Also return the image if you'd like a copy
return out
img1 = cv2.imread('pic3.png', 0) # Original image - ensure grayscale
img2 = cv2.imread('pic1.png', 0) # Rotated image - ensure grayscale
sift = cv2.SIFT()
# find the keypoints and descriptors with SIFT
kp1, des1 = sift.detectAndCompute(img1,None)
kp2, des2 = sift.detectAndCompute(img2,None)
# Create matcher
bf = cv2.BFMatcher()
# Perform KNN matching
matches = bf.knnMatch(des1, des2, k=2)
# Apply ratio test
good = []
for m,n in matches:
if m.distance < 0.75*n.distance:
# Add first matched keypoint to list
# if ratio test passes
good.append(m)
# Show only the top 10 matches - also save a copy for use later
out = drawMatches(img1, kp1, img2, kp2, good)
Based on what you are asking I am assuming you mean you have some sort of closed contour outlining the areas you want to bound your data point pairs to.
This is fairly simple for polygonal contours and more math is required for more complex curved lines but the solution is the same.
You draw a line from the point in question to infinity. Most people draw out a line to +x infinity, but any direction works. If there are an odd number of line intersections, the point is inside the contour.
See this article:
http://www.geeksforgeeks.org/how-to-check-if-a-given-point-lies-inside-a-polygon/
For point pairs, only pairs where both points are inside the contour are fully inside the contour. For complex contour shapes with concave sections, if you also want to test that the linear path between the points does not cross the contour, you perform a similar test with just the line segment between the two points, if there are any line intersections the direct path between the points crosses outside the contour.
Edit:
Since your contours are rectangles, a simpler approach will suffice for determining if your points are inside the rectangle.
If your rectangles are axis aligned (they are straight and not rotated), then you can use your values for top,left and bottom,right to check.
Let point A = Top,Left, point B = Bottom,Right, and point C = your test point.
I am assuming an image based coordinate system where 0,0 is the left,top of the image, and width,height is the bottom right. (I'm writing in C#)
bool PointIsInside(Point A, Point B, Point C)
{
if (A.X <= C.X && B.X >= C.X && A.Y <= C.Y && B.Y >= C.Y)
return true;
return false;
}
if your rectangle is NOT axis aligned, then you can perform four half-space tests to determine if your point is inside the rectangle.
Let Point A = Top,Left, Point B = Bottom,Right, double W = Width, double H = Height, double N = rotation angle, and Point C = test point.
for an axis aligned rectangle, Top,Right can be calculated by taking the vector (1,0) , multiplying by Width, and adding that vector to Top,Left. For Bottom,Right We take the vector (0,1), multiply by height, and add to Top,Right.
(1,0) is the equivalent of a Unit Vector (length of 1) at Angle 0. Similarly, (0,1) is a unit vector at angle 90 degrees. These vectors can also be considered the direction the line is pointing. This also means these same vectors can be used to go from Bottom,Left to Bottom,Right, and from Top,Left to Bottom,Left as well.
We need to use different unit vectors, at the angle provided. To do this we simply need to take the Cosine and Sine of the angle provided.
Let Vector X = direction from Top,Left to Top,Right, Vector Y = direction from Top,Right to Bottom,Right.
I am using angles in degrees for this example.
Vector X = new Vector();
Vector Y = new Vector();
X.X = Math.Cos(R);
X.Y = Math.Sin(R);
Y.X = Math.Cos(R+90);
Y.Y = Math.Sin(R+90);
Since we started with Top,Left, we can find Bottom,Right by simply adding the two vectors to Top,Left
Point B = new Point();
B = A + X + Y;
We now want to do a half-space test using the dot product for our test point. The first two test will use the test point, and Top,Left, the other two will use the test point, and Bottom,Right.
The half-space test is inherently based on directionality. Is the point in front, behind, or perpendicular to a given direction? We have the two directions we need, but they are directions based on the top,left point of the rectangle, not the full space of the image, so we need to get a vector from the top,left, to the point in question, and another from the bottom, right, since those are the two points we test against.
This is simple to calculate, as it is just Destination - Origin.
Let Vector D = Top,Left to test point C, and Vector E = Bottom,Right to test point.
Vector D = C - A;
Vector E = C - B;
The dot product is x1 * x2 + y1*y2 of the two vectors. if the result is positive, the two directions have an absolute angle of less than 90 degrees, or are roughly going in the same direction, a result of zero means they are perpendicular. In our case it means the test point is directly on a side of the rectangle we are testing against. Less than zero means an absolute angle of greater than 90 degrees, or they are roughly going opposite directions.
If a point is inside the rectangle, then the dot products from top left will be >= 0, while the dot products from bottom right will be <= 0. In essence the test point is closer to bottom right when testing from top left, but when taking the same directions when we are already at bottom right, it will be going away, back toward top,left.
double DotProd(Vector V1, Vector V2)
{
return V1.X * V2.X + V1.Y * V2.Y;
}
and so our test ends up as:
if( DotProd(X, D) >= 0 && DotProd(Y, D) >= 0 && DotProd(X, E) <= 0 && DotProd(Y, E) <= 0)
then the point is inside the rectangle. Do this for both points, if both are true then the line is inside the rectangle.