I’m working on this project where I’ve to separate some lines segments from others. I used Hough transform to detect these lines, however, I’m stuck on how I can extract only the lines I want. As you can see, in the image, I would like to extract the lines marked in red.
If someone has an idea of where I can find documentation that can help or provide some code help, I’ll be grateful.
I’ve provided my Hough transform code in case it can help with something.
img = cv2.imread('input.png')
lines_list = list()
if len(img.shape) == 3:
gray = cv2.cvtColor(img, cv2.COLOR_RGB2GRAY)
image_all = img.copy()
else:
gray = img.copy()
image_all = cv2.cvtColor(img,cv2.COLOR_GRAY2RGB)
edges = cv2.Canny(gray,50,150,apertureSize=3)
lines = cv2.HoughLinesP(
edges, # Input edge image
5, # Distance resolution in pixels
np.pi / 180, # Angle resolution in radians
150, # Min number of votes for valid line
np.array([]),
minLineLength = 90, # Min allowed length of line
maxLineGap = 40 # Max allowed gap between line for joining them
)
for points in lines:
# Extracted points nested in the list
x1,y1,x2,y2=points[0]
# Draw the lines joing the points On the original image
cv2.line(image_all,(x1,y1),(x2,y2),(0,100,255),4)
The final result should have only the lines marked in red.
Related
I'm trying to remove the grid lines in handwriting picture. I tried to use FFT to extract the grid pattern and remove it (this is from an answer in the original question, which is closed somehow. It has more background as well.). This image shows what I am able to get currently (Illustration result):
The first line is a real image with handwriting character. Since it's taken by phone in various conditions (light, direction, etc.), the grid line might not be perfect horizontal/vertical, and the color of grid line also varies and might be close the the color of characters. I turn it to grayscale, apply fft, and use tries to use thresholding to extract the patterns (in red rectangle, the illustration is using OTSU). Then I mask the image with the thresholding pattern, and use ifft to get the result. It fails on the real image obviously.
The second line is a real image of blank grid w/o handwriting character. From this, I think 3 lines (vertical and horizontal) in the center are the patterns I care.
The third line is a synthetic image w/ perfect grid lines. It's just for reference. And after applying the same algorithm, the grid lines could be removed successfully.
The fourth line is a synthetic image w/ perfect dashed grid lines, which is closer to the grid lines on real handwriting practice paper. It's also for reference. It shows the pattern of dashed lines are actually more complicated than 3 lines in the center. With the same algorithm, the grid lines could be removed almost completely as well.
The code I use is:
def FFTCV(img):
util.Plot(img, 'Input')
print(img.shape)
if len(img.shape) == 3 and img.shape[2] == 3:
img = cv.cvtColor(img, cv.COLOR_BGR2GRAY)
util.Plot(img, 'Gray')
dft = cv.dft(np.float32(img),flags = cv.DFT_COMPLEX_OUTPUT)
dft_shift = np.fft.fftshift(dft)
util.Plot(cv.magnitude(dft_shift[:,:,0],dft_shift[:,:,1]), 'fft shift')
magnitude_spectrum = np.uint8(20*np.log(cv.magnitude(dft_shift[:,:,0],dft_shift[:,:,1])))
util.Plot(magnitude_spectrum, 'Magnitude')
_, threshold = cv.threshold(magnitude_spectrum, 0, 1, cv.THRESH_BINARY_INV + cv.THRESH_OTSU)
# threshold = cv.adaptiveThreshold(
# magnitude_spectrum, 1, cv.ADAPTIVE_THRESH_MEAN_C, cv.THRESH_BINARY_INV, 11, 10)
# magnitude_spectrum, 1, cv.ADAPTIVE_THRESH_GAUSSIAN_C, cv.THRESH_BINARY_INV, 11, 10)
util.Plot(threshold, 'Threshold Mask')
fshift = dft_shift * threshold[:, :, None]
util.Plot(cv.magnitude(fshift[:,:,0],fshift[:,:,1]), 'fft shift Masked')
magnitude_spectrum = np.uint8(20*np.log(cv.magnitude(fshift[:,:,0],fshift[:,:,1])))
util.Plot(magnitude_spectrum, 'Magnitude Masked')
f_ishift = np.fft.ifftshift(fshift)
img_back = cv.idft(f_ishift)
img_back = cv.magnitude(img_back[:,:,0],img_back[:,:,1])
util.Plot(img_back, 'Back')
So I'd like to learn suggestions on how to extract the patterns for real images. Thanks very much.
Background
I've defined a tennis court with 3D points, and I'm trying to use OpenCV to project the 3D points as a 2D image.
Below are the 3D points I'm using to define the tennis court. For reference:
Left on the court is -X and Right is +X,
Up is the -Y and Down is +Y, and
Far side of net is +Z and Close side is -Z.
Point (0,0,0) is the intersection of the left side line with the net line (basically the left net post).
def DefineCourtPoints():
objp = {}
objp[1] = [0,0,39] # 1. Intersection of the upper base line with the left side line
objp[2] = [0,0,-39] # 2. Intersection of the lower base line with the left side line
objp[3] = [36,0,-39] # 3. Intersection of the lower base line with the right side line
objp[4] = [36,0,39] # 4. Intersection of the upper base line with the right side line
objp[5] = [4.5,0,39] # 5. Intersection of the upper base line with the left singles line
objp[6] = [4.5,0,-39] # 6. Intersection of the lower base line with the left singles line
objp[7] = [31.5,0,-39] # 7. Intersection of the lower base line with the right singles line
objp[8] = [31.5,0,39] # 8. Intersection of the upper base line with the right singles line
objp[9] = [4.5,0,21] # 9. Intersection of the left singles line with the upper service line
objp[10] = [31.5,0,21] # 10. Intersection of the right singles line with the upper service line
objp[11] = [4.5,0,-21] # 11. Intersection of the left singles line with the lower service line
objp[12] = [31.5,0,-21] # 12. Intersection of the right singles line with the lower service line
objp[13] = [18,0,21] # 13. Intersection of the upper service line with the center service line
objp[14] = [18,0,-21] # 14. Intersection of the lower service line with the center service line
objp[15] = [0,0,0] # 15. Intersection of the left side line with the net line (this will be considered (0,0) )
objp[16] = [36,0,0] # 16. Intersection of the right side line with the net line
return objp
For additional reference, here are the intrinsic matrix and distortion parameters for my camera (which were obtained using OpenCV and have been validated):
intrinsic_mtx = np.array([
[1882.77177, 0.0, 973.572122],
[0.0, 1880.83035, 537.299982],
[0.0, 0.0, 1.0]
])
distortion = np.array([0.232714433, -1.35119878, -0.00188551612, 0.00166434182, 2.50351701])
Here is the code I use to project each of the 3D points to 2D, and to graph the points/lines using Matplotlib.
def ProjectPoints(intrinsic_mtx, distortion, R, T, objp)
x_points = []
y_points = []
# Loop through each of the 3D points and project them to 2D.
for index in objp:
2d_point = cv2.projectPoints(
np.array(objp[index], dtype=np.float64),
cv2.Rodrigues(R)[0],
T,
intrinsic_mtx,
distortion
)[0][0][0]
print(2d_point)
x_points.append(2d_point[0])
y_points.append(2d_point[1])
# Graph the court boundary.
lines = [(1,2), # A line exists between point 1 and 2...
(2,3), # A line exists between point 2 and 3...
(3,4), # A line exists between point 3 and 4...
(4,1)] # A line exists between point 4 and 1...
x_lines = list()
y_lines = list()
for pair in lines:
for i in range(2):
x_lines.append(x_points[pair[i]-1])
y_lines.append(y_points[pair[i]-1])
# Append None to separate the lines.
x_lines.append(None)
y_lines.append(None)
fig, ax = plt.subplots()
ax.scatter(x_points, y_points)
ax.axis('equal')
ax.plot(x_lines, y_lines, c = 'r')
plt.show()
Problem
When I project the points and graph them, I expect to get something looking like a tennis court, but I just get a blob of points!
To troubleshoot, I decided to graph just outer edges of the court using the four corners of the court (which are the first four points in DefineCourtPoints). I expected to see a rectangle or parallelogram as these are valid projections of a rectangle. However, I get stuff like the following, which doesn't really make any sense since it has crossed lines:
The above result was obtained using R = [0, 0, 0]
and a T = [0, -10, 0]
Here are the coordinates of the four points for the above result, showing that I didn't cause the weird shape by somehow graphing the lines in the wrong order:
# Intersection of the upper base line with the left side line
[973.572122 , 1019.56417431]
# Intersection of the lower base line with the left side
line
[973.572122 , 55.03578969]
# Intersection of the lower base line with the right side
line
[-764.37105031, 55.03578969]
# Intersection of the upper base line with the right side line
[2711.51529431 , 1019.56417431]
Question
Why am I getting such weird projections for such a simple 3D object as a rectangle/tennis court boundary?
Is anyone getting the same results that I'm getting?
Thanks!
Try it yourself
import numpy as np
import cv2
import matplotlib.pyplot as plt
def DefineCourtPoints(): # Just the corners of the court.
objp = {}
objp[1] = [0,0,39] # 1. Intersection of the upper base line with the left side line
objp[2] = [0,0,-39] # 2. Intersection of the lower base line with the left side line
objp[3] = [36,0,-39] # 3. Intersection of the lower base line with the right side line
objp[4] = [36,0,39] # 4. Intersection of the upper base line with the right side line
objp = DefineCourtPoints()
intrinsic_mtx = np.array([
[1882.77177, 0.0, 973.572122],
[0.0, 1880.83035, 537.299982],
[0.0, 0.0, 1.0]
])
distortion = np.array([0.232714433, -1.35119878, -0.00188551612, 0.00166434182, 2.50351701])
R = np.array([0,0,0])
T = np.array([0,-10,0])
ProjectPoints(intrinsic_mtx, distortion, R, T, objp)
I have two images and I need to place the second image inside the first image. The second image can be resized, rotated or skewed such that it covers a larger area of the other images as possible. As an example, in the figure shown below, the green circle need to be placed inside the blue shape:
Here the green circle is transformed such that it covers a larger area. Another example is shown below:
Note that there may be some multiple results. However, any similar result is acceptable as shown in the above example.
How do I solve this problem?
Thanks in advance!
I tested the idea I mentioned earlier in the comments and the output is almost good. It may be better but it takes time. The final code was too much and it depends on one of my old personal projects, so I will not share. But I will explain step by step how I wrote such an algorithm. Note that I have tested the algorithm many times. Not yet 100% accurate.
for N times do this:
1. Copy from shape
2. Transform it randomly
3. Put the shape on the background
4-1. It is not acceptable if the shape exceeds the background. Go to
the first step.
4.2. Otherwise we will continue to step 5.
5. We calculate the length, width and number of shape pixels.
6. We keep a list of the best candidates and compare these three
parameters (W, H, Pixels) with the members of the list. If we
find a better item, we will save it.
I set the value of N to 5,000. The larger the number, the slower the algorithm runs, but the better the result.
You can use anything for Transform. Mirror, Rotate, Shear, Scale, Resize, etc. But I used warpPerspective for this one.
im1 = cv2.imread(sys.path[0]+'/Back.png')
im2 = cv2.imread(sys.path[0]+'/Shape.png')
bH, bW = im1.shape[:2]
sH, sW = im2.shape[:2]
# TopLeft, TopRight, BottomRight, BottomLeft of the shape
_inp = np.float32([[0, 0], [sW, 0], [sW, sH], [0, sH]])
cx = random.randint(5, sW-5)
ch = random.randint(5, sH-5)
o = 0
# Random transformed output
_out = np.float32([
[random.randint(-o, cx-1), random.randint(1-o, ch-1)],
[random.randint(cx+1, sW+o), random.randint(1-o, ch-1)],
[random.randint(cx+1, sW+o), random.randint(ch+1, sH+o)],
[random.randint(-o, cx-1), random.randint(ch+1, sH+o)]
])
# Transformed output
M = cv2.getPerspectiveTransform(_inp, _out)
t = cv2.warpPerspective(shape, M, (bH, bW))
You can use countNonZero to find the number of pixels and findContours and boundingRect to find the shape size.
def getSize(msk):
cnts, _ = cv2.findContours(msk, cv2.RETR_TREE, cv2.CHAIN_APPROX_NONE)
cnts.sort(key=lambda p: max(cv2.boundingRect(p)[2],cv2.boundingRect(p)[3]), reverse=True)
w,h=0,0
if(len(cnts)>0):
_, _, w, h = cv2.boundingRect(cnts[0])
pix = cv2.countNonZero(msk)
return pix, w, h
To find overlaping of back and shape you can do something like this:
make a mask from back and shape and use bitwise methods; Change this section according to the software you wrote. This is just an example :)
mskMix = cv2.bitwise_and(mskBack, mskShape)
mskMix = cv2.bitwise_xor(mskMix, mskShape)
isCandidate = not np.any(mskMix == 255)
For example this is not a candidate answer; This is because if you look closely at the image on the right, you will notice that the shape has exceeded the background.
I just tested the circle with 4 different backgrounds; And the results:
After 4879 Iterations:
After 1587 Iterations:
After 4621 Iterations:
After 4574 Iterations:
A few additional points. If you use a method like medianBlur to cover the noise in the Background mask and Shape mask, you may find a better solution.
I suggest you read about Evolutionary Computation, Metaheuristic and Soft Computing algorithms for better understanding of this algorithm :)
I have been working on performing camera calibration using ChAruCo boards.
Following the code here (my commented version is shown below), it appears that only every other image is used when performing the camera calibration - due to the decimator.
What could be a reason for this? Other than to save processing power, which seems unnecessary since this step is only performed once.
def read_chessboards(chessboard_images):
# Charuco base pose estimation.
print("POSE ESTIMATION STARTS:")
# Declare lists to store corner locations and IDs
allCorners = []
allIds = []
decimator = 0
# SUB PIXEL CORNER DETECTION CRITERION
criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 100, 0.00001)
# for each of the chessboard images
for im in chessboard_images:
print("=> Processing image {0}".format(im))
frame = cv2.imread(im) # read current image into frame variable
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY) # convert to grayscale
corners, ids, rejectedImgPoints = cv2.aruco.detectMarkers(gray, ARUCO_DICT) # detect markers present in image
# if there are any markers detected
if len(corners) > 0:
# SUB PIXEL DETECTION
for corner in corners:
# refine corner locations
# TODO: check if this works
cv2.cornerSubPix(gray, corner,
winSize=(3, 3),
zeroZone=(-1, -1),
criteria=criteria)
# interpolate position of ChArUco board corners.
res2 = cv2.aruco.interpolateCornersCharuco(corners, ids, gray, board)
print(f'Charuco corners at: {res2}')
# if 3+ corners are detected, add to allCorners list for every other image
if res2[1] is not None and res2[2] is not None and len(res2[1]) > 3 and decimator % 1 == 0:
allCorners.append(res2[1])
allIds.append(res2[2])
# why only every other chessboard image?
decimator += 1
imsize = gray.shape
return allCorners, allIds, imsize
Just realized that x % 1 always evaluates to 0, so it doesn't actually do anything. I guess it was included as an optional feature - if you change 1 to some other number.
I want to detect all rectangles in image and I use findContours in OpenCv , and I want to delete unnecessary shapes that have been identified by FindContours.
My image https://i.stack.imgur.com/eLb1s.png
My result: https://i.stack.imgur.com/xQqeF.png
My code:
img =cv2.imread('CD/A.png')
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
edges = cv2.Canny(gray,50,150,apertureSize = 3)
img1=np.ones(img.shape, dtype=np.uint8)*255
ret,thresh = cv2.threshold(gray,127,255,1)
(_,contours,h) = cv2.findContours(thresh,1,2)
for cnt in contours:
approx = cv2.approxPolyDP(cnt,0.01*cv2.arcLength(cnt,True),True)
if len(approx)==4:
cv2.drawContours(img1,[cnt],0,(0,255,0),2)
cv2.imshow('Detected line',img1)
cv2.waitKey(0)
cv2.destroyAllWindows()
I want to remove these extreme lines that exist within the rectangles :
https://i.stack.imgur.com/n9byP.png
Need your help guys .
One thing you could do is find the connected components and remove the ones that are small:
from skimage.morphology import label
import numpy as np
comps = label(thresh) # get the label map of the connected components
# The comps array will have a unique integer for each connected component
# and 0 for the background. np.unique gets the unique label values.
#
# Therefore, this loop allows us to pluck out each component from the image
for i in range(1, len(np.unique(comps))):
# comps == i will convert the array into True (1) if that pixel is in the
# i-th component and False (0) if it is not.
#
# Therefore, np.sum(comps == i) returns the "area" of the component
if np.sum(comps == i) < small_number:
# If the area is less than some number of pixels,
# set the pixels of this component to 0 in the thresholded image
thresh[comps == i] = 0
You can do the labeling with OpenCV as well with connectedComponentsWithStats or something like that but I'm more familiar with skimage.
If you can convert your image into a binary image (with a simple threshold), you can perform a morphological open operation which can help you filter out small lines in your image within the rectangle and then find contours again on the new image.
https://docs.opencv.org/trunk/d9/d61/tutorial_py_morphological_ops.html