Maybe someone has ideas how can we fill black pixels to white on the white digits and make that image more adopted to be recognizable
I was trying the Gaussian Blur with Kernel size (1,1), but it doesn't help effectively, sometimes digits on image become merged and this is much worst result
You may use the equivalent to MATLAB imfill, but the result is going to be binary image.
I found a Python implementation for imfill here (it uses Scikit-image).
Here is the code:
import cv2
import numpy as np
from skimage.morphology import reconstruction
def imfill(img):
# https://stackoverflow.com/questions/36294025/python-equivalent-to-matlab-funciton-imfill-for-grayscale
# Use the matlab reference Soille, P., Morphological Image Analysis: Principles and Applications, Springer-Verlag, 1999, pp. 208-209.
# 6.3.7 Fillhole
# The holes of a binary image correspond to the set of its regional minima which
# are not connected to the image border. This definition holds for grey scale
# images. Hence, filling the holes of a grey scale image comes down to remove
# all minima which are not connected to the image border, or, equivalently,
# impose the set of minima which are connected to the image border. The
# marker image 1m used in the morphological reconstruction by erosion is set
# to the maximum image value except along its border where the values of the
# original image are kept:
seed = np.ones_like(img)*255
img[ : ,0] = 0
img[ : ,-1] = 0
img[ 0 ,:] = 0
img[ -1 ,:] = 0
seed[ : ,0] = 0
seed[ : ,-1] = 0
seed[ 0 ,:] = 0
seed[ -1 ,:] = 0
fill_img = reconstruction(seed, img, method='erosion')
return fill_img
img = cv2.imread('5375.jpg', cv2.IMREAD_GRAYSCALE) # Read image as grayscale
img_thresh = cv2.threshold(img, 0, 255, cv2.THRESH_OTSU)[1] # Convert to B/W
fill_img = imfill(img_thresh)
cv2.imshow('img', img)
cv2.imshow('fill_img', fill_img)
cv2.waitKey()
cv2.destroyAllWindows()
Result:
Note:
You may get the same result using cv2.findContours and drawContours, but you should apply findContours on img_thresh.
In case you want closer result to the original image, you may use closing morphological operation, and use 'fill_img' as a mask:
closed_img = cv2.morphologyEx(img, cv2.MORPH_CLOSE, np.ones((35, 35)))
closed_img[fill_img == 0] = 0 # Set to zero where fill_img is zero.
Result:
Related
Context
My goal is to detect PV modules on the dataset of infrared images taken by a drone. I want to improve the edge detection so my algorithm performs better. Detected and labelled modules are then used to train a neural network.
Dataset
I have several hundred images taken at different times and from different altitudes. I guess their quality is not perfect - the environmental conditions could be better, e.g:
altitude - sometimes the edges between modules are not the images could be taken from a lower altitude so the edges are better visible.
capture time - sometimes the background (grass) is very hot. Most likely the images could be taken late morning/early afternoon.
However, I have to stick to what I have.
As you can see sometimes (e.g. image_3) the "middle line" is hardly visible.
Code
Preprocessing below is based on project I found on Github. Standard preprocessing and Canny Edge detection is used.
import cv2
import numpy as np
def detect_edges():
# image_path = "data/stackoverflow/TEMP_DJI_1_R (715).JPG"
# image_path = "data/stackoverflow/TEMP_DJI_6_R (720).JPG"
image_path = "data/stackoverflow/TEMP_DJI_5_R (657).JPG"
# read image
input_image = cv2.imread(image_path, cv2.IMREAD_COLOR)
cv2.imshow('input_image', input_image)
# scale image
image_scaling = 11.0
scaled_image_rgb = cv2.resize(src=input_image, dsize=(0, 0), fx=image_scaling, fy=image_scaling)
cv2.imshow('scaled_image', scaled_image_rgb)
# blur image
gaussian_blur = 7
blurred_image = cv2.blur(scaled_image_rgb, (gaussian_blur, gaussian_blur))
cv2.imshow('blurred_image', blurred_image)
# gray image
grayed_image = cv2.cvtColor(scaled_image_rgb, cv2.COLOR_BGR2GRAY)
cv2.imshow('grayed_image', grayed_image)
# red threshold
red_threshold = 120
red_channel = scaled_image_rgb[:, :, 2]
_, thresholded_image = cv2.threshold(red_channel, red_threshold, 255, 0, cv2.THRESH_BINARY)
# dilation and erosion
kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (9, 9))
closing = cv2.morphologyEx(thresholded_image, cv2.MORPH_CLOSE, kernel)
opening = cv2.morphologyEx(closing, cv2.MORPH_OPEN, kernel)
# min area
min_area = 250 * 200
contours, hierarchy = cv2.findContours(opening, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
areas = [cv2.contourArea(contour) for contour in contours]
discarded_contours = [area < min_area for area in areas]
contours = [contours[i] for i in range(len(contours)) if not discarded_contours[i]]
mask = np.zeros_like(grayed_image)
cv2.drawContours(mask, contours, -1, (255), cv2.FILLED)
mask = cv2.dilate(mask, kernel, iterations=5)
mask = cv2.blur(mask, (25, 25))
mask = mask.astype(np.float) / 255.
preprocessed_image = (grayed_image * mask).astype(np.uint8)
cv2.imshow('preprocessed_image', preprocessed_image)
hysteresis_min_thresh = 25
hysteresis_max_thresh = 40
# canny edge
canny_image = cv2.Canny(image=preprocessed_image, threshold1=hysteresis_min_thresh,
threshold2=hysteresis_max_thresh, apertureSize=3)
cv2.imshow('canny_image', canny_image)
cv2.waitKey()
Results
The results are not bad, however they must be improved before further processing.
What kind of operations would be best to distinguish panels from the background (grass)?
In the case of the images with hardly visible "middle" lines (image_3), are there any chances of finding that "internal" edge? Maybe for these images, I should rather focus on finding outer edges only and draw an artificial line in the middle to divide the whole panel into two?
I would like to create a program that is able to extract lines from a graph.
For example, if a graph like this is inputted, I would just want the red line to be outputted.
Below I have tried to do this using a hough line transformation, however, I do not get very promising results.
import cv2
import numpy as np
graph_img = cv2.imread("/Users/2020shatgiskessell/Desktop/Graph1.png")
gray = cv2.cvtColor(graph_img, cv2.COLOR_BGR2GRAY)
kernel_size = 5
#grayscale image
blur_gray = cv2.GaussianBlur(gray,(kernel_size, kernel_size),0)
#Canny edge detecion
edges = cv2.Canny(blur_gray, 50, 150)
#Hough Lines Transformation
#distance resoltion of hough grid (pixels)
rho = 1
#angular resolution of hough grid (radians)
theta = np.pi/180
#minimum number of votes
threshold = 15
#play around with these
min_line_length = 25
max_line_gap = 20
#make new image
line_image = np.copy(graph_img)
#returns array of lines
lines = cv2.HoughLinesP(edges, rho, theta, threshold, np.array([]),
min_line_length, max_line_gap)
for line in lines:
for x1,y1,x2,y2 in line:
cv2.line(line_image,(x1,y1),(x2,y2),(255,0,0),2)
lines_edges = cv2.addWeighted(graph_img, 0.8, line_image, 1, 0)
cv2.imshow("denoised image",edges)
if cv2.waitKey(0) & 0xff == 27:
cv2.destroyAllWindows()
This produces the output image below, which does not accurately recognize the graph line. How might I go about doing this?
Note: For now, I am not concerned about the graph titles or any other text.
I would also like the code to work for other graph images aswell, such as:
etc.
If the graph does not have many noises around it (like your example) I would suggest to threshold your image with Otsu threshold instead of looking for edges . Then you simply search the contours, select the biggest one (graph) and draw it on a blank mask. After that you can perform a bitwise operation on image with the mask and you will get a black image with the graph. If you like the white background better, then simply change all black pixels to white. Steps are written in the example. Hope it helps a bit. Cheers!
Example:
import numpy as np
import cv2
# Read the image and create a blank mask
img = cv2.imread('graph.png')
h,w = img.shape[:2]
mask = np.zeros((h,w), np.uint8)
# Transform to gray colorspace and threshold the image
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
_, thresh = cv2.threshold(gray,0,255,cv2.THRESH_BINARY_INV+cv2.THRESH_OTSU)
# Search for contours and select the biggest one and draw it on mask
_, contours, hierarchy = cv2.findContours(thresh,cv2.RETR_TREE,cv2.CHAIN_APPROX_NONE)
cnt = max(contours, key=cv2.contourArea)
cv2.drawContours(mask, [cnt], 0, 255, -1)
# Perform a bitwise operation
res = cv2.bitwise_and(img, img, mask=mask)
# Convert black pixels back to white
black = np.where(res==0)
res[black[0], black[1], :] = [255, 255, 255]
# Display the image
cv2.imshow('img', res)
cv2.waitKey(0)
cv2.destroyAllWindows()
Result:
EDIT:
For noisier pictures you could try this code. Note that different graphs have different noises and may not work on every graph image since the denoisiation process would be specific in every case. For different noises you can use different ways to denoise it, for example histogram equalization, eroding, blurring etc. This code works well for all 3 graphs. Steps are written in comments. Hope it helps. Cheers!
import numpy as np
import cv2
# Read the image and create a blank mask
img = cv2.imread('graph.png')
h,w = img.shape[:2]
mask = np.zeros((h,w), np.uint8)
# Transform to gray colorspace and threshold the image
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
_, thresh = cv2.threshold(gray,0,255,cv2.THRESH_BINARY_INV+cv2.THRESH_OTSU)
# Perform opening on the thresholded image (erosion followed by dilation)
kernel = np.ones((2,2),np.uint8)
opening = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, kernel)
# Search for contours and select the biggest one and draw it on mask
_, contours, hierarchy = cv2.findContours(opening,cv2.RETR_TREE,cv2.CHAIN_APPROX_NONE)
cnt = max(contours, key=cv2.contourArea)
cv2.drawContours(mask, [cnt], 0, 255, -1)
# Perform a bitwise operation
res = cv2.bitwise_and(img, img, mask=mask)
# Threshold the image again
gray = cv2.cvtColor(res,cv2.COLOR_BGR2GRAY)
_, thresh = cv2.threshold(gray,0,255,cv2.THRESH_BINARY_INV+cv2.THRESH_OTSU)
# Find all non white pixels
non_zero = cv2.findNonZero(thresh)
# Transform all other pixels in non_white to white
for i in range(0, len(non_zero)):
first_x = non_zero[i][0][0]
first_y = non_zero[i][0][1]
first = res[first_y, first_x]
res[first_y, first_x] = 255
# Display the image
cv2.imshow('img', res)
cv2.waitKey(0)
cv2.destroyAllWindows()
Result:
Here is the image from which I want to take the text out.
How to remove the black border and reduce the image to only 50?
Approach I took:
I tried to use corner detectors (corner peak and corner harris) and pick the first 2 coordinates from the left and last 2 coordinates from the right.
With those 4 coordinates I cropped the image and I further reduced by 5 on all sides.
Certainly not efficient way of doing it. I also looked at few segmentation also. Not able to get it right. I am using scikit image for solving this.
Using corners might not work since corner points can also be present in characters.
Here is what i tried with hough lines as described below:
1) First erode the image to minimize the gap between lines and characters
2) Use Hough line detection algorithm to detect and delete the lines
3) Dilate the image to get clear characters
4) Now we have characters and lines separated, so we can delete the lines by finding the connected components.
Here is the code implementation of the same in Python:
img = cv2.imread('D:\Image\st1.png',0)
ret, thresh = cv2.threshold(img, 150, 255, cv2.THRESH_BINARY_INV)
#dilate the image to reduce gap between characters and lines and get hough lines correctly
kernel = np.ones((3,3),np.uint8)
erosion = cv2.erode(thresh,kernel,iterations = 1)
#find canny edge image
canny = cv2.Canny(erosion,100,200)
minLineLength=img.shape[1]/4
lines = cv2.HoughLinesP(image=canny,rho=0.02,theta=np.pi/500, threshold=10,lines=np.array([]), minLineLength=minLineLength,maxLineGap=10)
a,b,c = lines.shape
# delete the lines
for i in range(a):
cv2.line(erosion, (lines[i][0][0], lines[i][0][1]), (lines[i][0][2], lines[i][0][3]), 0, 3, cv2.LINE_AA)
#erode the image
kernel = np.ones((3,3),np.uint8)
erosion = cv2.dilate(erosion, kernel, iterations=1)
# find connected components
connectivity = 4
nb_components, output, stats, centroids = cv2.connectedComponentsWithStats(erosion, connectivity, cv2.CV_32S)
sizes = stats[1:, -1]; nb_components = nb_components - 1
min_size = 250 #threshhold value for lines length
img2 = np.zeros((output.shape), np.uint8)
for i in range(0, nb_components):
if sizes[i] >= min_size:
img2[output == i + 1] = 255 #delete the line components
img = cv2.bitwise_not(img2)
Output image:
I have an image which I want to detect edges on that. I found Canny has been used a lot ( I don't know whether I have a better option than that). I have set the values as follow:
Imgproc.Canny(img, img, 10, 100, 3,true)
I've changed threshold values but don't see that much of a change in my image. Can anyone explain to me if there is a logical way to figure out numbers for threshold values (my image is gray scale)
Thank you...
I think this should be taken case by case, if you post some sample images would be useful, but I will try to answer anyways. Here is from Opencv Documents
Canny( detected_edges, detected_edges, lowThreshold, lowThreshold*ratio, kernel_size );
where the arguments are:
detected_edges: Source image, grayscale
detected_edges: Output of the detector (can be the same as the input)
lowThreshold: The value entered by the user moving the Trackbar
highThreshold: Set in the program as three times the lower threshold (following Canny’s recommendation)
kernel_size: We defined it to be 3 (the size of the Sobel kernel to be used internally)
What usually works for me is highThreshold = 255 and lowThreshold = 255/3
As Samer said it could be case by case. Here is some code that uses trackbars in opencv, and displays the canny image next to the original, in order to quickly experiment with different threshold values.
import cv2
import numpy as np
import matplotlib.pyplot as plt
def callback(x):
print(x)
img = cv2.imread('your_image.png', 0) #read image as grayscale
canny = cv2.Canny(img, 85, 255)
cv2.namedWindow('image') # make a window with name 'image'
cv2.createTrackbar('L', 'image', 0, 255, callback) #lower threshold trackbar for window 'image
cv2.createTrackbar('U', 'image', 0, 255, callback) #upper threshold trackbar for window 'image
while(1):
numpy_horizontal_concat = np.concatenate((img, canny), axis=1) # to display image side by side
cv2.imshow('image', numpy_horizontal_concat)
k = cv2.waitKey(1) & 0xFF
if k == 27: #escape key
break
l = cv2.getTrackbarPos('L', 'image')
u = cv2.getTrackbarPos('U', 'image')
canny = cv2.Canny(img, l, u)
cv2.destroyAllWindows()
You can use this equation it is useful and you can apply bluer to enhance it.
blurred_img = cv2.blur(img,ksize=(5,5))
med_val = np.median(img)
lower = int(max(0 ,0.7*median_pix))
upper = int(min(255,1.3*median_pix))
edges = cv2.Canny(image=img, threshold1=lower,threshold2=upper)
I am trying to detect foreground motion using opencv2 by removing static (mostly) BG elements. The method I am using is based on taking the mean of a series of images - representing the background. Then calculating one Standard deviation above and below that mean. Using that as a window to detect foreground motion.
This mechanism reportedly works well for moderately noisy environments like waving trees in the BG.
The desired output is a mask that can be used in a subsequent operation so as to minimise further processing. Specifically I am going to use optical flow detection within that region.
cv2 has made this much easier and the code is much simpler to read and understand. Thanks cv2 and numpy.
But I am having difficulty doing the correct FG detection.
Ideally I also want to erode/dilate the BG mean so as to eleminate 1 pixel noise.
The code is all togethr so you have a number of frames at the start (BGsample) to gather the BG data before FG detection starts. the only dependencies are opencv2 (> 2.3.1 ) and numpy (which should be included in > opencv 2.3.1 )
import cv2
import numpy as np
if __name__ == '__main__':
cap = cv2.VideoCapture(0) # webcam
cv2.namedWindow("input")
cv2.namedWindow("sig2")
cv2.namedWindow("detect")
BGsample = 20 # number of frames to gather BG samples from at start of capture
success, img = cap.read()
width = cap.get(3)
height = cap.get(4)
# can use img.shape(:-1) # cut off extra channels
if success:
acc = np.zeros((height, width), np.float32) # 32 bit accumulator
sqacc = np.zeros((height, width), np.float32) # 32 bit accumulator
for i in range(20): a = cap.read() # dummy to warm up sensor
# gather BG samples
for i in range(BGsample):
success, img = cap.read()
frame = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
cv2.accumulate(frame, acc)
cv2.accumulateSquare(frame, sqacc)
#
M = acc/float(BGsample)
sqaccM = sqacc/float(BGsample)
M2 = M*M
sig2 = sqaccM-M2
# have BG samples now
# start FG detection
key = -1
while(key < 0):
success, img = cap.read()
frame = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
#Ideally we create a mask for future use that is B/W for FG objects
# (using erode or dilate to remove noise)
# this isn't quite right
level = M+sig2-frame
grey = cv2.morphologyEx(level, cv2.MORPH_DILATE,
cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (3,3)), iterations=2)
cv2.imshow("input", frame)
cv2.imshow("sig2", sig2/60)
cv2.imshow("detect", grey/20)
key = cv2.waitKey(1)
cv2.destroyAllWindows()
I don't think you need to manually compute the mean and standard deviation use cv2.meanStdDev instead. In the code below, I'm using your average background matrix computed from
M = acc/float(BGsample)
So, now we can compute the mean and standard deviation of the average background image, and finally inRange is used to pull out the range that you wanted (i.e., the mean +/- 1 standard deviation).
(mu, sigma) = cv2.meanStdDev(M)
fg = cv2.inRange(M, (mu[0] - sigma[0]), (mu[0] + sigma[0]))
# proceed with morphological clean-up here...
Hope that helps!
my best guess so far. Using detectmin, max to coerce the fp sigma into grayscale for the cv2.inRange to use.
Seems to work OK but was hoping for better... plenty of holes in valid FG data.
I suppose it would work better in rgb instead of grayscale.
Can't get noise reduction using dilate or erode to work.
Any improvements ?
import cv2
import numpy as np
if __name__ == '__main__':
cap = cv2.VideoCapture(1)
cv2.namedWindow("input")
#cv2.namedWindow("sig2")
cv2.namedWindow("detect")
BGsample = 20 # number of frames to gather BG samples from at start of capture
success, img = cap.read()
width = cap.get(3)
height = cap.get(4)
if success:
acc = np.zeros((height, width), np.float32) # 32 bit accumulator
sqacc = np.zeros((height, width), np.float32) # 32 bit accumulator
for i in range(20): a = cap.read() # dummy to warm up sensor
# gather BG samples
for i in range(BGsample):
success, img = cap.read()
frame = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
cv2.accumulate(frame, acc)
cv2.accumulateSquare(frame, sqacc)
#
M = acc/float(BGsample)
sqaccM = sqacc/float(BGsample)
M2 = M*M
sig2 = sqaccM-M2
# have BG samples now
# calculate upper and lower bounds of detection window around mean.
# coerce into 8bit image space for cv2.inRange compare
detectmin = cv2.convertScaleAbs(M-sig2)
detectmax = cv2.convertScaleAbs(M+sig2)
# start FG detection
key = -1
while(key < 0):
success, img = cap.read()
frame = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
level = cv2.inRange(frame, detectmin, detectmax)
cv2.imshow("input", frame)
#cv2.imshow("sig2", M/200)
cv2.imshow("detect", level)
key = cv2.waitKey(1)
cv2.destroyAllWindows()