I've a binary image where removing green dot gets me separate line segments. I've tried using label_components() function from Julia but it labels only verticall joined pixels as one label.
I'm using
using Images
img=load("current_img.jpg")
img[findall(img.==RGB(0.0,0.1,0.0))].=0 # this makes green pixels same as background, i.e. black
labels = label_components(img)
I'm expecteing all lines which are disjoint to be given a unique label
(as was a funciton in connected component labeling in matlab, but i can't find something similar in julia)
Since you updated the question and added more details to make it clear, I decided to post the answer. Note that this answer utilizes some of the functions that I wrote here; so, if you didn't find documentation for any of the following functions, I refer you to the previous answer. I operated on several examples and brought the results in the continue.
Let's begin with an image similar to the one you brought in the question and perform the entire operation from the scratch. for this, I drew the following:
I want to perform a segmentation process on it and labelize each segment and highlight the segments using the achieved labels.
Let's define the functions:
using Images
using ImageBinarization
function check_adjacent(
loc::CartesianIndex{2},
all_locs::Vector{CartesianIndex{2}}
)
conditions = [
loc - CartesianIndex(0,1) ∈ all_locs,
loc + CartesianIndex(0,1) ∈ all_locs,
loc - CartesianIndex(1,0) ∈ all_locs,
loc + CartesianIndex(1,0) ∈ all_locs,
loc - CartesianIndex(1,1) ∈ all_locs,
loc + CartesianIndex(1,1) ∈ all_locs,
loc - CartesianIndex(1,-1) ∈ all_locs,
loc + CartesianIndex(1,-1) ∈ all_locs
]
return sum(conditions)
end;
function find_the_contour_branches(img::BitMatrix)
img_matrix = convert(Array{Float64}, img)
not_black = findall(!=(0.0), img_matrix)
contours_branches = Vector{CartesianIndex{2}}()
for nb∈not_black
t = check_adjacent(nb, not_black)
(t==1 || t==3) && push!(contours_branches, nb)
end
return contours_branches
end;
"""
HighlightSegments(img::BitMatrix, labels::Matrix{Int64})
Highlight the segments of the image with random colors.
# Arguments
- `img::BitMatrix`: The image to be highlighted.
- `labels::Matrix{Int64}`: The labels of each segment.
# Returns
- `img_matrix::Matrix{RGB}`: A matrix of RGB values.
"""
function HighlightSegments(img::BitMatrix, labels::Matrix{Int64})
colors = [
# Create Random Colors for each label
RGB(rand(), rand(), rand()) for label in 1:maximum(labels)
]
img_matrix = convert(Matrix{RGB}, img)
for seg∈1:maximum(labels)
img_matrix[labels .== seg] .= colors[seg]
end
return img_matrix
end;
"""
find_labels(img_path::String)
Assign a label for each segment.
# Arguments
- `img_path::String`: The path of the image.
# Returns
- `thinned::BitMatrix`: BitMatrix of the thinned image.
- `labels::Matrix{Int64}`: A matrix that contains the labels of each segment.
- `highlighted::Matrix{RGB}`: A matrix of RGB values.
"""
function find_labels(img_path::String)
img::Matrix{RGB} = load(img_path)
gimg = Gray.(img)
bin::BitMatrix = binarize(gimg, UnimodalRosin()) .> 0.5
thinned = thinning(bin)
contours = find_the_contour_branches(thinned)
thinned[contours] .= 0
labels = label_components(thinned, trues(3,3))
highlighted = HighlightSegments(thinned, labels)
return thinned, labels, highlighted
end;
The main function in the above is find_labels which returns
The thinned matrix.
The labels of each segment.
The highlighted image (Matrix, actually).
First, I load the image, and binarize the Gray scaled image. Then, I perform the thinning operation on the binarized image. After that, I find the contours and the branches using the find_the_contour_branches function. Then, I turn the color of contours and branches to black in the thinned image; this gives me neat segments. After that, I labelize the segments using the label_components function. Finally, I highlight the segments using the HighlightSegments function for the sake of visualization (this is the bonus :)).
Let's try it on the image I drew above:
result = find_labels("nU3LE.png")
# you can get the labels Matrix using `result[2]`
# and the highlighted image using `result[3]`
# Also, it's possible to save the highlighted image using:
save("nU3LE_highlighted.png", result[3])
The result is as follows:
Also, I performed the same thing on another image:
julia> result = find_labels("circle.png")
julia> result[2]
14×16 Matrix{Int64}:
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 4 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 4 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 4 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 4 0 0 0 0 0 0
0 1 1 0 0 0 3 3 0 0 0 5 5 5 0 0
0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
As you can see, the labels are pretty clear. Now let's see the results of performing the procedure in some examples in one glance:
Original Image
Labeled Image
import cv2
def clear(img):
back = cv2.imread("back.png", cv2.IMREAD_GRAYSCALE)
img = cv2.bitwise_xor(img, back)
ret, img = cv2.threshold(img, 120, 255, cv2.THRESH_BINARY_INV)
return img
def threshold(img):
ret, img = cv2.threshold(img, 120, 255, cv2.THRESH_BINARY_INV)
img = cv2.cvtColor(img, cv2.COLOR_RGB2GRAY)
ret, img = cv2.threshold(img, 248, 255, cv2.THRESH_BINARY)
return img
def fomatImage(img):
img = threshold(img)
img = clear(img)
return img
img = fomatImage(cv2.imread("1566135246468.png",cv2.IMREAD_COLOR))
cv2.imwrite("aa.png",img)
This is my code. But when I tried to identify it with tesseract-ocr, I got a warning.
Warning: Invalid resolution 0 dpi. Using 70 instead.
How should I set up dpi?
AFAIK, OpenCV doesn't set the dpi of PNG files it writes, so you are looking at work-arounds. Here are some ideas...
Method 1 - Use PIL/Pillow instead of OpenCV
PIL/Pillow can write dpi information into PNG files. So you would:
Step 1 - Convert your BGR OpenCV image into RGB to match PIL's channel ordering
from PIL import Image
RGBimage = cv2.cvtColor(BGRimage, cv2.COLOR_BGR2RGB)
Step 2 - Convert OpenCV Numpy array onto PIL Image
PILimage = Image.fromarray(RGBimage)
Step 3 - Write with PIL
PILimage.save('result.png', dpi=(72,72))
As Fred mentions in the comments, you could equally use Python Wand in much the same way.
Method 2 - Write with OpenCV but modify afterwards with some tool
You could use Python's subprocess module to shell out to, say, ImageMagick and set the dpi like this:
magick OpenCVImage.png -set units pixelspercentimeter -density 28.3 result.png
All you need to know is that PNG uses metric (dots per centimetre) rather than imperial (dots per inch) and there are 2.54cm in an inch, so 72 dpi becomes 28.3 dots per cm.
If your ImageMagick version is older than v7, replace magick with convert.
Method 3 - Write with OpenCV and insert dpi yourself
You could write your file to memory using OpenCV's imencode(). Then search in the file for the IDAT (image data) chunk - which is the one containing the image pixels and insert a pHYs chunk before that which sets the density. Then write to disk.
It's not that hard actually - it's just 9 bytes, see here and also look at pngcheck output at end of answer.
This code is not production tested but seems to work pretty well for me:
#!/usr/bin/env python3
import struct
import numpy as np
import cv2
import zlib
def writePNGwithdpi(im, filename, dpi=(72,72)):
"""Save the image as PNG with embedded dpi"""
# Encode as PNG into memory
retval, buffer = cv2.imencode(".png", im)
s = buffer.tostring()
# Find start of IDAT chunk
IDAToffset = s.find(b'IDAT') - 4
# Create our lovely new pHYs chunk - https://www.w3.org/TR/2003/REC-PNG-20031110/#11pHYs
pHYs = b'pHYs' + struct.pack('!IIc',int(dpi[0]/0.0254),int(dpi[1]/0.0254),b"\x01" )
pHYs = struct.pack('!I',9) + pHYs + struct.pack('!I',zlib.crc32(pHYs))
# Open output filename and write...
# ... stuff preceding IDAT as created by OpenCV
# ... new pHYs as created by us above
# ... IDAT onwards as created by OpenCV
with open(filename, "wb") as out:
out.write(buffer[0:IDAToffset])
out.write(pHYs)
out.write(buffer[IDAToffset:])
################################################################################
# main
################################################################################
# Load sample image
im = cv2.imread('lena.png')
# Save at specific dpi
writePNGwithdpi(im, "result.png", (32,300))
Whichever method you use, you can use pngcheck --v image.png to check what you have done:
pngcheck -vv a.png
Sample Output
File: a.png (306 bytes)
chunk IHDR at offset 0x0000c, length 13
100 x 100 image, 1-bit palette, non-interlaced
chunk gAMA at offset 0x00025, length 4: 0.45455
chunk cHRM at offset 0x00035, length 32
White x = 0.3127 y = 0.329, Red x = 0.64 y = 0.33
Green x = 0.3 y = 0.6, Blue x = 0.15 y = 0.06
chunk PLTE at offset 0x00061, length 6: 2 palette entries
chunk bKGD at offset 0x00073, length 1
index = 1
chunk pHYs at offset 0x00080, length 9: 255x255 pixels/unit (1:1). <-- THIS SETS THE DENSITY
chunk tIME at offset 0x00095, length 7: 19 Aug 2019 10:15:00 UTC
chunk IDAT at offset 0x000a8, length 20
zlib: deflated, 2K window, maximum compression
row filters (0 none, 1 sub, 2 up, 3 avg, 4 paeth):
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
(100 out of 100)
chunk tEXt at offset 0x000c8, length 37, keyword: date:create
chunk tEXt at offset 0x000f9, length 37, keyword: date:modify
chunk IEND at offset 0x0012a, length 0
No errors detected in a.png (11 chunks, 76.5% compression).
While I am editing PNG chunks, I also managed to set a tIME chunk and a tEXt chunk with the Author. They go like this:
# Create a new tIME chunk - https://www.w3.org/TR/2003/REC-PNG-20031110/#11tIME
year, month, day, hour, min, sec = 2020, 12, 25, 12, 0, 0 # Midday Christmas day 2020
tIME = b'tIME' + struct.pack('!HBBBBB',year,month,day,hour,min,sec)
tIME = struct.pack('!I',7) + tIME + struct.pack('!I',zlib.crc32(tIME))
# Create a new tEXt chunk - https://www.w3.org/TR/2003/REC-PNG-20031110/#11tEXt
Author = "Author\x00Sir Mark The Great"
tEXt = b'tEXt' + bytes(Author.encode('ascii'))
tEXt = struct.pack('!I',len(Author)) + tEXt + struct.pack('!I',zlib.crc32(tEXt))
# Open output filename and write...
# ... stuff preceding IDAT as created by OpenCV
# ... new pHYs as created by us above
# ... new tIME as created by us above
# ... new tEXt as created by us above
# ... IDAT onwards as created by OpenCV
with open(filename, "wb") as out:
out.write(buffer[0:IDAToffset])
out.write(pHYs)
out.write(tIME)
out.write(tEXt)
out.write(buffer[IDAToffset:])
Keywords: OpenCV, PIL, Pillow, dpi, density, imwrite, PNG, chunks, pHYs chunk, Python, image, image-processing, tEXt chunk, tIME chunk, author, comment
Recently I study the image processing.
When I go through the problem of filling the hole, it confuses me (I assume that the people able to answer the question is familiar with the step of doing this so I skip to the problem):
Let's say if I have a binary image like this:
0 0 0 0 0 0 0
0 0 1 1 0 0 0
0 1 0 0 1 0 0
0 1 0 0 1 0 0
0 0 1 0 1 0 0
0 0 1 0 1 0 0
0 1 0 0 0 1 0
0 1 0 0 0 1 0
0 1 1 1 1 0 0
0 0 0 0 0 0 0
And the book says to start form the region that is inside of the hole and perform the dilation operation and set the bound in case it fills the whole image.
I have no problem understanding the whole process, but if I try to code it, how can I only deal with a specific region (in the hole for this case)? Or the actual implement would be different method ?
If you can assume that the object with holes does not touch the border of the image, you can create an intermediate image where you call flood fill (with value e.g. 2) on the top left pixel. Any remaining '0' pixels have to be inside the contour. Take the position of the first encountered remaining '0' pixel and flood fill it in the original image.
I'm looking at the example: https://github.com/fchollet/keras/blob/master/examples/conv_lstm.py
This RNN is actually predicting the next frame of the movie, so the output should be a movie too (according to the test data fed in). I wonder if there are information lost due to the conv layers with padding.
For example, the underlying Tensorflow is padding bottom right, if there is a big padding: (n stands for numbers)
n n n n 0 0 0
n n n n 0 0 0
n n n n 0 0 0
n n n n 0 0 0
0 0 0 0 0 0 0
0 0 0 0 0 0 0
0 0 0 0 0 0 0
when we do the second conv, the bottom right corner will always be 0, which means the back propagation will never be able to capture anything there. As in this case a movie(a square moves on the whole screen), will it lost the information when the validation label is on the bottom right corner?
The answer is yes after asking a Ph.D. doing AI research.
I have a vector of integers. I want to add this vector to a particular row/column in the Mat object. Is this possible. I have been looking at the API and am unable to find anything.
Try cv::Mat::row() and cv::Mat::col().
there 's a constructor for Mat(and an assignment operator, too), that takes a vector as input, so the same applies to rows of a Mat ( which are Mat's again )
Mat big(5,5,CV_32S); // 5x5, 1channel, int mat
vector<int> vec(5); vec[0]=1; vec[2] = 17; vec[4]=13; // make a row vec
big.row(1) = vec; // careful, does not work for col(), since that returns a copy
0 0 0 0 0
1 0 17 0 13
0 0 0 0 0
0 0 0 0 0
0 0 0 0 0
i was wrong here, sorry. both row() and col() make a copy of the original data, so assigning to that is useless.