Wrong annotation: x = 0, y = 0, < 0 or > 1 with YOLO - machine-learning

Im getting this error when i train my dataset using yolo, i understand this mean that the labels are not in bounds of the image but ie source by data from open images. I get this error for each and every file.
Wrong annotation: x = 0, y = 0, < 0 or > 1, file: data/obj/3149c61b7407b0c5.txt
When converting the labels, the class was initially 101 for all, this gave me an error so I converted all of them to 0 using this
import glob
for filepath in glob.iglob('./**/*.txt', recursive=True):
with open(filepath) as file:
s = file.read()
s = s.replace('101 ', '0 ')
with open(filepath, "w") as file:
file.write(s)
Heres the content of one file "3149c61b7407b0c5.txt" which also gave this error.
0 1085.6260855566406 1254.4786898033353 2.9297560546874593 102.1827418215693
0 1086.0899527441406 607.6991307711304 3.857441406250018 1176.3009893473438
0 1137.8566324316407 1307.114119795808 91.84562460937491 5.3656234452045295
0 1186.680802158203 607.1194858238297 3.8867578125000364 1175.290322583892
0 1187.6573546972656 1361.4528185098725 3.8672361230469505 101.16129032792523
0 1234.0440929785157 1195.3281309872493 92.75390625 3.2795265516661787
0 1234.0284875097657 604.9840441022702 90.90814453124995 1177.3440653795917
0 1235.4991806640626 1360.3872916281202 91.875 101.11822602580656
0 1236.9639953222656 17.328152487227214 92.75390625 4.290258067785163
0 1283.3262900390625 7.146366677539043 3.8965437597655637 14.01077505600043
0 1431.255040830078 1251.7474424893667 94.71674785156256 101.12901075597932
0 1430.7726089941407 1630.7969258238081 91.77738281250004 5.3978934344627785
0 1477.198360751953 1626.5442385087865 4.86328125 5.365623446172403
0 1528.974786533203 1789.3657435657651 90.85930644531254 103.20425816668744
0 1579.2491416015625 1898.0120122775286 3.828125 99.01077386239739
0 1625.6456455078126 1519.5280764679483 92.73437579101551 4.344172047279395
0 1627.1007041015625 1467.355033670012 91.875 100.0859898946889
0 1674.4591220703126 1358.7162488302274 4.853545517578141 102.13981699359162
0 1772.593927158203 1950.1948291506785 3.876963085937632 5.35483871502201
0 1871.709142001953 2003.349592810873 4.863222255859455 101.10751591723597
0 1871.2404211035157 1843.6894421635711 3.9062204980468778 5.33334387530102

Yolo labels are done as a proportion of the image width/height. This is so that the label stays true even if the image is resized. The label format is:
<class id> <x> <y> <width> <height>
So for example, a label that says
0 150 50 450 100
On a 600x200 image would be
0 .25 .25 .75 .5

Related

How to get connected components label in a binary image?

I've a binary image where removing green dot gets me separate line segments. I've tried using label_components() function from Julia but it labels only verticall joined pixels as one label.
I'm using
using Images
img=load("current_img.jpg")
img[findall(img.==RGB(0.0,0.1,0.0))].=0 # this makes green pixels same as background, i.e. black
labels = label_components(img)
I'm expecteing all lines which are disjoint to be given a unique label
(as was a funciton in connected component labeling in matlab, but i can't find something similar in julia)
Since you updated the question and added more details to make it clear, I decided to post the answer. Note that this answer utilizes some of the functions that I wrote here; so, if you didn't find documentation for any of the following functions, I refer you to the previous answer. I operated on several examples and brought the results in the continue.
Let's begin with an image similar to the one you brought in the question and perform the entire operation from the scratch. for this, I drew the following:
I want to perform a segmentation process on it and labelize each segment and highlight the segments using the achieved labels.
Let's define the functions:
using Images
using ImageBinarization
function check_adjacent(
loc::CartesianIndex{2},
all_locs::Vector{CartesianIndex{2}}
)
conditions = [
loc - CartesianIndex(0,1) ∈ all_locs,
loc + CartesianIndex(0,1) ∈ all_locs,
loc - CartesianIndex(1,0) ∈ all_locs,
loc + CartesianIndex(1,0) ∈ all_locs,
loc - CartesianIndex(1,1) ∈ all_locs,
loc + CartesianIndex(1,1) ∈ all_locs,
loc - CartesianIndex(1,-1) ∈ all_locs,
loc + CartesianIndex(1,-1) ∈ all_locs
]
return sum(conditions)
end;
function find_the_contour_branches(img::BitMatrix)
img_matrix = convert(Array{Float64}, img)
not_black = findall(!=(0.0), img_matrix)
contours_branches = Vector{CartesianIndex{2}}()
for nb∈not_black
t = check_adjacent(nb, not_black)
(t==1 || t==3) && push!(contours_branches, nb)
end
return contours_branches
end;
"""
HighlightSegments(img::BitMatrix, labels::Matrix{Int64})
Highlight the segments of the image with random colors.
# Arguments
- `img::BitMatrix`: The image to be highlighted.
- `labels::Matrix{Int64}`: The labels of each segment.
# Returns
- `img_matrix::Matrix{RGB}`: A matrix of RGB values.
"""
function HighlightSegments(img::BitMatrix, labels::Matrix{Int64})
colors = [
# Create Random Colors for each label
RGB(rand(), rand(), rand()) for label in 1:maximum(labels)
]
img_matrix = convert(Matrix{RGB}, img)
for seg∈1:maximum(labels)
img_matrix[labels .== seg] .= colors[seg]
end
return img_matrix
end;
"""
find_labels(img_path::String)
Assign a label for each segment.
# Arguments
- `img_path::String`: The path of the image.
# Returns
- `thinned::BitMatrix`: BitMatrix of the thinned image.
- `labels::Matrix{Int64}`: A matrix that contains the labels of each segment.
- `highlighted::Matrix{RGB}`: A matrix of RGB values.
"""
function find_labels(img_path::String)
img::Matrix{RGB} = load(img_path)
gimg = Gray.(img)
bin::BitMatrix = binarize(gimg, UnimodalRosin()) .> 0.5
thinned = thinning(bin)
contours = find_the_contour_branches(thinned)
thinned[contours] .= 0
labels = label_components(thinned, trues(3,3))
highlighted = HighlightSegments(thinned, labels)
return thinned, labels, highlighted
end;
The main function in the above is find_labels which returns
The thinned matrix.
The labels of each segment.
The highlighted image (Matrix, actually).
First, I load the image, and binarize the Gray scaled image. Then, I perform the thinning operation on the binarized image. After that, I find the contours and the branches using the find_the_contour_branches function. Then, I turn the color of contours and branches to black in the thinned image; this gives me neat segments. After that, I labelize the segments using the label_components function. Finally, I highlight the segments using the HighlightSegments function for the sake of visualization (this is the bonus :)).
Let's try it on the image I drew above:
result = find_labels("nU3LE.png")
# you can get the labels Matrix using `result[2]`
# and the highlighted image using `result[3]`
# Also, it's possible to save the highlighted image using:
save("nU3LE_highlighted.png", result[3])
The result is as follows:
Also, I performed the same thing on another image:
julia> result = find_labels("circle.png")
julia> result[2]
14×16 Matrix{Int64}:
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 4 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 4 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 4 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 4 0 0 0 0 0 0
0 1 1 0 0 0 3 3 0 0 0 5 5 5 0 0
0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
As you can see, the labels are pretty clear. Now let's see the results of performing the procedure in some examples in one glance:
Original Image
Labeled Image

How to save dpi info in py-opencv?

import cv2
def clear(img):
back = cv2.imread("back.png", cv2.IMREAD_GRAYSCALE)
img = cv2.bitwise_xor(img, back)
ret, img = cv2.threshold(img, 120, 255, cv2.THRESH_BINARY_INV)
return img
def threshold(img):
ret, img = cv2.threshold(img, 120, 255, cv2.THRESH_BINARY_INV)
img = cv2.cvtColor(img, cv2.COLOR_RGB2GRAY)
ret, img = cv2.threshold(img, 248, 255, cv2.THRESH_BINARY)
return img
def fomatImage(img):
img = threshold(img)
img = clear(img)
return img
img = fomatImage(cv2.imread("1566135246468.png",cv2.IMREAD_COLOR))
cv2.imwrite("aa.png",img)
This is my code. But when I tried to identify it with tesseract-ocr, I got a warning.
Warning: Invalid resolution 0 dpi. Using 70 instead.
How should I set up dpi?
AFAIK, OpenCV doesn't set the dpi of PNG files it writes, so you are looking at work-arounds. Here are some ideas...
Method 1 - Use PIL/Pillow instead of OpenCV
PIL/Pillow can write dpi information into PNG files. So you would:
Step 1 - Convert your BGR OpenCV image into RGB to match PIL's channel ordering
from PIL import Image
RGBimage = cv2.cvtColor(BGRimage, cv2.COLOR_BGR2RGB)
Step 2 - Convert OpenCV Numpy array onto PIL Image
PILimage = Image.fromarray(RGBimage)
Step 3 - Write with PIL
PILimage.save('result.png', dpi=(72,72))
As Fred mentions in the comments, you could equally use Python Wand in much the same way.
Method 2 - Write with OpenCV but modify afterwards with some tool
You could use Python's subprocess module to shell out to, say, ImageMagick and set the dpi like this:
magick OpenCVImage.png -set units pixelspercentimeter -density 28.3 result.png
All you need to know is that PNG uses metric (dots per centimetre) rather than imperial (dots per inch) and there are 2.54cm in an inch, so 72 dpi becomes 28.3 dots per cm.
If your ImageMagick version is older than v7, replace magick with convert.
Method 3 - Write with OpenCV and insert dpi yourself
You could write your file to memory using OpenCV's imencode(). Then search in the file for the IDAT (image data) chunk - which is the one containing the image pixels and insert a pHYs chunk before that which sets the density. Then write to disk.
It's not that hard actually - it's just 9 bytes, see here and also look at pngcheck output at end of answer.
This code is not production tested but seems to work pretty well for me:
#!/usr/bin/env python3
import struct
import numpy as np
import cv2
import zlib
def writePNGwithdpi(im, filename, dpi=(72,72)):
"""Save the image as PNG with embedded dpi"""
# Encode as PNG into memory
retval, buffer = cv2.imencode(".png", im)
s = buffer.tostring()
# Find start of IDAT chunk
IDAToffset = s.find(b'IDAT') - 4
# Create our lovely new pHYs chunk - https://www.w3.org/TR/2003/REC-PNG-20031110/#11pHYs
pHYs = b'pHYs' + struct.pack('!IIc',int(dpi[0]/0.0254),int(dpi[1]/0.0254),b"\x01" )
pHYs = struct.pack('!I',9) + pHYs + struct.pack('!I',zlib.crc32(pHYs))
# Open output filename and write...
# ... stuff preceding IDAT as created by OpenCV
# ... new pHYs as created by us above
# ... IDAT onwards as created by OpenCV
with open(filename, "wb") as out:
out.write(buffer[0:IDAToffset])
out.write(pHYs)
out.write(buffer[IDAToffset:])
################################################################################
# main
################################################################################
# Load sample image
im = cv2.imread('lena.png')
# Save at specific dpi
writePNGwithdpi(im, "result.png", (32,300))
Whichever method you use, you can use pngcheck --v image.png to check what you have done:
pngcheck -vv a.png
Sample Output
File: a.png (306 bytes)
chunk IHDR at offset 0x0000c, length 13
100 x 100 image, 1-bit palette, non-interlaced
chunk gAMA at offset 0x00025, length 4: 0.45455
chunk cHRM at offset 0x00035, length 32
White x = 0.3127 y = 0.329, Red x = 0.64 y = 0.33
Green x = 0.3 y = 0.6, Blue x = 0.15 y = 0.06
chunk PLTE at offset 0x00061, length 6: 2 palette entries
chunk bKGD at offset 0x00073, length 1
index = 1
chunk pHYs at offset 0x00080, length 9: 255x255 pixels/unit (1:1). <-- THIS SETS THE DENSITY
chunk tIME at offset 0x00095, length 7: 19 Aug 2019 10:15:00 UTC
chunk IDAT at offset 0x000a8, length 20
zlib: deflated, 2K window, maximum compression
row filters (0 none, 1 sub, 2 up, 3 avg, 4 paeth):
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
(100 out of 100)
chunk tEXt at offset 0x000c8, length 37, keyword: date:create
chunk tEXt at offset 0x000f9, length 37, keyword: date:modify
chunk IEND at offset 0x0012a, length 0
No errors detected in a.png (11 chunks, 76.5% compression).
While I am editing PNG chunks, I also managed to set a tIME chunk and a tEXt chunk with the Author. They go like this:
# Create a new tIME chunk - https://www.w3.org/TR/2003/REC-PNG-20031110/#11tIME
year, month, day, hour, min, sec = 2020, 12, 25, 12, 0, 0 # Midday Christmas day 2020
tIME = b'tIME' + struct.pack('!HBBBBB',year,month,day,hour,min,sec)
tIME = struct.pack('!I',7) + tIME + struct.pack('!I',zlib.crc32(tIME))
# Create a new tEXt chunk - https://www.w3.org/TR/2003/REC-PNG-20031110/#11tEXt
Author = "Author\x00Sir Mark The Great"
tEXt = b'tEXt' + bytes(Author.encode('ascii'))
tEXt = struct.pack('!I',len(Author)) + tEXt + struct.pack('!I',zlib.crc32(tEXt))
# Open output filename and write...
# ... stuff preceding IDAT as created by OpenCV
# ... new pHYs as created by us above
# ... new tIME as created by us above
# ... new tEXt as created by us above
# ... IDAT onwards as created by OpenCV
with open(filename, "wb") as out:
out.write(buffer[0:IDAToffset])
out.write(pHYs)
out.write(tIME)
out.write(tEXt)
out.write(buffer[IDAToffset:])
Keywords: OpenCV, PIL, Pillow, dpi, density, imwrite, PNG, chunks, pHYs chunk, Python, image, image-processing, tEXt chunk, tIME chunk, author, comment

Possibility of only dealing with specific region of binary image

Recently I study the image processing.
When I go through the problem of filling the hole, it confuses me (I assume that the people able to answer the question is familiar with the step of doing this so I skip to the problem):
Let's say if I have a binary image like this:
0 0 0 0 0 0 0
0 0 1 1 0 0 0
0 1 0 0 1 0 0
0 1 0 0 1 0 0
0 0 1 0 1 0 0
0 0 1 0 1 0 0
0 1 0 0 0 1 0
0 1 0 0 0 1 0
0 1 1 1 1 0 0
0 0 0 0 0 0 0
And the book says to start form the region that is inside of the hole and perform the dilation operation and set the bound in case it fills the whole image.
I have no problem understanding the whole process, but if I try to code it, how can I only deal with a specific region (in the hole for this case)? Or the actual implement would be different method ?
If you can assume that the object with holes does not touch the border of the image, you can create an intermediate image where you call flood fill (with value e.g. 2) on the top left pixel. Any remaining '0' pixels have to be inside the contour. Take the position of the first encountered remaining '0' pixel and flood fill it in the original image.

Cannot understand Keras ConvLSTM2D - edited

I'm looking at the example: https://github.com/fchollet/keras/blob/master/examples/conv_lstm.py
This RNN is actually predicting the next frame of the movie, so the output should be a movie too (according to the test data fed in). I wonder if there are information lost due to the conv layers with padding.
For example, the underlying Tensorflow is padding bottom right, if there is a big padding: (n stands for numbers)
n n n n 0 0 0
n n n n 0 0 0
n n n n 0 0 0
n n n n 0 0 0
0 0 0 0 0 0 0
0 0 0 0 0 0 0
0 0 0 0 0 0 0
when we do the second conv, the bottom right corner will always be 0, which means the back propagation will never be able to capture anything there. As in this case a movie(a square moves on the whole screen), will it lost the information when the validation label is on the bottom right corner?
The answer is yes after asking a Ph.D. doing AI research.

Manipulating data of Mat in opencv

I have a vector of integers. I want to add this vector to a particular row/column in the Mat object. Is this possible. I have been looking at the API and am unable to find anything.
Try cv::Mat::row() and cv::Mat::col().
there 's a constructor for Mat(and an assignment operator, too), that takes a vector as input, so the same applies to rows of a Mat ( which are Mat's again )
Mat big(5,5,CV_32S); // 5x5, 1channel, int mat
vector<int> vec(5); vec[0]=1; vec[2] = 17; vec[4]=13; // make a row vec
big.row(1) = vec; // careful, does not work for col(), since that returns a copy
0 0 0 0 0
1 0 17 0 13
0 0 0 0 0
0 0 0 0 0
0 0 0 0 0
i was wrong here, sorry. both row() and col() make a copy of the original data, so assigning to that is useless.

Resources