How can I improve recognition? - opencv

Question

I set myself the task of recognizing passports, but I can’t completely recognize all areas. Tell me, what can help? Used a different filtering and canny algorithm, but something is missing. Код не может распознать серию и номер документа, а также мелкие символы, иногда не может распознать имя или фамилию совсем....
# import the necessary packages
from PIL import Image
import pytesseract
import argparse
import cv2
import os
import numpy as np
# построить разбор аргументов и разбор аргументов
ap = argparse.ArgumentParser()
ap.add_argument("-i", "--image" )
ap.add_argument("-p", "--preprocess", type=str, default="thresh")
args = vars(ap.parse_args())
# загрузить пример изображения и преобразовать его в оттенки серого
image = cv2.imread ("pt.jpg")
gray = cv2.cvtColor (image, cv2.COLOR_BGR2GRAY)
gray = cv2.Canny(image,300,300,apertureSize = 3)
# check to see if we should apply thresholding to preprocess the
# image
if args["preprocess"] == "thresh":
gray = cv2.threshold (gray, 0, 255, cv2.THRESH_BINARY | cv2.THRESH_OTSU)[1]
# make a check to see if median blurring should be done to remove
# noise
elif args["preprocess"] == "blur":
gray = cv2.medianBlur (gray, 3)
# write the grayscale image to disk as a temporary file so we can
# apply OCR to it
filename = "{}.png".format (os.getpid ())
cv2.imwrite (filename, gray)
# load the image as a PIL/Pillow image, apply OCR, and then delete
# the temporary file
pytesseract.pytesseract.tesseract_cmd = r'C:\Program Files\Tesseract-OCR\tesseract.exe'
text = pytesseract.image_to_string (image, lang = 'rus+eng')
os.remove (filename)
print (text)
os.system('python gon.py > test.txt') # doc output file
# show the output images
cv2.imshow ("Image", image)
cv2.imshow ("Output", gray)
cv2.waitKey (0)

Answer 1

It's easier (and faster) for Tesseract to recognize text when you provide it only with the regions that contain the text you want to interpret, in your case the big, black letters in the middle, for example:
I'm referring to running Tesseract only in the regions in green. Since the document's structure is predictable, you could easily find these regions as follows:
binarize and invert the image (black = empty)
use opencv's connectedComponentsWithStats() function to get a list of all connected components with their positions and size
you can hardcode thresholds to filter only the characters you want, or get a histogram of areas and use statistics to define thresholds dinamically
on the remaining connected components use morphological operations (e.g. dilation with a horizontal kernel) to connect letters together horizontally
get the bounding box of the final connected components
Optional: postprocessing will work much better in these isolated boxes
Feed each bounding box as a separate Mat to tesseract, it will greatly simplify the problem.