Extraction of sunken portions from image in hieroglyphics - opencv

Question

Currently I am trying to extract the hieroglyphics symbols from images like this one.
What I have done is used hough transform to find lines and split the image in portions to make it easier for me. But I tried a set of algorithms to extract the sunken letters from the image and I hit a dead end..
What I have tried is a mixture of morphological operations and edge detection and contour finding.
So are there any algorithms devised to do something like this or any hint will be appreciated.

Answer 1

You can up-sample the input image, apply some smoothing, and find the Otsu threshold, then use this threshold to find Canny edges with different window sizes.
For the larger window (5 x 5), you get a noisy image that contains almost all the edges you need, plus noise.
For the smaller window (3 x 3), you get a less noisy image, but some of the edges are missing.
If this less noisy image is not good enough, you can try morphologically reconstructing it using the noisy image as the mask. Here, I've linked some diagonal edge segments in the noisy image using a morphological hit-miss transform and then applied the reconstruction.
Using a
Mat k = (Mat_<int>(3, 3) <<
0, 0, 1,
0, -1, 0,
1, 0, 0);
kernel for linking broken edges, you get a thinner outline.
Please note that in the c++ code below, I've used a naive reconstruction.
Mat im = imread("rsSUY.png", 0);
/* up sample and smooth */
pyrUp(im, im);
GaussianBlur(im, im, Size(5, 5), 5);
/* find the Otsu threshold */
Mat bw1, bw2;
double th = threshold(im, bw1, 0, 255, THRESH_BINARY | THRESH_OTSU);
/* use the found Otsu threshold for Canny */
Canny(im, bw1, th, th/2, 5, true); /* this result would be noisy */
Canny(im, bw2, th, th/2, 3, true); /* this result would be less noisy */
/* link broken edges in more noisy image using hit-miss transform */
Mat k = (Mat_<int>(3, 3) <<
0, 0, 1,
0, -1, 0,
0, 0, 0);
Mat hitmiss;
morphologyEx(bw1, hitmiss, MORPH_HITMISS, k);
bw1 |= hitmiss;
/* apply morphological reconstruction to less noisy image using the modified noisy image */
Mat kernel = getStructuringElement(MORPH_ELLIPSE, Size(3, 3));
double prevMu = 0;
Mat recons = bw2.clone();
for (int i = 0; i < 200; i++)
{
dilate(recons, recons, kernel);
recons &= bw1;
Scalar mu = mean(recons);
if (abs(mu.val[0] - prevMu) < 0.001)
{
break;
}
prevMu = mu.val[0];
}
imshow("less noisy", bw2);
imshow("reconstructed", recons);
waitKey();

Answer 2

The best bet for this task is machine learning. You can:
Crop or mark a few samples for each letter
Train an SSD (Single-shot Multibox Detector) using these samples
The advantage is that you will be able to detect all letters in an image in one pass.

Answer 3

I have a rather simple but not so perfect solution.
1. Finding the optimal higher and lower threshold based on the median of the green channel of the image
Upper threshold image:
Lower threshold image:
2. Subtracting the two images followed by median filtering:
3. Canny edge detection:
To get a better finish you need to follow this up by some morphological operations.