I am not very expirienced in image processing...but have acquired some very noisy SEM images and it's hard to distinguish the particles I want to segment from the background. I know it's a general question but still...can you direct me to how I should go about it?
thank you in advance
Adi
It's difficult to be too specific without knowing what you want to achieve. If it's just improvement of the image in visual terms, a median filter would be my first suggestion. Use a small kernel size to avoid eroding edges too much.
As an example, here is a section of your original unprocessed image:
And here is the same section after applying a median filter with a 2 pixel radius:
It looks to me like the image is also slightly defocused, to which you can attempt some deconvolution algorithms like the Wiener filter, for example.
If you want to process the image in some way, for example to count the blobs (although that image is a really poor starting point), you can threshold it to get a binary image and then use morphological operations to refine the content. For example, I took the median filtered image, thresholded to binary and then performed a morphological open operation (an erode followed by a dilate):
To refine the segmentation and to split some of the touching particles, you can try a watershed segmentation on the binary image, like this:
Note that all of these images I produced using ImageJ, if you want to experiment further.
Related
I am currently working on an animal identification using footprint recognition project. My main task is to process an animal footprint taken from natural substrate and identify the animal the footprint belongs to. The first step is to preprocess the image and extract the ROI. This is where I am having difficulty as the processed images contain a lot of noise.
I have carried out a series of preprocessing steps all of which did reduce the noise but not sufficiently. The image below shows the outcome I have achieved thus far.
In order of left to right, the first image in the top row is an example of an animal footprint that I need to classify. The second image is one example of an image that will be used to train the system and classify the animal species (in this case a bear species). The third and fourth image in the first row show the grayscale and logarithmic transformation of the test image respectively.
The first image in the bottom row is a median blur of the image, the second shows adaptive thresholding. The third image shows the result of an eight-neighbour connectivity test where any pixel missing a neighbour is removed. The fourth image shows the image when erosion is applied after dilation. The last image shows the detected contours.
I have tried working with the contours to remove contours less than a certain area but it still didn't produce a better representation of the image. Displaying the largest contour just displays the entire image.
Using connected components detects a large number because of the high level of noise. I have tried working with blob detection and again did not achieve the desired results.
Im looking for the best and most efficient way to denoise the images and extract ROI.
Sample image:
One simple and effective way is to open the binary image then close it. Opening would help you with the white dots in the center of the image and closing would fill the undesired black spots in the white areas and finally you would have a neat footprint.
I applied binary threshold and followed it with morphological closing operation.
This is what I obtained:
This is the result of binary thresholding.
This is the result of morphological closing.
You have to do further processing to extract the foot perfectly.
I would suggest applying contour now. It should work fine.
I'm working with Infra Red image that is an output of a 3D sensor. This sensors project a Infra Red pattern in order to draw a depth map, and, because of this, the IR image has a lot of white spots that reduce its quality. So, I want to process this image to make it smoother in order to make it possible to detect objects laying in the surface.
The original image looks like this:
My objective is to have something like this (which I obtained by blocking the IR projecter with my hand) :
An "open" morphological operation does remove some noise, but I think first there should be some noise removal operation that addresses the white dots.
Any ideas?
I should mention that the algorithm to reduce the noise has to run on real time.
A median filter would be my first attempt .... possibly followed by a Gaussian blur. It really depends what you want to do with it afterwards.
For example, here's your original image after a 5x5 median filter and 5x5 Gaussian blur:
The main difficulty in your images is the large radius of the white dots.
Median and morphologic filters should be of little help here.
Usually I'm not a big fan of these algorithms, but you seem to have a perfect use case for a decomposition of your images on a functional space with a sketch and an oscillatary component.
Basically, these algorithms aim at solving for the cartoon-like image X that approaches the observed image, and that differs from Y only through the removal of some oscillatory texture.
You can find a list of related papers and algorithms here.
(Disclaimer: I'm not Jérôme Gilles, but I know him, and I know that
most of his algorithms were implemented in plain C, so I think most of
them are practical to implement with OpenCV.)
What you can try otherwise, if you want to try simpler implementations first:
taking the difference between the input image and a blurred version to see if it emphasizes the dots, in which case you have an easy way to find and mark them. The output of this part may be enough, but you may also want to fill the previous place of the dots using inpainting,
or applying anisotropic diffusion (like the Rudin-Osher-Fatemi equation) to see if the dots disappear. Despite its apparent complexity, this diffusion can be implemented easily and efficiently in OpenCV by applying the algorithms in this paper. TV diffusion can also be used for the inpainting step of the previous item.
My main point on the noise removal was to have a cleaner image so it would be easier to detect objects. However, as I tried to find a solution for the problem, I realized that it was unrealistic to remove all noise from the image using on-the-fly noise removal algorithms, since most of the image is actually noise.. So I had to find the objects despite those conditions. Here is my aproach
1 - Initial image
2 - Background subtraction followed by opening operation to smooth noise
3 - Binary threshold
4 - Morphological operation close to make sure object has no edge discontinuities (necessary for thin objects)
5 - Fill holes + opening morphological operations to remove small noise blobs
6 - Detection
Is the IR projected pattern fixed or changes over time?
In the second case, you could try to take advantage of the movement of the dots.
For instance, you could acquire a sequence of images and assign each pixel of the result image to the minimum (or a very low percentile) value of the sequence.
Edit: here is a Python script you might want to try
I've been trying to work on an image processing script /OCR that will allow me to extract the letters (using tesseract) from the boxes found in the image below.
Following alot of processing, I was able to get the picture to look like this
In order to remove the noise I inverted the image followed by floodfilling and gaussian blurring to remove noise. This is what I ended up with next.
After running it through some threholding and erosion to remove the noise (erosion being the step that distorted the text) I was able to get the image to look like this before running it through tesseract
This, while a pretty good rendering, allows for fairly accurate results through tesseract. Though it sometimes fails because it reads the hash (#) as a H or W. This leads me to my question!
Is there a way using opencv, skimage, PIL (opencv preferably) I can sharpen this image in order to increase my chances of tesseract properly reading my image? OR Is there a way I can get from the third to final image WITHOUT having to use erosion which ultimately distorted the text in the image.
Any help would be greatly appreciated!
OpenCV does has functions like filter2D that convolves arbitrary kernel with given image. In particular you can use kernels that are used for image sharpening. The main question is whether this will improve the results of your OCR library or not. The image is already pretty sharp and the noise in the image is not a result of blur. I never worked with teseract myself, but I am fairly sure that it already does all the noise reduction it could. And 'helping' him in this process may actually have opposite effect. For example any sharpening process tends to amplify noise (as opposite to noise reduction processes that usually are blurring images). Most of computer vision libraries give better results when provided with raw (unprocessed) images.
Edit (after question update):
There multiple ways to do so. The first one that I would test is this: Your first binary image is pretty clean and sharp. Instead of of using morphological operations that reduce quality of letters switch to filtering contours. Use findContours function to find all contours in the image and store their hierarchy (i.e. which contour is inside which). From all the found contours you actually need only the contours on first and second levels, i.e. outer and inner contours of each letter (contours at zero level are the outermost contours). Other contours can be discarded. Among the contours that do belong to first level you can discard those whose bounding box is too small to be a real letter. After those two discarding procedures I would expect that most of the remaining contours are the ones that are parts of the letters. Draw them on white image and run OCR. (If you want white letters on black background you will need to invert the order of vertices in the contours).
I have a basic question.
What are the advantages of upsampling an Image?
Does it help me in edge detection?
I have not found much useful information on the internet.
It depends on the image. It can help if you have extremely jagged edges. At the worst it does nothing. So, you pay in processing time for a potential improvement.
Usually we need to convert an image to a size different than its
original.
For this, there are two possible options:
Upsize the image (zoom in)
Downsize it (zoom out)
As an example, you could want to do your calculations (e.g. segmentation) on a downsized Image, later on you want to work on the original Image data again, so you upsize your Output (e.g. Segmentation mask) again.
Finding better results on upsized Images when applying edge detection
can rise from the following:
With edge detectors (e.g. canny, not only gradient computation) a blurring algorithm is usually connected. If you use some sort of blurring mask in preprocessing, it is possible that you can obtain similar behavior by its modification (decreasing, or increasing power of blurring) as in the case of image resize.
As we know Fourier Transform is sensitive to noises(like salt and peppers),
how can it still be used for image recognization?
Is there a FT expert here?
Update to actually answer the question you asked... :) Pre-process the image with a non-linear filter to suppress the salt & pepper noise. Median filter maybe?
Basic lesson on FFTs on matched filters follows...
The classic way of detecting a smaller image within a larger image is the matched filter. Essentially, this involves doing a cross correlation of the larger image with the smaller image (the thing you're trying to recognize).
For every position in the larger image
Overlay the smaller image on the larger image
Multiply all corresponding pixels
Sum the results
Put that sum in this position in the filtered image
The matched filter is optimal where the only noise in the larger image is white noise.
This IS computationally slow, but it can be decomposed into FFT (fast Fourier transform) operations, which are much more efficient. There are much more sophisticated approaches to image matching that tolerate other types of noise much better than the matched filter does. But few are as efficient as the matched filter implemented using FFTs.
Google "matched filter", "cross correlation" and "convolution filter" for more.
For example, here's one brief explanation that also points out the drawbacks of this very oldschool image matching approach: http://www.dspguide.com/ch24/6.htm
Not sure exactly what you're asking. If you are asking about how FFT can be used for image recognition, here are some thoughts.
FFT can be used to perform image "classification". It can't be used to recognize different faces or objects, but it can be used to classify the type of image. FFT calculates the spacial frequency content of the image. So for example, natural scene, face, city scene, etc. will have different FFTs. Therefore you can classify image or even within image (e.g. aerial photo to classify terrain).
Also, FFT is used in pre-processing for image recognition. It can be used for OCR (optical character recognition) to rotate the scanned image into correct orientation. FFT of typed text has a strong orientation. Same thing for parts inspection in industrial automation.
I don't think you'll find many methods in use that rely on Fourier Transforms for image recognition.
In the case of salt and pepper noise, it can be considered high frequency noise, and thus you could low pass filter your FFT before making a comparison with the target image.
I would imagine that it would work, but that different images that are somewhat similar (like both are photographs taken outside) would register as being the same image.