I have a bunch of images that roughly speaking look like a bunch of stripes orientated in a random direction (so the stripes could be horizontal, vertical, at 45 degrees etc). I've noticed quite a few have a shift in them i.e. the stripes are misaligned along some line perpendicular to the stripes.
I'd like to get rid of these but I have too many images to go through by hand. Here's what I've tried so far.
If I threshold the image by curvature, I get a pretty good binary mask of the stripes (so it's just a black and white image of the stripes). I then tried to do a convolution with a kernel that I thought might emphasise the shift, but it didn't work. My kernel was also specific to pictures that were orientated at 0 degrees, so even if that had worked I would've had to then figure out how to apply the same idea to different orientations.
EDIT: adding in an example
original picture. The shift is a bit subtle so I've drawn a black line on it to make it more obvious where it is.
The image after curvature thresholding.
A vertical Sobel followed by large horizontal Gaussian filtering reveals the breaks as faint but continuous lines. There is some hope to detect them by binarization, despite parasites.
Related
I'm trying to figure out how to automatically cut some images like the one below (this is a negative film), basically, I want to remove the blank parts at the top and at the bottom. I'm not looking for complete code for it, I just want to understand a way to do it. The language is not important at this point, but I think this kind of thing normally is accomplished with Python.
I think there are several ways to do that, ranging from simple to complex. You can see the problem as detecting white rectangles or segmenting the image I would say.
I can suggest you opencv (which is available in more than one language, among which python), you can have a look here at the image processing examples
First we need to find the white part, then remove it.
Finding the white part
Thresholding
Let's start with an easy one: thresholding
Thresholding means dividing the image into two parts (usually black and white). You can do that by selecting a threshold (in your case, the threshold would be towards white - or black if you invert the image). By doing so, however, you may also threshold some parts of the images (for example the chickens and the white part above the kid). Do you have information about position of the white stripes? Are they always on top and bottom of the image? If so, you can apply the thresholding operation only on the top 25% and bottom 25% of the image. You would then most likely not "ruin" the image.
Finding the white rectangle
If that does not work or you would like to try something else, you can see the white stripes as rectangles and try to find their contour. You can see how in this tutorial. In this case you do not get a binary image, but a bounding box of the white areas. You most likely find the chickens also in this case, but by looking at the bounding box is easy to understand which one are correct and which one not. You can also check this calculating areas of the bounding box (width * height) and keep only the big ones.
Removing the part
Once you get the binary image (white part and not white part) or the bounding box, you have to crop the image. This also can be done in several ways, but I think the easiest one would be cropping by just selecting the central part of the image. For example, if the image has H pixels vertically, you would keep only the pixel from H1 (the height of the first white space) to H-H2 (where H2 is the height of the second white space). There is no tutorial on cropping, but several questions here on SO, as for example this one.
Additional notes
You could use more advanced segmentation algorithms as the Watershed one or even learn and use advanced techinques as machine learning to do that (here an article), as you can see the rabbit hole is pretty deep in this case. But I believe that would be an overkill and already the easy techniques would give you some results in this case.
Hope this was helpful and have fun!
After segmentation of Objects in noisy data, I need to fit the best possible retangulat fit.
currently I just use opencv findContours and minAreaRect which will give me all around. I know that those objects will always be horizontal in the image with a maximum small angle like in this image.
This can be seen as the green rectanlge in the images, however I would need something like the red drawn rectangles, or even just the middle line (blue) since thats what I do need in the end.
Further, I also do have some conjunctions, like seen in this image:
Here I want to only detect the horizontal part and maybe know that there could be a junction.
Any idea how to solve this problem? I need some fast approach and have not found anything feasable yet.
Got much better results using distance transform (as mentioned from #Micka) on the masked Image, afterwars find the Line with the biggest distance as the middle of the rectangle (using some Filters, cuting off the curve) and in the End fitting a Line on the middle estimate.
Using GPUImage, I am able to detect corners of a book/page in an image. But sometimes, it will pass more than 4 points, in which case I will need to process and figure out the best rectangle out of these points. Here's an example:
What's the most efficient way to figure out the best rectangle in this case?
Thanks
If you're using a corner detection algorithm, then you can filter results based on the relative strength of the detected corner. The contrast at the book corners relative to your current background appears to be much stronger than the contrast at the point found in the wood grain. Are there relative magnitudes associated with each point, or do you just get the points? Setting thresholds for edge strengths can mean a lot of fiddling unless the intensities of the foreground and background are relatively constant.
Your sample image could be blurred or morphed. For example, the right morphological "close" on light pixels could eliminate the texture in the wood grain without having an effect on the size and shape of the book. (http://en.wikipedia.org/wiki/Mathematical_morphology)
Another possibility is to shrink the image to a much smaller size and then perform detection on that. Resizing the image will tend to wipe out tiny details such as whatever wood grain pattern is currently being detected.
Picking the right lens and lighting can make the image easier to process. Try to simplify the image as much as possible before processing it. As mentioned above, "dark field" lighting that would illuminate just the book edges would present a much simpler image for processing. Writing down the constraints can make it more obvious which solution will be most robust and simplest to implement. Finding any rectangle anywhere in an image is very difficult; it's much easier to find a light rectangle on a dark background if the rectangle is at least 100 x 100 pixels in size, rotated no more than 15 degrees from square to the image edges, etc.
More involved solutions can be split into two approaches:
Solving the problem using given only 4 or more (x,y) points.
Using a different image processing technique altogether for the sample image.
1. Solving the program given only the points
If you generally only have 5 or 6 points, and if you are confident that 4 of those points will belong to the corners of the rectangles that you want, then you can try this:
Find the convex hull of all points. The convex hull is the N-gon that completely encompasses all points. If the points were pegs sticking up, and if you stretched a rubber band around them and let it snap into place, then the final shape of the rubber band is a convex hull. Algorithms that find convex hulls typically return a list of points that ordered counterclockwise from the bottom leftmost point.
Make a copy of your point list and remove points from the copy until only four points remain. These four remaining points will still be ordered counterclockwise.
Calculate the angle formed by each set of three successive points: points 1, 2, 3, then 2, 3, 4, then 3, 4, 1, and so on.
If an angle is outside a reasonable tolerance--less than 70 degrees or greater than 110 degrees--skip back to step 2 and remove the next point (or set of points).
Store the min and max angles for each set of 4 points.
Repeat steps 2 - 6, removing a different point (or points) each time.
Track the set of points for which the min and max angles are closest to 90 degrees.
http://en.wikipedia.org/wiki/Convex_hull
There are a number of other checks and constraints that could be introduced. For example, if the point-to-point distances for 3 successive points in the convex hull (pts N to N+1, and N+1 to N+2) are close to the expected width and height of the book, then you might mark these as known good points and only test the remaining points to see which is the fourth point.
The technique above can get unwieldy if you get quite a few points, but it may work if two or three of the book corner points are expected to be found on the convex hull.
For any geometric problem, I always recommend checking out GeometricTools.com, which has a lot of great, optimized source code for all sorts of problems. It's very handy to have the book as well, especially if you can find a cheap copy using AddAll.com.
http://www.geometrictools.com/
2. Other image processing techniques for your sample image
Although I could be wrong, it appears that GPUImage doesn't have many general-purpose image processing algorithms. Some other image processing algorithms could make this problem much simpler to solve.
Though there isn't space to go into it here, one of the keys to successful image processing is appropriate lighting. Make sure you're lighting is consistent. A diffuse light that evenly illuminates the book and the background would work well. You can simplify the problem using funkier lighting: if you have four lights (or a special ring light), you can provide horizontal illumination from the top, bottom, left, and right that will cause the edges of the book to appear bright and other surfaces to appear dark.
http://www.benderassoc.com/mic/lighting/nerlite/Darkfield.htm
If you can use some other GPU libraries to do image processing, then one of the following techniques could work nicely:
Connected component labeling (a.k.a. finding blobs). It shouldn't be too hard to use either binary thresholding or a watershed algorithm to separate the white blob that is the book from the rest of the background. Once the blob for the book is identified, finding the corners is easier. (http://en.wikipedia.org/wiki/Connected-component_labeling) In OpenCV you can find the "contours."
Generate an list of edge points, then have four separate line-fitting tools search from top to bottom, right to left, bottom to top, and left to right to find the four strong (and mostly straight) edges associated with the book. In your sample image, though, either the book cover is slightly warped or the camera lens has introduced barrel distortion.
Use a corner detector designed to find light corners on a dark background. If you will always be looking for a white book on a wood grain background, you can create a detector to find white corners on a brown background.
Use a Hough technique to find the four strongest lines in the image. (http://en.wikipedia.org/wiki/Hough_transform)
The algorithmic technique that works best will depend on your constraints: are you looking for rectangles only of a certain size? is the contrast between foreground and background consistent? can you introduce lighting to simplify the appearance of the image? and so on.
I uploaded an example image for better understanding: http://www.imagebanana.com/view/kaja46ko/test.jpg
In the image you can see some scanlines and a marker (the white retangle with the circle in it). I want OpenCV to go along a specified area (in the example outlined trough the scanlines) that should be around 5x5. If that area contains a gradient from black to white, I want OpenCV to save the position of that area, so that I can work with it later.
The final result would be to differentiate between the marker and the other retangles separated trough black and white lines.
Is something like that possible? I googled a lot but I only found edge detectors but that's not what I want, I really need the detection of the black to white gradient only.
Thanks in advance.
it would be a good idea to filter out some of the areas by calculating their histogram.
You can use cvCalcHist for the task, then you can establish some threshold to determine if the black-white pixels percentage corresponds to that of a gradient. This will not solve the task but it will help you in reducing complexity.
Then, you can erode the image to merge all the white areas. After applying threshold, it would be possible to find connected components (using cvFindContours) that will separate images in black zones or white zones. You can then detect gradients by finding 5x5 areas that contain both a piece of a white zone and black zone simultaneously.
hope it helps.
Thanks for your answerer dnul, but it didn't really help me work this out. I though about a histogram to approach the problem but it's not quite what I want.
I solved this problem by creating a 40x40 matrix which holds 5x5 matrix's containing the raw pixel data in all 3 channels. I iterated trough each 40px-area and inside iterated trough each border of 5px-area. I checked each pixel and saved the ones which are darker then a certain threshold a storage.
After the iteration I had a rough idea of how many black pixels their are, so I checked each one of them for neighbors with white-pixels in all 3 channels. I then marked each of those pixels and saved them to another storage.
I then used the ransac algorithm to construct lines out of these points. It constructs about 5-20 lines per marker edge. I then looked at the lines which meet each other and saved the position of those that meet in a square angle.
The 4 points I get from that are the edges of the marker.
If you want to reproduce this you would have to filter the image in beforehand and apply a threshold to make it easier to distinguish between black and white pixels.
A sample picture, save after finding the points and before constructing the lines:
http://www.imagebanana.com/view/i6gfe6qi/9.jpg
What you are describing is edge detection. This is exactly how, say, the Canny edge detector works. It looks for dark pixels near light pixels, and based on a threshold that you pass in (There is also the adaptive canny, which figures out the threshold for you), and sets them to all black or all white (aka 'marks' them).
See here:
http://docs.opencv.org/doc/tutorials/imgproc/imgtrans/canny_detector/canny_detector.html
I'm trying to render 6 spot lights to create a point light for a shadow mapping algorithm.
I'm not sure if I'm doing this right, I've more or less followed the instructions here when setting up my view and projection matrices but the end result looks like this:
White areas are parts which are covered by one of the 6 shadow maps, the darker areas are ones which aren't covered by the shadowmaps. Obviously I don't have a problem with the teapots and boxes having their shadows projected onto the scene, however as you can see the 6 shadow maps have blindspots. Is this how a cubed shadow map is supposed to look? It doesn't look like a shadowmap of a point light source...
Actually you can adjust your six spots to have cones that perfectly fill each face of your cubemap. You can achieve this by setting each cone's aperture to create a circumscribed circle around each cubemap face. In this case you don't have to worry about overlapping, since the would be overlapping parts are out of the faces' area.
In other terms: adjust the lights' projection matrix' FOV, so it won't the view frustum that includes the light cone, but the cone will include the view frustum.
The a whole implementation see this paper.
What you're seeing here are a circle and two hyperbolas -- conic sections -- exactly the result you might expect if you took a double ended cone and intersected it with a plane.
This math may seem removed from the situation but it explains your problem. A spotlight creates a cone of light, and you can't entirely fill a solid space with a bunch of cones coming from the same point. (I'd suggest rolling up a bunch of pieces of paper and taping them together at the points to try it out.)
However, as you get far from the origin of your simulated-point-source, the cones converge to their assymptotes, and there is an infinitesimally-narrow gap in the light.
One option to solve this is to change the focus of the cones so that they overlap slightly -- this will create areas that are overexposed, but the overexposure will only become obvious as you get farther away. So long as all of your objects are near the point light source, this might not be much of an issue.
Another option is to move the focus of all of the lights much closer to their sources. This way, they'd converge to their assymptotes more quickly.