I'd like to register two images with the findTransformECC function offered by OpenCV.
My images have a irregular surrounding border I'd like to mask. I worked with feature based matching functions from the Feature2D-library and findHomography which worked quite well and offered a masking of image parts that should not be taken into account for estimating the transformation parameters.
findTransformECC doesn't offer such a masking, therefore I clipped the images by a centered rectangular shape. The clipped images are aligned very well after transformation. Since I'm using MOTION_EUCLIDEAN - which is just a rotation and translation - I thought I could use exactly the same transformation matrix for aligning the images of original extent - but I was proved wrong. The images aren't correct aligned after transforming them. The orientation of transformed images seem to be OK but images show a wrong translation. My thought was when input-images are clipped with exactly the same centered clipping-area and the rotation is performed around the center the final translation operation should fit as well?
Any suggestions appreciated.
In OpenCV 3.* masking is possible with the findTransformECC function. You can use the argument inputMask in the function.
Related
I'm working on a project with ARKit and I'm trying to do a perspective correction of the ARFrame.capturedImage to orient a piece of paper sitting on a detected plane so I can feed that into a CoreML model which expects images to be taken from directly overhead.
ARKit gives me the device orientation relative to the plane (ARCamera.transform, ARCamera.eulerAngles, and ARCamera.projectionMatrix all look promising).
So I have the orientation of the camera (and I know the plane is horizontal since that's all ARKit detects right now).. but I can't quite figure out how to create a GLKMatrix4 that will perform the correct perspective correction.
Originally I thought it would be as easy as transforming by the inverse of ARCamera.projectionMatrix but that doesn't appear to work at all; I'm not entirely sure what that matrix is describing.. it doesn't seem to change much based on the device orientation.
I've tried creating my own matrix using GLKMatrix4Rotate and the roll/pitch/yaw but that didn't work.. I couldn't even get it working with a single axis of rotation.
I found GLKMatrix4MakePerspective, GLKMatrix4MakeOrtho, and GLKMatrix4MakeFrustum which seem to do perspective transforms but I can't figure out how to take the information I have and translate it to the inputs of those functions to make the proper perspective transformation.
Edit:
As an example to better explain what I'm trying to do, I used the Perspective Warp tool in Photoshop to transform an example image; what I want to know is how to come up with a matrix that will perform a similar transform given the info I have about the scene.
I ended up using iOS11 Vision's Rectangle Detection and then feeding it into Core Image's CIPerspectiveCorrection filter.
I solved using OpenCV perspective transformation. (https://docs.opencv.org/trunk/da/d6e/tutorial_py_geometric_transformations.html,https://docs.opencv.org/2.4/modules/imgproc/doc/geometric_transformations.html#getperspectivetransform)
If you're able to get the corners of your paper in the scene (for example with an ARReferenceImage and project them in 2D), take them. Otherwise you can try to detect the corners through OpenCV directly (see https://stackoverflow.com/a/12636153/9298773) from the UIImage taken from sceneView.snapshot() with sceneView of type ARSceneView. In this last case I'd suggest you to binarize first and to change the MAX_CORNERS variable in the snippet at the link above to 4 (the 4 corners of your paper).
Then create a new cv::Mat with width and height of your choice respecting the proportion width and height of your paper and do perspective transform. For a guideline of this last paragraph, take a look at the section "Perspective Correction using Homography" at this link: https://www.learnopencv.com/homography-examples-using-opencv-python-c/#download. Succintly: you ask opencv to find an appropriate transform to project your prospected paper points into a perfectly rectangular plane (your new cv::Mat)
I am using GPUImageSobelEdgeDetectionFilter from project GPUImage for edge detection.
My requirement is that I want to detect edges in an image but only at centre frame of 200 x 200 and rest of the image should not be touched.
There is no direct api in framework to provide CGRect for edge detection coordinates. I do have an alternate approach of cropping down the original image and passing it for Edge Detection and finally super-imposing on the original one. But this sounds like a hack to me.
Any idea if there is a direct way to do it?
Only way to do that is as you suggest, do a crop and work with the cropped image.
If you're willing to switch over to the newer GPUImage 2, this is one of the core new features in that version of the framework. Filters can be partially applied to any region of an image, leaving the remainder of the image untouched. This includes the Sobel edge detection, and the masking of the image can be done using arbitrary shapes:
To partially apply a Sobel edge detection filter, you'd set up the filter chain as normal, then set a mask to the filter. In the below, the mask is a circle, generated to match a 480x640 image:
let circleGenerator = CircleGenerator(size:Size(width:480, height:640))
edgeDetectionFilter.mask = circleGenerator
circleGenerator.renderCircleOfRadius(0.25, center:Position.center, circleColor:Color.white, backgroundColor:Color.transparent)
The area within the circle will have the filter applied, and the area outside will simply passthrough the previous pixel colors.
This uses a stencil mask to perform this partial rendering, so it doesn't slow rendering by much. Unfortunately, I've pretty much ceased my work on the Objective-C version of GPUImage, so this won't be getting backported to that older version of the framework.
I'm working on an algorithm that counts patterns (bars) in a specific image. It seemed to me very simple at the first look, but I realized the complexity quickly.
I have tried simple thresholding, template matching (small sliding windows), edge detection...
I have just few images like this one. so I think that a machine learning algorithm can't give better results! but I still need suggestions.
I think you have enough data from your images. You need to crop from your images only the bars. You would get several dozens of small images for each image. After that you can resize all the images to some predefined size (for example 24X24 pixels) use a descriptor like HOG and SVM for the learning. For the false just use any other areas from your images.
This may not work in all cases, but since these are round bars, you can also try using circle detection. Both matlab(find circles) and opencv(hough circle transform) support this hough circle transformation. One issue is that you have to play with the parameters a bit (matlab is more simplistic than open cv) but that is true of almost any method.
These methods work better with larger images so I resized yours. You also need to know the radius of the circles to look for. If your camera position is constant, this shouldn't change much. This code is taken from the matlab documentation page I linked. It doensn't find all the circles, but some tuning may help
im = imread('http://i.stack.imgur.com/NRwUq.jpg');
%find circles doesn't work well on small images, I made the image
%three times larger, if you have larger images you should use those for
%better results
bim = imresize(im, 3*size(im));
%find and display circles
[centers, radii] = imfindcircles(bim,[8 20],'ObjectPolarity','bright',...
'Sensitivity',0.9);
imshow(bim);
h = viscircles(centers,radii);
number_of_bars = numel(centers)
I added green dots to circles the detector missed and blue X's over incorrect detection. I did these by hand, but the red circles were located by matlab.
How can I recognize some basic (usually rotated) shapes:
circle,
"empty" circle,
cross,
empty triangle,
using OpenCV? What's the most straightforward way? It would be nice if the user could "define" his own shapes somehow (load a template image, possibly).
I'd like to map each recognized shape to its type (circle, triangle etc.) and a position of its center. (And its rotation if possible).
For circles HoughCircles could be used (there's no rotation in this case, too). But what about the others? Template matching doesn't support rotation and scaling, right?...
Here's the input:
You are right that regular template matching are not rotation, scale invariant. Take a look at OpenCV's matchShapes. Internally, it uses HuMoments. You will need to use findContours to find each individual object. Now once you have done this, you will probably find matchShapes couldn't distinguish Circle from Ring. A simple way to solve this is to use the hierarchy structure from findContours. If there is a hole (large enough) inside a Circle, that's probably a Ring.
I intend to make a program which will take stereo pair images, taken by a single camera, and then correct and crop them so that when the images are viewed side by side with the parallel or cross eye method, the best 3D effect will be achieved. The left image will be the reference image, the right image will be modified for corrections. I believe OpenCV will be the best software for these purposes. So far I believe the processing will occur something like this:
Correct for rotation between images.
Correct for y axis shift.
Doing so will I imagine result in irregular black borders above and below the right image so:
Crop both images to the same height to remove borders.
Compute stereo-correspondence/disparity
Compute optimal disparity
Correct images for optimal disparity
Okay, so that's my take on what needs doing and the order it occurs in, what I'm asking is, does that seem right, is there anything I've missed, anything in the wrong order etc. Also, which specific functions of OpenCV would I need to use for all the necessary steps to complete this project? Or is OpenCV not the way to go? Much thanks.
OpenCV is great for this.
There is a whole chapter in:
And all the sample code for this in the book ships with the opencv distribution
edit: Roughly the steps are:
Remap each image to remove lens distortions and rotate/translate views to image center.
Crop pixels that don't appear in both views (optional)
Find matching objects in each view (stereoblock matching) create disparity map
Reproject disparity map into 3D model