Finding Area that CGPath Intersects CGRect - ios

I'm working on an iOS app where I need to be able to see how much of a CGPath is within the screen bounds to ensure that there is enough for the user to still touch. The problem is, when the shape is in the corner, all the methods I would normally use (and everything I can think to try) fails when the path is in the corner.
Here's a pic:
How can I calculate how much of that shape is on screen?

Obvious answers are to do it empirically by pixel painting or to do it analytically by polygon clipping.
So to proceed empirically you'd create a CGBitmapContext the size of your viewport, clear it to a known colour such as (0, 0, 0), paint on your polygon in another known colour, like say (1, 1, 1) then just run through all the pixels in the bitmap context and add up the total number you can find. That's probably quite expensive, but you can use lower resolution contexts to get more approximate results if that helps.
To proceed analytically you'd run a polygon clipping algorithm such as those described here to derive a new polygon from the original, which is just that portion of it that is actually on screen. Then you'd get the area of that using any of the normal formulas.
It's actually a lot easier to clip a convex polygon than a concave one, so if your polygons are of a fixed shape then you might consider using a triangulation algorithm, such as ear clipping or decomposition to monotone edges and then performing the clipping and area calculation on those rather than the original.

You could get an approximation using this approach:
Let reg be the intersection of the screen with the bounding box of the shape
Choose N random points in reg and check if they are contained in the shape
The area of the shape can be estimated by (area of reg)*(number of points contained in shape)/N
You can choose the parameter N based on the size of reg to get a good and fast approximation

Related

Extract contour path from an image

Let's say I have a 16x16 black & white bitmap image
Here white pixels indicate empty space and black pixels indicate filled space.
I want to extract all of it's contour lines that surround black pixels, including holes and nested contour lines. (see the second image)
Let's define a coordinate space for pixels
top-left pixel -> index (0,0)
top-right pixel -> index (15,0)
bottom-left pixel -> index (0,15)
bottom-right pixel -> index (15,15)
Contour lines also have their coordinate space
top-left corner of top-left pixel -> index (0,0)
top-right corner of top-right pixel -> index (16,0)
bottom-left corner of bottom-left pixel -> index (0,16)
bottom-right corner of bottom-right pixel -> index (16,16)
Finally, contour lines are defined as a sequence of points in that coordinate space.
On the second image I marked 3 contours to demonstrate what the desired output should look like.
Path1 (RED): 1(1,0) 2(2,0) 3(2, 3) 4(3,3) 5(0,3) ... 23(4,4) 24(1, 4)
Hole1 of Path1 (BLUE): 1(7,5) 2(7,6) 3(6,6) ... 13(11,6) 14(11,5)
Path2 (RED again): 1(8,6) 2(10,6) 3(10,8) 4(8,8)
...
Note that the order of points in contour is important. Winding difference for holes is not that important, but we should somehow indicate "hole" property of that contour.
I solved this problem using ClipperLib, but it is more like a brute-force approach in my opinion, if we ignore what happens inside the ClipperLib.
Here's a brief description of the algorithm.
First, define a 16x16 subject polygon from which we will be subtracting all white pixels
Scan the image matrix row by row
On each row extract all contiguous white rectangle shapes as a clipping polygon
Do the polygon clipping by subtracting all collected white rectangular polygons from initial 16x16 subject polygon
Extract path data (including holes) from ClipperLib's PolyTree solution
I'm wondering if there is a better way to solve this problem?
Using ClipperLib seems overkill here, as it addresses general polygons by means of complex intersection detection and topological reconstruction algorithms, whereas your problem is more "predictable".
You can proceed in two steps:
use a standard contouring algorithm, such as used by cv.findContours. (It is an implementation of "Satoshi Suzuki and others. Topological structural analysis of digitized binary images by border following. Computer Vision, Graphics, and Image Processing, 30(1):32–46, 1985.")
from the contours, which link pixel centers to pixel centers, derive the contours that follow the pixel edges. This can probably be achieved by studying the different configurations of sequences of three pixels along the outline.
You can use boundary tracing algorithms for this. I personally use Moore-Neighbor tracing, because it's intuitive and straightforward to implement. You first find the boundary contours, and then come up with a hole searching algorithm (you may need to combine parts of scanline fill algorithm). Once you find a hole, you can apply the same boundary tracing algorithm, but in opposite direction.
You can definitely use libraries like OpenCV to find contours, but it my experience, it may produce degenerate output incompatible with other libraries, such as poly2tri used to decompose polygons into triangles.
If we take your input sample image, then the red path could be considered self-intersecting (vertices 7 and 23 are touching), which may lead to failed polygon decomposition. You may need to figure out a way to find and treat those objects as separate, if that's a problem. However, the newest Clipper2 is going to have triangulation unit that could handle such degenerate input, if you ever need to solve this problem down the road.

Finding vertexes for construction of minimum size bounding box / convex hull

I have an array of data from a grayscale image that I have segmented sets of contiguous points of a certain intensity value from.
Currently I am doing a naive bounding box routine where I find the minimum and maximum (x,y) [row, col] points. This obviously does not provide the smallest possible box that contains the set of points which is demonstrable by simply rotating a rectangle so the longest axis is no longer aligned with a principal axis.
What I wish to do is find the minimum sized oriented bounding box. This seems to be possible using an algorithm known as rotating calipers, however the implementations of this algorithm seem to rely on the idea that you have a set of vertices to begin with. Some details on this algorithm: https://www.geometrictools.com/Documentation/MinimumAreaRectangle.pdf
My main issue is in finding the vertices within the data that I currently have. I believe I need to at least find candidate vertices in order to reduce the amount of iterations I am performing, since the amount of points is relatively large and treating the interior points as if they are vertices is unnecessary if I can figure out a way to not include them.
Here is some example data that I am working with:
Here's the segmented scene using the naive algorithm, where it segments out the central objects relatively well due to the objects mostly being aligned with the image axes:
.
In red, you can see the current bounding boxes that I am drawing utilizing 2 vertices: top-left and bottom-right corners of the groups of points I have found.
The rotation part is where my current approach fails, as I am only defining the bounding box using two points, anything that is rotated and not axis-aligned will occupy much more area than necessary to encapsulate the points.
Here's an example with rotated objects in the scene:
Here's the current naive segmentation's performance on that scene, which is drawing larger than necessary boxes around the rotated objects:
Ideally the result would be bounding boxes aligned with the longest axis of the points that are being segmented, which is what I am having trouble implementing.
Here's an image roughly showing what I am really looking to accomplish:
You can also notice unnecessary segmentation done in the image around the borders as well as some small segments, which should be removed with some further heuristics that I have yet to develop. I would also be open to alternative segmentation algorithm suggestions that provide a more robust detection of the objects I am interested in.
I am not sure if this question will be completely clear, therefore I will try my best to clarify if it is not obvious what I am asking.
It's late, but that might still help. This is what you need to do:
expand pixels to make small segments connect larger bodies
find connected bodies
select a sample of pixels from each body
find the MBR ([oriented] minimum bounding rectangle) for selected set
For first step you can perform dilation. It's somehow like DBSCAN clustering. For step 3 you can simply select random pixels from a uniform distribution. Obviously the more pixels you keep, the more accurate the MBR will be. I tested this in MATLAB:
% import image as a matrix of 0s and 1s
oI = ~im2bw(rgb2gray(imread('vSb2r.png'))); % original image
% expand pixels
dI = imdilate(oI,strel('disk',4)); % dilated
% find connected bodies of pixels
CC = bwconncomp(dI);
L = labelmatrix(CC) .* uint8(oI); % labeled
% mark some random pixels
rI = rand(size(oI))<0.3;
sI = L.* uint8(rI) .* uint8(oI); % sampled
% find MBR for a set of connected pixels
for i=1:CC.NumObjects
[Y,X] = find(sI == i);
mbr(i) = getMBR( X, Y );
end
You can also remove some ineffective pixels using some more processing and morphological operations:
remove holes
find boundaries
find skeleton
In MATLAB:
I = imfill(I, 'holes');
I = bwmorph(I,'remove');
I = bwmorph(I,'skel');

Calculation of center point for the localization of robot in 3D data

I am trying to find a reliable method to calculate the corner points of a container. From these corner point’s idea is to calculate the center point of the container for the localization of robot, it means that the calculated center point will be the destination of robot in order to pick the container. For this I am looking for any suggestions to calculate the corner points or may be if any possibility to calculate the center point directly. Up to this point PCL library C/C++ is used for the processing of the 3D data.
The image below is the screenshot of the container.
thanks in advance.
afterApplyingPassthrough
I did the following things:
I binarized the image (black pixels = 0, green pixels = 1),
inverted the image (black pixels = 1, green pixels = 0),
eroded the image with 3x3 kernel N-times and dilated it with same kernel M-times.
Left: N=2, M=1;Right: N=6, M=6
After that:
I computed contours of all non-zero areas and
removed the contour that surrounded entire image.
This are the contours that remained:
I do not know how "typical" input image looks like in your case. Since I only have access to one sample image, I would rather not speculate about "general solution" that will be suitable for you. But to solve this particular case, you could analyze every contour in the following way:
compute rotatated rectangle that fits best around your contour (you need something similar to minAreaRect from OpenCV)
compute areas of rectangle and contour interior
if the difference between contour area and the area of the rotated bounding rectangle is small, the contour has approximately rectangular shape
find the contour that is both rectangular and satisfies some other condition (for example: typical area of the container). Assume that this belongs to container and compute its center.
I am not claiming that this is a solution that will work well in real world scenarios. It is also not fast. You should view it as a "sketch" that shows how to extract some useful information.
I assume the wheels maintain the cart a known offset from the floor and you can identify the floor. Filter out all points which are too close to the floor (this will remove wheels and everything but cart which will help limit data and simplify later steps.
If you isolate the cart, you could apply a simple average point (centroid), alternately, if that is not precise, you could try finding the bounding box of the isolated cart (min max in primary directions) and then take the centroid of that bounding box (this should be more accurate, but will still need a slight vertical offset due to the top handles).
If you can not isolate the cart or the other methods are not working well, you could try using PCL sample consensus specifically SACMODEL_LINE. This will be an involved strategy, but will give very solid results, basically run through and find each line and subtract its members from the cloud so as to find the next best line. After you have your 4 primary cart lines, use their parameters to find your centroid. *this would also be robust against random items being in or on the cart as well as carts of various sizes (assuming they always had linear perpendicular walls)

Opencv: How to Fit an Image into Non-Rectangular Contour?

I want to fit an image of a clown like face into a contour of another face (a person).
I am detecting the persons face and getting a elliptical-like contour.
I can figure out the center, radius, highest, lowest, left-most and right-most points.
How do I fit the clown face (a square image which I can make elliptical by cutting the face out of the empty background of a png and then detecting the contour) into the persons face?
Or at the least, how do I fit a polygon into another polygon.
I can fit a rectangular image into a rectangular contour with ease, but faces aren't that shape.
Python preferable, but C++ is also manageable, thank you.
Edit: Visual representation as requested:
I have
and I want to make it like this:
but I want the clown face to stretch over the guys face and fit within the blue contour.
I think the keyword you are looking for is Active Appearance Models. First, you need to fit a model to first face (such as this one), which lays inside the contour. Then, you should fit the same model to the clown face. After that, since you have fitted same model to both faces, you can stretch it as you need.
I haven't use AAM myself and I'm not an expert about it, so my explanation might not be enough or might not be exactly correct, but I'm sure it will give you some insight.
A simple and good answer to this question is to find the extreme top, bottom, left, and right points on your contour (head) and then resize your mask to match the aspect ration and place it to cover the 4 points.
Because human heads are elliptical you can use fitEllipse() to give you those 4 points. This will automagically fix any problems with the person tilting their head because regardless of the angle you will know which point is top, bottom, left, and right.
The relevant code for finding the ellipse is:
vector<Point> contour;
// Do whatever you are doing to populate this vector
RotatedRect ellipse = fitEllipse(Mat(contour));
There is also an example as well as documentation for RotatedRect.
// Resize your mask with these sizes for optimum fit
ellipse.size.width
ellipse.size.height
You can rotate your image like this.
UPDATE:
You may also want to find the contour's extreme points to know how much you need to scale your image to ensure that all of the face is covered.

Understanding Distance Transform in OpenCV

What is Distance Transform?What is the theory behind it?if I have 2 similar images but in different positions, how does distance transform help in overlapping them?The results that distance transform function produce are like divided in the middle-is it to find the center of one image so that the other is overlapped just half way?I have looked into the documentation of opencv but it's still not clear.
Look at the picture below (you may want to increase you monitor brightness to see it better). The pictures shows the distance from the red contour depicted with pixel intensities, so in the middle of the image where the distance is maximum the intensities are highest. This is a manifestation of the distance transform. Here is an immediate application - a green shape is a so-called active contour or snake that moves according to the gradient of distances from the contour (and also follows some other constraints) curls around the red outline. Thus one application of distance transform is shape processing.
Another application is text recognition - one of the powerful cues for text is a stable width of a stroke. The distance transform run on segmented text can confirm this. A corresponding method is called stroke width transform (SWT)
As for aligning two rotated shapes, I am not sure how you can use DT. You can find a center of a shape to rotate the shape but you can also rotate it about any point as well. The difference will be just in translation which is irrelevant if you run matchTemplate to match them in correct orientation.
Perhaps if you upload your images it will be more clear what to do. In general you can match them as a whole or by features (which is more robust to various deformations or perspective distortions) or even using outlines/silhouettes if they there are only a few features. Finally you can figure out the orientation of your object (if it has a dominant orientation) by running PCA or fitting an ellipse (as rotated rectangle).
cv::RotatedRect rect = cv::fitEllipse(points2D);
float angle_to_rotate = rect.angle;
The distance transform is an operation that works on a single binary image that fundamentally seeks to measure a value from every empty point (zero pixel) to the nearest boundary point (non-zero pixel).
An example is provided here and here.
The measurement can be based on various definitions, calculated discretely or precisely: e.g. Euclidean, Manhattan, or Chessboard. Indeed, the parameters in the OpenCV implementation allow some of these, and control their accuracy via the mask size.
The function can return the output measurement image (floating point) - as well as a labelled connected components image (a Voronoi diagram). There is an example of it in operation here.
I see from another question you have asked recently you are looking to register two images together. I don't think the distance transform is really what you are looking for here. If you are looking to align a set of points I would instead suggest you look at techniques like Procrustes, Iterative Closest Point, or Ransac.

Resources