I'd like to detect fingers in the image and remove them for book scanning.
I have searched for finger detection and object removal projects but none of them could easily be applied or the results were not good. So, I might have to start writing my own code..
Could you recommend useful algorithms for the below items?
I think,
1) Drawing a contour (including shadows)
2) Select the finger region
3) Context-aware-fill where the finger was
I tried,
https://github.com/VPanjeta/Deep-Object-Removal
Thanks in advance
Finger image
1.color segmentation for selecting finger
2.clear border (with finger) like Matlab imclearborder. https://www.mathworks.com/help/images/ref/imclearborder.html
Related
I am developing horizontal plane detection application using ARKit. It seems to be working fine. Once floor is detected I am trying to place SCNPlane 2meter Hight and 2meter width horizontally from the centre point(detected floor). It is also working fine when floor is empty. If floor has some objects(obstacles like furniture) then SCNPlane is being placed over the object instead of the floor(under the object). How to detect only Horizontal floor excluding the objects. please guide me. thanks
When you are searching and have found the floor the ARKit will put out a grid, normally people use some kind of grid image to display this, but some don't want to show it. Once the grid has placed you place a SCNPlane, which i assume has an physical body as you say it falls towards the floor / furniture?
You can do this in 3 ways:
You can to stop the worldTrackingConfiguration once the floor has
been found.
You can once the floor has been found, fetch that Y-position and bind every object to fall towards that Y-position.
I guess you could check if the Y-position of the new detection overlaps with the floor detection, then it's fine else it's not. (I have not tested this one)
I need to be able to interact with a representation of a cilinder that has many different parts in it. When the users taps over on of the small rectangles, I need to display a popover related to the specific piece (form).
The next image demonstrates a realistic 3d approach. But, I repeat, I need to solve the problem, the 3d is NOT required (would be really cool though). A representation that complies the functional needs will suffice.
The info about the parts to make the drawing comes from an API (size, position, etc)
I dont need it to be realistic really. The simplest aproximation would be to show a cilinder in a 2d representation, like a rectangle made out of interactable small rectangles.
So, as I mentioned, I think there are (as I see it) two opposite approaches: Realistic or Simplified
Is there a way to achieve a nice solution in the middle? What libraries, components, frameworks that I should look into?
My research has led me to SceneKit, but I still dont know if I will be able to interact with it. Interaction is a very important part as I need to display a popover when the user taps on any small rectangle over the cylinder.
Thanks
You don't need any special frameworks to achieve an interaction like this. This effect can be achieved with standard UIKit and UIView and a little trigonometry. You can actually draw exactly your example image using 2D math and drawing. My answer is not an exact formula but involves thinking about how the shapes are defined and break the problem down into manageable steps.
A cylinder can be defined by two offset circles representing the end pieces, connected at their radii. I will use an orthographic projection meaning the cylinder doesn't appear smaller as the depth extends into the background (but you could adapt to perspective if needed). You could draw this with CoreGraphics in a UIView drawRect.
A square slice represents an angle piece of the circle, offset by an amount smaller than the length of the cylinder, but in the same direction, as in the following diagram (sorry for imprecise drawing).
This square slice you are interested in is the area outlined in solid red, outside the radius of the first circle, and inside the radius of the imaginary second circle (which is just offset from the first circle by whatever length you want the slice).
To draw this area you simply need to draw a path of the outline of each arc and connect the endpoints.
To check if a touch is inside one of these square slices:
Check if the touch point is between angle a from the origin at a.
Check if the touch point is outside the radius of the inside circle.
Check if the touch point is inside the radius of the outside circle. (Note what this means if the circles are more than a radius apart.)
To find a point to display the popover you could average the end points on the slice or find the middle angle between the two edges and offset by half the distance.
Theoretically, doing this in Scene Kit with either SpriteKit or UIKit Popovers is ideal.
However Scene Kit (and Sprite Kit) seem to be in a state of flux wherein nobody from Apple is communicating with users about the raft of issues folks are currently having with both. From relatively stable and performant Sprite Kit in iOS 8.4 to a lot of lost performance in iOS 9 seems common. Scene Kit simply doesn't seem finished, and the documentation and community are both nearly non-existent as a result.
That being said... the theory is this:
Material IDs are what's used in traditional 3D apps to define areas of an object that have different materials. Somehow these Material IDs are called "elements" in SceneKit. I haven't been able to find much more about this.
It should be possible to detect the "element" that's underneath a touch on an object, and respond accordingly. You should even be able to change the state/nature of the material on that element to indicate it's the currently selected.
When wanting a smooth, well rounded cylinder as per your example, start with a cylinder that's made of only enough segments to describe/define the material IDs you need for your "rectangular" sections to be touched.
Later you can add a smoothing operation to the cylinder to make it round, and all the extra smoothing geometry in each quadrant of unique material ID should be responsive, regardless of how you add this extra detail to smooth the presentation of the cylinder.
Idea for the "Simplified" version:
if this representation is okey, you can use a UICollectionView.
Each cell can have a defined size thanks to
collectionView:layout:sizeForItemAtIndexPath:
Then each cell of the collection could be a small rectangle representing a
touchable part of the cylinder.
and using
collectionView:(UICollectionView *)collectionView
didSelectItemAtIndexPath:(NSIndexPath *)indexPath
To get the touch.
This will help you to display the popover at the right place:
CGRect rect = [collectionView layoutAttributesForItemAtIndexPath:indexPath].frame;
Finally, you can choose the appropriate popover (if the app has to work on iPhone) here:
https://www.cocoacontrols.com/search?q=popover
Not perfect, but i think this is efficient!
Yes, SceneKit.
When user perform a touch event, that mean you knew the 2D coordinate on screen, so your only decision is to popover a view or not, even a 3D model is not exist.
First, we can logically split the requirement into two pieces, determine the touching segment, showing right "color" in each segment.
I think the use of 3D model is to determine which piece of data to show in your case if I don't get you wrong. In that case, the SCNView's hit test method will do most of work for you. What you should do is to perform a hit test, take out the hit node and the hit's local 3D coordinate of this node, you can then calculate which segment is hit by this touch and do the decision.
Now how to draw the surface of the cylinder would be the only left question, right? There are various ways to do, for example simply paint each image you need and programmatically and attach it to the cylinder's material or have your image files on disk and use as material for the cylinder ...
I think the problem would be basically solved.
If I have a sprite node, just a white circle somewhere on the screen, how am I able to make it so when I drag, let's say downwards and slightly to the left, the circle sprite would launch upwards and to the right and then gradually come down, like a golf shot.
Another way of explaining the mechanic is the Angry Birds game, where you launch the birds of the slingshot, the birds move in the opposite direction of your drag and gradually come down.
For another live example of the mechanics of the circle, look at the app, Desert Golfing.
Thanks, and if you don't know what I mean just comment and I'll try to explain it better.
OPTIONAL: If you do know how to do the slingshot type mechanic for the circle, do you also know how to add an arrow to the screen so users know which way the circle will launch?
Ill try to break your problem down into small steps that you could then solve yourself:
Detect the swipe:
use a UIPanGestureRecognizer. You will be able to implement a method that is called whenever a user drags their finger in a certain direction.
Here are some good references:
- Pan Gesture Official Documentation
- A very useful question that can serve you as a guide
Detect the magnitude of the swipe in order to impart an impulse
Check out the second link above. What you have to do is in the method for the gesture recognizer you will detect certain flags such as when the user starts the pan or ends the pan. Then, you can check for the location at those moments. With the Pythagorean theorem you should be able to get the distance and use that as the magnitude.
Apply impulse:
Create a physics body for your sprite and then make sure that you have gravity set inside your physics world. This allows the sprite to move in a parabolic motion. Then, use applyImpulse: on your physics body with your magnitude.
Regarding the arrow, you can easily do some delegation from within your pan gesture handler that gets the magnitude of the swipe and projects a reflection that your arrow will then show. Your question is pretty loaded so going into more detail is impossible, but best of luck. Hope this helps!
The application I am working on requires a 4px bar height with a full screen size width. I need to be able to select this 4px bar and move it around. I also can not change the size of this bar it has to be 4px in height.
This wouldn't be that big of an issue if I wasn't using OpenGL to create the object. OpenGL obviously does not have its own selection features so I am needing to program my own.
Initially after research I built a color selector to identify the object. How my color selector works is what ever x and y my finger touch returns from touchesBegan: is the pixel I grab from a screenshot of the OpenGL View. The issue with this is finger location is not precise at all. If I use the mouse it works perfect...
I decided to maybe loop through a buffer zone of the selected x and y but unfortunately a screenshot of the OpenGL view has antialiasing happens to the image when it's stored in memory and the buffer returns several shade of my objects color. I could possibly do a comparative color look up, to see if its in the range of colors but that seems overly complicated with how much I have already had to do. Plus cycling through the buffer zone isn't quick.
I also have thought maybe just remembering the location of my line on the screen and if my finger is close to that location just know that that's the one I want to select and move it around.
The future of this application can have up to 4 lines just like this so, I want something more secure then just knowing the location of where it is in memory.
What better way is there out there of handling selection of small objects?
How about maintaining an array of frames for the four objects, but expand the heights to something more manageable (8px or bigger)? Then, a touch within the larger region could be compared against the array (using CGRectContainsPoint). If you get a hit, then "snap to" the center point of the smaller (4px) rectangle before beginning the drag.
I do something like this by maintaining a list of "drop targets" for drag & drop, where it snaps to the drop target when it gets pretty close. Don't know if I'm conveying the idea very well, but it ought to work.
If the four 4px rectangles are going to be contiguous or very close together, you'll have to be able to make the selected one stand out or the user won't be able to tell which they're dragging -- but you could do that by making it bigger (maybe 6-8 px) then bringing it to the front so it overlays its adjacent neighbors.
More of an idea than an answer I guess.
John,
I would suggest a different approach. As you've discovered, touches in iOS are very imprecise. Apple usually suggests that the "hit box" for your controls be at least 40x40 points. I've gone as small as 30x30 points, but that starts to get hard.
What I would suggest you do is to factor your code so the app knows where the line is, and keeps track of it as a logical object. Then in your touch handler, interpret touches based on a large "buffer area" around the things you want the user to be able to move. If you just have a single horizontal bar, this should work great. Where you'll get into trouble is if you have multiple, thin horizontal bars that are close together. In that case you might need to rethink your app design and find another way to solve the problem.
As for the implementation details, you might add a pan gesture recognizer to your OpenGL view, and have it notify the OpenGL view of touch and drag actions. Then your OpenGL view can use knowledge of where your draggable objects are to decide how to interpret the touches.
I want to identify squares/rectangles inside my UIImageView (or UIImage).
I looked at "Very simple image recognition on iOS", but that's not quite what I'm looking.
At the moment I have an UIImageView which is given a UIImage from time to time.
Most of the UIImagees has black squares/rectangles like this:
.
But the corners may (or may not) have rounded edges.
How can I identify the first black square/rectangle's size?
The end result would be to resize my UIImageView to make the first black square in the UIImage fill the screen. Like so:
If your images will always be sharp black squares in a horizontal row, you could use corner detection to identify the rectangles, then pick out the four leftmost corners. I have three variants of corner detectors in my open source GPUImage framework based on the Harris, Noble, and Shi-Tomasi corner detection methods.
Running a GPUImageHarrisCornerDetectionFilter against your boxes with a threshold of 0.4 and sensitivity of 4.0 yields the following result:
They're a little hard to see, but red crosshairs mark where the detector found the corners of your boxes. Again, you just need to take the leftmost four points to find your target rectangle, and then simply scale your image or view so that this rectangle now fills your view.
An example of how to run such feature detection can be found in either the FilterShowcase or FeatureExtractionTest example within my framework. I describe the process by which I do this in this answer over at Signal Processing.
It seems easiest solution would be:
sum up all pixels vertically to the top-most row (like an excel table)
rows with the smallest/biggest value are your "gap" region
width can be derived from (2).
From what I understood about your question, you need to implement the Canny Edge Detection Algorithm for detecting the edges of the black borders in your image.
For this you should use the image processing framework available at the following links
Google
Github
Use the ImageWrapper *Image::cannyEdgeExtract(float tlow, float thigh)function from the Image.m file.