It seems HitResult only gives us intersection with a surface (plane) or a point cloud. How can I get a point in the middle of air with my click, and thus put an object floating in the air?
It really depends on what you mean by "in the air". Two possibilities I see:
"Above a detected surface" Do a normal hit test against a plane, and offset the returned pose by some Y distance to get the hovering location. For example:
Pose.makeTranslation(0, 0.5f, 0).compose(hitResult.getHitPose())
returns a pose that is 50cm above the hit location. Create an anchor from this and you're good to go. You also could just create the anchor at the hit location and compose with the y translation each frame to allow for animating the hover height.
"Floating in front of the current device position" For this you probably want to compose a translation on the right hand side of the camera pose:
frame.getPose().compose(Pose.makeTranslation(0, 0, -1.0f)).extractTranslation()
gives you a translation-only pose that is 1m in front of the center of the display. If you want to be in front of a particular screen location, I put some code in this answer to do screen point to world ray conversion.
Apologies if you're in Unity/Unreal, your question didn't specify so I assumed Java.
The reason why you see so often a hit result being interpreted as the desired position by the user is that actually there is no closed-form solution for this user interaction. Which of the infinite possible positions along the ray starting from the camera pointing towards the scene was desired by the User? 2D coordinates from a click still leave the third dimension undefined.
As you said "middle of the air", why not take the centre between the camera position and the hitresult?
You can extract the current position using pose.getTranslation https://developers.google.com/ar/reference/java/com/google/ar/core/Pose.html#getTranslation(float[],%20int)
Related
I built a complete structure as a node (with its child nodes) and the user will walk through it using ARKit.
At some point, if the user cannot continue because of some real obstacle in the real world, I added a "pause" button which should freeze whatever the user currently sees in front of the camera, the user could then move freely to some other open space and when the user will release the pause button he/she will be able to resume where they left off (only someplace else in the real world).
A while ago I asked about it in the Apple Developer forum and an Apple Frameworks Engineer gave the following reply:
For "freezing" the scene, you could transform the anchor's position (in world coordinates) to camera coordinates, and then anchor your content to the camera. This will give you the effect that the scene is "frozen", i.e., does not move relative to the camera.
I'm currently not using an anchor because I don't necessarily need to find a flat surface. Rather, my node is placed at a certain position relative to where we start at (0,0,0).
My question is how do I exactly do what the Apple engineer told me to do?
I have the following code which I'm still stuck with. When I add the node to the camera (pointOfView, last line of the code below), it does freeze in place, but I can't get it to freeze in the same position and orientation as it was before it was frozen.
#IBAction func pauseButtonClicked(_ sender: UIButton) {
let currentPosition = sceneView.pointOfView?.position
let currentEulerAngles = sceneView.pointOfView?.eulerAngles
var internalNodeTraversal = lastNodeRootPosition - currentPosition! // for now, lastNodeRootPosition is (0,0,0)
internalNodeTraversal.y = lastNodeRootPosition.y + 20 // just so it’s positioned a little higher in front of the camera
myNode?.removeFromParentNode() // remove the node from the Real World view. Looks like this line has no effect and just adding the node as a child to the camera (pointOfView) is enough, but it feels more right to do this anyway.
myNode?.position = internalNodeTraversal // the whole node is moved respectively in the opposite direction from the root to where I’m standing to reposition the camera in my current position inside the node
// myNode?.eulerAngles = (currentEulerAngles! * -1) — this code put the whole node in weird positions so I removed it
myNode?.eulerAngles.y = currentEulerAngles!.y * -1 // opposite orientation of the node so the camera will be oriented in the same direction
myNode?.eulerAngles.x = 0.3 // just tilting it up a little bit to have a better view, more similar to the view as before it was locked to the camera
// I don’t think I need to change the eulerAngles.z
myNode!.convertPosition(internalNodeTraversal, to: sceneView.pointOfView) // I’m not sure I wrote this correctly. Also, this line doesn’t seem tp change anything
sceneView.pointOfView?.addChildNode(myNode!) // attaching the node to the camera so it will remain stuck while the user moves around until the button is released
}
So I first calculate where in the node I'm currently standing and then I change the position of the node in the opposite direction so that the camera will now be in that position. That seems to be correct.
Now I need to change the orientation of the node so that it will point in the right direction and here things get funky. I've been trying so many things for days now.
I use the eulerAngles for the orientation. If I set the whole vector multiplied by -1, it would show weird orientations. I ended up only using the eulerAngles.y which is the left/right orientation and I hardcoded the x orientation (up/down).
Ultimately what I have in the code above is the closest that I was able to get. If I'm pointing straight, the freeze will be correct. If I turn just a little bit, the freeze will be pretty close as well. Almost the same as what the user saw before the freeze. But the more I turn, the more the frozen image is off and more slanted. At some point (say I turn 50 or 60 degrees to the side) the whole node is off the camera and cannot be seen.
Somehow I have a feeling that there must be an easier and more correct way to achieve the above.
The Apple engineer wrote to "transform the anchor's position (in world coordinates) to camera coordinates". For that reason I added the "convertPosition" function in my code, but a) I'm not sure I used it correctly and b) it doesn't seem to change anything in my code if I have that line or not.
What am I doing wrong?
Any help would be very much appreciated.
Thanks!
I found the solution!
Actually, the problem I had was not even described as I didn't think it was relevant. I built the AR nodes 2 meters in front of the origin (-2 for the z-coordinate) while the center of my node was still at the origin. So when I changed the rotation or eulerAngles, it rotated around the origin so my nodes moved in a large curve and in fact also changed their position as a result.
The solution was to use a simdPivot. Instead of changing the position and rotation of the node itself, I created a translation matrix and a rotation matrix which was at the point of the camera (where the user is standing) and I then multiplied both matrices. Now when I added the node as a child of the camera (pointOfView) this would freeze the image and in effect show exactly what the user was seeing before it was frozen as the position is the same and the rotation is exactly around the user's standing position.
My application involves a user pointing their phone like a remote—as opposed to an 'AR window'—at virtual nodes. (See illustration.) These nodes are manually placed in the environment and aren't anchored to a physical plane. They will be floating.
To do this, I would need to cast along the Y axis from the camera/screen center and test for any hits along that ray. In Unity, I would do this by doing a Raycast hit test from the camera's transform.top, but I'm not sure how to do this in SceneKit.
I have had success using sceneView.hitTest on the Z axis, but this isn't my ultimate use case. I have also tried using scene.hitTestWithSegment(), using the camera world space as the from but I'm not sure how to get the to. I'm guessing I would need to cast a ray along the local Y axis and get a point along it, which just leads me back to my original problem.
Any tips on where to look? Is there a similar convenience to casting from a local transform.top in SceneKit?
Thank you, in advance!
Illustration:
** Update **
Per Josh Homann's answer below, I wound up solving it like so:
// Set local segment end and convert to world space
let segmentLocalEnd:SCNVector3 = SCNVector3(0,20,0)
let segmentWorldEnd:SCNVector3 = sceneView.pointOfView!.convertPosition(segmentLocalEnd, to: nil)
// Get hits
let hitResults:[SCNHitTestResult] = scene.hitTestWithSegment(
from: sceneView.pointOfView!.worldPosition,
to: segmentWorldEnd,
options: [SCNHitTestOption.firstFoundOnly.rawValue: true]
)
You already have most of the answer. In an AR view you would project a ray from the camera along the look at vector. In your case though you want to go perpendicular to the look at vector, so instead you just project a ray from the camera along the up vector (0,1,0) (assuming you haven't rotated your space and positive Y is up). In SceneKit you can't actually test along a infinite ray, so just pick a sufficiently large value and pass it into hitTestWithSegment(from:to:options:)
Using SceneKit
I want to make the gray transparent box to disappear and only show the colored boxes when the user zooms in.
So I want to detect when that box's edges are starting to fall off the screen as I zoom, so I can hide the gray box accordingly.
First thoughts, but there may be better solutions:
You could do an unprojectPoint on the node and check against screen coordinates, do the +/- math on object size and skip Z. I "think" that would work
You can do some physics based collision detection against an invisible box or plane geometries that acts as your screen edges, has some complexity if your view is changing, but testing would be easy - just leave visible until you get what you want, then isVisible=false
isNode(insideFrustomof: ) - returns boolean on whether it "might" be visible. I'm assuming "might" means obscured by other geometry which in your case, shouldn't matter (edit) on second thought, that doesn't solve your problem but I'll leave it in here for reference.
I'm working on developing iOS AR application using ARKit + Core location. And the points which are displayed on the map using coordinates move from place to place when I go. But I need they are displayed on the same place.
Here you can see the example of what I mean:
https://drive.google.com/file/d/1DQkTJFc9aChtGrgPJSziZVMgJYXyH9Da/view?usp=sharing
Could you help to handle with this issue? How can I have fixed places for points using coordinates? Any ideas?
Thanks.
Looks like you attach objects to planes. However, when you move the ARKit extends the existing planes. As a result if you put points, for example, at the center of the plane, then the center is always updated. You need to recalculate the coordinates of the point and place objects correctly.
The alternative is not to add objects to planes (or in relation to them). If you need to "put" object on a plane, then the best way is to wait, until the plane will be directed enough (it will not change his direction significantly if you will move), then select a point on the plane where you want to put your object, then convert this point coordinate to global coordinates (as a result if plane will change his size the coordinate you have will not be changed at all), and finally put object in root (or another object that it's not related to the plane).
I am working on a small education demo which should measure height and width of the object using iOS camera.
EDIT:
I have a new theory to measure the width of an object.
In above image, if i can get Angle α and Angle ß, i can get width of the unknown side by using trigonometry formulas. I have the values of b1 and b2 already.
OLD:
Right now, i am focusing on measuring length only.
As per my knowledge it should be 3 step process.
User snaps one end of the object.
User snaps other end of the object.
User snaps center of the object. (Suggest me a better way for these please)
I get the approximate measurements using above process, but for the 3rd step, in which user snaps the center of the object. I want to show pointer location on screen (as camera overlay) to help user determine the center of the object.
This is how i am doing it right now.
How can i draw pointer location for 3rd step?
Note: Please suggest alternative/best way to make it possible. I would love another suggestions. Thanks.!!
First of all I must appreciate your work you have done till now. Another good thing is the way of explaining, salute!!!!!
After reading of your question, I feel that you dont need code, you can do it. I think you need direction only.
As per your explanation, you want to record angle of rotation of the device.
If you want to measure angle of rotation, you have to use compas readings. But compas readings will change if user tild the device. So you have to use accelerometer to measure tilding of device.
In short you have to make some combination and equation of both compas and accelerometer readings. Use compas to measure angle and use accelerometer to measure tilding of device.
If you want further information to implement it, you can ask me.
Hope this will help you....