I want to let the user control an object moving over the surface of a static sphere. Using two buttons to rotate the direction of the object clockwise and anti-clockwise as it constantly moves forward similar to asteroids.
In scene kit there are three different orientation properties for an SCNNode and I really don't know where to start. I understand how to execute everything except the rotation around the sphere.
You're looking for a parameterization of the surface of the sphere. You can find this online (but it can be tricky if you don't know the magic words to enter for your searches). Check out the entry on MathWorld.
The surface of the sphere is parameterized by two angle variables, call them s and t. Note that one variable will run from zero to 2 pi, and the other will run only from zero to pi. This is a gotcha that can be easy to miss. To convert these angles to rectangular (x, y, z) coordinates, you use the formula:
x = r cos(s) sin(t)
y = r sin(s) sin(t) // Yes it's sin(t) twice, that's not a typo.
z = r cos(t)
I find the following visualization helpful. A curve in a plane (the xy-plane, for example) sweeps out an angle from zero to pi, half a rotation and corresponds to the parameter s. If you set t equal to pi/2, so sin(t) = 1, then you can see how x and y turn into standard rectangular coordinates for a circular section. After the s parameter sweeps out half a circle, you can rotate that half circle all the way around from zero to 2 pi, to form a full sphere, and that full sweep corresponds to the parameter t.
If you represent your object's position by coordinates (s, t) then you can, for the most part, safely convert to rectangular coordinates using the formula above without worrying about the domain of either parameter; however if s or t grow without bound (say, because your object orbits continuously for a long time) it might be worth the small extra effort to normalize the parameters. I'm not sure how sin or cos behave for very large inputs.
Related
I'd like to get the pose (translation: x, y, z and rotation: Rx, Ry, Rz in World coordinate system) of the overhead camera. I got many object points and image points by moving the ChArUco calibration board with a robotic arm (like this https://www.youtube.com/watch?v=8q99dUPYCPs). Because of that, I already have exact positions of all the object points.
In order to feed many points to solvePnP, I set the first detected pattern (ChArUco board) as the first object and used it as the object coordinate system's origin. Then, I added the detected object points (from the second pattern to the last) to the first detected object points' coordinate system (the origin of the object frame is the origin of the first object).
After I got the transformation between the camera and the object's coordinate frame, I calculated the camera's pose based on that transformation.
The result looked pretty good at first, but when I measured the camera's absolute pose by using a ruler or a tape measure, I noticed that the extrinsic calibration result was around 15-20 millimeter off for z direction (the height of the camera), though almost correct for the others (x, y, Rx, Ry, Rz). The result was same even I changed the range of the object points by moving a robotic arm differently, it always ended up to have a few centimeters off for the height.
Has anyone experienced the similar problem before? I'd like to know anything I can try. What is the common mistake when the depth direction (z) is inaccurate?
I don't know how you measure the z but I believe that what you're measuring with the ruler is not z but the euclidean distance which is computed like so:
d=std::sqrt(x*x+y*y+z*z);
Let's take an example, if x=2; y=2; z=2;
then d will be d~3,5 so 3.5-2=1.5 is the difference you get between z and the ruler when you said around 15-20 millimeter off for z direction.
let's say I am placing a small object on a flat floor inside a room.
First step: Take a picture of the room floor from a known, static position in the world coordinate system.
Second step: Detect the bottom edge of the object in the image and map the pixel coordinate to the object position in the world coordinate system.
Third step: By using a measuring tape measure the real distance to the object.
I could move the small object, repeat this three steps for every pixel coordinate and create a lookup table (key: pixel coordinate; value: distance). This procedure is accurate enough for my use case. I know that it is problematic if there are multiple objects (an object could cover an other object).
My question: Is there an easier way to create this lookup table? Accidentally changing the camera angle by a few degrees destroys the hard work. ;)
Maybe it is possible to execute the three steps for a few specific pixel coordinates or positions in the world coordinate system and perform some "calibration" to calculate the distances with the computed parameters?
If the floor is flat, its equation is that of a plane, let
a.x + b.y + c.z = 1
in the camera coordinates (the origin is the optical center of the camera, XY forms the focal plane and Z the viewing direction).
Then a ray from the camera center to a point on the image at pixel coordinates (u, v) is given by
(u, v, f).t
where f is the focal length.
The ray hits the plane when
(a.u + b.v + c.f) t = 1,
i.e. at the point
(u, v, f) / (a.u + b.v + c.f)
Finally, the distance from the camera to the point is
p = √(u² + v² + f²) / (a.u + b.v + c.f)
This is the function that you need to tabulate. Assuming that f is known, you can determine the unknown coefficients a, b, c by taking three non-aligned points, measuring the image coordinates (u, v) and the distances, and solving a 3x3 system of linear equations.
From the last equation, you can then estimate the distance for any point of the image.
The focal distance can be measured (in pixels) by looking at a target of known size, at a known distance. By proportionality, the ratio of the distance over the size is f over the length in the image.
Most vision libraries (including opencv) have built in functions that will take a couple points from a camera reference frame and the related points from a Cartesian plane and generate your warp matrix (affine transformation) for you. (some are fancy enough to include non-linearity mappings with enough input points, but that brings you back to your time to calibrate issue)
A final note: most vision libraries use some type of grid to calibrate off of ie a checkerboard patter. If you wrote your calibration to work off of such a sheet, then you would only need to measure distances to 1 target object as the transformations would be calculated by the sheet and the target would just provide the world offsets.
I believe what you are after is called a Projective Transformation. The link below should guide you through exactly what you need.
Demonstration of calculating a projective transformation with proper math typesetting on the Math SE.
Although you can solve this by hand and write that into your code... I strongly recommend using a matrix math library or even writing your own matrix math functions prior to resorting to hand calculating the equations as you will have to solve them symbolically to turn it into code and that will be very expansive and prone to miscalculation.
Here are just a few tips that may help you with clarification (applying it to your problem):
-Your A matrix (source) is built from the 4 xy points in your camera image (pixel locations).
-Your B matrix (destination) is built from your measurements in in the real world.
-For fast recalibration, I suggest marking points on the ground to be able to quickly place the cube at the 4 locations (and subsequently get the altered pixel locations in the camera) without having to remeasure.
-You will only have to do steps 1-5 (once) during calibration, after that whenever you want to know the position of something just get the coordinates in your image and run them through step 6 and step 7.
-You will want your calibration points to be as far away from eachother as possible (within reason, as at extreme distances in a vanishing point situation, you start rapidly losing pixel density and therefore source image accuracy). Make sure that no 3 points are colinear (simply put, make your 4 points approximately square at almost the full span of your camera fov in the real world)
ps I apologize for not writing this out here, but they have fancy math editing and it looks way cleaner!
Final steps to applying this method to this situation:
In order to perform this calibration, you will have to set a global home position (likely easiest to do this arbitrarily on the floor and measure your camera position relative to that point). From this position, you will need to measure your object's distance from this position in both x and y coordinates on the floor. Although a more tightly packed calibration set will give you more error, the easiest solution for this may simply be to have a dimension-ed sheet(I am thinking piece of printer paper or a large board or something). The reason that this will be easier is that it will have built in axes (ie the two sides will be orthogonal and you will just use the four corners of the object and used canned distances in your calibration). EX: for a piece of paper your points would be (0,0), (0,8.5), (11,8.5), (11,0)
So using those points and the pixels you get will create your transform matrix, but that still just gives you a global x,y position on axes that may be hard to measure on (they may be skew depending on how you measured/ calibrated). So you will need to calculate your camera offset:
object in real world coords (from steps above): x1, y1
camera coords (Xc, Yc)
dist = sqrt( pow(x1-Xc,2) + pow(y1-Yc,2) )
If it is too cumbersome to try to measure the position of the camera from global origin by hand, you can instead measure the distance to 2 different points and feed those values into the above equation to calculate your camera offset, which you will then store and use anytime you want to get final distance.
As already mentioned in the previous answers you'll need a projective transformation or simply a homography. However, I'll consider it from a more practical view and will try to summarize it short and simple.
So, given the proper homography you can warp your picture of a plane such that it looks like you took it from above (like here). Even simpler you can transform a pixel coordinate of your image to world coordinates of the plane (the same is done during the warping for each pixel).
A homography is basically a 3x3 matrix and you transform a coordinate by multiplying it with the matrix. You may now think, wait 3x3 matrix and 2D coordinates: You'll need to use homogeneous coordinates.
However, most frameworks and libraries will do this handling for you. What you need to do is finding (at least) four points (x/y-coordinates) on your world plane/floor (preferably the corners of a rectangle, aligned with your desired world coordinate system), take a picture of them, measure the pixel coordinates and pass both to the "find-homography-function" of your desired computer vision or math library.
In OpenCV that would be findHomography, here an example (the method perspectiveTransform then performs the actual transformation).
In Matlab you can use something from here. Make sure you are using a projective transformation as transform type. The result is a projective tform, which can be used in combination with this method, in order to transform your points from one coordinate system to another.
In order to transform into the other direction you just have to invert your homography and use the result instead.
Is it possible to use SceneKit's unprojectPoint to convert a 2D point to 3D without having a depth value?
I only need to find the 3D location in the XZ plane. Y can be always 0 or any value since I'm not using it.
I'm trying to do this for iOS 8 Beta.
I had something similar with JavaScript and Three.js (WebGL) like this:
function getMouse3D(x, y) {
var pos = new THREE.Vector3(0, 0, 0);
var pMouse = new THREE.Vector3(
(x / renderer.domElement.width) * 2 - 1,
-(y / renderer.domElement.height) * 2 + 1,
1
);
//
projector.unprojectVector(pMouse, camera);
var cam = camera.position;
var m = pMouse.y / ( pMouse.y - cam.y );
pos.x = pMouse.x + ( cam.x - pMouse.x ) * m;
pos.z = pMouse.z + ( cam.z - pMouse.z ) * m;
return pos;
};
But I don't know how to translate the part with unprojectVector to SceneKit.
What I want to do is to be able to drag an object around in the XZ plane only. The vertical axis Y will be ignored.
Since the object would need to move along a plane, one solution would be to use hitTest method, but I don't think is very good in terms of performance to do it for every touch/drag event. Also, it wouldn't allow the object to move outside the plane either.
I've tried a solution based on the accepted answer here, but it didn't worked. Using one depth value for unprojectPoint, if dragging the object around in the +/-Z direction the object doesn't stay under the finger too long, but it moves away from it instead.
I need to have the dragged object stay under the finger no matter where is it moved in the XZ plane.
First, are you actually looking for a position in the xz-plane or the xy-plane? By default, the camera looks in the -z direction, so the x- and y-axes of the 3D Scene Kit coordinate system go in the same directions as they do in the 2D view coordinate system. (Well, y is flipped by default in UIKit, but it's still the vertical axis.) The xz-plane is then orthogonal to the plane of the screen.
Second, a depth value is a necessary part of converting from 2D to 3D. I'm not an expert on three.js, but from looking at their library documentation (which apparently can't be linked into), their unprojectVector still takes a Vector3. And that's what you're constructing for pMouse in your code above — a vector whose z- and y-coordinates come from the 2D mouse position, and whose z-coordinate is 1.
SceneKit's unprojectPoint works the same way — it takes a point whose z-coordinate refers to a depth in clip space, and maps that to a point in your scene's world space.
If your world space is oriented such that the only variation you care about is in the x- and y-axes, you may pass any z-value you want to unprojectPoint, and ignore the z-value in the vector you get back. Otherwise, pass -1 to map to the far clipping plane, 1 for the near clipping plane, or 0 for halfway in between — the plane whose z-coordinate (in camera space) is 0. If you're using the unprojected point to position a node in the scene, the best advice is to just try different z-values (between -1 and 1) until you get the behavior you want.
However, it's a good idea to be thinking about what you're using an unprojected vector for — if the next thing you'd be doing with it is testing for intersections with scene geometry, look at hitTest: instead.
i have three points in 2D and I want to draw a spline curve passing through them. How do I calculate the middle point (x1 and y1 as in quadTo)? i want to implement free curve like denon eq curve
For the first segment of the curve, you can probably use addQuadCurveToPoint, picking a control point with the same y value as the second point (and I picked an x value half way between the two end points):
For the second portion of the curve, you can't use quad curve, because you need two control points (or, you'd have to break it up into two quad curves, which is more hassle than its worth, IMHO). So use addCurveToPoint, using control point y values that are the same value as the y value of the point to which the control point refers (and, again, I picked x values half way between the x values of the two end points):
There are lots of permutations of this idea, but I hope this illustrates the concept. I'd suggest you start playing around with UIBezierPath and addCurveToPoint until you achieve the desired effect.
My goal is to move a shape in the virtual world in such a way so that it ends up where the mouse pointer is on the canvas.
What i have:
-mouse position (x,y) on a Canvas3D object
-Point3d object of where a pick ray starting from the Canvas3D viewport intersects with the first scene object. (point in 3D space of where i want to start the drag)
What i want:
-Some way to translate the Point3d's coordinates so that the initial point of intersection (the Point3d object) is always overlapping the the mouse position on the canvas (same as when i used the pick ray to determine what the user clicked on from the Canvas3D object).
Thanks!
It sounds as if you want to use the plane which is parallel to the background plane and contains the intersection point with the object. You can use this plane even when the mouse moves beyond the actual background as it's just a mathematical concept which stretches to infinity.
I'm not a Java programmer so I can't give you code but I am a mathematician so here's equation you need ;)
Let P denote the original intersection point and call the background plane unit normal n. This normal is also the normal of our plane of interest. Let R denote a point on the ray and l denote it's unit direction vector.
Then the equation of the plane is (x-P).n = 0 for a point x in the plane (the . denotes dot product of two vectors). The equation of a point on the ray is x = R + t*l where t is any real number. The ray therefore intersects the plane when
(t*l + R - P).n = 0
i.e. when
t = (P - R).n / ( l.n )
This gives you a t value to plug back into your ray equation to give the intersection point.