Questions about how scene kit translations work, and accelerometer - ios

I have done a tiny bit of 3d graphics in the past. When you move or rotate a Scene Kit sprite does it automatically update its translation matrix, or do you have to make it yourself?
Are "position" and "eulerAngles" both properties that are... absolute.
For example if I am in sprite kit and set the translation to (1, 0) it will be at that point relative to the origin.
And if I set the z rotation to 90 it will be rotated 90 degrees.
And if I incrament the translation (with +=) x it will start going in a line.
And same for zRotation if incremented it will rotate. In scene kit if I do similar things to the translation and euler angle values will they do the same thing?
Also what exactly does the accelerometer think its measuring, it is like the amount of motion in a certain period? So basically is it the delta between the two simultaneous points that the device was in.
Yes, this question is definitely broad, however they are much better placed here, then scattered in three tiny posts.

Doe, let me see if I can help
Translation matrix? It has a TRANSFORM matrix that includes translation, scale and rotation, and yes, it is automatically updated when you change one of these 3, and vice-versa.
If I understood well, yes, just like in SpriteKit. They are related to their parent coordinates. The position (1,0,0) would mean the Node (its center, unless you change its pivot (anchorPoint in spriteKit)) will be at distance 1 along the X axis of its parent from its parent origin).
The same works for the rotation, if a NodeA has 30 degrees rotation at axis X and you add a NodeB with 20 degrees rotation at X in NodeA, you would have the NodeA having visually a 50 degrees rotation at X.
Accelerometer measures the acceleration forces given to the device in a specific moment, in the three axis of the device. Its unit is not [m^2/s] but [Gravity/s] (would be approximately [10m^2/s]). An important detail is that this measure includes the gravity acceleration as well.
So, if you try to measure the acceleration with the device standing ortogonal to the ground, you would expect (0, 0, -1) (or 0,0,1, if upside down).
Lying down the device on the ground it would be (0, 1or-1, 0) (depending if the screen is facing the ground or the ceiling)
For for every tick (of update rate of the accelerometer) it calculates what was the acceleration imposed to the device at that moment. That's not the delta itself, but it can be easily calculated if you store the values.

Related

finding the depth in arkit with SCNVector3Make

the goal of the project is to create a drawing app. i want it so that when i touch the screen and move my finger it will follow the finger and leave a cyan color paint. i did created it BUT there is one problem. the paint DEPTH is always randomly placed.
here is the code, just need to connect the sceneView with the storyboard.
https://github.com/javaplanet17/test/blob/master/drawingar
my question is how do i make the program so that the depth will always be consistent, by consistent i mean there is always distance between the paint and the camera.
if you run the code above you will see that i have printed out all the SCNMatrix4, but i none of them is the DEPTH.
i have tried to change hitTransform.m43 but it only messes up the x and y.
If you want to get a point some consistent distance in front of the camera, you don’t want a hit test. A hit test finds the real world surface in front of the camera — unless your camera is pointed at a wall that’s perfectly parallel to the device screen, you’re always going to get a range of different distances.
If you want a point some distance in front of the camera, you need to get the camera’s position/orientation and apply a translation (your preferred distance) to that. Then to place SceneKit content there, use the resulting matrix to set the transform of a SceneKit node.
The easiest way to do this is to stick to SIMD vector/matrix types throughout rather than converting between those and SCN types. SceneKit adds a bunch of new accessors in iOS 11 so you can use SIMD types directly.
There’s at least a couple of ways to go about this, depending on what result you want.
Option 1
// set up z translation for 20 cm in front of whatever
// last column of a 4x4 transform matrix is translation vector
var translation = matrix_identity_float4x4
translation.columns.3.z = -0.2
// get camera transform the ARKit way
let cameraTransform = view.session.currentFrame.camera.transform
// if we wanted, we could go the SceneKit way instead; result is the same
// let cameraTransform = view.pointOfView.simdTransform
// set node transform by multiplying matrices
node.simdTransform = cameraTransform * translation
This option, using a whole transform matrix, not only puts the node a consistent distance in front of your camera, it also orients it to point the same direction as your camera.
Option 2
// distance vector for 20 cm in front of whatever
let translation = float3(x: 0, y: 0, z: -0.2)
// treat distance vector as in camera space, convert to world space
let worldTranslation = view.pointOfView.simdConvertPosition(translation, to: nil)
// set node position (not whole transform)
node.simdPosition = worldTranslation
This option sets only the position of the node, leaving its orientation unchanged. For example, if you place a bunch of cubes this way while moving the camera, they’ll all be lined up facing the same direction, whereas with option 1 they’d all be in different directions.
Going beyond
Both of the options above are based only on the 3D transform of the camera — they don’t take the position of a 2D touch on the screen into account.
If you want to do that, too, you’ve got more work cut out for you — essentially what you’re doing is hit testing touches not against the world, but against a virtual plane that’s always parallel to the camera and a certain distance away. That plane is a cross section of the camera projection frustum, so its size depends on what fixed distance from the camera you place it at. A point on the screen projects to a point on that virtual plane, with its position on the plane scaling proportional to the distance from the camera (like in the below sketch):
So, to map touches onto that virtual plane, there are a couple of approaches to consider. (Not giving code for these because it’s not code I can write without testing, and I’m in an Xcode-free environment right now.)
Make an invisible SCNPlane that’s a child of the view’s pointOfView node, parallel to the local xy-plane and some fixed z distance in front. Use SceneKit hitTest (not ARKit hit test!) to map touches to that plane, and use the worldCoordinates of the hit test result to position the SceneKit nodes you drop into your scene.
Use Option 1 or Option 2 above to find a point some fixed distance in front of the camera (or a whole translation matrix oriented to match the camera, translated some distance in front). Use SceneKit’s projectPoint method to find the normalized depth value Z for that point, then call unprojectPoint with your 2D touch location and that same Z value to get the 3D position of the touch location with your camera distance. (For extra code/pointers, see my similar technique in this answer.)

How to I get more reliable Y position tracking for the Google Tango in Unity?

We have a unity scene that uses arealearning which has been extremely reliable and consistent about XZ position. However we are noticing that sometimes the tango delta camera’s Y position will "jump up" very high in the scene. When we force the tango to relocalize (by covering the sensors for a few seconds), the Y position remains very off. At other times, the Y position varies a 0.5 - 1.5 unity units when we first start up our Unity app on the tango and are holding it in the exact same position in the exact same room using the same ADF file. Is there a way to get a more reliable Y position tracking and/or correct for these jumps?
(All the XYZ coordinate is in the Unity convention in this context, x is right, y is up, z is forward)
Y position should work same as XZ coordinates, it relocalized to the height based on the ADF origin.
But note that, the ADF's origin is where you started learning(recording) ADF. Let's say you started the learning session by holding the device normally, then the ADF's origin might be a little bit higher than ground level. When you construct a virtual world to relocalize, you should take the height difference into consideration.
Another thing to check is that making sure there's no offset or original location set for DeltaPoseController prefab. DeltaPoseController will take the initial starting transformation as a offset, and add up pose on it. For example, if my DeltaPoseController's starting position is at (0,1,0), and my pose from device is (0,1,1), then the actually position for DeltaPoseController in Unity would be (0,2,1). This applies to both translation and rotation.
Another advanced (and preferred) way of defining ground level is to use the depth sensor to find out the ground height. In the Unity Augmented Reality example, it showed how to detect the plane and place a marker on it. You can easily apply the similar method to the ground plane, do a PlaneFinding and place the ground at the right height in Unity world space.

iPhone augmented reality Euler angles rotation – roll issue

I’m working on an iOS augmented reality application.
It is location-based, not marker-based.
I use the GPS, compass and accelerometers to get latitude, longitude, altitude and the 3 euler angles: yaw, pitch and roll. I know using NSLog() that those 6 variables contain valid data.
My application shows some 3d objects over the camera view.
It works fine as long as I use everything but the roll angle.
If I add that third angle, the rotation applied to my opengl world is not good. I do it that way in the main OpenGL draw method
glRotatef(pitch, 1, 0, 0);
glRotatef(yaw, 0, 1, 0);
//glRotatef(roll, 0, 0, 1);
I think there is something wrong with this approach but am certainly not a specialist. Maybe I should create some sort of unique rotation matrix rather than 3 different ones?
Maybe that’s not possible easily? After all most desktop video games, FPS and the like, just let the user change the yaw and the pitch using the mouse, so only 2 angles, not 3. But unlike the mouse, which is a 2d device, a phone used for augmented reality can move in any angles.
But then again, all AR tutorials I have seen online couldn’t handle ‘roll’ properly. ‘Rolling’ your phone would either completely mess AR stuff up or do nothing at all, using some roll-compensation strategies.
So my question is, assuming I have my 3 Euler angles using the phone sensors, how should I apply them to my 3d opengl view?
I think you're likely talking about gimbal lock.
The essence of the problem is that if you rotate with Eulers then there's always a sequence to it. For example, you rotate around x, then around y, then z. But then one axis can always becomes ambiguous because a preceding can move it onto a different axis.
Suppose the rotation were 0 degrees around x, 90 degrees around y, then 20 degrees around z. So you do the x rotation and nothing has changed. You do the y rotation and everything moves 90 degrees. But now you've moved the z axis onto where the x axis was previously. So the z rotation will appear to be around x.
No matter what most people's instincts tell them, there's no way to avoid the problem. The kneejerk reaction is that you'll always rotate around the global axes rather than the local one. That doesn't resolve the problem, it just reverses the order. The z rotation could then the y rotation — which has already occurred — into an x rotation.
You're right that you should aim to create a unique description of rotation separated from measuring angles.
For augmented reality it's actually not all that difficult.
The accelerometer tells you which way down is. The compass tells you which way north is. The two may not be orthogonal though — the compass reading should vary from being exactly at a right angle to the floor on the equator to being exactly parallel to the accelerometer at the poles.
So:
just accept the accelerometer vector as down;
get the cross product of down and the compass vector to get your side vector — it should point along a line of longitude;
then get the cross product of your side vector and your down vector to get a north vector that is suitably perpendicular.
You could equally use the dot product to remove that portion of the compass vector that is in the direction of gravity and cross product from there.
You'll want to normalise everything.
That gives you three basis vectors, so just put them directly into a matrix. No further work required.

Blackberry Device Movement Angle Difference

I am working on a Blackberry application in which I need to retrieve the Angle difference when the device moves. It means the difference of angle between when the movement starts and ends. It must be 25 degrees to call some function.
In simple words, call a function when the device moves by 25 degrees.
Please read AccelerometerSensor docs, it is available in API 4.7.0 and higher. All data that you can retrieve is described in class AccelerometerData, it is orientation and acceleration (gravity data).
It is described more in details how to get angle from gravity sensor data in JavaME docs, "Mobile Sensor API" section:
If the phone was placed flat, the accelerometer would tell us that the acceleration along the z-axis (up and down) is about 1000 (this value represents 1G). The accelerations along the X and Y axises (sideways) would be about 0 since the phone is sitting still and gravity only works downwards. Flipping the phone over with the screen facing down, the accelerometer would give us the value of -1000 on the Z-axis. Standing on its side, would give us a value of 1000 or -1000 along either the X- or the Y-axis, depending on which side you put it. Putting the phone in a 45 degree angle along the X-axis would give us a value of ±707 on the Z-axis and ±707 on the Y-axis, since gravity cannot affect either axis with its full force (You can easily calculate what the value should be for a certain angle for each axis using the sine and cosine functions). Using the values from the X and Y-axis from the accelerometer, we can determine the position of the phone at any time, and then use that value to move our ship to avoid the incoming asteroids.
So, having accelerometer data for all 3 axes we may figure it out what is horizontal angle of a device.

What is this rotation behavior in XNA?

I am just starting out in XNA and have a question about rotation. When you multiply a vector by a rotation matrix in XNA, it goes counter-clockwise. This I understand.
However, let me give you an example of what I don't get. Let's say I load a random art asset into the pipeline. I then create some variable to increment every frame by 2 radians when the update method runs(testRot += 0.034906585f). The main thing of my confusion is, the asset rotates clockwise in this screen space. This confuses me as a rotation matrix will rotate a vector counter-clockwise.
One other thing, when I specify where my position vector is, as well as my origin, I understand that I am rotating about the origin. Am I to assume that there are perpendicular axis passing through this asset's origin as well? If so, where does rotation start from? In other words, am I starting rotation from the top of the Y-axis or the x-axis?
The XNA SpriteBatch works in Client Space. Where "up" is Y-, not Y+ (as in Cartesian space, projection space, and what most people usually select for their world space). This makes the rotation appear as clockwise (not counter-clockwise as it would in Cartesian space). The actual coordinates the rotation is producing are the same.
Rotations are relative, so they don't really "start" from any specified position.
If you are using maths functions like sin or cos or atan2, then absolute angles always start from the X+ axis as zero radians, and the positive rotation direction rotates towards Y+.
The order of operations of SpriteBatch looks something like this:
Sprite starts as a quad with the top-left corner at (0,0), its size being the same as its texture size (or SourceRectangle).
Translate the sprite back by its origin (thus placing its origin at (0,0)).
Scale the sprite
Rotate the sprite
Translate the sprite by its position
Apply the matrix from SpriteBatch.Begin
This places the sprite in Client Space.
Finally a matrix is applied to each batch to transform that Client Space into the Projection Space used by the GPU. (Projection space is from (-1,-1) at the bottom left of the viewport, to (1,1) in the top right.)
Since you are new to XNA, allow me to introduce a library that will greatly help you out while you learn. It is called XNA Debug Terminal and is an open source project that allows you to run arbitrary code during runtime. So you can see if your variables have the value you expect. All this happens in a terminal display on top of your game and without pausing your game. It can be downloaded at http://www.protohacks.net/xna_debug_terminal
It is free and very easy to setup so you really have nothing to lose.

Resources