I am trying to implement an ARCamera.projectPoint function using projectionMatrix and viewMatrix from the camera.
Going to create something like this:
let position = simd_float3(x: 1, y: 2, z: 3)
let position4 = simd_float4(x: position.x, y: position.y, z: position.z, w: 1)
let projectionMatrix = frame.camera.projectionMatrix(for: .landscapeRight, viewportSize: frame.camera.imageResolution, zNear: 0.001, zFar: 1000.0)
let viewMatrix = frame.camera.viewMatrix(for: .landscapeRight)
let projection = position4 * projectionMatrix * viewMatrix
let arkitProjection = frame.camera.projectPoint(position, orientation: .landscapeRight, viewportSize: frame.camera.imageResolution)
assert(projection.x == Float(arkitProjection.x))
assert(projection.y == Float(arkitProjection.y))
But I don't know how to implement it correctly. Hope for you help.
You were very close! What you have in projection is actually a "clip space" position (also note I fixed the order of the matrix multiplication). You then need to convert that to a "normalized device coordinate" (which has coordinates from -1 to +1 in xyz dimensions) using "perspective divide". Then finally you need to linear map it to the position in the image based on the image size.
let position = simd_float3(x: 1, y: 2, z: 3)
let position4 = simd_float4(x: position.x, y: position.y, z: position.z, w: 1)
let projectionMatrix = frame.camera.projectionMatrix(for: .landscapeRight, viewportSize: frame.camera.imageResolution, zNear: 0.001, zFar: 1000.0)
let viewMatrix = frame.camera.viewMatrix(for: .landscapeRight)
let clipSpacePosition = projectionMatrix * viewMatrix * position4
let normalizedDeviceCoordinate = clipSpacePosition / clipSpacePosition.w
let projection = CGPoint(
x: (CGFloat(normalizedDeviceCoordinate.x) + 1) * (frame.camera.imageResolution.width / CGFloat(2)),
y: ((-CGFloat(normalizedDeviceCoordinate.y) + 1)) * (frame.camera.imageResolution.height / CGFloat(2))
)
let arkitProjection = frame.camera.projectPoint(position, orientation: .landscapeRight, viewportSize: frame.camera.imageResolution)
assert(abs(projection.x - arkitProjection.x) < 0.01)
assert(abs(projection.y - arkitProjection.y) < 0.01)
Further reading:
http://www.opengl-tutorial.org/beginners-tutorials/tutorial-3-matrices/#the-projection-matrix (generally useful info about clip space and projection)
https://developer.apple.com/documentation/arkit/tracking_and_visualizing_faces (search "perspective divide")
https://learnopengl.com/Getting-started/Coordinate-Systems
Related
I would like to project ARPlaneAnchor point to captured image coordinate system.
At this moment i am able to get plane point and project it to view coordinate system by method
func translateTransform(_ x: Float, _ y: Float, _ z: Float) -> float4x4 {
var tf = float4x4(diagonal: SIMD4<Float>(repeating: 1))
tf.columns.3 = SIMD4<Float>(x: x, y: y, z: z, w: 1)
return tf
}
extension ARPlaneAnchor {
func worldPoints() -> (SCNVector3, SCNVector3, SCNVector3, SCNVector3) {
let worldTransform = transform * translateTransform(center.x, 0, center.z)
let width = extent.x
let height = extent.z
let topLeft = worldTransform * translateTransform(-width / 2.0, 0, -height / 2.0)
let topRight = worldTransform * translateTransform(width / 2.0, 0, -height / 2.0)
let bottomLeft = worldTransform * translateTransform(-width / 2.0, 0, height / 2.0)
let bottomRight = worldTransform * translateTransform(width / 2.0, 0, height / 2.0)
let pointTopLeft = SCNVector3(
x: topLeft.columns.3.x,
y: topLeft.columns.3.y,
z: topLeft.columns.3.z
)
let pointTopRight = SCNVector3(
x: topRight.columns.3.x,
y: topRight.columns.3.y,
z: topRight.columns.3.z
)
let pointBottomLeft = SCNVector3(
x: bottomLeft.columns.3.x,
y: bottomLeft.columns.3.y,
z: bottomLeft.columns.3.z
)
let pointBottomRight = SCNVector3(
x: bottomRight.columns.3.x,
y: bottomRight.columns.3.y,
z: bottomRight.columns.3.z
)
return (
pointTopLeft,
pointTopRight,
pointBottomLeft,
pointBottomRight
)
}
And then by projectPoint convert 3D point to 2D like in example
guard let point = arPlane?.worldPoints().0 else { return }
let pp = self.sceneView.session.currentFrame?.camera.projectPoint(SIMD3.init(point), orientation: .landscapeRight, viewportSize: self.view.bounds.size)
But how to convert this point to capturedImage coordinate system?
I am using the below to modify UICollectionViewLayoutAttributes. Method 1 works but not Method 2, anyone can help explain the difference?
guard let newAttrs = attributes?.copy() as? UICollectionViewLayoutAttributes,
let imageViewInfo = imageViewInfo else { return nil }
let center = imageViewInfo.center
//Method 1
newAttrs.transform = CGAffineTransform(scaleX: scale, y: scale)
let x = (newAttrs.center.x-center.x) * scale + igCenter.x
let y = (newAttrs.center.y-center.y) * scale + igCenter.y
newAttrs.center = CGPoint(x: x, y: y)
//Method 2
newAttrs.transform = CGAffineTransform(translationX: -center.x, y: -center.y)
.scaledBy(x: scale, y: scale)
.translatedBy(x: igCenter.x, y: igCenter.y)
https://developer.apple.com/documentation/uikit/uiview/1622459-transform
to change position of the view/cell, modify 'center' instead.
I am creating a circle (Using CGContext) with angle of 90, 180 & 280. As we know that 90 degree angle shape like letter 'L', but I am getting opposite shape of 'L', then Angles.
Following this image, I don't want to get it like this, so what can I do ? please let me know.
Thanks
You can fix it with one of following solution.
1) Rotate your view (button) to -90 degree.
Or
2) Trickle values of angle while drawing as following:
func drawCircle(angles : Double) {
let angleToDraw = angles - 90
let startCircleAngle = CGFloat(269.9/180 * M_PI)
let endCircleAngle = CGFloat(270/180 * M_PI)
let center = CGPointMake(self.frame.size.width / 2.0, self.frame.size.height / 2.0);
let ctx = UIGraphicsGetCurrentContext()
CGContextBeginPath(ctx!)
CGContextSetLineWidth(ctx!, 5)
CGContextSetStrokeColorWithColor(ctx!,UIColor.redColor().CGColor)
let x:CGFloat = center.x
let y:CGFloat = center.y
let radius: CGFloat = 100.0
CGContextAddArc(ctx!, x, y, radius, startCircleAngle , endCircleAngle, 1)
CGContextAddArc(ctx!, x, y, 1, startCircleAngle, startCircleAngle, 0)
CGContextSaveGState(ctx!)
CGContextTranslateCTM(ctx!, x, y)
// rotation supplied in radians
CGContextRotateCTM(ctx!, CGFloat(M_PI*(angleToDraw)/180))
CGContextStrokePath(ctx!)
let path = CGPathCreateMutable()
CGPathMoveToPoint(path, nil, 0, 0)
CGPathAddLineToPoint(path, nil, radius, 1)
CGContextAddPath(ctx!, path)
CGContextSetLineWidth(ctx!, 4)
CGContextScaleCTM(ctx!, -1.0, 1.0)
CGContextSetStrokeColorWithColor(ctx!,UIColor.blackColor().CGColor)
CGContextStrokePath(ctx!)
}
I'm trying to create a game in which i have a object 1 rotatiing in a circle and another object appears and places itself ontop of object 1. currently the object just rotates around object 1 without stacking ontop of it. how do i get the object to stack itself on top and then follow it's orbit? here's my code now.
let player = SKSpriteNode(imageNamed: "Light")
override func didMoveToView(view: SKView) {
// 2
backgroundColor = SKColor.whiteColor()
// 3
player.position = CGPoint(x: size.width * 0.5, y: size.height * 0.5)
// 4
player.size = CGSize(width:70, height:60 )
addChild(player)
let dx = player.position.x - self.frame.width / 2
let dy = player.position.y - self.frame.height / 2
let rad = atan2(dy, dx)
circle = UIBezierPath(arcCenter: CGPoint(x: size.width / 2, y: size.height / 2), radius: 120, startAngle: rad, endAngle: rad + CGFloat(M_PI * 4), clockwise: true)
let follow = SKAction.followPath(circle.CGPath, asOffset: false, orientToPath: true, speed: 100)
player.runAction(SKAction.repeatActionForever(follow))
}
func addMonster() {
let monster = SKSpriteNode(imageNamed: "plate")
// Determine where to spawn the monster along the Y axis
let actualY = random(min: monster.size.height/1, max: size.height - monster.size.height/1)
// Position the monster slightly off-screen along the right edge,
// and along a random position along the Y axis as calculated above
monster.position = CGPoint(x: size.width * 0.5 + monster.size.width/2, y: actualY)
// Add the monster to the scene
addChild(monster)
// Determine speed of the monster
let actualDuration = random(min: CGFloat(2.0), max: CGFloat(3.0))
// Create the actions
let actionMove = SKAction.moveTo(CGPoint(x: -monster.size.width/2, y: actualY), duration: NSTimeInterval(actualDuration))
let follow = SKAction.followPath(circle.CGPath, asOffset: false, orientToPath:true, speed: 100)
monster.runAction(SKAction.repeatActionForever(follow))
}
Your verbal description appears to have something like the following in mind (you can copy and paste this into an iOS playground):
import UIKit
import SpriteKit
import XCPlayground
let scene = SKScene(size: CGSize(width: 400, height: 400))
let view = SKView(frame: CGRect(origin: .zero, size: scene.size))
view.presentScene(scene)
XCPlaygroundPage.currentPage.liveView = view
scene.backgroundColor = .darkGrayColor()
scene.anchorPoint = CGPoint(x: 0.5, y: 0.5)
let player = SKShapeNode(rectOfSize: CGSize(width: 70, height: 60), cornerRadius: 5)
let radius: CGFloat = 120
let circle = CGPathCreateWithEllipseInRect(CGRect(x: -radius, y: -radius, width: radius * 2, height: radius * 2), nil)
let followCircle = SKAction.followPath(circle, asOffset: false, orientToPath: true, speed: 100)
player.runAction(.repeatActionForever(followCircle))
let monsterSize = CGSize(width: 35, height: 30)
let monster = SKShapeNode(rectOfSize: monsterSize, cornerRadius: 4)
monster.fillColor = .redColor()
monster.runAction(.repeatActionForever(followCircle))
scene.addChild(player)
scene.addChild(monster)
Where you make use of a random(min:max:) function that is probably implemented along the lines of:
public func random(min min: CGFloat, max: CGFloat) -> CGFloat {
return min + (CGFloat(arc4random()) / CGFloat(UInt32.max)) * (max - min)
}
But you also have the code that seems to be saying:
let y = random(
min: scene.size.height / -2 + monsterSize.height / 2,
max: scene.size.height / 2 - monsterSize.height / 2
)
let xFrom = scene.size.width / 2
let xTo = scene.size.width / -2
monster.position = CGPoint(x: xFrom, y: y)
monster.runAction(
.moveToX(xTo, duration: NSTimeInterval(random(min: 2, max: 3)))
)
But this is unused. Instead the monster is set to follow the same circle path as the player. And so I am not sure really what exactly you are trying to achieve. (By the way, the simplest way of getting the monster "stack on top" of the player would be to make it the child node of the player...)
Hope some of this guesswork helps you in some way (to refine your question and code sample if nothing else).
I created a circular array objects, no problems.
When I rotate there appears to be a sinc or gaussian function at the center.
The camera is a z 60, radius of structure is 30.
Initial view, no artifacts
Rotated 90 deg up, no artifacts
Rotated 180 deg, artifact appears in center of object
Continued rotation, artifact is still there.
The code for the object is here
class func Build(scene: SCNScene) -> SCNNode {
let radius: Double = 30.0
let numberOfStrands: Int = 24
//Create the base chromosomes.
let baseChromosome = SCNBox(width: 4.0, height: 24, length: 1.0, chamferRadius: 0.5)
let baseChromosomes = DNA.buildCircleOfObjects(baseChromosome, numberOfItems: numberOfStrands, radius: radius)
baseChromosomes.position = SCNVector3(x: 0, y: 0.5, z: 0)
return baseChromosomes
}
class func buildCircleOfObjects(_geometry: SCNGeometry, numberOfItems: Int, radius: Double) -> SCNNode {
var x: Double = 0.0
var z: Double = radius
let theta: Double = (M_PI) / Double(numberOfItems / 2)
let incrementalY: Double = (M_PI) / Double(numberOfItems) * 2
let nodeCollection = SCNNode()
nodeCollection.position = SCNVector3(x: 0, y: 0.5, z: 0)
for index in 1...numberOfItems {
x = radius * sin(Double(index) * theta)
z = radius * cos(Double(index) * theta)
let node = SCNNode(geometry: _geometry)
node.position = SCNVector3(x: Float(x), y: 0, z:Float(z))
let rotation = Float(incrementalY) * Float(index)
node.rotation = SCNVector4(x: 0, y: 1, z: 0, w: rotation)
nodeCollection.addChildNode(node)
}
return nodeCollection
}
}
Using these settings
cameraNode.camera?.xFov = 60
cameraNode.camera?.yFov = 60
cameraNode.camera?.zFar = 1000
cameraNode.camera?.zNear = 0.01
the artifacts disappeared. I think this is a problem with zFar, it should have clipped the back surface uniformly not like a lens aberration.
Falling out of Viewing frustum may be the one to cause most problem. However there is another common cause by misunderstanding the behavior of SCNView's allowsCameraControl, that is when you Pinch to zoom in or zoom out (change the camera's fieldOfView), but the pointOfView's position reminds the same.
Your nodes still stand BEHIND the pointOfView, however small the camera's zNear is. Hence you see them got clipped. The code below may help you in this case, at least avoid the problem.
sceneView.pointOfView!.simdPosition
= float3(0, 0, -scene.rootNode.boundingSphere.radius)