Getting object at location on screen in Metal - ios

Question

I'm using Metal and I want the user to be able to tap certain objects onscreen. I'm using a tap gesture recognizer, and so I have a CGPoint, but I don't know how to find the object at that point. Is there either a way to get the onscreen coordinates for metal, or a way to transform the CGPoint into the metal coordinate space?

Answer 1

I ended up doing this with a raytracing-type technique. I used a bounding sphere for each object and calculated whether or not the ray intersected it and how far from the ray's origin the intersection was, with the ray being from the camera through the onscreen point of clicking. The code is as follows:
var location = gestureRecognizer.locationInView(self.view)
var proj_matrix = projectionMatrix.copy()
var view_matrix = worldMatrix.copy()
var x = (2.0 * location.x) / self.view.bounds.size.width - 1.0;
var y = 1.0 - (2.0 * location.y) / self.view.bounds.size.height;
var newLocation = simpleVertex(x: Float(x), y: Float(y), z: Float(-1.0))
var projSpaceLoc = GLKMatrix4MultiplyVector4(GLKMatrix4Invert(proj_matrix.matrix(), nil), GLKVector4Make(newLocation.x, newLocation.y, newLocation.z, 1.0))
projSpaceLoc = GLKVector4Make(projSpaceLoc.x, projSpaceLoc.y, -1.0, 0.0)
var viewSpaceLoc = GLKMatrix4MultiplyVector4(GLKMatrix4Invert(view_matrix.matrix(), nil), projSpaceLoc)
GLKVector4Normalize(viewSpaceLoc)
//tappable is an array of nodes that can be tapped
for node in tappable {
var r: Float = node.r //this is the radius of the bounding sphere for that node
var c: GLKVector4 = node.origin //this is the origin of the bounding sphere
var little_c = GLKVector4DotProduct(c, c) - powf(r, 2)
var b = GLKVector4DotProduct(viewSpaceLoc, c)
var discriminant = powf(b, 2) - little_c
//If the discriminant is positive, their are two points of intersection, if it is 0, one point, and if it is negative, no points.
if (discriminant >= 0) {
var t_1 = (Float(-1.0)*b) + sqrtf(discriminant)
var t_2 = (Float(-1.0)*b) - sqrtf(discriminant)
if (intersect != nil) {
if (min(t_1, t_2) < intersect.intersect_point) {
intersect = (node, min(t_1, t_2))
}
} else {
intersect = (node, min(t_1, t_2))
}
}
}