How to scale SCNNodes to fit within a bounding box - ios

I'm downloading Collada DAE scenes and rendering them in SceneKit, but having trouble getting the downloaded node to "fit" within its parent node. I mainly care about scaling it's y-height to fit in the parent node.
Here's the code I'm using:
// Reset everything
node.position = SCNVector3(0, 0, 0)
node.pivot = SCNMatrix4Identity
node.scale = SCNVector3(1, 1, 1)
// Calculate absolute bounding box
let (min, max) = node.boundingBox
let size = max - min
// Get the biggest dimension of the node
guard let scaleRef = [size.x, size.y, size.z].max() else {
return
}
// Desired bounding box is of size SCNVector3(0.4, 0.4, 0.4)
let parentSize: Float = 0.4
let ratio = (parentSize/scaleRef)
node.scale = SCNVector3(node.scale.x * ratio, node.scale.y * ratio, node.scale.z * ratio)
//Correctly position the node at the bottom of its parent node by setting pivot
let (minNode, maxNode) = node.boundingBox
let dx = (maxNode.x - minNode.x) / 2
let dy = minNode.y
let dz = (maxNode.z - minNode.z) / 2
node.pivot = SCNMatrix4Translate(node.pivot, dx, dy, dz)
node.position.x = dx * ratio
node.position.y = dy * ratio
node.position.z = dz * ratio
This seems to work for most cases, but I'm ending up with a few that scale incorrectly.
For example, this object scales correctly (Poly Astornaut):
MIN: SCNVector3(x: -1.11969805, y: -0.735845089, z: -4.02169418)
MAX: SCNVector3(x: 1.11969805, y: 0.711179018, z: 0.0)
NEW SCALE: SCNVector3(x: 0.099460572, y: 0.099460572, z: 0.099460572)
This one does not (Basketball player):
MIN: SCNVector3(x: -74.8805618, y: 0.0459594727, z: -21.4300499)
MAX: SCNVector3(x: 74.8805618, y: 203.553589, z: 15.6760511)
NEW SCALE: SCNVector3(x: 0.00196552835, y: 0.00196552835, z: 0.00196552835)
Screenshots:
Correct scaling
Incorrect scaling

So what I ended up doing was a little different, as I wanted to scale my asset to exactly the reference size.
In my .scn file, going to the Node Inspector tab and looking at the bounding box of the asset I'm working with:
Know the size of your reference image. This is something you should be measuring in the real world, and entering it in your Assets.xcassets resource
This reference image size is passed to you in the func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) function. I access mine through the following code:
guard let imageAnchor = anchor as? ARImageAnchor else { return }
let referenceImage = imageAnchor.referenceImage
print(referenceImage.physicalSize.width)
print(referenceImage.physicalSize.height)
Wherever you want to scale, you'll simply use algebra to scale the bounding box size, to the desired size. So our scale is simply
let threeDimensionalAssetToRealReferenceImageScale = referenceImageWidth / threeDimensionalAssetBoundingBoxWidth
Call scale on your SCNNode
I wrote this up, as I was very confused with this situation, I hope it helps someone.

Related

How to extract outer lips from its feature point using vision framework in swift

I implemented addFaceLandmarksToImage function to crop the outer lips of the image. addFaceLandmarksToImage function first detects the face on the image using vision, convert the face bounding box size and origin to image size and origin. then I used face landmark outer lips to get the normalized point of the outer lips and draw the line on the outer lips of the image by connecting all the normalized points.
Then I implemented logic to crop the image. I first converted the normalized point of the outer lips into the image coordinate and used the find point method to get left, right, top and bottom-most points and extracted outer lips by cropping it and showed in processed image view. The issue with this function is output following is not as expected. Also if I use an image other then the one that is in the following link, it crops images other then outer lips. I could not figure out where have I gone wrong, is it on calculation of cropping rect or should I use another approach(using OpenCV and region of interest (ROI)) to extract the outer lips from the image?
video of an application
func addFaceLandmarksToImage(_ face: VNFaceObservation)
{
UIGraphicsBeginImageContextWithOptions(image.size, true, 0.0)
let context = UIGraphicsGetCurrentContext()
// draw the image
image.draw(in: CGRect(x: 0, y: 0, width: image.size.width, height: image.size.height))
context?.translateBy(x: 0, y: image.size.height)
context?.scaleBy(x: 1.0, y: -1.0)
// draw the face rect
let w = face.boundingBox.size.width * image.size.width
let h = face.boundingBox.size.height * image.size.height
let x = face.boundingBox.origin.x * image.size.width
let y = face.boundingBox.origin.y * image.size.height
let cropFace = self.image.cgImage?.cropping(to: CGRect(x: x, y: y, width: w, height: h))
let ii = UIImage(cgImage: cropFace!)
// outer lips
context?.saveGState()
context?.setStrokeColor(UIColor.yellow.cgColor)
if let landmark = face.landmarks?.outerLips {
var actualCordinates = [CGPoint]()
print(landmark.normalizedPoints)
for i in 0...landmark.pointCount - 1 {
// last point is 0,0
let point = landmark.normalizedPoints[i]
actualCordinates.append(CGPoint(x: x + CGFloat(point.x) * w, y: y + CGFloat(point.y) * h))
if i == 0 {
context?.move(to: CGPoint(x: x + CGFloat(point.x) * w, y: y + CGFloat(point.y) * h))
} else {
context?.addLine(to: CGPoint(x: x + CGFloat(point.x) * w, y: y + CGFloat(point.y) * h))
}
}
// Finding left,right,top,buttom point from actual coordinates points[CGPOINT]
let leftMostPoint = self.findPoint(points: actualCordinates, position: .leftMost)
let rightMostPoint = self.findPoint(points: actualCordinates, position: .rightMost)
let topMostPoint = self.findPoint(points: actualCordinates, position: .topMost)
let buttonMostPoint = self.findPoint(points: actualCordinates, position: .buttonMost)
print("actualCordinates:",actualCordinates,
"leftMostPoint:",leftMostPoint,
"rightMostPoint:",rightMostPoint,
"topMostPoint:",topMostPoint,
"buttonMostPoint:",buttonMostPoint)
let widthDistance = -(leftMostPoint.x - rightMostPoint.x)
let heightDistance = -(topMostPoint.y - buttonMostPoint.y)
//Cropping the image.
// self.image is actual image
let cgCroppedImage = self.image.cgImage?.cropping(to: CGRect(x: leftMostPoint.x,y: leftMostPoint.x - heightDistance,width:1000,height: topMostPoint.y + heightDistance + 500))
let jj = UIImage(cgImage: cgCroppedImage!)
self.processedImageView.image = jj
}
context?.closePath()
context?.setLineWidth(8.0)
context?.drawPath(using: .stroke)
context?.saveGState()
// get the final image
let finalImage = UIGraphicsGetImageFromCurrentImageContext()
// end drawing context
UIGraphicsEndImageContext()
imageView.image = finalImage
}}
normalized points of the outer lips of an image:-
[(0.397705078125, 0.3818359375),
(0.455322265625, 0.390625),
(0.5029296875, 0.38916015625),
(0.548828125, 0.40087890625),
(0.61279296875, 0.3984375),
(0.703125, 0.37890625),
(0.61474609375, 0.21875),
(0.52294921875, 0.1884765625),
(0.431640625, 0.20166015625),
(0.33203125, 0.34423828125)]
actual coordinate points of the outer lips of an image: -
[(3025.379819973372, 1344.4951847679913),
(3207.3986613331363, 1372.2607707381248),
(3357.7955853380263, 1367.633173076436),
(3502.7936454042792, 1404.6539543699473),
(3704.8654099646956, 1396.9412916004658),
(3990.2339324355125, 1335.2399894446135),
(3711.035540180281, 829.2893117666245),
(3421.039420047775, 733.6522934250534),
(3132.5858324691653, 775.3006723802537),
(2817.9091914743185, 1225.7201781179756)]
I also tried using the following method that uses CIDetector to get the mouth position and extracting the outer lips through cropping. The output wasn't good.
func focusonMouth() {
let ciimage = CIImage(cgImage: image.cgImage!)
let options = [CIDetectorAccuracy: CIDetectorAccuracyHigh]
let faceDetector = CIDetector(ofType: CIDetectorTypeFace, context: nil, options: options)!
let faces = faceDetector.features(in: ciimage)
if let face = faces.first as? CIFaceFeature {
if face.hasMouthPosition {
let crop = image.cgImage?.cropping(to: CGRect(x: face.mouthPosition.x, y: face.mouthPosition.y, width: face.bounds.width - face.mouthPosition.x , height: 200))
processedImageView.image = imageRotatedByDegrees(oldImage: UIImage(cgImage: crop!), deg: 90)
}
}
}
There are two problems:
input image on iOS can be rotated with orientation property marking how it is rotated. Vision Framework will do the job, but coordinates will be rotated. The simplest solution is to supply image that is up-oriented (normal rotation).
position and size of located landmarks are relative to position and size of the detected face landmarks. So found position and size should be scaled by size and offset of the face found, not whole image.

Repeat Texture on X axis over 3dObject in ARkit SCNODE - SWIFT

I wanted to move the texture over the SCNode in my current scene.
I am currently animating the node but i want the texture to run over the object instead of animating it.
let moveup = SCNAction.moveBy(x: 0.01, y: 0, z: 0, duration: 1)
moveup.timingMode = .easeInEaseOut
moveup.speed = CGFloat(20.5)
let moveDown = SCNAction.moveBy(x: -0.01, y: 0, z: 0, duration: 1)
moveDown.timingMode = .easeInEaseOut;
moveDown.speed = CGFloat(20.5)
let moveupSequence = SCNAction.sequence([moveup, moveDown])
print(moveupSequence.duration, moveupSequence.speed)
WaterNode?.runAction(moveupSequence)
This is my code for annimating the object. I need to know how can i access the texture and move them to X axis.
There is contentsTransform property of material
let action = SCNAction.customAction(duration: 1) { (node, elapsedTime) in
let c = node.geometry?.materials.first?.diffuse.contentsTransform
node.geometry?.materials.first?.diffuse.contentsTransform = SCNMatrix4MakeTranslation((c?.m41)! + 0.01, (c?.m42)!, (c?.m43)!)
}

Scene Kit node not rotating

I am trying to rotate by scene kit node, however it is not rotating.
I want it to rotate around the y axis. It is a sphere.
let node = SCNNode()
node.geometry = SCNSphere(radius: 1)
node.geometry?.firstMaterial?.diffuse.contents = UIImage(named: "diffuse.png")
node.geometry?.firstMaterial?.specular.contents = UIImage(named: "specular.png")
node.geometry?.firstMaterial?.emission.contents = UIImage(named: "emission.png")
scene.rootNode.addChildNode(node)
let action = SCNAction.rotate(by:360 * CGFloat((Double.pi)/180.0), around: SCNVector3(x: 0, y: 1, z: 0), duration: 8)
let repeatAction = SCNAction.repeatForever(action)
node.runAction(repeatAction)
Are you certain it isn't rotating? I'm not sure what your diffuse, specular and emission images are but maybe because it's a sphere it just doesn't really look like it is rotating because it looks the same from all sides.
Running the same action on a cube in viewDidAppear is working for me, so it could just be that you can't really notice the sphere rotating. I used this code:
let node = SCNNode()
node.geometry = SCNBox (width: 0.5, height: 0.5, length: 0.5, chamferRadius: 0.1)
node.position = SCNVector3Make(0, 0, -1)
sceneView.scene.rootNode.addChildNode(node)
let action = SCNAction.rotate(by:360 * CGFloat((Double.pi)/180.0), around: SCNVector3(x: 0, y: 1, z: 0), duration: 8)
let repeatAction = SCNAction.repeatForever(action)
node.runAction(repeatAction)
The cube is rotating fine with the way that you rotated the SCNNode.
Hope this helps

Strange behavior with SceneKit Node

I created a circular array objects, no problems.
When I rotate there appears to be a sinc or gaussian function at the center.
The camera is a z 60, radius of structure is 30.
Initial view, no artifacts
Rotated 90 deg up, no artifacts
Rotated 180 deg, artifact appears in center of object
Continued rotation, artifact is still there.
The code for the object is here
class func Build(scene: SCNScene) -> SCNNode {
let radius: Double = 30.0
let numberOfStrands: Int = 24
//Create the base chromosomes.
let baseChromosome = SCNBox(width: 4.0, height: 24, length: 1.0, chamferRadius: 0.5)
let baseChromosomes = DNA.buildCircleOfObjects(baseChromosome, numberOfItems: numberOfStrands, radius: radius)
baseChromosomes.position = SCNVector3(x: 0, y: 0.5, z: 0)
return baseChromosomes
}
class func buildCircleOfObjects(_geometry: SCNGeometry, numberOfItems: Int, radius: Double) -> SCNNode {
var x: Double = 0.0
var z: Double = radius
let theta: Double = (M_PI) / Double(numberOfItems / 2)
let incrementalY: Double = (M_PI) / Double(numberOfItems) * 2
let nodeCollection = SCNNode()
nodeCollection.position = SCNVector3(x: 0, y: 0.5, z: 0)
for index in 1...numberOfItems {
x = radius * sin(Double(index) * theta)
z = radius * cos(Double(index) * theta)
let node = SCNNode(geometry: _geometry)
node.position = SCNVector3(x: Float(x), y: 0, z:Float(z))
let rotation = Float(incrementalY) * Float(index)
node.rotation = SCNVector4(x: 0, y: 1, z: 0, w: rotation)
nodeCollection.addChildNode(node)
}
return nodeCollection
}
}
Using these settings
cameraNode.camera?.xFov = 60
cameraNode.camera?.yFov = 60
cameraNode.camera?.zFar = 1000
cameraNode.camera?.zNear = 0.01
the artifacts disappeared. I think this is a problem with zFar, it should have clipped the back surface uniformly not like a lens aberration.
Falling out of Viewing frustum may be the one to cause most problem. However there is another common cause by misunderstanding the behavior of SCNView's allowsCameraControl, that is when you Pinch to zoom in or zoom out (change the camera's fieldOfView), but the pointOfView's position reminds the same.
Your nodes still stand BEHIND the pointOfView, however small the camera's zNear is. Hence you see them got clipped. The code below may help you in this case, at least avoid the problem.
sceneView.pointOfView!.simdPosition
= float3(0, 0, -scene.rootNode.boundingSphere.radius)

SCNShape with bezier path

I am tying to draw a 3d line in Scenekit iOS, i am trying below code to draw line,
func path() -> UIBezierPath
{
var bPath = UIBezierPath()
bPath.moveToPoint(CGPointMake(0, 0))
bPath.addLineToPoint(CGPointMake(10, 10))
bPath.addLineToPoint(CGPointMake(5, 5))
bPath.closePath()
return bPath
}
and the below code is to load the line in SCNNode,
let sap = SCNShape()
sap.path = path()
let carbonNode = SCNNode(geometry: sap)
carbonNode.position = SCNVector3Make(0, 0, 15)
scene.rootNode.addChildNode(carbonNode)
But, i am getting blank screen.
try changing your geometry's materials. You might have a white object on a white background.
edit:
it also turns out that the vertices of your triangle are colinear. You thus end up with a degenerate triangle.
Your path has no surface area since you start at (0,0), draw a line to (10, 10) and then a second line to (5,5) which is already on the line between (0,0) and (10, 10). Then you close the path by drawing another line to (0,0).
Changing either of the three points creates a path with a surface area.
Also, the z-axis points out of the screen. This means that depending on where your camera is positioned, you may be placing the carbonNode behind the camera. If instead you meant to position it further into the scene you should likely use a negative z value.
Here's some code I wrote to address the problem.
// Material colors
let cyanMaterial = SCNMaterial()
cyanMaterial.diffuse.contents = UIColor.cyanColor()
let anOrangeMaterial = SCNMaterial()
anOrangeMaterial.diffuse.contents = UIColor.orangeColor()
let aPurpleMaterial = SCNMaterial()
aPurpleMaterial.diffuse.contents = UIColor.purpleColor()
// A bezier path
let bezierPath = UIBezierPath()
bezierPath.moveToPoint(CGPointMake(-0.25, 0))
bezierPath.addLineToPoint(CGPointMake(0, -0.25))
bezierPath.addLineToPoint(CGPointMake(0.25, 0))
bezierPath.addLineToPoint(CGPointMake(0, 0.25))
bezierPath.closePath()
// Add shape
let shape = SCNShape(path: bezierPath, extrusionDepth: 0.75)
shape.materials = [cyanMaterial, anOrangeMaterial, aPurpleMaterial]
let shapeNode = SCNNode(geometry: shape)
shapeNode.position = SCNVector3(x: 0.2, y: 0.75, z: 0.1);
self.rootNode.addChildNode(shapeNode)
shapeNode.rotation = SCNVector4(x: -1.0, y: -1.0, z: 0.0, w: 0.0)
Keep in mind that the numbers, the Scene Kit scale, are in meters. So set the numbers similarly to those that work for the other shapes you create.

Resources