Here below my code and camera image to load 2d UIView into sceneView. I already tried to load into material content but getting node many times blank.
let annotationNode = SCNNode()
let planeGeoMetry:SCNPlane = SCNPlane()
planeGeoMetry.firstMaterial?.fillMode = .fill
planeGeoMetry.firstMaterial?.diffuse.contents = view
annotationNode.name = name
annotationNode.geometry = planeGeoMetry
self.sceneView.scene.rootNode.addChildNode(annotationNode)
From Apples docs on SCNMaterialProperty.contents:
SceneKit cannot use a layer that is already being displayed elsewhere (for example, the backing layer of a
UIView
object)
Personally the easiest way of displaying information on a plane is using a SpriteKit Scene.
Related
I would like to display a mesh with iOS ARKit with SceneKit and not RealityKit.
I am able to add a material to the mesh, like so which displays the mesh just fine
let edgesMaterial = SCNMaterial()
edgesMaterial.fillMode = .lines
edgesMaterial.lightingModel = .constant
edgesMaterial.transparency = 1.0
edgesMaterial.diffuse.contents = UIColor.red
scnGeom.materials = [edgesMaterial]
However, this adds overlapping lines of the wireframes in the scene. With RealityKit the fix is rather simple by just adding
arView.environment.sceneUnderstanding.options.insert(.occlusion)
How can I get the same effect with a scenekit scene? Basically I want to add a occlusion material with lines. Any pointers?
UPDATE:
I can get something close but not quite the same visualization by using this:
sceneView.debugOptions.insert([.showWireframe])
let occMaterial = SCNMaterial()
occMaterial.colorBufferWriteMask = SCNColorMask(rawValue: 0)
scnGeom.materials = [occMaterial]
The lines appear white and also any object I place in the scene will render the wireframe automatically now with this
My ultimate goal is to have an SCNNode representing an image floating in space. This is more or less easily accomplished with the current code I have below, but the problem is that the back side of the image isn't rendered and is thus transparent from the back. I want to be able to display a different image on the back so that there is something to see from both sides. the isDoubleSided property doesn't work here because it simply mimics what's on the front. Any Ideas? I looked into the idea of creating my own geometry from Sources and Elements but it seemed very complex for what should be really simple.
My current code:
private func createNode() -> SCNNode{
let scaleFactor = image.size.width/0.2
let width = image.size.width/scaleFactor
let height = image.size.height/scaleFactor
let geometry = SCNPlane(width: width, height: height)
let material = SCNMaterial()
material.diffuse.contents = image
geometry.materials = [material]
return SCNNode(geometry: geometry)
}
Thanks!
Since you want different images, you need to use different materials. SceneKit allows specifying material per geometry element. SCNPlane has only one element, that's why isDoubleSided just mirrors image on the back side. You have two options here:
Create two SCNPlane geometries, orient them back to back and assign different images to each geometry.firstMaterial.diffuse.contents
Create custom SCNGeometry from SCNGeometrySource (4 vertices of plane) and two SCNGeometryElements (one for each side: 2 triangles, 6 indices), and assign array of two materials (different images) to geometry.
The first option is easier, but looks more like a workaround.
I just started looking at ARKitExample from apple and I am still studying. I need to do like interactive guide. For example, we can detect something (like QRCode), in that area, can I show with 1 label ?
Is it possible to add custom view (like may be UIVIew, UIlabel) to surface?
Edit
I saw some example to add line. I will need to find how to add additional view or image.
let mat = SCNMatrix4FromMat4(currentFrame.camera.transform)
let dir = SCNVector3(-1 * mat.m31, -1 * mat.m32, -1 * mat.m33)
let currentPosition = pointOfView.position + (dir * 0.1)
if button!.isHighlighted {
if let previousPoint = previousPoint {
let line = lineFrom(vector: previousPoint, toVector: currentPosition)
let lineNode = SCNNode(geometry: line)
lineNode.geometry?.firstMaterial?.diffuse.contents = lineColor
sceneView.scene.rootNode.addChildNode(lineNode)
}
}
I think this code should be able to add custom image. But I need to find the whole sample.
func updateRenderer（_ frame： ARFrame）{
drawCameraImage（withPixelBuffer：frame.capturedImage)
let viewMatrix = simd_inverse(frame.came.transform)
let prijectionMatrix = frame.camera.prijectionMatrix
updateCamera(viewMatrix, projectionMatrix)
updateLighting(frame.lightEstimate?.ambientIntensity)
drawGeometry(forAnchors: frame.anchors)
}
ARKit isn't a rendering engine — it doesn't display any content for you. ARKit provides information about real-world spaces for use by rendering engines such as SceneKit, Unity, and any custom engine you build (with Metal, etc), so that they can display content that appears to inhabit real-world space. Thus, any "how do I show" question for ARKit is actually a question for whichever rendering engine you use with ARKit.
SceneKit is the easy out-of-the-box, no-additional-software-required way to display 3D content with ARKit, so I presume you're asking about that.
SceneKit can't render a UIView as part of a 3D scene. But it can render planes, cubes, or other shapes, and texture-map 2D content onto them. If you want to draw a text label on a plane detected by ARKit, that's the direction to investigate — follow the example's, um, example to create SCNPlane objects corresponding to detected ARPlaneAnchors, get yourself an image of some text, and set that image as the plane geometry's diffuse contents.
Yes you can add custom view in ARKit Scene.
Just make image of your view and add it wherever you want.
You can use following code to get image for UIView
func image(with view: UIView) -> UIImage? {
UIGraphicsBeginImageContextWithOptions(view.bounds.size, view.isOpaque, 0.0)
defer { UIGraphicsEndImageContext() }
if let context = UIGraphicsGetCurrentContext() {
view.layer.render(in: context)
let image = UIGraphicsGetImageFromCurrentImageContext()
return image
}
return nil
}
I am trying to create a 3d model that is moving above a 2d background. I read somewhere else that in order to do that I need to create a SCNNode() with SCNPlane() geometry and use my backgroundimage as the material of the SCNPlane(). However I have no clue how to add materials to a geometry structure, can you help me?
So far this is my code:
let background = SCNNode()
background.geometry = SCNPlane()
First add your texture image to your assets catalogue, say "Background.jpg", to Assets.xcassets
Then
let background = SCNNode()
background.geometry = SCNPlane.init(width: 100, height: 100) // better set its size
background.geometry?.firstMaterial?.diffuse.contents = "Background.jpg"
scene.rootNode.addChildNode(background)
It seems the only way to make circles in a SKScene is to either upload a circular image or use SKShapeNodes.
What if we want to let the user import pictures from their address book or photo gallery and turn those pictures into circles (as opposed to rectangles) to be added to a SKScene.
Is this possible?
Yes, it's possible. You need to create a mask node, which is just a plain shape picture, it's a blank circle in your case, and add it to a crop node. Please consider the following:
func addPhotoToFrame(photoFrame: SKSpriteNode) {
let pictureNode = SKSpriteNode(imageNamed: "picture")
pictureNode.name = "PictureNode"
let maskNode = SKSpriteNode(imageNamed: "picture-frame-mask")
maskNode.name = "Mask"
let cropNode = SKCropNode()
cropNode.addChild(pictureNode)
cropNode.maskNode = maskNode
photoFrame.addChild(cropNode)
}