ARKit Showing broken UIWebView - ios

I'm trying to detect an image using ARImageTrackingConfiguration and when the image is detected, an UIWebView shows up instead of the image scanned. I read a lot of problems regarding the new WBWebView but is not working either.
The problem it seems that is the UIWebView is not being updated from the main thread even thought I'm using the DispatchMain.
On a related topic (the code I use is pretty much the same), if I try to put a SKVideoNode instead of a UIWebView, the video gets played super laggy with some big pixels on top if I set plane.cornerRadius = 0.25
The code is the following
func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) {
guard let imageAnchor = anchor as? ARImageAnchor else { return }
updateQueue.async {
let physicalWidth = imageAnchor.referenceImage.physicalSize.width
let physicalHeight = imageAnchor.referenceImage.physicalSize.height
// Create a plane geometry to visualize the initial position of the detected image
let mainPlane = SCNPlane(width: physicalWidth, height: physicalHeight)
mainPlane.firstMaterial?.colorBufferWriteMask = .alpha
// Create a SceneKit root node with the plane geometry to attach to the scene graph
// This node will hold the virtual UI in place
let mainNode = SCNNode(geometry: mainPlane)
mainNode.eulerAngles.x = -.pi / 2
mainNode.renderingOrder = -1
mainNode.opacity = 1
// Add the plane visualization to the scene
node.addChildNode(mainNode)
// Perform a quick animation to visualize the plane on which the image was detected.
// We want to let our users know that the app is responding to the tracked image.
self.highlightDetection(on: mainNode, width: physicalWidth, height: physicalHeight, completionHandler: {
DispatchQueue.main.async {
let request = URLRequest(url: URL(string: "https://www.facebook.com/")!)
let webView = UIWebView(frame: CGRect(x: 0, y: 0, width: 400, height: 672))
webView.loadRequest(request)
let webViewPlane = SCNPlane(width: xOffset, height: xOffset * 1.4)
webViewPlane.cornerRadius = 0.25
let webViewNode = SCNNode(geometry: webViewPlane)
webViewNode.geometry?.firstMaterial?.diffuse.contents = webView
webViewNode.position.z -= 0.5
webViewNode.opacity = 0
rootNode.addChildNode(webViewNode)
webViewNode.runAction(.sequence([
.wait(duration: 3.0),
.fadeOpacity(to: 1.0, duration: 1.5),
.moveBy(x: xOffset * 1.1, y: 0, z: -0.05, duration: 1.5),
.moveBy(x: 0, y: 0, z: -0.05, duration: 0.2)
])
)
}
})
}
}
VIDEO VERSION
func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) {
guard let imageAnchor = anchor as? ARImageAnchor else { return }
updateQueue.async {
let physicalWidth = imageAnchor.referenceImage.physicalSize.width
let physicalHeight = imageAnchor.referenceImage.physicalSize.height
// Create a plane geometry to visualize the initial position of the detected image
let mainPlane = SCNPlane(width: physicalWidth, height: physicalHeight)
mainPlane.firstMaterial?.colorBufferWriteMask = .alpha
// Create a SceneKit root node with the plane geometry to attach to the scene graph
// This node will hold the virtual UI in place
let mainNode = SCNNode(geometry: mainPlane)
mainNode.eulerAngles.x = -.pi / 2
mainNode.renderingOrder = -1
mainNode.opacity = 1
// Add the plane visualization to the scene
node.addChildNode(mainNode)
// Perform a quick animation to visualize the plane on which the image was detected.
// We want to let our users know that the app is responding to the tracked image.
self.highlightDetection(on: mainNode, width: physicalWidth, height: physicalHeight, completionHandler: {
let size = imageAnchor.referenceImage.physicalSize
var videoNode = SKVideoNode()
switch imageAnchor.name {
case "Elephant-PostCard":
videoNode = SKVideoNode(fileNamed: "Movies/cat.mp4")
case "puppy":
videoNode = SKVideoNode(fileNamed: "puppy.mov")
default:
break
}
// invert our video so it does not look upside down
videoNode.yScale = -1.0
videoNode.play()
let videoScene = SKScene(size: CGSize(width: 1280, height: 720))
videoScene.anchorPoint = CGPoint(x: 0.5, y: 0.5)
videoScene.addChild(videoNode)
let plane = SCNPlane(width: size.width, height: size.height)
plane.cornerRadius = 0.25
plane.firstMaterial?.diffuse.contents = videoScene
let planeNode = SCNNode(geometry: plane)
plane.firstMaterial?.isDoubleSided = true
mainNode.addChildNode(planeNode)
})
}
}

UIWebView from UIKit module is now deprecated.
Use WKWebView from WebKit module instead.

Related

How to make sure that only one video will be played at a time?

I created an AR App that plays a video on a particular image anchor. I wanted to extend that application to deal with more than 1 image. Now when I scan 1st image it plays the video, and when I scan the 2nd image it plays another video. The Problem that occurs here is both videos are playing simultaneously.
The Code:
func renderer(_ renderer: SCNSceneRenderer, nodeFor anchor: ARAnchor) -> SCNNode? {
let node = SCNNode()
if let imageAnchor = anchor as? ARImageAnchor {
//below code needs to be duplicated in order to add more news
if imageAnchor.referenceImage.name == "harrypotter" {
let videoNode = SKVideoNode(fileNamed: "harrypotter.mp4")
videoNode.play()
let videoScene = SKScene(size: CGSize(width: 480, height: 360))
videoNode.position = CGPoint(x: videoScene.size.width / 2, y: videoScene.size.height / 2)
videoNode.yScale = -1.0
videoScene.addChild(videoNode)
let plane = SCNPlane(width: imageAnchor.referenceImage.physicalSize.width, height: imageAnchor.referenceImage.physicalSize.height)
plane.firstMaterial?.diffuse.contents = videoScene
let planeNode = SCNNode(geometry: plane)
planeNode.eulerAngles.x = -.pi / 2
node.addChildNode(planeNode)
}
if imageAnchor.referenceImage.name == "deatheater" {
let videoNode = SKVideoNode(fileNamed: "deatheater.mp4")
videoNode.play()
let videoScene = SKScene(size: CGSize(width: 480, height: 360))
videoNode.position = CGPoint(x: videoScene.size.width / 2, y: videoScene.size.height / 2)
videoNode.yScale = -1.0
videoScene.addChild(videoNode)
let plane = SCNPlane(width: imageAnchor.referenceImage.physicalSize.width, height: imageAnchor.referenceImage.physicalSize.height)
plane.firstMaterial?.diffuse.contents = videoScene
let planeNode = SCNNode(geometry: plane)
planeNode.eulerAngles.x = -.pi / 2
node.addChildNode(planeNode)
}
i tried to use 2 if conditions which helps to play a video on a particular image anchor. Now when I scan 1st image it plays the video, and when I scan the 2nd image it plays another video. The Problem that occurs here is both videos are playing simultaneously. I want that whenever I play the another video the 1st video should be stop automatically.
Welcome to StackOverflow Ankur!
Here are some suggestions to get what you want and improve your code.
Instead of duplicating code for each image, create a function that does all this.
Keep track of the currently playing song so that you can stop it.
I'm not familiar with these APIs but you'd get something like this:
class Player
{
private let rootNode = SCNNode()
private var playingNode: SCNNode?
func renderer(_ renderer: SCNSceneRenderer, nodeFor anchor: ARAnchor)
{
if let imageAnchor = anchor as? ARImageAnchor
{
if let newNode = playVideo(names: imageAnchor.referenceImage.name)
{
<inset code to stop playing 'playingNode' and remove it>
playingNode = newNode
}
}
}
private func playVideo(named name: String) -> SCNNode?
{
guard <check if video file with 'name' exists> else { return nil }
let videoNode = SKVideoNode(fileNamed: "\(name).mp4")
videoNode.play()
let videoScene = SKScene(size: CGSize(width: 480, height: 360))
videoNode.position = CGPoint(x: videoScene.size.width / 2,
y: videoScene.size.height / 2)
videoNode.yScale = -1.0
videoScene.addChild(videoNode)
let plane = SCNPlane(width: imageAnchor.referenceImage.physicalSize.width,
height: imageAnchor.referenceImage.physicalSize.height)
plane.firstMaterial?.diffuse.contents = videoScene
let planeNode = SCNNode(geometry: plane)
planeNode.eulerAngles.x = -.pi / 2
return planeNode
}
}
Good luck.

Cannot Display Sprites Above SK3DNode

I'm displaying some basic 3D geometry within my SpriteKit scene using an instance of SK3DNode to display the contents of a SceneKit scene, as explained in Apple's article here.
I have been able to position the node and 3D contents as I want using SceneKit node transforms and the position/viewport size of the SK3DNode.
Next, I want to display some other sprites in my SpriteKit scene overlaid on top of the 3D content, but I am unable to do so: The contents of the SK3DNode are always drawn on top.
I have tried specifying the zPosition property of both the SK3DNode and the SKSpriteNode, to no avail.
From Apple's documentation on SK3DNode:
Use SK3DNode objects to incorporate 3D SceneKit content into a
SpriteKit-based game. When SpriteKit renders the node, the SceneKit
scene is animated and rendered first. Then this rendered image is
composited into the SpriteKit scene. Use the scnScene property to
specify the SceneKit scene to be rendered.
(emphasis mine)
It is a bit ambiguous withv regard to z-order (it only seems to mention the temporal order in which rendering takes place).
I have put together a minimal demo project on GitHub; the relevant code is:
1. SceneKit Scene
import SceneKit
class SceneKitScene: SCNScene {
override init() {
super.init()
let box = SCNBox(width: 10, height: 10, length: 10, chamferRadius: 0)
let material = SCNMaterial()
material.diffuse.contents = UIColor.green
box.materials = [material]
let boxNode = SCNNode(geometry: box)
boxNode.transform = SCNMatrix4MakeRotation(.pi/2, 1, 1, 1)
self.rootNode.addChildNode(boxNode)
}
required init?(coder: NSCoder) {
fatalError("init(coder:) has not been implemented")
}
}
2. SpriteKit Scene
import SpriteKit
class SpriteKitScene: SKScene {
override init(size: CGSize) {
super.init(size: size)
// Scene Background
self.backgroundColor = .red
// 3D Node
let objectNode = SK3DNode(viewportSize: size)
objectNode.scnScene = SceneKitScene()
addChild(objectNode)
objectNode.position = CGPoint(x: size.width/2, y: size.height/2)
let camera = SCNCamera()
let cameraNode = SCNNode()
cameraNode.camera = camera
objectNode.pointOfView = cameraNode
objectNode.pointOfView?.position = SCNVector3(x: 0, y: 0, z: 60)
objectNode.zPosition = -100
// 2D Sprite
let sprite = SKSpriteNode(color: .yellow, size: CGSize(width: 250, height: 60))
sprite.position = objectNode.position
sprite.zPosition = +100
addChild(sprite)
}
required init?(coder aDecoder: NSCoder) {
fatalError("init(coder:) has not been implemented")
}
}
...And the rendered result is:
(I want the yellow rectangle above the green box)
I made a Technical Support Incident with Apple about this and they just got back to me. The solution is actually very very simple.
If you want 2D sprites to render on top of SK3DNodes, you need to stop the contents of the SK3DNodes from writing to the depth buffer.
To do this, you just need to set writesToDepthBuffer to false on the SCNMaterial.
...
let material = SCNMaterial()
material.diffuse.contents = UIColor.green
material.writesToDepthBuffer = false
...
Boom. Works.
Please note that this is just something I stumbled upon. I have no idea why it works and I probably wouldn't trust it without further understanding, but maybe it'll help find an explanation or a real solution.
It seems that having an SKShapeNode (with a fill) alongside an SK3DNode (either as a sibling, part of a sibling tree, or child), draws it in proper order. The SKShapeNode doesn't seem to need to intersect with the SK3DNode either.
The fill is important, as having a transparent fill makes this not work. Stroke doesn't seem to have any effect.
An SKShapeNode of extremely small size and almost zero alpha fill works too.
Here's my playground:
import PlaygroundSupport
import SceneKit
import SpriteKit
let viewSize = CGSize(width: 300, height: 150)
let viewportSize: CGFloat = viewSize.height * 0.75
func skview(color: UIColor, index: Int) -> SKView {
let scene = SKScene(size: viewSize)
scene.backgroundColor = color
let view = SKView(
frame: CGRect(
origin: CGPoint(x: 0, y: CGFloat(index) * viewSize.height),
size: viewSize
)
)
view.presentScene(scene)
view.showsDrawCount = true
// Draw the box of the 3d node view port
let viewport = SKSpriteNode(color: .orange, size: CGSize(width: viewportSize, height: viewportSize))
viewport.position = CGPoint(x: viewSize.width / 2, y: viewSize.height / 2)
scene.addChild(viewport)
return view
}
func cube() -> SK3DNode {
let mat = SCNMaterial()
mat.diffuse.contents = UIColor.green
let box = SCNBox(width: viewportSize, height: viewportSize, length: viewportSize, chamferRadius: 0)
box.firstMaterial = mat
let boxNode3d = SCNNode(geometry: box)
boxNode3d.runAction(.repeatForever(.rotateBy(x: 10, y: 10, z: 10, duration: 10)))
let scene = SCNScene()
scene.rootNode.addChildNode(boxNode3d)
let boxNode2d = SK3DNode(viewportSize: CGSize(width: viewportSize, height: viewportSize))
boxNode2d.position = CGPoint(x: viewSize.width / 2, y: viewSize.height / 2)
boxNode2d.scnScene = scene
return boxNode2d
}
func shape() -> SKShapeNode {
let shape = SKShapeNode(rectOf: CGSize(width: viewSize.height / 4, height: viewSize.height / 4))
shape.strokeColor = .clear
shape.fillColor = .purple
return shape
}
func rect(_ color: UIColor) -> SKSpriteNode {
let sp = SKSpriteNode(texture: nil, color: color, size: CGSize(width: 200, height: viewSize.height / 4))
sp.position = CGPoint(x: viewSize.width / 2, y: viewSize.height / 2)
return sp
}
// The original issue, untouched.
func v1() -> SKView {
let v = skview(color: .red, index: 0)
v.scene?.addChild(cube())
v.scene?.addChild(rect(.yellow))
return v
}
// Shape added as sibling after the 3d node. Notice that it doesn't overlap the SK3DNode.
func v2() -> SKView {
let v = skview(color: .blue, index: 1)
v.scene?.addChild(cube())
v.scene?.addChild(shape())
v.scene?.addChild(rect(.yellow))
return v
}
// Shape added to the 3d node.
func v3() -> SKView {
let v = skview(color: .magenta, index: 2)
let box = cube()
box.addChild(shape())
v.scene?.addChild(box)
v.scene?.addChild(rect(.yellow))
return v
}
// 3d node added after, but zPos set to -1.
func v4() -> SKView {
let v = skview(color: .cyan, index: 3)
v.scene?.addChild(shape())
v.scene?.addChild(rect(.yellow))
let box = cube()
box.zPosition = -1
v.scene?.addChild(box)
return v
}
// Shape added after the 3d node, but not as a sibling.
func v5() -> SKView {
let v = skview(color: .green, index: 4)
let parent = SKNode()
parent.addChild(cube())
parent.addChild(rect(.yellow))
v.scene?.addChild(parent)
v.scene?.addChild(shape())
return v
}
let container = UIView(frame: CGRect(origin: .zero, size: CGSize(width: viewSize.width, height: viewSize.height * 5)))
container.addSubview(v1())
container.addSubview(v2())
container.addSubview(v3())
container.addSubview(v4())
container.addSubview(v5())
PlaygroundPage.current.liveView = container
TL;DR
In your code, try:
...
let shape = SKShapeNode(rectOf: CGSize(width: 0.01, height: 0.01))
shape.strokeColor = .clear
shape.fillColor = UIColor.black.withAlphaComponent(0.01)
// 3D Node
let objectNode = SK3DNode(viewportSize: size)
objectNode.addChild(shape)
...

Scene is modified when rendering callback

I had the following code which produced the error:
func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) {
if anchor is ARImageAnchor {
let phoneScene = SCNScene(named: "Phone_01.scn")!
let phoneNode = phoneScene.rootNode.childNode(withName: "parentNode", recursively: true)!
// rotate the phone node
let rotationAction = SCNAction.rotateBy(x: 0, y: 0.5, z: 0, duration: 1)
let inifiniteAction = SCNAction.repeatForever(rotationAction)
phoneNode.runAction(inifiniteAction)
phoneNode.position = SCNVector3(anchor.transform.columns.3.x,anchor.transform.columns.3.y + 0.1,anchor.transform.columns.3.z)
node.addChildNode(phoneNode)
}
}
Scene is modified in a rendering callback of another scene.
So I replaced it with the following:
func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) {
if anchor is ARImageAnchor {
DispatchQueue.global().async {
let phoneScene = SCNScene(named: "Phone_01.scn")!
let phoneNode = phoneScene.rootNode.childNode(withName: "parentNode", recursively: true)!
DispatchQueue.main.async {
// rotate the phone node
let rotationAction = SCNAction.rotateBy(x: 0, y: 0.5, z: 0, duration: 1)
let inifiniteAction = SCNAction.repeatForever(rotationAction)
phoneNode.runAction(inifiniteAction)
phoneNode.position = SCNVector3(anchor.transform.columns.3.x,anchor.transform.columns.3.y + 0.1,anchor.transform.columns.3.z)
node.addChildNode(phoneNode)
}
}
}
}
And now the error is gone and everything works OK. My question is: is that the correct solution? Should I switch to background thread to load the scene and then to main thread to add the nodes. Are nodes even added on the main thread?
Try something like this in the method delegate. This was an example of an old proyect.
DispatchQueue.main.async {
if let imageAnchor = anchor as? ARImageAnchor {
let plane = SCNPlane(width: imageAnchor.referenceImage.physicalSize.width, height: imageAnchor.referenceImage.physicalSize.height)
plane.firstMaterial?.diffuse.contents = UIColor(white: 1.0, alpha: 0.5)
let planeNode = SCNNode(geometry: plane)
planeNode.eulerAngles.x = -.pi
node.addChildNode(planeNode)
...
}
}

How to create 3D Virtual Content with SceneKit in Swift 4

I am using this to create a simple 3D object in Single View application. In a default configuration, the following code places a 10-centimeter cube 20 centimeters in front of the camera's initial position
let cubeNode = SCNNode(geometry: SCNBox(width: 0.1, height: 0.1, length: 0.1, chamferRadius: 0))
cubeNode.position = SCNVector3(0, 0, -0.2)
sceneView.scene.rootNode.addChildNode(cubeNode)
When plane detection is enabled, ARKit adds and updates anchors for each detected plane. To add visual content for these anchors, implement
ARSCNViewDelegate methods such as the following:
func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) {
guard let planeAnchor = anchor as? ARPlaneAnchor else { return }
let plane = SCNPlane(width: CGFloat(planeAnchor.extent.x), height: CGFloat(planeAnchor.extent.z))
let planeNode = SCNNode(geometry: plane)
planeNode.position = SCNVector3Make(planeAnchor.center.x, 0, planeAnchor.center.z)
planeNode.transform = SCNMatrix4MakeRotation(-Float.pi / 2, 1, 0, 0)
node.addChildNode(planeNode)
}
How can i rander the 3D Object.

Adding an SKLabelNode to Scene Kit View

I am building a game using Scene Kit. In order to present the score I wanted to use an SKLabelNode on the screen, however, when I attach it to a SCNNode, it looks very blurry:
Here is the code that I have written to do this, please let me know if there is a better way to go about doing this without having the text be so blurry. Thank you so much!
func initHUD() {
let skScene = SKScene(size: CGSize(width: 100, height: 100))
skScene.backgroundColor = UIColor(white: 0.0, alpha: 0.0)
labelNode = SKLabelNode()
labelNode.fontSize = 20
labelNode.position.y = 50
labelNode.position.x = 50
skScene.addChild(labelNode)
let plane = SCNPlane(width: 1, height: 1)
let material = SCNMaterial()
material.lightingModelName = SCNLightingModelConstant
material.doubleSided = true
material.diffuse.contents = skScene
plane.materials = [material]
hudNode = SCNNode(geometry: plane)
hudNode.name = "HUD"
hudNode.rotation = SCNVector4(x: 1, y: 0, z: 0, w: 3.14159265)
hudNode.position = SCNVector3(x:0, y: 1, z: -5)
}
func updateHUD() {
labelNode.text = "\(score)"
}
The typical way to do a HUD for a SceneKit scene is to create a SpriteKit SKScene and set it as the overlaySKScene of your SceneKit view. Then it renders at full resolution and always at the same view-relative size and position.

Resources