My goal is to have an app where I can scan a certain image and display an AR-Generated Model next to it. I am following this project.
I already imported the image I want to scan and that is working as expected. The problem now is to import my own Model. And that is where I struggle.
What I want is to display a 2D Image on a simple PlaneScene on top or next to the detected image.
I managed to create a .dae file from Blender. I also imported it into Xcode and converted it inside XCode to a .scn file.
This is how it looks:
As you can see the image is not correctly displayed.. It is just white. Also when running the app, the Object is not being displayed. I can not see it anywhere, but it is also not throwing an error.
This is my .dae file: File
Maybe I dont even need Blender for that but I couldn't find a way to display an image onto a Plane in XCode...
What am I missing here? I feel like there is something wrong with my Model. Any help is appreciated!
As I can see, when I open your file in Blender and go to the Front view:
Your Plane is wrong oriented in the space. One looks at it from the side and it is not well centered. (but this is probably what you want).
But you don't need an imported geometry object. Apple has a solution to this.
I recommend you to use the built in plane (SCNPlane, what is a SCNGeometry object) for your project. It will have a front view oriantation by default and you can easely move the offset position.
Let me know if you need an example. But it should be easy to find out how to do this.
PS: If you want to import Models this here is a useful extension to do this:
extension SCNNode {
convenience init(named name: String) {
self.init()
guard let scene = SCNScene(named: name) else {return}
for childNode in scene.rootNode.childNodes {addChildNode(childNode)}
}
}
Related
I've replaced the tree.glb model in the ThreeJS placeground example (https://github.com/8thwall/web/tree/master/examples/threejs/placeground), but it's not showing. It works fine when using tree.glb.
To debug, I've also tried replacing it with the jellyfish-model.glb available in the examples, but it also doesn't show when tapping on the floor plane.
Is there something wrong with my code, or with the .glb models I'm replacing tree.gbl with?
const modelFile = 'tree.glb' // 3D model to spawn at tap
to
const modelFile = 'jellyfish-model.glb' // 3D model to spawn at tap
File structure on github: 8thwall-3js-test-github
Ideally, I'd like to replicate what I've done using Unity+Vuforia in this example (which basically places a .png onto a floor plane): https://www.youtube.com/watch?v=poWvXVB4044
I'd start by looking at the scale of the 3d model. The tree model in the link you provided is quite large, so it's being scaled down in size. See https://github.com/8thwall/web/blob/master/examples/threejs/placeground/index.js#L7-L8
Prove to yourself that the model is being loaded by adding a console.log('model added!') type statement into animateIn() (as that is the model loaded handler)
My guess is that your jellyfish-model.glb is there, just very small. Trying adjusting startScale and endScale to larger values and see if that helps.
If I run Xcode's ARKit project template, I can move around the default ship as expected.
I have tried importing some dae models that I have exported from Blender, switching the ship to the new item but keeping the code the same. They always show in view but position in front the camera. If I try to move around the model it sticks infront of the camera and doesn't stay in the same virtual space.
I have also tried importing a dae from another tutorial and that works as expected.
Does anything need setting when exporting a 3d model as a Collada dae file to make sure it will anchor to a position in ARKit?
When importing 3D-models for ARKit make sure to check:
the scale of your file. One unit equals 1 meter
The model should be centered in the coordinate origin to make it easy to position the model
Try to import the file in the SceneKit editor in Xcode and check if it gets imported correctly.
I'm currently trying to combine the following sources:
Apples SceneKit Vehicle Demo, Resp. its Swift version,
ARKit by example, and resp. its Swift version.
Each project on its own works like a charm (although I changed the vehicle demo so that the car can be controlled by on-screen buttons).
Now, when I try to combine both projects to create an augmented reality racing game, I run into problems regarding the size of the .dae model of the car: it's too big.
I can scale the model using the (chassis) nodes .scale property, but as soon as I add the SCNPhysicsVehicle properties and behaviour, the car gets reset(?) to its original size. I tried to scale the model in Xcode (open dae file, change scale), but its bounding box remains the same - that tells me that the rescaling didn't work properly.
Any hints?
1)you can scale the dae models by art.scnassets directly.
art.scnassets -> car.dae -> node inspector -> transforms -> scale the object
2) can scale 3dmodel by SCNAction
let scene = SCNScene(named: "art.scnassets/cup.dae")!
let node = scene.rootNode.childNode(withName: "cup", recursively: true)!
let action = SCNAction.scale(by: sender.scale, duration: 1.0)
node.runAction(action)
What I like to do is use Blender or some other 3d modeling program to resize your dae model to work in meters. Everything in ARKit is based on meters, so by sticking to the same metric you can get all your models to play well together without having to guess what the scale factor needs to be.
I'm not sure how to fix the model directly in Xcode. However, you can fix it in blender. Start by importing the object into blender. Select the object and observe it's dimensions. Scale the object to the desired dimensions and apply them by hittin Ctrl + A, and selecting scale. Alternatively, from the object menu, you can select Apply -> Scale. Now you can export your model with the corrected size.
I am attempting to experiment with Apple’s Fox game SceneKit example (link below) by adding a model with a simple animation like the ‘panda.scn’ and ‘walk.scn’ assets.
I can create a static model with no joints or animation that works: e.g. In Maya (2017) I add a simple sphere, export selection to FBX_DAE (COLLADA) file, drag it into the project in XCode and convert it to a SCN file. I can then drag that model into the ‘level.scn’, position and scale it as I’d expect.
However as soon as I add any animation or joints to my model I lose the ability to position and scale the model in XCode.
In Maya I add two joints to my sphere select the sphere and joints and export as above. When I examine the model SCN in XCode (either in isolation or as a reference within another scene) I find that I cannot apply any translation or scaling. XCode lets me move the xyz locators in the GUI and update scale but the model does not change. I can see the mesh and joints in the outline view and I have tried moving the joints instead of the mesh, but they do nothing. I have even tried ignoring the GUI and positioning the model in code just as the panda character is set up, but applying positions or transforms to the node does nothing - it always appears at the origin and default scale.
Ignoring that for the moment, my understanding from looking at the ‘walk.scn’ file is that an animation is just an export of the joints with keyframes. I have tried to reproduce that by exporting only the joints to a separate DAE and importing it, then applying that in code as they do with the ‘walkAnimation’. This seems to do nothing as well.
I have experimented with various settings of the FBX_DAE export dialog including baking animations (By the way - what is the difference between baking an animation on export vs baking it in Maya before export? The former seems to do something and the other does nothing in my tests.)
I would dearly love a workflow for creating some simple character animations in Maya and getting them into SceneKit. Any help is greatly appreciated.
For reference:
Apple’s source: https://developer.apple.com/library/content/samplecode/Fox/Introduction/Intro.html
Maya scene for my trivial ball model with two joints:
http://pat.net/misc/ball1.mb
and the DAE output from Maya for that:
http://pat.net/misc/ball1.dae
UPDATE:
Mnuages's answer appears to be correct in that when I added a parent control over the geometry and joints SceneKit then allowed me to move them. But even after brushing up on my understanding of how these nodes and their transforms relate in Maya I do not feel that I have a real understanding of how SceneKit is interpreting them. (Would love to read some docs on that would illuminate this more if they exist).
UPDATE #2:
I was finally able to create an animation by doing the following: 1) Export either the full scene or just the joint with the animation being sure to select the "bake animation" option in the DAE export dialog or bake the entire animation using Key->Bake Animation. 2) It only works if I load the DAE file in scenekit instead of converting it to an SCN. Converting to SCN format seems to lose the animation.
what happens is that the node named joint1 is animated by the joint1-anim animation. So even if you move joint1 in the editor, what you see on screen is the result after the animation is evaluated.
If you create an intermediate node, say joint1-parent, and make joint1 as child node of joint1-parent, then you'll be able to translate and rotate joint1-parent freely and see the effects on joint1.
As for why moving pSphere does not change anything, it's the same idea. Just like the animation overrides the position, the skeleton will reposition the mesh.
The problem is in the title. I need to access the SCNGeometry of a SCNNode and then edit the SCNMaterial collection of the SCNGeometry but when I inspect the node almost all its properties are set to nil including the geometry property. The weird thing is that all the nodes have been loaded from a .dae file and are all rendered fine in the scene. Any insight would be much appreciated.
are you sure that you are retrieving the correct node? What API do you use to do that? What do you see in the SceneKit editor that's built into Xcode?