I have been searching for animating 3D objects via Sceneform. I am very new to AR.
Can anybody provide a sample for 3D animation like a moving human?
Sceneform does not support animated renderables (like support for the animated FBX file format) right now. You can only move or rotate objects but you can't get something like a walking human easily.
Sceneform SDK for Android v1.7.0 supports animation (15th February 2019).
Sceneform includes an optional animation library, com.google.ar.sceneform:animation which enables animation playback.
Added ModelAnimator and AnimationData classes. Sceneform now has the ability to play animated models.
Added SkeletonNode class which can be used to bind nodes to bones in a skinned renderable, making it possible to attach objects to bones, access the positions of bones, and manipulate the positions of bones directly.
Added AugmentedFaceNode to the UX library which can be used to render visuals with ARCore's Augmented Faces feature. See the new sample in the /samples/augmentedfaces/ directory.
Added Vector3.equals(Vector3) and Quaterion.equals(Quaterion).
Exposed Quaternion(Vector3 eulerAngles) and Quaternion.eulerAngles() publicly.
Sceneform lets you import models with animation. You can use Sceneform APIs to play back and control the animation, and attach nodes to a model's skeleton.
For instance, the Sceneform Animation sample includes files used to build models of Andy the android and a baseball hat. The Andy model contains animation data, while the baseball hat is a non-animated model. When you run the sample, Andy breakdances and waves his arms while the hat remains fixed to his head using a node.
It's important to differentiate between model animation in Sceneform versus property animation in Android.
Model animations are created ahead of time by artists using modeling and animation software. They contain Skeletal Animation Data. These animations must be exported as fbx files, then imported into a sfb file (binary asset) to be used in Sceneform.
Property animation is a fundamental Android concept and is not specific to Sceneform. This kind of animation can change any mutable value on a Java object that has a getter and a setter. The animated values can be set dynamically, but cannot be packaged into an sfb file.
Hope this helps.
I have a 3d model of a human being standing. I implemented it into an project using arkit and can place it somewhere in the room. So far so good, but I would like to add an animation to the 3d model. For example when I press the buttonDance that it starts dancing. Not to move it up and down, but to add an animation to it.
What are keywords to make this work or does anyone have a brief way of doing this? Maybe what software to use or is it possible within sceneKit maybe?
You can use services such as Mixamo to generate an animation for your character.
I would advise you to use 3D models in Collade (.DAE) format because this format includes all your animations inside. You will have to clean the .DAE file to collect all the bone animations into one animation, more info here.
You will then need to read the animation from the .DAE file and add it to the node (your 3D model). Esteban Herrera has a great blog post on how to animate 3D models with ARKit.
We are creating iOS message extension app which is similar to Apple's Animoji app, we have created our own models (Some 3d faces) and we have .dae files for these models. We are converting these files to scn files in XCode by using option "Convert to scenkit scene file format (.scn)". When we see this scn file in XCode viewer, camera's Perspective view is shown for the model, and in this perspective view the face of the model looks thinner. Once we tap on the Perspective option drop down and choose "Front" option, model's face is in correct posture and is in its normal shape. We want this shape to show in the app also while running, but everytime we show model only the thinner face (perspective view) is visible. Is there a way to set camera to front for the model programmatically ? Is there any specific step we need to take while exporting the dae file ? so that by default camera view is front only.
the "front" camera in the Xcode editor is an orthographic camera (its usesOrthographicProjection is set to YES).
Alternatively you can change the fieldOfView or focalLength properties to reduce the amount of deformation due to perspective. You can find an illustration in this post on the Photography site.
You are looking for an orthographic projection for the camera. To setup it correctly you can add some camera object to the scene graph in .scn XCode viewer and choose "camera" option in drop down list. Camera projection type is perspective by default therefore you should change it to orthographic.
P.S. can you share some screenshot of your 3d face? Is it better than Memoji?)
Looking at the apple sample AR app, there are many realistic looking objects (cup, candle, etc). However working with the scene kit editor on Xcode it is clear that this only allows you to create basic objects.
My question is, what software/file can be used to create realistic scn objects? I'm sure that there is software that allows you to create 3D models and covert them to scn files. I just don't know which software to use or what files can be converted to scn
Note: I understand that this question may be too vague/broad for the Stackoverflow guidelines. I just don't know where to pose my question and this seems like the best place
To get some existing models to work with, here is what I did...just the basics that I know.
I went to Turbosquid and found a 3D model that would work for me.
Make sure it has the OBJ files. Buy and download.
Next download Blender. Import the OBJ file you just bought.
Export a DAE file.
Drag the DAE files and any textures (png files) into the .scnassets folder in your project.
Click on the DAE model in the .scnassets folder. Click to select the object in the Scene graph.
Click the globe in the upper right.
I clicked Diffuse and selected one of the PNGs I dragged in to apply it to the model
You can also skip the Blender conversion and just use one of the free online OBJ to DAE conversion tools. Google it. And try to buy a cheap $5 range model on Turbosquid that just has a OBJ file and not a lot of other piece parts. They are too big and create other issues as a starter approach anyway.
Update After watching a Apple WWDC presentation on Model IO in Xcode, I now see that you could drop in a OBJ file into your .scnassets folder. Select that file, go to Editor and select Convert to SCN file. That should work also but I have not tried. Worth trying with a copy of your OBJ file.
Update December 2018:
I've been working more with 3D files. Here is some incremental help on this issue.
Try using MeshLab to import your 3D model and convert it to a .DAE file. Drag the .DAE file into a folder in Xcode. That is what you are going to use to display in your app.
http://www.meshlab.net/
If your source 3D model is a .OBJ file, there are two related files that should be in the same folder as the .OBJ file. These are a *.mtl file and a *.jpg or *.BMP file. The .mtl file can be opened with TextEdit.
Open and make sure it has a line that says: map_Kd *.jpg. The .jpg is the texture image the wraps around the 3D mesh file. I've found that it is best to make sure your texture file is in .jpg format. If it isn't, change it to .jpg format (in Preview for example by resaving it as a jpeg) and then edit the .mtl file for the new .jpg file name.
I had some texture files that were .bmp and I just converted to .jpg, edited the .mtl file and I was good.
The second issue is the name of the node in the .obj file. The .obj file can also be opened with TextEdit. The .obj file should reference the .mtl file in the same folder. If not, you have a problem.
And here is the tricky part. When you are adding a childNode to the rootNode in a SceneKit scene, you have to fill in the "withName:" text name. Open the converted .DAE file that you have made from your .obj + .jpg + .mtl (all three are used when importing into MeshLab but after exporting to .DAE, there is only the .DAE file), and search for "node id =" . It might say: "node id="node". If so, the word "node" is the name of the childNode you enter for your text name in the "withName:" property of the scene.rootNode.childNode(withName: "node", recursively: true) call.
You can change your node name ID to node if it isn't already.
Hope this helps. Many hours of work and help from others to understand this next round of working with 3D models.
Xcode's SceneKit editor isn't a 3D art authoring package — just like its SpriteKit editor and Interface Builder components aren't equivalent to the likes of Photoshop, Illustrator, Pixelmator, Affinity Designer, etc. To create 3D art assets yourself, you'll need to learn Blender, Maya, 3DS Max, or one of the other major 3D authoring tools. Beware, the learning curve to becoming a 3D artist is a bit steeper than learning how to paint in 2D.
What the SceneKit editor is for is taking the output from a 3D artist and preparing or combining it for use in a SceneKit-based app or game — tweaking material definitions so they look right with SceneKit's renderer, arranging separate assets to create a game level or other scene, adding dynamic SceneKit-specific features like particle effects and physics, etc.
You bridge between these two worlds by exporting assets from your 3D art tools in one of the formats SceneKit can import. Digital Asset Exchange (.dae) is one of the best options here, but through SceneKit's lower level counterpart, Model I/O, you can also import other formats like OBJ or Pixar USD.
When you open those in Xcode, you get the SceneKit editor, so you can start marking SceneKit-specific edits and save the results for use in your app as .scn files.
There are a few things you can do in the process of authoring and prepping 3D assets that makes them look more realistic in ARKit. The ARKit session from WWDC (and the ReadMe file in the sample code project attached to that session) includes a few such tips:
use physically based materials
"bake" ambient occlusion and other static lighting effects
add invisible shadow planes
You can create your realistic 3D models in .DAE format which is supported by many tools. Then in Xcode, you can convert that .DAE file to .SCN format. For conversion, check this
How to convert .DAE to .SCN
In SceneKit's Scene graph you can import 3D assets with animation (from Blender, Maya or Houdini), create 3D primitives (SCNBox, SCNSphere, SCNPlane, SCNCylinder, etc) and assign UV-mapped textures on 5 SceneKit's shaders (Blinn, Lambert, Phong, Constant and PBR).
The proper way to create life-like animated models is to use professional 3D authoring tools like Autodesk Maya, Autodesk 3dsMax, Maxon Cinema 4D, The Foundry Modo, Blender or SideFX Houdini. These apps allow you create not only a perfect geometry but also realistic UV-mapped textures containing render passes for diffuse, transparency, metalness, bump, and occlusion slots of SceneKit's material editor.
When your 3d model and set of UV-mapped textures are ready for use all you need to do is to save a .dae or .usdz file. Format .dae must be converted to .scn. For this select a .dae file and apply command from menu:
// Editor - Convert to SceneKit file format...
Then for converted .scn model you can choose a Physically Based shader (PBR) in Material Inspector. Physically Based shaders for .scn and .usdz makes your models look realistically believable.
Using SketchUp, I made a DAE file, with a basic shape which has a few textures too.
The DAE file also comes with a folder the contains those textures.
I also am using SceneKit and have an existing scene, camera, light etc. I at the moment, I render many cubes into certain positions. How can I render the DAE model instead of the cube?
Cheers.
It's sure possible but please note that I am in objective C not swift(sorry).
Here is the code (Objective C but it's pretty easy to translate):
SCNScene *somthing = [SCNScene sceneNamed:#"mySketchupScene.dae"];
Now, when you import the DAE (we'll be calling it by it's real name Collada from here on out) into your app resources or .scnassets (perhaps by click-dragging) make sure that you first import the texture folder, then import the Collada.
I found that if you don't do the textures first, then Xcode messes up the matireals and you have to go manually re-assign all the textures.