I've imported a Wavefront OBJ file from a URL and now I'd like to insert it into my scene (SceneKit) on my iOS 9 app (in Swift). What I've done so far is:
let asset = MDLAsset(URL: localFileUrl)
print("count = \(asset.count)") // 1
Any help converting this to a SCNNode would be appreciated. According to to Apple's docs:
Model I/O can share data buffers with the MetalKit, GLKit, and SceneKit frameworks to help you load, process, and render 3D assets efficiently.
But I'm not sure how to get buffer from an MDLAsset into a SCNNode.
Turns out this quite easy as many of the ModelIO classes already bridge. I was doing import ModelIO which gave me access to all the ModelIO classes and likewise import SceneKit which gave me the SceneKit classes, but, I was missing import SceneKit.ModelIO to bring in the SceneKit support for ModelIO.
let url = NSURL(string: "url-to-your-obj-here")
let asset = MDLAsset(URL: url!)
let object = asset.objectAtIndex(0)
let node = SCNNode(MDLObject: object)
Easy as that...
Related
I have an .fbx file which is having animations.
I tried using AssimpKit to import .fbx file but could not import the file I have added the file in project it self.
I used following code:
let filePath = Bundle.main.path(forResource: "Gag01_08", ofType: "fbx")
let animFile = SCNAssimpScene(named: "Gag01_08", inDirectory: filePath, options: nil)
How to import fbx file for ARKit?
Can the animations will auto play?
I just checked the project-space of AssimpKit. It does not support the Assimp-release 4.1, which brings a more stable fbx-support with it.
To work around this issue you can try to upgrade the AssimpKit on your own or use assimp 4.1 as the native wrapper: without AssimpKit in it.
I'm importing some simple OBJ assets using ModelIO like so:
let mdlAsset = MDLAsset(url: url, vertexDescriptor: nil, bufferAllocator: nil, preserveTopology: true, error: nil)
... and then adding them to a SceneKit SCN file. But, whenever I have meshes that have both quads/tris (often the case, for example eyeball meshes), the resulting mesh is jumbled:
Incorrect mesh topology
Re-topologizing isn't a good option since I sometimes have low-poly meshes with very specific topology, so I can't just set preserveTopology to false... I need a result with variable topology (i.e. MDLGeometryType.variableTopology).
How do I import these files correctly preserving their original topology?
I reported this as a bug at Apple Bug Reporter on 25th of November, bug id: 35687088
Summary: SCNSceneSourceLoadingOptionPreserveOriginalTopology does not actually preserve the original topology. Instead, it converts the geometry to all quads, messing up the 3D model badly. Based on its name it should behave exactly like preserveTopology of Model IO asset loading.
Steps to Reproduce: Load an OBJ file that has both triangles and polygons using SCNSceneSourceLoadingOptionPreserveOriginalTopology and load the same file into an MDLMesh using preserveTopology of ModelIO. Notice how it only works properly for the latter. Even when you create a new SCNGeometry based on the MDLMesh, it will "quadify" the mesh again to contain only quads (while it should support 3-gons and up).
On December 13th I received a reply with a request for sample code and assets, which I supplied 2 days later. I have not received a reply since (hopefully because they are just busy from catching up from the holiday season...).
As I mentioned in my bug report's summary, loading the asset with Model I/O does work properly, but then when you create a SCNNode based on that MDLMesh it ends up messing up the geometry again.
In my case the OBJ files I load have a known format as they are always files also exported with my app (no normals, colors, UV). So what I do is load the information of the MDLMesh (buffers, facetopology etc) manually into arrays, from which I then create a SCNGeometry manually. I don't have a complete separate piece of code of that for you as it is a lot and mixed with a lot of code specific to my app, and it's in Objective C. But to illustrate:
NSError *scnsrcError;
MDLAsset *asset = [[MDLAsset alloc] initWithURL:objURL vertexDescriptor:nil bufferAllocator:nil preserveTopology:YES error:&scnsrcError];
NSLog(#"%#", scnsrcError.localizedDescription);
MDLMesh * newMesh = (MDLMesh *)[asset objectAtIndex:0];
for (MDLSubmesh *faces in newMesh.submeshes) {
//MDLSubmesh *faces = newMesh.submeshes.firstObject;
MDLMeshBufferData *topo = faces.topology.faceTopology;
MDLMeshBufferData *vertIx = faces.indexBuffer;
MDLMeshBufferData *verts = newMesh.vertexBuffers.firstObject;
int faceCount = (int)faces.topology.faceCount;
int8_t *faceIndexValues = malloc(faceCount * sizeof(int8_t));
memcpy(faceIndexValues, topo.data.bytes, faceCount * sizeof(int8_t));
int32_t *vertIndexValues = malloc(faces.indexCount * sizeof(int32_t));
memcpy(vertIndexValues, vertIx.data.bytes, faces.indexCount * sizeof(int32_t));
SCNVector3 *vertValues = malloc(newMesh.vertexCount * sizeof(SCNVector3));
memcpy(vertValues, verts.data.bytes, newMesh.vertexCount * sizeof(SCNVector3));
....
....
}
In short, the preserveTopology option in SceneKit isn't working properly. To get from the working version in Model I/O to SceneKit I basically had to write my own converter.
Trying to figure out how to solve my issue of applying shaders to my ARSCNView.
Previously, when using a standard SCNView, i have successfully been able to apply a distortion shader the following way:
if let path = Bundle.main.path(forResource: "art.scnassets/distortion", ofType: "plist") {
if let dict = NSDictionary(contentsOfFile: path) {
let technique = SCNTechnique(dictionary: dict as! [String : AnyObject])
scnView.technique = technique
}
}
Replacing SCNView with ARSCNView gives me the following error(s):
"Error: Metal renderer does not support nil vertex function name"
"Error: _executeProgram - no pipeline state"
I was thinking it's because that ARSCNView uses a different renderer than SCNView. But logging ARSCNView.renderingAPI tells me nothing about the renderer, and i can't seem to choose one when i construct my ARSCNView instance. I must be missing something obvious, because i can't seem to find a single resource when scouring for references online.
My initial idea was instead use a SCNProgram to apply the shaders. But i can't find any resources of how to apply it to an ARSCNView, or if it's even a correct/possible solution, SCNProgram seems to be reserved for materials.
Anyone able to give me any useful pointers of how to solve vertex+fragment shaders for ARSCNView?
SCNTechnique for ARSCNView does not work with GLSL shaders, instead Metal functions need to be provided in the technique's plist file under the keys metalVertexShader and metalFragmentShader.
To the contrary, documentation says any combination of shader should work:
You must specify both fragment and vertex shaders, and you must
specify either a GLSL shader program, a pair of Metal functions, or
both. If both are specified, SceneKit uses whichever shader is
appropriate for the current renderer.
So it might be a mistake, but I guess the documentation is outdated. Since all ARKit running devices also run Metal, GLSL support has not been added to ARSCNViews.
As iOS12 deprecates OpenGL this looks like planned.
I had this issue in ARKit iOS11.4 and 12 and it came down to a series of miss-spelt shaders. I hope this might help someone.
I am using the following code to render a scene in scenekit and it works perfectly when the dae file is loaded from art.scnassests folder.
let scene = SCNScene(named: "art.scnassets/idle.dae")
However I want to download the asset and apply it and I am getting an error
let documentsURL = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask).first!
let scene = SCNScene(named: documentsURL.absoluteString+"idle.dae")
A file named idle.dae exists in the folder.
I get the error: fatal error: unexpectedly found nil while unwrapping an Optional value
How to load the downloaded asset and apply dynamically? What am I doing wrong? Any pointers please? I am a noob to iOS programming.
Unless something has changed for iOS 11, you won't be able to download and instantiate a DAE file at runtime on iOS. They are compressed/compiled at build time using a utility named scntool.
Can you instead use one of the file formats supported by Model I/O? See https://developer.apple.com/videos/play/wwdc2015/602/?time=320 for the original list (Alembic .abc, Polygon .ply Triangles .stl, WaveFront .obj), and https://developer.apple.com/videos/play/wwdc2017/610/ for a quick discussion of Pixar's USD (Universal Scene Description).
If you're stuck with DAE files, Frederik Jacques has an article at https://the-nerd.be/2014/11/07/dynamically-load-collada-files-in-scenekit-at-runtime/ which outlines his experience reverse engineering the DAE processing pipeline. His technique allows downloaded SCN files which have been processed from DAE files on a server.
See also Load uncompressed collada file using iOS Scene Kit (with comments by an authoritative source) and https://forums.developer.apple.com/thread/38010.
I used to render 3d scene with openGL and metal on IOS, and the file format which I used was OBJ and CTM. These days I am trying Scene Kit. It seems that SceneKit only load DAE file. All the demos I can found on the Internet use DAE file , and I can't see the array of vertex and facet in their codes.
How can I load OBJ file or CTM file instead of DAE file?
Loading an OBJ file
It is as simple as passing MDLAsset a valid URL.
private func nodeForURL(url: NSURL) -> SCNNode
{
let asset = MDLAsset(URL: url)
let object = asset.objectAtIndex(0)
let node = SCNNode(MDLObject: object)
return node
}
This will not only correctly load the .obj file, but it will load referenced .mtl files.
you can do that by writing your own importer. Take a look at SCNGeometry, SCNGeometrySource and SCNGeometryElement.
edit: starting iOS 9.0 and OS X 10.11 SceneKit can open OBJ files or any other file format supported by Model I/O. You can use previously existing APIs to do that (such as +sceneNamed:) or the new +sceneWithMDLAsset: method.
As of iOS 9/OS X 10.11, you can use Model I/O's MDLAsset to import OBJ files (and a few other formats). How do you convert Wavefront OBJ file to an SCNNode with Model I/O has sample code.
EDIT: ModelIO can probably load OBJ files now. I haven’t tried this path myself. This answer was written before iOS 9 and OS X 10.11:
SceneKit can't load DAE files on iOS, actually, it actually precompiles the DAE files to an internal format for iOS devices.
If you want to transform your OBJs to DAEs you can write a simple importer/exporter on OS X to do so — on OS X SceneKit will actually read OBJ files (it's not documented but it works) and will write DAEs.
Or you could download the "Assimp" project on github if you'd like to try to read OBJs yourself, but it's going to be a bit of work getting it into SceneKit objects.
Further information regarding the supported file formats:
The following 3D file formats are supported by SceneKit and can be imported to an .scn file using the Scene Editor in Xcode:
DAE, OBJ, Alembic, STL and PLY files.
Source: WWDC 2015 Session "Enhancements to SceneKit" at 02:24