I'm tinkering with the Sceneform SDK's Augmented Image sample code after I completed the accompanying code lab. The completed sample adds two types of objects to the AR scene: one is modeled with a CAD software and loaded from an sfb binary (that's the green maze) and the other one is a red ball which is constructed run-time using the MaterialFactory and ShapeFactory.
A simple experiment is to remove the green maze to only have the red ball (and remove the physics engine of course as well). In that case however the red ball does not appear on the AR scene.
The interesting thing is that the green maze does not have to appear on the scene - by that I mean I don't have to create the Node, assign renderable, etc. https://github.com/CsabaConsulting/sceneform-android-sdk/blob/master/samples/augmentedimage/app/src/main/java/com/google/ar/sceneform/samples/augmentedimage/AugmentedImageNode.java#L139:
mazeNode = new Node();
mazeNode.setParent(this);
mazeNode.setRenderable(mazeRenderable.getNow(null));
But if I take out the loading code https://github.com/CsabaConsulting/sceneform-android-sdk/blob/master/samples/augmentedimage/app/src/main/java/com/google/ar/sceneform/samples/augmentedimage/AugmentedImageNode.java#L89
mazeRenderable =
ModelRenderable.builder()
.setSource(context, Uri.parse("GreenMaze.sfb"))
.build();
and most importantly the code in the setImage which waits until the model is fully loaded and built https://github.com/CsabaConsulting/sceneform-android-sdk/blob/master/samples/augmentedimage/app/src/main/java/com/google/ar/sceneform/samples/augmentedimage/AugmentedImageNode.java#L125
if (!mazeRenderable.isDone()) {
CompletableFuture.allOf(mazeRenderable)
.thenAccept((Void aVoid) -> setImage(image))
.exceptionally(
throwable -> {
Log.e(TAG, "Exception loading", throwable);
return null;
});
return;
}
The ball won't appear. The ball (and any other run-time constructed objects I add) won't appear if I take out this .isDone() section above. I haven't found any indicator in the AR Session or anywhere else which would indicate that something is not ready to work yet. In my application I may only use run-time built 3D objects, do I need an sfb only for the sake of those appearing?
This happened because implicitly the Factory based scene building contains CompletableFuture as well! More specifically the material building is a function which returns CompletableFuture.
Not realizing this, I haven't quoted the important code section in the question. You can see that the just below the Maze model loader instructions:
https://github.com/CsabaConsulting/sceneform-android-sdk/blob/master/samples/augmentedimage/app/src/main/java/com/google/ar/sceneform/samples/augmentedimage/AugmentedImageNode.java#L94
MaterialFactory.makeOpaqueWithColor(context, new Color(android.graphics.Color.RED))
.thenAccept(
material -> {
ballRenderable =
ShapeFactory.makeSphere(0.01f, new Vector3(0, 0, 0), material); });
Here we can see the tell-tale sign of .thenAccept( which reveals that makeOpaqueWithColor returns a Future. While the model loading was also in the code we also had this check later:
if (!mazeRenderable.isDone()) {
CompletableFuture.allOf(mazeRenderable)
That code unfortunately doesn't pay attention to the material which is also asynchronously built. But the wait for the 3D model load gives enough time so that the material building can finish as well when it is accessed. However as soon as I removed the maze together with the future waiter code section, there was no safeguard for waiting for the material building to finish. This caused the whole scene hierarchy to be constructed before the material actually is ready. This results in an invisible scene.
https://github.com/googlecodelabs/arcore-augmentedimage-intro/issues/7
Related
I am building a simple game in SceneKit. It's a car racing game on a highway.
My lane is a SCNNode which holds children that are actually flattened clones of other static non-moving geometry models (vegetation, grass roadside, etc.). The problem is happening when I remove parts of the road that are invisible in camera by simply:
func removeLastLane() {
guard let laneNode = lanes.first else { return }
if lanes.count > 0 {
lanes.removeFirst()
}
laneNode.removeFromParentNode()
}
In the console the error appears:
objc[37558]: __weak variable at 0x600002701cf0 holds 0x600002701d99 instead of 0x600002200b40. This is probably incorrect use of objc_storeWeak() and objc_loadWeak(). Break on objc_weak_error to debug.
I have tried to debug it, but the trace is not showing anything sensible.
When I'm not using the flattenedClone() method, then above error is not occurring, but then I have more draw calls :(
Do you know why this error occurs?
EDIT 1:
Added link to the obj-c implementation of failing method in runtime https://opensource.apple.com/source/objc4/objc4-646/runtime/objc-weak.mm
I'm importing some simple OBJ assets using ModelIO like so:
let mdlAsset = MDLAsset(url: url, vertexDescriptor: nil, bufferAllocator: nil, preserveTopology: true, error: nil)
... and then adding them to a SceneKit SCN file. But, whenever I have meshes that have both quads/tris (often the case, for example eyeball meshes), the resulting mesh is jumbled:
Incorrect mesh topology
Re-topologizing isn't a good option since I sometimes have low-poly meshes with very specific topology, so I can't just set preserveTopology to false... I need a result with variable topology (i.e. MDLGeometryType.variableTopology).
How do I import these files correctly preserving their original topology?
I reported this as a bug at Apple Bug Reporter on 25th of November, bug id: 35687088
Summary: SCNSceneSourceLoadingOptionPreserveOriginalTopology does not actually preserve the original topology. Instead, it converts the geometry to all quads, messing up the 3D model badly. Based on its name it should behave exactly like preserveTopology of Model IO asset loading.
Steps to Reproduce: Load an OBJ file that has both triangles and polygons using SCNSceneSourceLoadingOptionPreserveOriginalTopology and load the same file into an MDLMesh using preserveTopology of ModelIO. Notice how it only works properly for the latter. Even when you create a new SCNGeometry based on the MDLMesh, it will "quadify" the mesh again to contain only quads (while it should support 3-gons and up).
On December 13th I received a reply with a request for sample code and assets, which I supplied 2 days later. I have not received a reply since (hopefully because they are just busy from catching up from the holiday season...).
As I mentioned in my bug report's summary, loading the asset with Model I/O does work properly, but then when you create a SCNNode based on that MDLMesh it ends up messing up the geometry again.
In my case the OBJ files I load have a known format as they are always files also exported with my app (no normals, colors, UV). So what I do is load the information of the MDLMesh (buffers, facetopology etc) manually into arrays, from which I then create a SCNGeometry manually. I don't have a complete separate piece of code of that for you as it is a lot and mixed with a lot of code specific to my app, and it's in Objective C. But to illustrate:
NSError *scnsrcError;
MDLAsset *asset = [[MDLAsset alloc] initWithURL:objURL vertexDescriptor:nil bufferAllocator:nil preserveTopology:YES error:&scnsrcError];
NSLog(#"%#", scnsrcError.localizedDescription);
MDLMesh * newMesh = (MDLMesh *)[asset objectAtIndex:0];
for (MDLSubmesh *faces in newMesh.submeshes) {
//MDLSubmesh *faces = newMesh.submeshes.firstObject;
MDLMeshBufferData *topo = faces.topology.faceTopology;
MDLMeshBufferData *vertIx = faces.indexBuffer;
MDLMeshBufferData *verts = newMesh.vertexBuffers.firstObject;
int faceCount = (int)faces.topology.faceCount;
int8_t *faceIndexValues = malloc(faceCount * sizeof(int8_t));
memcpy(faceIndexValues, topo.data.bytes, faceCount * sizeof(int8_t));
int32_t *vertIndexValues = malloc(faces.indexCount * sizeof(int32_t));
memcpy(vertIndexValues, vertIx.data.bytes, faces.indexCount * sizeof(int32_t));
SCNVector3 *vertValues = malloc(newMesh.vertexCount * sizeof(SCNVector3));
memcpy(vertValues, verts.data.bytes, newMesh.vertexCount * sizeof(SCNVector3));
....
....
}
In short, the preserveTopology option in SceneKit isn't working properly. To get from the working version in Model I/O to SceneKit I basically had to write my own converter.
I'm trying to swap the fragement-shader used in a program. The fragment-shaders all have the same variables, just different calculations. I am trying to provide alternative shaders for lower level hardware.
I end up getting single color outputs (instead of a texture), does anyone have an idea what I could be doing wrong? I know the shaders are being used, due to the color changing accordingly.
//if I don't do this:
//WebGL: INVALID_OPERATION: attachShader: shader attachment already has shader
gl.detachShader(program, _.attachedFS);
//select a random shader, all using the same parameters
attachedFS = fragmentShaders[~~(Math.qrand()*fragmentShaders.length)];
//attach the new shader
gl.attachShader(program, attachedFS);
//if I don't do this nothing happens
gl.linkProgram(program);
//if I don't add this line:
//globject.js:313 WebGL: INVALID_OPERATION: uniform2f:
//location not for current program
updateLocations();
I am assuming you have called gl.compileShader(fragmentShader);
Have you tried to test the code on a different browser and see if you get the same behavior? (it could be standards implementation specific)
Have you tried to delete the fragment shader (gl.deleteShader(attachedFS); ) right after detaching it. The
previous shader may still have a pointer in memory.
If this does not let you move forward, you may have to detach both shaders (vertex & frag) and reattach them or even recreate the program from scratch
I found the issue, after trying about everything else without result. It also explains why I was seeing a shader change, but just getting a flat color. I was not updating some of the attributes.
I am trying to create an app similar to the Reactable.
The user will be able to drag "modules" like an oscillator or filter from a menu into the "play area" and the module will be activated.
I am thinking to initialize the modules as they intersect with the "play area" background object. However, this requires me to name the modules automatically, i.e.:
let osci = AKOscillator()
where osci will automatically count up to be:
let osci1 = AKOscillator()
let osci2 = AKOscillator()
...
etc.
How will I be able to do this?
Thanks
edit: I am trying to use an array by creating an array of
var osciArray = [AKOscillator]()
and in my function to add an oscillator, this is my code:
let oscis = AKOscillator()
osciArray.append(oscis)
osciArray[oscCounter].frequency = freqValue
osciArray[oscCounter].amplitude = 0.5
osciArray[oscCounter].start()
selectedNode.userData = ["counter": oscCounter]
oscCounter += 1
currentOutput = osciArray[oscCounter]
AudioKit.output = currentOutput
AudioKit.start()
My app builds fine, but once the app starts running on the Simulator I get error : fatal error: Index out of range
I haven't used AudioKit, but I read about it a while ago and I have quite a big interest in it. From what I understand from the documentation, it's structured pretty much like SpriteKit: nodes connected together.
I guess then that most classes in the library derive from a base class, just like everything in SpriteKit derives from the SKNode class.
Since you are linking the audio kit nodes with visual representations via SpriteKit nodes, why don't you simply subclass from an SKSpriteNode and add an optional audioNode property with the base class from AudioKit?
That way you can just use SpriteKit to interact directly with the stored audio node property.
I think there's a lot of AudioKit related code in your question, but to answer the question, you only have to look at oscCounter. You don't show its initial value, but I am guessing it was zero. Then you increment it by 1 and try to access osciArray[oscCounter] which has only one element so it should be accessed by osciArray[0]. Move the counter lower and you'll be better off. Furthermore, your oscillators look like local variables, so they'll get lost once the scope is lost. They should be declared as instance variables in your class or whatever this is part of.
my following code returns null ,
byte[] image1 = _videoControl.getSnapshot(null);
any suggestion please
Few important moments about VideoControl.getSnapshot method:
some manufacturers may not implement getSnapshot() method;
the viewfinder must actually be visible on the screen prior to calling getSnapShot();
if you attempt to take pictures too quickly, however, getSnapShot() may
return null. The camera requires time to clear out its buffer and
prepare for the next shot;
you may check MMAPI System Property for "video.snapshot.encodings" before capturing:
if (System.getProperty("video.snapshot.encodings") == null) {
// getSnapshot() is not supported
}
You may read this chapter from book "Advanced BlackBerry Development":
http://books.google.com/books?id=F4Qu-lpoVncC&pg=PA53&lpg=PA53#v=onepage&q&f=false
Since VideoControl.getSnapshot method is not supported by all devices I'd recommend to use another approach. You can start the native BB Camera app with this line of code:
Invoke.invokeApplication(Invoke.APP_TYPE_CAMERA, new CameraArguments());
and then using the FileSystemJournalListener catch the taken image.
The BB SDK on your PC contains samples. Search for 'fileexplorerdemo' sample to see the rest of details.