The Collada 3D model is flashing - ios

I have developed iOS application to have AR functionality using ARKit. I have used this project.
The application works with Collada (.dae) file dynamically which means the the client uploads the Collada file and all of the textures somewhere and the model gets shown accordingly with the help of this solution.
The application works mostly fine but for this one we experience the black part of the model flashing.
As you could see in the pictures most of the floor should be completely black(apart from one small part which is black and white). When we zoom in the model (make the model bigger) the model gets shown with less white spots on the floor(in the first picture almost no white spot but sometimes we can see some). When we zoom out and make the model smaller, we see many white spots on the floor and other places which should be black.[zoomed in the model with less white spot]
Is there any solution for this problem?


The issue you are seeing is known as Z-fighting.
In the given model, the black (bottom/floor) plane and the black/white patterned plane are very close so as the user moves further away the renderer doesn't accurately differentiate which plane should be shown first.
One solution is to set the readsFromDepthBuffer property of the plane's materials to false.
Once you have a reference to the node/plane, you can set the property to its material.
if let geom = node.geometry {
for material in geom.materials {
material.readsFromDepthBuffer = false
}
}

Related

Getting the current visible entities in RealityKit

Currently, RealityKit doesn't have any method that provides the currently visible entities. In SceneKit we do have a method for that particular functionality—nodesInsideFrustum(pointOfView).
Our internal solution is to create a big fake bounding box in front of the camera. We then check intersections between the "frustum" bounding box and each entity's bounding box. That, of course, is a bit cumbersome and inaccurate. I wonder if someone can come up with a better solution who is willing to share it.

You could combine two ARView methods:
ARView.project(position) to get the 2D point in screen space
ARView.bounds.contains(point) to know if it's visible on screen
But it's not enough, you also have to check if the object is behind you:
Entity.position(relativeTo: cameraAnchor) (with cameraAnchor being an AnchorEntity(.camera)) to have the local position
the sign of localPosition.z shows if it's in front or behind the camera

RealityKit: Render an object under a grounding shadow

I have a placement marker (a simple plane with green corners) to visualize detected planes (via ARRaycastResult) in an ARView. This placement marker uses UnlitMaterial with a texture. Everything works fine as long as there is no other object added.
When I add another object, RealityKit also adds grounding shadow (invisible plane) right under the object. It works as a shadow plane and occludes everything behind it - including my placement marker.
Here is a picture of the placement marker (part of which is hidden under the shadow plane):
Is there any way to prevent this clipping? I was lookig for something like rendering order (as ARKit does have one), but have not fond anything in RealityKit yet.
I would like to keep the shadow plane if possible.
Edit: Added official name of the invisible plane (grounding shadow)

Reality kit automatically adds "grounding shadows" if the AnchoringComponent.Target is of type "plane". According to an Apple Engineer this can be aso simulated using a DirectionalLight with a Shadow and placing a plane (with an OcclusionMaterial) under the model. So the grounding shadow is probably also made out of OcclusionMaterial. This also explains why this grounding shadow occludes other objects.
There is an option to disable rendering of these grounding shadows. It can be done by inserting of disableGroundingShadows option into ARView's renderOptions property.
arView.renderOptions.insert(.disableGroundingShadows)
I have not yet found a way to override OcclusionMaterial (if any).

8th Wall tap to place example not showing model replacement

I've replaced the tree.glb model in the ThreeJS placeground example (https://github.com/8thwall/web/tree/master/examples/threejs/placeground), but it's not showing. It works fine when using tree.glb.
To debug, I've also tried replacing it with the jellyfish-model.glb available in the examples, but it also doesn't show when tapping on the floor plane.
Is there something wrong with my code, or with the .glb models I'm replacing tree.gbl with?
const modelFile = 'tree.glb' // 3D model to spawn at tap
to
const modelFile = 'jellyfish-model.glb' // 3D model to spawn at tap
File structure on github: 8thwall-3js-test-github
Ideally, I'd like to replicate what I've done using Unity+Vuforia in this example (which basically places a .png onto a floor plane): https://www.youtube.com/watch?v=poWvXVB4044

I'd start by looking at the scale of the 3d model. The tree model in the link you provided is quite large, so it's being scaled down in size. See https://github.com/8thwall/web/blob/master/examples/threejs/placeground/index.js#L7-L8
Prove to yourself that the model is being loaded by adding a console.log('model added!') type statement into animateIn() (as that is the model loaded handler)
My guess is that your jellyfish-model.glb is there, just very small. Trying adjusting startScale and endScale to larger values and see if that helps.

How can I add a 3d object as a marker on Google Maps like Uber does

I want to add a 3d marker for showing cars on map with rotation like Uber does but I can't find any information on adding 3d objects on Google Maps SDK for iOS.
Would appreciate any help.
Apparently no one is seeing what OP and I are seeing so here's a video of a Uber car turning 90 degrees. Play it frame by frame and you'll notice that it's not a simple image rotation. Either Uber went through the trouble of doing ~360 angles of each vehicles, or it really is a 3D model. Doing 360 images of every car seems foolish to me.

From what I can tell, they are not using 3D objects. They are also not animating between 400 images of a car at a different angle. They're doing a mix of rotating image assets and animating between ~50-70 images of a car at different angles. The illusion is perfect because it really does look like they used 3D car models !
Look at this GIF of a Uber car turning a corner (Dropbox link):
We can clearly see that that the shadow and the car's view angle doesn't update as often as the car's rotation.
Here I overlaid 2 images of the car at different angles, but using the same car image:
We can see that the map is rotated ~5 degrees but the car image is perfectly clear because it hasn't changed, it was simply rotated.

Uber just released a blog post documenting this.
It looks like the vehicles were modeled in 3D software and then image assets depicting different angles were exported for the app. Depending on where the vehicle is on the map and its heading then a different asset is used.

First, they are NOT 3D Objects if that's what you referring to (It's possible to create one though, but waste of time) They are simply 3D image created in Photoshop or Illustrator (Mostly) that have 3D perspective (It's also retina optimized, that's why it looks very clear).
The reason you see that the car is rotated its because the UIImageView that the image is being held into is rotated (using CABasicAnimation mostly) using calculation base off of 3D device position (Same technology use for running apps to track your location etc), which you can use Core Location to retrieve that data.
It's a proccess, but very doable. Good Luck!

Thanks All answers are valid.
if you want you can see the video running, how it works
You can generate sprite sheet ( around 60 ) tiles
How i implement it and tools you need
3d source car model.
blender, animate camera using path animation elipse.
camera rotate around of car from top to bottom view
render 3d marker using sprite generated with blender, for angles use bearing change on location updates.
Your vehicle needs to be rendered to support most screens, so the base size for each tile was 64 px and I was scaling according to the dpi of the screens
Result implementation:
https://twitter.com/ronfravi/status/1133226618024022016?s=09

I believe a pair of marker images, one is the real marker, and another one is a darker blurry shadow can do the trick in a cheaper way. Putting the shadow marker beneath the marker, shifting X & Y axis to an amount where you feel the shadow would be put appropriately, and finally moving them as well as rotating them (on web version, you may need separate rotated images) at the same time should be able to [re]create the illusion.
As #Felix Lapalme already explained it beyond any easier, am not diving any deeper into explaning it.

check out my repo
I used a dea model and turned it according to the heading variable
https://github.com/hilalalhakani/HHMarker

I achieved this in Xamarin by rendering three.js in a webview then sending the image buffer directly to the marker instead of drawing it to the screen. It only took a couple of days, and for my use case it was needed because I want the drivers to be able to change the color and kind of car, but if this is not the functionality you need you're better of just using a sequence of rendered images.

If you want to Rotated your image as the marker. Want to show a moving object you can use .git image. It would be help full for you.
Swift 3
let position = CLLocationCoordinate2D(latitude: 51.5, longitude: -0.127)
//Rotate a marker
let degrees = 90.0
let london = GMSMarker(position: position)
london.title = "London"
//Customize the marker image
london.icon = UIImage(named: "YourGifImageName")
london.groundAnchor = CGPoint(x: 0.5, y: 0.5)
london.rotation = degrees
london.map = mapView
For more info Please check here

Physically based rendering - Pink material iOS

I am using SceneKit to show a 3D model on iOS. I would like to use PBR. At the moment I am using the scene editor not coding to edit my materials. I can set any field to PNGs at the models material (metalness, normals, etc..) except the roughness. When I set it, the whole model turns pink. If I use float value, there is no problem. Do you have any idea, what the problem could be?

Resources