I'm trying to make a simple collaborative app with RealityKit and the MultipeerConnectivity framework where all users can interact with the same entity. I have a simple RealityComposer .reality file with a cube that jumps up every time the user taps on it.
Setting up the multipeer connection and loading the Entity from the .reality file works fine so far.
But I really don't understand how to manage that only one cube is in the scene for every user in the session.
What I tried:
adding Tap Gesture to arView which creates an ARKit's ARAnchor (which is synced to all peers). After adding the anchor I disable the gesture recognizer.
in the didUpdateAnchor delegate function I am looking for this anchor (this is called on all peers as soon the anchor is synced), creating an AnchorEntity for this anchor, adding it to the scene, loading the Entity from the reality file and adding it as child of the EntityAnchor.
If I do this every peer seems to create a cube and I see two or more cubes in the scene.
How can I load the models (like meshes, sound, etc) on every peer for a synced ARAnchor and tell ARKit/RealityKit that this one and the same virtual object?
Related
I have an AR scene (built as a node with children nodes) that I want the user to walk inside it.
Because it can be large, I added a pause button in the corner so that if the user hits an obstacle in the real world, he'll just need to press-and-hold that button to freeze the scene, move elsewhere in the real world, and when the button will be released the scene would have moved as-is to that new location, retaining the same relative position and rotation inside that scene.
What I did is very simple. When button is touched down I just did:
sceneView.session.pause()
And when the button is Touch Up Inside I ran:
sceneView.session.run(configuration)
This seemed to do the trick.
However, after I clicked and moved once or twice more and moved back to where I was before, my whole AR scene suddenly jumped back to its previous location in the real world. So it did freeze and showed up in the new location for a second or two, but then it jumped back.
I tried to resume the session with several of the available options.
If I ran:
sceneView.session.run(configuration, options: [.resetTracking])
the whole scene would move to the new location, but it would be set to its initial position and rotation as if I was standing at (0,0,0) when I started the app, but now moved to the new location. In other words, it lost the current location and rotation that I was at when I pressed the pause button and just reset the scene.
All the other options that I tried had the same effect as if I didn't add any options.
I should add that I'm not using any anchors (so far) nor any ray casting.
I didn't try different combinations of those options (except removeExistingAnchors and resetTracking together), but I doubt that that would help. Am I wrong?
What am I doing wrong?
Anyone have a solution for my problem?
Or will I need to manually record the current position and rotation when pressing the pause button and then restore those with some position change of the node? I was hoping I could skip having to do it this way.
And if I do need to do it manually, any tips on how to achieve this?
Thanks for your help!
I am developing an app using iOS ARKit which has launched recently. I have a question if map and data that has been added can be extended to multiple sessions.
I am adding UI elements on top of map data, however if I close the app and open it again, i will have to do entire map detection and adding UI elements again. If there is a way to save the map please suggest how this can be achieved?
Nope.
There is no API for accessing the data ARKit uses internally for position/orientation tracking, nor for telling ARKit to save/restore such data itself.
I have an ARKit app which allows the user to add a cube to the scene. This works fine and I can see the cube. But when I push the app to the background and then move the device to another location (by walking to a different room) then ARKit session is unable to determine the correct position of my old nodes.
Is there anyway I can find a workaround this problem so that when the app is resumed from coming to foreground from the background then it still remembers the position of the nodes.
UPDATE: I am looking into saving the lat and long for the user and then somehow converting those lat and long to SCNVector3 to put the node.
You probably won't be able to keep the AR running in the background. Apple does not recommend pausing the session or interrupting it and trying to resume:
Avoid interrupting the AR experience. If the user transitions to another fullscreen UI in your app, the AR view might not be an expected state when coming back.
Use the popover presentation (even on iPhone) for auxiliary view controllers to keep the user in the AR experience while adjusting settings or making a modal selection. In this example, the SettingsViewController and VirtualObjectSelectionViewController classes use popover presentation.
The issue is that once the session gets interrupted, the device stops using it's mechanisms that keep track of AR Nodes as well as your location, might have to set up a mechanism that keeps the app running constantly in the background and running the ARSession through that. You might be able to find projects on github that allow for running in the background. Another issue you might face is Apple's limitations with running apps in the background, which is apparently 3 minutes.
If you're at all interested in restarting the AR Session one you came back in, you can see my answer on this thread.
I'm working on a SceneKit game using Swift, and I'm thinking about the best way to organize the various screens/ states surrounding the actual main game scene. I have a simple prototype which consists of three states:
Splash Screen (choose game type) ->
Game ->
Game over screen (choose to play again or return to splash)
In the prototype I opted to go for UIKit to do the UI elements, rather than a SpriteKit overlay. Although it's just buttons at the moment, which would be easy enough to do in SpriteKit, I might in future want to add sliders (eg an options screen to set control sensitivity etc), text entry etc, and then you end up recreating great chunks of uikit.
So at present, each of the 3 scenes described above has its own UIViewController. The Splash and Game ViewControllers have their own SCNViews and SCNScenes, and the GameOver is a modal that displays over the main Game scene.
This structure isn't really ideal, as it means that the main game SCNScene gets reloaded whenever the viewDidLoad of the GameViewController fires. The main SCNScene is quite detailed so takes a few seconds to load, and with repeated cycles this seems to create memory issues. After 2 cycles of Splash -> Game -> GameOver I get a memory warning, and after the third cycle Xcode loses the connection to the iPhone (which seems, in my experience, to be caused by memory issues).
I would like to have a main GameViewController that loads the most frequently reused assets once, but still be able to segue between the various menu screens.
So what are the options here?
I thought perhaps I could have a main GameViewController which controls the loading and unloading of SceneKit assets and has the sole SCNView. Its viewDidLoad method would only be fired once, when the app first starts. Then, each of the other views would have a very minimal UIViewController, which would be presented as modals over the main GameViewController, with "OverCurrentContext" selected.
I wanted to ask whether this pattern sounds feasible, and how others organise their SceneKit games. I'm still quite new to native iOS development, so it could be that I'm missing some basic game design pattern.
My Experience with the use of only one GameViewController (my current work for MAC OS X: i started a a 'small' 3D game with a MainMenu/start screen, a Hud and 2 or 3 complete different 3D Scenes - this shall be ported then to IOS, too):
In the last week i tried what you asked for, to use only one GameViewController for "all" that stuff - for me it seemed to be the easy and "good" way to do it - but even after lot of hours with all my tricks I've learned the last years i was not able to reload load another (or change) 3D scene (or even not another sprite kit scene hud) after a scenekit scene is loaded in a single ViewController.
Maybe there is difference between MAC OS X and IOS but i even tried this in an iOS version and it has the same behavior.
What i was able to do: You can modify a loaded 3D scene or change data in the sprite kit hud etc., so you are able to use one 3d scene to add or show the things up when they are needed but as far as my tests here showed, you will need one ViewController to show up a complete new different 3d scene or 2d/3d Menus etc. I will post my further experiences as soon as I'm a step further. i hope this helps a little.
I just started to learn SceneKit/SpriteKit to develop games, and while I understood the game part (the SCNScene, in my case), I’m not so sure about the menu/hud part.
What I want to do: create an Inventory Screen to show the items the player has in his bag. This screen will appear when the user taps the bag Icon in the HUD. Furthermore, the Inventory Screen must have a UICollectionView-like list to show each item.
Game Hud:
Inventory Screen:
My question is: should I mix SpriteKit with UIKit to try to accomplish this? Or use just one of them? Should everything be done just with SKScenes that are shown on top of each other (by adding to the OverlaySKScene)?
Currently, I’m presenting the HUD as a SKScene (OverlaySKScene). When the user taps the bag icon, I’m loading a new SKScene (with a bunch of SKSpriteNodes, SKShapeNodes etc) and putting it as the current OverlaySKScene. Now, I’m trying to do the CollectionView part, but don’t know if I should try to implement everything from scratch with SpriteKit or if it’s possible to reuse UICollectionView, with my SKScene as a delegate.
I'm not sure if the way I'm doing is the right way, either.
Any help would be appreciated.
Thanks in advance.