When adding a text MeshResource, with no angle and with a fixed world position, it looks fine from the camera perspective.
However, when the user walks to the other side of the text entity and turns around, it looks mirrored.
I don't want to use the look(at_) API since I only want to rotate it around the Y-axis 180 degrees and when the user passes it again to reset the angle to 0.
First we have to put text in anchor that will stay in the same orientation even when we rotate text. Then add textIsMirrored variable that will handle rotation when changed:
class TextAnchor: Entity,HasAnchoring {
let textEntity = ModelEntity(mesh: .generateText("text"))
var textIsMirrored = false {
willSet {
if newValue != textIsMirrored {
if newValue == true {
textEntity.setOrientation(.init(angle: .pi, axis: [0,1,0]), relativeTo: self)
} else {
textEntity.setOrientation(.init(angle: 0, axis: [0,1,0]), relativeTo: self)
}
}
}
}
required init() {
super.init()
textEntity.scale = [0.01,0.01,0.01]
anchoring = AnchoringComponent(.plane(.horizontal, classification: .any, minimumBounds: [0.3,0.3]))
addChild(textEntity)
}
}
Then in your ViewController you can create anchor that will have Camera as a target so we can track camera position and create out textAnchor:
let cameraAnchor = AnchorEntity(.camera)
let textAnchor = TextAnchor()
For it to work you have to add it as a child of your scene (preferably in viewDidLoad):
arView.scene.addAnchor(cameraAnchor)
arView.scene.addAnchor(textAnchor)
Now in ARSessionDelegate function you can check camera position in relation to your text and rotate it if Z axis is below 0:
func session(_ session: ARSession, didUpdate frame: ARFrame) {
if cameraAnchor.position(relativeTo: textAnchor).z < 0 {
textAnchor.textIsMirrored = true
} else {
textAnchor.textIsMirrored = false
}
}
Related
I'm using a UIScrollView to display an image with various markers on top. The image view has a UILongPressGestureRecognizer that detects long presses. When the long press event is detected, I want to create a new marker at that location.
The problem I'm having is that when I zoom in or out, the location of the gesture recognizer's location(in: view) seems to be off. Here's a snippet of my implementation:
let longPressGestureRecognizer = UILongPressGestureRecognizer(target: self, action: #selector(self.onLongPress(gesture:)))
self.hostingController.view.addGestureRecognizer(longPressGestureRecognizer)
#objc func onLongPress(gesture: UILongPressGestureRecognizer) {
switch gesture.state {
case .began:
guard let view = gesture.view else { break }
let location = gesture.location(in: view)
let pinPointWidth = 32.0
let pinPointHeight = 42.0
let x = location.x - (pinPointWidth / 2)
let y = location.y - pinPointHeight
let finalLocation = CGPoint(x: x, y: y)
self.onLongPress(finalLocation)
default:
break
}
}
Please note that I'm using a UIViewControllerRepresentable that contains a UIViewController with a UIScrollView that is surfaced to my SwiftUI View. Maybe this might be causing it.
Here's the SwiftUI code:
var body: some View {
UIScrollViewWrapper(scaleFactor: $scaleFactor, onLongPress: onInspectionCreated) {
ZStack(alignment: .topLeading) {
Image(uiImage: image)
ForEach(filteredInspections, id: \.syncToken) { inspection in
InspectionMarkerView(
scaleFactor: scaleFactor,
xLocation: CGFloat(inspection.xLocation),
yLocation: CGFloat(inspection.yLocation),
iconName: iconNameForInspection(inspectionMO: inspection),
label: inspection.readableIdPaddedOrNewInspection)
.onTapGesture {
selectedInspection = inspection
}
}
}
}
.clipped()
}
Here's a link to a reproducible example project:
https://github.com/Kukiwon/sample-project-zoom-long-press-location
Here's a recording of the problem:
Link to video
Any help is greatly appreciated!
I don't use SwiftUI, but I've seen some quirky stuff looking at UIHostingController implementations, and it appears a specific quirk is hitting you.
I inset your scroll view by 40-pts and gave the main view a red background to make it a little easier to see what's going on.
First, add a 2-pixel blue line around the border of your sheet_hd image, and scroll all the way to the bottom-left corner. It should look like this:
As you zoom in, keeping the scroll at bottom-left, it will look like this:
So far, so good -- and using a long-press to add a marker works as expected.
However, as soon as we zoom out to less than 1.0 zoom scale:
we can no longer see the bottom edge of the image.
Zooming back in makes it more obvious:
And the long-press location is incorrect.
For further clarification, if we set .clipsToBounds = false on the scroll view, and set .alpha = 0.5 on the image view, we see this:
We can drag the view up to see the bottom edge, but as soon as we release the touch is bounces back below the frame of the scroll view.
What should fix this is to use this extension:
// extension to remove safe area from UIHostingController
// source: https://stackoverflow.com/a/70339424/6257435
extension UIHostingController {
convenience public init(rootView: Content, ignoreSafeArea: Bool) {
self.init(rootView: rootView)
if ignoreSafeArea {
disableSafeArea()
}
}
func disableSafeArea() {
guard let viewClass = object_getClass(view) else { return }
let viewSubclassName = String(cString: class_getName(viewClass)).appending("_IgnoreSafeArea")
if let viewSubclass = NSClassFromString(viewSubclassName) {
object_setClass(view, viewSubclass)
}
else {
guard let viewClassNameUtf8 = (viewSubclassName as NSString).utf8String else { return }
guard let viewSubclass = objc_allocateClassPair(viewClass, viewClassNameUtf8, 0) else { return }
if let method = class_getInstanceMethod(UIView.self, #selector(getter: UIView.safeAreaInsets)) {
let safeAreaInsets: #convention(block) (AnyObject) -> UIEdgeInsets = { _ in
return .zero
}
class_addMethod(viewSubclass, #selector(getter: UIView.safeAreaInsets), imp_implementationWithBlock(safeAreaInsets), method_getTypeEncoding(method))
}
objc_registerClassPair(viewSubclass)
object_setClass(view, viewSubclass)
}
}
}
Then, in viewDidLoad() in your UIScrollViewViewController:
override func viewDidLoad() {
super.viewDidLoad()
self.view.addSubview(self.scrollView)
self.pinEdges(of: self.scrollView, to: self.view)
// add this line
self.hostingController.disableSafeArea()
self.hostingController.willMove(toParent: self)
self.scrollView.addSubview(self.hostingController.view)
self.pinEdges(of: self.hostingController.view, to: self.scrollView)
self.hostingController.didMove(toParent: self)
self.hostingController.view.alpha = 0
}
Quick testing (obviously, you'll want to thoroughly test it) seems good... we can scroll all the way to the bottom, and long-press location is back where it should be.
I have an app that I am trying to update from SceneKit to RealityKit, and one of the features that I am having a hard time replicating in RealityKit is making an entity constantly look at the camera. In SceneKit, this was accomplished by adding the following billboard constraints to the node:
let billboardConstraint = SCNBillboardConstraint()
billboardConstraint.freeAxes = [.X, .Y]
startLabelNode.constraints = [billboardConstraint]
Which would allow the startLabelNode to freely rotate so that it was constantly facing the camera without the startLabelNode changing its position.
However, I can't seem to figure out a way to do this with RealityKit. I have tried the "lookat" method, which doesn't seem to offer the ability to constantly face the camera. Here is a short sample app where I have tried to implement a version of this in RealityKit, but it doesn't offer the ability to have the entity constantly face the camera like it did in SceneKit:
import UIKit
import RealityKit
import ARKit
class ViewController: UIViewController, ARSessionDelegate {
#IBOutlet weak var arView: ARView!
override func viewDidLoad() {
super.viewDidLoad()
arView.session.delegate = self
arView.environment.sceneUnderstanding.options = []
arView.debugOptions.insert(.showSceneUnderstanding) // Display a debug visualization of the mesh.
arView.renderOptions = [.disablePersonOcclusion, .disableDepthOfField, .disableMotionBlur] // For performance, disable render options that are not required for this app.
arView.automaticallyConfigureSession = false
let configuration = ARWorldTrackingConfiguration()
if ARWorldTrackingConfiguration.supportsSceneReconstruction(.mesh) {
configuration.sceneReconstruction = .mesh
} else {
print("Mesh Classification not available on this device")
configuration.worldAlignment = .gravity
configuration.planeDetection = [.horizontal, .vertical]
}
configuration.environmentTexturing = .automatic
arView.session.run(configuration)
UIApplication.shared.isIdleTimerDisabled = true // Prevent the screen from being dimmed to avoid interrupting the AR experience.
}
#IBAction func buttonPressed(_ sender: Any) {
let screenWidth = arView.bounds.width
let screenHeight = arView.bounds.height
let centerOfScreen = CGPoint(x: (screenWidth / 2), y: (screenHeight / 2))
if let raycastResult = arView.raycast(from: centerOfScreen, allowing: .estimatedPlane, alignment: .any).first
{
addStartLabel(at: raycastResult.worldTransform)
}
}
func addStartLabel(at result: simd_float4x4) {
let resultAnchor = AnchorEntity(world: result)
resultAnchor.addChild(clickToStartLabel())
arView.scene.addAnchor(resultAnchor)
}
func clickToStartLabel() -> ModelEntity {
let text = "Click to Start Here"
let textMesh = MeshResource.generateText(text, extrusionDepth: 0.001, font: UIFont.boldSystemFont(ofSize: 0.01))
let textMaterial = UnlitMaterial(color: .black)
let textModelEntity = ModelEntity(mesh: textMesh, materials: [textMaterial])
textModelEntity.generateCollisionShapes(recursive: true)
textModelEntity.position.x -= textMesh.width / 2
textModelEntity.position.y -= textMesh.height / 2
let planeMesh = MeshResource.generatePlane(width: (textMesh.width + 0.01), height: (textMesh.height + 0.01))
let planeMaterial = UnlitMaterial(color: .white)
let planeModelEntity = ModelEntity(mesh: planeMesh, materials: [planeMaterial])
planeModelEntity.generateCollisionShapes(recursive:true)
// move the plane up to make it sit on the anchor instead of in the middle of the anchor
planeModelEntity.position.y += planeMesh.height / 2
planeModelEntity.addChild(textModelEntity)
// This does not always keep the planeModelEntity facing the camera
planeModelEntity.look(at: arView.cameraTransform.translation, from: planeModelEntity.position, relativeTo: nil)
return planeModelEntity
}
}
extension MeshResource {
var width: Float
{
return (bounds.max.x - bounds.min.x)
}
var height: Float
{
return (bounds.max.y - bounds.min.y)
}
}
Is the lookat function the best way to get the missing feature working in RealityKit or is there a better way to have a Entity constantly face the camera?
k - I haven't messed with RK much, but assuming entity is a scenekit node? - then set constraints on it and it will be forced to face 'targetNode' at all times. Provide that works the way you want it to, then you may have to experiment with how the node is initially created IE what direction it is facing.
func setTarget()
{
node.constraints = []
let vConstraint = SCNLookAtConstraint(target: targetNode)
vConstraint.isGimbalLockEnabled = true
node.constraints = [vConstraint]
}
I was able to figure out an answer to my question. Adding the following block of code allowed the entity to constantly look at the camera:
func session(_ session: ARSession, didUpdate frame: ARFrame) {
planeModelEntity.look(at: arView.cameraTransform.translation, from: planeModelEntity.position(relativeTo: nil), relativeTo: nil)
}
I am currently trying to get a vertical flip animation for my card in my project's subview. I am using some dependencies (Shuffle Cocoapod), and I tried everything to attempt to get a flip animation from right to left of my card. It seems like I can change the background and other simple stuff of each single card, but nothing happens when trying to animate it. The goal would be to "show the back of the card" with another word in it (basically each card has a "front word" and a "back word". What am I getting wrong?
import UIKit
import Shuffle
class CardsViewController: UIViewController, SwipeCardStackDataSource, SwipeCardStackDelegate {
let cardStack = SwipeCardStack()
var actualIndex : Int = 0
var showingBack = false
override func viewDidLoad() {
super.viewDidLoad()
view.addSubview(cardStack)
cardStack.frame.size.height = 512
cardStack.frame.size.width = 384
cardStack.center.y = CGFloat(cardStack.frame.size.height/2)
cardStack.center.x = CGFloat(cardStack.frame.size.width/2)
cardStack.center = view.center
cardStack.dataSource = self
}
func card(fromImage word: String) -> SwipeCard {
let card = SwipeCard()
card.swipeDirections = [.left, .right]
for direction in card.swipeDirections {
card.setOverlay(CardOverlay(direction: direction), forDirection: direction)
}
card.content = CardContentView(withWord: word)
return card
}
func cardStack(_ cardStack: SwipeCardStack, cardForIndexAt index: Int) -> SwipeCard {
let theCard = card(fromImage: cardWords[actualIndex].word)
actualIndex = actualIndex + 1
return theCard
}
func numberOfCards(in cardStack: SwipeCardStack) -> Int {
return cardWords.count
}
let cardWords = [
DataOfWord(word: "comme", translation: "as"),
DataOfWord(word: "je", translation: "I"),
DataOfWord(word: "son", translation: "his"),
DataOfWord(word: "que", translation: "that")]
}
You can create a flip type of animation with the UIView's transition(with:duration:options:animations:completion:) method
Here's some code from one of my own apps where I flip the gameboard to display the level completion score on the "other side":
UIView.transition(with: gameBoardCollectionView, duration: 0.75, options: [.transitionFlipFromRight]) {
self.loadLevelEndArcadeView()
} completion: { (complete) in
self.animateLevelEndArcadeLabelViews()
}
So, inside the transition code block, you just load the views you want and when the animation completes, you'll be displaying something that represents the back side of a view. There are also other transition animations you can call upon if you want.
I'm trying to detect when the camera is facing my object that I've placed in ARSKView. Here's the code:
override func update(_ currentTime: TimeInterval) {
// Called before each frame is rendered
guard let sceneView = self.view as? ARSKView else {
return
}
if let currentFrame = sceneView.session.currentFrame {
//let cameraZ = currentFrame.camera.transform.columns.3.z
for anchor in currentFrame.anchors {
if let spriteNode = sceneView.node(for: anchor), spriteNode.name == "token", intersects(spriteNode) {
// token is within the camera view
let distance = simd_distance(anchor.transform.columns.3,
currentFrame.camera.transform.columns.3)
//print("DISTANCE BETWEEN CAMERA AND TOKEN: \(distance)")
if distance <= captureDistance {
// token is within the camera view and within capture distance
print("token is within the camera view and within capture distance")
}
}
}
}
}
The problem is that the intersects method is returning true both when the object is directly in front of the camera, as well as directly behind you. How can I update this code so it only detects when the spriteNode is in the current camera viewfinder? I'm using SpriteKit by the way, not SceneKit.
Here's the code I'm using to actually create the anchor:
self.captureDistance = captureDistance
guard let sceneView = self.view as? ARSKView else {
return
}
// Create anchor using the camera's current position
if sceneView.session.currentFrame != nil {
print("token dropped at \(distance) meters and bearing: \(bearing)")
// Add a new anchor to the session
let transform = getTransformGiven(bearing: bearing, distance: distance)
let anchor = ARAnchor(transform: transform)
sceneView.session.add(anchor: anchor)
}
func getTransformGiven(bearing: Float, distance: Float) -> matrix_float4x4 {
let origin = MatrixHelper.translate(x: 0, y: 0, z: Float(distance * -1))
let bearingTransform = MatrixHelper.rotateMatrixAroundY(degrees: bearing * -1, matrix: origin)
return bearingTransform
}
I have spent a while looking at this, and have come to the conclusion that trying to get the distance between the currentFrame.camera and the anchor doesn't work simply because it returns similar values irregardless of whether the anchor is infront of, or behind the camera. By this I mean that if we assume that our anchor is at point x, and we move forwards 1meter or backwards 1 meter, the distance from the camera and the anchor is still 1 meter.
As such after some experimenting I believe we need to look at the following variables and functions to help us detect whether our SKNode is infront of the camera:
(a) The zPosition of the SpriteNode which refers to:
The z-order of the node (used for ordering). Negative z is "into" the screen, Positive z is "out" of the screen
(b) open func intersects(_ node: SKNode) -> Bool which:
Returns true if the bounds of this node intersects with the
transformed bounds of the other node, otherwise false.
As such the following seems to do exactly what you need:
override func update(_ currentTime: TimeInterval) {
//1. Get The Current ARSKView & Current Frame
guard let sceneView = self.view as? ARSKView, let currentFrame = sceneView.session.currentFrame else { return }
//3. Iterate Through Our Anchors & Check For Our Token Node
for anchor in currentFrame.anchors {
if let spriteNode = sceneView.node(for: anchor), spriteNode.name == "token"{
/*
If The ZPosition Of The SpriteNode Is Negative It Can Be Seen As Into The Screen Whereas Positive Is Out Of The Screen
However We Also Need To Know Whether The Actual Frostrum (SKScene) Intersects Our Object
If Our ZPosition Is Negative & The SKScene Doesnt Intersect Our Node Then We Can Assume It Isnt Visible
*/
if spriteNode.zPosition <= 0 && intersects(spriteNode){
print("Infront Of Camera")
}else{
print("Not InFront Of Camera")
}
}
}
}
Hope it helps...
You can also use this function to check the camera's position :-
- (void)session:(ARSession *)session didUpdateFrame:(ARFrame *)frame; {
simd_float4x4 transform = session.currentFrame.camera.transform;
SCNVector3 position = SCNVector3Make(transform.columns[3].x,
transform.columns[3].y,
transform.columns[3].z);
// Call any function to check the Position.
}
I would give you a clue. Check the ZPosition like this.
if let spriteNode = sceneView.node(for: anchor),
spriteNode.name == "token",
intersects(spriteNode) && spriteNode.zPosition < 0 {....}
Creating a 2d game, and want to add moving platforms where I can control the pattern and rotation of the platform.
Example:
I want this platform to move in a clockwise motion, also when it reaches the top and bottom I want it to rotate accordingly as if it is facing the direction it is going (so the platform would essentially rotate 180 degrees as it arches at the top and bottom).
I can't use SKActions because I need the physics to work properly.
My idea is that I can use an Agent with behaviors and goals to do this. Not sure if I will need path finding as well.
I'm researching how to use these features, but the documentation and lack of tutorials is hard to decipher. I'm hoping someone could save my some time of trial and error by providing an example of how this would work.
Thanks in advance!
Using SKActions or adjusting the position manually will be sucky to get working the way you want, BUT it's always worth a shot to give it a go, since it would take 2 minutes to mock it up and see...
I'd suggest doing something like a Path class that sends velocity commands to the platform every frame ...
Actually, this could be a good exercise in learning the state machine..
MOVE RIGHT STATE: if X position > starting position X + 200, enter state "move down"
MOVE DOWN STATE: if Y position < starting position Y - 200, enter state "move left"
MOVE LEFT STATE: if X position < starting position X, enter state "move up"
MOVE UP STATE: if Y position > starting position Y, enter state "move right"
.. there are ways you could figure out to give it more curvature instead of just straight angle (when changing direction)
Otherwise you would have to translate that into a class / struct / component and give each platform it's own instance of it.
///
Another option is to take the physics out of the equation, and create a playerIsOnPlatform property... then you manually adjust the position of the player each frame... (or maybe a SKConstraint)
This would then require more code on jumping and such, and turns things to spaghetti pretty quickly (last time I tried it)
But, I was able to successfully clone this, using proper hit detection going that route:
https://www.youtube.com/watch?v=cnhlFZeIR7Q
UPDATE:
Here is a working project:
https://github.com/fluidityt/exo2/tree/master
Boilerplate swift stuff:
import SpriteKit
import GameplayKit
// Workaround to not having any references to the scene off top my head..
// Simply add `gScene = self` in your didMoveToViews... or add a base scene and `super.didMoveToView()`
var gScene = GameScene()
class GameScene: SKScene {
override func didMove(to view: SKView) {
gScene = self
}
var entities = [GKEntity]()
var graphs = [String : GKGraph]()
private var lastUpdateTime : TimeInterval = 0
func gkUpdate(_ currentTime: TimeInterval) -> TimeInterval {
if (self.lastUpdateTime == 0) {
self.lastUpdateTime = currentTime
}
let dt = currentTime - self.lastUpdateTime
for entity in self.entities {
entity.update(deltaTime: dt)
}
return currentTime
}
override func update(_ currentTime: TimeInterval) {
self.lastUpdateTime = gkUpdate(currentTime)
}
}
Here is the very basic component:
class Platforms_BoxPathComponent: GKComponent {
private enum MovingDirection: String { case up, down, left, right }
private var node: SKSpriteNode!
private var state: MovingDirection = .right
private lazy var startingPos: CGPoint = { self.node.position }()
#GKInspectable var startingDirection: String = "down"
#GKInspectable var uniqueName: String = "platform"
#GKInspectable var xPathSize: CGFloat = 400
#GKInspectable var yPathSize: CGFloat = 400
// Moves in clockwise:
private var isTooFarRight: Bool { return self.node.position.x > (self.startingPos.x + self.xPathSize) }
private var isTooFarDown: Bool { return self.node.position.y < (self.startingPos.y - self.yPathSize) }
private var isTooFarLeft: Bool { return self.node.position.x < self.startingPos.x }
private var isTooFarUp: Bool { return self.node.position.y > self.startingPos.y }
override func didAddToEntity() {
print("adding component")
// can't add node here because nodes aren't part of scene yet :(
// possibly do a thread?
}
override func update(deltaTime seconds: TimeInterval) {
if node == nil {
node = gScene.childNode(withName: uniqueName) as! SKSpriteNode
// for some reason this is glitching out and needed to be redeclared..
// showing 0 despite clearly not being 0, both here and in the SKS editor:
xPathSize = 300
}
let amount: CGFloat = 2 // Amount to move platform (could also be for for velocity)
// Moves in clockwise:
switch state {
case .up:
if isTooFarUp {
state = .right
fallthrough
} else { node.position.y += amount }
case .right:
if isTooFarRight {
state = .down
fallthrough
} else { node.position.x += amount }
case .down:
if isTooFarDown {
state = .left
fallthrough
} else { node.position.y -= amount }
case .left:
if isTooFarLeft {
state = .up
fallthrough
} else { node.position.x -= amount }
default:
print("this is not really a default, just a restarting of the loop :)")
node.position.y += amount
}
}
}
And here is what you do in the sks editor: