For automated testing reasons, I want to generate low-level touch events programmatically. We are using MonkeyTalk, which abstracts to a higher level in many cases such that only taps seem to work properly when the app does not use gesture recognizers. I am trying to fix the swipe functionality, but am having trouble understanding what exactly I need to send to the touchesMoved method (touchesEnded and touchedBegan are relatively straightforward).
Related
I am working with an app that supports swipe gestures with multi-fingers (2 finger down swipe, etc.), and would like to simulate this in XCUITests. I see that XCUIElement contains a bunch of functions like SwipeUp() and SwipeLeft(), however they all see to be for only single fingers. I don't see any other APIs that look like they would allow simulating a two-finger down swipe, for instance.
Does anyone know of a way to do this?
Unfortunately, there isn’t a solution for this until we’re given a specific call for it; the existing gestures are all synchronous so you can’t simulate individual fingers doing different things at the same time.
There are pinch and rotate functions, but they can’t be made to do what you’re looking for.
How do I receive raw touch events on iOS for creating my own gestures using my own state machine? I'm looking for the analogue of Android's MotionEvent for multi-touch gestures. I need to be able to draw on the background box, but I do not need OS components on this box.
iOS provides a gesture recognizer for defining custom gestures, and it defines UITouch objects that deliver to the application. However, the former is incapable of the implementing the complexity of my system, particularly at the efficiency I need, and the latter does preprocessing, such as to determine the tapCount. My system must itself decide whether a touch is a tap or not.
Additionally, I need to ascertain for myself whether multiple simultaneously-touching fingers form a gesture or not. I can't have iOS interpreting four-finger gestures input (exclusively) on top of my viewing area, though I could live with iOS interpreting five-finger gestures. It's okay for iOS to preempt control from my view if any finger of the gesture starts outside of the view.
The application is a gesturing alternative to the keyboard, so it would ideally be able to operate in the keyboard area, in case this affects the answer.
Is this possible on iOS? What's the approach? I'd like to write a simple program outputting touch events to prove it can be done. Thanks for your help!
UPDATE: I was largely able to answer the question by playing with the Multitouch Visualizer app. Each tap is registered as it occurs, and gestures of 4 fingers or fewer appear to be received without iOS interfering. iOS takes over for some 5 finger gestures, but only if they satisfy certain (unknown) initial characteristics. I don't know how this is done, but I imagine via UITouch events.
I am trying to take screenshots of my iOS app. Before taking a screenshot, I need to get the app to an appropriate state. To get to an appropriate state, a lot of swiping is required.
This would have been fine if I have actual devices, but I don't. So I need to perform swipes on a simulator using a trackpad. I find this very hard, and sometimes I can't swipe properly so the gesture is not recognised.
I thought of using the UI Testing library to programmatically perform swipes. However, my app is actually a game, and random events happen. Writing code to handle these random events would not be worth the time. It would be best if I am in control of the swiping.
I have also thought of adding buttons on the UI of the app. When they are pressed a swipe is simulated. Then I can just click those buttons instead of swiping with my trackpad, which is way easier. However, these buttons will then appear on the screenshot, which I obviously don't want users to see.
Also note that I can't use a tap gesture recogniser as a replacement for the swipe gesture recognisers, because I need to detect swipes in all four directions and do different things depending on the direction.
Essentially, how can I perform a "swipe" more easily on the simulator? It would be great if I can do this by pressing keys on my keyboard, or maybe there is a feature in Xcode that allows me to do this which I am not aware of?
I suggest you automate the UI test.
Recording a test from live actions is a standard Xcode UI test feature. Fastlane is the icing on the cake to automate the capture of screenshots too.
Fastlane has the tools to automatically run a UI test and capture screenshots in all device resolutions. You can even record the actions by recording a UI test and play it back.
Check it out here:
Fastlane Screenshot
Even if you do not wish to use Fastlane, you can record the gestures in a unit test and have it pause.
I'm new to iOS development and instinctively turn towards handling touch events because my code is cross-platform and this maps more closely to other input devices like a mouse. But obviously for multi-touch, it is neater to just use built-in gesture functionality.
However can one do both - track a single touch directly as a kind of cursor, while also supporting pinch, rotate, etc?
Yes, they can both happen at the same time but they can (and do by default) interact.
The gesture has properties such as cancelsTouchesInView for example which influence which events are sent on to the view. See also delaysTouchesBegan which influences which are sent and when.
Is it possible to record set of touch events on iPhone and then playback?
I have searched alot but could not find any answer. if its possible, can anyone explain with an example.
I m not looking for testing purpose. Within my application, instead of creating animation, i just want to record set of events and then want to playback to explain the app flow to the users.
Regards.
Recording is pretty simple. Look at the various "Responding to Touch Events" and "Responding to Motion Events" methods on UIResponder. Just create your own UIView subclass (since UIView inherits from UIResponder) and keep a copy of the events passed into the relevant methods.
Playback is a bit more complicated; there's no way to make UITouch or UIEvent objects (so you can't make a fake event and pass it on to -[UIApplication sendEvent:]). But, there's nothing stopping you from manually parsing an array of Event objects and handling it on your own (aside from it being some kind of ugly code).
There's no built-in macro capability, but you could certainly build that ability into your application. You'll need to do more than just play back events, though. Touches aren't normally visible, but if you're trying to explain how to use your app to the user you'll probably want to have some sort of visual representation for the touches that trigger different responses similar to the way the iOS Simulator uses white dots to represent multiple touches when you hold down the option key.
Assuming that you can solve that problem, two strategies for easily recording user actions come to mind:
Use the Undo Manager: NSUndoManager is already set up to "record" undoable events. If you invest some time into making everything in your app undoable, you could (maybe) perform a set of actions, undo them all to move them to the redo stack, and then save the events in the redo stack as your script.
Use Accessibility: The Accessibility framework sends notifications whenever user interface elements are touched. Your app could use those notifications to create a playback script. You'll still need to write the code to play back the events in the script, though.
You could mirror your application with AirServer and use any screen capture software to make the video.