I'm beginning to experiment with EarlGrey a little bit, having done UI Testing using the XCUITest for some months now. I'm running into the classic problem of being unable to dismiss system alerts, which is strange as it looks as though Google implemented a matcher for system alerts called grey_systemAlertViewShown(). I'm trying to detect system alerts using GREYCondition. Here's what I've tried:
- (void)waitForAndDismissSystemAlertForSeconds:(NSInteger)seconds {
GREYCondition *interactableCondition = [GREYCondition conditionWithName:#"isInteractable" block:^BOOL{
// Fails if element is not interactable
NSError *error;
[[EarlGrey selectElementWithMatcher:grey_systemAlertViewShown()] assertWithMatcher:grey_interactable() error:&error];
if (error) {
return NO;
} else {
NSError *allowButtonError;
[[EarlGrey selectElementWithMatcher:grey_accessibilityLabel(#"Allow")] assertWithMatcher:grey_notNil() error:&allowButtonError];
if (!allowButtonError) {
[[EarlGrey selectElementWithMatcher:grey_accessibilityLabel(#"Allow")] performAction:grey_tap()];
}
return YES;
}];
[interactableCondition waitWithTimeout:seconds];
}
I've also tried using addUIInterruptionMonitorWithDescription as described here (but using EarlGrey code to do basically what I am doing above in the interruption monitors): Xcode 7 UI Testing: how to dismiss a series of system alerts in code
Neither approach works. Breakpoints don't fire for the non-error case in my GREYCondition, and the interruption monitor doesn't dismiss my alert either.
Does anybody know if EarlGrey supports dismissing system alerts?
As the docs for grey_systemAlertViewShown indicate, grey_systemAlertViewShown merely checks to see if system alert views are shown. A better usage of the API would be to assert that system alert is not shown (maybe because the test app has mocked out the code that causes system alerts).
Code that taps a button that requests causes system alert to be shown (for ex: requests user's geo location) comes here...
// Assert that in the test app system alert view is not shown because we have mocked out the part of code that requests user location.
[[EarlGrey selectElementWithMatcher:grey_anything()] assertWithMatcher:grey_not(grey_systemAlertViewShown())];
As of this writing system alert views cannot be dismissed by EarlGrey. Alertviews that are launched by the app can be dismissed. The FAQ has a question that indicates that EarlGrey tests will fail if modal dialogs are present.
The best way we found to get around this was to include a launch argument for testing, in which we wouldn't register the app for notifications.
Something like:
if [[[NSProcessInfo processInfo] arguments] containsObject:argument] {
return;
}
before you call
[[UIApplication sharedApplication] registerUserNotificationSettings:settings];
[[UIApplication sharedApplication] registerForRemoteNotifications];
This way, the "Do you want to allow push notifications..." alert won't show.
It is possible to grant all the required permissions using AppleSimulatorUtils util.
This approach eliminates the need to dismiss alerts and saves time.
Install an util by entering next commands in the Terminal app
brew tap wix/brew
brew install applesimutils
And grant permission(s) with
applesimutils --byId <simulator UDID> --bundle <bundle identifier> --setPermissions "notifications=YES"
For more info and examples please refer to
https://github.com/wix/AppleSimulatorUtils
EarlGreyImpl.invoked(fromFile: #file, lineNumber: #line).selectElement(with: grey_text("Click")).perform(grey_tap())
//use above code it might work for your problem
Related
For people wanting to reply quickly without reading the post: I am not hitting any memory limits. Read the whole post for details.
My WatchKit extension cannot properly function without the user first being "onboarded" through the phone app. Onboarding is where the user must accept the permissions that we require, so it's very crucial.
On my WatchKit extension, I wanted to display a simple warning for users who had not finished onboarding within our phone app yet.
As such, I thought I'd get the status of onboarding from the phone in two ways:
When the user opens the app/the app is activated (I use the willActivate method to detect this)
When the app finishes onboarding it sends a message to the watch of its completion (if the extension is reachable, of course)
Both of these combined would ensure that the status of onboarding is always kept in sync with the watch.
I wrote the first possibility in, utilizing reply handlers to exchange the information. It worked just fine, without any troubles. The warning telling the user to complete disappears, the extension does not crash, and all is well.
I then wrote in the second possibility, of the extension being reachable when the user finishes onboarding (with the phone then directly sending the companion the new status of onboarding). My extension crashes when it receives this message, and I am stuck with this odd error.
Program ended with exit code: 0
My extension does not even get a chance to handle the new onboarding status, the extension just quits and the above error is given to me.
I am not hitting any sort of memory limit. I have read the technical Q&A which describes what a memory usage limit error looks like, and I don't receive any sort of output like that whatsoever. As well, before the extension should receive the message, this is what my memory consumption looks like.
I have monitored the memory consumption of the extension right after finishing onboarding, and I see not a single spike indicating that I've gone over any kind of threshold.
I have tried going line by line over the code which manages the onboarding error, and I cannot find a single reason that it would crash with this error. Especially since the reply handler method of fetching the onboarding status works so reliably.
Here is the code of how I'm sending the message to the watch.
- (void)sendOnboardingStatusToWatch {
if(self.connected){
[self.session sendMessage:#{
LMAppleWatchCommunicationKey: LMAppleWatchCommunicationKeyOnboardingComplete,
LMAppleWatchCommunicationKeyOnboardingComplete: #(LMMusicPlayer.onboardingComplete)
}
replyHandler:nil
errorHandler:^(NSError * _Nonnull error) {
NSLog(#"Error sending onboarding status: %#", error);
}];
}
}
(All LMAppleWatchCommunicationKeys are simply #define'd keys with exactly their key as the string value. ie. #define LMAppleWatchCommunicationKey #"LMAppleWatchCommunicationKey")
Even though it's never called by the extension, here is the exact receiving code of the extension which handles the incoming data, if it helps.
- (void)session:(WCSession *)session didReceiveMessage:(NSDictionary<NSString *, id> *)message {
NSString *key = [message objectForKey:LMAppleWatchCommunicationKey];
if([key isEqualToString:LMAppleWatchCommunicationKeyOnboardingComplete]){
BOOL newOnboardingStatus = [message objectForKey:LMAppleWatchCommunicationKeyOnboardingComplete];
[[NSUserDefaults standardUserDefaults] setBool:newOnboardingStatus
forKey:LMAppleWatchCommunicationKeyOnboardingComplete];
dispatch_async(dispatch_get_main_queue(), ^{
for(id<LMWCompanionBridgeDelegate> delegate in self.delegates){
if([delegate respondsToSelector:#selector(onboardingCompleteStatusChanged:)]){
[delegate onboardingCompleteStatusChanged:newOnboardingStatus];
}
}
});
}
}
Before including this onboarding-related code, my WatchKit extension was tested by over 100 people, without any troubles. I am using the exact same custom error dialogue that I was using before, just with a different string. I cannot for the life of me figure out what is causing this crash, and the ambiguity of it has given me very little to work with.
Any help would be greatly appreciated. Thank you very much for taking your time to read my post.
Edit: I just tried creating a symbolic breakpoint for exit(), which is never hit. If I call exit() myself, it calls the breakpoint, so I know the breakpoint itself is working.
I am writing UI test cases using the new Xcode 7 UI Testing feature. At some point of my app, I ask the user for permission of camera access and push notification. So two iOS popups will show up: "MyApp Would Like to Access the Camera" popup and "MyApp Would Like to Send You Notifications" popup. I'd like my test to dismiss both popups.
UI recording generated the following code for me:
[app.alerts[#"cameraAccessTitle"].collectionViews.buttons[#"OK"] tap];
However, [app.alerts[#"cameraAccessTitle"] exists] resolves to false, and the code above generates an error: Assertion Failure: UI Testing Failure - Failure getting refresh snapshot Error Domain=XCTestManagerErrorDomain Code=13 "Error copying attributes -25202".
So what's the best way of dismissing a stack of system alerts in test? The system popups interrupt my app flow and fail my normal UI test cases immediately. In fact, any recommendations regarding how I can bypass the system alerts so I can resume testing the usual flow are appreciated.
This question might be related to this SO post which also doesn't have an answer: Xcode7 | Xcode UI Tests | How to handle location service alert?
Thanks in advance.
Xcode 7.1
Xcode 7.1 has finally fixed the issue with system alerts. There are, however, two small gotchas.
First, you need to set up a "UI Interuption Handler" before presenting the alert. This is our way of telling the framework how to handle an alert when it appears.
Second, after presenting the alert you must interact with the interface. Simply tapping the app works just fine, but is required.
addUIInterruptionMonitorWithDescription("Location Dialog") { (alert) -> Bool in
alert.buttons["Allow"].tap()
return true
}
app.buttons["Request Location"].tap()
app.tap() // need to interact with the app for the handler to fire
The "Location Dialog" is just a string to help the developer identify which handler was accessed, it is not specific to the type of alert.
I believe that returning true from the handler marks it as "complete", which means it won't be called again. For your situation I would try returning false so the second alert will trigger the handler again.
Xcode 7.0
The following will dismiss a single "system alert" in Xcode 7 Beta 6:
let app = XCUIApplication()
app.launch()
// trigger location permission dialog
app.alerts.element.collectionViews.buttons["Allow"].tap()
Beta 6 introduced a slew of fixes for UI Testing and I believe this was one of them.
Also note that I am calling -element directly on -alerts. Calling -element on an XCUIElementQuery forces the framework to choose the "one and only" matching element on the screen. This works great for alerts where you can only have one visible at a time. However, if you try this for a label and have two labels the framework will raise an exception.
Objective - C
-(void) registerHandlerforDescription: (NSString*) description {
[self addUIInterruptionMonitorWithDescription:description handler:^BOOL(XCUIElement * _Nonnull interruptingElement) {
XCUIElement *element = interruptingElement;
XCUIElement *allow = element.buttons[#"Allow"];
XCUIElement *ok = element.buttons[#"OK"];
if ([ok exists]) {
[ok tap];
return YES;
}
if ([allow exists]) {
[allow tap];
return YES;
}
return NO;
}];
}
-(void)setUp {
[super setUp];
self.continueAfterFailure = NO;
self.app = [[XCUIApplication alloc] init];
[self.app launch];
[self registerHandlerforDescription:#"“MyApp” would like to make data available to nearby Bluetooth devices even when you're not using app."];
[self registerHandlerforDescription:#"“MyApp” Would Like to Access Your Photos"];
[self registerHandlerforDescription:#"“MyApp” Would Like to Access the Camera"];
}
Swift
addUIInterruptionMonitorWithDescription("Description") { (alert) -> Bool in
alert.buttons["Allow"].tap()
alert.buttons["OK"].tap()
return true
}
Gosh.
It always taps on "Don't Allow" even though I deliberately say tap on "Allow"
At least
if app.alerts.element.collectionViews.buttons["Allow"].exists {
app.tap()
}
allows me to move on and do other tests.
For the ones who are looking for specific descriptions for specific system dialogs (like i did) there is none :) the string is just for testers tracking purposes. Related apple document link : https://developer.apple.com/documentation/xctest/xctestcase/1496273-adduiinterruptionmonitor
Update : xcode 9.2
The method is sometimes triggered sometimes not. Best workaround for me is when i know there will be a system alert, i add :
sleep(2)
app.tap()
and system alert is gone
God! I hate how XCTest has the worst time dealing with UIView Alerts. I have an app where I get 2 alerts the first one wants me to select "Allow" to enable locations services for App permissions, then on a splash page the user has to press a UIButton called "Turn on location" and finally there is a notification sms alert in a UIViewAlert and the user has to select "OK". The problem we were having was not being able to interact with the system Alerts, but also a race condition where behavior and its appearance on screen was untimely. It seems that if you use the alert.element.buttons["whateverText"].tap the logic of XCTest is to keep pressing until the time of the test runs out. So basically keep pressing anything on the screen until all the system alerts are clear of view.
This is a hack but this is what worked for me.
func testGetPastTheStupidAlerts() {
let app = XCUIApplication()
app.launch()
if app.alerts.element.collectionViews.buttons["Allow"].exists {
app.tap()
}
app.buttons["TURN ON MY LOCATION"].tap()
}
The string "Allow" is completely ignored and the logic to app.tap() is called evreytime an alert is in view and finally the button I wanted to reach ["Turn On Location"] is accessible and the test pass
~Totally confused, thanks Apple.
The only thing I found that reliably fixed this was to set up two separate tests to handle the alerts. In the first test, I call app.tap() and do nothing else. In the second test, I call app.tap() again and then do the real work.
On xcode 9.1, alerts are only being handled if the test device has iOS 11. Doesn't work on older iOS versions e.g 10.3 etc. Reference: https://forums.developer.apple.com/thread/86989
To handle alerts use this:
//Use this before the alerts appear. I am doing it before app.launch()
let allowButtonPredicate = NSPredicate(format: "label == 'Always Allow' || label == 'Allow'")
//1st alert
_ = addUIInterruptionMonitor(withDescription: "Allow to access your location?") { (alert) -> Bool in
let alwaysAllowButton = alert.buttons.matching(allowButtonPredicate).element.firstMatch
if alwaysAllowButton.exists {
alwaysAllowButton.tap()
return true
}
return false
}
//Copy paste if there are more than one alerts to handle in the app
#Joe Masilotti's answer is correct and thanks for that, it helped me a lot :)
I would just like to point out the one thing, and that is the UIInterruptionMonitor catches all system alerts presented in series TOGETHER, so that the action you apply in the completion handler gets applied to every alert ("Don't allow" or "OK"). If you want to handle alert actions differently, you have to check, inside the completion handler, which alert is currently presented e.g. by checking its static text, and then the action will be applied only on that alert.
Here's small code snippet for applying the "Don't allow" action on the second alert, in series of three alerts, and "OK" action on the remaining two:
addUIInterruptionMonitor(withDescription: "Access to sound recording") { (alert) -> Bool in
if alert.staticTexts["MyApp would like to use your microphone for recording your sound."].exists {
alert.buttons["Don’t Allow"].tap()
} else {
alert.buttons["OK"].tap()
}
return true
}
app.tap()
This is an old question but there is now another way to handle these alerts.
The system alert isn't accessibly from the app context of the app you are launched in, however you can access the app context anyway. Look at this simple example:
func testLoginHappyPath() {
let app = XCUIApplication()
app.textFields["Username"].typeText["Billy"]
app.secureTextFields["Password"].typeText["hunter2"]
app.buttons["Log In"].tap()
}
In a vacuum with a simulator already launched and permissions already granted or denied, this will work. But if we put it in a CI pipeline where it gets a brand new simulator, all of the sudden it won't be able to find that Username field because there's a notification alert popping up.
So now there's 3 choices on how to handle that:
Implicitly
There's already a default system alert interrupt handler. So in theory, simply trying to typeText on that first field should check for an interrupting event and handle it in the affirmative.
If everything works as designed, you won't have to write any code but you'll see an interruption logged and handled in the log, and your test will take a couple seconds more.
Explicitly via interruptionmonitor
I won't rewrite the previous work on this, but this is where you explicitly set up an interruptionmonitor to handle the specific alert being popped up - or whatever alerts you expect to happen.
This is useful if the built-in handler doesn't do what you want - or doesn't work at all.
Explicitly via XCUITest framework
In xCode 9.0 and above, you can switch between app contexts fluidly by simply defining multiple XCUIApplication() instances. Then you can locate the field you need via familiar methods. So to do this explicitly would look like the following:
func testLoginHappyPath() {
let app = XCUIApplication()
let springboardApp = XCUIApplication(bundleidentifier: "com.apple.springboard")
if springboardApp.alerts[""FunHappyApp" would like permission to own your soul."].exists {
springboardApp.alerts.buttons["Allow"].tap()
}
app.textFields["Username"].typeText["Billy"]
app.secureTextFields["Password"].typeText["hunter2"]
app.buttons["Log In"].tap()
}
Sounds like the approach to implementing camera access and notifications are threaded as you say, but not physically managed and left to chance when and how they are displayed.
I suspect one is triggered by the other and when it is programatically clicked it wipes out the other one as well (which Apple would probably never allow)
Think of it you're asking for a users permission then making the decision on their behalf? Why? Because you can't get your code to work maybe.
How to fix - trace where these two components are triggering the pop up dialogues - where are they being called?, rewrite to trigger just one, send an NSNotification when one dialogue has been completed to trigger and display the remaining one.
I would seriously discourage the approach of programatically clicking dialogue buttons meant for the user.
One question and one issue:
I have the following code:
- (void) registerForLocalCalendarChanges
{
[[NSNotificationCenter defaultCenter] addObserver:self selector:#selector(localCalendarStoreChanged) name:EKEventStoreChangedNotification object:store ];
}
- (void) localCalendarStoreChanged
{
// This gets call when an event in store changes
// you have to go through the calendar to look for changes
[self getCalendarEvents];
}
These methods are in a class/object called CalendarEventReporter which contains the method getCalendarEvents (in the callback).
Two things:
1) If the app is in the background the callback does not run. Is there a way to make it do that?
2) When I bring the app back into the foreground (after having changed the calendar on the device) the app crashes without any error message in the debug window or on the device. My guess is that the CalendarEventReporter object that contains the callback is being garbage-collected. Is that possible? Any other thoughts on what might be causing the crash? Or how to see any error messages?
1) In order for the app to run in the background you should be using one of the modes mentioned in the "Background Execution and Multitasking section here:
uses location services
records or plays audio
provides VOIP
services
background refresh
connection to external devices
like through BLE
If you are not using any of the above, it is not possible to get asynchronous events in the background.
2) In order to see the crash logs/call stack place an exception breakpoint or look into the "Device Logs" section here: Window->Organizer->Devices->"Device Name" on left->Device Logs on Xcode.
To answer your first question, take a look at https://developer.apple.com/library/ios/documentation/iphone/conceptual/iphoneosprogrammingguide/ManagingYourApplicationsFlow/ManagingYourApplicationsFlow.html
What I did to get code running in the background is to do something like
In the .h file
UIBackgroundTaskIdentifier backgroundUploadTask;
In the .m file
-(void) functionYouWantToRunInTheBackground
{
self.backgroundUploadTask = [[UIApplication sharedApplication] beginBackgroundTaskWithExpirationHandler:^{
[self endBackgroundUpdateTask];
}];
//code to do something
}
-(void) endBackgroundUpdateTask
{
[[UIApplication sharedApplication] endBackgroundTask: self.backgroundUploadTask];
self.backgroundUploadTask = UIBackgroundTaskInvalid;
}
The code above I pretty much learned from objective c - Proper use of beginBackgroundTaskWithExpirationHandler
As for your second question, you should set a breakpoint where code is supposed to run when you bring the app back to the foreground. No one can figure out why an app crashes if not given enough code or information.
The solution to the second part of the question was to raise the scope of the object containing the callback code. I raised it to the level of the containing ViewController. This seems to work. I still can't figure out how to raise the Notification (i.e. execute the call back) if the notification comes while the app is in the background/suspended. This prevented the object containing the callback from being cleaned up.
So I had this idea to test the implementation of my screen tracking (with Google Analytics) on my app using UI automation.
The original idea was to build a UI script to go through the screens while checking if the tracking events are being sent accordingly. I need this as sometimes I'm not able to compose everything out of view controllers or the events are not forwarded in the expected order. Regardless of that, I should test this aspect of my app as well and I thought that UI automation was the answer.
I have implemented a script to go through the screens using the UI automation instrument and this is working correctly. I even went so far as using tuneup js to make the code more streamlined and easier to follow.
I was expecting to have something like (in general terms, the syntax is only a simplification):
Being on screen X
Tap button A
Expect screen Y and tracking event for the screen Y
However, as far as I was able to check, testing the screen tracking is something that is not possible with the UI automation.
Or am I missing something?
I thought of creating an invisible view that stays on top of all the view hierarchy and changing its name every time a new screen is loaded to allow me to test it with UI automation but the idea sounded a little over the top...
What do you people suggest? Look for another UI automation tool? Do it with unit testing instead?
Thanks in advance for any help
You could use a UIAlertView and inspect those alerts. Instead of sending the analytics events you can pop up the alert so you can check on it in UIAutomation.
Analytics abstraction frameworks like AnalyticsKit provide an easy way to change the analytics provider. And AnalyticsKit even has an example for that (take a look at the AnalyticsKitDebugProvider class). So the changes to your production code are minimal.
You could use a build configuration where you set a build variable to control the initialization of your analytics
id<AnalyticsKitProvider> provider
#ifdef USE_UI_AUTOMATION_ANALYTICS
provider = [[TestAutomationProvider alloc] init];
#else
provider = [[RealProvider alloc] initWithApiKey:API_KEY];
#endif
[AnalyticsKit initializeLoggers:#[provider]];
In UIAutomation you can test for the alert coming up. You can utilize assertions.js out of the tuneup.js package to write a function like this
function checkForAlert()
{
var alert = null;
retry( function() {
log("wait until alert appaers");
alert = UIATarget.localTarget().frontMostApp().alert();
assertNotNull(alert, "No alert found");
assertTrue("The name you can choose for the alert" == alert.name());
}, 5, 1.0);
return alert;
};
This combines waiting for the alert and testing if it finally appear. If the alert not appears, the test will fail.
In your test you use this in the following way:
var analyticAlert = checkForAlert() // if alert appears it will be in the var, otherwise the test fails at this point.
analyticAlert.buttons()["OK"].tap(); // dismiss the alert
To make this work you also need to set an onAlert handler. Otherwise UIAutomation would try to dismiss your alert immediately. This has to be done before your tests code. Alert handling is explained in the UIAutomation docs.
function MyOnAlertHandler(alert)
{
if("The name you choose"==alert.name()) // filter all alerts created by analytics provider
{
return true; // handle alert in your test
}
return false // automaticly dismiss all other
}
UIATarget.onAlert = MyOnAlertHandler; // set the alert handler
I want to make one internal application(not want to put it on AppStore). In which i have following scenario.
I am putting my app(say "XApp") in background. Now, i open other app in foreground(Say "YApp"). Now, i want to get the current elements hieararchy of the YApp in Xapp. I am having development provisioning profile/certificate for the Yapp.
I just want the Element hierarchy, almost similar to what i get, if i run the UIAutomation of instruments and throw the following command through .js script.
UIATarget.localTarget().frontMostApp().logElementTree();
In short, i need same result as from above command, but without connecting device to any PC via usb cable. It is OK, if it is by using private apis or for jailbroken device. As i have mentioned that, i dont want to put it on App Store.
EDIT:
I can find elements hierarchy of my own app, i.e. for XApp itself, by runnning following code as,
-(void) showViewHeirarchy:(UIView*)v
{
for (UIView *vv in v.subviews)
{
NSLog(#"SubView:%#",[vv description]);
NSLog(#"count:%lu",(unsigned long)vv.subviews.count);
if (vv.subviews.count>0)
{
[self showViewHeirarchy:vv];
}
}
}
Calling above function after viewDidLoad method completed its execution(not in viewDidLoad), and calling it such as:
UIWindow *v=[[UIApplication sharedApplication] keyWindow];
[self showViewHeirarchy:v];
So, can anyone know how to get refrence of UIApplication object of one App from other App(running in background), so that my job will became easier.