I have a set of XCUItests that exercise a bit of functionality in my app. Currently, I have the following function that pulls up a keyboard, types some stuff in, then taps "Search" (which is the equivalent in Enter in this context.
func clickSearchOnKeyboard(_ app: XCUIApplication) {
XCTAssert(app.staticTexts["Search"].waitForExistence(timeout: 10))
app.textFields["SearchItemView.SearchTextFieldID"].clearAndEnterText(testData.productData.valid.styleColor)
XCUIApplication().buttons["Search"].tap()
}
However, after updating the simulators to iOS 13, this test will fail because now when the keyboard is first pulled up we get a "What's New" fly-out explaining the new swipe functionality.
I think I can just add an If clause to my test code to handle this the first time it pops up, but I'm wondering if anyone has found out a way to disable these sorts of things for simulator testing:
Something in the Init() method that would disable "What's New" type of pop ups?
Some clever function that could always intercept this event and click "Continue"?
EXTRA BONUS POINTS: These automated tests run as part of an automated pipeline. As part of that, an assumption is that these tests run against a completely brand new simulator set (so we can't reuse existing simulators). Specifically, we blow away simulators (using Erase All Contents & Settings) prior to every run. So whatever solution will need to be completely portable and require 0 manual intervention.
Something else?
I think just testing for the dialog and dismissing it if it appears should be the best approach.
You could also launch an apple tool and trigger the popup after the erase & reset operation is complete as part of your initial tooling setup.
e.g.
xcrun simctl erase deviceuuid
xcodebuild test-without-building test name:DismissKeyboardTour destination=deviceuuid
xcodebuild test-without-building mytestsuite destination=deviceuuid
I would think you would only need to do this once per test suite run (eg all the tests)
You can run tests on clones of an already configured simulator (use xcrun simctl clone ...)
If you want to create simulators from scratch, then add a git repository in simulator folder, configure it as you like (skip keyboard onboarding in your case) and use git status to know what shall be changed in order to configure your simulators in scripts.
Detailed:
Create new simulator
Create a new git repository from its folder
Configure simulator the way you like
Watch changes in the simulator with git
Based on these changes add steps to your CI/CD runner script (prior test running)
Related
I was looking for smart way for always automatically run unit test before start archiving. The idea is that if I start all process (by clicking on archive button or by doing something else) unit test will start running. If unit test succeed, archive process will be started, otherwise not. I know it is nor difficult to do that with using Jenkins, etc. but I would like to have simple config/scheme in Xcode which will do the job.
Has anyone some experiences with that?
Select you scheme and go to edit scheme and select "Build".
Check mark Archive for Unit test case target as shown in below image.
Seems like it's just one setting called "Test after build" in Xcode. Someone posted a blog about it.
I am trying to use UIAutomation for testing an iOS 8 Today extension widget. I can change the target to the extension, launch it, but then unable to do anything after that.
Has anyone had any success in using UIAutomation with the extensions?
I hope I have the right end of the stick on this one.
I think you want to create a set of UI Tests which run like you can do with a normal application.
I found that I was not able to create a specific UI Tests target to then set the target application as WidgetExtension, as you would normally.
However I was able to set the WidgetExtension target to build and run onto a device (>= iOS 14) then within my UI Tests, I was able to record some steps which would allow me to write a clearer UI Test (place the cursor inside a test function to use record, you might already know this).
The tricky bit would be keeping the WidgetExtension target up-to-date onto the testing simulator to run your automated tests.
Having some issues getting a release build of an Air IOS Ipad app running (starling based).
I am doing a release build, and saving the ipa file on my drive. I double click the ipa to get it into iTunes, and from there I do an app install onto the iPad.
Once on the ipad, the app fails to load any external files.
I put up a preloader, and attempt to load assets via Loadermax, however, it seems to fail.
Here is the code section that the app fails on: (NOTE: this all works in debug mode, both in Air simulator, and debug mode on device via usb)
import com.greensock.events.LoaderEvent;
import com.greensock.loading.ImageLoader;
import flash.filesystem.File;
...
...
var _imageLoader:ImageLoader;
...
...
private function loadAssets():void{
var _appFile:File = File.applicationDirectory;
_appFile = _appFile.resolvePath("assets/statics/blueImage.png");
_imageLoader = new ImageLoader(_appFile.nativePath, {onComplete:onImageLoaded, onFail:onImageLoadFail});
}
private function onImageLoaded(e:LoaderEvent):void{
traceOut("OH MY GOD BATMAN, IT WORKED!!!"); // traceOut is a helper method that prints to a textfield on the display
}
private function onImageLoadFail(e:LoaderEvent):void{
traceOut("back to the Bat Cave...");
}
...
...
In debug mode, the above code shows:
OH MY GOD BATMAN, IT WORKED!!!
In release build:
back to the Bat Cave...
Anyone know where I am going off the rails? I am not using any ANE's or such.
Based in SDK: "http://ns.adobe.com/air/application/14.0"
PCs are not case sensitive but Ios is. While in simulator or debug mode a file name "myFile" would work even if it's really "myfile" (note the cap difference) on release mode it will fail.
If that doesn't work then try using the url property of File instead of nativePath. Also you can easily check is the file exist prior to loading it:
if(_appFile.exist)
Also when depending on custom frameworks you force yourself on depending on their shortcomings. Use a classic Loader instance to repeat the same operation and see if that one succeed. You'll be surprise the number of times a custom framework can fail on simple operations simply because the way it's setup internally.
Firstly, Thank you to to the individuals who assisted in getting to the root of this problem and solving, especially BotMaster.
The issue really has to do with Flashbuilder, and a bug that is associated with it's Build Release wizard.
After reading this:
https://forums.adobe.com/message/5750011
What I did was:
click the "Export Release Build" button in Flashbuilder,
IMPORTANT: check the box for "Keep bin-release-temp folder"
Click Next
Click Cancel
Via Windows Explorer, navigated to the bin-debug folder, and copied the "assets" and "configuration" folders to the "bin-debug-temp" folder that was generated by flash-builder
I return to Flash builder, and Click on "Export Release Build" button again.
Check the box for "Keep bin-release-temp folder"
Click Next
Check the boxes beside "assets" and "configuration", Click Finish.
the newly generated ipa file had the folders I required.
Once again, thanks to those who contributed to resolve this bug.
I am using UIAutomation to test an app, and I would like to find out my code coverage. But since javascript has no preprocessor, that means that gcov and similar are not an option. Has anyone come up with a solution for this?
For Xcode version 4.5 and newer:
Set the “Generate Test Coverage Files” build setting to Yes.
Set the “Instrument Program Flow” build setting to Yes.
This will generate code coverage files every time you run your application in the simulator and exit the application. A detailed explanation of these two steps can be found at the beginning of http://qualitycoding.org/xcode-code-coverage/.
For any unit tests, code coverage files will be generated every time you hit the test button and the tests complete. For UIAutomation, it is a little bit more tricky. You have to ensure the application exits at the conclusion of your tests. The easiest way I found to do this is to turn off multitasking. Add UIApplicationExitsOnSuspend in your MyAppName-Info.plist file and set this option to 'YES'. Run your UI automation test and at the end of it you can exit the app either by manually pressing the home button in the simulator or using the UIATarget.localTarget().deactivateAppForDuration() method.
Note: if your app has any UI Automation tests that rely on the deactivateAppForDuration() method, the tests will terminate upon running the command.
Code Coverage is only used for Unit Testing, there is no Code Coverage for UIAutomation because there is no way to tell how many elements on screen has been "touched" by UIAutoamtion
I have a set of automated test cases set up in instruments using tuneup.js to test an app. I decided to use tuneup.js as it allowed me to separate my tests into individual test cases and run the whole set from one individual script, this works fine if all the tests run ok, however if one fails, all the tests fail as the simulator is left in an unknown state (I have written my tests so they all start and end on the same login screen) Is there a way to reset the simulator, or restart the app between test cases?
Try to launch tests from the command line. UI Automation allows to execute only one test in one run. After the test will be completed (does not matter if it was failed or passed) - application will be kicked by the system (UIAutomation). At least it works with real devices.
Your command line launch script will work in the following manner:
1. Reads configuration file (can be any file txt or xml) with path to your tests. At this point you will have an array with path to your tests and total tests count.
2. Then using simple 'for' loop (from 1 to 'testcount') it will launch UIAutomation with required parameters. One of the parameters will be the path to your test script that was read from the configuration file.
You can also put the path to the 'configuration file' as a parameter for your command line launch script. This will allow you to run any test set simply calling the launch script with required configuration file.
I wrote a script that will reset the contents & settings of all versions and devices for the iOS Simulator. It grabs the device names and version numbers from the menu, so it will include any new devices or iOS versions that Apple releases simulators for.
It's easy to run manually or use in a build-script. I would suggest adding it as a Pre-Action Run Script before the build.
https://github.com/michaelpatzer/ResetAllSimulators
Having failed tests leave your app in an unknown state is one of the main problems with using Apple's instruments tool as-is. We solved this in a framework called Illuminator (on GitHub, and inspired by tuneup.js) in two ways.
First, we wrote an automation bridge -- a channel for RPC with the app being tested, which allows us to reset our app before each test.
In cases where that's not sufficient, the Illuminator test runner has an option to re-run each failed test in its own pristine launch of the simulator (e.g. with --retest 1x,solo).