Is it possible to configure Xcode in such a way so when I run all of my test each one will be performed on a "new iPhone Simulator". I'm testing UI related stuff and I want each test to start on a newly opened application.
Currently, application is starting only once and all of the tests are starting one after another and sometimes the ending state after one test is not good for the next test to start.
Solved
I used XCTool (https://github.com/facebook/xctool)
I was able to run one particulatr test by invoking command:
xctool -workspace WorkspaceName.xcworkspace -scheme Workspace -sdk iphonesimulator run-tests -only TestTarget:TestClass/test_Some_Method
Then I simply wrote a script to run all of the tests I had. This way every test triggers simulator to run application once again from scratch.
You could, before the next test, delete the app as you would do it in a real iPhone.
Related
I'm working on building a UI Test suite for my iOS app. I need to test my app's functionality on several different devices, but right now I have to select the simulator I want, run the tests, and then repeat.
Now that Xcode supports multiple simulators running in parallel, is there a way to run the UI tests across several different device simulators at the same time?
Run the following command in the same directory as your project to run your tests in parallel from the command line:
xcodebuild test -scheme "YourSchemeName" -destination 'platform=iOS Simulator,OS=11.2,name=iPhone 8' -destination 'platform=iOS Simulator,OS=11.2,name=iPhone 6s' -configuration "Debug" ENABLE_TESTABILITY=YES SWIFT_VERSION=4.0 ONLY_ACTIVE_ARCH=YES
You can add -destination 'platform=iOS Simulator,OS=11.2,name=iPhone 8' for a different destination for as many different destinations as you would like.
For a list of simulator names and OSs that are available, run the command:
instruments -s devices
Bear in mind that if you are running tests in the simulator, you will not see the simulators on your screen when running tests through the command line.
In Xcode:
Select your target scheme in Xcode, and "Edit Scheme..."
Find the settings for "Test", and press on the "Info" tab
You'll see a list of your Unit and UI tests, press on the associated
"Options..." button
Select "Execute in parallel on Simulator"
Optionally select "Randomize execution order"
Commandline:
Look at this answer here
i am facing a problem. I want to do a ui test for my application.
i have written six test class where my test cases are included.
I can run my test cases using
"command U" or the clicking in the run button.
But i don't want to do like this.
I want to run my test classes using code i mean using
xctest because after complete my execution, i have to do another task.
can anyone please give an example how i can run my test classes using xctest codding from a class file.
i dont want to use shell script or jenkins, i want it by using xctest framework.
Running tests with xcodebuild
The xcodebuild command-line tool drives tests just like the Xcode IDE. Run xcodebuild with the action test and specify different destinations with the -destination argument. For example, to test MyApp on the local OS X “My Mac 64 Bit,” specify that destination and architecture with this command:
> xcodebuild test -project MyAppProject.xcodeproj -scheme MyApp -destination 'platform=OS X,arch=x86_64'
If you have development-enabled devices plugged in, you can call them out by name or id. For example, if you have an iPod touch named “Development iPod touch” connected that you want to test your code on, you use the following command:
> xcodebuild test -project MyAppProject.xcodeproj -scheme MyApp -destination 'platform=iOS,name=Development iPod touch'
Tests can run in Simulator, too. Use the simulator to target different form factors, operating systems, and OS versions easily. Simulator destinations can be specified by name or id. For example:
> xcodebuild test -project MyAppProject.xcodeproj -scheme MyApp -destination 'platform=Simulator,name=iPhone,OS=8.1'
Source
Edit your Xcode scheme.
Click the disclosure triangle next to "Test" to reveal more options.
Select "Post-actions"
Click '+' to add any actions to run after tests are complete.
As you can see, your choices are to run a script, or send a simple email. For more information on configuring actions, see https://michele.io/the-project-file-part-2/
I am investigating how to use XCTest to close our automation test gap.
From the developer document, I only see something like this:
xcodebuild test -project MyAppProject.xcodeproj -scheme MyApp
https://developer.apple.com/library/tvos/documentation/DeveloperTools/Conceptual/testing_with_xcode/chapters/08-automation.html#//apple_ref/doc/uid/TP40014132-CH7-SW1
I would like to know if there is a way to run XCTest(UI test) against an archived build (.ipa)?
or
Can we just seperate the build and test so that we can build first and test against that build later?
Thanks
If you want to separate the build and test stages, you can use the xcodebuild flags build-for-testing (which builds a xctestrun file) and test-without-building (which runs it). You may want to build-without-testing to a generic device if you want the build-for-testing to be used on multiple devices.
Check out https://medium.com/xcblog/speed-up-ios-ci-using-test-without-building-xctestrun-and-fastlane-a982b0060676 for more details
Running XCTest-based tests from the cmd-line causes the tests to silently pass, even when a test fails when run via the IDE.
Added a new XCTest-based test target to an iOS App.
Ran tests with the following command:
xcodebuild clean test -workspace VoucherBlaster.xcworkspace -scheme VoucherBlaster -configuration Debug -destination OS=7.1,name=iPad
xcodebuild reported tests as passing, even though the example test failed.
Used SenTest instead and the build reported failed tests, as expected.
Questions:
Is this a common/known issue? (If it is I'd normally be surprised that the problem made it all the way into Xcode5.1 GA, but it wouldn't be the first time :-/ )
Is there a work-around?
I've set up jenkins on my local machine. Than I created a sample app with one failing and one succeeding unit test. When I issue the following command in the terminal
xcodebuild -scheme 'SampleWithTest' -sdk iphonesimulator7.0
-destination platform='iOS Simulator',OS=7.0,name='iPhone Retina (3.5-inch)' clean test
than it clean build the project, starts the simulator and makes the tests. This is how it should work.
When I run the same command via jenkins, it clean builds the project and ends with ** TEST SUCCEEDED ** without starting the simulator or print out the test results. Jenkins just fakes, that the tests was successful.
I test it on my local machine. Nothing headless, remote, source control, slave, code signing or whatever could make any trouble.
The Jenkins Xcode Plugin has not been updated to work with Xcode 5 and the XCTest framework. See this issue. Jenkins and the Xcode Plugin do not understand XCTest output.
Additionally, Xcode 5 changed how Xcode interacts with the simulator and devices when running tests. While there is something like a "headless mode" now for the simulator, access to that functionality isn't available to the Xcode plugin. This means that in some cases, it looks to the Xcode plugin as if the tests run and exit immediately, which is not accurate.
Xcode is known to be pretty tricky to setup with Jenkins, Travis or other non-Apple CI solutions.
Unit Tests support is even worse so you should try using xctool instead.
Related blog entry here.