I have created UITesting like this. I need to run those 3 file in sequence and I need to name like this. I think it is quite bad. I want to name more logically (instead of Test1,etc). Is it possible to rename and run in sequence?
Another one is I need to name my method alphabetically. I want to name randomly and run in sequence. How shall I do?
- (void)test2CreatePost
- (void)test3ClickLikeInNewsfeed
Functions nesting is fine for running UI tests from XCode instantly only. But when tests are running from command line, they are running by the alphabetic sequence. I've read that this issue was fixed for unit tests in XCode 8, but it seems not for UI tests.
For instance, now I have the following UI tests hierarchy:
UITests
TestCaseB
TestCaseA
When I run them from XCode 8.2.1 they are executed in the same sequence as they are described, i.e.:
TestCaseB
TestCaseA
But when I run them from command line using xcodebuild test TEST_TARGET_NAME=<TargetName> -scheme <UITestsSchemeName> destination 'platform=iOS Simulator,name=iPhone 7' they are running in another sequence:
TestCaseA
TestCaseB
Be careful.
You could achieve this by nesting your functions.
// Note that this function contains tests but does not follow the 'testXXX' naming convention
func login() {
// Implement your test here
}
func newsfeed() {
// Implement your test here
}
func moreTab() {
// Implement your test here
}
// Here is the test method that will run the tests in a user-specified order
func testSomething() {
login()
newsfeed()
moreTab()
}
You can use the procedure explained in this answer to achieve this.
Running individual XCTest (UI, Unit) test cases for iOS apps from the command line
Basically you need to override testInvocations present in XCTestCase, then you have control over the order of test case execution.
If you need more information on how to achieve this add a comment.
Related
Objective
Running simple unit-test for an iOS app.
Setup
Xcode (Version 14.1)
Sample Test:
import XCTest
#testable import SDKTestAppSwift
class SDKTestAppSwiftTests: XCTestCase {
func testExample() throws {
print("Test: testExample")
// This is an example of a functional test case.
let base = "asdfghjkl123"
XCTAssert(base == "asdfghjkl123")
}
}
Next to the ..
func testExample() throws {
.. Xcode provides a nice little 'start single test'-button.
I used to klick it to start the test, locally (Xcode IDE on mac).
This used to work properly.
Problem
Klicking the 'start single test'-button, an overlay appears:
For the sake of this question to be machine-readable:
There is no scheme and/or test plan that contains every test you are trying to run.
Create a new scheme and/or test plan containing the tests you want to run.
To close the overlay an 'OK'-Button is delivered only.
Misleading solution (?!)
Just before there was no such scheme needed.
I wonder why would that changes 'suddenly'.
From my point ov view Scheme/Testplans are executed 'on startup' of the device-emulator (or likewise), which is not my objective in the first place.
However - creating a TestPlan via:
File > New > File... > TestPlan
Adding the SDKTestAppSwiftTests-class using the +-Button.
... no success ... :-/
Several other attempts
It seems my system is not the only one facing likewise issues.
Problems with naming of test plans
xcodebuild: Tests cannot be run because the test plan “Scheme” could not be read
No matter if there is a 'test-plan' and whatever name I assigned it - this is no solution.
Default test plan
https://developer.apple.com/forums/thread/133495
I seriously do not know how to create a 'default test plan' if it was not the way decribed before.
Quit Xcode and delete 'old configurations'
Xcode: No Scheme
xcodebuild says does not contain scheme
Something about files with old configurations which have to be deleted
In this case I am not sure which file and where to find it exactly.
Conclusion
Even though one of the obove may help you, for me the issue is not solved, yet.
Anyways
Thanks for reading & sharing - any help is appreciated :)
For me I fixed this through my scheme → Edit Scheme → Test → Plus Sign (Add Test Target) and then adding my unit test target.
There is another solution ...
First - Manage Schemes
Second - Create new Scheme
Third -> Celebrate, it's done .. :)
I'm using swift 5.1 and XCTestCase framework, however Xcode fails to recognize that I have unit tests. Is there something I'm missing? All of the documentation on Apple shows that I am doing things correctly.
Your functions have to have the prefix 'test'. In your case it would be func testTrueWhenPassIsEmpty for example.
Apple Documentation and a good starter tutorial is here: https://www.hackingwithswift.com/read/39/2/creating-our-first-unit-test-using-xctest
Each test function has to have the 'test' word in front function name. So Xcode can recognize that this function is a test.
For example in your case it should be :
testReturnsTrueWhenPassInNil() { ... }
instead of:
returnsTrueWhenPassInNil() { ... }
I'm trying to set the language and the region of my UI Testing target, and unfortunately it doesn't seem to work.
I've tried both ways, first:
Product | Scheme | Edit Scheme
Run | Options
Application Language: French
Application Region: France
Test | Arguments
Use the Run action's arguments and environment variables: Checked
And I try this other way:
Product | Scheme | Edit Scheme
Test | Arguments
Use the Run action's arguments and environment variables: Unchecked
Arguments passed at launch:
-AppleLanguages (French)
-AppleLocale fr_FR
Each way resulting in the UI Testing still happening in English. Moreover, when I hit the record button, it runs in French...
So if I do something wrong, I would be happy to know!
Thanks in advance!
I have figured it out. I set the locale settings in the launchArguments for testing temporary in Xcode.
override func setUp() {
super.setUp()
// Put setup code here. This method is called before the invocation of each test method in the class.
// In UI tests it is usually best to stop immediately when a failure occurs.
continueAfterFailure = false
app = XCUIApplication()
app.launchArguments = [
"-inUITest",
"-AppleLanguages",
"(de)",
"-AppleLocale",
"de_DE"
]
For CI I use fastlane from Felix Krause and get localized screenshots with snapshot.
Try changing the specs on the simulator. It worked for me every time.
On Simulator:
Settings > General > Language & Region > iPhone Language --> change here
I am facing the issue of a test that has undefined step(s) not being flagged as a failed test.
In the Java code we use Selenium 2/WebDriver and tests are driven by Ant and run in a Continuous Integration environment.
For the following scenario:
#test1
Scenario: Run test with an undefined step
Given I am logged in to the application //working
And I view the test example //working
Then the tree panel exists in the layout //undefined step
The following is a snippet of what is seen in the console:
#test1
Scenario: Run test with an undefined step
Given I am logged in to the application
And I view the test example
Then the tree panel exists in the layout
1 scenario (1 undefined)
3 steps (1 undefined, 2 passed)
The ant target used to run the test:
ant test.cuke.firefox -Dwebtest.server="http://localhost:9944" -Dwebtest.cuke.options="--tags #test1"|wac
I read that using the --strict flag gets the tests to fail.
But I've no idea of where I need to mention the flag.
Is it in the build.xml file? If so, where exactly - as wherever I've tried hasn't helped.
Is it in the cucumber.yml file?
There are 2 such files:
i) \lib\cucumber.jruby\gems\cucumber-0.8.7
ii) \lib\cucumber.jruby\gems\gherkin-2.1.5-java
If not in these files, where else?
Could you please point to where and how the flag needs to be set?
I've tried looking up the help but nothing has helped (probably I'm looking in all the wrong places!)
Thanks!
You need to set the strict option:
http://cukes.info/api/cucumber/jvm/javadoc/cucumber/api/junit/Cucumber.Options.html#strict()
Edit: You can set this flag in the RunCukesTest like:
#RunWith(Cucumber.class)
#Cucumber.Options(
format = {"html:target/cucumber-html-report"},
strict = true)
public class RunCukesTest {
}
I am trying to run xUnit tests (from an F# module, if it makes any difference) using TestDriven.NET, but whatever I do I get this error:
It looks like you're trying to execute an xUnit.net unit test.
For xUnit 1.5 or above (recommended):
Please ensure that the directory containing your 'xunit.dll' reference also contains xUnit's
test runner files ('xunit.dll.tdnet', 'xunit.runner.tdnet.dll' etc.)
For earlier versions:
You need to install support for TestDriven.Net using xUnit's 'xunit.installer.exe' application.
You can find xUnit.net downloads and support here:
http://www.codeplex.com/xunit
I tried following the suggestions, i.e. I copied the files
xunit.dll.tdnet
xunit.extensions.dll
xunit.gui.clr4.exe
xunit.runner.tdnet.dll
xunit.runner.utility.dll
xunit.runner.utility.xml
xunit.xml
to the folder with xunit.dll and I ran xunit.installer.exe. How can I get it to work?
I just figured out that I forgot to make the test a function in F# (so it was just a value). The error message can't be more misleading though!
You have two problems:
your Fact is broken:-
If you hover over the
please work
bit, you'll see something like: unit -> int
For a Fact to be picked up by an xUnit runner, it needs to yield `unit (void).
Hence, one key thing to get right first is to not return anything. In other words, replace your 123 with () (or an Assertion).
You can guard against this by putting a :unit stipulation on the test:-
[<Fact>]
let ``please work`` () : unit = 123
This will force a compilation error.
TestDriven.NET is reporting it cannot find the xunit.tdnet modules
It's critical to get step 1 right first. Then retry and the problem should be gone
If it remains...
Either try the VS-based runner which should work as long as it's installed and xunit.dll is getting to your output dir or look at the docs for your version of TD.NET for detailed troubleshooting notes (exec summary is if the .tdnet file was in your out dir or you undo and redo the xunit.installer from the folder containing the packages it should just work, esp if you are on latest)