Xcode UITesting test for enabled segmented control fails in iOS12 - ios

Question

Before I submitted this to Apple as a bug report, I wanted to double check I am not doing something very silly.
I have attached a sample project that demonstrates the problem. I have two segmented controls, one of which controls the enabled state of the other.
https://www.dropbox.com/s/dq2x9srbme4genb/EnabledControlsProblem.zip?dl=0
If you click on the disabled button, it disabled the second segmented control.
I have a basic UI test that replicates this behaviour which is constructed like this:
XCTAssertTrue(app.segmentedControls.buttons["Enabled"].exists)
XCTAssertTrue(app.segmentedControls.buttons["Enabled"].isSelected)
XCTAssertTrue(app.segmentedControls.buttons["First"].exists)
XCTAssertTrue(app.segmentedControls.buttons["First"].isSelected)
XCTAssertFalse(app.segmentedControls.buttons["Second"].isSelected)
XCTAssertTrue(app.segmentedControls.containing(.button, identifier: "First").firstMatch.isEnabled)
app.segmentedControls.buttons["Disabled"].tap()
XCTAssertFalse(app.segmentedControls.containing(.button, identifier: "First").firstMatch.isEnabled)`
If you run this test on an iOS 11.4 device it works as I think it should and succeeds. However, if I switch to a iOS 12.1 simulator device, the test fails on the last line (which is checking if the second control is enabled). Visually state of the control changes as it should. I have also tried this same test on an actual iPhone device running 12.0
Any attempts to sleep before the last check, don't make a difference.
Is there something fundamentally wrong with what I have done or is this a bug or a change in iOS12 that I missed?

Answer 1

I used to have the same type of problems with UI testing and I discovered a better method in pretty much all cases, and this lies in the key point that all tests rely on the Accessibility Framework. I was getting these weird seemingly buggy results all the time.
1. Better Debugging
Debugging goes much easier when you use the following command, which will show the hierarchy of elements:
print("Current App Tree: \(app.debugDescription)")
The hierarchy will help you ensure that the element does exists and how the app/device/simulator is seeing it.
2. Use Accessibility Identifiers
Instead of finding elements as you are, understand that all the tests are built on the accessibility framework. This seems insignificant at first, but I found it made life much easier. The framework is much happier when all elements have an accessibility identifier. Basically each view has a property: "accessibilityIdentifier". This will show up in the app debug listed above. If you do not set this, the framework will have a hard time finding your elements. If you're using storyboards, this is easy to set under Accessibility. If you're using code, set the property as follows:
myView.accessibilityIdentifier = "Enable State Selection"
Then in your UI Tests you can find the element like this:
app.buttons.matching(identifier: "Enable State Selection").element
In your specific case, it looks like it is part of a UI group of some kind involved in segmented controls. I would make a two step finding like this:
myUIGroup.accessibilityIdentifier = "Controls"
Then you can do a two step finding like this:
let controlsView = app.segmentedControls.matching(identifier: "Controls").element
let enabledButtons = controlsView.buttons.matching(identifier: "Enable State Selection").element
XCTAssert(enabledButtons.exists)
Overall I find that you at the beginning of your UI Testing class define these as class globals:
let app = XCUIApplication()
lazy var controlsView = app.segmentedControls.matching(identifier: "Controls").element
lazy var enabledButtons = controlsView.buttons.matching(identifier: "Enable State Selection").element
Then in your test function the code is very clean, and simply:
XCTAssert(enabledButtons.exists)