so I've recently started testing an iOS app with xctest. I'm on a time model view where I would like to change the number in a cell. The number in this cell is the number of days after which the selected time model repeats itself. But I'm unable to access this textfield and change the number as it does not have a label / name / static text. When I record a tap on this field, Xcode gives me a strange element hierarchy which I've defined as the parameter 'onDay' below
func testRepetitionTypeMonthsOnDay() throws {
let app = XCUIApplication()
let tablesQuery = app.tables
let cellsQuery = tablesQuery.cells
XCTAssertTrue(app.navigationBars["Zeitmodelle"].waitForExistence(timeout: standardTimeout)) //wait for the time model view to open
app.staticTexts["Wiederholend"].firstMatch.tap() //tapping on a cell
XCTAssertTrue(app.navigationBars["Wiederholend"].waitForExistence(timeout: standardTimeout)) // wait for the cell to open
let repetitionType = app.tables.cells["Am, Am Tag, Expand"] //cell 1 from screenshot
let onDay = tablesQuery.children(matching: .cell).element(boundBy: 7).children(matching: .other).element(boundBy: 1).children(matching: .other).element.children(matching: .textField).element //cell 2
let endDate = app.tables.cells["Endet, Endet nicht, Expand"] // cell 3 from screenshot
onDay.tap()
onDay.clearAndEnterText("5")
}
However, Xcode cannot find the parameter onDay that itself has generated in the previous step, tap on it and enter a new text. I've attached a screenshot of the app with this cell here. The cells above and below the marked cell can be identified easily and the assertions for their existence work. The marked cell, however, is a different matter as it does not have a label / static text. Do you have an idea how I can get around this? Thanks a lot!
So I've finally found a way to access this element. Instead of going into the cell view I've accessed the textfields array from tables view.
func testRepetitionTypeMonthsOnDay() throws {
let app = XCUIApplication()
let tablesQuery = app.tables
XCTAssertTrue(app.navigationBars["Zeitmodelle"].waitForExistence(timeout: standardTimeout)) //wait for the time model view to open
app.staticTexts["Wiederholend"].firstMatch.tap() //tapping on a cell
XCTAssertTrue(app.navigationBars["Wiederholend"].waitForExistence(timeout: standardTimeout)) // wait for the cell to open
let onDay = tablesQuery.textFields.element(boundBy: 3)
onDay.tap()
onDay.clearAndEnterText("5")
}
Because the app had a mixture of picker wheels and textfields in the same view, it was a bit problematic for Xcode to find the element I wanted from a general element index. But the use of the textfields array filtered the index list down to only textfields. Xcode was then able to identify the element without any conflict and perform the test. I hope this helps anyone having the same problem.
I am building UI automation tests for an app with a complex view hierarchy.
In my situation, I would like to use automation to tap on a UITextField subclass. That textfield is contained within a collectionView, which is itself contained within another collection view.
That textview has a known accessibility label+identifier, but I can't find any way to access this view via the normal XCTest api. For example, the following does not locate the element:
let app = XCUIApplication()
let textField = app.textFields["FOO"]
let textField2 = app.secureTextFields["FOO"]
let textField3 = app.otherElements["FOO"]
As another way to get a reference to the textfield, I tried setting up as a UIAccessabilityContainer, but I'm not sure I'm using it correctly.
//inside the textfield
//should return false if using UIAccessibilityContainer?
self.isAccessibilityElement = false
let e = UIAccessibilityElement(accessibilityContainer: self)
e.accessibilityIdentifier = placeholder
e.accessibilityLabel = placeholder
e.accessibilityTraits = .allowsDirectInteraction
//append this element to the window's list of elements
var eles = window?.accessibilityElements ?? []
eles.append(e)
window?.accessibilityElements = eles
Doing this seems to break all of my existing automation code using the "normal" api. What is the correct way to go about this?
I can't seem to get my array printed onto my textview. I have an array and have set my textview (texthold) to the randomIndex of the array.. Anything you guys see I'm doing wrong?
This is my button that when clicked is supposed to get a random string from the devices array and print it into the textview field. I am getting no errors just nothing displays onto the textview when the button is clicked
#IBAction func Generate(_ sender: Any) {
let devices = ["Apple Watch", "iPhone", "iPad", "Mac", "Apple TV"]
// Generate a random index
let randomIndex = Int(arc4random_uniform(UInt32(devices.count)))
// Get a random item
let randomItem = devices[randomIndex]
texthold.text = devices[randomIndex]
}
This code works in my program. Check your layout using "Debug view hierarchy tool", maybe it will help.
And:
texthold.text = randomItem
Name methods with lowercase (Generate -> generate)
Check if your button touch calls this method
Check if your UI items have connected outlets.
There may be problem with changing text from background thread, but if you really have this method connected from storyboard/xib as method for outlet's action - this situation impossible.
And check that your code is being called on .touchUpInside action of your button (second screenshot in storyboard)!
This is my simple UITest (customizing order of tabs in tabbarcontroller):
func testIsOrderOfTabsSaved() {
let app = XCUIApplication()
let tabBarsQuery = app.tabBars
tabBarsQuery.buttons["More"].tap()
app.navigationBars["More"].buttons["Edit"].tap()
tabBarsQuery.buttons["Takeaway"].swipeLeft()
tabBarsQuery.buttons["In Restaurant"].swipeLeft()
//here, how to get position of moved button with text "In Restaurant"?
NOTE:
It is possible to get XCUIElement from XCUIElementQuery by index. Can I do this fro the other way?
It seems that the queries automatically return in order based on position on screen.
for i in 0...tabsQuery.buttons.count {
let ele = tabsQuery.buttons.elementBoundByIndex(i)
}
Where the index i represents the position of the button in the tab bar, 0 being the leftmost tab, i == tabsQuery.buttons.count being the rightmost.
You have various ways to create a position test. The simplest way is to get buttons at indices 0 and 1, then get two buttons by name and compare the arrays are equal: (written without testing)
let buttonOrder = [tabBarsQuery.buttons.elementAtIndex(0), tabBarsQuery.buttons.elementAtIndex(1)]
let takeawayButton = buttons["Takeaway"];
let restaurantButton = buttons["In Restaurant"];
XCTAssert(buttonOrder == [takeawayButton, restaurantButton])
Another option is to directly get the frame of each button and assert that one X coordinate is lower than the other.
To answer your specific question about getting the index of an XCUIElement in a XCUIElementQuery, that's absolutely possible. Just go through all the elements in the query and return the index of the first one equal to the element.
An XCUIElement alone isn't able to tell you its position in within the XCUIElementQuery. You can search the XCUIElementQuery to discover its offset if you know something about the XCUIElement. In the below let's imagine that "More" is the identifier (change 'identifier' to 'label' if that is what you're working with).
If you want to find the offset of the "More" button (as posted in the original question) then:
var offsetWithinQuery : Int? = nil // optional since it's possible the button isn't found.
for i in 0...tapBarsQuery.buttons.count{
if tapBarsQuery.elementAtIndex(i).indentifier == "More" {
offSetWithinQuery = i
}
After the above loop exits, you'll either have the offset or it'll be nil if "More" isn't found.
We use KIF for our functional testing, and it uses the accessibility label of elements to determine where to send events. I'm currently trying to test the behaviour of a UISegmentedControl, but in order to do so I need to set different accessibility labels for the different segments of the control. How do I set the accessibility label for a particular segment?
As Vertex said,
obj-c
[[[self.segmentOutlet subviews] objectAtIndex:3] setAccessibilityLabel:#"GENERAL_SEGMENT"];
swift
self.segmentOutlet.subviews[3].accessibilityLabel = "GENERAL_SEGMENT"
some advice so you don't go crazy like I did:
To scroll in accessibility mode swipe three fingers
The indexes of the segments are backwards than you would expect, i.e. the furthest segment to the right is the 0th index and the furthest to the left is the n'th index where n is the number of elements in the UISegmentControl
I'm just getting started with KIF myself, so I haven't tested this, but it may be worth a try. I'm sure I'll have the same issue soon, so I'd be interested to hear if it works.
First, UIAccessibility Protocol Reference has a note under accessibilityLabel that says:
"If you supply UIImage objects to display in a UISegmentedControl, you can set this property on each image to ensure that the segments are properly accessible."
So, I'm wondering if you could set the accessibilityLabel on each NSString object as well and be able to use that to access each segment with KIF. As a start, you could try creating a couple of strings, setting their accessibility labels, and using [[UISegmentedControl alloc] initWithItems:myStringArray]; to populate it.
Please update us on your progress. I'd like to hear how this goes
Each segment of UISegmentedControl is UISegment class instance which subclass from UIImageView. You can access those instances by subviews property of UISegmentedControl and try to add accessibility for them programmatically.
You can't rely on the index in the subviewsarray for the position. For customisation of the individual subviews I sort the subviews on their X Position before setting any propery.What would also be valid for accesibilityLbel.
let sortedViews = self.subviews.sorted( by: { $0.frame.origin.x < $1.frame.origin.x } )
sortedViews[0].accessibilityLabel = "segment_full"
sortedViews[1].accessibilityLabel = "segment_not_full"
This is an old question but just in case anyone else runs up against this I found that the segments automatically had an accessibility label specified as their text. So if two options were added of Option 1 and Option 2. A call to
[tester tapViewWithAccessibilityLabel:#"Option 2"];
successfully selected the segment.
The solutions with using an indexed subview is not working since you cannot rely on a correct order and it will be difficult to change the number of segments. And sorting by origin does not work, since the frame (at least for current versions) seems to be always at x: 0.
My solution:
(segmentedControl.accessibilityElement(at: 0) as? UIView)?.accessibilityLabel = "Custom VoiceOver Label 1"
(segmentedControl.accessibilityElement(at: 1) as? UIView)?.accessibilityLabel = "Custom VoiceOver Label 2"
(segmentedControl.accessibilityElement(at: 2) as? UIView)?.accessibilityLabel = "Custom VoiceOver Label 3"
Seems to work for me and has the correct order. You also do not rely on an image. Not that pretty either but maybe more reliable than other solutions.
This is an old question but just in case anyone else runs up against this I found that the segments automatically had an accessibility label specified as their text.
Further to Stuart's answer, I found it really useful when writing test cases to turn on 'Accessibility Inspector' on the Simulator (Settings -> General -> Accessibility -> Accessibility Inspector). You'd be surprised how many elements already have accessibility labels included, like in the standard iOS UI elements or even third party frameworks.
Note: Gestures will now be different - Tap to view accessibility information, double tap to select. Minimizing the Accessibility Inspector window (by tapping the X button) will return the gestures back to normal.
You guys want to see how Apple recommends it be done?
It's FUGLY.
This is from this example:
func configureCustomSegmentsSegmentedControl() {
let imageToAccessibilityLabelMappings = [
"checkmark_icon": NSLocalizedString("Done", comment: ""),
"search_icon": NSLocalizedString("Search", comment: ""),
"tools_icon": NSLocalizedString("Settings", comment: "")
]
// Guarantee that the segments show up in the same order.
var sortedSegmentImageNames = Array(imageToAccessibilityLabelMappings.keys)
sortedSegmentImageNames.sort { lhs, rhs in
return lhs.localizedStandardCompare(rhs) == ComparisonResult.orderedAscending
}
for (idx, segmentImageName) in sortedSegmentImageNames.enumerated() {
let image = UIImage(named: segmentImageName)!
image.accessibilityLabel = imageToAccessibilityLabelMappings[segmentImageName]
customSegmentsSegmentedControl.setImage(image, forSegmentAt: idx)
}
customSegmentsSegmentedControl.selectedSegmentIndex = 0
customSegmentsSegmentedControl.addTarget(self,
action: #selector(SegmentedControlViewController.selectedSegmentDidChange(_:)),
for: .valueChanged)
}
They apply the accessibility labels to images, and then attach the images. Not too different from the above answer.
another option if not willing to set accesibility label might be calculating the poistion of each segment part and use
[tester tapScreenAtPoint:segementPosition];
to trigger the actions
If you look at the segmented control thru the accessibility inspector, you find that the segments are UISegment objects. Moreover, they turn out to be direct subviews of the UISegmentedControl. That fact suggests the following insanely crazy but perfectly safe Swift 4 code to set the accessibility labels of the segments of a UISegmentedControl:
let seg = // the UISegmentedControl
if let segclass = NSClassFromString("UISegment") {
let segments = seg.subviews.filter {type(of:$0) == segclass}
.sorted {$0.frame.minX < $1.frame.minX}
let labels = ["Previous article", "Next article"] // or whatever
for pair in zip(segments,labels) {
pair.0.accessibilityLabel = pair.1
}
}
As mentioned in the accepted answer, adding accessibilityLabel to the text should do the trick:
let title0 = "Button1" as NSString
title0.accessibilityLabel = "MyButtonIdentifier1"
segmentedControl.setTitle("\(title0)", forSegmentAt: 0)
let title1 ="Button2" as NSString
title1.accessibilityLabel = "MyButtonIdentifier2"
segmentedControl.setTitle("\(title1)", forSegmentAt: 1)
XCode 12 / iOS 14.3 / Swift 5
This is an old post but I encountered the same problem trying to set accessibility hints for individual segments in a UISegmentedControl. I also had problems with some of the older solutions. The code that's currently working for my app borrows from replies such as those from matt and Ilker Baltaci and then mixes in my own hack using UIView.description.
First, some comments:
For my UISegmentedControl with 3 segments, the subview count is 3 in the viewDidLoad() and viewWillAppear() of the parent UIVIewController. But the subview count is 7 in viewDidAppear().
In viewDidLoad() or viewWillAppear() the subview frames aren't set, so ordering the subviews didn't work for me. Apparently Benjamin B encountered the same problem with frame origins.
In viewDidAppear(), the 7 subviews include 4 views of type UIImageView and 3 views of type UISegment.
UISegment is a private type. Working directly with the private API might flag your app for rejection. (see comment below)
type(of:) didn't yield anything useful for the UISegment subviews
(HACK!) UIView.description can be used to check the type without accessing the private API.
Setting accessibility hints based on X order tightly couples UI segment titles and hints to their current positions. If user testing suggests a change in segment order, then changes must be made both in the UI and in the code to set accessibility hints. It's easy to miss that.
Using an enum to set segment titles is an alternative to relying on X ordering set manually in the UI. If your enum inherits from String and adopts the protocols CaseIterable and RawRepresentable, then it's straightforward to create titles from the enum cases, and to determine the enum case from a segment title.
There's no guarantee the following will work in a future release of the framework, given the reliance on description.contains("UISegment") but it's working for me. Gotta move on.
override func viewDidAppear(_ animated: Bool) {
super.viewDidAppear(animated)
// get only the UISegment items; ignore UIImageView
let segments = mySegmentedControl.subviews.compactMap(
{ $0.description.contains("UISegment") ? $0 : nil }
)
let sortedSegments = segments.sorted(
by: { $0.frame.origin.x < $1.frame.origin.x }
)
for i in 0 ..< sortedSegments.count {
let segment = sortedSegments[i]
// set .accessibilityHint or .accessibilityLabel by index
// or check for a segment title matching an enum case
// ...
}
}
On Private APIs and Rejection
I'm referring to the April 2016 comment from #dan in Test if object is an instance of class UISegment:
It's a private class. You can check it with [...
isKindOfClass:NSClassFromString(#"UISegment")] but that may get your
app rejected for using private api or stop working in the future if
apple changes the internal class name or structure.
Also:
What exactly is a Private API, and why will Apple reject an iOS App if one is used?
"App rejected due to non-public api's": https://discussions.apple.com/thread/3838251
As Vortex said, the array is right to left with [0] starting on the right. You can set every single accessibility option by accessing the subviews. Since the subviews are optional, it is good to pull out the subview first, and then assign the accessibility traits that you want. Swift 4 example for a simple two option segment control:
override func viewDidLayoutSubviews() {
super.viewDidLayoutSubviews()
guard let rightSegment = segmentControl.subviews.first, let leftSegment = segmentControl.subviews.last else { return }
rightSegment.accessibilityLabel = "A label for the right segment."
rightSegment.accessibilityHint = "A hint for the right segment."
leftSegment.accessibilityLabel = "A label for the left segment."
leftSegment.accessibilityHint = "A hint for the left segment."
}