UIAccessibilityElement `accessibilityCustomActions` not working properly - ios

I have created an instance of UIAccessibilityElement in order to provide a set of custom actions together with some additional information (i.e. accessibilityLabel + accessibilityHint)
The problem is that VoiceOver doesn't announce the existence of custom actions. They are there, they work, but don't get announced. Also, custom actions' hint is not being announced as well.
Any ideas?
Code to generate the element is below:
private lazy var accessibilityOverviewElement: UIAccessibilityElement = {
let element = UIAccessibilityElement(accessibilityContainer: self)
element.accessibilityLabel = viewModel.accessibilityOverviewTitle
element.accessibilityHint = viewModel.accessibilityOverviewHint
element.isAccessibilityElement = true
let close = UIAccessibilityCustomAction(
name: viewModel.accessibilityCloseActionTitle,
target: self,
selector: #selector(self.accessibilityDidClose))
close.accessibilityHint = viewModel.accessibilityCloseActionHint
let expand = UIAccessibilityCustomAction(
name: viewModel.accessibilityExpandActionTitle,
target: self,
selector: #selector(self.accessibilityDidExpand))
expand.accessibilityHint = viewModel.accessibilityExpandActionHint
element.accessibilityCustomActions = [close, expand]
return element
}()
I compute the element's frame in viewDidLayoutSubviews()
override func viewDidLayoutSubviews() {
super.viewDidLayoutSubviews()
var frame = view.bounds
frame.size.height = SleepAidMinifiedPlayerViewController.defaultHeight
accessibilityOverviewElement.accessibilityFrameInContainerSpace = frame
}
Finally, I need to be able to enable/disable accessibility since this view controller slides from the bottom and hides, but it's not completely removed from the view hierarchy (so VoiceOver still focuses on its elements)
func setAccessibility(enabled isEnabled: Bool) {
view.accessibilityElements = isEnabled ? [accessibilityOverviewElement, /* + other accessible elements*/].compactMap { $0 } : []
}
Thanks!

Any ideas?
I already created a radar about this problem: VoiceOver doesn't read out the custom actions anymore - Nov 4, 2019 at 5:01 PM – FB7426771.
Description: "Natively, in iOS 13, VoiceOver doesn't announce available actions even if they're present: example in the alarms settings, select an alarm and no actions is read out (it's OK in iOS 12) while they exist.
Moreover, if I create an element in an app with custom actions, they won't be announced in iOS 13 but they can be used if I know they're here (up and down swipe to get them).
However, if i use an older app targeting iOS 12, my elements containing custom actions are perfectly spelled out with the "actions available" announced with an iOS 12 device while the iOS 13 device does announce them 'sometimes'.
Please correct this huge turning back in the next iOS 13.3 version because it's extremely penalizing for the VoiceOver users."
No answers since but it's important to deliver a solution in a future version: I'm looking forward to seeing this correction in the next release notes.
However, your implementation should make your app work as desired, that's not the problem in my view ⇒ there are many useful examples (code + illustrations) if you need further explanations about some VoiceOver implementations.
Make your app run under iOS 12 and notice that it works while it's not the case under iOS 13.😰
⚠️ ⬛️◼️🔳▪️ EDIT ▪️🔳◼️⬛️ ⚠️ (2020/03/17)
The problem is that VoiceOver doesn't announce the existence of custom actions. They are there, they work, but don't get announced. Also, custom actions' hint is not being announced as well.
Even if you didn't mention your iOS version you're working with, I think this is iOS 13 because this weird behavior has been introduced making itself scarce in this version: no WWDC videos or info on the Apple website. 😤
This dedicated a11y site mentioned this modification ⟹ "iOS 13 introduced a new custom actions behavior: the "actions available" announcement isn't always present anymore.
It was previously offered to every element containing custom actions but, now, it will occur when you navigate to another element that contains a different set of actions.
The purpose is to prevent repetitive announcements on elements where the same actions are present as the previous element." 🤓
Take a look at this SO answer that highlights a response from a Technical Support Incident about this subject. 😉
Conclusion: if you need to use the announcement of the custom actions on each element they're implemented, use iOS 12 otherwise you'll have to work with this new behavior that wasn't explained anywhere and is definitely not efficient for the VoiceOver users ⟹ the Apple Technical Support claims that's the way it works from now.😰
⚠️ ⬛️◼️🔳▪️ EDIT ▪️🔳◼️⬛️ ⚠️ (2022/11/15)
I haven't this problem anymore, even in iOS 15. 🥳
If you're still in the same still bad situation in iOS 16, I suggest to check you've ticked the box Accessibility-VoiceOver-Verbosity-Actions-Speak in your device settings to make it work as expected (⟹ source). 👍
However, I've had no news from Apple regarding my TSI. 😵‍💫

Related

iOS 15 DualSense is opening App Library on PS button long press

On iOS 15 a long press on the PS Button of the DualSense controller is opening the App Library and I don't receive a callback via the valueChangedHandler function. The App library which will be opened looks like this
This is how I handle all controller inputs:
func handleController(controller: GCController) {
controller.extendedGamepad?.valueChangedHandler = { [weak self] (gamepad: GCExtendedGamepad, element: GCControllerElement) in
guard let self = self else {
return
}
// no feedback received when performing a long press on the PS button
}
Can the game library be suppressed somehow? Sony's PS Remote Play app somehow manages to suppress it, but I don't know how, nor can I find anything in Apple's official API documentation.
Edit: Seems this problem only occurs on iPads, on iPhones this problem doesn't exist. Is there some API or anything on iPads to suppress this behaviour? I assume the most majority of users don't want to open the App Library in the middle of the game.
If someone ever faces the same problem you can actually disable system gestures for the Home button.
In Swift all you have to add is this line (controller is a GCController object)
controller.physicalInputProfile.buttons[GCInputButtonHome]?.preferredSystemGestureState = .disabled
In ObjectiveC it would work like this
controller.physicalInputProfile.buttons[GCInputButtonHome].preferredSystemGestureState = GCSystemGestureStateDisabled;
Thanks to the Apple employee who helped me here
https://developer.apple.com/forums/thread/711905
Edit: on tvOS this isn't working as the PS button (menu button) of a controller always have to act as home event
https://developer.apple.com/forums/thread/715012

iOS 11.4 not asking Privacy Usage ( Privacy - Motion Usage Description has been set )

I'm stumped, iOS 11.4 ( 15F79 ), iPhone 6. Cannot get the App to Ask for Motion Data. info.plist has been set via the editor and double checked via the info.plist open in textWrangler, Also deleted key and saved via textWrangler.
<key>NSMotionUsageDescription</key>
<string>This app needs your Phones motion manager to update when the phone is tilted. Please allow this App to use your phones tilt devices</string>
I have deleted then reinstalled the app about 10 times. I have restared the phone 5 times. I have checked through settings and my app does NOT show up in Privacy-Motion and Fitness or anywhere else in settings. I am using a free developer account, maybe that has something to do with it?
I created a new Xcode game template and changed nothing apart from importing CoreMotion and this code
**** Edited, sorry I forgot to say I had started the instance, just forgot to put it here, just in case someone thinks that's the problem ************
let motionManager = CMMotionManager()
override func didMove(to view: SKView) {
motionManager.startDeviceMotionUpdates()
if motionManager.isDeviceMotionActive == true {
motionManager.accelerometerUpdateInterval = 0.2
motionManager.startAccelerometerUpdates(to: OperationQueue.current!, withHandler: {
(accelerometerData: CMAccelerometerData!, error: NSError!) in
let acceleration = accelerometerData.acceleration
print(accelerometerData)
} as! CMAccelerometerHandler)
}else{
print(CMMotionActivityManager.authorizationStatus().rawValue)
}
which prints a 0 ( an Enum - case not determined ) to the console.
In my actual app it was a 3 ( same Enum - case Denied ).
As I've said, I have uninstalled, reinstalled, edited plist via Xcode and text wrangler ( a code editor ) , tried different versions of the code above, tried the code in different places ( in did move to view, in class )tried code off apple docs. etc.... I haven't been asked the NSUsage question and the App keeps crashing.
I have looked for ways to get the Alert fired up, As in CLLocationManager.requestWhenInUseAuthorization() but I cannot find a comparable CMMotion version ( I don't think there is one. ) I have created a new swift file , imported Foundation and CMMotion and just put that code there, But still no Alert asking for Motion Data.
I tried a single view app template instead of a game template thinking that might be the issue, Nope.
What do I do?
Any help Appreciated. Thanks
You are confusing two related but different classes.
CMMotionManager gives access to accelerometer, magnetometer and gyroscope data. It does not require any user permission as this information is not considered privacy related.
In your else clause you are checking the authorisation status of CMMotionActivityManager. This object reports the device motion type (walking, running, driving). This information is considered privacy related and when you create an instance of this class and request data from it, the permissions alert is displayed.
The reason your else is being triggered is because you are checking isDeviceMotionActive; this will be false until you call startDeviceMotionUpdates, which you never do. Even if you used isAccelerometerActive you would have a problem because you call startAccelerometerUpdates in the if clause which will never be reached.
You probably meant to check isAccelerometerAvailable. If this returns false then there isn't much you can do; the device doesn't have an accelerometer.
Update
It doesn't make sense to check isDeviceMotionActive immediately after calling startDeviceMotion:
You know it's active; you just started it
I imagine the start up takes some time, so you could expect to get false if you check immediately.
Apple recommends that you do not have more than one observer in place for each motion device type, so the purpose of check the is...Active to ensure you don't call start... again if you have already done so.
If you only want gyroscope data then you don't need to call startDeviceMotionUpdates at all.

How to test right-to-left language in iOS XCTest unit tests?

Is there a way to switch an XCTest unit test into the right-to-left mode to test Arabic version of the app where sentences are written from right to left of the screen? My app code logic behaves differently based on language direction. I would like to verify this functionality in a unit test. What I need to do is to switch the app into the right-to-left language mode from an XCTest unit test case.
One can run the app in the right-to-left mode by changing the Scheme's Application language settings to Right-to-left Pseudolanguage. Is there a way to do similar thing in a unit test?
My imperfect solution
I ended up changing semanticContentAttribute of a view under test to .ForceRightToLeft. It does what I need to do. It does not feel like a very clean approach though. Firstly, it only works in iOS 9. Secondly, it looks like I am tinkering with my app views on a low level from the unit test. Instead, I would prefer to switch the whole app's language to right-to-left if it is possible.
class MyTests: XCTestCase {
func testRightToLeft() {
if #available(iOS 9.0, *) {
let view = UIView()
view.semanticContentAttribute = .ForceRightToLeft
// Test code involving the view
}
}
}
There's no easy way to do this right now with testing/UI testing besides passing in environment flags or setting the semanticContentAttribute as you are doing now. Filing a bug to Apple is highly recommended.
You can also change the device language & region in the scheme. This means you'll need separate schemes for the various LTR/RTL tests you want to run:
Xcode even provides pseudo-languages for extra-long string & RTL testing.
You can detect the writing direction via
let writingDirection = UIApplication.sharedApplication().userInterfaceLayoutDirection
switch writingDirection {
case .LeftToRight:
//
case .RightToLeft:
//
default:
break // what now? You are obviously using iOS 11's topToBottom direction…
}
To set different languages and locales on startup this might be a proper solution.
What you are looking for is Automated UI-Testing
This example JavaScript code changes the device orientation for example:
var target = UIATarget.localTarget();
var app = target.frontMostApp();
//set orientation to landscape left
target.setDeviceOrientation(UIA_DEVICE_ORIENTATION_LANDSCAPELEFT);
UIALogger.logMessage("Current orientation now " + app.interfaceOrientation());
//reset orientation to portrait
target.setDeviceOrientation(UIA_DEVICE_ORIENTATION_PORTRAIT);
UIALogger.logMessage("Current orientation now " + app.interfaceOrientation());
For testing, if your layout has changed to RTL or LTR you could try to access specific UI Elements and check their content against an expected content. So here is another example to check the contents of a TableViewCell from the official docs:
The crux of testing is being able to verify that each test has been performed and that it has either passed or failed. This code example runs the test testName to determine whether a valid element recipe element whose name starts with “Tarte” exists in the recipe table view. First, a local variable is used to specify the cell criteria:
var cell = UIATarget.localTarget().frontMostApp().mainWindow() \
.tableViews()[0].cells().firstWithPredicate("name beginswith 'Tarte'");
Next, the script uses the isValid method to test whether a valid element matching those criteria exists in the recipe table view.
if (cell.isValid()) {
UIALogger.logPass(testName);
} else {
UIALogger.logFail(testName);
}
If a valid cell is found, the code logs a pass message for the testName test; if not, it logs a failure message.
Notice that this test specifies firstWithPredicate and "name
beginsWith 'Tarte'". These criteria yield a reference to the cell for
“Tarte aux Fraises,” which works for the default data already in the
Recipes sample app. If, however, a user adds a recipe for “Tarte aux
Framboises,” this example may or may not give the desired results.
If you want to test a specific scheme:
Executing an Automation Instrument Script in Xcode
After you have created your customized Automation template, you can execute your test script from Xcode by following these steps:
Open your project in Xcode.
From the Scheme pop-up menu (in the workspace window toolbar), select Edit Scheme for a scheme with which you would like to use your script.
Select Profile from the left column of the scheme editing dialog.
Choose your application from the Executable pop-up menu.
Choose your customized Automation Instrument template from the Instrument pop-up menu.
Click OK to approve your changes and dismiss the scheme editor dialog.
Choose Product > Profile.
Instruments launches and executes your test script.

Sencha iOS picker bug

My app is nearly completed, but there's one bug which I have to get sorted before release. The app uses Cordova 3.4 and Sencha to build a "native" app for iOS and Android (the bug only relates to iOS)
Basically, when the picker value is changed, unless the user is quick enough in how they click Done, it reverts to the previous value - hard to explain! Here is a video showing the bug in action.
As mentioned before, this is only a problem on iOS (Android is fine). It is also worth noting that when there are two value options in other pickers in the app this bug does not exist. For example, the picker for time (hours & minutes) and date (day & month) do not have this bug - only single value pickers have the issue.
Any ideas?
I have just had to fix this issue within our product, and boy debugging on the iPhone is a right pain when you only have a Windows desktop!
Essentially what seemed to be happening was that when a slot's selection changed, the internal selectedIndex property was being updated, however the _value was not - and it seems that it's the _value that is being consulted.
I created a new slot class as follows, that overrides doItemTap to ensure that value is set appropriately (me._value = me.getValue(true);):
Ext.define('Ext.ux.FixedSlot', {
extend: 'Ext.picker.Slot',
xtype : 'fixedslot',
doItemTap: function(list, index, item, e, event) {
var me = this;
me.selectedIndex = index;
me.selectedNode = item;
me._value = me.getValue(true);
me.scrollToItem(item, true);
}
});
Then in my picker definition config (we have a class defined as a subclass of field.Select), I instructed it to use my new slot type (defaultType: 'fixedslot'):
Ext.define('Ext.ux.MyFixedPicker', {
extend: 'Ext.field.Select',
config : {
defaultPhonePickerConfig : { defaultType: 'fixedslot' }
}
});
I'm hoping that helps you avoid some of the pain of my last six hours! I still can't explain exactly why/where in the Sencha Touch source that's important, but for right now it appears to fix the problem and meet our packaging deadline!

UIAutomation with Instruments - How to tap copy/paste buttons?

I'm using Instruments for iOS automation and I can't seem to figure out how to tap options on the copy/paste menu. When I do a logElementTree(),I see that we are returning a UIEditingMenu and then three elements (which correspond to options of that menu, such as copy/paste, etc..). I am attempting to place this into a variable, and then trying to "tap" that variable but I cannot get that to work. Here is a sample of my code:
var target = UIATarget.localTarget();
var app = target.frontMostApp();
var window = app.mainWindow();
//This generates the highlighted text
app.dragInsideWithOptions({startOffset:{x:0.45, y:0.6}, endOffset:{x:0.45, y:0.6}, duration:1.5});
var copy = app.editingMenu.elements.withName("copyButton");
copy.tap();
Instruments returns, "0) UIAElementNil". In addition to the above, I've also tried:
app.elements.withName("copyButton")
window.elements.withName("copyButton")
So, I can get the editingMenu to produce the available options, but I cannot figure out a way to tap or select one of those options. I'm not quite sure I know how to reference those options to begin with.
Does anyone have any ideas?
Thanks!
You should try app.editingMenu().elements()[index].tap() where index is the index of the option you want to tap from the array of elements returned. I got my one working this way.
Hey.
First of all, I was always using .elements() not .elements... but it is JS, so it may be invoking function that is assigned to object property..?
Anyway, maybe this edit menu is not internal window of the app, but it is system level menu, that is invoked, when you do the drag? If that is true, try:
UIATarget.localTarget().frontMostApp().elements().withName("copyButton").tap();
But as I see in apple reference your version with calling app.editingMenu() should be fine...
Maybe try calling buttons by position, and you will see which respond:
UIATarget.localTarget().frontMostApp().editingMenu().elements()[0].tap;
UIATarget.localTarget().frontMostApp().editingMenu().elements()[1].tap;
UIATarget.localTarget().frontMostApp().editingMenu().elements()[2].tap;
You should find position of correct one this way. When you have it's position you can check its properties by button.logElement();. With this inf you you should be able to switch back to .withName method instead hardcoded position.
I did this similar to yoosiba but with editingMenu element names.
Using Xcode 4.5.1 and device running iOS 6.
Using Alex Vollmer's excellent tuneup_js for target, app and vtap().
Otherwise you can use UIATarget.localTarget().frontMostApp() and tap().
NOTE: vtap() will delay and retry tapping. Without this you may need to add your own delays.
// tap in textFieldA to see editingMenu.
app.mainWindow().textFields()["textFieldA"].vtap();
app.editingMenu().elements()["Select All"].vtap();
app.editingMenu().elements()["Copy"].vtap();
// must delay before attempting next tap
target.delay(2);
// ... navigate to different section of the app
// tap in textFieldB to see editingMenu.
app.mainWindow().textFields()["textFieldB"].vtap();
// paste clipboard contents copied from textFieldA into textFieldB
app.editingMenu().elements()["Paste"].vtap();
target.delay(2);

Resources