How does UIAutomation determine whether a UIAElement.isVisible() - ios

I have a view with the following structure:
GrandView
-Parent View1
-Parent View2
--Child View1
--Child View2
The child views take up almost all of the bounds of Parent View2, but there is still some space around the edges. I can select Parent View2 in the simulator with the accessibility inspector if I click on the edges. I can also tap Parent View2 in UIAutomation if I use:
tapWithOptions({tapOffset:{x:0.15, y:0.95}});
However, my calls to isVisible() always return 0. I expect that if I can tap the element, or select it with the accessibility inspector, it should return 1.
How does UIAutomation determine whether a UIAElement is visible?

Every operation you perform against an element has a timeout. This isn't obvious from the Apple documentation, but if you look at: setTimeout , it tells us:
The timeout value establishes a grace period for object resolution. If an object representing a UI element becomes available within the grace period, an attempt is made to instantiate that object from information retained by the instrument.
setTimeout itself just changes the default value (as do push and pop). What you really want to do is perform your action on your view, and fail on the timeout if it never becomes available (the default timeout is 5 seconds). The WWDC 2010 session "Automating User Interface Testing with Instruments" does go into this a little, it's available on the ADC WWDC 2010 page, with both video and slides. In your case, you'd want to execute the tap() on your view. If, for some reason, that view isn't available to UIAutomation within 5 seconds, you should see an exception. Experiment with changing the timeout by doing:
var oldTimeout = target.timeout();
target.pushTimeout(10);
before your code, and
target.popTimeout(oldTimeout);
after.

If it's a UIView, it should be driven the the hidden property. If it's not a view, and it's a container, it should be driven by accessibilityElementsHidden .
In general though, you don't want to use this for UIAutomation. Instead, whatever you were going to do on the view - in this case, a tap() - go ahead and do it, and let the system throw an error if it times out. In general this is the model you want to follow in your scripts rather than testing whether something is available first. Sine UIAutomation is DOM scripting the UIAccessibility information, when things like animated view transitions happen things get out of sync. Sometimes the script executes faster than the UI animates, and sometimes the opposite! waitForInvalid may be a shortcut to a solution for you.
Alex Vollmer's tuneup.js library for UIAutomation makes writing tests much easier, and is easy to extend.
https://github.com/alexvollmer/tuneup_js

Related

How to scroll to an element which is not inside the viewport in Appium?

I am using appium java client with uiautomator2 as the automation engine for android. The application has a scrollview from which I need to fetch text off TextView. some of those are within the viewport when the screen is loaded and some of those are outside viewport. How can I fetch the text of the nodes in a right way? Right now, I have a specific test device on which I know which element is visible and which is not. So for the later, I have implemented a scrollTo method to scroll down to.
I have seen solutions on keeping scrolling until the element is in viewport. I didn't like it as it will hamper the performance of the tests. because the test could still be waiting for an element before scrolling next and that time is simply wasted.
What could be a better strategy to handle this? We are using pagefactory pattern and mostly using xpath, accessibility label and id to identify the elements. In case it helps, the app under test is a react native app.
I suggest using UiScrollable for that.
If you use page object pattern and #AndroidFindBy annotation, you can simply put:
#AndroidFindBy(uiSelector = "UiScrollable(\"className(\"android.widget.ScrollView\")\").scrollIntoView(resourceId(\"id_of_element_to_scroll_to\"))")
private MobileElement buttonOfTheScreen;
Here you need to define two locators:
one for the element which will be swiped on, let call it swipe base - className(\"android.widget.ScrollView\") in the example above
the second locator is for element that is out of the screen - resourceId(\"id_of_element_to_scroll_to\")
No additional actions are required. Once you need to interact with your element, e.g. getText() or click(), uiautomator will swipe to it automatically.
Alternatively, you can use:
String locator = "UiScrollable(\"className(\"android.widget.ScrollView\")\").scrollIntoView(resourceId(\"id_of_element_to_scroll_to\"))";
driver.findElement(MobileBy.AndroidUIAutomator(locator)).click();
Note: this approach works only if the element you are looking for is under the bottom edge of your screen.

iOS Accessibility - Change AccessibilityElementsHidden without delay?

I am working on a view in which certain elements will be removed and re-added as accessible items depending on the state of the view. I have been able to successfully achieve the functionality I desire by setting AccessibilityElementsHidden to toggle the state.
However, I am finding that there is a brief pause (~1-2 seconds) between this field being set before the Accessibility Layout is updated, which can allow the user to highlight a deactivated accessibility element if they are moving at a reasonable pace. If they are focused on an item as it is being disabled it makes it difficult to re-orient oneself in the VoiceOver interface.
I have found methods to immediately update the display of the interface (by means of SetNeedsLayout() and LayoutIfNeeded() on the main thread) but unfortunately this does not trigger the Accessibility Layout update.
I have also tried using UIAccessibilityPostNotification.LayoutChanged but like the changing of the AccessibilityElementsHidden property, this also takes a moment to propagate to the view.
Can anyone provide some insight as to what I need to do to ensure the user cannot put themselves in a bad state before the Accessible Layout is applied?

Seeking advice for "dual mode" for increment/decrement of control with UIAccessibilityTraitAdjustable

I implemented a custom iOS control (looking similar to a ruler), where the user can easily scroll to select one of about 300 values.
My control also has the trait UIAccessibilityTraitAdjustable and implements the methods accessibilityIncrement() and accessibilityDecrement() of the protocol UIAccessibilityAction to update its value. So far everything works as intended.
The problem is that adjusting the value via VoiceOver can be tedious, since each swipe gesture increments/decrements the value only by one. Is there a good way to offer a second mode where the updates happen, e.g. in steps of ten?
UIAccessibilityTraitAdjustable uses a fixed, developer-defined increment size. You can adjust the slider with a larger step size in -accessibilityIncrement if you think users will find it tedious to adjust the control one unit at a time. VoiceOver users desiring finer control can use the pass-through gesture (double tap and hold) to interact with the control directly, sliding to adjust it in single-percent increments. This does, however, assume users have fine motor control.
If you feel strongly that UIAccessibility should support a "fine-grain" adjustment mode, file an enhancement request with supporting use cases.

In iOS, is there a relationship between run loop and display refresh?

In iOS, is there a relationship between run loop and display refresh? If so how?
Apple has a very good WWDC 2012 video describing in detail what happens at the end of each run loop when the current CATransaction commits and drawing happens and animations begin. You might also enjoy reading the less technical explanation of the "redraw moment" in my book:
http://www.apeth.com/iOSBook/ch17.html#_drawing_animation_and_threading
Apple's View Programming Guide for iOS / View and Window Architecture / View Architecture Fundamentals / The View Drawing Cycle says this:
When the contents of your view change, you do not redraw those changes directly. Instead, you invalidate the view using either the setNeedsDisplay or setNeedsDisplayInRect: method. These methods tell the system that the contents of the view changed and need to be redrawn at the next opportunity. The system waits until the end of the current run loop before initiating any drawing operations. This delay gives you a chance to invalidate multiple views, add or remove views from your hierarchy, hide views, resize views, and reposition views all at once. All of the changes you make are then reflected at the same time.
(Emphasis added.)

UIViewController viewDidLoad vs. viewWillAppear: What is the proper division of labor?

I have always been a bit unclear on the type of tasks that should be assigned to viewDidLoad vs. viewWillAppear: in a UIViewController subclass.
e.g. I am doing an app where I have a UIViewController subclass hitting a server, getting data, feeding it to a view and then displaying that view. What are the pros and cons of doing this in viewDidLoad vs. viewWillAppear?
viewDidLoad is things you have to do once. viewWillAppear gets called every time the view appears. You should do things that you only have to do once in viewDidLoad - like setting your UILabel texts. However, you may want to modify a specific part of the view every time the user gets to view it, e.g. the iPod application scrolls the lyrics back to the top every time you go to the "Now Playing" view.
However, when you are loading things from a server, you also have to think about latency. If you pack all of your network communication into viewDidLoad or viewWillAppear, they will be executed before the user gets to see the view - possibly resulting a short freeze of your app. It may be good idea to first show the user an unpopulated view with an activity indicator of some sort. When you are done with your networking, which may take a second or two (or may even fail - who knows?), you can populate the view with your data. Good examples on how this could be done can be seen in various twitter clients. For example, when you view the author detail page in Twitterrific, the view only says "Loading..." until the network queries have completed.
It's important to note that using viewDidLoad for positioning is a bit risky and should be avoided since the bounds are not set. this may cause unexpected results (I had a variety of issues...)
This post describes quite well the different methods and what happens in each of them.
currently for one-time init and positioning I'm thinking of using viewDidAppear with a flag, if anyone has any other recommendation please let me know.
Initially used only ViewDidLoad with tableView. On testing with loss of Wifi, by setting device to airplane mode, realized that the table did not refresh with return of Wifi. In fact, there appears to be no way to refresh tableView on the device even by hitting the home button with background mode set to YES in -Info.plist.
My solution:
-(void) viewWillAppear: (BOOL) animated { [self.tableView reloadData];}
Depends, Do you need the data to be loaded each time you open the view? or only once?
Red : They don't require to change every time. Once they are loaded they stay as how they were.
Purple: They need to change over time or after you load each time. You don't want to see the same 3 suggested users to follow, it needs to be reloaded every time you come back to the screen. Their photos may get updated... you don't want to see a photo from 5 years ago...
viewDidLoad: Whatever processing you have that needs to be done once.
viewWilLAppear: Whatever processing that needs to change every time the page is loaded.
Labels, icons, button titles or most dataInputedByDeveloper usually don't change.
Names, photos, links, button status, lists (input Arrays for your tableViews or collectionView) or most dataInputedByUser usually do change.

Resources