UIAutomation with Instruments - How to tap copy/paste buttons? - ipad

I'm using Instruments for iOS automation and I can't seem to figure out how to tap options on the copy/paste menu. When I do a logElementTree(),I see that we are returning a UIEditingMenu and then three elements (which correspond to options of that menu, such as copy/paste, etc..). I am attempting to place this into a variable, and then trying to "tap" that variable but I cannot get that to work. Here is a sample of my code:
var target = UIATarget.localTarget();
var app = target.frontMostApp();
var window = app.mainWindow();
//This generates the highlighted text
app.dragInsideWithOptions({startOffset:{x:0.45, y:0.6}, endOffset:{x:0.45, y:0.6}, duration:1.5});
var copy = app.editingMenu.elements.withName("copyButton");
copy.tap();
Instruments returns, "0) UIAElementNil". In addition to the above, I've also tried:
app.elements.withName("copyButton")
window.elements.withName("copyButton")
So, I can get the editingMenu to produce the available options, but I cannot figure out a way to tap or select one of those options. I'm not quite sure I know how to reference those options to begin with.
Does anyone have any ideas?
Thanks!

You should try app.editingMenu().elements()[index].tap() where index is the index of the option you want to tap from the array of elements returned. I got my one working this way.

Hey.
First of all, I was always using .elements() not .elements... but it is JS, so it may be invoking function that is assigned to object property..?
Anyway, maybe this edit menu is not internal window of the app, but it is system level menu, that is invoked, when you do the drag? If that is true, try:
UIATarget.localTarget().frontMostApp().elements().withName("copyButton").tap();
But as I see in apple reference your version with calling app.editingMenu() should be fine...
Maybe try calling buttons by position, and you will see which respond:
UIATarget.localTarget().frontMostApp().editingMenu().elements()[0].tap;
UIATarget.localTarget().frontMostApp().editingMenu().elements()[1].tap;
UIATarget.localTarget().frontMostApp().editingMenu().elements()[2].tap;
You should find position of correct one this way. When you have it's position you can check its properties by button.logElement();. With this inf you you should be able to switch back to .withName method instead hardcoded position.

I did this similar to yoosiba but with editingMenu element names.
Using Xcode 4.5.1 and device running iOS 6.
Using Alex Vollmer's excellent tuneup_js for target, app and vtap().
Otherwise you can use UIATarget.localTarget().frontMostApp() and tap().
NOTE: vtap() will delay and retry tapping. Without this you may need to add your own delays.
// tap in textFieldA to see editingMenu.
app.mainWindow().textFields()["textFieldA"].vtap();
app.editingMenu().elements()["Select All"].vtap();
app.editingMenu().elements()["Copy"].vtap();
// must delay before attempting next tap
target.delay(2);
// ... navigate to different section of the app
// tap in textFieldB to see editingMenu.
app.mainWindow().textFields()["textFieldB"].vtap();
// paste clipboard contents copied from textFieldA into textFieldB
app.editingMenu().elements()["Paste"].vtap();
target.delay(2);

Related

How to unfocus from a WebView in UWP

I'm working on a UWP app that hosts a WebView which runs in a separate process.
var webView = new Windows.UI.Xaml.Controls.WebView(WebViewExecutionMode.SeparateProcess)
This results in a behavior that if the WebView has the focus, the containing app can't regain the focus by itself by simply trying to focus on a UI element.
The app supports keyboard shortcuts which may result in different elements getting the focus, but it's not working correctly when the focus is captured by the WebView. The target element seems to be getting the focus but it seems as if the process itself is not activated (as the real focus resides in a different process I suppose...).
I'm currently trying to activate the app programmatically through protocol registration in an attempt to regain focus.
I added a declaration in the app manifest for a custom protocol mycustomprotocol coupled with the following activation overload
protected override void OnActivated(IActivatedEventArgs args)
{
if (eventArgs.Uri.Scheme == "mycustomprotocol")
{ }
}
And the following code to invoke the activation:
var result = await Windows.System.Launcher.LaunchUriAsync(new Uri("mycustomprotocol:"));
Seems to be working only on some computers, on others (not while debugging the app, only when executed unattached) instead of regaining focus the app's taskbar icon just flashes orange.
I've created a sample project showing the problem and the semi working solution here
Any insight on any of this would be great.
I can reproduce your issue. I found that when we switch the focus with the mouse, the focus can be transferred to the TextBlock. So you could solve this question through simulating mouse input.
Please use the following code to instead FocusTarget.Focus(FocusState.Programmatic).
As follows:
InputInjector inputInjector = InputInjector.TryCreate();
var infoDown = new InjectedInputMouseInfo();
// adjust your mouse position to the textbox through changing infoDown.DeltaX,infoDown.DeltaY
infoDown.DeltaX = 10; //change
infoDown.DeltaY = -150; //change
infoDown.MouseOptions = InjectedInputMouseOptions.LeftDown;
var infoUp = new InjectedInputMouseInfo();
infoUp.DeltaX = 0;
infoUp.DeltaY = 0;
infoUp.MouseOptions = InjectedInputMouseOptions.LeftUp;
inputInjector.InjectMouseInput(new[] { infoDown, infoUp });
Note: If you use the input injection APIs, you need to add inputInjectionBrokered Capabilitiy in your Package.appxmanifest.
But this Capabilitiy is a restricted Capabilitiy, you can’t publish this app in store, which can’t pass the verification.
I've been in discussions with a WebView software engineer. The problem is that the separate process still wants to own focus if you try to move the focus away from the webview. His solution is to ask the other process' web engine to give up focus with the following call:
_= webView.InvokeScriptAsync("eval", new string[] { "window.departFocus('up', { originLeft: 0, originTop: 0, originWidth: 0, originHeight: 0 });" });
You can call it before trying to change the focus to your target. I ran various tests and it works consistently.

Can not access the programatically added UIView in XCTest

Below is my recorded XCTest
let app = XCUIApplication()
let tablesQuery = app.tables
tablesQuery.staticTexts["Video"].tap()
tablesQuery.staticTexts["\tWindowed"].tap()
app.buttons["Launch"].tap()
app.buttons["Popout Video"].tap()
app.children(matching: .window).element(boundBy: 0).children(matching: .other).element(boundBy: 1).tap()
When I am trying to run the test the last part that is:
app.children(matching: .window).element(boundBy: 0).children(matching: .other).element(boundBy: 1).tap()
is not accessible. It does not throw any error but the last line of code is not executed.
I have tried solving the issue by referring to the following stackoverflow question :
Xcode UI Tests can't find views that are added programatically
Also , I have referred to the following Apple Documentations: https://developer.apple.com/library/content/documentation/UserExperience/Conceptual/iPhoneAccessibility/Making_Application_Accessible/Making_Application_Accessible.html#//apple_ref/doc/uid/TP40008785-CH102-SW2
https://developer.apple.com/reference/uikit/uiview
But all these doesn't seem to solve the issue. Any help will be greatly appreciated.
The line in question is very brittle. It's possible that your app's views are not always available in the same order every time the app launches, depending on race conditions, so what was recorded in your recording session does not necessarily work for all launches of the app. You'll probably find that the code is actually running, but isn't tapping the element you expected.
To make your code less brittle, add an accessibility identifier to the UIView in your app code, and use the otherElements query to find the UIView.
// app code
let view: UIView!
view.accessibilityIdentifier = "myAccessibilityIdentifier"
// UI test code
app.otherElements["myAccessibilityIdentifier"].tap()

Get all options from Android Spinner using Appium

I am trying to pull all the options off an android spinner, using Appium. With Selenium, you can use the Select object and do something like getOptions (I forget the exact syntax). I need the text from all the options in the spinner.
Considering the spinner options are accessible through Appium. Getting all the values of the options on the spinner shall work as follows :
List<WebElement> spinnerList = driver.findElements(getBy("identifier")); //where identifier would vary on how you can access the elements
String spinnerListElementText[index]; //e.g. to store Text of all the options
for (int index = 0; index < spinnerList.size(); index++) {
String spinnerListElementText[index] = spinnerList.get(index).getText();
}
In Appium, there are test frameworks called Uiautomator, Uiautomator2 and Espresso, respectively. The thing that you were trying to get is not provided by Uiautomator or Uiautomator2 test frameworks. The only way with those frameworks is to click the spinner and get page source of spinner with visible elements. You could try to use Espresso framework. This is the reason:
Uiautomator: It is a test framework which provides a black-box testing for developers. It means that you can not get internal codes of the application.
Espresso: It is a test framework which provides a grey-box testing for developers. It means that you can get internal codes of the application, find elements which are not visible in the page (off-screen elements).
Try to use Espresso framework in Appium.

How to test right-to-left language in iOS XCTest unit tests?

Is there a way to switch an XCTest unit test into the right-to-left mode to test Arabic version of the app where sentences are written from right to left of the screen? My app code logic behaves differently based on language direction. I would like to verify this functionality in a unit test. What I need to do is to switch the app into the right-to-left language mode from an XCTest unit test case.
One can run the app in the right-to-left mode by changing the Scheme's Application language settings to Right-to-left Pseudolanguage. Is there a way to do similar thing in a unit test?
My imperfect solution
I ended up changing semanticContentAttribute of a view under test to .ForceRightToLeft. It does what I need to do. It does not feel like a very clean approach though. Firstly, it only works in iOS 9. Secondly, it looks like I am tinkering with my app views on a low level from the unit test. Instead, I would prefer to switch the whole app's language to right-to-left if it is possible.
class MyTests: XCTestCase {
func testRightToLeft() {
if #available(iOS 9.0, *) {
let view = UIView()
view.semanticContentAttribute = .ForceRightToLeft
// Test code involving the view
}
}
}
There's no easy way to do this right now with testing/UI testing besides passing in environment flags or setting the semanticContentAttribute as you are doing now. Filing a bug to Apple is highly recommended.
You can also change the device language & region in the scheme. This means you'll need separate schemes for the various LTR/RTL tests you want to run:
Xcode even provides pseudo-languages for extra-long string & RTL testing.
You can detect the writing direction via
let writingDirection = UIApplication.sharedApplication().userInterfaceLayoutDirection
switch writingDirection {
case .LeftToRight:
//
case .RightToLeft:
//
default:
break // what now? You are obviously using iOS 11's topToBottom direction…
}
To set different languages and locales on startup this might be a proper solution.
What you are looking for is Automated UI-Testing
This example JavaScript code changes the device orientation for example:
var target = UIATarget.localTarget();
var app = target.frontMostApp();
//set orientation to landscape left
target.setDeviceOrientation(UIA_DEVICE_ORIENTATION_LANDSCAPELEFT);
UIALogger.logMessage("Current orientation now " + app.interfaceOrientation());
//reset orientation to portrait
target.setDeviceOrientation(UIA_DEVICE_ORIENTATION_PORTRAIT);
UIALogger.logMessage("Current orientation now " + app.interfaceOrientation());
For testing, if your layout has changed to RTL or LTR you could try to access specific UI Elements and check their content against an expected content. So here is another example to check the contents of a TableViewCell from the official docs:
The crux of testing is being able to verify that each test has been performed and that it has either passed or failed. This code example runs the test testName to determine whether a valid element recipe element whose name starts with “Tarte” exists in the recipe table view. First, a local variable is used to specify the cell criteria:
var cell = UIATarget.localTarget().frontMostApp().mainWindow() \
.tableViews()[0].cells().firstWithPredicate("name beginswith 'Tarte'");
Next, the script uses the isValid method to test whether a valid element matching those criteria exists in the recipe table view.
if (cell.isValid()) {
UIALogger.logPass(testName);
} else {
UIALogger.logFail(testName);
}
If a valid cell is found, the code logs a pass message for the testName test; if not, it logs a failure message.
Notice that this test specifies firstWithPredicate and "name
beginsWith 'Tarte'". These criteria yield a reference to the cell for
“Tarte aux Fraises,” which works for the default data already in the
Recipes sample app. If, however, a user adds a recipe for “Tarte aux
Framboises,” this example may or may not give the desired results.
If you want to test a specific scheme:
Executing an Automation Instrument Script in Xcode
After you have created your customized Automation template, you can execute your test script from Xcode by following these steps:
Open your project in Xcode.
From the Scheme pop-up menu (in the workspace window toolbar), select Edit Scheme for a scheme with which you would like to use your script.
Select Profile from the left column of the scheme editing dialog.
Choose your application from the Executable pop-up menu.
Choose your customized Automation Instrument template from the Instrument pop-up menu.
Click OK to approve your changes and dismiss the scheme editor dialog.
Choose Product > Profile.
Instruments launches and executes your test script.

iOS - Finding elements by name in instruments?

How can I find a textfiled by name in automation testing?
I am able to get the element by index which is not helpfull.
In my nib the accessibility label of the button is set to "btn", but when executing that line of code it throws an exception because it can't find the button. Any suggestions?
var target = UIATarget.localTarget();
var mainWindow = target.frontMostApp().mainWindow();
// This does not work
mainWindow.buttons()["btn"].tap();
// This works
mainWindow.buttons()[0].tap();
How about recording the event within the Javascript editor (from the device). That usually shows you the options.
Or you can log the element tree by sending
rootView.logElementTree();

Resources