So I have recently stumbled across a "fun" thing for automation and test case runs in general - and that is that iOS is very persistent with the users cache and won't let you delete the apps cache.
The actual issue:
first test case runs fine and logs in the user
the second test case expects the user to be logged out | expects to run on a fresh app - without any left over cache from the previous user <- so because of that it FAILS
I have tried several solutions:
adding capabilities like noReset or/and fullReset
stopping the driver (deleting the app and reinstating it again)
basically everything I could find - I tried
I have managed to fix the issue for SIMULATOR but not for the REAL DEVICE.
If you are doing any appium webdriver automation for your mobile applications - you should have encountered this issue and had to deal with it somehow - would really appreciate your opinion on this topic! (Also I am using SauceLabs for the real devices)
P.s. I don't want to make silly workarounds like loging out the user after each test case - so the next one could start from a fresh app and so on. IMHO this would be a really bad practise | shot-lived workaround.
Edit - code examples.
Capabilities:
config.capabilities = [
{
platformName: 'iOS',
'appium:platformVersion': getRandomOsVersion(osVersions),
'appium:deviceName': 'iPhone .*',
'appium:automationName': 'XCUITest',
'appium:app': 'storage:filename=app.ipa',
'appium:autoAcceptAlerts': true,
'appium:connectHardwareKeyboard': true,
'appium:shouldTerminateApp': true,
'appium:newCommandTimeout': 240,
'sauce:options': {
build: buildName,
name: buildName,
allowTouchIdEnroll: false,
sauceLabsImageInjectionEnabled: false,
},
},
];
Test cases:
describe('Login functionality', () => {
beforeEach(async () => {
await driver.reset();
(await WelcomeScreen.getStartedButton).waitForDisplayed();
});
it('Login with user (finished profile)', async () => {
await WelcomeScreen.getStartedButton.click();
await LoginScreen.submitLogin(USER_CREDENTIALS.USER_WITH_FINISHED_PROFILE);
await expect(await FeaturesTourScreen.letsGetYouStarted).toBeDisplayed();
});
it('Login with user incorrect credentials', async () => {
await WelcomeScreen.getStartedButton.click();
await LoginScreen.submitLogin(USER_CREDENTIALS.INCORRECT_CREDENTIALS);
await expect(await LoginScreen.alertMesage).toBeDisplayed();
});
});
Real Device background
Real devices behave differently in comparison to Simulators. When you install an iOS app on a real device it needs to be resigned so it is allowed to install the app. Removing an app "might" remove all data that has been stored on the device / keystore, but data could still be attached to your iCloud account.
There are no Appium commands to remove the data stored in the keystore or in the app itself
WebdriverIO background
This is not even 100% specific for WebdriverIO, but when you run 2 tests (it) from a single file the end state of the first test will be the starting state of the second test. You can control that with a beforeEach/afterEach, but this will have no/less effect on real devices.
WebdriverIO spins up a single driver for every test file. In theory this should lead to a clean simulator / device when the second test-file is starting
Sauce Labs background
When you are using Sauce Labs you don't have the option to remove the app between the first and second it, because when you remove the app it needs to be resigned and this is not supported. Options like noReset|fullReset won't have an effect on iOS real devices.
If you need to have all data cleared then you need to:
or have a backdoor in your app to do this through Appium
run the second test case in a new spec file. WebdriverIO spins up a new instance for every test file and in between the app is removed and the device is cleaned
I would suggest you ask the developers of the application to add data cleanup logic in the code itself that runs when a specific launch argument is provided when starting the application. Then you can use the Appium capability processArguments to provide that argument to the app when starting the test and the app itself will cleanup the data. Example for providing arguments from Appium issues, Appium XCUITest driver official docs describing the capability
Related
Hi I try to write my first test for an Ionic App and I cant figure out howto do, as the documentation seems to miss some information.
ionic:
I have created a pipeline that builds an APK and and IPA file
I can run those files in the emulators (Android & iOS)
Appium
The server runs, and can connect to the local running Android Emulator
I've create the node.js project as documented in the getting started guide
The following works too:
async function main () {
const client = await wdio.remote(opts);
await client.deleteSession();
}
Result:
[Instrumentation] OK (1 test)
[Instrumentation] The process has exited with code
But now the documentation seems to miss a step, as it starts with (Automating hybrid apps):
driver
.contexts().then(function (contexts) { // get list of available views. Returns array: ["NATIVE_APP","WEBVIEW_1"]
return driver.context(contexts[1]); // choose the webview context
})
How do I get a driver instance in JavaScript, based on the client instance?
Also I have an "async function main ()" now. Is there no pattern to write it like async function test_01, test_02, or more meaningful?
I would like to launch Appium Inspector session to attach to the currently running Android app on my device. The problem I'm facing is that whenever I start the session, the running app is getting killed and launched again. Is there any way to avoid that?
I'm currently using the following config:
{
"appium:automationName": "UiAutomator2",
"platformName": "android",
"appium:app": "/my/path/to/apk",
"appium:noReset": true
}
Not sure, but those capabilities might help:
"appium:noReset": true,
"appium:autoLaunch": false,
Also, remove appium:app capability and try to run without it (possibly appium will just create an empty session without app and will wait for new instructions).
If it doesn't work, replace appium:app with:
"appium:appPackage": ...,
"appium:appActivity":...
Update
The final capabilities set
"appium:autoLaunch": false, // to disable app relaunch on Appium session start
"appium:noReset": true,
"appium:dontStopAppOnReset": true, // to prevent closing the app Appium Inspector's session stopped
"appium:app"
I can't open devtools in the built version of my electron app. Therefore i want to find another solution to log any errors that only occur in the production version.
Is there any good way to get some console.logs from an electron application if its already built?
Obviously i can debug the “development” version (running npm run dev) of my electron app by opening the chrome dev tools. But i can’t find any way to enable them inside my production application.
I am using the newsest version of electron-vue
Thanks for any help in advance.
Here's what worked for me on Mac.
In terminal type lldb path/to/build.app
In the opened debugger type run --remote-debugging-port=8315. It should open a window of your app.
Open Chrome at http://localhost:8315/
Click on the name of the app. For example, Webpack App.
If you don't see anything in the opened tab, focus on the window of your app.
Launch your Electron application with the --remote-debugging-port=8315 flag set and navigate to chrome://inspect/#devices in Chrome 80+. Then click Configure... and add localhost:8315 as a discovery server.
Then, wait for your Electron instance to appear in the devices list and click inspect.
Enabling the Chrome devtools in production can be done in various ways:
A environment variable:
E.g. under Windows set ELECTRON_ENV=development&& myapp.exe
Pass a special parameter to your app
E.g. myapp.exe --debug
Debug mode via user settings (if you have persistent settings)
Menu entry to open the devtools
Can be combined with 1.-3. to only have that menu entry when in debug mode
You can just check if 1.-3. are set and if they are, you simply open the devtools via mainWindow.webContents.openDevTools()
Personally I use a combination of 1., 3. and 4. I simply unlock a developer menu that allows me to open the devtools or opens the userdata folder for me.
If you also want to log critical errors, then electron-log looks like a rather popular option for electron.
On Mac just run open /Applications/WhatsApp.app --args --remote-debugging-port=8315 and then open http://localhost:8315
https://github.com/bytedance/debugtron
Debugtron is an app to debug in-production Electron based app. It is also built with Electron.
The way to do this is using the --remote-debugging-port flag.
Using Signal as an example, take the following steps:
start the application from the CLI
signal-desktop --remote-debugging-port
Open the debugging URL in a Google Chrome browser (in this case http://localhost:39733/), this will open a page with the app name on it .
Click the to open a screen were you can click around to use the app and see output in the devtools
Alternatively, you can open chrome://inspect/#devices in the Google Chrome browser and click "inspect" (underneath the app name) to open the same window
In my case, I had a runtime error crashing Electron before the web view was even shown. Chrome dev tools were not useful for debugging this kind of error.
Even stranger, the app loaded fine using the lldb commands:
lldb ./dist/mac/electron-quick-start-typescript.app
run --remote-debugging-port=8315
I managed to solve by writing the Node.js console log/error to a file. So I could see the console output:
import fs from 'fs';
import util from 'util';
// use a fixed path, to ensure log shows outside Electron dist
const logPath = `/Users/username/Sites/electron-server/debug-${isDev ? 'dev' : 'prod'}.log`;
const logFile = fs.createWriteStream(logPath, { flags: 'w' });
const logStdout = process.stdout;
console.log = function(...args) {
logFile.write(util.format.apply(null, args) + '\n');
logStdout.write(util.format.apply(null, args) + '\n');
}
console.error = console.log;
I found the error was a relative path issue. When the app is started in production mode the relative path did not point to the correct location.
I solved by using an absolute path, with Electron getAppPath() method:
- './renderer'
+ app.getAppPath() + '/renderer'
In the main/index.js at the end of section app.on('ready') just add:
mainWindow.webContents.openDevTools();
Just for debugging, when electron opens an empty window, but the development version works fine, this way is very helpful for me.
The answers above don't help for me. ( those with -remote-debugging-port)
put this code into your main.js or similar file, it will automatically open the dev-tool when started.
mainWindow.webContents.openDevTools()
June 2022 UPDATE:
Devtools can be explicitly enabled through on the BrowserWindow object.
mainWindow = new BrowserWindows({
...,
webPreferences: {
...,
devTools: true,
}
}
And then you can manually open it on load.
mainWindow.on('ready-to-show', () => {
if (!mainWindow) {
throw new Error('mainWindow is not defined')
}
mainWindow.show()
mainWindow.webContents.openDevTools()
})
I've iOS app that needs some privileges (GPS, Push notifications).
When app starts for a first time iOS asks user if they're ok with granting those permissions to application.
I've written some UITests and want to automate running them on locally connected iPhone.
The problem is that I cannot override permissions questions and my tests fails.
I found out that application deployed by IDE (Xamarin Studio) will ask for permissions, but application deployed via UITests will not.
So I tried with .AppBundle(path_to_app) but it says this is only valid for deploying to Simulator.
SetUp : System.Exception : This app bundle is not valid for running on
a simulator. To fix this issue please ensure that your target device
is a simulator. DTPlatformName is 'iphoneos', not 'iphonesimulator' in
the apps Info.plist.
Like it's trying to deploy iPhone app to Simulator. But Target in Xamarin Studio is set to real device.
I tried to add .DeviceIdentifier. When Used with .InstalledApp it was starting up (still asking for permissions).
But when I used DeviceIdentifier and AppBundle there was the same error as above.
My tests works fine on Test Cloud. They work fine on Simulator.
They work fine when I deploy to device manually, start app and approve permissions then run UI tests.
What I cannot achieve is to make UITests override permissions questions on real device.
Anyone made this work?
Last thing is that I found is in documentation for AppBundle method
"Will force a run on simulator"
https://developer.xamarin.com/api/member/Xamarin.UITest.Configuration.iOSAppConfigurator.AppBundle/p/System.String/
So I may be doomed with the task but maybe someone knows a workaround?
You can dismiss system dialogs with UITest by using InvokeUIA. The test below works by tapping the "OK" button of an iOS system alert:
[Test]
public void AppLaunches ()
{
app.Screenshot ("First screen.");
app.InvokeUia ("uia.query('[:view {:marked \"OK\"}]')");
app.InvokeUia ("uia.tapMark(\"OK\")");
}
A working sample app & UITest is also here:
https://github.com/King-of-Spades/InvokeUia-for-System-Dialogs
Warning about system dialogs in Test Cloud
The reason that you don't see this issue in Test Cloud is because Test Cloud automatically dismisses the system alerts; so usually you don't have to worry about it. However, if your alert launches too soon; so that it appears before the automation has fully started your app, then it will be unable to detect & dismiss the alert and cause your test to fail.
So you want to make sure that when running your app in Test Cloud that the request for permissions are delayed, or you can even deactivate them if they aren't explicitly needed for a particular test. More information is available in this Calabash guide: https://github.com/calabash/calabash-ios/wiki/Managing-Privacy-Alerts%3A--Location-Services%2C-APNS%2C-Contacts
(Even though it's Calabash, you can use the same strategy in UITest; albeit with a C# syntax.)
Update for Xcode 8 / iOS 10
Xcode 8 / iOS 10 removed UIAutomation, so the InvokeUIA workaround will only continue to be possible if you're using Xcode 7 and iOS 7-9. References:
UITest: https://developer.xamarin.com/guides/testcloud/UITest/xcode7/
Calabash: https://developer.xamarin.com/guides/testcloud/calabash/xcode7/#Automation_API
For real devices you dont need any of those.
{
app = ConfigureApp
.iOSAppBundle
.StartApp();
}
this piece of code is good enough, if you are connecting real device to the system, then select that before running.
We use this to execute the UI tests on iPhones:
ConfigureApp.iOS.InstalledApp("com.appcenter.UITestDemo").StartApp();
InstalledApp requires you to build an IPA using the Debug
configuration & a valid provisioning profile, and preinstalling it on
the target device.
https://github.com/microsoft/appcenter-Xamarin.UITest-Demo/blob/main/UITestDemo.UITest/AppInitializer.cs
To confirm system dialogs we use this:
private Query ConfirmLocalNetworkPermissionDialogButton
=> AppInitializer.Platform == Platform.iOS
? new Query(x => x.ClassFull("_UIAlertControllerActionView").Marked("OK"))
: x => x.Class("AppCompatButton").Marked("button1");
I am performing following steps
Set capabilities and launch ABC app. By providing app path
capabilities.setCapability("app", "/Users/changdeojadhav/Library/Developer/Xcode/DerivedData/ABC/Build/Products/Debug-iphonesimulator/ABC.app");
capabilities.setCapability("bundleId","com.abc.ABC-Demo");
Perform some actions
quit driver by driver.quit()
Set capabilities for Xyz app. And launch XYZ app
Perform some steps
quit driver by driver.quit()
relaunch ABC app as mentioned in step #1.
Expected is "App ABC should retain it's state" but ABC gets reset.
I have launched appium with --no-reset parameter.
Any idea about what I am missing here
Thanks
As far as I can tell, there is currently no solution to reopening the application after going to the home screen without clearing the app from cache.
In past versions of iOS/Appium, the solution was to do:
from appium import webdriver
driver = webdriver.Remote('http://0.0.0.0:4723/wd/hub', desired_caps)
driver.close_app()
app = driver.find_element_by_xpath('//UIAApplication/UIAWindow/UIAScrollView/UIAButton[#name="sampleApp"]')
app.click()
However this currently crashes Appium
I will update this question when I log a github issue for it.
The Appium help page says that it only supports multiple app testing in a single test session for Android without Selendroid:
iOS: Support for automating multiple apps in one session: No
Android:Support for automating multiple apps in one session: Yes (but not when using the Selendroid backend)
http://appium.io/slate/en/master/?ruby#toc_27
I'm guessing that is why you are having this issue, and it's most likely an Instruments/XCode issue.
I was able to relaunch the same app without it resetting its state with Appium 1.3.1 running with Xcode 6.1 on a Mac Mini running Mavericks. I did not try launching another app in between launches. I'm driving the automation from C#.
protected AppiumDriver GetAppiumDriver(bool forRestart = false)
{
DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.SetCapability("appium-version", "1.0");
capabilities.SetCapability("platformName", "iOS");
capabilities.SetCapability("platformVersion", "7.1");
capabilities.SetCapability("deviceName", "iPhone Simulator");
capabilities.SetCapability("app", _appPath);
capabilities.SetCapability("locationServicesEnabled", true);
if (forRestart)
{
capabilities.SetCapability("noReset", true);
}
AppiumDriver driver = new AppiumDriver(_serverUrl), capabilities, new TimeSpan(0, 5, 0));
return driver;
}
public void iOSMobileAppBasicUITest()
{
// Initially Launch the app with the noReset capability at its default value of false to ensure a clean starting point.
_driver = GetAppiumDriver(false);
//Shut down the app.
_driver.Quit();
// Launch the app again, this time with the noReset capability set to true.
_driver = GetAppiumDriver(true);
// Use _driver to do whatever UI automation is desired.
// Optional: Send the app to the background so that iOS does state preservation.
_driver.BackgroundApp(3);
// Close the app.
_driver.CloseApp();
// Alternative: _driver.Quit();
// Launch the app.
_driver.LaunchApp();
// Alternative: _driver = GetAppiumDriver(true);
...
As I known, Appium by default runs in fast reset mode, and it tries to clear app's data when the session ends (as a result of the invocation of quit() in this case). If your want to keep app's data, the option --no-reset should work for you.