I have been trying to use Calabash to test some simple iOS apps. I was able to run the app on the simulator and all steps work fine. Now I want to test it on an iOS device, but I am having some problems. I run this command with all the necessary info:
BUNDLE_ID=... DEVICE_TARGET=... DEVICE_ENDPOINT=http://MY_IP:37265 cucumber
After a while the app opens in the iPhone and I get the usual information on console:
Feature: Running a test
As an iOS developer
I want to have a sample feature file
So I can begin testing quickly
Scenario: Example steps # features/my_first.feature:6
But it stays there forever. I left it for a while and it does nothing.
I have tried Appium, as well, for testing and it worked fine both on the simulator and on the iPhone, so I guess is not a problem with Profiles and the device itself.
What could be causing this issue?
My setup:
- Mac OS X 10.10.3
- Xcode Version 6.3 (6D570)
- iPhone 4 - iOS 7.1.2 (11D257)
- calabash-cucumber (0.14.0)
- cucumber (1.3.19)
UPDATE
Trying curl http://192.168.200.168:37265/version (as suggested by #alannichols) I get:
Failed to connect to 192.168.200.168 port 37265: Operation timed out
But I got my test to move to the first step executing the -cal target from Xcode (on the device) and running the command:
NO_LAUNCH=1 DEVICE_TARGET=... DEVICE_ENDPOINT=http://192.168.200.168:37265 cucumber
Still I am getting an error, but this time is a timeout error:
Operation timed out - connect(2) for "192.168.200.168" port 37265 (Errno::ETIMEDOUT)
Does it mean that I am reaching the iPhone by somehow I am not able to send requests?
If the app opens and nothing happens, it's usually to do with the device endpoint setting being incorrect. The app opens based on the bundle ID, but then calabash tries to interact with the app via the device endpoint.
Try running the app in the calabash console and if it behaves the same way as in your tests it'll be an issue with the device endpoint. Some ideas of things to check are that you're on the same network on the phone and the machine the test is running on, whether your vpn is interfering with the communication somehow or whether instruments is open on the test machine.
When the app is open you should be able to run
curl http://your.ip.address:37265/version
and get a response with the version number of calabash. If you can't get that then calabash won't be able to do anything in your tests.
Related
I am using Xcode Version 13.4.1 (13F100) on a MacBook Pro with macOS Monterey 12.4 and programming in Objective-C.
I would like to log debug messages when my app is running on the iPhone device AND is NOT attached to Xcode. I tried OSLog like described here: https://www.avanderlee.com/debugging/oslog-unified-logging/
However when I open the Console App and select my connected iPhone after running the app the log is empty and I have to explicitly click on "Start streaming" only then I can see my log when I start the app again.
I would like to log when the iPhone is not connected and the app is running and then close the app and connect the iPhone and see the log from the app run. I don't want live streaming but look into past logs.
Is there a way to do that?
I wrote to Apple Developer Support and they said the following:
If you log in your code using OSLog, then you can retrieve those
messages using the Terminal command ‘log’ later on.
For example, with the device named “iPhone" connected to the Mac, you
can retrieve the log info from the last 3 minutes with:
> % sudo log collect --device-name iPhone --last 3m
You can then use Console to look through the saved logs, or you can
parse out the log data from the command line with the ‘log' command.
See its man page for usage.
Here a link on how to use OSLog:
https://www.avanderlee.com/debugging/oslog-unified-logging/
In Objective-C this worked for me:
#import <os/log.h>
os_log(OS_LOG_DEFAULT, "Some message");
I followed Apple Developer Support's advice and it worked for me.
I just updated to iOS 10 on my test phone and Xcode 8. I ran the code migration and was running tests successfully on my simulated devices.
However, once I started trying to test on my real device things stopped working smoothly. After running my Sprite Kit game for around 5 - 10 seconds it would crash and render the phone inoperable until I unplugged it from the computer. After I unplug it the game runs as normal and is still really smooth.
This are the startup notifications printed to the console:
2016-09-15 22:20:49.490959 AppName[411:38688] [DYMTLInitPlatform] platform initialization successful
2016-09-15 22:20:49.684189 AppName[411:38634] Metal GPU Frame Capture Enabled
2016-09-15 22:20:49.684758 AppName[411:38634] Metal API Validation Enabled
This is the error I receive after the app crashes when the phone is plugged in:
/BuildRoot/Library/Caches/com.apple.xbs/Sources/Metal/Metal-85.83/ToolsLayers/Debug/MTLDebugRenderCommandEncoder.mm:130: failed assertion `indexBufferOffset(12) + (indexCount(480) * 2) must be <= [indexBuffer length](960).'
(lldb)
try to disable Metal verification by editing the scheme:
(Command-Option-R)
go to the Options tab.
Set Metal API Validation to Disabled.
This worked for me...
I can resolve this, unchecking "Debug executable" in the Scheme
Command-Option-R
"Wait for executable to be launched"
manual run
works for me from time to time.
I've iOS app that needs some privileges (GPS, Push notifications).
When app starts for a first time iOS asks user if they're ok with granting those permissions to application.
I've written some UITests and want to automate running them on locally connected iPhone.
The problem is that I cannot override permissions questions and my tests fails.
I found out that application deployed by IDE (Xamarin Studio) will ask for permissions, but application deployed via UITests will not.
So I tried with .AppBundle(path_to_app) but it says this is only valid for deploying to Simulator.
SetUp : System.Exception : This app bundle is not valid for running on
a simulator. To fix this issue please ensure that your target device
is a simulator. DTPlatformName is 'iphoneos', not 'iphonesimulator' in
the apps Info.plist.
Like it's trying to deploy iPhone app to Simulator. But Target in Xamarin Studio is set to real device.
I tried to add .DeviceIdentifier. When Used with .InstalledApp it was starting up (still asking for permissions).
But when I used DeviceIdentifier and AppBundle there was the same error as above.
My tests works fine on Test Cloud. They work fine on Simulator.
They work fine when I deploy to device manually, start app and approve permissions then run UI tests.
What I cannot achieve is to make UITests override permissions questions on real device.
Anyone made this work?
Last thing is that I found is in documentation for AppBundle method
"Will force a run on simulator"
https://developer.xamarin.com/api/member/Xamarin.UITest.Configuration.iOSAppConfigurator.AppBundle/p/System.String/
So I may be doomed with the task but maybe someone knows a workaround?
You can dismiss system dialogs with UITest by using InvokeUIA. The test below works by tapping the "OK" button of an iOS system alert:
[Test]
public void AppLaunches ()
{
app.Screenshot ("First screen.");
app.InvokeUia ("uia.query('[:view {:marked \"OK\"}]')");
app.InvokeUia ("uia.tapMark(\"OK\")");
}
A working sample app & UITest is also here:
https://github.com/King-of-Spades/InvokeUia-for-System-Dialogs
Warning about system dialogs in Test Cloud
The reason that you don't see this issue in Test Cloud is because Test Cloud automatically dismisses the system alerts; so usually you don't have to worry about it. However, if your alert launches too soon; so that it appears before the automation has fully started your app, then it will be unable to detect & dismiss the alert and cause your test to fail.
So you want to make sure that when running your app in Test Cloud that the request for permissions are delayed, or you can even deactivate them if they aren't explicitly needed for a particular test. More information is available in this Calabash guide: https://github.com/calabash/calabash-ios/wiki/Managing-Privacy-Alerts%3A--Location-Services%2C-APNS%2C-Contacts
(Even though it's Calabash, you can use the same strategy in UITest; albeit with a C# syntax.)
Update for Xcode 8 / iOS 10
Xcode 8 / iOS 10 removed UIAutomation, so the InvokeUIA workaround will only continue to be possible if you're using Xcode 7 and iOS 7-9. References:
UITest: https://developer.xamarin.com/guides/testcloud/UITest/xcode7/
Calabash: https://developer.xamarin.com/guides/testcloud/calabash/xcode7/#Automation_API
For real devices you dont need any of those.
{
app = ConfigureApp
.iOSAppBundle
.StartApp();
}
this piece of code is good enough, if you are connecting real device to the system, then select that before running.
We use this to execute the UI tests on iPhones:
ConfigureApp.iOS.InstalledApp("com.appcenter.UITestDemo").StartApp();
InstalledApp requires you to build an IPA using the Debug
configuration & a valid provisioning profile, and preinstalling it on
the target device.
https://github.com/microsoft/appcenter-Xamarin.UITest-Demo/blob/main/UITestDemo.UITest/AppInitializer.cs
To confirm system dialogs we use this:
private Query ConfirmLocalNetworkPermissionDialogButton
=> AppInitializer.Platform == Platform.iOS
? new Query(x => x.ClassFull("_UIAlertControllerActionView").Marked("OK"))
: x => x.Class("AppCompatButton").Marked("button1");
Running calabash-ios cucumber for simple iPhone app. Simulator is working perfectly. When trying on physical device, app loads, crashes, then loads again, and crashes again, on a loop.
Error message:
Unable to start. Make sure you've set APP_BUNDLE_PATH to a build supported by this simulator version
Calabash::Cucumber::Launcher::StartError: Time out waiting for UIAutomation run-loop Error while writing to fifo. RunLoop::Fifo::NoReaderConfiguredError.
Logfile /var/folders/9s/yttj4cz93m98hqs6bfw2x8xr0000gn/T/run_loop20150522-12657-dr3m49/run_loop.out
2015-05-22 19:58:11 +0000 Fail: An error occurred while trying to run the script.
Instruments Trace Complete (Duration : 2.594758s; Output : /var/folders/9s/yttj4cz93m98hqs6bfw2x8xr0000gn/T/run_loop20150522-12657-dr3m49/trace.trace)
to run the script. UIAScriptAgentSignaledException
(Calabash::Cucumber::Launcher::StartError)
/Library/Ruby/Gems/2.0.0/gems/calabash-cucumber-0.14.3/lib/calabash-cucumber/launcher.rb:766:in `new_run_loop'
/Library/Ruby/Gems/2.0.0/gems/calabash-cucumber-0.14.3/lib/calabash-cucumber/launcher.rb:631:in `relaunch'
Steps:
loaded app-cal on device
ran in command-line using:
$ DEVICE_TARGET=udid BUNDLE_ID=com.company.app DEVICE_ENDPOINT=http://ip:37265
Error while writing to fifo. RunLoop::Fifo::NoReaderConfiguredError
The probably cause of this is that the device is not enabled for UIAutomation. The device must be enabled for development. A device is enabled for development if it appears in Xcode's Device's window (Shift + Command +2) and you must enable on-device testing manually by visiting the device's Settings.app > Developer tab.
If you have recently upgraded the iOS version we recommend that you check the UIAutomation settings again and perform a restart. Often, after an upgrade, Xcode cannot download debugging information from the device. Look at Xcode's Device's window. The device must appear without any warnings.
Sometimes devices get into a bad state and Xcode cannot communicate with it. We recommend trying to interact with the device with Apple's Instruments to clear this bad state.
All this information and more can be found on the Calabash iOS Hot Topics page. https://github.com/calabash/calabash-ios/wiki/Hot-Topics#6-errnoeintr-interrupted-system-call
Tool: IBM Mobilefirst TestWorkbench 8.6.0.1
OS: Windows 7
Device: iPhone 6 Version: 8.1.2
Laptop and iPhone are on same Network.
Click on Add Devices icon in Mobile Devices tab. Test Workbench URL pops up.
Entered the URL in my Safari browser in iPhone.
IBM Rational Test Workbench Mobile Client opens in my browser.
Problem
The Device does not show up in Mobile Devices Tab.
Able to record a Test in my iPhone6. After recording, getting a pop up in Test Workbench which says "Click here to generate Test Recordings". When i click that i get the below error in Error log
Plug-in: com.ibm.rational.test.lt.recorder.moeb
Message: CRRTWM1201E Unexpected exception while performing test generation.
Exception Stack Trace:
java.io.IOException: Unable to access device used for recording.
at com.ibm.rational.test.lt.recorder.moeb.ui.wizards.CreateTestFromRecordingLogsWizard.getSessionProducer(CreateTestFromRecordingLogsWizard.java:150)
at com.ibm.rational.test.lt.recorder.moeb.ui.wizards.CreateTestFromRecordingLogsWizard.createRecordingSession(CreateTestFromRecordingLogsWizard.java:128)
at com.ibm.rational.test.lt.recorder.moeb.ui.wizards.CreateTestFromRecordingLogsWizard.access$0(CreateTestFromRecordingLogsWizard.java:125)
at com.ibm.rational.test.lt.recorder.moeb.ui.wizards.CreateTestFromRecordingLogsWizard$1.run(CreateTestFromRecordingLogsWizard.java:91)
at org.eclipse.swt.custom.BusyIndicator.showWhile(BusyIndicator.java:70)
at com.ibm.rational.test.lt.recorder.moeb.ui.wizards.CreateTestFromRecordingLogsWizard.performFinish(CreateTestFromRecordingLogsWizard.java:86)
at org.eclipse.jface.wizard.WizardDialog.finishPressed(WizardDialog.java:853)
at org.eclipse.jface.wizard.WizardDialog.buttonPressed(WizardDialog.java:438)
at org.eclipse.jface.dialogs.Dialog$2.widgetSelected(Dialog.java:619)
at org.eclipse.swt.widgets.TypedListener.handleEvent(TypedListener.java:248)
at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:84)
at org.eclipse.swt.widgets.Display.sendEvent(Display.java:4353)
Problem Guesses: Had instrumented the iOS app in Mac with jdk1.6
Presently uploaded the instrumented iOS app in Test Workbench in Windows Laptop which has jdk1.7. Could this be the problem ?
No the jdk should not be the issue.
The fact that your device doesn't appear in the mobile devices tab is not normal. This seems to be the origin of the issue.
The version 8.6.0.1 seems to be old... I recommand first that you update to the last one (8.7), re-instrumente the app and try again please ?
Thanks
Dominique