How to successfully run iOS simulator from Bamboo agent script - ios

I'm trying to run tests using xcodebuild through a Bamboo job script task. It opens the simulator, but it stays black and nothing happens, and the script fails due to timeout. Running the identical xcodebuild script goes fine via terminal. Here's the only related log message I was able to find:
Unable to connect to SimulatorBridge
Simulator[30414] <Error>: Unable to determine device.
Things I've already tried:
removing simulator and adding new ones
checking that bamboo agent is running under default user (which happens to be admin)
reseting simulator in advance using xcrun simctl erase

running open -a /Applications/Xcode.app/Contents/Developer/Applications/Simulator.app/ does the job. source

Related

Xcode 14 is not getting attached to Simulator in Debug mode

While debugging any iOS application, Xcode builds and run successfully, also launches a simulator but it is not able to attach debugger from Xcode 14 to Simulator iOS app.
It throws an error in Xcode:
Xcode console says:
Could not attach to pid : “75997” Domain: IDEDebugSessionErrorDomain
Code: 3 Failure Reason: attach failed (Not allowed to attach to
process. Look in the console messages (Console.app), near the
debugserver entries, when the attach failed. The subsystem that
denied the attach permission will likely have logged an informative
message about why it was denied.) User Info: {
DVTRadarComponentKey = 855031;
IDERunOperationFailingWorker = DBGLLDBLauncher;
RawUnderlyingErrorMessage = "attach failed (Not allowed to attach to process. Look in the console messages (Console.app), near the
debugserver entries, when the attach failed. The subsystem that
denied the attach permission will likely have logged an informative
message about why it was denied.)"; }
Tried with re-installing Xcode and Command line tools, but issue persists.
Steps:
Build and run app with Debug executables true
Simulator gets launched
Error on Xcode and it gets disconnected from simulator
Is there a possibility that it can be blocked by any other app, if so how to identify ?
I solved the issue by doing the steps below:
SOLUTION 1:
Edit the scheme
Select Run
Select Build Configuration "Debug"
Select "Wait for the executable to be launched"
Close
Run the app
see the screenshot here
You will receive an error message when you run the app again.
Close the app.
Edit the scheme again and change "Wait for the executable to be launched" to "Automatically".
Now your simulator will work as expected.
SOLUTION 2:
Delete your app folder in Derived Data
/Users/[your username]/Library/Developer/Xcode/DerivedData/
Search your app name folders and delete them.
SOLUTION 3:
Open the terminal and run:
sudo DevToolsSecurity -enable
This command enables the Developer Mode. After running the command you will receive this message: [Developer mode is now enabled].
Sometimes it's happen, when you are trying to build too fast after a build has started.
First of all, quit your current simulator, delete derived data and clean the project, wait until whole processing is complete. Then do some changes / editing if required (as we clean all data) and try running on a different simulator.
Check certificates in key-chain access, if you found any untrusted certificates in key-chain access remove it.
Also if you are using beta version just make sure it is downloaded from Apple official website.
Came here to share my (strange) solution if anybody crosses the same problem with no answers working. In my case, no solution from this nor other SO answers to similar questions worked (cleaning, rebuilding project, updating, clearing derived data..)
My setup is:
Macbook M1 2022 + Xcode Version 14.2 (14C18)
This Xcode version could not attach to any of my apps (fresh new or previously developed) on different simulators (11, 12, 14) with iOS versions 13.7 - 16.2.
Editing scheme solution from answer above was not working also so I ended up in leaving scheme as:
Build configuration: Release
Debug executable Deselected(!)
Launch Automatically
And magically debugging app in Xcode started working even though it's release version with debug option deselected.. Selecting Debug executable again makes Xcode not possible in attaching to app process
Just to clarify: above steps actually makes app to run as in DEBUG mode - Xcode attaches to pid, I can do console logs and breakpoints. This Xcode version behaves opposite than options selected

Self Hosted Agent on macOS not working with Azure Pipeline

This is an issue I've been facing for the past week.
I have an M1 Mac Mini setup as a self-hosted agent to run iOS related builds/archives through Azure Pipelines.
I'm able to build/archive locally on the machine itself using terminal. Mobile provisioning and code signing are setup and working fine.
When I run the agent interactively using run.sh, I am able to run the pipeline successfully. When I try running the agent as a launchd service using ./svc.sh install -> ./svc.sh start, it always fails with the same error:
** ARCHIVE FAILED **
The following build commands failed:
ValidateEmbeddedBinary /Users/<user>/Library/Developer/Xcode/DerivedData/<project>/Build/Intermediates.noindex/ArchiveIntermediates/<app-name>/BuildProductsPath/Debug-iphoneos/<app-name>.app (in target '<app-name>' from project '<project>')
(1 failure)
##[error]Bash exited with code '65'.
The command I'm using to archive is
xcodebuild -workspace ~/repos/<project>/<project>.xcworkspace -scheme <app-name> -allowProvisioningUpdates clean archive -archivePath build/<app-name> -destination generic/platform=iOS
Again, this command works when I run it locally and when I run the agent interactively.
I removed app/project names for privacy reasons.
I've searched high and low as to why this error is occurring and I've tried virtually all suggestions to fix it. For the life of me I don't know why the command would work locally and when the agent is run interactively, but does not work when the agent is run as a launchd service.
Edit: I’ve noticed a pattern: the archive fails with that error after every successful run. In other words the command successfully archives but when I run it again, it fails. If I run it once more, it’s successful. This occurs when running it through Azure Pipelines and locally via terminal. I have no words!

How to solve 'Test runner exited before starting test execution' error for iPhone Devices

Context -
We have E2E Testing running on the CI on the iPhone devices.
The script performs the following steps -
Build the Test runner app from code.
Extracts the app file from the IPA.
Re-codesign the app with the required Provision Profiles.
Creates a .xctestrun file
Uses the xcodebuild command like the following to run the test -
xcodebuild -sdk iphoneos -destination id=<device_id> -xctestrun <path_to_xctestrun_file> -resultBundlePath <path_to_test_bundle_file> test-without-building -only-testing:E2ETestsBundle.xctest/<test_filter>
The above command installs the runner and target app into the device and starts executing the test.
The apps get installed and uninstalled at each run during a single execution.
Problem -
During the execution, the xcodebuild CLI tool, suddenly loses connection to the device and is not able to install the apps into the device and then results in the following error -
Testing failed:
E2ETestsBundle.xctest:
E2ETestsBundle-Runner encountered an error (Early unexpected exit, operation never finished bootstrapping - no restart will be attempted. (Underlying Error: Test runner exited before starting test execution. If you believe this error represents a bug, please attach the log file at /var/folders/1r/0xlnbmzd0yj3m_cc7p95yb4r0000gn/T/tmpl7d9p1s6/testResults.bundle/Staging/1_Test/Diagnostics/E2ETestsBundle-Configuration-Test Scheme Action-Iteration-1/E2ETestsBundle/Session-E2ETestsBundle-2022-04-13_205520-muaMCx.log))
** TEST EXECUTE FAILED **
Points to note -
The xcodebuild CLI tool will keep on failing with the error once it occurred until recovered.
Even while the xcodebuild is in a failed state, the devices are still accessible via Xcode IDE and other CLI tools like libimobiledevice and are able to install both the test runner and target applications.
This is occurring intermittently across all devices models and across subversions of iOS 14 and 15.
Recovery -
The only way to recover is to reboot the device.
Even after a reboot, the devices start facing this exception again after a few days of execution.
As we can’t keep on rebooting the device every time before execution, we need a solution to fix this problem.
Any help would be appreciated!!

Calabash-ios: Run test physical device installed ipa without xcodeproj

I am new with calabash-ios and have a dummy question is:
How possibly could I run a calabash test with real device which have installed ipa file (included calabash framework) and without xcodeproj?
Problem is I am doing QA and Dev only gives me ipa file to test without source code.
UPDATE:
I am able to run the ipa file in console and cucumber by adding BUNDLE_ID. And make sure the CFNetwork.framework is including in app framework to establish the connection.
But I got another problem to run the scenario:
1. When I run the command below:
DEVICE_ENDPOINT=http://192.168.1.9:37265 NO_LAUNCH=1 BUNDLE_ID=com.example.appname DEVICE_TARGET=UDID cucumber
Then I got this error:
A JSON text must at least contain two octets! (JSON::ParserError)
features/my_first.feature:8:in `Then I touch "LOG-IN/ CREATE ACCOUNT"'
all other steps was skipped
Then I run the console code
DEVICE_ENDPOINT=http://192.168.1.9:37265 NO_LAUNCH=1 BUNDLE_ID=com.example.appname DEVICE_TARGET=UDID calabash-ios console
Then I run server_version
I got reply back that said connection SUCCEED
But when I run start_test_server_in_background
It took me too long to wait and I see the app my physical device kept crashed and launch again and again until it stop as:
Unable to start. Make sure you've set APP_BUNDLE_PATH to a build supported by this simulator version
Calabash::Cucumber::Launcher::StartError: Calabash::Cucumber::Launcher::StartError:
"Timed out waiting for UIAutomation run-loop Error while writing to fifo. RunLoop::Fifo::NoReaderConfiguredError.
2015-10-01 10:49:41.634 instruments[15640:10558880] WebKit Threading Violation - initial use of WebKit from a secondary thread.
2015-10-01 10:49:42.574 instruments[15640:10558978] Attempting to change event horizon while disengage
2015-10-01 10:49:42.577 instruments[15640:10558877] Attempting to change event horizon while disengage
Instruments Trace Error : Target failed to run: Permission to debug app
Anyone can help please! I tried to search some solution but they almost issue on simulator and they contain xcodeproj. As a QA, I dont have the source code, only .ipa file. :(
Thanks
You should set 3 variables:
$ DEVICE_TARGET=<UDID> \
DEVICE_ENDPOINT=http://<ip>:37265 \
BUNDLE_ID=com.example.YourApp cucumber
The device and the host computer should be on the same network.
The .ipa must be installed on your device.
You can do this in Xcode by dragging-and-dropping in the Device Manager window or with a third-party tool like ideviceinstaller (available with homebrew).
You should never set NO_LAUNCH.
http://calabashapi.xamarin.com/ios/file.ENVIRONMENT_VARIABLES.html#label-NO_LAUNCH
You also need to enable you device for develop and turn on UIAutomation in the Settings.app > Developer.
Enabling device for development
RunLoop::Fifo::NoReaderConfiguredError
UPDATE In response to this question, I rewrote the Testing on Physical Devices Calabash iOS wiki page. Wow, that content was ancient!
As long as the file is build with calabash then that is not a problem. You just call cucumber with APP_BUNDLE_PATH to where ever your .ipa file is and should be all set.
But you should be aware that .ipa needs to be build to match the target you run the tests on.

unable to run scripts on simulator from cron tab

We are using sikuli tool for mobile automation on our MAC OSX 10.7.3
I have written a shell script to open simulator and to run scripts on simulator
/Users/ezprintsqa/Downloads/FoneMonkey/bin/iphonesim launch /Users/ezprintsqa/Documents/Xcode/DerivedData/SnapCards- btmjdowsgmixpvdyfxfvhxtoihab/Build/Products/Debug-iphonesimulator/SnapCardsFoneMonkey.app&
/Users/ezprintsqa/Downloads/Sikuli-IDE.app/sikuli-ide.sh -r /Users/ezprintsqa/Downloads/SikuliWorkSpace/SnapcardsTestSuite1.sikuli
When I execute from command line I can see simulator opening and scripts getting executed on simulator.
But when I do same thing using cron tab I can see only simulator opening but the tool can't identify the simulator now to run scripts, the tool throws error message that it is unable to identify the simulator.
When the tool is able to identify the simulator while running from command prompt why is it unable to identify while running from crontab?
Thanks
You are saying:
the tool throws error message that it is unable to identify the simulator.
Do I understand right, that the script throws a FindFailed exception, meaning, that some image (might be the visual identification of the simulator window) is not found on the screen within the standard waiting time of 3 seconds?
I this is the case, you should give more time to the simulator to start up in the crontab case, either by adding a wait(n) at the beginning of the Sikuli script or use wait(image, n) to wait longer for the image to appear

Resources