Question: For this pull request, the tests do not pass on CircleCI but the tests pass locally. Why?
The CircleCI test output shows failures for all FBSnapshotTestCase tests. For example:
✗ testAdjustsFontSizeToFitWidth, ((comparisonSuccess__) is true) failed - Snapshot comparison failed: Error Domain=FBSnapshotTestControllerErrorDomain Code=1 "Unable to load reference image." UserInfo=0x7f85f36b0a50 {NSLocalizedFailureReason=Reference image not found. You need to run the test in record mode, NSLocalizedDescription=Unable to load reference image., FBReferenceImageFilePathKey=/Users/distiller/TTTAttributedLabel/Example/TTTAttributedLabelTests/ReferenceImages_32/TTTAttributedLabelTests/testAdjustsFontSizeToFitWidth#2x.png}
[…]
✗ testAttributedTruncationToken, ((comparisonSuccess__) is true) failed - Snapshot comparison failed: Error Domain=FBSnapshotTestControllerErrorDomain Code=1 "Unable to load reference image." UserInfo=0x7f85f35b06d0 {NSLocalizedFailureReason=Reference image not found. You need to run the test in record mode, NSLocalizedDescription=Unable to load reference image., FBReferenceImageFilePathKey=/Users/distiller/TTTAttributedLabel/Example/TTTAttributedLabelTests/ReferenceImages_32/TTTAttributedLabelTests/testAttributedTruncationToken#2x.png}
However, the same tests pass locally:
On CircleCI the tests use the ReferenceImages_32 directory, which does not exist:
/Users/distiller/TTTAttributedLabel/Example/TTTAttributedLabelTests/ReferenceImages_32/TTTAttributedLabelTests/testAdjustsFontSizeToFitWidth#2x.png
However, the images do exist in ReferenceImages_64. I expect the _64 directory to be used because the circle.yml file specifies to use the iPhone 6 simulator:
dependencies:
pre:
- xcrun instruments -w "iPhone 6 (8.3 Simulator)" || exit 0
override:
- sudo gem install cocoapods xcpretty obcd -N
- pod install --project-directory=Example
test:
override:
- set -o pipefail && xcodebuild -workspace 'Example/Espressos.xcworkspace' -scheme 'Espressos'
-sdk iphonesimulator -destination "platform=iOS Simulator,name=iPhone 6"
GCC_INSTRUMENT_PROGRAM_FLOW_ARCS=YES GCC_GENERATE_TEST_COVERAGE_FILES=YES clean test | xcpretty -c
--report junit --output ${CIRCLE_TEST_REPORTS}/junit.xml
What causes this?
How can I get the tests passing on CircleCI?
Possibly related: this FBSnapshotTestCase issue
Have you tried running the build without the cache? This often fixes these kinds of issues for me. To build without cache, just click the button at the top right.
Related
I have a jenkins server connecting to a remote mac mini through ssh to execute a shell script that has to build an IPA from a unity project.
When the shell script is executed locally on the mac mini everything goes fine. But when the shell script is ran from jenkins (with the exact same parameter and the same user) it fails codesigning the archive.
I will share with you the obfuscated shell script as well as the build log.
Thank you for your help in advance.
The shell script :
#!/bin/bash
# Consider directory paths initialized in parameter here
#
#
# Consider git cleaning / fetching commit here
#
#
# Consider environment / version and build name controls here
#
#
# Start Unity Build :
/Applications/Unity2017.4.10f1/Unity.app/Contents/MacOS/Unity -batchmode -quit -projectPath "$SOURCE_PATH" -executeMethod "BuildManager.BuildPlayer" -logFile "$BUILD_LOG_FILE" -buildEnvironment "$ENVIRONMENT" -buildPlatform "IOS" -buildPath "$TARGET_BUILD_DIR" -overrideVersion "$OVERRIDE_VERSION"
if [ ! -d "${TARGET_BUILD_DIR}/Unity-iPhone.xcodeproj" ]
then
echo "[ERR]Exporting unity project to Xcode failed."
exit 1
else
echo "Build successfull"
fi
#
#
# Consider initializing a param for the provisioning profile file path
#
#
# Consider initializing a param for the plist file path
cd $TARGET_BUILD_DIR
# archive generated xcode project
xcodebuild -scheme "Unity-iPhone" -archivePath "${DEPLOY_DIR_ROOT}/${BUILD_NAME}_${FILE_FORMAT_VERSION}/archive.xcarchive" -sdk iphoneos -configuration Release PROVISIONING_PROFILE="${PROVISIONING_PROFILE_PATH}" archive
if [ $? != 0 ]; then
echo "FAILED ARCHIVING XCODE PROJECT"
exit 1
fi
# export ipa from archive
xcodebuild -exportArchive -archivePath "${DEPLOY_DIR_ROOT}/${BUILD_NAME}_${FILE_FORMAT_VERSION}/archive.xcarchive" -exportOptionsPlist "${PLIST_PATH}" -exportPath "${DEPLOY_DIR_ROOT}/${BUILD_NAME}_${FILE_FORMAT_VERSION}"
if [ $? != 0 ]; then
echo "FAILED EXPORTING IPA FROM ARCHIVE"
exit 1
fi
#
# Section reserved for uploading the ipa to relevant remote storage
#
exit 0
So everything works like a charm (even the build can be installed on a device) when the shell script is ran locally from the terminal on the mac mini.
When it comes to run the shell script remotely through ssh it fails to codesign the archive. The user used over ssh is the same than the one used locally to run the script.
Here is the error :
CodeSign
/#######/Library/Developer/Xcode/DerivedData/Unity-iPhone-#########/Build/Intermediates.noindex/ArchiveIntermediates/Unity-iPhone/InstallationBuildProductsLocation/Applications/#########.app
(in target: Unity-iPhone) cd /#########/xcodeProjPath export
CODESIGN_ALLOCATE=/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/codesign_allocate
Signing Identity: "#########" Provisioning Profile: "iOS Team
Provisioning Profile: #########"
(#########)
/usr/bin/codesign --force --sign ######### --entitlements
/#########/Library/Developer/Xcode/DerivedData/Unity-iPhone-#########/Build/Intermediates.noindex/ArchiveIntermediates/Unity-iPhone/IntermediateBuildFilesPath/Unity-iPhone.build/Release-iphoneos/Unity-iPhone.build/#########.app.xcent
--timestamp=none /#########/Library/Developer/Xcode/DerivedData/Unity-iPhone-#########/Build/Intermediates.noindex/ArchiveIntermediates/Unity-iPhone/InstallationBuildProductsLocation/Applications/#########.app
/#########/Library/Developer/Xcode/DerivedData/Unity-iPhone-#########/Build/Intermediates.noindex/ArchiveIntermediates/Unity-iPhone/InstallationBuildProductsLocation/Applications/#########.app:
errSecInternalComponent
Command CodeSign failed with a nonzero exit code
** ARCHIVE FAILED **
I m kind of stuck right now since all my attempts didn't work at all ...
Thank you in advance for your help.
EDIT:
mac mini on macOS High Sierra Version 10.13.6 (17G65)
xcode Version 10.0 (10A255)
Ok so for all of you guys struggling around this tricky subject ( totally invisible if we don't know enough about macOS ) there is kind of security system that still makes the difference between a local user and a remote access to a user.
So the keychain handling your keys and certificates ( used by codesign to sign your build ) is not usable out of the box for the remote user. Its needs to be unlocked first !!!
To know about the available keychains on your system just type in on your terminal :
security list-keychains
You should see something like :
"/Users/'YOURUSER'/Library/Keychains/login.keychain-db"
"/Library/Keychains/System.keychain"
And you guessed it right there, you have to unlock the keychain of your user ! Juste run this :
security unlock-keychain -p 'USER_PASSWORD' 'PATH_TO_USER_KEYCHAIN'
And that's it.
N.B:
Please let me know if I understood something wrong about all this.
My tests do not start executing, and always timeout every time I try to use an xcodebuild command.
The command that I used is the following:
xcodebuild -workspace App.xcworkspace -scheme 'AppName' -sdk iphonesimulator -configuration 'UI_Automation' CODE_SIGN_STYLE='Manual' CODE_SIGN_IDENTITY='iPhone Developer: John Smith (XXXXXXXX)' PROVISIONING_PROFILE_SPECIFIER='John Smith PP Name' DEVELOPMENT_TEAM='ABC Company Apple DEV' -destination 'platform=iOS Simulator,name=iPhone 8,OS=11.2' -destination-timeout 600 -only-testing:AppUITests clean test
It goes through a process where the app is built, but then the tests do not start executing. As a matter of fact, the Simulator does not even start, and I get the following error.
Testing failed:
Timed out waiting for AX loaded notification If you believe this error represents a bug, please attach the log file at /var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/com.apple.dt.XCTest/IDETestRunSession-60333F56-66CA-4C34-8460-3846DCF59C14/AppUITests-F74C4FDD-17EE-44AD-A2BA-DDB1BC0A1D7E/Session-AppUITests-2017-12-28_150211-qwlao1.log
Does anyone know how to fix this? I have been racking my brain around this for the past 2 weeks and can't figure it out.
xcode build command changed since xcode 9. So you have to change your build command.
Follow the mentioned steps:
First build the project. Command for build app is
xcodebuild build-for-testing -workspace "Your.xcworkspace" -scheme "UpgradeAll" -destination "platform=iOS Simulator,name=iPad Air,OS=11.0" -derivedDataPath "All"
Then execute the test command using this.
xcodebuild test-without-building -xctestrun "All/Build/Products/UpgradeAll_iphonesimulator11.0-x86_64.xctestrun" -destination "platform=iOS Simulator,name=iPad Air,OS=11.0" '-only-testing:YourTestbundleName' -derivedDataPath 'build/reports/bundleName' | tee xcodebuild.log | ios-sim start --devicetypeid "iPad-Air, 11.0"
I got this error when trying to run my UI Tests through jenkins. What I did was:
Set up Jenkins as a Launch Agent instead of a daemon. (more info here https://medium.com/#ved.pandey/setting-up-jenkins-on-mac-osx-50d8fe16df9f)
Grant the jenkins user admin privileges under System Preferences -> Users & Groups -> Jenkins user -> 'Allow user to administer this computer'.
Rebooted my Mac Mini and the UI Tests work now.
I have a complete command to deploy the xCode project on real device.
i.e
xcodebuild -workspace jamesAppV2.xcworkspace -scheme jamesAppV2 -configuration Debug -destination 'platform=iOS,name=Shujaat’s iPad' clean test
its working fine using the command line.
Todo: I wanted to execute this command via a shell script.
here is my complete shell script deploy.sh so for.
#!/bin/bash
#My First Script
#Info to be configured
current_path=$(pwd)
appName="jamesApp"
jamesApp_workspace="jamesAppV2.xcworkspace"
echo "Searching for $jamesApp_workspace workspace..."
if [[ $(ls $jamesApp_workspace) ]]; then
echo "$jamesApp_workspace found in current directory."
echo "Listing all installed and connected devices..."
instruments -s devices
echo "Copy + Paste from above devices"
echo "specify name of your decice to launch $appName"
read d_device_name
echo "building workspace for $d_device_name..."
build_cmd=(xcodebuild -workspace jamesAppV2.xcworkspace -scheme jamesAppV2 -configuration Debug)
destination="'platform=iOS,name=$d_device_name'"
build_cmd+=(-destination "$destination" clean test)
echo "${build_cmd[#]}"
# Here it prints the valid command given above
"${build_cmd[#]}"
else
echo "$jamesApp_workspace workspace not found"
echo "Make sure your current path contains the $jamesApp_workspace workspace"
echo "Place this file i.e deploy.sh within the directory containing $jamesApp_workspace workspace"
fi;
Problem:
I have done like
build_cmd=(xcodebuild -workspace jamesAppV2.xcworkspace -scheme jamesAppV2 -configuration Debug)
destination="'platform=iOS,name=$d_device_name'"
build_cmd+=(-destination "$destination" clean test)
echo "${build_cmd[#]}" #Prints valid command
"${build_cmd[#]}"
but gives error on execution
xcodebuild: error: option 'Destination' requires at least one parameter of the form 'key=value'
if I run the above command via command line its working perfectly but If I run this via shell script its not working.
I have referred I want to concatenate arguments of xcodebuild as string, which have space in it ,then run this command to concatenate the xcodebuild command
The shell removes the single quotes in the original command, therefore you should not have any when creating the array either.
I am also trying to execute the command in a similar way by passing it via a string. The command works without the double quotes anywhere on the command for me.
example:
$ xcodebuild -project ~/ios_projects/example.xcodeproj -scheme Xcode9-XCtest destination id=EBCDFH7S-DCJD-EE8D-DSKDKD78
I am using phonegap CLI 3.1 and XCode5. I am trying to generate the build for release mode through Phonegap CLI and Xcrun. I don't want to use Phonegap Build to upload the mobileprovision or whatever the process of them. I want to do it by xcrun to assign the mobileprovison to release build.
1) cordova build ios --release
Compiling app on platform "ios" via command
"/Applications/MAMP/htdocs/MyTest/MyTestApp/platforms/ios/cordova/build" --release
Platform "ios" compiled successfully.
2) sudo xcrun -sdk iphoneos PackageApplication -v "ios/build/emulator/MyTestApp.app" -o "/Users/mymac/Desktop/Testnew/MyTestApp.ipa" --sign "iPhone Distribution: NAME (TEAM_ID)" --embed "MyTestApp_Dis.mobileprovision"
Packaging application: 'ios/build/emulator/MyTestApp.app'
Arguments: embed=MyTestApp_Dis.mobileprovision verbose=1 output=/Users/mymac/Desktop/Testnew/MyTestApp.ipa sign=iPhone Distribution: NAME (TEAM_ID)
Environment variables:
HOME = /Users/mymac
SUDO_GID = 20
SDKROOT = /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS7.0.sdk
VERSIONER_PERL_PREFER_32_BIT = no
MAIL = /var/mail/root
SSH_AUTH_SOCK = /tmp/launch-zsBMC8/Listeners
LANG = en_US.UTF-8
USER = root
LOGNAME = root
__CF_USER_TEXT_ENCODING = 0x0:0:0
USERNAME = root
PATH = /usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/local/git/bin
SUDO_USER = mymac
SHELL = /bin/bash
TERM = xterm-256color
SUDO_COMMAND = /usr/bin/xcrun -sdk iphoneos PackageApplication -v ios/build/emulator/MyTestApp.app -o /Users/mymac/Desktop/Testnew/MyTestApp.ipa --sign iPhone Distribution: NAME (TEAM_ID) --embed MyTestApp_Dis.mobileprovision
SUDO_UID = 501
VERSIONER_PERL_VERSION = 5.12
Output directory: '/Users/mymac/Desktop/Testnew/MyTestApp.ipa'
Temporary Directory: '/tmp/W81FhZ9VAH' (will NOT be deleted on exit when verbose set)
+ /bin/cp -Rp ios/build/emulator/MyTestApp.app /tmp/W81FhZ9VAH/Payload
Program /bin/cp returned 0 : []
Checking original app
/usr/bin/codesign --verify -vvvv ios/build/emulator/MyTestApp.app
Program /usr/bin/codesign returned 1 : [ios/build/emulator/MyTestApp.app: code object is not signed at all
In architecture: i386
]
Codesign check fails : ios/build/emulator/MyTestApp.app: code object is not signed at all
In architecture: i386
Done checking the original app
Embedding 'MyTestApp_Dis.mobileprovision'
/bin/rm -rf /tmp/W81FhZ9VAH/Payload/MyTestApp.app/embedded.mobileprovision
Program /bin/rm returned 0 : []
/bin/cp -rp MyTestApp_Dis.mobileprovision /tmp/W81FhZ9VAH/Payload/MyTestApp.app/embedded.mobileprovision
Program /bin/cp returned 0 : []
/usr/bin/codesign -d --entitlements /tmp/W81FhZ9VAH/entitlements_rawixGWnKhi /tmp/W81FhZ9VAH/Payload/MyTestApp.app
Program /usr/bin/codesign returned 1 : [/tmp/W81FhZ9VAH/Payload/MyTestApp.app: code object is not signed at all
]
error: Failed to read entitlements from '/tmp/W81FhZ9VAH/Payload/MyTestApp.app'
Hmmh, I'm having a similar problem like Shashi.
When running 'cordova buld ios [--release]' from shell and then doing a 'xcrun ...' afterwards it works for me okay.
BUT: When running this sequence from within a script, I receive a "Codesign check fails ..." error too ...
If I insert (like) a "wait" cycle inside my script between the cordova and the xcrun call, it works.
So - to me - it seems, as if cordova returns to shell while it isn't completely finished (?)
Fact is if I code my script like
#!/bin/bash
cordova build ios --release
sleep 5
sh -c "xcrun ..."
it's working for me.
Question: Is it a bug in cordova/phonegap ???
So, finally I got everything to work okay ... :D
The problem of Jenkins complaining about a failed 'codesign ...' run is a MacOS (configuration) issue
The crucial thing is to allow the Jenkins access to the keychain of the system. The allowed access for the Login-shell of the Jenkins user is different from the Jenkins server process running under the Jenkins user account (!)
For now I realize this by running the unlock of the login.keychain within the Jenkins job before running my build script
like: in the Jenkins job for "execute shell"
security unlock-keychain -p password /Users/Shared/Jenkins/Library/Keychains/login.keychain
echo ##### building now ######################
./buildit.sh ios --release -v
This may not be the 100% nicest solution - but for now it works :P
See as well: [1]: Keychain won't unlock from Jenkins script unless user logged in
Meanwhile I found:
Fact is, that - when cordova exits and returns to shell - cordova related activities are NOT completed yet!
It takes a while after the cordova exit, for the 'platforms/ios/AppName/_CodeSignature/CodeResources' file to show up. This file obviously is essential for the 'codesign' which is started by xcrun command to succeed.
So I do in my script (which i call 'buildit.sh')
#!/bin/bash
[...]
cordova build ios --release
signaturefile="platforms/ios/build/device/$appname/_CodeSignature/CodeResources"
echo DEBUG:signatur file is $signaturefile
while [ ! -f $signaturefile ]
do
echo waiting
sleep 1
done
xcrun ...
Then the whole build/packaging process in one script succeeds.
However: Running the script from my ContinuousIntegration server Jenkins, I observe that this criteria may be essential, but not enough. From the CI I still get a
/usr/bin/codesign --verify -vvvv [...]
Program /usr/bin/codesign returned 1 : [...] code object is not signed at all
error!??
EDIT (05.12.2013): This is due to the fact that the Jenkins service couldn't access the keychain. E.g. doing in the Jenkins job an unlock of the keychain prior running the build script sorts it. (May not be the most elegant solution, but at least it prooves the problem not to be in the scripting :)
In order to skip the code signing you can perform a manual build from the console like this:
xcodebuild clean build CODE_SIGN_IDENTITY="" CODE_SIGNING_REQUIRED=NO
Use additionally the -configuration, -target and -sdk parameters in order to define your build settings.
To Disable Code Signing:
*Go to /Applications.
Right click on XCode and select 'Show Package Contents'.
Copy Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS5.1.sdk/SDKSettings.plist to your desktop. (Make sure to actually copy and paste. No drag and drop)
Open it and under DefaultProperties set CODE_SIGNING_REQUIRED to NO.
Copy it back and replace the original file.
Restart XCode.
Open your project.
In Project Navigator select your project and open Build Settings section of your porject (and not any particular target)
Under Code Signing find Code Signing Identity and for both Debug and Release modes set Any iOS SKD to Don't Code Sign.
Now you should be able to build your project without any errors.*
To make an IPA:
In 'Project Navigator' select Products
Right click on [NameOfYourProject].app and select 'Show in Finder'.
Create a folder and name it Payload
Move [NameOfYourProject].app to Payload.
Compress Payload and rename it to [NameOfYourProject].ipa
I've been trying to figure out how to run our Unit Tests from the command line, so that we can automate them.
I'm using XCode 4.5.2, and building an IOS application.
First I tried using this:
xcodebuild -target "Unit Tests" -configuration "Debug (test syncserver)" -sdk iphonesimulator6.0 clean build TEST_AFTER_BUILD=YES
That would run the compile, but the test didn't run.
So, after reading other solutions around here, I installed ios-sim and setup my target to run this script afterward:
(Note that I had to replace occurences of "TEST_HOST" with "CODESIGNING_FOLDER_PATH" as TEST_HOST doesn't seem to exist in my builds.)
#!/bin/bash
if [ "$RUN_UNIT_TEST_WITH_IOS_SIM" = "YES" ]; then
test_bundle_path="$BUILT_PRODUCTS_DIR/$PRODUCT_NAME.$WRAPPER_EXTENSION"
echo ios-sim launch "$(dirname "$CODESIGNING_FOLDER_PATH")" --setenv DYLD_INSERT_LIBRARIES=/../../Library/PrivateFrameworks/IDEBundleInjection.framework/IDEBundleInjection --setenv XCInjectBundle="$test_bundle_path" --setenv XCInjectBundleInto="$CODESIGNING_FOLDER_PATH" --args -SenTest All "$test_bundle_path"
ios-sim launch "$(dirname "$CODESIGNING_FOLDER_PATH")" --setenv DYLD_INSERT_LIBRARIES=/../../Library/PrivateFrameworks/IDEBundleInjection.framework/IDEBundleInjection --setenv XCInjectBundle="$test_bundle_path" --setenv XCInjectBundleInto="$CODESIGNING_FOLDER_PATH" --args -SenTest All "$test_bundle_path"
echo "Finished running tests with ios-sim"
else
"${SYSTEM_DEVELOPER_DIR}/Tools/RunUnitTests"
fi
Now, when I run the same xcodebuild command as before, I get this:
/bin/sh -c
"\"/Users/johnlussmyer/tu/ondeck/OnDeck/build/OnDeck.build/Debug (test
syncserver)-iphonesimulator/Unit
Tests.build/Script-4CECE52812D5043F0063EC6A.sh\"" ios-sim launch
/Users/johnlussmyer/tu/ondeck/OnDeck/build/Debug (test
syncserver)-iphonesimulator --setenv
DYLD_INSERT_LIBRARIES=/../../Library/PrivateFrameworks/IDEBundleInjection.framework/IDEBundleInjection
--setenv XCInjectBundle=/Users/johnlussmyer/tu/ondeck/OnDeck/build/Debug (test
syncserver)-iphonesimulator/UnitTests.app --setenv
XCInjectBundleInto=/Users/johnlussmyer/tu/ondeck/OnDeck/build/Debug
(test syncserver)-iphonesimulator/UnitTests.app --args -SenTest All
/Users/johnlussmyer/tu/ondeck/OnDeck/build/Debug (test
syncserver)-iphonesimulator/UnitTests.app [DEBUG] Session could not be
started: Error Domain=DTiPhoneSimulatorErrorDomain Code=1 "Unknown
error." UserInfo=0x7fcf04b03190 {NSLocalizedDescription=Unknown
error., DTiPhoneSimulatorUnderlyingErrorCodeKey=-1} Finished running
tests with ios-sim
Any suggestions on what to try next?
From the log, it seems you have several wrong settings. The correct syntax is:
ios-sim launch YourApp.app
--setenv DYLD_INSERT_LIBRARIES=/../../Library/PrivateFrameworks/IDEBundleInjection.framework/IDEBundleInjection
--setenv XCInjectBundle=YourTest.octest
--setenv XCInjectBundleInto=YourApp.app/YourApp
--args -SenTest All YourTest.octest
You could see that
the app path is incorrect after launch command
XCInjectBundle should be your octest instead of app
XCInjectBundleInto should be your app binary path instead of app folder
The last argument should also be your octest
If you can't see TEST_HOST in your environment variables, you may want to go through them to find some propers ones to make your bash script correct.
A bit old question.. but I fall here some similar question.
My two cents for OSX/ iOS (tested under iOS13 / Catalina / Xcode 11). (hope it can help someone..)
func underXCTest()->Bool{
let environment = ProcessInfo.processInfo.environment
if let _ = environment["XCInjectBundleInto"]{
return true
}
return false
}
Under testing return false.