Tests that are working locally are skipped in travis-ci - ios

I am trying in vain to integrate one of my projects with travis CI. Locally, everything is fine.
I have the following in my .travis.yml:
language: objective-c
before_install:
- brew update
- brew upgrade xctool
osx_image: xcode7.2
xcode_workspace: PsychoApp.xcworkspace
xcode_scheme: PsychoApp
xcode_sdk: iphonesimulator9.2
The build seems to pass without any issues, but the tests are skipped (both ui and unit tests).
I get the following message (only unit tests to stay short):
run-test PsychoAppTests.xctest (iphonesimulator9.2, iPhone 4s, application-test)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
skipping: No test cases to run or all test cases were skipped.
After that, the build is marked as successful.
Details about the project:
Created with XCode 7.2 using default single view application template
Swift language
Uses Cocoapods for several dependencies, hence the use of workspaces
Only one scheme, Psychoapp, created by default with the correct targets set for testing
Using default test target created by xcode at the start of the project
Tests are running locally
Did anyone experience similar issues?
Best regards
============
EDIT:
As Greg Combs mentioned, I was able to make my tests be executed by fixing issues regarding some of my swift files not being visible to my test targets (both UI and unit tests).
As a bonus, here is a travis script that has not given me any issues:
language: objective-c
osx_image: xcode7.2
script:
xctool -workspace PsychoApp.xcworkspace -scheme PsychoApp test -sdk iphonesimulator9.2 -destination platform='iOS Simulator,OS=9.2,name=iPhone 6s'

At least in my case, this was caused by xctool hiding the fact that some of the tests didn't successfully execute due to runtime errors. Looking deeper into the logs, I had to fix some issues in my dependency frameworks.

Related

Xcode build phases: Use previous build warning output in "run script" phase

I want to have a run script that looks at the number of previous warnings (probably during the compile phase), and generates an error which stops the run/buid if the number of warnings is too large.
I am working on an app that has a large amount of warnings that happen during compile. They do not stop the app from compiling, but I want to stop devs from adding more warnings as they add features.
I have looked everywhere, but so far have not found a way to use previous build output in the run script phase of the build. Is there any way of doing this? Some kind of env variable, or a way of monitoring the build output in a custom script as it happens?
I see a similar question here:
Is there a way to access Xcodes current build output from a build phase run script?
But it seems that the output is only available after the build is complete in that answer. Is there any other option that would allow the build to be failed before it finishes?
Any insight into the build system in Xcode would be appreciated! Cheers
Additional context:
All I need is for Xcode's default warnings to have a feature similar to SwiftLint's warning_threshold
https://stackoverflow.com/a/52256459/7623867
I have looked into that problem as well for quite some time (I wanted to use warnings for quality measurement). The only way I have found is to compile the whole project with xcodebuild, with a command similar to:
xcrun xcodebuild -project MyProject.xcodeproj/ -scheme MyScheme -destination "platform=iOS Simulator" build -quiet | tee xcodebuild.log
and then counting the warnings using AWK or some other tool.
Here is an example.

How to unit test iOS package with `swift test`?

I have an iOS library with Swift package manager. I am trying to run unit tests with swift test
The reason why I am not using xcodebuild is, I am trying to remove the .xcodeproj from my source control. Also, there is a warning when we create xcodeproj with spm, that generate-xcodeproj will be deprecated soon.
TL;DR
This library depends on Lottie
I tried just running swift test in the root directory, but it gives lot of errors. like /.build/checkouts/lottie-ios/lottie-swift/src/Public/Animation/AnimationView.swift:859:11: error: cannot find 'superview' in scope
I tried
swift test -Xswiftc "-sdk" -Xswiftc "`xcrun --sdk iphonesimulator --show-sdk-path`" -Xswiftc "-target" -Xswiftc "x86_64-apple-ios14.5-simulator"
Then it gives the error error: module 'XCTest' was created for incompatible target x86_64-apple-macos10.15. I did not specify macos10.15 anywhere, I am on macOS 11.5.1
I can build the package using the above command by replacing test with build. What is wrong with my test command?
As I understand this is because I am not specifying the target simulator, is it possible to specify the target simulator with swift test?
swift test only works for macOS testing. It does not support cross builds for iOS.
You do not need an existing Xcode project to use xcodebuild. Run the command, xcodebuild -list to initialize a Package.swift file for testing.
See the example here.

Error running xcodebuild parallel jobs with same Cocoa Pod in frameworks & app

We have an Xcode workspace (for an iOS app) which consists of multiple frameworks and a reference application which links to and uses the frameworks and the frameworks themselves as well as the reference application use the same pod (or pods).
I've created a simple application that exhibits the same error, it's up on Github (https://github.com/hds/ios-app-with-sdks). I'll just explain everything in terms on the sample application, as it should be easier to follow that way.
The project dependancy chart looks like this:
AppWithSDKs
|- TheSDK
| |- ReactiveCocoa
|- OtherSDK
| |- ReactiveCocoa
|- ReactiveCocoa
ReactiveCocoa is one of the pods that I'm using in multiple places, so I've used in in the sample project as well. I don't think that ReactiveCocoa has anything to do with this error.
Building this project from the command line with xcodebuild produces a variety of errors, always related to the pod used in multiple parts of project. The most common error is in the code signing step, but I get errors related to setting file permissions and ownership as well. The errors always complain about the file that is to be signed (or have its permissions set) not being present.
The build command is:
xcodebuild -verbose -workspace AppWithSDK.xcworkspace -scheme AppWithSDK -sdk iphoneos8.4 -derivedDataPath Build -configuration Release archive -archivePath Build/Products/
An example of an error that I've seen is:
/usr/sbin/chown -RH <user>:<group> /path/to/project/AppWithSDK/Build/Build/Intermediates/ArchiveIntermediates/AppWithSDK/IntermediateBuildFilesPath/UninstalledProducts/ReactiveCocoa.framework
chown: /path/to/project/AppWithSDK/Build/Build/Intermediates/ArchiveIntermediates/AppWithSDK/IntermediateBuildFilesPath/UninstalledProducts/ReactiveCocoa.framework/Headers/UITextView+RACSignalSupport.h: No such file or directory
Command /usr/sbin/chown failed with exit code 1
The suspicion is that xcodebuild is doing something in parallel that shouldn't be performed in parallel, but I don't know what that is. If we run xcodebuild with -jobs 1 then there are no errors. The project also builds fine when it's compiled from the Xcode app.
Obviously the problem isn't a showstopper at the moment, but if we have to build in serial on the command line, it's going to make our Jenkins builds much slower.
Thanks in advance for any guidance or help.

Teamcity Step tests (Command Line) failed when testing an iOS app

I am trying to run tests automatically using Teamcity, but it seems that when the agent is compiling the project, it is not done correctly because when I run the command like for running test, I am getting the following error:
fatal error: 'Pushwoosh/PushNotificationManager.h' file not found
#import <Pushwoosh/PushNotificationManager.h>
[16:48:51][Step 2/2] ^
[16:48:51][Step 2/2] 1 error generated.
However, when I run manually the same commands I get no errors, but my test running. The application that I want to run was written in objective-c, and the command lines I am using are the following:
$ git clone REPOSITORY_URL (runs sucessfully)
$ pod install (runs sucessfully)
$ xctool -workspace Supermaxi.xcworkspace -scheme Supermaxi test -only FunctionalTests:TestCase000_Registration/test00_WrongEmail -sdk iphonesimulator (fails and I get the previous described error)
After being helped by I friend of mine, we found the solution to this issue in this link.
It turns out that the main problem was, as the link states, that CocoaPods uses symbolic links in it’s directory structure, and TeamCity can have trouble copying symbolic links in the VCS checkout to build agent, so I had to set the Version Control Settings of Teamcity as it is explained in the previous link.
Therefore, I have just changed the VCS checkout mode from Automatically on server to Automatically on agent (if supported by VCS roots). After that I run the agent again, and it worked properly.

Jenkins hangs when running Xcode Unit Tests

I am attempting to run my Xcode unit tests using Jenkins. I can run them from Terminal on my build server using this command with out any issues:
xcodebuild test -scheme MyProject -destination OS=7.0,name="iPad"
CONFIGURATION_BUILD_DIR=/Users/dev/Jenkins/Home/workspace/MyProject/build
However when I try to add that command to my build script (buildscript.sh) file, it hangs when running the unit tests. I also tried adding a new Xcode task to Jenkins and running with these parameters:
XcodeSchema File: MyProject
Configuration: Debug
Custom xcode build arguments: test -destination OS=7.0,name="iPad" CONFIGURATION_BUILD_DIR=/Users/dev/Jenkins/Home/workspace/MyProject/build
It is hanging on a build file in the debug-iphone simulator folder.
** BUILD SUCCEEDED **
setenv SYSTEM_DEVELOPER_DIR /Applications/Xcode.app/Contents/Developer
.
.
.
/bin/sh -c /Users/dev/Library/Developer/Xcode/DerivedData/MyProject
-ggdmamhydtxxzadkbjuusdjmholn/Build/Intermediates/MyProject.build/Debug
-iphonesimulator/MyProjectApplicationTests.build
I took a look at the build file and the environment variable it is using is set correctly.
MyProjectApplicationTest.build
#!/bin/sh
# Run the unit tests in this test bundle.
"${SYSTEM_DEVELOPER_DIR}/Tools/RunUnitTests"
# "${SYSTEM_DEVELOPER_DIR}/Tools/RunPlatformUnitTests"
# "${PROJECT_DIR}/XcodeScripts/RunPlatformUnitTests
The RunUnitTests file exists in that directory.
I don't know if Jenkins is having issues running the simulator. I spent most of the day yesterday racking my brain on this. If anyone has any ideas on what the issue may be, please let me know.
I was also facing the same problem and found that jenkins user is not correctly set up to run test case on simulator. Follow instruction given on below website:
https://content.pivotal.io/blog/ios-ci-with-jenkins
see the section "SETTING UP THE JENKINS USER".
hope it will work for you as well.

Resources