Using xcodebuild I can run a specific test or group of tests. But is there a way to list all tests without running them, using xcodebuild or any other tool that comes with xcode? Facebook's xctool can do this but I hope to not take dependency on third party tools.
Better late than never - I also needed to do this and couldn't find a way using xcode, so I wrote a python script that parses the test files and gets the names of XCTest test functions:
import re
import os
all_tests = []
TEST_FILES = os.path.expanduser("~/git/ios-proj/Test")
test_case_pattern = re.compile(r"^\s*func\s(test\w*)\(\)[\s\w]*\{")
for test_file in os.listdir(TEST_FILES):
all_tests += [test_case_pattern.match(line)[1] for line in
open(os.path.join(TEST_FILES, test_file), "r").readlines() if test_case_pattern.match(line)]
print(all_tests)
You can get the test names by using "strings". it's not an xcode command, but is normally available on the mac. Run "strings" on the following file and grep whatever marks your tests. in my case, all tests start with "test_":
strings <Tests-Target>-Runner.app/PlugIns/<Tests-Target>.xctest/<Tests-Target>|grep "test_"
an easy way to find this file is by running:
grep -r <single test name> <derived data folder for your project>
I cannot think of any way to sort automatically... but
If you want to test one of the classes by one command So you can run your test one by one through this command :
xcodebuild test -destination 'platform=iOS Simulator,name=iPhone 7,OS=11.1' '-only-testing:APPNAME-TESTTARGET/NAMEOFTHECLASS'
also you can see your list of targets, builds configurations, and schemes by using this command :
xcodebuild -list
Related
I want to have a run script that looks at the number of previous warnings (probably during the compile phase), and generates an error which stops the run/buid if the number of warnings is too large.
I am working on an app that has a large amount of warnings that happen during compile. They do not stop the app from compiling, but I want to stop devs from adding more warnings as they add features.
I have looked everywhere, but so far have not found a way to use previous build output in the run script phase of the build. Is there any way of doing this? Some kind of env variable, or a way of monitoring the build output in a custom script as it happens?
I see a similar question here:
Is there a way to access Xcodes current build output from a build phase run script?
But it seems that the output is only available after the build is complete in that answer. Is there any other option that would allow the build to be failed before it finishes?
Any insight into the build system in Xcode would be appreciated! Cheers
Additional context:
All I need is for Xcode's default warnings to have a feature similar to SwiftLint's warning_threshold
https://stackoverflow.com/a/52256459/7623867
I have looked into that problem as well for quite some time (I wanted to use warnings for quality measurement). The only way I have found is to compile the whole project with xcodebuild, with a command similar to:
xcrun xcodebuild -project MyProject.xcodeproj/ -scheme MyScheme -destination "platform=iOS Simulator" build -quiet | tee xcodebuild.log
and then counting the warnings using AWK or some other tool.
Here is an example.
We are setting up a Swift iOS Project, build with fastlane, to be analysed by SonarQube with the SonarSwift Plugin.
Everything works so far except for information about the Tests.
We archived to add Code coverage by generating a report with slather (llvm-cov) and renaming it Coverage.report and filling sonar.swift.coverage.reportPath with the path.
We also generate *.junit files and feed them into sonar.junit.reportsPath. But those seem to never be included.
We tried only the path ./reports and naming the files with TEST- prefix and .xml ending. We also tried a direct Path ./reports/TEST-report.xml.
Did anybody got this to work? If yes, How? Is this even supported by SonarQube?
According to https://community.sonarsource.com/t/number-of-unit-tests-is-zero/26104/6
JUnit reports only support Java.
For Swift, we need to convert the report to a generic type
Firstly, For Coverage (https://gist.github.com/lucascorrea/80f3d57a7f97b2365ef39839eefe68a1)
run test to generate test data
convert to coverage data with xcrun llvm-cov show that sonar can recognize
configure sonar.swift.coverage.reportPath to the report path generated by xcrun
you can run sonar-scanner to submit coverage data
sh("xcodebuild -workspace ../brewdog.xcworkspace -scheme brewdog -sdk iphonesimulator -enableCodeCoverage YES -destination 'platform=iOS Simulator,name=iPhone 7,OS=11.3' GCC_GENERATE_TEST_COVERAGE_FILES=YES build test -derivedDataPath ../DerivedData")
sh("xcrun llvm-cov show -instr-profile=$(find ../DerivedData -iname 'Coverage.profdata') ../DerivedData/Build/Products/Debug-iphonesimulator/brewdog.app/brewdog > ../DerivedData/Coverage.report")
Secondly, For UnitTest (https://github.com/azohra/forsis)
After you done this, you can check information like count of unit tests, success, fail, duration (There are some pictures in the above link)
This one is the same, you need to convert the test execution report to generic type with this tool
set up the convert
**[IMPORTANT]**configure sonar.tests AND sonar.testExecutionReportPaths in sonar-project.properties
run sonar-scanner
After done the two things
You can check the coverage AND unit test in the column in sonarqube website!!
After solving this Omitted code blocks from clang AST for ObjectiveC, I've tried it on a small Objective C .m file along with an appropriate compile_commands.json and it works properly and I get the entire syntax tree.
Now I'm trying to see if it's possible to run it on the entire xcodebuild
[
{
"directory" : "/Users/xx/Desktop/iOSApplication",
"command" : "xcodebuild clean build CODE_SIGN_IDENTITY=\"\" CODE_SIGNING_REQUIRED=NO -project /Users/xx/Desktop/iOSApplication/iOSApplication.xcodeproj/",
"file" : "iOSApplication.xcodeproj"
}
]
When I tried to then run clang-check -ast-dump /Users/xx/Desktop/iOSApplication/iOSApplication.xcodeproj it gives me the errors error: unknown argument: '-project' and error: unable to handle compilation, expected exactly one compiler job in ''
Is it actually possible to run the AST based checker on the entire xcodeproject? Or how should I go about compiling the files 1 at a time?
I've managed to generate the compile_commands.json by following this guide here http://docs.oclint.org/en/stable/guide/xcodebuild.html
However, I'd still like to be able to run my RecursiveASTVisitor on the entire projects. Or alternatively, pass in the xcode project and enumerate all the source files would probably work too.
Anyone has ideas how to go about passing entire xcodebuild project as parameter for RecursiveASTVisitor?
Right now I'm running my ASTVisitor like this ./MyASTChecker ~/Desktop/directory/sample1.m but I'd like to make it do something like ./MyASTChecker ~/Desktop/directory/sampleproject.xcodeproj
The way I do it is at compile time using scan-build. This works for me with cmake/make based projects.
scan-build --use-analyzer=clang -enable-checker <checker_name> make
This will read the makefile and build everything in there while running the specified checker on each file as it's compiled. If you only want to build with some flags or a specific target, you can do this:
scan-build --use-analyzer=clang -enable-checker <checker_name> make <build_options>
If you instead have a cmake based project. You might first want to generate a makefile in a build directory. I do this for that:
cmake <path_to_cmakelists> -DCMAKE_CXX_COMPILER=c++-analyzer -DCMAKE_CC_COMPILER=ccc-analyzer
This followed by scan-build from above will compile and run checker on the source files while each file is being compiled.
I have only tried this with CMAKE / MAKE but should work with xcode like this:
scan-build --use-analyzer=clang -enable-checker <checker_name> xcodebuild
and with build options
scan-build --use-analyzer=clang -enable-checker <checker_name> xcodebuild <build_options>
You can read more about scan-build here
In my application I am using KIF framework for testing. Now for all test cases in my class I want to generated file with all cases and their result. How to achieve this?
open Terminal run the following command
sudo install ocunit2junit
then move project folder (cd project path)
and run following command
xcodebuild test -workspace appName.xcworkspace -scheme schemName
-destination 'platform=iOS,id=deviceID' 2>&1|ocunit2junit
I got shell script for build and deploy xCode projects.
https://github.com/ciryon/xcodebuild-script
I am trying to run the script but getting the following error.
xcodebuild: error: The workspace named "jamesAppV2" does not contain a scheme named "". The "-list" option can be used to find the names of the schemes in the workspace.
Note:
Its pointing me to scheme named "" (empty), What could be the problem any hint.
here is my scheme -list
xcodebuild -workspace jamesAppV2.xcworkspace -list
Information about workspace "jamesAppV2":
Schemes:
jamesAppV2
jamesAppV2Tests
OAuth2
OAuth2iOS
AFNetworking
Alamofire
Firebase
Flurry-iOS-SDK
ForecastIOClient
ForecastIOClient-ForecastIOClient
Kingfisher
Pods-jamesAppV2
Starscream
SwiftyJSON
You can use fastlane.gym which is a wrapper around xcodebuild that simplifies its usage and fixes many problems.
I had a quick look at the scripts. It appears that they are attempting to use the shared scheme files to locate the scheme. So two things spring to mind, first - why not grep the xcodebuild -list command instead. Secondly, a blank result from the script may be caused by running the script on a project that doesn't have any shared schemes.
Finally I note that the scripts are quite old. This probably means that they are rather out of date and using old techniques (iso-sim) for example. I suspect that you might be better off starting from scratch and building a new script piece by piece using the latest commands offered by xcodebuild. It may take some time, but will be worth it in the long run.