I'm developing a modular app with >30 modules, where each module has its own Unit test target. I also have a special test scheme which runs all Unit tests from all the targets.
The problem is - it's too slow. While the tests run fast, it takes a lot of time for Xcode to switch from one test target to another. The "all tests" scheme is supposed to be run in CI so I would really like to improve the performance.
I tried to convert all Unit test targets into static frameworks and link them to a new "merged" Unit test target. In this target I then created a single AllTests.swift file importing the frameworks and manually invoked tests on each of them. The performance improvement was ~4x.
However, this will be quite hard to maintain this file. My assumption is, since they are linked statically, there can be a way to make Xcode run imported in such a way tests automatically. If this works I'll be able to fully automate this process, keeping test targets for development and generating the "merged" target in CI.
What else I tried:
Investigated the option of using SourceKitten to parse the project and generate the AllTests.swift file automatically. This would require building the project one more time, eliminating the performance improvement.
Instead of linking frameworks, include references to source files into the merged target. This introduces a bunch of "multiple files with the same name" and "same declaration" errors that cannot be automatically resolved.
Made all declarations in the static frameworks public.
Googled a lot. This SO question is quite close but the solution doesn't work for me.
So the main question is - is there a way to make Xcode automatically include Unit tests that are part of a static framework the current target is linked against? Or any other suggestion on how I can automate this process will be very much appreciated.
Related
I've got a mixed ObjC/Swift codebase with a very mixed-up structure, and the tests no longer run under Xcode 10.2 (details below). I'm trying to make the minimum structural changes to get tests back, so that I have a safety net for the refactoring that will come next. After taking what seems to me to be the obvious first step, the tests fail to build (details again below).
I'm interested in any of
solutions for the original problem (to get tests running again in the messy setup)
solutions for the problem after refactoring (to get refactored tests building in the somewhat tidier setup)
suggestions for build settings to verify or include in the description here, to get clearer on what's going wrong
I'm not interested in advice about how to structure a fresh project or create new targets with more sensible configurations: extracting code from these targets is also non-trivial, so I really need test coverage back before I start any refactoring.
Original situation (very messy)
project myCompany
app target app which builds module MyCompany
contains both Swift and ObjC code, with dependencies both ways
Defines Module = Yes, Product Module Name = MyCompany
test target myCompanyTests
Defines Module = No, Product Module Name = MyCompany
also contains both Swift and ObjC code
CocoaPods for external dependencies, also a bunch of internal Swift modules with dependencies managed by hand
Test files are included only in the myCompanyTests target, but many code files are included in both app and myCompanyTests targets. I think this and the two targets defining the same module name was probably in order to avoid having to import the app target into tests, because of problems with Swift/ObjC interop (see the next section). Building tests produces warnings about classes implemented in two places:
objc[9724]: Class _TtC12MyCompany12DiaryFetcher is implemented in both
/Users/tikitu/Library/Developer/CoreSimulator/Devices/556CAC28-B90B-4B6B-A87C-1A1450795051/data/Containers/Bundle/Application/495F33C2-F7FC-4AE6-B3FE-6908D6361B55/mycompany-develop.app/mycompany-develop (0x10d724060)
and
/Users/tikitu/Library/Developer/Xcode/DerivedData/mycompany-bifciplpbqaeqqdrmhivpjgnojri/Build/Products/Debug-iphonesimulator/mycompany-develop.app/PlugIns/myCompanyTests.xctest/myCompanyTests (0x13566dc38).
One of the two will be used. Which one is undefined.
As of Xcode 10.2, myCompanyTests builds successfully but running the tests fails with an EXC_BAD_ACCESS in swift_checkMetadataState somewhere inside the UIApplicationMain call. My guess is that this is related to the module-name/files-in-both-targets shenanigans.
First attempt at "fix the obvious mistakes"
As a first attempt to tidy things up somewhat I've done the following:
Remove all non-test files from the myCompanyTests target
Rename myCompanyTests Product Module Name to MyCompanyTests
Add #testable import MyCompany in lots of swift tests
Then I start running into Swift/ObjC interop problems, because I need to call Swift code in the app target from ObjC code in the test target. Things I've tried:
#import "MyCompany-Swift.h" in an objc .m test file
'MyCompany-Swift.h' file not found
#import <MyCompany-Swift.h> in an objc .m test file
'MyCompany-Swift.h' file not found
#import <MyCompany/MyCompany-Swift.h> in an objc .m test file (yes you can see I don't actually understand this mechanism)
'MyCompany/MyCompany-Swift.h' file not found
#import "MyCompanyTests-Swift.h" in all objc .m test files that need access to the app target
the generated file MyCompanyTests-Swift.h includes a line #import MyCompany; which looks very promising!
but that line fails with the error Module 'MyCompany' not found
Especially this last one looks suspicious to me, as I would expect the Xcode generated file should "just work". I guess I've overridden some user setting somewhere that's getting in the way: I'd be delighted with any suggestions for places to check.
I won't accept this (yet) as it's incomplete information, but I believe at least that I know why the fixed-up version is failing: the location of MyCompany-Swift.h isn't in the unit-test target's header search paths, because it's the app target. Reference e.g. https://github.com/jeremy-w/MixedLanguageTesting (which suggests adding an ad-hoc entry to the header search paths).
I believe the issue to be that only a framework target auto-creates the swift-to-objc header file Module-Swift.h: neither the app target nor a test target appear to do so.
I successfully resolved these issues by creating two new framework targets
* MyCompanyStuffThatUsedToBeInApp (containing everything that used to be in the app target except main.h), which does bidirectional swift/objc interop and
* MyCompanyTesting (a framework target imported by the tests, which likewise does bidirectional swift/objc interop for test-only code).
There may still be simpler ways to tidy this up, but this one at least is proven to work.
We are at 400+ targets in xcode. It still works fine but there has to be a better way to set this up by keeping the same code base but not having all those targets which could slow down xcode.
Android Studio lets you update the appname, which loads that folder from disk so only that project is loaded to run and program against. In XCode that is not the case, all targets are available.
It's been years but is there a better way now, with hundreds of targets that doesnt involve Git or Branching? The questions in regards to this are old and only for a few projects, we are talking hundreds here.
Your question lacks enough context to make a specific recommendation but in general...
Use Frameworks
If you can, combine sensible things into a single (or multiple) framework target. Frameworks can be more than fancy wrappers around a dynamic library, they can contain helper tools and such as well.
Use Workspaces
If there is a logical grouping to your existing targets you can separate them out into their own Xcode projects. Once you have them in their own projects you can create a workspace that references those individual projects. Even if the combined workspace loads in everything upfront (I don't think it does tho) you can still open and use the separate projects for a fast and fluid experience when working on the components.
Use static libraries
If you have a ton of targets such that one requires A, B, and C, but another needs B, C, D then you can actually put A, B, C, and D together in a static library and rely on the linker to strip out unused code from each individual target. This obviously does not reduce the number of targets you have, but you can make the static library its own project and include it in a common workspace. This will also speed up compilation as the files only need be compiled once.
Parameterize Targets or Use Schemes
If your targets are simply wrapping some external build tool/script with hardcoded parameters (I've actually seen this) you can actually pass a ton of existing variables from xcode to these external tools and eliminate "duplicate" targets. Similarly you can add new schemes if some of your targets are simply permutations of each other. A good example I've seen of this are people that have a separate target for "sanitized" (address sanitizer, etc) builds you can instead create a sanitization scheme instead of a target.
Use "Script" Build Phases
If some of your targets are doing something such as linting then you can instead employ a script build phase to call the linter instead of having a separate target to do it.
Offload Targets to an External build System
Xcode can have targets that simply call out to an external tool/script using the Script build phase (and using variable parameters as mentioned above). This can make sense to do if you already use another build system (make, cmake, etc) for another platform. Use Xcode only for the Mac/iOS specific targets and offload everything else to a cross platform build system.
If the build system outputs errors in a format Xcode understands it will even show file and line errors the same as native Xcode targets. I've seen people write thin wrappers around external tools to catch parse and reprint errors into such a format.
We started to split our swift iOS app into multiple frameworks to speed up our TDD feedback cycle. Each of these framework is a project with it's own unit test target. All of these are part of a larger workspace.
The issue is we can't run multiple test targets with our current CI setup. I was wondering if it's possible to bundle together all of our test targets into one for CI but keep them separated for development (maybe by importing/running tests from all the different targets into a "CI unit test target")?
You could certainly add another target and add REFERENCES to the files for the unit tests. They may more may not need modification of the module import names. A file may be referenced in multiple projects, and this could even be automated, as there are tools such as PBXProj for python to allow reading/writing of the Xcode project.
Alternatively, but I am not sure, it may be possible to create an aggregate target with the other items added - this is just speculation.
Perhaps a better question is why can't multiple test targets be run? Surely a script or fast lane could help?
I have a custom Framework I use within my normal App target as well as the corresponding UnitTest target. Turns out that confuses the runtime in such way that it is unable to choose the correct implementation since it has multiple choices:
objc[35580]: Class AClass is implemented in both ../MyApp.app/MyApp and ../MyApp.app/MyAppTests. One of the two will be used. Which one is undefined.
That of course leads to weird behavior if you try to check an object's class hierarchy or do any other class related checks.
So it boils down to the following two questions:
I don't see similar logs for e.g. UIKit components, but this framework is also linked to both targets. Have I incorrectly compiled the framework?
Is it just a trivial configuration issue I missed?
PS: I already checked similar posts like 1 or 2, but although everything is configured as described, the problem remains.
You have added the dependency framework to the Tests target. This is flawed thinking. Since your primary application ALSO exports the SAME framework you will receive warnings about duplicated symbols for any classes found in the framework.
By removing your framework from the test target you can resolve the warnings. Remember, you're not losing any functionality by not linking against the same framework in the test target. Trust me, your code is still there.
I ran into a similar problem here: Xcode5: creating new testing target
The key is to create a new unit testing bundle, point it to your original target, and then don't do anything else! If you start including frameworks and source files into the test target, it'll generate these linking errors. The test target is supposed to "inject" the test classes into the actual target, not create a new separate target on it's own. So you just need to import the header files in your test class, and write your test cases.
I think the bundle should only "read" the framework's header files but not build the sources and leave that task to the App (remove the Framework .m files from the UnitTest target).
Right now the App and the UnitTest are both building the Framework, thus the duplicated classes.
I am currently working on an existing iOS project in Xcode 4, and I wanted to add unit-testing using Xcode's built-in unit testing framework (OCUnit).
I followed this guide http://developer.apple.com/library/mac/#documentation/developertools/Conceptual/UnitTesting/02-Setting_Up_Unit_Tests_in_a_Project/setting_up.html on setting up unit testing, but when I try to run the tests, I get the following error:
No architectures to compile for (ARCHS=x86_64, VALID_ARCHS=armv7 armv7s).
I really don't want to migrate everything to a new project, because this project is a very large and unweildy one. However, I really do want to add unit testing.
Thanks in advance!
After trying to add unit tests several times over, I discovered that in the page for my test target, under build settings -> architectures, there is an entry named valid architectures. I tried messing with the architectures setting, and that did not work, but changing the valid architectures for the unit test target to
$(ARCHS_STANDARD_32_64_BIT)
made it compile!
I still have to deal adding missing frameworks in the test target, but this approach answers my main question.