Xcode 11 is recompiling (nearly?) my whole project, even if I just change a local private variable, or change a value of a constant in local scope, sometimes even in local private function scope. I sometime can get 2 or 3 changes with quick builds as expected, but soon enough it decides to recompile everything again (which takes too long).
Any ideas what might be going on? Is Xcode not able to determine what's changed, why does it recompile so much other stuff (even other modules).
Any advice is highly appreciated, thanks!
We had the same problem and we fixed it. Twice.
Incremental build (same build machine):
before: ~10m
after: ~35s
HOW?
Let's start with our experience first. We had a massive Swift/Obj-C project and that was the main concern: build times were slow and you had to create a new project to implement a new feature (literally). Bonus points for never-working syntax highlighting.
Theory
To truly fix this you have to truly understand how build system works.
For example, let's try this code snippet:
import FacebookSDK
import RxSwift
import PinLayout
and imagine you use all of these imports in your file. And also this file depends on another file, which depends on another libraries, which in turn uses another libraries etc.
So to compile your file Xcode has to compile every library you mentioned and every file it depends on, so if you change one of the "core" files Xcode has to rebuild literally whole project.
Xcode build is multi-threaded, but it consists of many single-threaded trees.
So on the first step of every incremental build Xcode is deciding which files have to be re-compiled and builds an AST tree. If you change a file which is acting as "dependable" on other files, so every other file which acts as "dependent" has to be recompiled.
So the first advice is to lower coupling. Your project parts have to be independent of each other.
Obj-C/Swift bridge
Problem with those trees if you're using a Obj-C/Swift bridge, Xcode has to go through more phases than usual:
Perfect world:
Builds Obj-C code
Build Swift code
Obj-C/Swift bridge:
[REPEATABLE STEP] Build Swift code, which is needed to compile Obj-C code
[REPEATABLE STEP] Build Obj-C code, which is needed to compile Swift code
Repeat 1 & 2 until you have only non-dependable Swift & Obj-C code left
Build Obj-C code
Build Swift code
So if you change something from step 1 or 2, you're basically in a trouble.
The best solution is to minimize Obj-C/Swift Bridge (and remove it from your project).
If you don't have an Obj-C/Swift Bridge, that's awesome and you're good to go to the next step:
Swift Package Manager
Time to move to SwiftPM (or at least configure your Cocoapods better).
Thing is, most frameworks with default Cocoapods configuration drag along with themselves a lot of stuff you don't need.
To test this create an empty project with only one dependency like PinLayout, for example and try to write this code with Cocoapods (default configuration) and SwiftPM.
import PinLayout
final class TestViewController: UIViewController {
}
Spoiler: Cocoapods will compile this code, because Cocoapods will import EVERY IMPORT of PinLayout (including UIKit) and SwiftPM will not because SwiftPM imports frameworks atomically.
Dirty hack
Do you remember Xcode build is multi-threaded?
Well, you can abuse it, if you are able to split your project to many independent pieces and import all of them as independent frameworks to your project. It does lower the coupling and that was actually the first solution we used, but it wasn't in fact very effective, because we could only reduce incremental build time to ~4-5m which is NOTHING compared to the first method.
There's no golden bullet here, but plenty of things to check:
Make sure you're actually using the Debug configuration in your scheme
See below for how to ensure you're using incremental builds versus whole module per matt's advice. Also make sure your Optimization Level for Debug builds is none.
If you're using type-inference heavy frameworks like RxSwift, adding explicit type annotations can speed up build times.
If the project is very big, you could consider refactoring out logical groups of source files into frameworks, but that may be too drastic of a change than you'd prefer
It might help if you provided some more specifics about the project: are you statically linking any libraries? Is it a framework or app target? How big and what swift version are you using? Do you have any custom Build Phases like linters or code generation that could be skipped sometimes?
Related
I read this article about Whole Module Optimization (WMO). I am curious if I fully benefit from the optimizations if I place all the swift files from Cocoapods directly in my main project, since then, WMO sees all the swift files as a whole and can make optimizations like function inlining and generic specialization across my whole project, instead of per module (pod).
How does WMO works for multiple pods? Does it do WMO per module/pod? If so, I think a project can benefit even more from WMO if all the swift files are in the same project, since then WHO can optimize everything as a whole, instead of per module/pod. Or am I missing something?
So before going in production, remove all the pods, place the .swift files directly in the project and run WMO looks to me like I will benefit more from WMO than not doing it.
Yes.
Whether it's worth the trouble is another question, but yes to all of it.
(I've done it this way for years, and still do in some projects, but on newer projects I've given up and accepted that frameworks are life and just let CocoaPods be CocoaPods. It's just too much work to do it by hand. But that's opinion. The answer is "yes.")
I work in a team on an iOS project that has grown to enormous size in terms of Swift code.
It takes about 10 minutes to build the project from the clean state and, most scrutinizing, 30 seconds to build and run the project after changing anything in the code, even if that code pertains to a single line in a private method in Swift file symbols from which are not used anywhere else.
We've tried lots of thing to improve build times, including techniques from this nice resource https://github.com/fastred/Optimizing-Swift-Build-Times
Nothing helped, you still need to wait the whole 30 seconds after changing every minor thing to see it in the app.
We use Xcode 10, the "New build system" with the compilation mode set to Incremental. If I build the project via Perform Action > Build With Timing Summary, the longest phase is "Swift code compilation" which is nothing new. We suspect that Xcode tries to follow conservative compilation decision making and rebuilds every Swift file that could potentially have any connection to the modified Swift code. And it seems that Xcode is wrong most of the time and does redundant work.
I kinda miss Objective-C days when the compiler would look at the all import/include statements and only rebuild explicitly declared dependencies, which meant blazing fast build times.
So I now think that maybe we could break our project into modules and ubiquitously use import in Swift to tell the compiler what Swift files depend on what other Swift files.
Is there a good and perhaps automated way to modularize a big project into many small components to speed up regular lets-try-how-it-works builds?
You can split your projects into Cocoa Touch Frameworks. -> https://www.raywenderlich.com/5109-creating-a-framework-for-ios
Add each framework to its own git repo. It may be private repo.
Create private cocoapods. -> https://guides.cocoapods.org/making/private-cocoapods.html
Add new dependencies to your project podfile. -> pod 'YourFramework', :git => 'FrameworkGitRepoPath'
I might be late to the party here, but we are using this tool: https://github.com/yonaskolb/XcodeGen/
Not only did it allow us to easily split the project into modules, it completely eliminated conflicts in the project file, because it doesn't need to be added to the repository any more.
Reading the String Manifesto, I saw a paragraph about avoiding the Foundation import when not necessary.
Why is this a concern to the Swift team? Apart from aesthetics and code tidiness, do imports come with a cost?
Could importing unnecessary frameworks impact performance, memory usage, packaged app size or build time?
The page you reference is just saying that they'd like to see more String methods built directly into Swift. Currently some tasks like case-sensitive string comparison require importing Foundation.
Ok I'm going to give the rest of your question a shot. The answer is it depends on what you are importing.
Looking at this answer:
Since Xcode 5, there is a new feature introducing precompiled sources database. Xcode 5 basically compiles all the required frameworks just once, saves builds in the database and that already compiled pieces uses while compiling your code
Also looking at this question:
We see that importing UIKit in every file that uses it is necessary. If the above solution is correct than regardless of how many times UIKit or Foundation is imported it is only compiled once. Thus importing a standard library more than once as no affect on compile time.
However, importing something for the first time would affect compile time because that Library now needs to be compiled when it previously was not needed.
Example if I have to import Foundation in a small Swift program the compile time will be slowed down. When it comes to iOS apps it's basically impossible to not import UIKit which also imports Foundation so I don't think this is worth worrying about since every app will have to compile these libraries as well.
Additionally, we need to look at imports that result from things like Cocoa Pods and Carthage:
Looking at this repo:
There are two ways you can embed third-party dependencies in your projects:
as a source that gets compiled each time you perform a clean build of your project (examples: CocoaPods, git submodules, copy-pasted code, internal libraries in subprojects that the app target depends on)
as a prebuilt framework/library (examples: Carthage, static library distributed by a vendor that doesn’t want to provide the source code)
CocoaPods being the most popular dependency manager for iOS by design leads to longer compile times, as the source code of 3rd-party libraries in most cases gets compiled each time you perform a clean build. In general you shouldn’t have to do that often but in reality, you do (e.g. because of switching branches, Xcode bugs, etc.).
Carthage, even though it’s harder to use, is a better choice if you care about build times. You build external dependencies only when you change something in the dependency list (add a new framework, update a framework to a newer version, etc.). That may take 5 or 15 minutes to complete but you do it a lot less often than building code embedded with CocoaPods.
Looking at this blog:
We see that there are more precise imports that can be used if one is concerned about compile time. Such as import UIKit.UITableViewController
As far as performance and binary size goes, any unused symbols are already optimized out of the final binary by the Swift compiler. If there’s no reference to it at compile time then it’s removed, meaning that importing a framework but not using particular parts of it shouldn’t have any negative implications.
Another blog states:
In this WWDC 2016 talk, Apple suggests replacing dynamic frameworks with static archives to mitigate this. To take this approach, we rebuilt as many of our dynamic frameworks as possible statically and then merged them into a single monolithic dynamic framework named AutomaticCore.
The difference was dramatic: our app’s launch time was cut in half.
TLDR: It's not worth worrying about importing standard things like Foundation or UIKit they are only compiled once and just about every app uses them so any performance decrease is shared. However, if you are importing third party Frameworks you might want to use a static archive to help with compile time.
While there is no doubt that importing frameworks increases the raw file size of your app, they are generally very well compressed. And if you need the framework, you have no choice but to import it. Practically, however, file size is not an issue when it comes to frameworks--relatively large ones, like Mapbox, can be included at the cost of just a few MB to your end product. And it's a shared cost by all apps so it's a wash.
Framework code is also stored in shared libraries, not in your executable, which is a very important distinction. Regardless of which Apple platform you're programming for, read Apple's OS X documentation on frameworks because the general concepts are the same. In it, it briefly talks about the relationship between framework inclusion and performance:
If you are worried that including a master header file may cause your
program to bloat, don’t worry. Because OS X interfaces are implemented
using frameworks, the code for those interfaces resides in a dynamic
shared library and not in your executable. In addition, only the code
used by your program is ever loaded into memory at runtime, so your
in-memory footprint similarly stays small.
As for including a large number of header files during compilation,
once again, don’t worry. Xcode provides a precompiled header facility
to speed up compile times. By compiling all the framework headers at
once, there is no need to recompile the headers unless you add a new
framework. In the meantime, you can use any interface from the
included frameworks with little or no performance penalty.
https://developer.apple.com/library/content/documentation/MacOSX/Conceptual/BPFrameworks/Tasks/IncludingFrameworks.html
That said, don't import frameworks that are already imported by default from "umbrella" frameworks. For example, importing UIKit automatically imports Foundation so don't import both. Does this really make a difference? No, but caring about the details, even the unimportant ones, is what will snowball someone into a programmer other people want to hire and work with. I can't tell you how many programmers I've collaborated with in the past who had the "who cares" attitude about things like this. It's no surprise their code was always the sloppiest and they're the ones always out of work.
I have 2 frameworks created by me that use (both of them) a library also created by me.
The first framework initialize the library and makes all its workflow. After finishing the first framework, the second one must start.
But when the second one is going to start, after initializing the library, the app using both frameworks crashes with a exc_bad_access error.
Apparently the library is created correctly, but if i comment the line of code to initialize the library in the second framework, the workflow continues (it crashes later because it has no library initialization).
Is there anything I'm doing wrong? Should I use two separate libraries instead?
EDIT:
Imagine the situation:
Framework A has this methods: start, stop. And while it works it delegate to the methods: infoFromA,frameworkAFinished.
Framework B has this methods: start, stop. And while it works it delegate to the methods: infoFromB,frameworkBFinished.
Both start methods initialize the static library mentioned (lets call it problematicLibrary).
Both frameworks present some views to make its functionality. So lets make an example of the app workflow.
At the app view viewWillAppear method, I start the Framework A just using:
[FrameworkA start]; , this will initialize the library and present a view. With this view (using my problematicLibrary) some info will be delegated to the infoFromA delegated method. And after all the info is delegated, it will delegate to frameworkAFinished.
When the FrameworkA has delegated to the frameworkAFinished I start the next Framework: [FrameworkB start]. As the other Framework, it will initialize the library and present a view. While debugging, all the initialization of the library is done (create the instances of the needed objects and a new instance of the library is created) and while its presenting the view it goes through the viewDidLoad method and then it throws an exc_bad_access error at the problematicLibrary initialization line (which has been done before and continued to present the view!!) without going into any other method from the view.
I have checked if the initialization is doing well, and all the variables were at null value before the initialization, and a new memory address is given to the library object.
This sounds strongly like a symbol conflict to me. I'm just surprised the linker didn't catch it, but I assume that's because you're using a static library in both your frameworks instead of simply yet another framework.
Generally speaking, I'd warn that "this is a bad idea™". What you're trying to introduce into your design is basically dependency management. Like a lot of blog articles and specifically this SO answer suggest, you should avoid packaging frameworks (and by extension libraries) into frameworks.
What most likely happens in your scenario is this (I admit I'm guessing a bit here): You linked the library into Framework A. Thus, the library becomes a fixed part of it. Its symbols are in it, even if you did not expose them to the app in any header files or the like. As long as you use only that, everything works smoothly. Then comes Framework B, of which the library is also a fixed part. Even though you can't see it from your app, the very same symbols are inside it. This, however, clashes with the symbols that were already loaded by Framework A, hence a crash (as said, this would usually be caught by the linker, but I guess you "tricked" it by building the frameworks beforehand and packaged the library in them). Maybe somebody else can explain this in more detail, but that quickly becomes besides the point as to how you would go for a solution. From how I see it, it just won't work this way.
So here's a suggestion for how you can solve your problem:
If you really, really need to split like this (two frameworks used in your app using the same dependency), I'd suggest removing the library from the frameworks (i.e. make them depend on it, but not package the actual .a file in them) and documenting that properly. Then add the library to your app project, just like you added the frameworks.
If you want to make this fancy and easily installable into other apps, I'd recommend setting up a private CocoaPods repository and turn your frameworks into private pods. You can then easily define the library (or rather "new Framework C") as a dependency for Framework A and Framework B. When you then pod install in your app, cocoapods figures out the dependency and adds Framework C automatically to your project. This scenario is exactly what cocoapods (or any dependency manager for that matter) was designed for. It automates and helps in the project setup, so that the final build (the app) doesn't have to figure out dynamically what it can and can't use. The end result is the same.
Trying to duplicate that "in code" quickly becomes messy. Frameworks trying to figure out things of the surrounding app/project that uses them (like "is my dependency so-and-so already linked? if not, can I load my own version of the library?") can lead to a lot of pain.
Okay, in response to your comment I'll try my hand at a more detailed How-To for the non-cocoapods setup. As a preface, though, let me say that that's kinda hard to do on top of my head, as I have no ready sample project to toy around with. Since this is one of those "set it up once and then forget aout it for a long time" I have to admit my recollection of these things is a bit fuzzy, so consider this as a sort of "rough direction". There might be things you need to configure differently than how I recall them. Other SO user are thus hereby invited to edit and correct my answer here, too. :)
First, I have to say I am not exactly sure whether you need to convert your static library into a framework or not for this, I think you don't so I'll go from here (I have never used a static library in that way).
That means you can leave the project that builds your library as is. On second thought, you probably have to do this to be able to link against the library without making it a part of the framework that uses it. I will still refer to this code as "library" in the below points, but assume that you're able to turn it into a dynamic framework as well.
The projects building Framework A and Framework B should be changed. I don't know how you have this set up (as one project with various targets, whether you have a "development application" as part of it to test the frameworks on themselves, etc.), but the important thing is that in the target that builds a framework, the library should be linked (i.e. in the "Link Binary With Libraries" build phase), but not copied (i.e. it must not be in the "Copy Bundle Ressources" build phase). It might be tricky to set up any development/test target you use to run, depending on how you did that so far. For example you might have to manually edit Library Search paths in your build settings to be able to properly compile the framework.
Lastly, you need to change your final app's project settings, obviously. The library that was originally part of Framework A and B now needs to be linked to from its project directly, and, obviously, it needs to be copied into the bundle as well. Note that any projects that include either of your frameworks (A or B or both) in the future must do so, otherwise they won't work, because these frameworks expect the library to be present (but no longer "bring it with them").
In spite of this long-ish text wall, it shouldn't be that complicated, I think, but you may still want to check out how cocoapods can support you with something like this, perhaps it's helpful in the future. The linked article expects basic knowledge on how to use a pod and write one, but the other guides on the site explain that pretty well. I am just saying this because in the end, when using cocoapods in a project to add multiple libraries/frameworks that introduce dependencies, it basically sets up your project in the way I described above. It uses some fancy shell scripts that ensure everything is copied to the correct folders and such, but overall speaking that's how it works: A pod's podspec tells cocoapods what additional pods to include in your app for it to work (a dependecy the pod expects to be there, but doesn't "bring in" by itself).
Check if they are both compiling for the same target.
Check if they have access to the same target membership.
Check build phases to see that they are both there.
I think because the first library is not 'well' referencing the second one.
Also i think that you can't import a framework inside another one.
To make things easier, you can merge the two frameworks on a single one.
Also you can add a callback (using protocols or closures) that informs for the end of the first workflow, and you use this callback to initialize the second framework. This way the initialization will not be made automatically.
After iPhone app that I'm writing in Swift become quite big (> 150 .swift files + various Objective-C libs), Xcode start behave pretty badly:
every second compilation I get various errors, e.g.:
Command failed due to signal: Segmentation fault: 11
compilation take enormous amount of time (> 2 min on MacBook Pro Retina)
and so on.
I just wonder if everyone has same problems and maybe someone found a way to reduce this nightmare?
What I have done so far — I split project into several dynamic frameworks that I link from main project, it helps to reduce compile time, but introduce some new problems.
I also use iRamDisk to keep DerivedData folder in RAM and periodically delete all files from it, it sometimes helps with SourceKit crashes.
Swift toolchain is still a bit gross, you'll need to use some temporary workarounds until Apple fixes it (see UPDATES below)
Here is a list of items that you can do to keep yourself from going crazy.
Slowness caused by immature Swift compiler
Change your development workflow using Injection for Xcode. Once you installed the plugin, you'll be able to inject code changes in your simulator\device without recompiling. You don't need to hardcode\modify anything in your project. We started using it recently at work and it made a huge impact on our side, even if it doesn't apply to every use case (for example you can't create new functions, you can only modify the existing ones).
Some particular code constructs that the compiler doesn't like and takes too much time to compile. The most common problem is with the Type Checker that slows down compile time exponentially based on how many type checks it needs to do (read more here for practical examples and here for a detailed explanation). In order to identify if you are suffering from this problem you can follow this blog post, you will gather information about the functions that creates slowness by using some compiler additional flags. Alternatively you can use this Xcode plugin to identify the source of the build slowness.
Use dynamic frameworks wisely, where it makes sense. A framework recompilation will be done only when you modify one of its Swift files (dynamic frameworks are only available for iOS >= 7).
Condense code in the same files. Lowering the number of Swift files speeds up the compile process sensibly. You can easily achieve it enabling "Whole module optimization" by adding a user-defined custom flag SWIFT_WHOLE_MODULE_OPTIMIZATION and set it to YES and at the same time set optimization level to none (to disable optimizations that would make it slow) OUTDATED You may consider to use this gist, it's a build script that collapses all your code in a "merge.swift" file.
You'll need to create a new target for it, but it is worth a
try.
Double check things listed here (there are a few some more misc reasons because the compilation is slow)
OUTDATED Try the approach described in this blog post, it involves creating a build script that generates a make file. It requires manual intervention on the build script (it contains the list of swift files).
OUTDATED Try this hacked up incremental compilation technique
UPDATE: Incremental builds introduced on Swift 1.2 (Xcode 6.3)
Apple finally introduced incremental builds with Swift 1.2 (shipped with Xcode 6.3). It's not still perfect, but it's a huge improvement.
From now on a class is recompiled only when it is changed (or when one of the class it depends on has been changed).
However the compiler still can’t understand if the changes to a class are to its interface or not. So any kind of change to a class causes a recompilation of that class and all of its dependencies.
UPDATE: Recompile dependent classes only when public interface changes introduced on Swift 2.1 (Xcode 7.1)
Starting from Swift 2.1 (Xcode 7.1), the dependent classes are recompiled only when you change the public interface of a class, and not at every change. This makes an huge difference in particular for big projects.
Project (mis)configuration (not related to Swift)
Be sure that "Build Active Architecture Only" is YES for debug.
Be sure that you didn't add pre\post compilation scripts that take too much time.
Apple has some advices for speeding up your Xcode build in Technical Note 2190. Have you thought about creating and precompiling an own framework for outsourcing unchanged Swift modules or some/all Objective-C code?
Remove all type inferences in Swift.
This SO topic has some nice ideas and this blog post suggest to
stop generating dSYM bundles and
avoid compiling with -O4 if using Clang.
Although lots of these improvements are related to Objective-C, I am quite sure, that some of them are still relevant for Swift.
The (re)compiling is a known issue that I am sure will be resolved soon. Some recommendations:
Use Objective C where possible - it compiles fast even if it is a part of a Swift project
Split code to frameworks
Specify types instead of leaving it up to the compiler to infer them
Again, there is a good chance that this will be fixed soon, so perhaps it is best not to make big investments in rewriting or reorganizing the code at this point in time.
you could try:
upgrading the amount of RAM in your computer
if you have multiple .swift files that do things on the same view controller, try condensing them into one .swift file per view controller
tweaking the settings under compile sources to see if you have any duplicates or if there are any scripts or settings you can add to make it compile faster...
you can also take a look at this post's answers for some hints as to what you can do to slow down compile time
I've discovered that one of the main causes of segmentation faults and slow compilation is hardcoding big arrays and dictionaries, especially when declaring them as global constants and trying to access values from them from within another .swift file. When I store all that data inside plists, these problems are gone.
In my experience avoid creating the large swift files, when I started a project in my new company, there was a 'UIViewController' with more than 2000 lines, little changes on this file taking much time to build, I made 4 extensions with <500 lines from that class, my speed improvement was incredible.