I am building an iPhone application that I need to be position-independent. I am linking against third-party libraries to which I do not have source code.
If such a library has not been compiled with -fPIC and therefore is not position-independent, can I still link against it to produce a valid PIE binary?
Will the dynamic loader handle any text relocations that occur due to code from this library.
If not, what are my possible options to resolve this situation and still produce a PIE binary?
As far as I know, whether a source file was built with -fPIC or not, is not important unless you plan to create a dynamic library out of its object file. Actually most static libraries aren't built in a position independent manner. Not that that this would be forbidden, but it will result in a slightly bigger and slower binary later on (as the compiler/linker will not be able to perform certain types of optimizations).
And a binary runs in an address space of its own, it doesn't have to be position independent at all. That's the reason why dynamic libraries must be position independent as they are shared among different binaries that all require different memory positions for themselves, so it must be possible for the dynamic linker to move the library elsewhere in the process space of a binary at runtime.
Or to say it in even simpler words: Just link against that static library and everything will be fine.
Related
When you build an app,
You have to compile and then link.
Whether you link a framework dynamically or statically, you still have to compile them both.
It’s just that for a static library, you link at build time and pay its price through extra time.
With a dynamic library, you link at launch time i.e. defer the extra cost, but eventually you’ll have to endure the link time…
Like my question is, for dynamic linking, the compilation still happens during build time. It’s not like that the OS has a compiler and compiles and then links app launch time. I imagine if that was correct then app launches for dynamic linking would be terrible…
Do I have it right?
And if a library is pre-compiled, then while you save on the compilation step, it doesn't in any way affect the linking. Still — depending on if it's static vs. dynamic library you will have to pay the cost of linking eventually.
Not an IOS-specific question.
Libraries (static or dynamic) are not necessarily compiled at build time. Many are provided as binary files without source code and simply linked to your program. If you have the source code, you can set up your build process to build the libraries.
Statically-linked libraries are combined into your executable file at build time by the linker. Dynamically-linked libraries are not part of your executable file, and are linked to your program at run time by the operating system's program loader.
Dynamic linking is generally very fast. Modern compile-time linkers can be very sophisticated and can do exotic things like generate and compile code.
So far we're describing classic compilation and linking.
Now, there are newer languages that are a hybrid of compilation and interpretation. In these languages, a library contains an intermediate representation (IL) that is may be interpreted by a runtime, or compiled to native code just-in-time (JIT).
We haven't even discussed interpreted languages like Javascript.
Modern systems often blur the traditional distinctions between interpreted and compiled languages and do "all of the above". Sometimes the code running to accomplish something in an app is not even on your device, but is running on a server "in the cloud".
So, your homework now is to read up on the terms "native code", "assembly code", "intermediate language", "compiler", "JIT", "runtime", and "interpreted language".
I have main application target. And I'm moving some code into frameworks to reduce compile time of the huge project.
Default framework type is Dynamic (BuildSettings -> Mach-O Type). I understand benefits of using Apple's dynamic frameworks as several apps will use the same framework and each application size will be lower (because the app size doesn't include this dynamic frameworks).
But if I'm using my own frameworks and only in my application, why should I choose dynamic frameworks.
It looks like the app size will be bigger with dynamic frameworks Source and application start time will be longer (because it needs to connect all these dynamic libraries, but with static they are already a part of app executable).
App store size probably will be bigger as well with dynamic frameworks as well.
Would be helpful if somebody can fill me in what benefits can we get using dynamic frameworks 🤝
Your assessment is mostly correct.
It is possible to not directly link against frameworks, but instead load them on demand with dlopen.
This can be used both for a plugin system where only one of many available libraries will be needed, or to defer the loading of particularly heavy frameworks, which would actually reduce the launch time of your app.
Further reasons I can think of for using dynamically linked frameworks are:
Licensing reasons.
If you're developing a closed-source library for other developers to use, then a static library normally has a lot more information still embedded than a dynamic library and with a static library you can easily conceal the fact that you're using it, both of which you might not want.
If you have symbol clashes (e.g. due to linking against a static library multiple times, as might be the case with the Rust standard library), then you can split the different codebases into frameworks in order to separate the namespaces.
In building an objective-c static library, I noticed that the .a file (fat file from simulator and iPhone) is quite large. In particular, it was originally 5.7mb. I found this post and set my build settings Generate Debug Symbols to No, decreasing the lib size to 1.7mb.
This was a big improvement, but is there anything else that can be done? The implementation and header files alone take up ~100kb.
In case it's part of your concern, a static library is just the relevant .o files archived together plus some bookkeeping. So a 1.7mb static library — even if the code within it is the entire 1.7mb — won't usually add 1.7mb to your product. The usual rules about dead code stripping will apply.
Beyond that you can reduce the built size of your code. The following probably isn't a comprehensive list.
In your target's build settings look for 'Optimization Level'. By switching that to 'Fastest, Smallest -Os' you'll permit the compiler to sacrifice some speed for size.
Make sure you're building for thumb, the more compact ARM code. Assuming you're using LLVM that means making sure you don't have -mno-thumb anywhere in your project settings.
Also consider which architectures you want to build for. Apple doesn't allow submission of an app that supports both ARMv6 and the iPhone 5 screen and have dropped ARMv6 support entirely from the latest Xcode. So there's probably no point including that at this point.
What is the easiest way to produce a Mach-O object file that does not have the SUBSECTIONS_VIA_SYMBOLS flag set, such that the linker (with -dead_strip) will not later try to cut the text section into pieces and guess which pieces are used?
I can use either a command-line option to llvm/gcc (4.2.1) that will prevent it from emitting .subsections_via_symbols in the first place, or a command-line tool that will remove the flag from an existing object file.
(Writing such a tool myself based on the Mach-O spec is an option, but if possible I'd rather not reinvent the wheel that hard).
Platform: iOS, cross-compiling from OSX with XCode 4.5.
Background: We're supplying a static library that other companies build into apps. When our library encounters a problem it produces a crash report with a stack trace and certain other key information that (if we're lucky) we get to analyze later. Typically the apps as deployed have been stripped of debug information so interpreting stack traces is a problem. If we were making the app ourselves we would just save the DWARF debug data from before stripping and use that to decode the addresses in the incoming crash reports. But we can't depend on the app makers supplying us with such data from their linking steps.
What we're doing instead is to let the crash report include the run-time address of selected function; from that we can deduce the offset between addresses in our linker map and addresses in the crash report. We're linking our entire library incrementally into a single .o before we stuff it into an .a; since it does only one big thing there wouldn't be much to save from removing unused functionality from it when the app is eventually linked. Unfortunately there's a few small pieces of code in the library that are sometimes not used (alternative API entry points for the main functionality, small helper functions for interpreting our error codes and the like), and if the app developer links with -dead_strip, it disturbs the address reconstruction of crash reports that the relative offsets in the final app differ from the linker map from our incremental link operation.
We can't realistically ask all app developers to disable dead-code stripping in their build process, so it seems a better way forward if we could mark our .o as "not dead-strippable" and have the eventual app linking respect that.
I solved it.
The output of an incremental link operation only has MH_SUBSECTIONS_VIA_SYMBOLS set if all the input objects have it set. And an object file produced from assembler input only has it set if there's an explicit directive set. So one can remove the flag by linking with an empty assembler input:
echo > empty.s
$(CC) $(CFLAGS) input.o empty.s -nostdlib -Wl,r -o output.o
If you're building a commercial iOS SDK that will be included in other people's code and you have third party libraries that you have a license to, is there an effective way to simplify the library / framework structure by not exporting those 3rd party symbols in a static lib?
I appreciate I could instruct developers to check for overlapping symbols, but I'd like to minimize instructions. Ie, just want them to drop the lib into their project and off they go. I also do not want to export my third party symbols as they may change in later projects.
Unfortunately, there isn't a lot to be done here very easily. A static library is just a bunch of .o files glued together. There is no linker step to determine what pieces are actually needed between .o's. That's not done until the final link step (by your customer).
That said, there are some things you can think about:
First, whenever possible, avoid including sub-libraries inside of a static library. This is really dangerous if it's possible for the customer to have other copies of the same sub-library. Your situation seems to be difference since your sub-library is licensed, so the customer is unlikely to have multiple copies, but for example, you should never include a static copy of libcurl in your static library. You must ask the customer to link libcurl themselves, or else things will explode quite badly for them. (See Linking static libraries, that share another static library.) Again, this sounds like it may not apply to you, but keep it in mind if you have common open-source libraries in the mix.
An old-school solution for dealing with visibility is to glue together your compile units. This is a fancy way of saying "concatenate all your .c/.m files into one massive file and compile that." Any function you mark "static" will not be visible outside this compile unit, and so shouldn't be exported. This also happens to increase the possibility of compiler inlining and other optimizations (particularly without fancy link-time optimization) inside of your library.
See Symbol Exporting Strategies. You have several options:
marking symbols as static (probably not possible in this case since they come from a 3rd party)
Use an exported symbols list or an unexported symbols list
Set the visibility attribute of the symbol directly (again, probably not possible in this case)
Use -fvisibility command line option when compiling (probably your best bet here)
Use weak imports
Use a weak library
These are explained at the link above.