I am trying to find a way of wrapping an existing C library with an Apple .framework structure. The key sticking point is avoiding the need to specify the Search Headers field in Settings.
Typically in a framework you specify something like:
#import <Foundation/Foundation.h>
where Foundation is the framework name and the .h file is an umbrella header.
When testing with existing code, for the sake of argument OpenSSL, the project is using #include <openssl/file.h> internally to refer to its files. Once you want to place this inside a framework for convenience every include naturally needs to be changed to <NameOfFramwork/openssl/file.h> or you must add the $(SRCROOT)/Path/To/Frameworks/NameOfFramework.framework/Headers to the search path. This is terribly inconvenient and kills a lot of the value of the framework format. It only becomes worse when you want to wrap multiple SDK versions of the library in an XCFramework.
I'm wondering specifically if the ModuleMap can help avoid the need to change the #includes? I've added a modulemap I'm creating as a test.
Experimental module.modulemap
framework module LibreSSL [extern_c] {
umbrella header "LibreSSL.h"
export *
module * { export * }
explicit module LibreSSL_Private [extern_c] {
umbrella "Headers/include"
link "LibreSSL"
export *
}
}
One unsatisfying solution is to switch to an XCFramework comprised of Libraries instead of Frameworks.
xcodebuild \
-create-xcframework \
-library <path/to/library> \
-header <path/to/headers> \
-output MyCool.xcframework
Then header search paths work correctly, but you lose the .modulemap and the nice framework structure that keeps everything together neatly per SDK build.
Related
I am trying to package C code into a Swift module, let's call it CModule. Once I have put it into the base folder of a project (SwiftModule) and configured the Search Paths, I could get auto-complete to work in the Swift files, as well as detect errors/warnings.
The problem is, it won't recognize the module when it's imported, and Jump to Definition results in the following error:
Couldn't Generate Swift Representation
Error (from SourceKit):
"Could not load module: CModule"
CModule contains the following files:
src folder, containing the implementation and an umbrella header exposing all required functions (by importing other sub-headers) such
that main_header.h imports componentA.h, componentB.h etc
module.modulemap, containing only:
module CModule [system] {
header "src/main_header.h"
export *
}
The CModule folder is then put at the root of the SwiftModule project.
The framework and import search paths are defined as follows: $(PROJECT_DIR)/CModule (recursive)
Considering auto-complete and others work, I am confident the import search path is set correctly. No idea about the framework search path.
Additional details:
There are no spaces anywhere in my paths.
SwiftModule is a "framework" that will eventually be put on CocoaPods.
There are no public headers but the base SwiftModule header. CModule headers are all defined as Project headers.
I have a static framework B which I use inside a framework A. B has a class C I want to also expose in A, i.e. in the illustration below I want the App to be able to use the class.
App > framework A > static framework B > class C
If I try to include the header for C in the public headers of A, I get 'duplicate interface definition'. But as the code is compiled into A from B, I just need to expose the class interface – presumably that will allow the App to link correctly...?
I achieved this through a bit of a workaround.
When a static library/framework is compiled into another framework/static library, its symbols are still exposed (verify through nm <binary> | grep <symbol>). This means you just need your App code to know about those symbols – i.e. including the header for class C in framework A.
In your headers in framework A, you need to include the class C header. When compiling the framework you need to use "ClassC.h", but when the framework is being used you need <FrameworkName/ClassC.h>, as that is its location in the framework included by the App. To do this you use a macro which is removed in a run script phase.
So, in your framework A target build phases, make sure your header for Class C is in the 'public' section of your copy headers phase, then add a 'Run Script' phase and paste the following:
TARGET_MACRO="TARGET_FRAMEWORK"
cd -P "$BUILT_PRODUCTS_DIR/$WRAPPER_NAME/Headers/"
perl -0pi -e "s/\#if ${TARGET_MACRO}.*?\#else\r?\n?(.*?)\r?\n?\#endif/\1/sg" *.h
Change `TARGET_FRAMEWORK" to be whatever, or leave it.
In Framework A build settings, under preprocessor macros, add TARGET_FRAMEWORK=1 for all build configurations.
Then, in your Framework A headers that include Class C, use:
#if TARGET_FRAMEWORK
#import "ClassC.h"
#else
#import <FrameworkA/ClassC.h>
#endif
BACKGROUND
I'm building an iOS app (which I'll just call MyApp from here) that will rely on calculations done by several separate static libraries (which I'll call Lib1, Lib2, Lib3,...). Each library is built in it's own project, then imported into a single workspace (so the workspace will contain MyApp, Lib1, Lib2, ...). More details on how this is set up here. The libraries are used by other products that are independent from MyApp, so I want to minimize any changes in the libraries. The libraries are also written in (plain) C, so there are no header files.
Certain function names are used by multiple libraries (so both Lib1 and Lib2 might each have a DoStuff method). Functions with the same name generally do the same thing, but there are some specifics about how that do it that can be different between libraries, so the actual code in DoStuff on Lib1 might be quite different than the code in DoStuff on Lib2. It would be very difficult to write one universal DoStuff that would be exactly the same in each library.
THE ISSUE
While the app is running, it isn't calling the correct DoStuff from the correct library. I found out about this because the wrong function was called during a debug session (which eventually caused the app to crash, due to the subtle differences in the DoStuff functions).
WHAT I'M LOOKING FOR
Each library has only one entry point from MyApp, and each entry point is uniquely named. If DoStuff is called from the entry point method of Lib1 (or any other method on Lib1, for that matter), then I want it to call the DoStuff method on Lib1. What's the best way to make that happen?
Is there any way (maybe through a setting somewhere in XCode) I can make it so that each library is it's own namespace? That would be my preferred way to fix the issue. I guess I could go through and rename the duplicate functions so that they are all unique (so the DoStuff method on Lib1 could be renamed to Lib1DoStuff, or something similar), but there are hundreds of functions that could have duplicate names, and we are going to be adding hundreds of libraries to the project, so having to go in and rename all the functions by hand and fix all the calls to them would take a significant amount of time, and my boss doesn't see that as a viable option.
UPDATE
After looking at the comments from Josh Caswell and some of the links he provided, it looks like it might be possible to automatically rename all the functions when the libraries are compiled, and that would be the best way to try to fix THE ISSUE above. From what I've seen, the objcopy that gets mentioned in a couple of the links in the comments isn't support on iOS. I eventually came across this blog entry, which talks about creating custom build rules for Xcode targets, and this blog that talks about custom build settings and build phases.
Am I right to assume that I can use scripts at some point in the build process to automatically append to the name of all the functions in each of my libraries, instead of doing it manually as I described in the last paragraph of the WHAT I'M LOOKING FOR section above? If so, which is the correct part of the build process to make those changes? Lastly, what would the syntax look like for doing something like that? The 'scripts' used in the different parts of the build processes certainly doesn't look like Obj-C. I've never used these 'scripts' before, so I'm completely in the dark on how I'd use them, and that's what I'm looking for help with.
I tried to be as clear as I could, but if there are any questions on what I'm asking please let me know.
Why isn't xcode calling the correct library function?
Let's say I have 3 C libraries as mentioned by you. Let's say it has the following code.
Library 1 - test1lib.a with code:
#include <stdio.h>
void doStuff()
{
printf("\nDoing stuff for lib1\n");
}
void uniqueEntryPoint1()
{
printf("\nUnique entry point for lib1\n");
doStuff();
}
Library 2 - test2lib.a with code:
#include <stdio.h>
void doStuff()
{
printf("\nDoing stuff for lib2\n");
}
void uniqueEntryPoint2()
{
printf("\nUnique entry point for lib2\n");
doStuff();
}
Library 3 - test3lib.a with code:
#include <stdio.h>
void doStuff()
{
printf("\nDoing stuff for lib3\n");
}
void uniqueEntryPoint3()
{
printf("\nUnique entry point for lib3\n");
doStuff();
}
Here each library has a unique function and one common function doStuff()
When we add these 3 libraries to xcode and link them. xcode links but does not load all the objects files. Let's say the objective C code is like this:
#implementation ViewController
- (void)viewDidLoad {
[super viewDidLoad];
// Do any additional setup after loading the view, typically from a nib.
uniqueEntryPoint1();
}
The output is
Unique entry point for lib1
Doing stuff for lib1
In this case xcode will only load symbols which are referred (library 1 objects) in this case.
If you are read about linker flags/options such as -all_load, -force_load and -objC, you will have better understanding.
If we add -all_load linker option, it will force linker to load all the objects of libraries so we will get the following error in xcode
ld: 2 duplicate symbols for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
The reason this fails as the linker detects that doStuff() is redefined multiple times.
The only way around this problem is change the linker input i.e. symbols present in these 3 libraries. This is already mentioned by Josh in the comments. I will add my $0.02 to it.
Possible Solutions
Solution 1:
The best solution (self explanatory) is to change the source code if you have access to the same.
Solution 2:
Use objcopy to rename or prefix the function as provided in the answer of this How to deal with symbol collisions between statically linked libraries?
Now your doubt on how to find the objcopy.
Option 1:
You can use this project https://github.com/RodAtDISA/llvm-objcopy. This will be tricky to compile as it builds along with llvm. You will have to follow instructions at http://llvm.org/docs/GettingStarted.html and http://llvm.org/docs/CMake.html.
If you rewrite the https://github.com/RodAtDISA/llvm-objcopy/blob/master/llvm-objcopy.cpp, you can probably reuse the parsing and object rewriting logic without depending upon llvm.
Option 2:
Compile and reuse binutils objcopy from https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;a=blob;f=binutils/objcopy.c;h=2636ab4bcb34cf1e1e54db9933018a805b366727;hb=HEAD
Solution 3:
You can follow the answer provided by Richard in this link Rewriting symbols in static iOS libraries. This is more of a hack but using a hex editor you can rewrite the symbols if their length is kept the same. If you have more symbols and its a big library, you can consider using https://sourceforge.net/projects/bbe-/ and nm to write a script.
All of this is a considerable effort but apparently there is no shortcut.
Here is a quotation from the other post:
I'm working in a iOS project that includes a static library created by another company. The library include an old version of AFNeworking and I don't have any source files.
Now i need to use a more recent (and less bugged) version of afneworking, but i cannot include the same class twice in the project (of course) because all the "duplicate symbols"
My problem is that I'm preparing an iOS framework and I want to avoid this kind of situation in the future. I'm not talking about AFNetworking, but other quite popular iOS framework. In addition I applied some custom changes in the original framework code.
The one way to avoid "duplicate symbols" and "Class X is implemented in both Y and Z. One of the two will be used" that comes to my mind is to add some prefix to the original framework classes, but is this the right solution?
UPDATE 1:
I tried to apply John's solution but no joy. I have created a simplified project (here is the link to the repo) with two classes FrameworkClass which is present in framework target only, and SharedClass which is present in both framework and application targets, so maybe you can see if I'm doing something wrong. After application did launch I'm still getting:
objc[96426]: Class SharedClass is implemented in both .../TestFramework.framework/TestFramework and .../SymbolsVisibilityTest.app/SymbolsVisibilityTest. One of the two will be used. Which one is undefined
UPDATE 2:
Here is my output from nm based on the provided sample project's framework-output:
0000000000007e14 t -[FrameworkClass doFramework]
0000000000007e68 t -[SharedClass doShared]
U _NSLog
U _NSStringFromSelector
00000000000081f0 s _OBJC_CLASS_$_FrameworkClass
U _OBJC_CLASS_$_NSObject
0000000000008240 s _OBJC_CLASS_$_SharedClass
00000000000081c8 s _OBJC_METACLASS_$_FrameworkClass
U _OBJC_METACLASS_$_NSObject
0000000000008218 s _OBJC_METACLASS_$_SharedClass
0000000000007fb0 s _TestFrameworkVersionNumber
0000000000007f70 s _TestFrameworkVersionString
U ___CFConstantStringClassReference
U __objc_empty_cache
U _objc_release
U _objc_retainAutoreleasedReturnValue
U dyld_stub_binder`
UPDATE 3:
I did manage to "hide" SharedClass symbols by applying the solution by #bleater and my output from nm is now:
U _NSLog
U _NSStringFromSelector
00001114 S _OBJC_CLASS_$_FrameworkClass
U _OBJC_CLASS_$_NSObject
00001100 S _OBJC_METACLASS_$_FrameworkClass
U _OBJC_METACLASS_$_NSObject
U ___CFConstantStringClassReference
U __objc_empty_cache
U _objc_release
U _objc_retainAutoreleasedReturnValue
U dyld_stub_binder`
But I'm still getting double implementation warning in Xcode.
You should limit the visibility of symbols in any framework or library you are developing. Set the default visibility to hidden, and then explicitly mark all the functions in the public interface as visible.
This avoids all the problems you have described. You can then include any version of any public library (AFNetworking, SQLite, etc.), without fear of future conflict because anything linking to your framework or library won't be able to "see" those symbols.
To set the default visibility to hidden you can go into the project settings and set "Symbols Hidden by Default" to YES. It is set to NO unless you change it.
There are at least a couple of ways to mark the symbols from your public interface as "Visible". One is by using an exports file, another is to go through and explicitly mark certain functions as visible:
#define EXPORT __attribute__((visibility("default")))
EXPORT int MyFunction1();
The define is obviously just for convenience. You define EXPORT once and then just add EXPORT to all of your public symbols.
You can find official apple documentation on this here:
Runtime Environment Programming Guide
Update:
I took a look at your sample project, and it looks like I pointed you in the wrong direction. It appears that you can only truly hide C and C++ symbols. So if your were having this problem with a C lib (like sqlite), setting the default visibility to hidden would work. It looks like the nature of the Objective C runtime prevents you from truly making the symbols invisible. You CAN mark the visibility on these symbols, but with Objective-C it appears that is just a way to have the linker enforce what you should or shouldn't be able to use from the library (while still leaving them visible).
So if you redefine a Objective-C symbol in different compilation unit with the same name (by perhaps compiling in a new version of a popular open source library), then you will still have a conflict.
I think your only solution at this point is to do what you first suggested and prefix the symbols you are including into your framework with a unique identifier. It isn't a very elegant solution, but with the limits of the objective C runtime I believe it is probably the best solution available.
So the blog post by Kamil Burczyk was a good starting point, thanks for the hint Michał Ciuba! It has covered most of the symbols, but it didn't cope with categories and class clusters. You can see what category methods are still exposed without any change by invoking nm with parameter list sth like:
nm MyLibrary | grep \( | grep -v "\[LIBRARYPREFIX" | grep -v \(MyLibrary | grep -v ") prefix_"
When it comes to categories we have 3 groups of categories, and they all require a specific, different approach:
Categories on classes that has been renamed by NamespacedDependencies.h
Categories on classes not renamed by NamespacedDependencies.h
Categories on class clusters like NSString, NSArray...
Ad 1.
Everything is ok - class name will be prefixed so category will exist on prefixed sumbol in object file
Ad 2.
This problem occours whenever inside of the dependency we have category on a class like NSObject. It would be exposed without any change in object file, thus would cause a conflict. My approach was to internally rename NSObject to PREFIX_NSObject, this ofcourse requires me also to create and add the PREFIX_NSObject class implementation to the project (empty implementation, just a subclass of original NSObject)
#import "PREFIX_NSObject.h"
#ifndef NSValueTransformer
#define NSValueTransformer __NS_SYMBOL(NSObject)
#endif
Ad 3.
We cannot apply Ad 2. approach here. Actual objects created by let's say PREFIX_NSArray class methods are still of type that wont derive from my presumable PREFIX_NSArray class, so this doesn't make sense as category methods defined on PREFIX_NSArray won't be visible on NSArray derived objects. I ended up by manually prefixing methods of those categories in source code.
It's kind of crazy workflow, but at least gives warranty that everything will be 'invisible' and won't cause a conflict.
It's always good idea to run nm to check if all category symbols are hidden:
nm MyLibrary | grep \( | grep -v "\[LIBRARYPREFIX" | grep -v \(MyLibrary | grep -v ") prefix_"
I have a file which looks something like this:
#include <Shlwapi.h>
...
void SomeFunction()
{
UrlCombineW(...)
}
This compiled just fine until I installed another Delphi component in C++ Builder IDE but now reports unresolved external for UrlCombineW. The above call was fine before installing this component.
It seems that the component is overwriting this in some way so I need to explicitly tell the compiler where to look for UrlCombineW. This is a function from Shlwapi.dll.
Compiler does not complain, but how do I explicitly tell the linker where to look for this function and avoid unresolved external error?
Expanding my comment to an answer.
You need to link to Shlwapi.lib in order for the linker to find the functions. (This explanation glosses over a few things, but a .lib, a library file, can be either a static or import library. A static library contains the functions themselves - it's basically a collection of .obj files bundled together; an import library says that functions X, Y and Z are found in a specific DLL.) Either way, if you link the .lib in you will get the functions that you need.
There are a couple of ways to do tell the linker to link in the file:
Use #pragma comment(lib, "Filename.lib") in a .cpp file somewhere. For your case, this is #pragma comment(lib, "Shlwapi.lib").
Add it to the project options, which in turn adds it to the linker command line. In C++ Builder you do this by actually adding the .lib file to the project, ie drag and drop it onto the project in the Project Manager, or use File > Add To Project.
Which you prefer is up to you. I tend to link to localized things locally - so in my code, there's only one unit which uses Shlwapi.h and the fact it does so is an implementation detail hidden from the outside, it's not shown in the interface. Therefore, in that file, I link using #pragma comment at the point I include the header. On the other hand, if you have something used far more widely - to pick the widest example, kernel32.lib - I would add that the project itself. (Note this is an example, you don't actually need to explicitly link to kernel32, that will be done for you!)