Out of educational purposes I'm trying to use Eigen in my pet project. In order to speed the math I included macro EIGEN_USE_BLAS which activates use of blas library.
But I encountered an issue when I tried to upload my project to Testflight. The Apple response to that was:
ITMS-90338: Non-public API usage - The app references non-public
symbols in My-Project: _saxpy_, _sgemm_, _sgemv_, _strmm_,
_strmv_. If method names in your source code match the private Apple
APIs listed above, altering your method names will help prevent this
app from being flagged in future submissions. In addition, note that
one or more of the above APIs may be located in a static library that
was included with your app. If so, they must be removed. For further
information, visit the Technical Support Information at
http://developer.apple.com/support/technical/
I know that Apple has blas library as a part of Accelerate.framework and it should be use strictly through Accelerate API. But the thing is that Eigen also has their own blas included in their source and the library actually has no intention of using Accelerate.framework private guts. So that is why Apple asks me to rename those functions or remove them completely in order to get rid of that inconvenience.
But I'm not even sure that it is possible to do - to tune Eigen and its blas to use alternative names.
Is there maybe a way to solve that issue in some efficient manner? Or maybe I don't know something about Eigen usage in iOS environment?
I was lucky to figure out the problem with few tips from my fellow colleagues and a strong will to get the job done, and now I want to share with you my conclusions.
The problem: Eigen doesn't have BLAS binary provided, it only has it's headers. And when one uses EIGEN_USE_BLAS macro and it actually works, that means that Apple is linking Eigen BLAS functions to their binary - BLAS binary which is a part of Accelerate.framework. The trickiest part is that your project doesn't necessarily should have Accelerate.framework inside. XCode will add necessary files automatically and of course it won't bother to tell you that.
What's weird for me the most here is that using "naked" BLAS provided by Apple's Accelerate.framework is a clear violation because it's private API. So... from my perspective it's kind of XCode violates this rule without my intention and tells me nothing about it. Strange...
So, in order to fix it, you need to build BLAS or OpenBLAS binary (I have built OpenBLAS), add it to the project and link through Other linker flags in Build settings. After that the error is gone and you're able to upload your build to Testflight.
In order to save you some time in building OpenBLAS for iOS, I will leave you few links. I think those would be enough:
http://www.programmersought.com/article/2638161057/ - very clear tutorial on how to build OpenBLAS
https://github.com/xianyi/OpenBLAS/tree/release-0.2.21 - source code of the library
https://github.com/xianyi/OpenBLAS/issues/1531 - discussion where you can find solutions to problems you might encounter
What I found the most confusing part of this whole story is the message from Apple. Here's another copy of that:
ITMS-90338: Non-public API usage - The app references non-public
symbols in My-Project: saxpy, sgemm, sgemv, strmm, strmv. If
method names in your source code match the private Apple APIs listed
above, altering your method names will help prevent this app from
being flagged in future submissions. In addition, note that one or
more of the above APIs may be located in a static library that was
included with your app. If so, they must be removed. For further
information, visit the Technical Support Information at
http://developer.apple.com/support/technical/
I'm not a native English speaker but I feel pretty confident to say that this text is a non-sense and has absolutely nothing to do with the solving of the actual problem.
It says absolutely nothing about the fact that private API was linked to my code and that is the problem. And that I need to provide my own backend for those functions in order to make error disappear. It says about names matching and that altering my names will help to solve the problem, when it clearly will not in my case.
I hope Apple will bring more clarity into this problem and that errors descriptions will have more sense in the future.
Related
My goal is to build a swift iOS framework which uses two other frameworks (included as separate projects) and which shouldn't reveal the source code after built.
Is there some text/guide/documentation which would explain and navigate me through the process of building such a framework properly and correctly?
I built framework with aggregate target adding and linking frameworks on which my custom framework is dependent using run script as indicated here. I was able to add built of my custom framework to my custom app, together with other two dependencies (again as a separate projects), and run it on the device. However, I am not convinced by the correctness of my custom framework built.
Moreover, I was not able to upload the archive to the Appstore due to the various errors of "Unsupported architectures...", "CFBundleIdentifier Collision...", "Invalid Bundle...", "Invalid Binary" and so on. After sorting these errors out according to the various stackoverflow answers and installing the app from the TestFlight, the app crashed after launch and wasn't working at all.
I was checking various blog posts, stackoverflow questions/answers and Apple Framework Programming Guide but nothing gave me comprehensive understanding on building custom framework under conditions described above.
Everything I did was just following step-by-step tutorials without explanation of the purpose of the steps. I am sure I am missing the basics. Could you please help me and give me some guides?
I can understand you frustration. I, a while ago too searched probably for many documents on how to write a framework correctly but like you I also didn't find anything really that satisfying. From my own experiences I can give a couple of advices.
NO External Libraries
In my opinion DO NOT use external libraries in your own framework. I don't really know what your frameworks purpose is but most of the stuff you want to do can be done without using external libraries. Depending on other libraries is not a good idea especially if its a framework you are working on. Anytime these libraries get updated or even worse if they don't you will have to wait for them to be updated or find another library.So rather than this happening later on I think its better if you do it from the start. So loose the external libraries.
Universal Framework Binary
Second one is pretty easy. Generating a universal framework. I suggest you don't use a script. Most of the scripts I found were either outdated or they didn't work at all. Later on I found out that actually it was pretty easy to generate one on your own. You can do this by building your project once for a real device and one for the simulator.Then you can generate a universal binary by using the command lipo -create "Your simulator executable path" "your iOS device executable path" -output "your framework name". What this does is that it combines your two executable files and generates a universal one. Then you can just go and copy your simulator documents from the modules file and paste them in you iphoneos modules file. I am going to share a link were you can go through the walkthrough yourself. https://medium.com/wireless-registry-engineering/create-a-universal-fat-swift-framework-b7409bbfa18f
Use Objective-C(If you can)
This one is bit of a tricky one unless you know objective-c. What I would recommend is that you implement your framework using Objective-C and writing a swift wrapper around it. I would not have said this if you were creating an iOS app but in case of a framework I still think you should go for objective-c. This is because Objective-c has been around for over 30 years and most of the very old apps are in objective-c. If you want your framework to easily be used by older apps coded in objective-c I recommend you go with it. I have read tons of posts on how people have problems trying to use frameworks written in swift in their objective-c apps. Swift will be the first and probably only choice in the near future but not just yet. On the plus side if you still haven't you will have learnt Objective-C which will give definitely give you a better understanding on how things work. It will be challenging but I promise you it will be worth your while .I have a good read on this which you can checkout yourself. https://academy.realm.io/posts/altconf-conrad-kramer-writing-iOS-sdk/
Naming Conventions
This is a pretty straight forward one. I suggest you stick to apples naming conventions. This is because you will be sharing your code this time and people will look for familiarity when trying to integrate your framework. This will make your code easier to understand. You can check out these two links for more info.https://github.com/raywenderlich/objective-c-style-guide (obj-c) https://github.com/raywenderlich/swift-style-guide (swift)
Access Control
This in my opinion is an important one. When working on you framework think before you implement a class or a function. Consider if you would like someone else to be able to use that part of your code. You may want to limit the user while they use your framework and correct access control is the way to do it. You can easily guide the users so the users do exactly what you want them to do with your framework.
Document Your Code
This is a must if you want your framework be a professional one. You should be documenting every function and variable the user will use. Documenting and explaining what your code does makes a lot of peoples lives easy. You don't one anyone trying to understand what your code does for half an hour while you could have easily written a small explanation for what the parameters do and one that function or variable should be used for.
Test Your Code
Last but not least do write tests for your code. This does take some time but it assures you that your code works the way it should.
Look at other good frameworks
You should definitely checkout other open source libraries and look at what they have done. Usually there is no point in reinventing the wheel unless you are doing something absolutely different but even then there are very familiar ways to do things. I can suggest you check out the mantle sdk(https://github.com/Mantle/Mantle). Another one is the very popular Alamofire sdk(https://github.com/Alamofire/Alamofire) and also the Realm sdk(https://github.com/realm/realm-cocoa). These are good examples of frameworks. Take a look at them. Look how they have done things. It will give you an insight on how your framework should look like.
I know all of these points may also be valid if you were writing an app but what makes these a must is the fact that you will be sharing your code with others. You may manage by not doing some of these while implementing an app but for a framework things do change a little bit. It is always a pleasure to work with easy to use frameworks which make coding a pleasure. These types of small things will make your framework preferable. Happy coding.
I recently tried to use Armadillo on iOS to do some matrix computing. The App worked on my development iPhone, but Apple gave me the error message when trying to publish it in the Appstore. It seems that Armadillo calls some BLAS functions which are internal. I searched the web with the message, but had not found anything useful. I also found calling BLAS functions with "cblas_" prefix, e.g. cblas_dgemv, directly from my code would not cause the error. However, that made the use of armadillo meaningless.
I wonder if anyone has encountered the same problem, and what the solution is. I’m suspecting it’s something related to some macro in config.hpp. Thank you very much for your kindly help.
Error message:
Non-public API usage:
• The app references non-public symbols in ***: _sgemm_, _sgemv_, _ssyrk_
The Accelerate BLAS implementation supports a bunch of redundant symbols to facilitate divergent function naming schemes of various fortran compilers. Strictly speaking these are intended to be used (by your fortran compiler) so you probably have some arguing ground that they are not private interfaces. If the AppStore is still giving you trouble, file a bug against Apple and ask them to fix the bookkeeping on the interfaces so they can be used.
It would be simpler to just start using the cblas_ interfaces in the headers though.
My company makes a static library for iOS apps. One annoying step for developers is that they have to manually link against all the required frameworks that the library uses, and failing to do so leads to somewhat confusing compiler errors.
I would have previously thought this wasn't possible, but the company Chartboost claims to automatically link against non-default frameworks like AdSupport and StoreKit. Based on my testing in their sample app (available from the linked page), so far this appears to be the case (Even when disabling "Enable Modules" and "Link Frameworks Automatically" in the app that links against the static library).
Is there some way to enable this feature when creating a static library? I've tried enabling modules and the "Link Frameworks Automatically" LLVM options in Xcode, but so far haven't been able to get it working.
There's a piece of code called CBDynamicallyLoadedDependencies that calls dlopen() on the appropriate system library before making the function or method call.
My original answer wasn't correct. the dlopen() call is just in the x86 code. On the device, it's something different, but my ARM assembly isn't strong enough to figure it out. All I can say is that there's a piece of code that's acting as a trampoline to the desired functions and that references the system library files (like /System/Library/Frameworks/AdSupport.framework/AdSupport).
But the point is that it's not a simple project trick that makes it work normally. There's internal code involved.
They might use modules you mentioned and #import instead of #import, which should make the libraries link automatically.
For reference, check this question.
This might not be what you're looking for, but if you don't yet support Cocoapods, I would strongly advise taking a look:
http://cocoapods.org/
(Edit: Cocoapods is essentially linked to Xcode. Other IDEs will need another solution.)
You can advertise Cocoapods to developers as the "easy" way to work with your library, and the manual method as...well, the manual method. ;)
I'm not aware of any industry resistance to Cocoapods, so I don't see a downside to supporting it, and it does solve the problem you're talking about (albeit in a roundabout sort of way).
Also I've found Chartboost VERY developer-friendly. You might even reach out to them and ask.
I have read that class-dump utility is used to dump headers from iphone private api's. However, it does work only for objective-c frameworks. I wanted to know how it works for frameworks written in C, example - IOSurface, IOMobileFramebuffer etc.
The second part of the question is very generic. I have an app with me downloaded from cydia or istore. How do i go about reverse engineering the app on a jailbroken deivce (if that is actually needed). To be specifc, I am able to locate the executable binary and the dylibs. I am able to see the frameworks being used. But, how do I make out what functions inside the frameworks are being called by the app ?
Thanks.
I would recommend to break down this question to two questions:
1) "I wanted to know how it works for frameworks written in C, example - IOSurface, IOMobileFramebuffer etc."
I had exactly the same question:
Getting signatures of private API methods for iOS
The answer is
a) Try to google C method to see whether somebody else has disassembled it and found method signature.
b) If nobody did this before, you can be the first who will do it
2) "how do I make out what functions inside the frameworks are being called by the app ?"
There are two types of references to functions in frameworks/libraries:
Compile time references
You use some disassembler, it will list all compile time references to frameworks/dylibs.
Runtime references
These are references when somebody does dlopen, dlsym or NSBundle to use some functions.
You will have to disassemble and look/grep through disassembled code to find where they are used. There will be strings with the names of methods which are used.
The library can be found here:
http://lpsolve.sourceforge.net/
The demo code on the site shows them doing a #include "lp_lib.h" and then making API calls to the library.
My question is: how do I get the various .so files that the site provides in the download of the library into a format and into xcode in a way that lets me include the .lib and make API calls to it?
Compiling lp_solve for the armv7 architecture is incredibly difficult and has gotten the better of many talented people. I say this because the above answer makes it seem like if you follow the instructions closely enough you will be able to compile, then drag and drop.
I would use GLPK which I got to work on iOS. https://github.com/wisaruthk/MyGLPK.
The gentleman that compiled this package for iOS initially attempted to do it for lp_solve, but could not get it to work. This package will allow you to do a proof of concept, but infects your app with the GPL license. If you want to put your app on iTunes, you'll have to figure out how to get your hands on a commercially friendly package.
AlgLib is the cheapest commercially available library, but it does not have mixed integer programming. Other solutions Gurobi, Frontline Solver, NAG, Rogue Wave charge in excess of $5,000 for a developer license and will cut you a deal at scale to only charge your users $350 for a run time license.
In iOS, you cannot use dynamic linking, i.e., you will have to link lp_solve statically. The docs have instructions on how to build the static library here.
When you're done compiling the static library, simply drag'n'drop the resulting .a file into the project and add it to the linking step of your target. That should be it. However, since lp_solve is LGPL-licensed, pay attention to licensing issues, see here.