I am building a menu bar app witih rumps and using py2app to export. When I am building using Alias mode its completely fine.
When I am building for production however it prompts me this error:
ValueError: New Mach-O header is too large to relocate in '/Users/isaacng/Developer/Menu Bar Spotify/Rumpify/dist/rumpify.app/Contents/Resources/lib/python3.10/lib-dynload/cryptography/hazmat/bindings/_rust.so' (new size=2264, max size=2260, delta=72)
I have tried rolling back a few versions of py2app and nothing. I found this FAQ in py2app but I really don't understand how to rebuild binaries with enough space?
https://py2app.readthedocs.io/en/latest/faq.html
This error can be avoided by rebuilding binaries with enough space in the Mach-O headers, either by using the linker flag “-headerpad_max_install_names” or by installing shared libraries in a deeply nested location (the path for the install root needs to be at least 30 characters long).
Very greatful if anyone can figure this out! I really don't want to use pyInstaller!!
Related
When I tried to archiving the project directly using Xcode menu (Product - Archive), it works well. But when I tried to execute it from the Integration menu (bot), I got this error:
Build operation failed without specifying any errors. Individual build
tasks may have failed for unknown reasons. One possible cause is if
there are too many (possibly zombie) processes; in this case,
rebooting may fix the problem. Some individual build task failures (up
to 12) may be listed below.
What happen? I couldn't find any error messages and completely have no clue on what's going on. I have tried to reboot the macmini and also revert the changes but it still.
UPDATE (Edit 3)
The issue has been fixed on Xcode 11, from beta 3:
Xcode uses response files by default to pass input files to the Swift compiler. To turn this behavior off, set USE_SWIFT_RESPONSE_FILE to NO.
You can use an unlimited number of Swift files in a target. (35879960)
Old answer
I've only seen this error arising when the total of files (notice all their respective absolute paths count) exceed the command line length limit (looks like it's imposed by the OS, currently 262144 bytes on my rMBP). It's a known issue.
To fix this (AFAIK), you have 2 options:
Fast (short term): Put your project on a shorter path on the server (like moving the project from /Users/mrjimoy_05_server/myprojects/mycoolproject/ to /p/mycoolproject)
Better (long term): While the first solution might work, for now, you'll probably reach the same point where you're now in the near future. So a better solution is to modularize your app (separate it into frameworks/projects). Since every module will get built separately, it'll be much harder to reach the limit and get this error again.
I hope it helps.
PD: Looks like the error thrown by the New Build System is:
unable to spawn process (File exists)
Edit 1
The error thrown by the New Build System on Xcode 10 now is:
unable to spawn process (Argument list too long)
Edit 2
The Swift team have solved this issue, but it also needs some work from the Xcode team, which hasn't been done yet on the latest released Xcode version (10.2)
Some iOS 9 devices in the wild seem to crash with the error message that I receive from the very basic crash reporting in Xcode only
dyld: malformed mach-o: load commands size (16464) > 16384
Unfortunately that's all the info I get. I can't even debug or reproduce locally.
Can anyone hint me into the right direction here?
It occurs after updating my Cocoapods, so I guess there's one of them (or their dependency) that misbehaves.
After some investigation of my mach-O binary, I saw that the sizeofcmds is really 16464.
If I understand correctly, there seems to be a load command size limit of 16384, can anyone confirm this?
Does that mean I should remove dylibs and everything should be fine?
At WWDC18 I went to an Apple engineer who is working on dyld. Here’s what he had to say:
The Dyld code is downloadable from https://opensource.apple.com (the one specific to us can be found inside macOS 10.12)
For iOS 9 the maximum size of load commands is indeed 16k aka 1 memory page (There’s no way around it! This is imposed by the OS itself. For customer service telling people to update to iOS 10 (all devices that run iOS 9 can except for iPhone 4S) would be viable.)
Since iOS 10 the maximum size of commands is 32k
Majority of the size of the load commands is determined by strings (paths) of the frameworks (use command otool -L to see them
Possible solutions:
Use less libraries (that was our goto solution thus far, but we will change to umbrella libraries (see below))
Shortening names (might screw up header lookup of cocoa pods, maybe use head maps to fix that inside the Xcode build process → maybe more (high-level) info in WWDC18 session “Behind the scenes of the Xcode Build Process”)
Try to build static archives for libraries (should not have dynamic resources otherwise make copy phases and figure out where resources are)
Build frameworks that re-export other frameworks (umbrella frameworks). Use -reexport-l as a linker flag (not done often) → gonna make some runtime overhead when starting the app, also uses a bit more memory (man ld → for info on re-exports)
The engineer recommended to file a bugreport via bugreport.apple.com, because in the future even hitting the 32k limit is possible.
I found a solution that will (at least temporarily) work for me - but I still encourage everyone to provide a real solution and more detailed insights.
Anyway:
Extract your binary
Xcode Archive
Export as IPA
Rename YourApp.ipa to YourApp.zip and extract
Navigate to the subfolder payload to find your YourApp.app
Right click & Show Package Contents of your YourApp.app file
Copy your binary YourApp (no file extension) to a different location
Investigate your mach-o binary
Run otool -f on your binary
Note the align for both architectures are listed which, for me, says 2^14 (16384). This seems to be the threshold for the size of load commands.
Run otool -l on your binary
You'll see that the different architectures and their load commands are listed - as well as their sizeofcmds (size of commands).
Now the funny thing:
For arm64, the sizeofcmds (16464) was larger than the align (16384), while it wasn't for armv7.
Now I haven't found enough documentation on this, but I assume that align symbolizes a threshold that should not be reached by the load command size. And also that it adjusts automatically (since we are definitely not having that many frameworks in our app, there have to be apps that have more).
So I guess the error came from this unlikely case, that the sizeofcmds was different in between the architectures AND that one of them was actually valid (so that the align was not automatically adjusted).
Please correct me if I'm wrong, I am just assuming here and I really really want to understand why this happens.
Solve the issue
Remove frameworks until you are under the sizeofcmds for both architectures.
I know this is not scalable, we were lucky (and stupid) that we still had one barely used framework in there that we could easily remove.
Fortunately this only seems to be an issue on iOS9 and will therefore loose relevance over the next months, nevertheless, we should find out if I'm right
Investigation ideas
My assumption that the align is automatically adjusted could be investigated by just putting in more and more frameworks to see if it actually does.
If so, adding frameworks would also solve the original issue - not nice, but at least slightly more scalable.
Sidenote
I don't feel like I shed enough light on the origins of this issue and I had a lot of assumptions.
While my solution works, I really hope you feel encouraged to investigate this as well and give a better answer.
So here's the problem:
The Mach-O header size is expected to be 16k (optimized for the platform's pagesize). In the reference by rachit it's basically the same thing but the limit is 32K. Both are correct in that this is hard limit of dyld, the loader.
The total size of load commands exceeds this max size. Removing frameworks and libraries works for you because that removes LC_LOAD_DYLIB commands (And, there is no reason why you'd need so many frameworks anyway). Instead of removing frameworks, build your app from the ground up starting with the core frameworks, and adding so long as you get linker errors.
btw, 'Align' has nothing to do with this - Alignment refers to the fat (universal) architecture slices, and doesn't have anything to do with the Mach-O.
I was able to resolve this for my team after reviewing the result of otool -l. Turns out we had the same directory included in our framework search paths 5x causing our dylibs to be added as rpaths 5x.
I've got a very curious error to share regarding Xcode localization process. I will try to share as much detail as legally possible.
From Xcode, I am trying to export an XLIFF file to send to our translators, via "Editor > Export for Localizations". However, this immediately throws error with the message:
The operation couldn't be completed. Argument list too long
This is indeed confusing, as I cannot find a more verbose log anywhere (I have already tried checking my Console.app). So, I spent quite a few time googling – to no avail. I couldn't find similar case like this. The error itself happens only when I am trying to export for localization. I can build and run the app just fine.
Facts
~ $ xcodebuild -version
Xcode 8.2
Build version 8C38
~ $ xcode-select -version
xcode-select version 2347.
~ $ echo $PATH
/Users/david.christiandy/.rbenv/shims:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Users/david.christiandy/arctools/arcanist/bin:/usr/local/go/bin:/Library/Frameworks/Mono.framework/Versions/Current/Commands
I am using Xcode 8.2 on macOS Sierra 10.12.5.
The error happens only when I try exporting from localization. This is also true when I run the localization process via xcodebuild -exportLocalizations.
I can build and run the app just fine. (I believe) there's no problem with my header search paths.
Attempts
Thought there was something wrong in the code, so I tried to run the export process (via xcodebuild command) in a CI. Somehow, it's working. For the record, I am using Bitrise CI with the same stack as my system (Xcode 8.2.x, macOS 10.12)
Asked colleagues to run export process on their machines, and they have the same error.
This leads me to think that there must be something wrong with the configuration. So I made a standalone project to confirm that the export process fails consistently. Turns out, it works just fine!
So, the hypothesis I got currently is:
There's probably something wrong in the code, and
There might be tools/software (that most of our iOS engineers installed) that might contribute to the error (since the CI completes just fine).
I don't know why the CI can run the export process just fine, and I don't know when it might suddenly stop functioning (just like our local machines).
Appreciate any help on this matter. Thank you!
I also asked this question on Apple developer forums, here is the link: https://forums.developer.apple.com/thread/86762
“Argument list too long” sounds like E2BIG, which you get when you try to run a child process with a huge argument list (I believe the current limit is 256 KiB). I suspect that Export for Localizations is running some sort of command line tool to do that work (probably the extractLocStrings tool, which you’ll find lurking within Xcode’s app bundle) and passing it full paths to each of the files in your project. Depending on how many files you have and how long those paths are, it’s easy to run into problems like this.
One of the ‘fun’ things about bugs like this is that they are dependent on where you place your project. Things might work if the project is at the top of your home directory but fail if it’s nested deep inside a subdirectory.
That also suggests a potential workaround, namely, to move your project further up in the directory hierarchy.
Finally, you should definitely file a bug about this. I believe we’ve seen this before (r. 30703294) but your report will help reinforce that this is causing problems for developers in the field. Please post your bug number, just for the record.
Several days ago before I read this answer, I managed to get the export working by deleting some folders via Xcode (remove references only). Initially I suspected that there's an invalid format within the folders that I deleted, but when I tried deleting other folders, the export process works just fine.
I also tried exporting strings using Xcode 9, and I didn't encounter the problem. So hopefully this bug is only for Xcode 8.3.3 and below.
I am trying to get some a medium-to-large sized code base that is, frankly, well written with a high degree of portability.
I decided to package it as a loadable bundle (plugin) and piggy-backed off of one of the template app projects and followed some tutorials about adding a target for loadable bundles within an app.
Also, this loadable bundle depends on a custom framework which I built for iOS and added it as a dependent for the loadable bundle. ie. The plugin links to a framework wrapper for a static lib.
The custom framework built successfully. Granted I have not yet verified that it works. The idea is to test the integrated functionality.
My build settings are largely defaults with the exception of some preprocessor defines.
Because I don't really understand the code base yet, I am literally adding one file-at-a-time to the plugin target and building cleanly every 3-4 files added.
The build completes successfully but with many, many warnings as follows, with paths to intermediate build results...etc.:
"file was built for unsupported file format with a series of hex characters () which is not the architecture being linked (armv7s)". When I converted the hex chars to ascii it just showed "#1 /Users/my-username/? ".
When I do a 'file' on any .o in the intermediate build results, I get "ASCII c program text, with very long lines"
What am I doing wrong? What does that mean?
Thank you so much for your time.
The short answer is this:
If you get this message, then your project settings are messed up.
If you are linking your app against custom frameworks, make sure they are built as fat binaries
You will need to know very clearly the meanings of active architecture and how it is used and whether or not you want to only build the active architecture for your app, or all of the possible architectures.
If you are, like me, inheriting a slew of portable code that depended heavily on gcc and its extensions, expect to make changes around builtin* attributes and to make heavy use of __clang to make available macros that used to be defined through the GNUC et al.
Also, you will need to use the -E for clang to debug/understand the preprocessing and the file inclusion. That said, don't forget to take it out because effectively what will happen is that your .o will just contain text and the build may succeed, but the linker will give you the odd message subject of this question.
Finally, do understand that Xcode, like any piece of complex software, is buggy. Sometimes, it will keep settings that you get rid off. In my case, I included custom frameworks which I built after placing them in a local dir. Then I deleted them from the project and opted to trash when prompted. The build kept failing because the linker for some reason was looking for the local directory. You would have to edit the *.pbxproj and manually remove them.
I have a strange issue that I haven't seen or read about anywhere else. My Xcode no longer parses all of my files to find issues/errors. It will display any issues or errors in a file I am currently viewing, and these will persist thereafter, but with 400+ files I can't reasonably visit each one to make it compile. Xcode can build successfully if the code is valid, however I can never know when that is, since I see no errors. Indexing occurs, but no compiling takes place.
Also, when building, the status bar states that Xcode is compiling x out of however many total compile sources but the total number is usually not even close to the actual total number. For example it may say compiling x out of 40 files when there are actually 400 compile sources.
I wasn't doing anything out of the ordinary from my usual tasks when this issue began. New projects will compile, then randomly stop compiling after a period of time.
Things I have tried:
nuking derived data
quitting Xcode/restarting computer
Recloning the project from a known working repository. It will build but won't find errors/issues when they do exist. There should be at least a handful of minor issues but none appear.
Reinstalling Xcode (4.5.2)
tried both GCC 4.2 and LLVM 4.1 compilers
Has anyone experienced this issue before and found a solution? This problem has made my job extremely difficult and any help would be much appreciated.
It does sound like corruption of DerivedData. Try this:
Clean your target: cmd+shift+K or Product->Clean
Quit xcode
Delete the contents of /Users/your_user/Library/Developer/Xcode/DerivedData
Restart xcode
(It's safe to delete this folder's contents. But if you're nervous about that, back it up first).
You should be good to go.