Swift WHOLE_MODULE_OPTIMIZATION improves compile time, but causes lldb/Xcode crash - ios

TL;DR
Before
SWIFT_WHOLE_MODULE_OPTIMIZATION = NO
Debug compile takes 10-15 minutes
Release compile takes 25+ minutes
po works fine in LLDB
After
SWIFT_WHOLE_MODULE_OPTIMIZATION = YES
Debug compile takes 1-2 minutes
Release compile takes ~8 minutes
po always causes Xcode to crash
Any idea why the horrible compile times based on this info, and/or why Xcode might be crashing?
Deets
I'm working on a large 100% Swift project (there are 3rd party libraries in Objective-C, but all our code is Swift). We have been having atrocious compile times, usually around 10-15 minutes to compile the debug configuration and 30+ minutes to compile the release configuration.
This project has been very difficult to work with because of the horrible compile times. I've been searching for ways to improve this, particularly through build settings and for months had no luck. One thing I overlooked was SWIFT_WHOLE_MODULE_OPTIMIZATION, particularly because any mention of it claims it will increase a project's compile time.
So the other day we enabled SWIFT_WHOLE_MODULE_OPTIMIZATION and lo and behold we have a 10x improvement on compile times.
The problem is, now whenever we're debugging the project and try printing an object in lldb with po myObject Xcode immediately crashes. Here's some info from the crash log:
Process: Xcode [5860]
Path: /Applications/Xcode.app/Contents/MacOS/Xcode
Identifier: com.apple.dt.Xcode
Version: 6.4 (7720)
Build Info: IDEFrameworks-7720000000000000~8
App Item ID: 497799835
App External ID: 812725084
Code Type: X86-64 (Native)
Parent Process: ??? [1]
Responsible: Xcode [5860]
Date/Time: 2015-08-05 15:53:08.265 -0600
OS Version: Mac OS X 10.11 (15A235d)
Report Version: 11
Time Awake Since Boot: 13000 seconds
Crashed Thread: 20
Exception Type: EXC_BAD_ACCESS (SIGSEGV)
Exception Codes: KERN_INVALID_ADDRESS at 0x000000000000008f
Exception Note: EXC_CORPSE_NOTIFY
VM Regions Near 0x8f:
-->
__TEXT 000000010ef62000-000000010ef63000 [ 4K] r-x/rwx SM=COW /Applications/Xcode.app/Contents/MacOS/Xcode
Application Specific Information:
ProductBuildVersion: 6E35b
Here's the stack trace on the crashed thread:
Thread 20 Crashed:: <DBGLLDBSessionThread (pid=6402)>
0 com.apple.LLDB.framework 0x0000000116b09ab4 swift::ArchetypeBuilder::resolveArchetype(swift::Type) + 68
1 com.apple.LLDB.framework 0x0000000116b0f808 std::__1::__function::__func<substConcreteTypesForDependentTypes(swift::ArchetypeBuilder&, swift::Type)::$_6, std::__1::allocator<substConcreteTypesForDependentTypes(swift::ArchetypeBuilder&, swift::Type)::$_6>, swift::Type (swift::Type)>::operator()(swift::Type&&) + 152
2 com.apple.LLDB.framework 0x0000000116bc0986 swift::Type::transform(std::__1::function<swift::Type (swift::Type)> const&) const + 54
3 com.apple.LLDB.framework 0x0000000116bc0f2b swift::Type::transform(std::__1::function<swift::Type (swift::Type)> const&) const + 1499
4 com.apple.LLDB.framework 0x0000000116bc0bbb swift::Type::transform(std::__1::function<swift::Type (swift::Type)> const&) const + 619
5 com.apple.LLDB.framework 0x0000000116bc0c0a swift::Type::transform(std::__1::function<swift::Type (swift::Type)> const&) const + 698
6 com.apple.LLDB.framework 0x0000000116b0c8f2 swift::ArchetypeBuilder::substDependentType(swift::Type) + 50
7 com.apple.LLDB.framework 0x0000000116e9554e (anonymous namespace)::LowerType::visitAnyStructType(swift::CanType, swift::StructDecl*) + 270
8 com.apple.LLDB.framework 0x0000000116e92e66 swift::Lowering::TypeConverter::getTypeLoweringForUncachedLoweredType(swift::Lowering::TypeConverter::TypeKey) + 150
9 com.apple.LLDB.framework 0x0000000116e92b39 swift::Lowering::TypeConverter::getTypeLowering(swift::Lowering::AbstractionPattern, swift::Type, unsigned int) + 2361
10 com.apple.LLDB.framework 0x0000000116f8f711 lldb_private::SwiftSILManipulator::emitLValueForVariable(swift::VarDecl*, lldb_private::SwiftExpressionParser::SILVariableInfo&) + 1521
11 com.apple.LLDB.framework 0x00000001172ac7ee (anonymous namespace)::LLDBNameLookup::emitLValueForVariable(swift::VarDecl*, swift::SILBuilder&) + 102
12 com.apple.LLDB.framework 0x0000000116ebb162 swift::Lowering::SILGenFunction::emitInitializationForVarDecl(swift::VarDecl*, swift::Type) + 98
13 com.apple.LLDB.framework 0x0000000116ebbc74 swift::ASTVisitor<(anonymous namespace)::InitializationForPattern, void, void, void, std::__1::unique_ptr<swift::Lowering::Initialization, std::__1::default_delete<swift::Lowering::Initialization> >, void, void>::visit(swift::Pattern*) + 404
14 com.apple.LLDB.framework 0x0000000116ebbc57 swift::ASTVisitor<(anonymous namespace)::InitializationForPattern, void, void, void, std::__1::unique_ptr<swift::Lowering::Initialization, std::__1::default_delete<swift::Lowering::Initialization> >, void, void>::visit(swift::Pattern*) + 375
15 com.apple.LLDB.framework 0x0000000116ebba0d swift::Lowering::SILGenFunction::visitPatternBindingDecl(swift::PatternBindingDecl*) + 45
16 com.apple.LLDB.framework 0x0000000116f0617c swift::Lowering::SILGenFunction::visitBraceStmt(swift::BraceStmt*) + 284
17 com.apple.LLDB.framework 0x0000000116ecd1c0 swift::Lowering::SILGenFunction::emitFunction(swift::FuncDecl*) + 320
18 com.apple.LLDB.framework 0x0000000116ea3966 swift::Lowering::SILGenModule::emitFunction(swift::FuncDecl*) + 246
19 com.apple.LLDB.framework 0x0000000116ea3828 swift::Lowering::SILGenModule::visitFuncDecl(swift::FuncDecl*) + 168
20 com.apple.LLDB.framework 0x0000000116ea579b swift::Lowering::SILGenModule::emitSourceFile(swift::SourceFile*, unsigned int) + 427
21 com.apple.LLDB.framework 0x0000000116ea5c22 swift::SILModule::constructSIL(swift::Module*, swift::SILOptions&, swift::SourceFile*, llvm::Optional<unsigned int>, bool, bool) + 386
22 com.apple.LLDB.framework 0x0000000116ea5d42 swift::performSILGeneration(swift::SourceFile&, swift::SILOptions&, llvm::Optional<unsigned int>, bool) + 98
23 com.apple.LLDB.framework 0x00000001172aa617 lldb_private::SwiftExpressionParser::Parse(lldb_private::Stream&, unsigned int, unsigned int, unsigned int) + 10715
24 com.apple.LLDB.framework 0x000000011706b3e8 lldb_private::ClangUserExpression::Parse(lldb_private::Stream&, lldb_private::ExecutionContext&, lldb_private::ExecutionPolicy, bool, unsigned int) + 1064
25 com.apple.LLDB.framework 0x000000011706cdb4 lldb_private::ClangUserExpression::Evaluate(lldb_private::ExecutionContext&, lldb_private::EvaluateExpressionOptions const&, char const*, char const*, lldb_private::SharingPtr<lldb_private::ValueObject>&, lldb_private::Error&, unsigned int, std::__1::shared_ptr<lldb_private::Module>*) + 628
26 com.apple.LLDB.framework 0x00000001171d1696 lldb_private::Target::EvaluateExpression(char const*, lldb_private::StackFrame*, lldb_private::SharingPtr<lldb_private::ValueObject>&, lldb_private::EvaluateExpressionOptions const&) + 376
27 com.apple.LLDB.framework 0x000000011716d75c lldb_private::SwiftLanguageRuntime::GetObjectDescription(lldb_private::Stream&, lldb_private::ValueObject&) + 668
28 com.apple.LLDB.framework 0x00000001170464e6 lldb_private::ValueObject::GetObjectDescription() + 370
29 com.apple.LLDB.framework 0x000000011548e228 lldb::SBValue::GetObjectDescription() + 76
30 com.apple.dt.dbg.DebuggerLLDB 0x00000001153f3c9e -[DBGLLDBDataValue _lldbValueObjectDescription] + 24
31 com.apple.dt.dbg.DebuggerLLDB 0x00000001153f3b7f -[DBGLLDBDataValue lldbDescription] + 29
32 com.apple.dt.dbg.DebuggerLLDB 0x00000001154023dc __87-[DBGLLDBSession printDescriptionOfDataValueToConsole:runAllThreads:completionHandler:]_block_invoke + 182
33 com.apple.dt.dbg.DebuggerLLDB 0x0000000115402e6d -[DBGLLDBSession handleNextActionWithState:withRunPending:] + 424
34 com.apple.dt.dbg.DebuggerLLDB 0x00000001153fdf44 DBGLLDBSessionThread(void*) + 980
35 libsystem_pthread.dylib 0x00007fff8ec12cb3 _pthread_body + 131
36 libsystem_pthread.dylib 0x00007fff8ec12c30 _pthread_start + 168
37 libsystem_pthread.dylib 0x00007fff8ec10419 thread_start + 13
Seems like we may be on the forefront of new technology here because I haven't found much help on this issue yet. I'm wondering if we have some sort of unusual configuration that is causing the compile time issues, or if there is a known reason why lldb might be crashing. Its the same on multiple different machines, with El Capitan, Yosemite, Xcode 6.3, Xcode 6.4. Any help is appreciated!

I've worked and I am actively working on projects that are probably even larger than yours (50+ libraries, hundreds of custom classes with thousands of LoC). It takes few seconds to compile and up to 2 minutes for release build (Late 2013 MBP), even for full Swift project.
I would highly recommend to look for other causes, as what this option tells you is that your problem is somewhere else since it affects app performance, not building speed. Maybe you have your .pch file full of stuff that does not belong there or something, it can cause this kind of slowing.
Other interesting thing is use of inference. I personally declare type for every single variable just because it is easier to read for others, but there is more to it. Since the inference can get complicated, it can cause compiler to take A LOT of time to figure out what your code actually does. Read this example article, though it was more like alpha-problem. BUT MAYBE, just maybe, you have something like this in your code as well. What would be worth doing is trying to remove all the swift code, compile libraries if that works well, then you know the problem is in your swift code and it will be probably connected to this.
Also, if you provide WHAT actually slows your build time (because you can see whole build process with timestamps, and there is high probability that one step will just take forever), it can help pinpoint the problem more accurately.
Hope it helps!
Edit: Another article on this interested topic

I managed to fix this in our project. (12 mins total compile time)
The key is to set the SWIFT_WHOLE_MODULE_OPTIMIZATION = YES as you say in the user defined settings
But you need to do one more change. You need to set the optimization level to None in debug otherwise you will not be able to debug your app.
I wrote a blog post about it here:
https://tech.zalando.com/blog/improving-swift-compilation-times-from-12-to-2-minutes/

Related

Does Crashlytics encounter assertionFailure as crash?

I'm a bit confused and can't find proper information about this case. As we all know from docs:
Use this function to stop the program, without impacting the
performance of shipping code, when control flow is not expected to
reach the call—for example, in the default case of a switch where you
have knowledge that one of the other cases must be satisfied. To
protect code from invalid usage in Release builds, see
preconditionFailure(_:file:line:).
However, I'm getting crash report in Crashlytics when my asserionFailure fires.
Crashed: com.apple.main-thread
0 libswiftCore.dylib 0x1b0c15efc specialized _assertionFailure(_:_:file:line:flags:) + 440
1 libswiftCore.dylib 0x1b0a316b8 assertionFailure(_:file:line:) + 96
2 OneFit 0x100a3d238 AdditionalUserInfoRouter.enqueueRoute(with:animated:completion:) + 64 (AdditionalUserInfoRouter.swift:64)
3 OneFit 0x100a53040 protocol witness for MVVMRouter.enqueueRoute(with:animated:completion:) in conformance AdditionalUserInfoRouter + 4374458432 (<compiler-generated>:4374458432)
4 OneFit 0x100638224 MVVMRouter.enqueueRoute(with:) + 32 (MVVMRouter.swift:32)
5 OneFit 0x100a6a6c8 closure #2 in AdditionalUserInfoViewModel.close() + 132 (AdditionalUserInfoViewModel.swift:132)
6 OneFit 0x100a676a0 thunk for #escaping #callee_guaranteed (#guaranteed [Subscription]) -> (#error #owned Error) + 4374541984 (<compiler-generated>:4374541984)
7 OneFit 0x100a6fb94 partial apply for thunk for #escaping #callee_guaranteed (#guaranteed [Subscription]) -> (#error #owned Error) + 4374576020 (<compiler-generated>:4374576020)
8 PromiseKit 0x10344d048 $s10PromiseKit8ThenablePAAE4done2on5flags_AA0A0CyytGSo17OS_dispatch_queueCSg_8Dispatch0J13WorkItemFlagsVSgy1TQzKctFyAA6ResultOyARGcfU_yycfU_ + 64
9 PromiseKit 0x103426614 $sIeg_IeyB_TR + 28
10 libdispatch.dylib 0x1a35a9610 _dispatch_call_block_and_release + 24
11 libdispatch.dylib 0x1a35aa184 _dispatch_client_callout + 16
12 libdispatch.dylib 0x1a355c1d0 _dispatch_main_queue_callback_4CF$VARIANT$mp + 1044
13 CoreFoundation 0x1a385a3c4 __CFRUNLOOP_IS_SERVICING_THE_MAIN_DISPATCH_QUEUE__ + 12
14 CoreFoundation 0x1a38553b8 __CFRunLoopRun + 2004
15 CoreFoundation 0x1a38548bc CFRunLoopRunSpecific + 464
16 GraphicsServices 0x1ad6c0328 GSEventRunModal + 104
17 UIKitCore 0x1a78ea6d4 UIApplicationMain + 1936
18 OneFit 0x1003b3784 main + 39 (AppDelegate.swift:39)
19 libdyld.dylib 0x1a36df460 start + 4
Does Crashlytics logs asserts as crashes or I have some real crash?
UPD: Firebase/Crashlytics support response:
I don't know of any particular Crashlytics-related behavior around
assertionFailure, and unless it's terminating the main app thread I
would not expect us to view it as a crash. I see there's already an
answer on the stackoverflow post - does that clear things up?
Assertions are not supposed to be enabled in release builds with -O optimisations level. This means that in final release code there will be like a blank line and so Crashlytics has nothing to crash at.
Double-check your target's build settings, if you created it from scratch or copied it from debug you may have left assertions enabled.
From docs:
In playgrounds and -Onone builds (the default for Xcode’s Debug
configuration), stop program execution in a debuggable state after
printing message.
In -O builds, has no effect.
In -Ounchecked builds, the optimizer may assume that this function is
never called. Failure to satisfy that assumption is a serious
programming error.

Swift compiler segfaulting in handleDeferredImports

We have started seeing a strange problem, appearing somewhat out of nowhere, where the compiler segfaults in handleDeferredImports. This doesn't always happen, sometimes I need to clean & rebuild 2-3 times to trigger it again, so it's very hard to determine what's the root cause of it.
I've also not been able to reproduce it in a separate sample project, but after two days of looking for solutions I'm drawing a blank.
This is the segfault: error: Segmentation fault: 11 with following stack:
0 swift 0x000000011307064a PrintStackTraceSignalHandler(void*) + 42
1 swift 0x000000011306fdfe SignalHandler(int) + 302
2 libsystem_platform.dylib 0x00007fff76391b3d _sigtramp + 29
3 libsystem_platform.dylib 000000000000000000 _sigtramp + 2311513312
4 swift 0x0000000110605df0 swift::ClangImporter::Implementation::handleDeferredImports() + 512
5 swift 0x00000001106058dc swift::ClangImporter::Implementation::importHeader(swift::ModuleDecl*, llvm::StringRef, swift::SourceLoc, bool, std::__1::unique_ptr<llvm::MemoryBuffer, std::__1::default_delete<llvm::MemoryBuffer> >, bool) + 1804
6 swift 0x0000000110606754 swift::ClangImporter::importBridgingHeader(llvm::StringRef, swift::ModuleDecl*, swift::SourceLoc, bool, bool) + 932
7 swift 0x000000011010acfd swift::CompilerInstance::performSema() + 2029
8 swift 0x000000010f2f859b performCompile(swift::CompilerInstance&, swift::CompilerInvocation&, llvm::ArrayRef<char const*>, int&, swift::FrontendObserver*, swift::UnifiedStatsReporter*) + 731
9 swift 0x000000010f2f4dc5 swift::performFrontend(llvm::ArrayRef<char const*>, char const*, void*, swift::FrontendObserver*) + 7717
10 swift 0x000000010f29aa35 main + 1349
11 libdyld.dylib 0x00007fff761a808d start + 1
12 libdyld.dylib 0x0000000000000123 start + 2313519255
There's no indication to individual files or headers.
We do use a mix of Objective-C and Swift, importing ProjectModuleName-Swift.h in a lot of files, and similar, have a very large ProjectModuleName-Bridging-Header.h file. In addition to this, for legacy reasons, there's a ProjectModuleName-Prefix.pch for default Objective-C includes (yes, I know that's awful).
What's strange here is the segfault in handleDeferredImports which is different from all the other Swift compiler segfault issues found on Stack Overflow.
Swift version is 4.2, and we're using the "New Build System" in Xcode 10 (it also segfaults on the "Legacy Build System").
Workaround for the Swift 4 compiler was to enable Whole Module optimisation for all builds.
The issue itself appears to been fixed in Swift 5.0
Should anyone see this issue in the future, please refer to https://bugs.swift.org/browse/SR-9528

Stored value type does not match pointer operand type! - store i64, swift.bridge, dictionaryUpCasts

We have recently migrated code from Swift 2.3 to 4.0.
The app working fine when run in debug mode on devices as well as simulators. No issues in compile and build.
But as soon as it is archived, it fails in compilation with the following error. And, more importantly, it is not pointing to any location (no file, no function, no statement) in the code.
Even I am finding difficulty in deducing - where and what the issue is.
It is built using Xcode 9.0. Previously I thought it is the issue of Beta version as I was using the Beta and GM seed of XCode 9.0 on Mac OS Sierra.
Stored value type does not match pointer operand type!
store i64 %119, %swift.bridge** %128, align 8, !dbg !402
%swift.bridge*<unknown>:0: error: fatal error encountered during compilation; please file a bug report with your project and the crash log
<unknown>:0: note: Broken function found, compilation aborted!
0 swift 0x000000010454adba PrintStackTraceSignalHandler(void*) + 42
1 swift 0x000000010454a1f6 SignalHandler(int) + 662
2 libsystem_platform.dylib 0x00007fffacd39b3a _sigtramp + 26
3 libsystem_platform.dylib 0x0000000000000004 _sigtramp + 1395418340
4 libsystem_c.dylib 0x00007fffacbbe420 abort + 129
5 swift 0x0000000100eb9fe7 swift::performFrontend(llvm::ArrayRef<char const*>, char const*, void*, swift::FrontendObserver*)::$_0::__invoke(void*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, bool) + 551
6 swift 0x00000001045083fb llvm::report_fatal_error(llvm::Twine const&, bool) + 571
7 swift 0x00000001045081ba llvm::report_fatal_error(char const*, bool) + 42
8 swift 0x00000001044de6b6 (anonymous namespace)::VerifierLegacyPass::runOnFunction(llvm::Function&) + 54
9 swift 0x000000010448baaa llvm::FPPassManager::runOnFunction(llvm::Function&) + 506
10 swift 0x0000000104493609 llvm::legacy::FunctionPassManagerImpl::run(llvm::Function&) + 377
11 swift 0x00000001044933fa llvm::legacy::FunctionPassManager::run(llvm::Function&) + 410
12 swift 0x0000000101033778 swift::performLLVM(swift::IRGenOptions&, swift::DiagnosticEngine*, llvm::sys::SmartMutex<false>*, llvm::GlobalVariable*, llvm::Module*, llvm::TargetMachine*, swift::version::Version const&, llvm::StringRef, swift::UnifiedStatsReporter*) + 5016
13 swift 0x00000001010376d5 ThreadEntryPoint(swift::irgen::IRGenerator*, llvm::sys::SmartMutex<false>*, int) + 181
14 swift 0x000000010103772f void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (*)(swift::irgen::IRGenerator*, llvm::sys::SmartMutex<false>*, int), swift::irgen::IRGenerator*, llvm::sys::SmartMutex<false>*, int> >(void*) + 47
15 libsystem_pthread.dylib 0x00007fffacd4393b _pthread_body + 180
16 libsystem_pthread.dylib 0x00007fffacd43887 _pthread_body + 0
17 libsystem_pthread.dylib 0x00007fffacd4308d thread_start + 13
Stack dump:
0. Running pass 'Module Verifier' on function '#_T0s17_dictionaryUpCasts10DictionaryVyq0_q1_GACyxq_Gs8HashableRzsAFR0_r2_lFTfq4g_nSi_SQySayACyS2SGGGSiAHSgTg5'
This is a bug in Swift code generation.
Select Project, go to your desired Target, and under build settings, you may try by changing code optimization level before you compile. Try whole-module-optimization with the respective configuration, as shown below:
With Swift 3.0 this is the default, and Swift 4.0 shouldn't be any different, IMHO.

Xcode 8.2.1/Swift 3.0 Product/Archive failing with Segmentation Fault 11

I went to create a Archive through Xcode and was presented with
Command failed due to signal: Segmentation fault: 11
#selector(NSDecimalNumberBehaviors.scale)
0 swift 0x00000001067b93ad PrintStackTraceSignalHandler(void*) + 45
1 swift 0x00000001067b8b56 SignalHandler(int) + 790
2 libsystem_platform.dylib 0x00007fffd13feb3a _sigtramp + 26
3 swift 0x0000000106722d51 llvm::MCDwarfLineTableHeader::getFile(llvm::StringRef&, llvm::StringRef&, unsigned int) + 3169
4 swift 0x0000000105ae7c38 llvm::DwarfDebug::endModule() + 1080
5 swift 0x0000000105abe571 llvm::AsmPrinter::doFinalization(llvm::Module&) + 2481
6 swift 0x00000001065edee5 llvm::FPPassManager::doFinalization(llvm::Module&) + 53
7 swift 0x00000001065f789b llvm::legacy::PassManager::run(llvm::Module&) + 1803
8 swift 0x000000010391420a performLLVM(swift::IRGenOptions&, swift::DiagnosticEngine&, llvm::sys::SmartMutex<false>*, llvm::GlobalVariable*, llvm::Module*, llvm::TargetMachine*, llvm::StringRef) + 8234
9 swift 0x0000000103910ccb ThreadEntryPoint(swift::irgen::IRGenerator*, llvm::sys::SmartMutex<false>*, int) + 171
10 swift 0x0000000103910d53 void* std::__1::__thread_proxy<std::__1::tuple<void (*)(swift::irgen::IRGenerator*, llvm::sys::SmartMutex<false>*, int), swift::irgen::IRGenerator*, llvm::sys::SmartMutex<false>*, int> >(void*) + 99
11 libsystem_pthread.dylib 0x00007fffd1408aab _pthread_body + 180
12 libsystem_pthread.dylib 0x00007fffd14089f7 _pthread_body + 0
13 libsystem_pthread.dylib 0x00007fffd14081fd thread_start + 13
The project compiles and runs perfectly fine (Cmd+R)
I've tried:
Clearing the derived data
Rebooting
Searching both Google and SO for any viable options which might match my circumstances.
I've had this before, but have always had some idea of "where" the problem lies, there doesn't seem to be any indication from the stack trace of where to begin to look
The project is (reasonably) large and contains a number of other libraries managed through Carthage, so not easy to post.
I have a "sneaking" suspicion that having opened the project (by mistake) in Xcode 8.3b1 "might" be part of the issue
By following this answer I can make it work, but this is not a long term solution I want to live with.
I know this is long shot and it seems to every question on this has a different cause, but I'd really like to know ANY suggestions that I might look into to solve this issue

iOS 9 GPU crash: gpus_ReturnGuiltyForHardwareRestart

I'm getting a crash periodically with a gpus_ReturnGuiltyForHardwareRestart.
The crash is sporadic, and occurs in a complex multi-threaded app. Sample stack trace below.
It seems to occur (which is rare) when the UI and related handlers are being stressed (think hyperactive user making lots of gestures as quickly as the app and system will allow), with lots of resulting calls to behind-the-scenes and up-front rendering.
Researching gpus_ReturnGuiltyForHardwareRestart suggests this may be due to a buffer issue, e.g. a buffer overrun due to an incorect binding or failure to unbind.
gpus_ReturnGuiltyForHardwareRestart crash
(not relevant, but I did look: gpus_ReturnGuiltyForHardwareRestart)
My understanding is that a buffer gets corrupted in some way, and the crash happens sometime later when the corrupted buffer is accessed.
I've been through the code and made sure every bound buffer and texture is subsequently unbound to prevent unwanted/unintentional changes by later code; still getting the crash.
I did just come across these, suggesting the possibility of a bug in the OS:
iOS 9 Beta 4 crash: gpus_ReturnGuiltyForHardwareRestart (original poster indicated was fixed in 9 beta 5)
Re: OpenGLES driver crash in iOS9 Beta2/3
However, I'm currently testing on 9.3.4, so it seems this would be fixed.I've tried making sure all buffers are properly unbound after use, and tried periodic use of glFlush(), both without success.
Does anybody have experience with this, any knowledge on potential sources, means of tracking down causes, or fixes?
Stack trace:
Date/Time: 2016-08-11T20:20:40Z
Launch Time: 2016-08-11T20:15:22Z
OS Version: iPhone OS 9.3.4 (13G35)
Report Version: 104
Exception Type: SIGSEGV
Exception Codes: SEGV_ACCERR at 0x1
Crashed Thread: 0
Thread 0 Crashed:
0 libGPUSupportMercury.dylib 0x0000000191bd9f28 gpus_ReturnGuiltyForHardwareRestart + 12
1 libGPUSupportMercury.dylib 0x0000000191bdaec4 gpusSubmitDataBuffers + 168
2 GLEngine 0x0000000195e9f1e4 gliPresentViewES_Exec + 172
3 GLEngine 0x0000000195e9f0fc gliPresentViewES + 80
4 OpenGLES 0x000000018576bc44 -[EAGLContext presentRenderbuffer:] + 68
5 NWFPApp 0x00000001001ef8fc -[NWFPIOSGLView renderNormalBuffers] (NWFPIOSGLView.mm:543)
6 NWFPApp 0x00000001001ef814 -[NWFPIOSGLView renderAll] (NWFPIOSGLView.mm:516)
7 NWFPApp 0x00000001001ee780 -[NWFPIOSGLView doInContext:] (NWFPIOSGLView.mm:135)
8 NWFPApp 0x00000001001ef770 -[NWFPIOSGLView drawView] (NWFPIOSGLView.mm:494)
9 NWFPApp 0x0000000100203cc8 -[NWFPGLChoreographer displayLinkEvent:] (NWFPGLChoreographer.m:128)
10 QuartzCore 0x000000018614022c CA::Display::DisplayLinkItem::dispatch() + 36
11 QuartzCore 0x00000001861400e0 CA::Display::DisplayLink::dispatch_items(unsigned long long, unsigned long long, unsigned long long) + 420
12 IOKit 0x0000000183885e54 IODispatchCalloutFromCFMessage + 368
13 CoreFoundation 0x00000001835ad030 __CFMachPortPerform + 176
14 CoreFoundation 0x00000001835c57d4 __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE1_PERFORM_FUNCTION__ + 52
15 CoreFoundation 0x00000001835c4f0c __CFRunLoopDoSource1 + 432
16 CoreFoundation 0x00000001835c2c64 __CFRunLoopRun + 1796
17 CoreFoundation 0x00000001834ecc50 CFRunLoopRunSpecific + 380
18 GraphicsServices 0x0000000184dd4088 GSEventRunModal + 176
19 UIKit 0x00000001887ce088 UIApplicationMain + 200
20 NWFPApp 0x00000001002af1a0 main (main.m:30)
21 ??? 0x000000018308a8b8 0x0 + 0

Resources