Proper way of preventing GPU access with Core Image when app is in background - ios

I'm experiencing crashes from customers, with the following backtrace:
0 libGPUSupportMercury.dylib 0x3542ae2e gpus_ReturnNotPermittedKillClient + 10
1 IMGSGX543RC2GLDriver 0x30bbf5e5 SubmitPacketsIfAny + 245
2 GLEngine 0x32f827db glFinish_Exec + 167
3 CoreImage 0x31fb85b7 CI::GLESContext::recursive_render(CI::Node const*, CGRect, bool) + 219
4 CoreImage 0x31fbb351 CI::GLESContext::render(CI::Node*, CGRect) + 41
5 CoreImage 0x31fc2901 CI::image_get_cgimage(CI::Context*, CI::Image*, CGRect, CGColorSpace*, CI::PixelFormat) + 1313
6 CoreImage 0x31fa8427 -[CIContext createCGImage:fromRect:format:colorSpace:] + 487
7 CoreImage 0x31fa81e9 -[CIContext createCGImage:fromRect:] + 89
8 App 0x0013c9db -[PZTiledImageLayer drawInContext:] (PZTiledImageLayer.m:129)
Which are due to accessing the GPU when the app is in background (not allowed).
The code leading to this crash is this:
if([UIApplication sharedApplication].applicationState == UIApplicationStateActive)
{
cg_image = [self.imageContext createCGImage:im fromRect:rclip];
}
This means the app changes state after my check, but before GPU is accessing within the Core Image API.
What is the proper way of handling the situation of app background state when using this Core Image API?

If this is happening on a background thread, you could try moving it to the main thread. That way the execution of this function is not interleaved with the AppDelegate callback
- (void)applicationDidEnterBackground:(UIApplication *)application
and you can implement some signalling there without worrying about concurrency.

Related

Crash on one specific Unity scene in iOS build - TexturesMetal::AddCreatedTexture

I am having a crash only on one specific scene in my AR Foundation Unity build with the following error.
Exception Type: EXC_BAD_ACCESS (SIGSEGV)
Exception Subtype: KERN_INVALID_ADDRESS at 0x0000000000000020
VM Region Info: 0x20 is not in any region. Bytes before following region: 4362387424
REGION TYPE START - END [ VSIZE] PRT/MAX SHRMOD REGION DETAIL
UNUSED SPACE AT START
__TEXT 10404c000-104050000 [ 16K] r-x/r-x SM=COW ...lycoroTestApp
Termination Signal: Segmentation fault: 11
Termination Reason: Namespace SIGNAL, Code 0xb
Terminating Process: exc handler [5380]
Triggered by Thread: 28
Thread 28 name:
Thread 28 Crashed:
0 libobjc.A.dylib 0x0000000195ac5ac0 objc_retain + 16 (objc-object.h:636)
1 UnityFramework 0x00000001066d5320 TexturesMetal::AddCreatedTexture(TextureID, id, bool) + 32 (TexturesMetal.mm:481)
2 UnityFramework 0x0000000106634828 GfxDeviceWorker::RunCommand(ThreadedStreamBuffer&) + 21972 (GfxDeviceWorker.cpp:0)
3 UnityFramework 0x00000001066aa2ec GfxDeviceWorkerAutoreleasePoolProxy + 68 (GfxDeviceMetal.mm:5242)
4 UnityFramework 0x0000000106636bd0 GfxDeviceWorker::RunExt(ThreadedStreamBuffer&) + 92 (GfxDeviceWorker.cpp:389)
5 UnityFramework 0x0000000106636b68 GfxDeviceWorker::Run() + 156 (GfxDeviceWorker.cpp:373)
6 UnityFramework 0x000000010662ef70 GfxDeviceWorker::RunGfxDeviceWorker(void*) + 12 (GfxDeviceWorker.cpp:352)
7 UnityFramework 0x00000001062289f8 Thread::RunThreadWrapper(void*) + 496 (Thread.cpp:81)
8 libsystem_pthread.dylib 0x00000001c9777c74 _pthread_start + 288 (pthread.c:887)
9 libsystem_pthread.dylib 0x00000001c977c878 thread_start + 8
At first I thought it was because I was missing the Environmental Probe manager in that scene (no). Then I thought perhaps it was because I was using a specific shader that didn't use a _MainTex. But I use that shader no problem in other working scenes.
And then I built the scene with only a simple primitive and my UI and still received this error. After that crash the scene did open successfully, but obviously the problem is still there. The UI is the exact same as the other working scenes.
I can't find out much about this specific error: TexturesMetal::AddCreatedTexture - only that there may be this bug with GLES3 and PVRTC textures - but they aren't present in the most recent build.
https://issuetracker.unity3d.com/issues/ios-crash-in-uploadtexture-at-texturesmeta-dot-mm-or-drawbufferranges-at-gfxdevicegles-dot-cpp-using-atlas-with-pvrtc-compress
The scene works perfectly in Android - it only crashes in iOS.
Any ideas? Full report below.
I ran a Product>Analysis in Xcode and it gave me this warning:
/Users/xcodeclub/Downloads/Eliot Silver/Classes/UI/UnityView.mm:200:5: nil returned from a method that is expected to return a non-null value
For this bit of Code in UnityView.mm:
static Class UnityRenderingView_LayerClassMTL(id self_, SEL _cmd)
{
return NSClassFromString(#"CAMetalLayer");
}
static Class UnityRenderingView_LayerClassNULL(id self_, SEL _cmd)
{
return NSClassFromString(#"CALayer");
}
#implementation UnityRenderingView
(Class)layerClass
{
return nil;
}
(void)InitializeForAPI:(UnityRenderingAPI)api
{
IMP layerClassImpl = api == apiMetal ? (IMP)UnityRenderingView_LayerClassMTL : (IMP)UnityRenderingView_LayerClassNULL;
class_replaceMethod(object_getClass([UnityRenderingView class]), #Selector(layerClass), layerClassImpl, UIView_LayerClass_Enc);
}
#End

iOS application is getting crashed if we continuously publish / unpublish the audio and video track, TwillioVideo SDK

Description
in my application I am publishing and unpublishing the audio and video tracks when user mute / unmute audio and enable / disable the camera. After 2 three continues tap the application is getting crashed.
I tried to disable the mic button interaction until the audio is not published, as we get callback in localParticipantDidPublishAudioTrack but for un-publish there is no callback. That logic is also crashing. How can I fix this crash
Also please suggest what will be the side effect if I will enable/disable the audio track instead of publish / unpublish. Also what is the recommended approach for both audio and video
Steps to Reproduce
While on the call try monkey tap on mute button and from the code inside publish & unpublish the audio track on each respective tap
Code
#IBAction func micControlAction(_ sender: Any) {
//Toggle mic
guard let participant = callViewModel.getCallParticipant(type: .patient) else { return }
if let audioTrack = participant.audioTrack as? LocalAudioTrack {
if audioTrack.isEnabled {
(participant.participant as? LocalParticipant)?.unpublishAudioTrack(audioTrack)
}
else {
(participant.participant as? LocalParticipant)?.publishAudioTrack(audioTrack)
}
audioTrack.isEnabled = !audioTrack.isEnabled
//This will call the observer and the UI will gets updated according to the need
participant.micEnabled.property = audioTrack.isEnabled
}
}
Expected behavior: The user should be able to mute / unmute the mic without any delay.
Actual behavior: Application crashes 100% of time
Logs
libc++abi.dylib: Pure virtual function called!
Pure virtual function called!
Versions
Video iOS SDK 4.6.0 via CocoaPods
Xcode 13.1
iOS Version 13.6.1
iOS Device iPad Air
Based on suggestion on this link, it seems that TwillioVideo 4.6.0 SDK is having issue and reverting to TwillioVideo SDK to version 4.4.0 works fine.
The crash log when using version 4.6.0:
Thread 31 name:
Thread 31 Crashed:
0 libsystem_kernel.dylib 0x00000001ba0e39c4 __pthread_kill + 8
1 libsystem_pthread.dylib 0x00000001f3333434 pthread_kill + 268 (pthread.c:1609)
2 libsystem_c.dylib 0x000000018e562f64 abort + 164 (abort.c:118)
3 libc++abi.dylib 0x000000019bd89bc4 abort_message + 132 (abort_message.cpp:78)
4 libc++abi.dylib 0x000000019bd89594 __cxa_pure_virtual + 24 (cxa_virtual.cpp:16)
5 TwilioVideo 0x0000000105686668 0x104fe4000 + 6956648
6 TwilioVideo 0x000000010568646c 0x104fe4000 + 6956140
7 TwilioVideo 0x0000000105420e0c 0x104fe4000 + 4443660
8 TwilioVideo 0x0000000105420db4 0x104fe4000 + 4443572
9 TwilioVideo 0x0000000105499924 0x104fe4000 + 4938020
10 TwilioVideo 0x0000000105499188 0x104fe4000 + 4936072
11 TwilioVideo 0x0000000105498128 0x104fe4000 + 4931880
12 TwilioVideo 0x0000000105499568 0x104fe4000 + 4937064
13 libsystem_pthread.dylib 0x00000001f332da60 _pthread_start + 148 (pthread.c:890)
14 libsystem_pthread.dylib 0x00000001f332cf5c thread_start + 8

iOS Core Data dispatch_async background queuing crash

I'm getting a non-reproduceable crash and I'm not sure why. I'm caching images on a background queue. The image names are properties on a Core Data NSManagedObject subclass, CCCard. At the same time, I have a collection view that is also accessing these CCCards.
Here is the relevant code and notes to follow.
//CCDeckViewController.m --------------------------------------
- (void)viewDidLoad {
// This will fetch the CCCard objects from Core Data.
self.cards = [[CCDataManager shared] cards];
[self cacheImages];
}
- (void)cacheCardImages {
// Because these are NSManagedObjects, I access them in the background
// via their object ID. So pull the IDs out here.
NSArray *cardIds = [self.cards valueForKey:#"objectID"];
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_LOW, 0), ^{
for (NSManagedObjectID *cardId in cardIds) {
// Fetch the actual CCCard object.
CCCard *card = (CCCard *)[[CCDataManager shared] objectWithId:cardId];
// Cache the card's image if it's not already there.
NSString *key = [card thumbnailCacheKey];
if ([self.cardImageCache objectForKey:key]) {
continue;
}
CCDiscardableImage *discardable = [CCHelper decompressedImageForPath:key size:[card thumbnailSize] tintable:[card tintable]];
[self.cardImageCache setObject:discardable forKey:key];
}
});
}
- (UICollectionViewCell *)collectionView:(UICollectionView *)collectionView cellForItemAtIndexPath:(NSIndexPath *)indexPath {
CCCard *card = self.cards[indexPath.item];
// This line calls the code that crashes.
UIColor *color = [card color];
// More code that returns the cell.
}
// CCCard.m --------------------------------------
- (UIColor *)color {
// Crash happens on this line.
if (self.colorId.integerValue == 0) {
// some code
}
}
Here are parts of the stack trace from the Crashlytics report I think are most relevant:
Thread #0: Crashed: com.apple.main-thread
EXC_BREAKPOINT 0x000000000000defe
0 CoreData 0x24c1b070 _sharedIMPL_pvfk_core + 247
1 CoreData 0x24c1b071 _sharedIMPL_pvfk_core + 248
2 myapp 0x4286f -[CCCard color] (CCCard.m:37)
3 myapp 0x40bbb -[CCDeckViewController collectionView:cellForItemAtIndexPath:] (CCDeckViewController.m:127)
Thread #4: com.apple.root.utility-qos
0 libsystem_pthread.dylib 0x2321920c RWLOCK_GETSEQ_ADDR
1 libsystem_pthread.dylib 0x23219295 pthread_rwlock_unlock + 60
2 libobjc.A.dylib 0x22cb8e01 rwlock_tt<false>::unlockRead() + 8
3 libobjc.A.dylib 0x22cb5af5 lookUpImpOrForward + 488
4 libobjc.A.dylib 0x22cb5903 _class_lookupMethodAndLoadCache3 + 34
5 libobjc.A.dylib 0x22cbbd7b _objc_msgSend_uncached + 26
6 CoreData 0x24c1d3ab _PFObjectIDFastEquals64 + 38
7 CoreFoundation 0x233eeed4 CFBasicHashFindBucket + 1820
8 CoreFoundation 0x233ee775 CFDictionaryGetValue + 116
9 CoreData 0x24c17037 _PFCMT_GetValue + 122
10 CoreData 0x24c16ec7 -[NSManagedObjectContext(_NSInternalAdditions) _retainedObjectWithID:optionalHandler:withInlineStorage:] + 58
11 CoreData 0x24c1b45d _PF_FulfillDeferredFault + 940
12 CoreData 0x24c1afcf _sharedIMPL_pvfk_core + 86
13 myapp 0x42991 -[CCCard imagePath] (CCCard.m:53)
14 myapp 0x41d5b __39-[CCDeckViewController cacheCardImages]_block_invoke (CCDeckViewController.m:295)
It's frustrating because it never happens during development, so can't test any theories. I'm trying to understand the code and crash report to figure out the problem now. My guess is that it has something to do with faulting because it accesses the CCCard object's instance method, but then crashes when trying to read the property from it. But why?
You are violating thread confinement.
Per the Core Data documentation, a NSManagedObjectContext and any NSManagedObject associated with it must only be accessed on the queue that it is assigned to.
Your dispatch_async violates that rule and needs to be corrected. There is no longer a design where you can use Core Data in a dispatch_async like that.
If you want the processing to be asynchronous then you should spawn a private NSManagedObjectContext that is a child of your main NSManagedObjectContext and then execute a -performBlock: against it. That will give you background processing that is configured properly.
Also, I highly recommend turning on -com.apple.CoreData.ConcurrencyDebug 1 as that will catch issues like this during development.

iOS screen become black little by little

I'm implementing an iOS app. and found that sometimes the screen became black little by little.
for example, in a view controller, there are a collection view, a button, a page controller, and sometimes I found the collection view became black(or invisible), only black background is shown, after 1-2 seconds, the button is gone, then the whole screen is black. but if I put the app into background, then bring it back, everything is ok.
here is the code:
- (void)application:(UIApplication *)application performFetchWithCompletionHandler:(void (^)(UIBackgroundFetchResult))completionHandler
{
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_BACKGROUND, 0), ^{
NSTimeInterval timeLeft = [UIApplication sharedApplication].backgroundTimeRemaining - 1.0;
if(timeLeft &lt 0)
{
completionHandler(UIBackgroundFetchResultNoData);
}
//create new thread to do something, and sleep at current thread
...create new threads
[NSThread sleep:timeLeft];
completionHandler(UIBackgroundFetchResultNewData);
});
}
the issue could be reproduced if I repeat the following actions several times:
Do some thing
Put app to background(press home button)
Bring app back to foreground
repeat 1 - 3
we found following error in organizer for our app:
: CoreAnimation: warning, deleted thread with uncommitted CATransaction; set CA_DEBUG_TRANSACTIONS=1 in environment to log backtraces.
after added , I got the following log:
Myapp[11496] : CoreAnimation: warning, deleted thread with uncommitted CATransaction; created by:
0 QuartzCore 0x31ca6a75 + 268
1 QuartzCore 0x31ca6929 + 224
2 QuartzCore 0x31cabddb + 86
3 QuartzCore 0x31cab9f7 + 150
4 UIKit 0x3203f919 + 344
5 UIKit 0x320bb11b + 138
6 UIKit 0x322b0ebf + 218
7 UIKit 0x322b1169 + 104
8 UIKit 0x322b1735 + 36
9 Myapp 0x002e538d __61-[AppDelegate application:performFetchWithCompletionHandler:]_block_invoke + 632
10 libdispatch.dylib 0x3a487d53 + 10
11 libdispatch.dylib 0x3a48d689 + 228
12 libdispatch.dylib 0x3a48d8dd + 56
13 libsystem_pthread.dylib 0x3a5b8c17 _pthread_wqthread + 298
14 libsystem_pthread.dylib 0x3a5b8adc start_wqthread + 8
my questions is:
how could application:performFetchWithCompletionHandler cause animation issue?
to answer questions:
1. I'm sure that phone is not going to sleep
2. source code. sorry the project is too big
Check that you are working with UI on main thread only, this should be the case.
Edit: - I've seen this one, although never used it(written by Peter Steinberger).
Also this answer can help avoiding several more problems.
The thread created to perform your fetch invokes a call to update your ui, when it finishes performing whatever task it is doing, and its no longer needed it is deallocated. UIUpdates MUST be performed on the main thread or they might not be performed right away.
Have you tried dispatching to the main thread whatever ui update you are performing there?
It's hard to know what you your completion handler is doing, but if it's updating UI try to dispatch them on the main thread
- (void)application:(UIApplication *)application performFetchWithCompletionHandler:(void (^)(UIBackgroundFetchResult))completionHandler
{
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_BACKGROUND, 0), ^{
NSTimeInterval timeLeft = [UIApplication sharedApplication].backgroundTimeRemaining - 1.0;
if(timeLeft < 0)
{
dispatch_async(dispatch_get_main_queue(), ^{
completionHandler(UIBackgroundFetchResultNoData);
});
}
//create new thread to do something, and sleep at current thread
...create new threads
[NSThread sleep:timeLeft];
dispatch_async(dispatch_get_main_queue(), ^{
completionHandler(UIBackgroundFetchResultNewData);
});
});
}

How to debug EXC_BAD_ACCESS in Apple's Accelerator code?

On sporadic occasions I'm getting an EXC_BAD_ACCESS (SIGSEV) in one of Apple's Accelerator framework methods, vDSP_fft2d_zip. The crash report gave the following for Thread 10. Again, most of the time this works like a champ. Any ideas on how to find the cause and fix it?
Exception Type: EXC_BAD_ACCESS (SIGSEGV)
Exception Subtype: KERN_INVALID_ADDRESS at 0x412f2740
Triggered by Thread: 10
. . .
Thread 10 Crashed:
0 libvDSP.dylib 0x2ddfcdb4 ___lldb_unnamed_function431$$libvDSP.dylib + 68
1 libvDSP.dylib 0x2ddfab66 ___lldb_unnamed_function427$$libvDSP.dylib + 562
2 libvDSP.dylib 0x2ddfa8ee vDSP_fft2d_zop + 434
3 libvDSP.dylib 0x2ddfa732 vDSP_fft2d_zip + 18
4 ASSIST for iPad 0x000ac32e -[Processor setupFilterForBubbleSizeMM:rectWidth:rectImageWidth:dataFilled:dataUnfilled:] (Processor.mm:813)
5 ASSIST for iPad 0x000af4d2 -[Processor runConvolutionProcessOnImage:aveImage:] (Processor.mm:1835)
6 ASSIST for iPad 0x000ac5f6 -[Processor processAnswers] (Processor.mm:869)
7 ASSIST for iPad 0x000ae9cc -[Processor process] (Processor.mm:1644)
8 ASSIST for iPad 0x000d7744 -[ReallTimeScanner processImage:] (ReallTimeScanner.mm:1046)
9 Foundation 0x2f296c82 __NSThread__main__ + 1058
10 libsystem_pthread.dylib 0x396b6c1a _pthread_body + 138
11 libsystem_pthread.dylib 0x396b6b8a _pthread_start + 98
12 libsystem_pthread.dylib 0x396b4c8c thread_start + 4
The relevant code looks like this:
#define FFT_SIZE 512
#define FFT_POWER 9
dataFilledIn->imagp=(float *) malloc(sizeof(float)*FFT_SIZE*FFT_SIZE);
dataFilledIn->realp=(float *) malloc(sizeof(float)*FFT_SIZE*FFT_SIZE);
dataUnfilledIn->imagp=(float *) malloc(sizeof(float)*FFT_SIZE*FFT_SIZE);
dataUnfilledIn->realp=(float *) malloc(sizeof(float)*FFT_SIZE*FFT_SIZE);
memset(dataFilledIn->imagp, 0, sizeof(float)*FFT_SIZE*FFT_SIZE);
memset(dataFilledIn->realp, 0, sizeof(float)*FFT_SIZE*FFT_SIZE);
memset(dataUnfilledIn->imagp, 0, sizeof(float)*FFT_SIZE*FFT_SIZE);
memset(dataUnfilledIn->realp, 0, sizeof(float)*FFT_SIZE*FFT_SIZE);
for(float y=0; y<FFT_SIZE; y+=0.25)
{
for(float x=0; x<FFT_SIZE; x+=0.25)
{
dataFilledIn->realp[(int)y*FFT_SIZE+(int)x] += 0.0625;
dataUnfilledIn->realp[(int)y*FFT_SIZE+(int)x] -= 0.0625;
}
}
vDSP_fft2d_zip(setup, dataFilledIn, 1, 0, FFT_POWER, FFT_POWER, kFFTDirection_Forward);
Answering my own question here: Be sure that the very first parameter sent in (type FFTSetup) to vDSP_fft2d_zip() is initialized correctly, and you should use this call
vDSP_destroy_fftsetup(setup);
to clean it up. I was initializing the setup var properly, like this
setup = vDSP_create_fftsetup(FFT_POWER, kFFTRadix2);
and then cleaning it up, but then was not initializing it again before subsequent calls to vDSP_fft2d_zip(). Unfortunately it only rarely caused a crash, which made it seem like the frequent calls to vDSP_fft2d_zip(setup, . . .) were working when they weren't. The crashes were sporadic making the real cause hard to trace.
There are some articles in Raywenderlich may help you:
Objectively Speaking: A Crash Course in Objective C for iOS 6
Objectively Speaking 2: A Crash Course in Objective C for iOS 6

Resources