I am trying to reproduce such crash(es):
There's Manual Reference Counting in my project. Also, there's a lot of multi-threading.
Some of the properties are not thread-safe. :(
I have just one assumption about the cause of this crash: some object is overreleased (?).
I've added automated UI tests (Appium), but they haven't helped yet.
Also, I've profiled for Zombies - everything seems ok.
Also, I've tried Xcode's Static Analyzer ( Product -> Analyze ), there're a lot of warnings, but none of them seems to be a cause of such crash (I've looked at the warning Incorrect decrement of reference count not owned at this point).
I created a test project with MRC, and added such code:
- (void)testAssumptions {
//#autoreleasepool
{
[self overReleaseNilValue];
[self overReleaseNotNilValue];
}
}
- (void)overReleaseNilValue {
NSIndexPath* path = [[NSIndexPath alloc] initWithIndex:42];
[path release];
[path release];
}
- (void)overReleaseNotNilValue {
NSIndexPath* path = nil;
[path release];
[path release];
}
Releasing an object twice doesn't crash neither with autorelease pool enabled nor without a pool.
So my questions are:
1. What can be another reason of such crash except of releasing already released object?
2. Are there ways to increase a probability of reproducing such crash? E.g. some env. variable which reduces some autorelease pool tolerance to unsafe code? Or some additional autorelease pool?
3. Why doesn't my test project code crash?
Any comments are highly appreciated.
Incorrect decrement of reference count not owned at this point
This would definitely the be warning to explore. It's almost certainly pointing you to at least one error.
Your test project is not actually testing anything (they're also named backwards I believe). There is no promise that over-releasing a value will cause a crash. overReleaseNotNilValue is well-defined behavior, and is absolutely will not crash (sending a message to nil does nothing). overReleaseNilValue is undefined behavior. I haven't dug into it, but I would expect NSIndexPath to be implemented with tagged pointers, which will not crash if you over-release them.
Undefined is undefined. It doesn't mean crash. Over-releasing a value can do anything. If you're lucky, it'll crash....
Also, there's a lot of multi-threading. Some of the properties are not thread-safe.
I would expect this to be at the heart of your problem if it's intermittent. I've worked on such projects. The solution is to fix the problems. You will not know which specific problem is the cause of your crashes. You may never know. You still must fix them all. It will take some time, but you have to make the code thread-safe, or its behavior is undefined.
As to debugging, you will want to do the following, in order:
Turn on the Runtime Sanitization options (under the Scheme editor, Run, Diagnostics). You especially will want the Thread Sanitizer for this.
Clear all Static Analyzer warnings. If any of them say your memory management is wrong, you have to clear those. The system is literally telling you where the problems are. Don't ignore it.
Clear all warnings. There should be zero warnings in your project. If there are lots of "false" warnings then you will never see the real warnings telling you exactly where your problems are. Eliminate all warnings.
I spent 8 months eliminating rare over-release crashes in one a well-written project written by expert developers and almost no threading. It can take a lot of time. You have to clear every problem. Just one incorrect release is enough to crash the program randomly.
Related
I check for the variable's existence, but still get the exc_bad_access! How is this possible? I have tried Zombies, Instruments, and the console, to no avail, and about all else my limited knowledge can muster. Any other suggestions??
if (intDealerCard) {
NSLog(#"%i", intDealerCard); // EXC_BAD_ACCESS - code 2 occurs here
}
Perhaps some things of note...
The code is being executed over 1000 times, as it is a simulation, and it will execute just fine numerous times. I'd like to think there is a problem with my code, but the verified check throws me off. Also, there are two different spots where this may happen, mostly in one of the spots, but occasionally in the other, but still, I'm not sure that is relevant seeing how the check is verified?!
EXC_BAD_ACCESS occurs when memory hasn't been managed properly. Many times an app will crash with this error message in one place, however the memory management error has occurred elsewhere in the logic -- perhaps an unexpected dealloc is occurring on the instance of the class contains the member that is crashing, or if the app is running simulations concurrently, a race condition between a shared resource, etc. In short, the few lines of code in your question are likely not the culprit, just a symptom of a more systemic error with how that variable's memory is being managed.
As you've run Zombies and nothing has shaken loose, consider running the static analyzer (Product > Analyze) -- this class of error can sometimes be detected via static analysis and present one or more code flows that could resulting early release, unexpected deallocation, etc. These analysis errors and compiler warnings should be repaired and then you can retest using NSZombies if the error persists.
Update 2: I found a workaround which is to synchronize MOC deallocating and saving. Please see the updated project.
https://github.com/shuningzhou/MOCDeadLock.git
Note: I made it fail more aggressively. Don't run it on a real device!
Update: A sample project to demonstrate this issue.
https://github.com/shuningzhou/MOCDeadLock.git
XCode 6.2: Unable to reproduce.
XCode 6.3: Reproducible.
XCode 6.4 beta: Reproducible.
========================== The Issue ===============================
Our app randomly stuck on OSSpinLockLockSlow after upgrading to XCode 6.3. In our project, we used NSOperation and NSOperationQueue to fetch data from our server and used Core Data for data persistence.
This issue never happened before! You can see from the stack trace that no calls are made by our code. I am not sure where to start debugging this. Could someone provide some guidance?
Thank you in advance!
Please see the stack trace
Edit:
We are using AFNetworking and our NSOperations are subclasses of AFHTTPRequestOperation. We added some custom properties and overrode the method -(void)start:
- (void)start;
{
//unrelated code...
NSString *completionQueueID = [NSString uuid];
const char *cString = [completionQueueID cStringUsingEncoding:NSASCIIStringEncoding];
self.completionQueue = dispatch_queue_create(cString, DISPATCH_QUEUE_SERIAL);
//unrelated code....
[super start];
}
For Core Data, We are following the thread-confinement pattern. We have separate managed object context for each thread, and the contexts share a static persistent store coordinator.
Edit 2:
More info: I found that this issue happens when the system exits multiple threads at the same time. We store the Managed Object Context in the thread dictionary, and they get released when the threads exit.
[[[NSThread currentThread] threadDictionary] setObject:dataManager forKey:#"IHDataManager"];
CPU usage is around 20%.
I have been experiencing precisely this issue. As per your stack trace, I have a bunch of threads stalled with _OSSpinLockLockSlow.
It appears to be a livelock situation with the spinlocks chained up together. Including some networking threads and core data. But as Rob pointed out, symptoms of livelock should include high CPU usages (spinlocks are all endlessly spinning). In my case (and in yours) this is not the case, CPU usage is low - simulator 'percent used' 20%, simulator overall in activity monitor 0.6% - so maybe it's a deadlock ;-)
Like you, I am using a thread-confinement pattern, separate managed object context per thread, single persistent store.
Following your observation that the hang always seems to follow deallocing of a bunch of threads, I checked that behaviour and can confirm that is the case.
This got me wondering why I had so many threads active. It turned out I was using gcd with a concurrent background queue:
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_BACKGROUND,0),^{
modelClass = [WNManagedObject classForID:mongoID];
dispatch_async(dispatch_get_main_queue(),^{
...
});
});
This snippet is part of some networking/JSON parsing code. 'classForID' was causing slight UI jitters on the main thread, so I backgrounded it.
In effect the concurrent background queue was spitting out a whole bunch of short-lived threads. This was completely unnecessary. Refactoring as a single serial queue fixed the thread excesses, which got rid of the spinlock issue. Finally I realised I didn't need to get the class at all, so this code has since been exorcised.
Problem fixed, but no explanation as to why this should suddenly become an issue with 8.3
I suspect that the same issue is touched on in this question (although Cocoalumberjack gets the blame there):
syscall_thread_switch iOS 8.3 race - CocoaLumberjack bug? how to debug this?
..and in this Cocoalumberjack bug report
https://github.com/CocoaLumberjack/CocoaLumberjack/issues/494
I am also using CocoaLumberjack but it does not feature in any of the problem threads, so I think that is a red herring. The underlying cause seems to be excess thread creation.
I have seen the issue in the simulator and on devices when tethered to XCode, but I have not experienced it when running independently of XCode. It is new to me in iOS 8.3 / XCode 6.3.1
Not really an answer, more of a diary of my own workaround for this weird issue, but maybe you'll find it useful.
If question is still actual - this is a bug in iOS: OpenRadar crash report
Also you may find this blog post useful: blog post
I think you should replace OSSpinLocks with something else to fix this in your app.
We encountered this bug in our Unity3d game. We didnt fixed this yet in our app because we do not have access to most of the native iOS code (we write our game on C# and we use a lot of 3-rd party native plugins). So I cannot recommend you something concrete about replacing OSSpinLock. Sorry for my English.
Update
Many Apple frameworks and libraries uses OSSpinLock internally, so you dont need to use it explicity to run into this issue.
I'm trying to debug an EXC_BAD_ACCESS crash using NSZombie. My app creates lots of large objects though and with NSZombie enabled they aren't getting released causing the app to crash in seconds. This means I can't even cause the EXC_BAD_ACCESS crash before the app crashes due to low memory.
Is there an alternative? Can I enable NSZombie on specific file instead of the entire project? How else could I debug this crash (I know it's caused by UIGestureRecognizer but I use them a lot so it doesn't narrow down the issue significantly).
Thanks.
Edit:
Thanks for the advice. I think I may have solved the issue and will report back after more testing.
Edit 2: Sovled the issue myself but selected the answer which seems like it would be a good solution to any similar issues in the future.
All I can think of is implementing it manually; create a proxy container that holds an object of type id and nominates that as -forwardingTargetForSelector: as well as getting it to respond to -isKindOfClass:, etc.
Disable ARC for the proxy and have it retain itself during init and check its own retainCount when nominating a forwarding target.
If the count is 1 then raise an exception or log a warning or whatever.
Have suspect classes wrap themselves in and return a proxy as the last line of their unit.
For possible bonus points, store [NSThread callStackSymbols] somewhere (probably on disk) during the proxy's unit so you can at least find out where the incorrectly managed object was created.
NSZombies was/is for apps that use their own memory management. If your app uses ARC then this won't help.
Create a new Breakpoint: All Exceptions
That should usually show you where you trigger the bad access.
I would like to know in what situation did you use -retainCount so far, and eventually the problems that can happen using it.
Thanks.
You should never use -retainCount, because it never tells you anything useful. The implementation of the Foundation and AppKit/UIKit frameworks is opaque; you don't know what's being retained, why it's being retained, who's retaining it, when it was retained, and so on.
For example:
You'd think that [NSNumber numberWithInt:1] would have a retainCount of 1. It doesn't. It's 2.
You'd think that #"Foo" would have a retainCount of 1. It doesn't. It's 1152921504606846975.
You'd think that [NSString stringWithString:#"Foo"] would have a retainCount of 1. It doesn't. Again, it's 1152921504606846975.
Basically, since anything can retain an object (and therefore alter its retainCount), and since you don't have the source to most of the code that runs an application, an object's retainCount is meaningless.
If you're trying to track down why an object isn't getting deallocated, use the Leaks tool in Instruments. If you're trying to track down why an object was deallocated too soon, use the Zombies tool in Instruments.
But don't use -retainCount. It's a truly worthless method.
edit
Please everyone go to http://bugreport.apple.com and request that -retainCount be deprecated. The more people that ask for it, the better.
edit #2
As an update,[NSNumber numberWithInt:1] now has a retainCount of 9223372036854775807. If your code was expecting it to be 2, your code has now broken.
NEVER!
Seriously. Just don't do it.
Just follow the Memory Management Guidelines and only release what you alloc, new or copy (or anything you called retain upon originally).
#bbum said it best here on SO, and in even more detail on his blog.
Autoreleased objects are one case where checking -retainCount is uninformative and potentially misleading. The retain count tells you nothing about how many times -autorelease has been called on an object and therefore how many time it will be released when the current autorelease pool drains.
I do find retainCounts very useful when checked using 'Instruments'.
Using the 'allocations' tool, make sure 'Record reference counts' is turned on and you can go into any object and see its retainCount history.
By pairing allocs and releases you can get a good picture of what is going on and often solve those difficult cases where something is not being released.
This has never let me down - including finding bugs in early beta releases of iOS.
Take a look at the Apple documentation on NSObject, it pretty much covers your question:
NSObject retainCount
In short, retainCount is probably useless to you unless you've implemented your own reference counting system (and I can almost guarantee you won't have).
In Apple's own words, retainCount is "typically of no value in debugging memory management issues".
Of course you should never use the retainCount method in your code, since the meaning of its value depends on how many autoreleases have been applied to the object and that is something you cannot predict. However it is very useful for debugging -- especially when you are hunting down memory leaks in code that calls methods of Appkit objects outside of the main event loop -- and it should not be deprecated.
In your effort to make your point you seriously overstated the inscrutable nature of the value. It is true that it is not always a reference count. There are some special values that are used for flags, for example to indicate that an object should never be deallocated. A number like 1152921504606846975 looks very mysterious until you write it in hex and get 0xfffffffffffffff. And 9223372036854775807 is 0x7fffffffffffffff in hex. And it really is not so surprising that someone would choose to use values like these as flags, given that it would take almost 3000 years to get a retainCount as high as the larger number, assuming you incremented the retainCount 100,000,000 times per second.
What problems can you get from using it? All it does is return the retain count of the object. I have never called it and can't think of any reason that I would. I have overridden it in singletons to make sure they aren't deallocated though.
You should not be worrying about memory leaking until your app is up and running and doing something useful.
Once it is, fire up Instruments and use the app and see if memory leaks really happen. In most cases you created an object yourself (thus you own it) and forgot to release it after you were done.
Don't try and optimize your code as you are writing it, your guesses as to what may leak memory or take too long are often wrong when you actually use the app normally.
Do try and write correct code e.g. if you create an object using alloc and such, then make sure you release it properly.
Never use the -retainCount in your code. However if you use, you will never see it returns zero. Think about why. :-)
You should never use it in your code, but it could definitely help when debugging
The examples used in Dave's post are NSNumber and NSStrings...so, if you use some other classes, such as UIViews, I'm sure you will get the correct answer(The retain count depends on the implementation, and it's predictable).
I am debugging an iPhone app that was written by someone else, it doesn't use arc. I have tried using arc but they have a lot of old code that uses some c style void pointers, so the program crashes when I do.
I have given up on that idea as a lost cause. The problem is that the allocations / leak tools don't show a memory leak, but the Activity monitor shows memory being leaked every time the user swipes to change a page. (about 1.5 Meg each swipe) Needless to say the program crashes.
Being a c++ programmer myself, with some objective c experience I don't understand all the ins and outs of where the memory is being held onto. Can someone give me some pointers on how to track this one down.
Thanks
I highly recommend spending a day and reading the Objective-C Programming Guide in depth (the retain/release/auto release sections) until it clicks. Also properties which are almost for sure in play here.
Specifically, some memory has been retained by either one too many retains, by a property setter doing that as a side effect, or a view holding onto UI elements (getting hidden but not removed from its super view).
In the Instruments Allocations view, you can see the types of objects that are not getting released - this should help track it down.
Another trick is to put log statements in dealloc methods, to see what large objects are NOT getting released (no log statements in the console).
Your last option is to either post your code and ask for help here, or hire someone to get you past this or to ARCify the code.
EDIT: I had an idea over the weekend on how one might track arbitrary objects - to get an indication of whether they were getting released or not, so I created an Object Tracker project. Using the Tracker class, you can mark objects for tracking, and when the object gets dealloced you'll see a log message in Xcode. Also, you can query the class object to see what objects are still living.
It works by using objc_setAssociatedObject() to attache itself to the tracked object, then logs itself when that object is dealloced (and thus releases the tracker object.
So, you can try a few objects in your project - views, scrollViews, images, arrays, etc, and see if you can at least find the object that isn't getting dealloced when it should be.
Hope this helps.