The following is my checking gcroot of the object 0330e9a8.
I found Its root is a hanle with its type is "RefCnt".
How could I get the actual reference count of it. And How could I know WHICH object obtained its reference?
0:042> !GCRoot 0330e9a8
Note: Roots found on stacks may be false positives. Run "!help gcroot" for
more info.
Scan Thread 5 OSTHread 534
Scan Thread 8 OSTHread 60c
Scan Thread 9 OSTHread 1120
Scan Thread 10 OSTHread 1b30
Scan Thread 11 OSTHread 52c
Scan Thread 12 OSTHread 1da8
Scan Thread 15 OSTHread 1cbc
Scan Thread 19 OSTHread 1808
Scan Thread 22 OSTHread 1b6c
Scan Thread 23 OSTHread 1e70
Scan Thread 0 OSTHread 1a34
Scan Thread 32 OSTHread 1840
DOMAIN(001C1050):HANDLE(RefCnt):d31460:Root:0330edf8(xxxxxxx)->
0330eedc(xxxxxxxxxxxx)->
0330e9a8(xxxxxxxxxxxxxx)
It's a GC handle that keeps .NET objects reachable for unmanaged code.
for example: subscribing to COM object event.
What types of roots would you normally see when running !gcroot and what do they mean?
ESP: ESP=Extended Stack Pointer, Object is in use on a stack
Strong: Strong reference, Typically a static variable
WeakLn: Weak Long Handle, A weak reference that is tracked through finalization (can be resurrected)
WeakSh: Weak Short Handle, A weak reference, can't be resurrected
Pinned: Pinned object, pinned at a specific address, can't move around during garbage collection.
RefCnt: Reference count, referenced as long as the reference count is > 0.
See: .NET Debugging Demos Lab 7: Memory Leak - Review
Related
I've written a simulator, which is distributed over two hosts. When I launch a few thousand processes, after about 10 minutes and half a million events written, my main Erlang (OTP v22) virtual machine crashes with this message:
no next heap size found: 18446744071789822643, offset 0.
It's always that same number - 18446744071789822643.
Because my server is very capable, the crash dump is also huge and I can't view it on my headless server (no WX installed).
Are there any tips on what I can look at?
What would be the first things I can try out to debug this issue?
First, see what memory() says:
> memory().
[{total,18480016},
{processes,4615512},
{processes_used,4614480},
{system,13864504},
{atom,331273},
{atom_used,306525},
{binary,47632},
{code,5625561},
{ets,438056}]
Check which one is growing - processes, binary, ets?
If it's processes, try typing i(). in the Erlang shell while the processes are running. You'll see something like:
Pid Initial Call Heap Reds Msgs
Registered Current Function Stack
<0.0.0> otp_ring0:start/2 233 1263 0
init init:loop/1 2
<0.1.0> erts_code_purger:start/0 233 44 0
erts_code_purger erts_code_purger:wait_for_request 0
<0.2.0> erts_literal_area_collector:start 233 9 0
erts_literal_area_collector:msg_l 5
<0.3.0> erts_dirty_process_signal_handler 233 128 0
erts_dirty_process_signal_handler 2
<0.4.0> erts_dirty_process_signal_handler 233 9 0
erts_dirty_process_signal_handler 2
<0.5.0> erts_dirty_process_signal_handler 233 9 0
erts_dirty_process_signal_handler 2
<0.8.0> erlang:apply/2 6772 238183 0
erl_prim_loader erl_prim_loader:loop/3 5
Look for a process with a very big heap, and that's where you'd start looking for a memory leak.
(If you weren't running headless, I'd suggest starting Observer with observer:start(), and look at what's happening in the Erlang node.)
I'm using Ipython parallel in an optimisation algorithm that loops a large number of times. Parallelism is invoked in the loop using the map method of a LoadBalancedView (twice), a DirectView's dictionary interface and an invocation of a %px magic. I'm running the algorithm in an Ipython notebook.
I find that the memory consumed by both the kernel running the algorithm and one of the controllers increases steadily over time, limiting the number of loops I can execute (since available memory is limited).
Using heapy, I profiled memory use after a run of about 38 thousand loops:
Partition of a set of 98385344 objects. Total size = 18016840352 bytes.
Index Count % Size % Cumulative % Kind (class / dict of class)
0 5059553 5 9269101096 51 9269101096 51 IPython.parallel.client.client.Metadata
1 19795077 20 2915510312 16 12184611408 68 list
2 24030949 24 1641114880 9 13825726288 77 str
3 5062764 5 1424092704 8 15249818992 85 dict (no owner)
4 20238219 21 971434512 5 16221253504 90 datetime.datetime
5 401177 0 426782056 2 16648035560 92 scipy.optimize.optimize.OptimizeResult
6 3 0 402654816 2 17050690376 95 collections.defaultdict
7 4359721 4 323814160 2 17374504536 96 tuple
8 8166865 8 196004760 1 17570509296 98 numpy.float64
9 5488027 6 131712648 1 17702221944 98 int
<1582 more rows. Type e.g. '_.more' to view.>
You can see that about half the memory is used by IPython.parallel.client.client.Metadata instances. A good indicator that results from the map invocations are being cached is the 401177 OptimizeResult instances, the same number as the number of optimize invocations via lbview.map - I am not caching them in my code.
Is there a way I can control this memory usage on both the kernel and the Ipython parallel controller (who'se memory consumption is comparable to the kernel)?
Ipython parallel clients and controllers store past results and other metadata from past transactions.
The IPython.parallel.Client class provides a method for clearing this data:
Client.purge_everything()
documented here. There is also purge_results() and purge_local_results() methods that give you some control over what gets purged.
I am writing/sending 40 bytes to a peripheral using CBCharacteristicWriteWithResponse (BLE112 using BGLib) I would have expected it to arrive in two blocks of 20 bytes.
Instead it is arriving in three blocks 18, 18 & 4.
Any idea why I am seeing this behaviour?
I have been sniffing the initial exchange after a connection.
Via an ATT_Exchange_MTU_Req the iDevice says it can support MTU of 135 bytes! (0x0087) the BLE112 then responds with an ATT_Exchange_MTU_Rsp of 23 bytes. iOS remembers and uses this value as the basis of future comms with the peripheral (the whole point of ATT_Exchange_MTU_Req/Rsp)
In the BTSpec4.0.pdf section 3.4.6.1 Prepare Write Request it says that the Part Attribute value has ATT_MTU-5 = 23 - 5 = 18 bytes to use contain a part of the value.
Had this confirmed by a BLE guru # Apple so reasonably sure this is correct.
There is a part of my app where I perform operations concurrently. They consist of initializing many CALayers and rendering them to bitmaps.
Unfortunately, during these operations (each takes about 2 seconds to complete on an iphone 4), the Dirty Size indicated by VM Tracker spikes to ~120MB. Allocations spike to ~12MB(does not accumulate) From my understanding, the Dirty size is memory that cannot be freed. so often, my app and all other apps in the background gets killed.
Incident Identifier: 7E6CBE04-D965-470D-A532-ADBA007F3433
CrashReporter Key: bf1c73769925cbff86345a576ae1e576728e5a11
Hardware Model: iPhone3,1
OS Version: iPhone OS 5.1.1 (9B206)
Kernel Version: Darwin Kernel Version 11.0.0: Sun Apr 8 21:51:26 PDT 2012; root:xnu-
1878.11.10~1/RELEASE_ARM_S5L8930X
Date: 2013-03-18 19:44:51 +0800
Time since snapshot: 38 ms
Free pages: 1209
Active pages: 3216
Inactive pages: 1766
Throttled pages: 106500
Purgeable pages: 0
Wired pages: 16245
Largest process: Deja Dev
Processes
Name UUID Count resident pages
geod <976e1080853233b1856b13cbd81fdcc3> 338
LinkedIn <24325ddfeed33d4fb643030edcb12548> 6666 (jettisoned)
Music~iphone <a3a7a86202c93a6ebc65b8e149324218> 935
WhatsApp <a24567991f613aaebf6837379bbf3904> 2509
MobileMail <eed7992f4c1d3050a7fb5d04f1534030> 945
Console <9925a5bd367a7697038ca5a581d6ebdf> 926 (jettisoned)
Test Dev <c9b1db19bcf63a71a048031ed3e9a3f8> 81683 (active)
MobilePhone <8f3f3e982d9235acbff1e33881b0eb13> 867
debugserver <2408bf4540f63c55b656243d522df7b2> 92
networkd <80ba40030462385085b5b7e47601d48d> 158
notifyd <f6a9aa19d33c3962aad3a77571017958> 234
aosnotifyd <8cf4ef51f0c635dc920be1d4ad81b322> 438
BTServer <31e82dfa7ccd364fb8fcc650f6194790> 275
CommCenterClassi <041d4491826e3c6b911943eddf6aaac9> 722
SpringBoard <c74dc89dec1c3392b3f7ac891869644a> 5062 (active)
aggregated <a12fa71e6997362c83e0c23d8b4eb5b7> 383
apsd <e7a29f2034083510b5439c0fb5de7ef1> 530
configd <ee72b01d85c33a24b3548fa40fbe519c> 465
dataaccessd <473ff40f3bfd3f71b5e3b4335b2011ee> 871
fairplayd.N90 <ba38f6bb2c993377a221350ad32a419b> 169
fseventsd <914b28fa8f8a362fabcc47294380c81c> 331
iapd <0a747292a113307abb17216274976be5> 323
imagent <9c3a4f75d1303349a53fc6555ea25cd7> 536
locationd <cf31b0cddd2d3791a2bfcd6033c99045> 1197
mDNSResponder <86ccd4633a6c3c7caf44f51ce4aca96d> 201
mediaremoted <327f00bfc10b3820b4a74b9666b0c758> 257
mediaserverd <f03b746f09293fd39a6079c135e7ed00> 1351
lockdownd <b06de06b9f6939d3afc607b968841ab9> 279
powerd <133b7397f5603cf8bef209d4172d6c39> 173
syslogd <7153b590e0353520a19b74a14654eaaa> 178
wifid <3001cd0a61fe357d95f170247e5458f5> 319
UserEventAgent <dc32e6824fd33bf189b266102751314f> 409
launchd <5fec01c378a030a8bd23062689abb07f> 126
**End**
On closer inspection, the dirty memory consists mostly of Image IO and Core Animation pages. multiple entries consisting of hundreds to thousands of pages. What does Image IO and Core Animation do exactly? and how can I reduce the Dirty Memory?
edit: tried doing this on a serial queue and no improvement on size of Dirty memory
another question. how large is too large for Dirty Memory and allocations?
Updated:
- (void) render
{
for (id thing in mylist) {
#autorelease {
CALayer *layer = createLayerFromThing(thing);
UIImage *img = [self renderLayer:layer];
[self writeToDisk:img];
}
}
}
in createLayerFromThing(thing); I actually creating a layer with a huge amount of sub layers
UPDATED
first screenshot for maxConcurrentOperationCount = 4
second for maxConcurrentOperationCount = 1
============================================================================================================================================================
and since it bringing down the number of concurrent operations barely made a dent,
I decided to try maxConcurrentOperationCount = 10
It's difficult to say what's going wrong without any details but here are a few ideas.
A. Use #autorelease. CALayers generate bitmaps in the backgound, which in aggregate can take-up lots of space if they are not freed in time. If you are creating and rendering many layers I suggest adding an autorelease block inside your rendering loop. This won't help if ALL your layers are nested and needed at the same time for rendering.
- (void) render
{
for (id thing in mylist) {
#autorelease {
CALayer *layer = createLayerFromThing(thing);
[self renderLayer:layer];
}
}
}
B. Also if you are using CGBitmapCreateContext for rendering are you calling the matching CGContextRelease? This goes also for CGColorRef.
C. If you are allocating memory with malloc or calloc are you freeing it when done? One way to ensure that this happens
Post the code for the rendering loop to provide more context.
I believe there are two possibilities here:
The items you create are not autoreleased.
The memory you are taking is what it is, due to the number of concurrent operations you are doing.
In the first case the solution is simple. Send an autorelease message to the layers and images upon their creation.
In the second case, you could limit the number of concurrent operations by using an NSOperationQueue. Operation queues have a property called maxConcurrentOperationCount. I would try with a value of 4 and see how the memory behavior changes from what you currently have. Of course, you might have to try different values to get the right memory vs performance balance.
Autorelease will wait till the end of the run loop to clean up. If you explicit release it can take it from the heap without filling the pool.
- (void) render {
for (id thing in mylist) {
CALayer *layer = createLayerFromThing(thing); // assuming this thing is retained
[self renderLayer:layer];
[layer release]; // this layer no longer needed
} }
Also run build with analyse and see if you have leaks and fix them too.
I have an older app that I converted to use ARC. The original version did not have memory leaks, and I fixed all of the errors after the conversion using the memory diagnostics in the debugger. Occasionally, the app will crash and the stack trace looks like ARC is stuck in either an infinite retain or release cycle, like this:
0 CoreFoundation 0x329046a6 CFRelease + 46
1 CoreFoundation 0x329046da CFRelease + 98
2 CoreFoundation 0x329046da CFRelease + 98
3 CoreFoundation 0x329046da CFRelease + 98
4 CoreFoundation 0x329046da CFRelease + 98
Unfortunately, the stack traces cut off at 512 entries, so I can't see where this chain started. Apple said this could happen if the alloc and init methods are separated and the init method returns a different object than alloc. I checked all of our allocations and they follow the recommended format:
Class *var = [[Class alloc] initMethod:arg];
Has anyone seen an infinite loop of this sort? I have not been able to catch it while using the Xcode debugger, and it is very inconsistent. Since ARC is injecting the retains and releases, I don't have any idea where the crash originates.