I am making an iOS app which can render big number of textures which I stream from disk on the fly. I use a NSCache for LRU cache of textures. There is one screen with a 3D model and one screen with a full screen detail of a texture where this texture can be changed with swiping. Kind of a very simple carousel. The app never takes more then 250MiB of RAM on 1GiB devices, the textures' cache works good.
For the fullscreen view I have a cache of VBOs based on the screen resolution and texture resolution (different texture coordinates). I never delete these VBOs and always check if the VBO is OK (glIsBuffer()). The screens are separate UIViewControllers and I use the same EAGLContext in both of them, no context sharing. This is OK as it is on the same thread.
All this is Open GL ES 2.0 and everything works good. I can switch between the 3D/2D screens, change the textures. The textures are created/deleted on the fly as needed based on the available memory.
BUT sometimes I get a random crash when rendering a full screen quad when calling:
glDrawArrays(GL_TRIANGLE_STRIP, 0, 4);
This can happen when I receive a lot of memory warnings in a row. Sometimes I can get hundreds of memory warnings in few seconds and the app works OK but sometimes it will crash while swiping to new full screen texture quad. This happens even for textures that were already rendered in full screen. It never crashes on the 3D model which uses the same textures.
The crash report is always on the glDrawArrays call (in my code) with a EXC_BAD_ACCESS KERN_INVALID_ADDRESS at 0x00000018 exception. The last call in a stack trace is always gleRunVertexSubmitARM. This happens on various iPads and iPhones.
It looks like a system memory pressure corrupts some GL memory but I do not know when, where and why.
I have also tried switching from VBO to the old way of having vertex data on the heap, where I first check if the vertex data is not NULL before calling glDrawArrays. The result is the same, random crashes in low memory situations.
Any ideas what could be wrong? The address 0x00000018 in the EXC_BAD_ACCESS is really bad but I do not know whose address it should be. Could a de-allocated texture or shader cause a EXC_BAD_ACCESS in glDrawArrays?
After several days of intensive debugging I finally figured it out. The problem was the NSCache storing OpenGL textures. Under memory pressure the NSCache starts removing items from it to free memory. The problem is that in that situation it does it on its own background thread (com.apple.root.utility-qos), thus there is no GL context for that thread (and no sharegroup with the main context) so the texture name is not a valid name (cannot be deleted) and will leak the GL memory. So after some memory warnings there was a lot of leaked GL textures, memory full and app finally crashed.
TL DR:
Do not use NSCache for caching OpenGL objects because it will leak them after a memory warning.
Related
I'm getting a lot of crashes in EAGLContext presentRenderbuffer on iOS 11, but only on iPhone 6/6+ and older.
As per this post, I think we've already ruled out VBO-related problems by rewriting everything to not use VBO/VAOs, but the crash wasn't fixed by that.
There are a few other questions on SO about this but no solution -- has anyone else been seeing the uptick in this crash and been able to resolve it?
TL;DR:
Here is what we know so far:
The crash is specific to iOS11, iPhone 5S/6/6+. It doesn’t occur on 6S and up.
The core of the OpenGL stack returns gpus_ReturnGuiltyForHardwareRestart
It occurs when we attempt to invoke [EAGLContext presentRenderbuffer] from a CAEAGLLayer
We don’t have a repro.
What we have tried so far:
Remove any reference to VBO/VAO in our rendering stack. Didn’t help.
We have tried reproing with a large range of drawing scenarios (rotation, resize, background/foreground). No luck.
As far as we can tell, there is nothing specific in our application logic between the iPhone 6 family and the iPhone 6S family.
Some clues (that could be relevant but not necessarily):
We know that when the presentRenderBuffer is invoked off main thread, and some CATransaction are occurring at the same time on the main thread, the crash rate goes up.
When presentRenderBuffer is invoked on main thread (along with the whole drawing pipeline), the crash rate goes slightly down but not drastically.
A substantial chunk (~20%) of the crashes occurs when the layer goes off screen and/or gets out of the view hierarchy.
Here is the stack trace:
libGPUSupportMercury.dylib gpus_ReturnGuiltyForHardwareRestart
1 AGXGLDriver gldUpdateDispatch
2 libGPUSupportMercury.dylib gpusSubmitDataBuffers
3 AGXGLDriver gldUpdateDispatch
4 GLEngine gliPresentViewES_Exec
5 OpenGLES -[EAGLContext presentRenderbuffer:]
From my experience I get this sort of crashes in these cases:
Calling OpenGL API when application is in UIApplicationStateBackground state.
Using objects (textures, VBO, etc) that was created in OpenGL context that have different shareGroup. This can happened if you don't call [EAGLContext setCurrentContext:..] before rendering or other work with OpenGL object.
Invalid geometry. For example this can happened if you allocate index buffer for bigger size that you need. Fill it with some values and then try to render with size that was used at allocation. Sometimes this works (tail of buffer is filled with 0, and you don't see any visual glitches). Sometimes it will crash (when tail of buffer filled with junk, and reference to point that is out of bounds).
Hope this helps in some way.
P.S. Maybe you tell some more info about your application? I write application that render vector maps at iOS and don't face any troubles with iOS 11 at this moment. Rendering pipeline is pretty simple CADisplayLink call our callback on main thread when we can render next frame. Each view with OpenGL scene can have several background contexts to load resources in background (ofc it have same shareGroup with main context).
To avoid writing to a constant buffer from both the gpu and cpu at the same time, Apple recommends using a triple-buffered system with the help of a semaphore to prevent the cpu getting too far ahead of the gpu (this is fine and covered in at least three Metal videos now at this stage).
However, when the constant resource is an MTLTexture and the AVCaptureVideoDataOutput delegate runs separately than the rendering loop (CADisplaylink), how can a similar triple-buffered system (as used in Apple’s sample code MetalVideoCapture) guarantee synchronization? Screen tearing (texture tearing) can be observed if you take the MetalVideoCapture code and simply render a full screen quad and change the preset to AVCaptureSessionPresetHigh (at the moment the tearing is obscured by the rotating quad and low quality preset).
I realize that the rendering loop and the captureOutput delegate method (in this case) are both on the main thread and that the semaphore (in the rendering loop) keeps the _constantDataBufferIndex integer in check (which indexes into the MTLTexture for creation and encoding), but screen tearing can still be observed, which is puzzling to me (it would make sense if the gpu writing of the texture is not the next frame after encoding but 2 or 3 frames after, but I don’t believe this to be the case). Also, just a minor point: shouldn’t the rendering loop and the captureOutput have the same frame rate for a buffered texture system so old frames aren’t rendered interleaved with recent ones.
Any thoughts or clarification on this matter would be greatly appreciated; there is another example from McZonk, which doesn’t use the triple-buffered system, but I also observed tearing with this approach (but less so). Obviously, no tearing is observed if I use waitUntilCompleted (equivalent to Open GL’s glfinish), but thats like playing an accordion with one arm tied behind your back!
I was attempting to debug why I wasn't seeing a new object (quad) being rendered, so I used the "Capture GPU frame" feature of Xcode. It usually works fine, but now it's giving me EXC_BAD_ACCESS in another render call, during glDrawElements.
Note that it seems similar to bugs I've seen, related to a mixed usage of VBOs and not. However, I'm definitely unbinding the VBO after usage, and disabling vertex attribute arrays:
glBindBuffer(GL_ARRAY_BUFFER, 0);
glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, 0);
glDisableVertexAttribArray(posAttr);
glDisableVertexAttribArray(texCoordAttr);
(Also, bear in mind that I'm only getting the crash when using "Capture GPU frame", not all the time)
What might I be doing wrong? Or could this be a bug in Xcode...?
This was indeed due to some GL state being left over, specifically a glVertexAttribPointer. The reason I didn't catch it was because it was an order of operations problem directly within the 3d engine itself: child objects were being iterated (and rendered) before some state was cleared up.
(Apologies, this was a tediously project-specific issue)
I have a question regarding OpenGL ES context size.
I have two OpenGL contexts running on iPad retina using GLKView. The view
is configured to have no depth / stencil / multisampling but only a 32 bit framebuffer.
A single buffer takes 12MB (2048*1536*4 bytes). Profiling my application reveals I have 3 IOKit allocations of 12MB plus one allocation of 12MB from Core Animation. I suspect they are all related. My guess is that Core animation caches the resulting frame buffer which explains the one 12MB coming from it.Also, I'm calling deleteDrawable on the GLKView which is hidden, which means that I would have expected a single 12MB buffer from IOKit and maybe another one from Core Animation. Does anyone have any experience with OpenGL memory consumption, how to reduce it and why do I see three IOKit allocations although I have only a single GLView at any given time?
I believe that iOS devices use triple buffering internally, which would explain the extra allocations. This was mentioned by John Carmack in an email printed here.
I have an iOS opengl es 2.0 app that needs to use a TON of large textures. Ideally 4096x4096. I have a struct array that contains all the info about the texture, and as I need to use each one I glGenTextures a new texture id and load the image there, free up the uiimage, etc. That all works great.
My app uses a bunch of textures for UI, image processing, etc. About 4-5 of the 15 I'm using for all of that are 4k x 4k. Rest are smaller. And then these load-as-needed textures are also 4k.
On loading about the 4th-5th of those the app crashes HARD. No console or debug. Just quits to the springboard in the middle of trying to load the next texture.
I don't have a memory leak - I ran instruments. I'm using ARC. I can post the crash report from the Organizer but it doesn't have much info. Just that my app's rpages was 170504.
I could post the image load code but its the same code I've used on all my apps for years. The new thing is pushing the system that hard and trying to load that many large textures.
Q1: Anyone have experience with using a ton of large textures?
So I resolved to the fact that I'll have to do preview res stuff at 1024x1024 and then final res stuff at 4096. The 1k images are now loading as needed and staying loaded. The 4k images will all be loaded one at a time into the same texture to be used and then move on to the next.
I wrote into my image loader a preview parameter and when set it shrinks the image to fit in 1024 during the load. Now Instead of crashing on the 4th or 5th I can add textures 'all day'. My GUESS is that I could do 16x as many as before. But I only need like 20-30 at a time. (only!) So far I've tried 20 with no memory warnings or crashes.
However.. if the app keeps running, because my textures are loaded at unique texture ids, at some point I would hit that spot where I need to unload one that's no longer needed to load the next one. This is probably very simple, but....
Q2: How do I free up a texture that's at an texture id when I no longer need it?
Q3: Will a memory warning tell me that I need to free up an open gl texture?
Q4: Aren't textures loaded on the PVR chip? Are they or how are they even taking up the phone's memory?
Thanks!
Removing Texture:
You have to use this GL call from the main thread.
glDeleteTextures(1, &_texture);
Memory warning is a general call to the application. It will not give you specific information. It is always better to remove unwanted textures from the memory if they are not needed anymore. Eg: We usually remove textures used in menu when the user moves to the In-Game screens, they are reloaded again when the user navigates back. This is much easier to manage memory than waiting for the system to call memory warning.
When you load PNG image, the data is decompressed and stored raw as array of colors per pixel. A 1K texture will use 4 mb despite of content/colors in the image. PVR is a hardware decompression chip which will decompress realtime when the image is used by the GPU, and the image file size you see is what memory it uses.