I'm trying to use multiple GLSL fragment shaders with OpenGL ES on iOS 7 and upwards. The shaders itself are running fine after the first call to glDrawArrays. Nevertheless, the very first call to glDrawArrays after the shaders and their program have been compiled and linked takes ages to complete. Afterwards some pipeline or whatever seems to have been loaded and everything goes smooth. Any ideas what the cause of this issue are and how to prevent it?
The most likely cause is that your shaders may not be fully compiled until you use them the first time. They might have been translated to some kind of intermediate form when you call glCompileShader(), which would be enough for the driver to provide a compile status and to act as if the shaders had been compiled. But the full compilation and optimization could well be deferred until the first draw call that uses the shader program.
A commonly used technique for games is to render a few frames without actually displaying them while some kind of intro screen is still shown to the user. This prevents the user from seeing stuttering that could otherwise result from all kinds of possible deferred initialization or data loading during the first few frames.
You could also look into using binary shaders to reduce slowdowns from shader compilation. See glShaderBinary() in the ES 2.0 documentation.
What actually helped speeding up the first draw call was the following (which is fine in my use case since I'm rendering a video so no depth testing is needed).
glDisable(GL_DEPTHTEST)
Related
When i run my app in ios 10 using xcode 8 i am getting following message in debug console, and by UI getting freezed can any one know why this is happening
ERROR
/BuildRoot/Library/Caches/com.apple.xbs/Sources/VectorKit/VectorKit-1228.30.7.17.9/GeoGL/GeoGL/GLCoreContext.cpp
1763: InfoLog SolidRibbonShader: ERROR
/BuildRoot/Library/Caches/com.apple.xbs/Sources/VectorKit/VectorKit-1228.30.7.17.9/GeoGL/GeoGL/GLCoreContext.cpp
1764: WARNING: Output of vertex shader 'v_gradient' not read by
fragment shader
Answer
One of the situations where you might get this warning in Xcode is when using an app that uses shaders such as the Maps app with an MKMapView. You'll find that the map view works as expected without that warning on a real device with real hardware/native OS.
In the sim the SolidRibbonShader fragment shader is not able to read the output of the v_gradient vertex shader probably because it's in beta or there might be an incompatibility between Xcode version and SIM version. However the shaders are recognized on a real device.
Explanation
Those shaders belong to the OpenGL Rendering Pipeline. The Rendering Pipeline is the sequence of steps that OpenGL takes when rendering objects.
The rendering pipeline is responsible for things like applying texture, converting the vertices to the right coordinate system and displaying the character on the screen etc.
There are six stages in this pipeline.
Per-Vertex Operation
Primitive Assembly
Primitive Processing
Rasterization
Fragment Processing
Per-Fragment Operation
Finally, an image appears on the screen of your device. These six stages are called the OpenGL Rendering Pipeline and all data used for rendering must go through it.
What is a shader?
A shader is a small program developed by you that lives in the GPU. A shader is written in a special graphics language called OpenGL Shading Language(GLSL).
A shader takes the place of two important stages in the OpenGL Rendering Pipeline: Per-Vertex Processing and Per-Fragment Processing stage. There is one shader for each of these two stages.
The ultimate goal of the Vertex Shader is to provide the final transformation of the mesh vertices to the rendering pipeline. The goal of the Fragment shader is to provide Coloring and Texture data to each pixel heading to the framebuffer.
Vertex shaders transform the vertices of a triangle from a local model coordinate system to the screen position. Fragment shaders compute the color of a pixel within a triangle rasterized on screen.
Separate Shader Objects Speed Compilation and Linking
Many OpenGL ES apps use several vertex and fragment shaders, and it is often useful to reuse the same fragment shader with different vertex shaders or vice versa. Because the core OpenGL ES specification requires a vertex and fragment shader to be linked together in a single shader program, mixing and matching shaders results in a large number of programs, increasing the total shader compile and link time when you initialize your app.
Update: the issue seems to be gone now on Xcode9/iOS11.
Firstly, the freezing problem happens only when run from Xcode 8 and only on iOS 10 (currently 10.0.2), whether in debug or release mode. MKMapView though seems fine when the app is distributed via App Store or 3rd party ad hoc distribution systems. The warnings you are seeing may or may not be related to the problem, I don't know.
What I've found is that the offending code is in MKMapView's destructor, and it doesn't matter what you do with the map view object or how you configure it, i.e. merely calling
[MKMapView new];
anywhere in your code will freeze the app. The main thread hangs on a semaphore and it's not clear why.
One of the things I've tried was to destroy the map view object in a separate thread but that didn't help. Eventually I decided to retain my map objects at least in DEBUG builds.
NOTE: this is a really sh*tty workaround but at least it will help you to debug your app without freezing. Retaining these objects means your memory usage will grow by about 45-50MB every time you create a view controller with a map.
So, let's say if you have a property mapView, then you can do this in your view controller's dealloc:
- (void)dealloc
{
#if DEBUG
// Xcode8/iOS10 MKMapView bug workaround
static NSMutableArray* unusedObjects;
if (!unusedObjects)
unusedObjects = [NSMutableArray new];
[unusedObjects addObject:_mapView];
#endif
}
To avoid writing to a constant buffer from both the gpu and cpu at the same time, Apple recommends using a triple-buffered system with the help of a semaphore to prevent the cpu getting too far ahead of the gpu (this is fine and covered in at least three Metal videos now at this stage).
However, when the constant resource is an MTLTexture and the AVCaptureVideoDataOutput delegate runs separately than the rendering loop (CADisplaylink), how can a similar triple-buffered system (as used in Apple’s sample code MetalVideoCapture) guarantee synchronization? Screen tearing (texture tearing) can be observed if you take the MetalVideoCapture code and simply render a full screen quad and change the preset to AVCaptureSessionPresetHigh (at the moment the tearing is obscured by the rotating quad and low quality preset).
I realize that the rendering loop and the captureOutput delegate method (in this case) are both on the main thread and that the semaphore (in the rendering loop) keeps the _constantDataBufferIndex integer in check (which indexes into the MTLTexture for creation and encoding), but screen tearing can still be observed, which is puzzling to me (it would make sense if the gpu writing of the texture is not the next frame after encoding but 2 or 3 frames after, but I don’t believe this to be the case). Also, just a minor point: shouldn’t the rendering loop and the captureOutput have the same frame rate for a buffered texture system so old frames aren’t rendered interleaved with recent ones.
Any thoughts or clarification on this matter would be greatly appreciated; there is another example from McZonk, which doesn’t use the triple-buffered system, but I also observed tearing with this approach (but less so). Obviously, no tearing is observed if I use waitUntilCompleted (equivalent to Open GL’s glfinish), but thats like playing an accordion with one arm tied behind your back!
I'm using OpenGL ES to write a custom UI framework on iOS. The use case is for an application, as in something that won't be updating on a per-frame basis (such as a game). From what I can see so far, it seems that the default behavior of the GLKViewController is to redraw the screen at a rate of about 30fps. It's typical for UI to only redraw itself when necessary to reduce resource usage, and I'd like to not drain extra battery power by utilizing the GPU while the user isn't doing anything.
I tried only clearing and drawing the screen once as a test, and got a warning from the profiler saying that an uninitialized color buffer was being displayed.
Looking into it, I found this documentation: http://developer.apple.com/library/ios/#DOCUMENTATION/iPhone/Reference/EAGLDrawable_Ref/EAGLDrawable/EAGLDrawable.html
The documentation states that there is a flag, kEAGLDrawablePropertyRetainedBacking, which when set to YES, will allow the backbuffer to retain things drawn to it in the previous frame. However, it also states that it isn't recommended and cause performance and memory issues, which is exactly what I'm trying to avoid in the first place.
I plan to try both ways, drawing every frame and not, but I'm curious if anyone has encountered this situation. What would you recommend? Is it not as big a deal as I assume it is to re-draw everything 30 times per frame?
In this case, you shouldn't use GLKViewController, as its very purpose is to provide a simple animation timer on the main loop. Instead, your view can be owned by any other subclass of UIViewController (including one of your own creation), and you can rely on the usual setNeedsDisplay/drawRect system used by all other UIKit views.
It's not the backbuffer that retains the image, but a separate buffer. Possibly a separate buffer created specifically for your view.
You can always set paused on the GLKViewController to pause the rendering loop.
I'm developing an iPad app that uses large textures in OpenGL ES. When the scene first loads I get a large black artifact on the ceiling for a few frames, as seen in the picture below. It's as if higher levels of the mipmap have not yet been filled in. On subsequent frames, the ceiling displays correctly.
This problem only began showing up when I started using mipmapping. One possible explanation is that the glGenerateMipmap() call does its work asynchronously, spawning some mipmap creation worker (in a separate process, or perhaps in the GPU) and returning.
Is this possible, or am I barking up the wrong tree?
Within a single context, all operations will appear to execute strictly in order. However, in your most recent reply, you mentioned using a second thread. To do that, you must have created a second shared context: it is always illegal to re-enter an OpenGL context. If already using a shared context, there are still some synchronization rules you must follow, documented at http://developer.apple.com/library/ios/ipad/#DOCUMENTATION/3DDrawing/Conceptual/OpenGLES_ProgrammingGuide/WorkingwithOpenGLESContexts/WorkingwithOpenGLESContexts.html
It should be synchronous; OpenGL does not in itself have any real concept of threading (excepting the implicit asynchronous dialogue between CPU and GPU).
A good way to diagnose would be to switch to GL_LINEAR_MIPMAP_LINEAR. If it's genuinely a problem with lower resolution mip maps not arriving until later then you'll see the troublesome areas on the ceiling blend into one another rather than the current black-or-correct effect.
A second guess, based on the output, would be some sort of depth buffer clearing issue.
I followed #Tommy's suggestion and switched to GL_LINEAR_MIPMAP_LINEAR. Now the black-or-correct effect changed to a fade between correct and black.
I guess that although we all know that OpenGL is a pipeline (and therefore asynchronous unless you are retrieving state or explicity synchronizing), we tend to forget it. I certainly did in this case, where I was not drawing, but loading and setting up textures.
Once I confirmed the nature of the problem, I added a glFinish() after loading all my textures, and the problem went away. (Btw, my draw loop is in the foreground and my texture loading loop - because it is so time consuming and would impair interactivity - is in the background. Also, since this may vary between platforms, I'm using iOS5 on an iPad 2)
Currently, I have a fixed timestep game loop running on a second thread in my game. The OpenGL context is on the same thread, and rendering is done once per frame after any updates. So the main "loop" has to wait for drawing each frame before it can proceed. This wasn't really a problem until I wrote my particle system. Upwards of 1500+ particles with a physics step of 16ms causes the framerate to drop just below 30, anymore and it's worse. The particle rendering can't be optimized anymore without losing capability, so I decided to try moving OpenGL to a 3rd thread. I know this is somewhat of an extreme case, but I feel it should be able to handle it.
I've thought of running 2 loops concurrently, one for the main stepping (fixed timestep) and one for drawing (however fast it can go). However the rendering calls pass in data that may be changed each update, so I was concerned that locking would slow it down and negate benefit. However, after implenting a test to do this, I'm just getting EXC_BAD_ACCESS after less than a second of runtime. I assume because they're trying to access the same data at the same time? I thought the system automatically handled this?
When I was first learning OpenGL on the iPhone, I had OpenGL setup on the main thread, and would call performSelectorOnMainThread:withObject:waitUntilDone: with the rendering selector, and these errors would happen any time waitUntilDone was false. If it was true, it would happen randomly sometimes, but sometimes I could let it run for 30 mins and it would be fine. Same concept as what's happening now I assume. I am getting the first frame drawn to the screen before it crashes though, so I know something is working.
How would this be properly handled, and if so would it even provide the speed up I'm looking for? Or would multiple access slow it down just as much?