How does drawRect and CGGraphicsContext work? - ios

I am working with some stuff in Core Graphic's and I am looking for some additional clarification regarding a couple of topics.
drawRect:
I have an understanding of this and know it is where all of the drawing aspect's of a UIView goes, but am just unclear as to what is happening behind the scene's. What happen's when I create a UIView and fill out drawRect then set another object's UIView to be that custom view? When is drawRect being called?
CGGraphicsContext:
I know what the purpose of this is and understand the concept, but I can't see exactly how it is working.
For example:
CGContextSaveGState(context);
CGContextAddRect(context, rect);
CGContextClip(context);
CGContextDrawLinearGradient(context, gradient, startPoint, endPoint, 0);
CGContextRestoreGState(context);
The code above is in my app and work's correctly. The thing that confuses me is how it is working. The idea of saving/restoring a context makes sense, but it appears like I am literally saving a context, using that exact same context to make change's, then restoring the same context once again. It just seem's like I am saving a context and then writing on top of that context, only to restore it. How is it getting saved to a point where when you restore it, it is a different instance of the context than what was just used to make changes? You use the same reference of the variable context in every situation.
Lastly I would appreciate any resource's for practice project's or example's on using Core Graphics. I am looking to improve my skill in the matter since I obviously don't have much at the current time.

What happen's when I create a UIView and fill out drawRect then set another object's UIView to be that custom view? When is drawRect being called?
Adding a view to a 'live' view graph marks the view's frame as in need of display. The main run loop then creates and coalesces invalid rects and soon returns to invoke drawing. It does not draw immediately upon invalidation. This is a good thing because resizing, for example, would result in significant overdrawing -- redundant work which would kill many apps' drawing performance. When drawing, a context is created to render to -- which ultimately outputs to its destination.
Graphics Contexts are abstractions which are free to work optimally for their destination -- a destination could be a device/screen, bitmap, PDF, etc.. However, a context handle (CGContextRef) itself refers to a destination and holds a set of parameters regarding its state (these parameters are all documented here). These parameter sets operate like stacks: Push = CGContextSaveGState, Pop = CGContextRestoreGState. Although the context pointer isn't changing, the stack of parameter sets is changing.
As far as resources, see Programming with Quartz. It's 8 years old now, and was originally written for OS X -- but that ultimately doesn't matter a whole lot because the fundamentals of the drawing system and APIs really haven't evolved significantly since then -- And that is what you intend to focus on. The APIs have been extended, so it would be good to review which APIs were introduced since 10.4 and see what problems they solve, but it's secretly a good thing for you because it helps maintain focus on the fundamental operation of the drawing system. Note that some functionalities were excluded from iOS (e.g. often due to floating point performance and memory constraints, I figure), so every example may not be usable on iOS, but I know of no better guide.
Tip: Your drawing code can be easily reused on OS X and iOS if you use Quartz rather than AppKit/UIKit. Plus, the Quartz APIs have a lower update frequency (i.e. the APIs tend to be longer lived).

-drawRect: gets called at some point after you (e.g. your view controller) have called the view's method -setNeedsDisplay or -setNeedsDisplayInRect:.
Saving the graphics state pushes the current graphics state onto a stack. The graphics state contains fill and stroke setting, the current transformation matrix etc. See Apple's documentation for details.
The Quartz 2D Programming Guide doesn't contain many examples but is otherwise quite thorough.

With quartz/ core graphics the context is literally a set of current parameters to use to draw the next drawing command on top of the previous drawing.
Saving the state let's you save all those parameters for later drawing commands that will reuse them.
Then you can set up a different set of parameters for some drawing commands.
Restoring the state gets you back to where you were.
I recommend the book
Programming with Quartz
2D and PDF Graphics in Mac OS X
Though a bit dated in some ways, it will really teach you how quartz / core graphics really flows.

Ok this is a very very deep topic to talk about. I'll explain a few things to my understanding & try to keep it simple. If I'm mistaken I hope someone can correct me out.
first of all there is concept of onscreen drawing and offscreen drawing. On screen drawing is taken place in GPU where offscreen drawing is taken place in CPU to draw things and then its given to GPU to display on the screen. Thats where drawRect() comes in to place (drawrect is only 1 way of doing the offscreen drawings btw). This is why in the drawRect template method (you will see when you make a subclass of UIView) there is a comment by Apple telling
"Only override drawRect: if you perform custom drawing. An empty implementation adversely affects performance during animation"
The reason is whenever there is drawRect method, the iOS would have to ask the CPU to takecare of the drawing which takes place in drawRect and hand it over to the GPU. (Dont get the idea that this is a bad thing :) ). So this is what happens in drawRect in an abstract level.
Now to the question of why save & restore same context over and over. Have you tried to read the description of the method in apple doc about save/restore context ? If you have, you'd notice that it shows all the graphical states which would be affected by this. Ok how does this help ?
Consider something like this. Lets say you're drawing on a rectangle where you have to limit this next part of the drawing on the right half of it and use shadows and antialiasing, etc. You can save your context before drawing the right side and set whatever properties you want and once you finish that, you can simply restore the context and you can continue with all the settings you had before without explicitly setting them again. It's a good practice as well when you do complex drawings as otherwise it would have weird outcomes you might not expect. something like this below
- drawRect()
{
CGContextSaveGState(context);
drawLeftPart(); // - 1
drawRightPart(); // - 2
someOtherDrawing(); // - 3
CGContextRestoreGState(context);
}
- drawLeftPart()
{
CGContextSaveGState(context);
// do your drawing
CGContextRestoreGState(context);
}
- drawRightPart()
{
CGContextSaveGState(context);
// do your drawing
CGContextRestoreGState(context);
}
- someOtherDrawing()
{
CGContextSaveGState(context);
// do your drawing
CGContextRestoreGState(context);
}
Now what ever properties you set in part 1 wont affect drawing of part 2 & 3 so forth.
Hope this helps,

Related

How to properly use setNeedsDisplayInRect for iOS apps?

I'm on Yosemite 10.10.5 and Xcode 7, using Swift to make a game targeting iOS 8 and above.
EDIT: More details that might be useful: This is a 2D puzzle/arcade game where the player moves stones around to match them up. There is no 3D rendering at all. Drawing is already too slow and I haven't even gotten to explosions with debris yet. There is also a level fade-in, very concerning. But this is all on the simulator so far. I don't yet have an actual iPhone to test with yet and I'm betting the actual device will be at least a little faster.
I have my own Draw2D class, which is a type of UIView, set up as in this tutorial. I have a single NSTimer which initiates the following chain of calls in Draw2D:
[setNeedsDisplay]; // which calls drawRect, which is the master draw function of Draw2D
drawRect(rect: CGRect)
{
scr_step(); // the master update function, which loops thru all objects and calls their individual update functions. I put it here so that updating and drawing are always in sync
CNT = UIGraphicsGetCurrentContext(); // get the curret drawing context
switch (Realm) // based on what realm im in, call the draw function for that realm
{
case rlm.intro: scr_draw_intro();
case rlm.mm: scr_draw_mm();
case rlm.level: scr_draw_level(); // this in particular loops thru all objects and calls their individual draw functions
default: return;
}
var i = AARR.count - 1; // loop thru my own animation objects and draw them too, note it's iterating backwards because sometimes they destroy themselves
while (i >= 0)
{
let A = AARR[i];
A.scr_draw();
i -= 1;
}
}
And all the drawing works fine, but slow.
The problem is now I want to optimize drawing. I want to draw only in the dirty rectangles that need drawing, not the whole screen, which is what setNeedsDisplay is doing.
I could not find any tutorials or good example code for this. The closest I found was apple's documentation here, but it does not explain, among other things, how to get a list of all dirty rectangles so far. It does not also explicitly state if the list of dirty rectangles is automatically cleared at the end of each call to drawRect?
It also does not explain if I have to manually clip all drawing based on the rectangles. I found conflicting info about that around the web, apparently different iOS versions do it differently. In particular, if I'm gonna hafta manually clip things then I don't see the point of apple's core function in the first place. I could just maintain my own list of rectangles and manually compare each drawing destination rectangle to the dirty rectangle to see if I should draw anything. That would be a huge pain, however, because I have a background picture in each level and I would hafta draw a piece of it behind every moving object. What I'm really hoping for is the proper way to use setNeedsDisplayInRect to let the core framework do automatic clipping for everything that gets drawn on the next draw cycle, so that it automatically draws only that piece of the background plus the moving object on top.
So I tried some experiments: First in my array of stones:
func scr_draw_stone()
{
// the following 3 lines are new, I added them to try to draw in only dirty rectangles
if (xvp != xv || yvp != yv) // if the stone's coordinates have changed from its previous coordinates
{
MyD.setNeedsDisplayInRect(CGRectMake(x, y, MyD.swc, MyD.shc)); // MyD.swc is Draw2D's current square width in points, maintained to softcode things for different screen sizes.
}
MyD.img_stone?.drawInRect(CGRectMake(x, y, MyD.swc, MyD.shc)); // draw the plain stone
img?.drawInRect(CGRectMake(x, y, MyD.swc, MyD.shc)); // draw the stone's icon
}
This did not seem to change anything. Things were drawing just as slow as before. So then I put it in brackets:
[MyD.setNeedsDisplayInRect(CGRectMake(x, y, MyD.swc, MyD.shc))];
I have no idea what the brackets do, but my original setNeedsDisplay was in brackets just like they said to do in the tutorial. So I tried it in my stone object, but it had no effect either.
So what do I need to do to make setNeedsDisplayInRect work properly?
Right now, I suspect there's some conditional check I need in my master draw function, something like:
if (ListOfDirtyRectangles.count == 0)
{
[setNeedsDisplay]; // just redraw the whole view
}
else
{
[setNeedsDisplayInRect(ListOfDirtyRecangles)];
}
However I don't know the name of the built-in list of dirty rectangles. I found this saying the method name is getRectsBeingDrawn, but that is for Mac OSX. It doesn't exist in iOS.
Can anyone help me out? Am I on the right track with this? I'm still fairly new to Macs and iOS.
You should really avoid overriding drawRect if at all possible. Existing view/technologies take advantage of any hardware capabilities to make things a lot faster than manually drawing in a graphics context could, including buffering the contents of views, using the GPU, etc. This is repeated many times in the "View Programming Guide for iOS".
If you have a background and other objects on top of that, you should probably use separate views or layers for those rather than redraw them.
You may also consider technologies such as SpriteKit, SceneKit, OpenGL ES, etc.
Beyond that, I'm not quite sure I understand your question. When you call setNeedsDisplayInRect, it will add that rect to those that need to be redrawn (possibly merging with rectangles that are already in the list). drawRect: will then be called a bit later to draw those rectangles one at a time.
The whole point of the setNeedsDisplayInRect / drawRect: separation is to make sure multiple requests to redraw a given part of the view are merged together, and drawing only happens once per redraw cycle.
You should not call your scr_step method in drawRect:, as it may be called multiple times in a cycle redraw cycle. This is clearly stated in the "View Programming Guide for iOS" (emphasis mine):
The implementation of your drawRect: method should do exactly one
thing: draw your content. This method is not the place to be updating
your application’s data structures or performing any tasks not related
to drawing. It should configure the drawing environment, draw your
content, and exit as quickly as possible. And if your drawRect: method
might be called frequently, you should do everything you can to
optimize your drawing code and draw as little as possible each time
the method is called.
Regarding clipping, the documentation of drawRect states that:
You should limit any drawing to the rectangle specified in the rect
parameter. In addition, if the opaque property of your view is set to
YES, your drawRect: method must totally fill the specified rectangle
with opaque content.
Not having any idea what your view shows, what the various method you call do, what actually takes time, it's difficult to provide much more insight into what you could do. Provide more details into your actual needs, and we may be able to help.

ios iPaint from WWDC 2012

In the Video Session 506 of Apples WWDC 2012 they showed a painting app which is made for high performance drawings (so the frame rate never gets below 30).
I tried to replicate the code but get stuck on multiple points.
What I am looking for is a basic drawing app (lines, Squares, Circles, bezier paths), which performs well even after hundreds of lines have been drawn.
The basic approach is to save the drawn lines (or circles bezierpaths etc) to an image after a certain numbers of them have been drawn, and then only refresh the new drawings, therefore not having to redraw all the already drawn lines.
But somehow I never get to a higher performance. How do I need to implement this? Do I need multiple layers? And how do I manage that not all layers in a view are redrawn, but only a certain sublayer?
If someone could provide me with a short example of a few lines drawn on an layer, then saving that layer to an image, an then drawing on top of that I would really appreciate it.
Thank you for any help to recreate the iPaint application, which is unfortunately not available for download from apple.
That is only half of the puzzle. The other half is to only refresh the minimum possible area of the view (via setNeedsDisplayInRect:). However, I have been through many different ways of drawing via Core Graphics. The caching is fine, but I don't use it anymore. I set the update rectangle as above, and then test each path before I stroke it (testing is fast, stroking is slow). If it is inside the update box, I stroke it, otherwise I ignore it.
I did not look at that session, but a traditional Quartz speedup has been to use CGLayers (not CALayers). You can think of a CGLayer as a cached drawing which may or may not be a bitmap (the system decides how best to cached it). If you have a backing bitmap context, you can use that as your "image" and draw the CGLayers into that (and then discard the layers) as you see fit. Read up on CGLayer (its in the Quartz documentation) and then see if this was what they talked about in that session.

Free hand painting and erasing using UIBezierPath and CoreGraphics

I have been trying so much but have no solution find out yet. I have to implement the painting and erasing on iOS so I successfully implemented the painting logic using UIBezierPath. The problem is that for erasing, I implemented the same logic as for painting by using kCGBlendModeClear but the problem is that I cant redraw on the erased area and this is because in each pass in drawRect i have to stroke both the painting and erasing paths. So is there anyway that we can subtract erasing path from drawing path to get the resultant path and then stroke it. I am very new to Core Graphics and looking forward for your reply and comments. Or any other logic to implement the same. I can't use eraser as background color because my background is textured.
You don't need to stroke the path every time, in fact doing so is a huge performance hit. I guarantee if you try it on an iPad 3 you will be met with a nearly unresponsive screen after a few strokes. You only need to add and stroke the path once. After that, it will be stored as pixel data. So don't keep track of your strokes, just add them, stroke them, and get rid of them. Also look into using a CGLayer (you can draw to that outside the main loop, and only render it to your rect in the main loop so it saves lots of time).
These are the steps that I use, and I am doing the exact same thing (I use a CGPath instead of UIBezierPath, but the idea is the same):
1) In touches began, store the touch point and set the context to either erase or draw, depending on what the user has selected.
2) In touches moved, if the point is a certain arbitrary distance away from the last point, then move to the last point (CGContextMoveToPoint) and draw a line to the new point (CGContextAddLineToPoint) in my CGLayer. Calculate the rectangle that was changed (i.e. contains the two points) and call setNeedsDisplayInRect: with that rectangle.
3) In drawRect render the CGLayer into the current window context ( UIGraphicsGetCurrentContext() ).
On an iPad 3 (the one that everyone has the most trouble with due to its enormous pixel count) this process takes between 0.05 ms and 0.15ms per render (depending on how fast you swipe). There is one caveat though, if you don't take the proper precautions, the entire frame rectangle will be redrawn even if you only use setNeedsDisplayInRect: My hacky way to combat this (thanks to the dev forums) is described in my self answered question here, Otherwise, if your view takes a long time to draw the entire frame (mine took an unacceptable 150 ms) you will get a short stutter under certain conditions while the view buffer gets recreated.
EDIT With the new info from your comments, it seems that the answer to this question will benefit you -> Use a CoreGraphic Stroke as Alpha Mask in iPhone App
Hai here is the code for making painting, erasing, undo, redo, saving as picture. you can check sample code and implement this on your project.
Here

-[CALayer setNeedsDisplayInRect:] causes the whole layer to be redrawn

I'm subclassing CALayer to provide my own drawing in method. For optimization I call -[MyLayer setNeedsDisplayInRect:] instead of -[MyLayer setNeedsDisplay]. In the drawing method I get the rect which should be redrawn via CGContextGetClipBoundingBox().
If I use this layer as the base layer of an UIView every thing works as expected. The problem arises, as soon as I use my custom layer as a sublayer of an other CALayer. Than CGContextGetClipBoundingBox() always returns the rect of the bounds of that layer.
Any ideas?
[EDIT]
It seems, that there is no guaranty, that the content of the CALayer is cached and only the dirty part gets redrawn. I did a small test and stored the rect that needs display as a separate property. The result was, that only this part was visible on the screen.
I'll now render to an image context and keep that image as a cache. In the draw method I'll only display the cached image.
Apple's documentation is unfortunately conflicting as the docs on -setNeedsDisplayInRect do not indicate whether the method works in practice. Based on my own experience, this technote sets it straight:
Note that, because of the way that iPhone/iPod touch/iPad updates its screen, the entire view will be redrawn if you call -setNeedsDisplayInRect: or -setNeedsDisplay:.
That being said, there are a number of things you can look into if you think that you are hitting a wall due to redundant drawing.
If drawing images, the biggest performance improvement you can make is to use images of the same dimensions at which you draw. If they're not, try to cache your image by rendering it to some offscreen bitmap context and bring it back later on.
Check out the shouldRasterize property on CALayer. This can be a godsend if you are trying to manipulate a layer whose sublayers potentially constitute a complex layer hierarchy. Be sure to check out how you're doing in Instruments by ticking the Color Hits Green and Misses Red box in the Core Animation instrument. If you see a lot of red, chances are using shouldRasterize is hurting more than it's helping.
Even better than shouldRasterize is to flatten your layer hierarchy, as then you can avoid the extra overhead that shouldRasterize incurs when flattening your layer hierarchy real time. Of course this is not always possible, but don't be afraid to try :)
If you're drawing images, try experimenting with your blending mode. If you happen to be drawing opaque images, there's no need to be using normal source over methods (which use both read/write bandwidth). Try kCGBlendModeCopy, which allows you to eliminate read bandwidth overhead.
Check out CGLayerRef, which allows you to cache Core Graphics output across various calls to your drawing methods. My experience is that, unless you're doing some hardcore pixel pushing, that this ends up being more costly than just redrawing. See this for an interesting read.
Above all, Instruments is your friend. Check out a couple videos from past WWDCs (2012, 2011, and 2010); they all have great info about how to fine-tune performance.
Please feel free to ask any further questions if something I've said makes little sense.

iOS " current graphics context" - What is that

When I draw lines and shapes etc I get the " current graphics context" in iOS.
What exactly though is " current graphics context" - I'm looking for the 30,000 foot description.
Right now I just copy and paste UI code, not exactly sure what it's doing.
A graphics context is the place where information about the drawing state is stored. This includes fill color, stroke color, line width, line pattern, winding rule, mask, current path, transparency layers, transform, text transform, etc. When using CoreGraphics calls, you specify the context to use to every single function. This means you can use multiple contexts at once, though typically you only use one. At the UIKit layer, there is the concept of a "current" graphics context, which is a graphics context that's used by all UIKit-level drawing calls (such as -[UIColor set] or UIBezierPath drawing). The current context is stored in a stack of contexts, so you can create a new context for some drawing, then when you finish with it the previous context is restored. Typically you get a context for free inside of -[UIView drawRect:] inside of CALayer display-related methods, but not otherwise.
It used to be that the "current" context was an application-wide global state, and therefore was not safe to touch outside of the main thread. As of iOS 4.0 (I believe), this became a thread-local state and UIKit-level drawing methods became safe to use on background threads.
The OS needs a place to save information, such as drawing state, which you don't want to specify in every single CG drawing command, such as in which bitmap or view to draw, the scale or other transform to use, the last color you specified, etc.
The context tells each CG call where to find all this "stuff" for your current drawing call. Give a different context to the exact same drawing call, and that call might draw to a different bitmap in a completely different view, with a different color, different scale, etc.
Basically it is a class in a platform (iOS, Android, JavaME and many others) that provides access to all the drawing/display capabilities provided for that platform. It varies a bit for different platforms of course, but this is the 30,000 foot description :)

Resources