UIView with drawRect/update image performance issue on Retina displays - ios

The problem in short:
LooksLike drawRect itself (even empty) leads to a significant performance bottleneck depending on device's resolution - the bigger screen is the worse things are.
Is there a way to speed up redrawing of view's content?
In more details:
I'm creating a small drawing app on iOS - user moves his finger over a display to draw a line.
The idea behind this is quite simple - while user moves his finger touchesMoved accumulates the changes into offscreen buffer image and invalidates the view to merge the offscreen buffer with the view's content.
The simple code snippet may look like this:
#interface CanvasView : UIView
...
end;
#implementation CanvasView{
UIImage *canvasContentImage;
UIImage *bufferImage
CGContextRef drawingContext;
}
- (void)touchesBegan:(NSSet *)touches withEvent:(UIEvent *)event{
// prepare drawing to start
UIGraphicsBeginImageContext(canvasSize);
drawingContext = UIGraphicsGetCurrentContext();
...
}
-(void)touchesMoved:(NSSet *)touches withEvent:(UIEvent *)event{
// draw to bufferImage
CGContextMoveToPoint(drawingContext, prevPoint.x, prevPoint.y);
CGContextAddLineToPoint(drawingContext, point.x, point.y);
CGContextStrokePath(drawingContext);
...
[self setNeedDisplay];
}
-(void) touchesEnded:(NSSet *)touches withEvent:(UIEvent *)event{
//finish drawing
UIGraphicsEndImageContext();
//merge canvasContentImage with bufferImage
...
}
-(void)drawRect:(CGRect)rect{
// draw bufferImage - merge it with current view's content
CGContextRef context = UIGraphicsGetCurrentContext();
CGContextDrawImage(context, self.bounds, canvasContentImage.CGImage);
CGContextDrawImage(context, imageRect, bufferImage.CGImage);
...
}
I've also implemented a small helper class to calculate fps rate.
So the approach above works rather good on non-retina screens producing nearly 60fps. However fps rate dramatically drops on retina-screens. For example on iPad Retina it is about 15-20 fps which is too slow.
The first obvious reason I thought is that setNeedsDisplay causes to redraw a full screen which is a big wast of resources. So I moved to setNeedsDisplayInRect to update only a dirty region. Surprisingly it didn't change anything regarding to performance (at lest nothing noticeable according to measurements and visually).
So I've started to try different approaches to figure out the bottleneck. When I've commented out all the drawing logic the fps rate still stayed at 15-20 - looks like the problem lies outside of drawing logic.
Finally when I've fully commented out the drawRect method the fps rises to 60. Not that I removed only the implementation but even a declaration. Not sure of my terminology so here is the results:
// -(void)drawRect:(CGRect)rect{
// // draw bufferImage - merge it with current view's content
// ...
// }
What is more interesting when I moved all the drawing code from drawRect method to touchMoved method it doesn't impact the performance, however the same amount of drawing/processing logic still remains comparing to the version with drawRect method - even updating the entire view every time still gives me 60fps.
One problem is that without drawRect I'm not able to visualize that changes.
So I've came to what pregenerated drawRect method warns about:
"Only override drawRect: if you perform custom drawing. An empty
implementation adversely affects performance during animation."
My guess is that the system creates and destroys graphics context every time custom drawRect is triggered leading to "dversely affects performance"
So the questions are:
Is there any way to speed up drawRect calls, like make the system reuse resources from call to call of drawRect or something?
If this is a dead end, what other approaches available to update view's content? Moving to OpenGL is not an option at the moment as there are a lot of code/logic already implemented and it will take a lot of effort to port it.
I'll be glad to provide any additional information needed.
And Thanks in advance for any suggestions!
EDIT:
After some more investigation and experiments I've came to using UIImageView and it's image property to update view's content dynamically. It gave a small performance improvements (the drawing is more stable at 19-22 fps). However it is still far from target 50-60fps.
One think I've noticed is that updating only a dirty part of offscreen buffer image really makes sense - without forcing the view's content update, pure logic of redrawing offscreen buffer gives around 60 fps.
But once I'm trying to attach the updated image to UIImageView.image property to update it's content the fps drops to mentioned 19-22. This is reasonable as assigning the property forces whole image to be redrawn on view's side.
So the question still remains - is there any way to updated only a specified portion of view's (UIImageView's) displaying content?

After spending several days I've came to unexpected (at least for myself) results.
I was able to achieve 30fps on retina iPad which is acceptable result for now.
The trick that worked for me was:
Subclass UIImageView
Use UIImageView.image property to update content - it gives a better results comparing to ordinary UIView with setNeedsDisplay/setNeedsDisplayInRect methods.
Use [self performSelector:#selector(setImage:) withObject:img afterDelay:0]; instead of just UIImageView.image = img to set the updated image to UIImageView.
The last point is a kind of spell for me now however it gives the minimal necessary frame rate (even if the original image is redrawn fully each frame not only dirty regions).
My guess if why performSelector helped to gain fps for updating view in my case is that scheduling setImage on view's queue optimizes possible internal idles which may occur during touch event processing.
This is only my guess and if anyone can provide relevant explanation I'll be glad to post it here or accept that answer.

Related

Handling multiple/frequent calls to UIView needsDisplayInRect efficiently

I am working on an iPad application which has a custom drawn UIView which basically covers the whole screen. It is an audio application and it has VU meters which get update messages over ethernet about 30 times a second which then tell the UI to redraw the VU meters on the screen.
The problem I have is that there are 16 VU meters at various positions in the view and I end up calling needsDisplayInRect for each one 30 times a second. If I do all of the drawing in the drawRect function then it works, but it's pretty sluggish as the CGRect passed in to drawRect is almost the entire screen.
In the OSX version there is the NSView getRectsBeingDrawn function which gave me a route to significantly improve the redraw performance. However, as far as I'm aware there is no equivalent function in iOS.
I've tried storing the CGRects that I've called then only redrawing them in drawRect, but it turns out that iOS often decides to redraw areas between the CGRects that have had needsDisplayInRect called on them, so I end up with big white rectangles on areas that I've not requested a redraw for (although I do get the performance improvement I'm expecting). The problem here is that I can't see a way to get the rects that are actually being redrawn, just the rect passed into drawRect which is essentially a union of all of the rects being redrawn.
So, is there any other way to speed up very frequent redraws of small areas of a UIView that I've missed?
The simplest solution is almost certainly to make a subview for each VU meter. Presumably you'd just have one VUMeterView class and make sixteen instances of it.

Best way to handle autoresizing of UIView with custom drawing

I'm working on a custom view, that has some specific Core Graphics drawings. I want to handle the view's autoresizing as efficiently as possible.
If I have a vertical line drawn in UIView, and the view's width stretches, the line's width will stretch with it. I want to keep the original width, therefore I redraw each time in -layoutSubviews:
- (void)drawRect:(CGRect)rect
{
[super drawRect:rect];
// ONLY drawing code ...
}
- (void)layoutSubviews
{
[super layoutSubviews];
[self setNeedsDisplay];
}
This works fine, however I don't think this is a efficient approach - unless CGContext drawing is blazing fast.
So is it really fast? Or is there better way to handle view's autoresizing? (CALayer does not support autoresizing on iOS).
UPDATE :
this is going to be a reusable view. And its task is to draw visual representation of data, supplied by the dataSource. So in practice there could really be a lot of drawing. If it is impossible to get this any more optimized, then there's nothing I can do... but I seriously doubt I'm taking the right approach.
It really depends on what you mean by "fast" but in your case the answer is probably "No, CoreGraphics drawing isn't going to give you fantastic performance."
Whenever you draw in drawRect (even if you use CoreGraphics to do it) you're essentially drawing into a bitmap, which backs your view. The bitmap is eventually sent over to the lower level graphics system, but it's a fundamentally slower process than (say) drawing into an OpenGL context.
When you have a view drawing with drawRect it's usually a good idea to imagine that every call to drawRect "creates" a bitmap, so you should minimize the number of times you need to call drawRect.
If all you want is a single vertical line, I would suggest making a simple view with a width of one point, configured to layout in the center of your root view and to stretch vertically. You can color that view by giving it a background color, and it does not need to implement drawRect.
Using views is usually not recommended, and drawing directly is actually preferred, especially when the scene is complex.
If you see your drawing code is taking a considerable toll, steps to optimize drawing further is to minimize drawing, by either only invalidating portions of the view rather than entirely (setNeedsDisplayInRect:) or using tiling to only draw portions.
For instance, when a view is resized, if you only need to draw in the areas where the view has changed, you can monitor and calculate the difference in size between current and previous layout, and only invalidate regions which have changed. Edit: It seems iOS does not allow partial view drawing, so you may need to move your drawing to a CALayer, and use that as the view's layer.
CATiledLayer can also give a possible solution, where you can cache and preload tiles and draw required tiles asynchronously and concurrently.
But before you take drastic measures, test your code in difficult conditions and see if your code is performant enough. Invalidating only updated regions can assist, but it is not always straightforward to limit drawing to a provided rectangle. Tiling adds even more difficulty, as the tiling mechanism requires learning, and elements are drawn on background threads, so concurrency issues also come in play.
Here is an interesting video on the subject of optimizing 2D drawing from Apple WWDC 2012:
https://developer.apple.com/videos/wwdc/2012/?include=506#506

alternative to UIView setCenter

My app, which is a game, includes a CADisplayLink timer which calls a function that instructs about 20 calls to UIView setCenter: for the various objects on screen every frame.
Time profiling it, this accounts for about 30% of all activity in the game and drastically reduces performance on older devices (anything lower than 5th generation ipod touch or iphone).
Are there any lightweight, low-overhead alternatives I can use to move objects (specifically UIViews) around the screen every frame?
EDIT:
Just to clarify, the center property of these UIViews must be set EVERY FRAME. I have a number of tiles that represent the ground in my game. They zip across the screen, only to be replaced by new tiles. After fiddling with the code for a couple hours to change the UIViews to CAlayers, I have it working at absolutely no performance gain. There surely is a better way to do this.
Some code to give a general idea of what is going on:
for(Object* o in gameController.entities){
[o step:curTimeMS];
}
gameController is, as one would think, a class that takes care of the main game functions. It includes its list of entities, which are all the objects on-screen. The step method on each of these entities is a virtual function, so it is specific to each entity - the curTimeMS variable is simply a timestamp so the object can calculate its delta position. In essence, each entity updates its layer.position property every frame, moving it at an appropriate speed across the screen.
I would recommend SpriteKit. It is a very powerful game / 2d animation framework created by apple.. Cocos2D is also a very powerful framework of similar type. You can create a new SpriteKit game straight from XC
If you want to stay in house with just UIKit stuff, check out UIView block based animations. Here is the jist of it.
[UIView animateWithDuration:numberOfSecondsTakenToAnimate animations: ^{
// do you animation here. i.e.: move view frame, adjust color.
} completions: ^(BOOL complete) {
// when the animation is complete, code in this block is executed.
}];
I just remembered Core Graphics. It is used in tandem with UIViews to create simple 2d graphics and is very powerful and very fast. Here is the jist of that.
CGContextRef cntxt = UIGraphicsGetCurrentContext();
CGContextBeginPath(cntxt);
CGContextMoveToPoint(cntxt, <x>, <y>);
CGContextAddLineToPoint(cntxt, <x>, <y>);
CGContextClosePath(cntxt);
[[UIColor <color>] setFill];
[[UIColor <color>] setStroke];
CGContextDrawPath(cntxt, kCGPathFillStroke);
Note: things in < > are variables / values specified by you.
If you want to go all out, take the time to learn Open GL. Beware, I have heard that this is extremely hard to learn.
If you need performance, do not use UIView. It is not designed to be fast.
Instead, have a single UIView that takes up the whole screen, with a bunch of CALayer objects inside the one view. Move the layers around.
CALayer works by talking direct to the GPU, so it's very fast. Perhaps even faster than OpenGL. UIView is using CALayer internally so they both behave approximately the same. The only real difference is any change to the position of a CALayer will be animated by default. You can easily turn the animation off, although in a game you probably want animation.
Your other option, of course, is to use OpenGL. But that's a lot more work.
EDIT: here is some sample code for changing the position of a layer properly: https://developer.apple.com/library/ios/documentation/Cocoa/Conceptual/CoreAnimation_guide/CreatingBasicAnimations/CreatingBasicAnimations.html#//apple_ref/doc/uid/TP40004514-CH3-SW8

How to add, rotate and drag line in IOS?

What would be the best way in accomplishing this? Let's say I have a method (not defined) that would allow a line with a fixed size and color to be drawn on the screen. This line would need to then accept rotate gestures and panning gestures in order to move around the screen. It won't resize, it only needs to rotate and translate.
What is the best way of going about this? Should lines be subviews or sublayers to parent view? What is the method for drawing a line in ios? How to handle multiple lines on screen? I just want someone to lead me down the right path in the ios graphics jungle.
Firstly, you need to consider how complex the whole drawing is. From your description it sounds like the task is relatively simple. If that is the case, then Core Graphics would be the way to go. If the drawing is significantly more complex, you should look at OpenGL ES and GLKit, though using OGL involves a fair bit more work
Assuming Core Graphics, I'd store the centre point, angle and length of the line, and change the angle and size using the gesture recognizers, and calculate the points to draw using basic trig. Loop over the points to draw in the view -drawRect method and draw each one with the appropriate CG functions - call [view setNeedsDisplay] or [view setNeedsDisplayInRect:areaToRedraw]to trigger the redraws. (The second method only redraws the part of the view you specify, and can be used to improved performance).
The first of a series of tutorials on Core Graphics is here.- the part on 'drawing lines' will be most relevant. I haven't done this one (I used the old edition of this book), but I've followed a lot of others from this site and found them very helpful
As a side note you'll probably need a way to focus on a particular line if you have more than one on the screen- an easy way would be to find the line centre point closest to the point the user touched.
It seems that the best API for drawing lines like you want is with Core Graphics. Put this code within your UIView's drawRect method:
/* Set the color that we want to use to draw the line */
[[UIColor redColor] set];
/* Get the current graphics context */
CGContextRef currentContext =UIGraphicsGetCurrentContext();
/* Set the width for the line */
CGContextSetLineWidth(currentContext,5.0f);
/* Start the line at this point */
CGContextMoveToPoint(currentContext,50.0f, 10.0f);
/* And end it at this point */
CGContextAddLineToPoint(currentContext,100.0f, 200.0f);
/* Use the context's current color to draw the line */
CGContextStrokePath(currentContext);
For the gesture recognition, use UIGestureRecognizers. Use the following methods
- (IBAction)handleRotate:(UIRotationGestureRecognizer *)recognizer
- (IBAction)handlePinch:(UIPinchGestureRecognizer *)recognizer
- (IBAction)handlePan:(UIPanGestureRecognizer *)recognizer

Resizing UIView from drawRect()

I have a UIView and drawing its content from drawRect(), but I need to resize the view sometimes depending on some calculation that are done during the drawing:
When I change the size, the drawing seams to scale and not to use the additional size gained from the resize.
This is how I'm resizing form drawRect()
CGRect limits = self.frame ;
limits.size.width = 400;
self.frame = limits;
It seams like the context is not aware of the resize and is drawing using the old sizes. I already try [self setNeedsDisplay] after resizing but the draw method is not called again.
Calculations should be done ahead of time and not during drawing, for performance reasons alone (though early calculations would also avoid this problem).
For example, if your calculations depend on factors X, Y and Z, arrange to redo those calculations whenever there is a change to X, Y or Z. Cache the results of the calculation if necessary, and resize your view frame at that time. The drawRect: implementation should only draw into the area it is given.
For anybody struggling with that.
Apparently [self setNeedsDisplay] won't have any effect if called from drawRect: (or any method called from drawRect:). Somewhy it just skips the call.
To make things happen use following instead
[self performSelector:#selector(setNeedsDisplay) withObject:nil afterDelay:0];
This will be called right when main queue is free again; just make sure to use some condition for calling needsDisplay inside of drawRect - or else you're going to get looped.

Resources