I'm attempting to optimize my app. It's quite visually rich, so has quite a lot of layered UIViews with large images and blending etc.
I've been experimenting with the shouldRasterize property on CALayers. In one case in particular, I have a UIView that consists of lots of sub views including a table. As part of a transition where the entire screen scrolls, this UIView also scales and rotates (using transforms).
The content of the UIView remains static, so I thought it would make sense to set view.layer.shouldRasterize = YES. However, I didn't see an increase in performance. Could it be that it's re-rasterizing every frame at the new scale and rotation? I was hoping that it would rasterize at the beginning when it has an identity transform matrix, and then cache that as it scales and rotates during the transition?
If not, is there a way I could force it to happen? Short of adding a redundant extra super-view/layer that does nothing but scale and rotate its rasterized contents...
You can answer your own question by profiling your application using the CoreAnimation instrument. Note that this one is only available in a device.
You can enable "Color hits in Green and Misses Red". If your layer remains red then it means that it is indeed rasterizing it every frame.
Related
Here is the iPad Simulator with four nested UIViews, drawing a custom background, and an inner UILabel. I am rotating the top UIView's CALayer, by getting it's layer and setting transform with a rotateY CATransform3D, animated on a separate thread (but the changes to transform are being sent on the main thread, naturally):
Note- this animation does not loop correctly, hence it appears to bounce.
The layers do animate as a whole, but curiously, the first child and it's descendants appear to be floating above the UIView with the transform applied!
The UIViews themselves are children in another UIView, which has a red background. There are no other transformations being applied anywhere else.
The positions for each UIView were set using setFrame initially at the start.
What is causing this strange behaviour, and how can I ensure the child UIViews transform with their parent, giving a flat appearance to the surface as a whole?
Well. Perhaps unsurprisingly I was doing something silly, but since I'd not used CALayer transforms before, I didn't know if things were acting up. I had overridden layoutSubviews on the UIViews I was creating, and the act of rotating the CALayer was triggering this call and then pushing the child components frame around, due to a bug.
The problem is that CALayers don't actually do 3D perspective by default. In order to do that you need to make a minor change to the layer's transform (which is of type CATransform3D)
You want to change the .m34 field of the transform to a small negative value. Try -1/200 to -1/500 as a starting range. If I remember correctly it should be the negative of 1 over the image height/width.
Change the .m34 property of the layer that you want to appear to "come off the page" and rotate in 3D. When you do that the Z setting of the layer does matter, and will both make closer layers bigger and also make layers that are further away disappear behind other things.
I suggest you do a Google search on "CATransform3D m34" for more information. There is a fair amount of information on the net about it.
I'm working on an iPhone app which will be displaying large scrollable and zoomable surface containing a grid of pictures and text labels over them. I need to be able to change the position of pictures individually and control the opacity level of labels. I tried to accomplish these goals by using UIScrollView and Core Animation.
The subview of UIScrollView contains two main sublayers: one for displaying pictures and one for the labels. Pictures are CALayers with their contents property set to CGImage and they are added as sublayers to the pictures layer. Labels are CATextLayers and they are sublayers of the second layer. Transparency of the labels layer changes depending on the zoom scale of the scroll view.
And here is the problem: everything works fine when the labels are fully opaque or fully transparent, but when they are semitransparent scrolling starts to be jerky and FPS drops to about 35. Obviously blending of these layers slows everything down, but I couldn't find a way to fix it. I will appreciate any ideas on how to improve the performance in this situation. Maybe there is a better way to draw text labels than using CATextLayer?
Is it possible for you to merge the two "main layers" of your UIScrollView into one? Also, is it possible for you to add layers directly to UIScrollView's layer instead of adding additional ones?
I find that I get huge performance wins by reducing the number of layers that exist for the sole purpose of containing other layers.
One solution is to add a shadow to the most background layers of both image layer and text layer.
There are number of shadow properties that you can tweak i.e. shadowPath, shadowColor, shadowOffset and shadowRadius - set each of them, don't miss any. Also set yourlayer.masksToBounds = NO.
Do not forget to add yourlayer.shouldRasterize = YES because this will have better performance impact.
I'm subclassing CALayer to provide my own drawing in method. For optimization I call -[MyLayer setNeedsDisplayInRect:] instead of -[MyLayer setNeedsDisplay]. In the drawing method I get the rect which should be redrawn via CGContextGetClipBoundingBox().
If I use this layer as the base layer of an UIView every thing works as expected. The problem arises, as soon as I use my custom layer as a sublayer of an other CALayer. Than CGContextGetClipBoundingBox() always returns the rect of the bounds of that layer.
Any ideas?
[EDIT]
It seems, that there is no guaranty, that the content of the CALayer is cached and only the dirty part gets redrawn. I did a small test and stored the rect that needs display as a separate property. The result was, that only this part was visible on the screen.
I'll now render to an image context and keep that image as a cache. In the draw method I'll only display the cached image.
Apple's documentation is unfortunately conflicting as the docs on -setNeedsDisplayInRect do not indicate whether the method works in practice. Based on my own experience, this technote sets it straight:
Note that, because of the way that iPhone/iPod touch/iPad updates its screen, the entire view will be redrawn if you call -setNeedsDisplayInRect: or -setNeedsDisplay:.
That being said, there are a number of things you can look into if you think that you are hitting a wall due to redundant drawing.
If drawing images, the biggest performance improvement you can make is to use images of the same dimensions at which you draw. If they're not, try to cache your image by rendering it to some offscreen bitmap context and bring it back later on.
Check out the shouldRasterize property on CALayer. This can be a godsend if you are trying to manipulate a layer whose sublayers potentially constitute a complex layer hierarchy. Be sure to check out how you're doing in Instruments by ticking the Color Hits Green and Misses Red box in the Core Animation instrument. If you see a lot of red, chances are using shouldRasterize is hurting more than it's helping.
Even better than shouldRasterize is to flatten your layer hierarchy, as then you can avoid the extra overhead that shouldRasterize incurs when flattening your layer hierarchy real time. Of course this is not always possible, but don't be afraid to try :)
If you're drawing images, try experimenting with your blending mode. If you happen to be drawing opaque images, there's no need to be using normal source over methods (which use both read/write bandwidth). Try kCGBlendModeCopy, which allows you to eliminate read bandwidth overhead.
Check out CGLayerRef, which allows you to cache Core Graphics output across various calls to your drawing methods. My experience is that, unless you're doing some hardcore pixel pushing, that this ends up being more costly than just redrawing. See this for an interesting read.
Above all, Instruments is your friend. Check out a couple videos from past WWDCs (2012, 2011, and 2010); they all have great info about how to fine-tune performance.
Please feel free to ask any further questions if something I've said makes little sense.
Blending and offscreen-rendering are both expensive in Core Animation.
One can see them in Core Animation instrument in Instruments, with Debug Options:
Here is my case:
Display 50x50 PNG images on UIImageViews. I want to round the images with a 6-point corer radius. The first method is to set UIImageView.layer's cornerRadius and masksToBounds which causes offscreen-rendering. The second method is to make PNG image copies with transparent corners which causes blending(because of the alpha channel).
I've tried both, but I can't see significant performance difference. However, I still want to know which is worse in theory and best practices if any.
Thanks a lot!
Well, short answer, the blending has to occur either way to correctly display the transparent corner pixels. However, this should typically only be an issue if you want the resulting view to also animate in some way (and remember, scrolling is the most common type of animation). Also, I'm able to recreate situations where "cornerRadius" will cause rendering errors on older devices (iPhone 3G in my case) when my views become complex. For situations where you do need performant animations, here are the recommendations I follow.
First, if you only need the resources with a single curve for the rounded corners (different scales are fine, as long as the desired curvature is the same), save them that way to avoid the extra calculation of "cornerRadius" at runtime.
Second, don't use transparency anywhere you don't need it (e.g. when the background is actually a solid color), and always specify the correct value for the "opaque" property to help the system more efficiently calculate the drawing.
Third, find ways to minimize the size of transparent views. For example, for a large border view with transparent elements (e.g. rounded corners), consider splitting the view into 3 (top, middle, bottom) or 7 (4 corners, top middle, middle, bottom middle) parts, keeping the transparent portions as small as possible and marking the rectangular portions as opaque, with solid backgrounds.
Fourth, in situations where you're drawing lots of text in scrollViews (e.g. highly customized UITableViewCell), consider using the "drawRect:" method to render these portions more efficiently. Continue using subviews for image elements, in order to split the render time between the overall view between pre-drawing (subviews) and "just-in-time" drawing (drawRect:). Obviously, experimentation (frames per second while scrolling) could show that violating this "rule-of-thumb" may be optimal for your particular views.
Finally, making sure you have plenty of time to experiment using the profiling tools (especially CoreAnimation) is key. I find that it's easiest to see improvements using the slowest device you want to target, and the results look great on newer devices.
After watching WWDC videos and having some experiments with Xcode and Instruments I can say that blending is better then offscreen rendering. Blending means that system requires some additional time to calculate color of pixels on transparent layers. The more transparent layers you have (and bigger size of these layers) then blending takes more time.
Offscreen rendering means that system will make more then one rendering iteration. At first iteration system will make rendering without visualization just to calculate bounds and shape of area which should be rendered. In next iterations system does regular rendering (depends on calculated shape) including blending if required.
Also for offscreen rendering system creates a separate graphics context and destroys it after rendering.
So you should avoid offscreen rendering and it's better to replace it with blending.
Hellothis weekend I started to watch the 2011 WWDC videos. I've found really interesting topics about iOS. My favorites were about performance and graphics, but I've found two of them apparently in contradiction. Of course there is something that I didn't get.
The sessions that I'm talking about are Understanding UIKit Rendering -121 and Polishing your app -105.
Unfortunately sample code from 2011 is still not downloadable, so is pretty hard to have an overall view.
In one session they explain that most of times offscreen rendering should be avoided during visualization in scrollview etc. They fix the performance issues in the sample code almost drawing everything inside the -drawRect method.
In the other session the performance issue (on a table view) seems to be due to too much code in the -drawRect method of the table's cells.
First is not clear to me when an OffScreen rendering is required by the system, I've seen in the video that some quartz function such as: cornerRadious, shadowOffset, shadowColor requires it, but does exist a general rule?
Second I don't know if I understood well, but it seems that when there is no offscreen rendering adding layers or views is the way to go.
I hope someone could bring light about that..
Thanks,
Andrea
I don't think there is a rule written down anywhere, but hopefully this will help:
First, let's clear up some definitions. I think offscreen vs onscreen rendering is not the overriding concern most of the time, because offscreen rendering can be as fast as onscreen. The main issue is whether the rendering is done in hardware or software.
There is also very little practical difference between using layers and views. Views are just a thin wrapper around CALayer and they don't introduce a significant performance penalty most of the time. You can override the type of layer used by a view using the +layerClass method if you want to have a view backed by a CAShapeLayer or CATileLayer, etc.
Generally, on iOS, pixel effects and Quartz / Core Graphics drawing are not hardware accelerated, and most other things are.
The following things are not hardware accelerated, which means that they need to be done in software (offscreen):
Anything done in a drawRect. If your view has a drawRect, even an empty one, the drawing is not done in hardware, and there is a performance penalty.
Any layer with the shouldRasterize property set to YES.
Any layer with a mask or drop shadow.
Text (any kind, including UILabels, CATextLayers, Core Text, etc).
Any drawing you do yourself (either onscreen or offscreen) using a CGContext.
Most other things are hardware accelerated, so they are much faster. However, this may not mean what you think it does.
Any of the above types of drawing are slow compared to hardware accelerated drawing, however they don't necessarily slow down your app because they don't need to happen every frame. For example, drawing a drop shadow on a view is slow the first time, but after it is drawn it is cached, and is only redrawn if the view changes size or shape.
The same goes for rasterised views or views with a custom drawRect: the view typically isn't redrawn every frame, it is drawn once and then cached, so the performance after the view is first set up is no worse, unless the bounds change or you call setNeedsDisplay on it.
For good performance, the trick is to avoid using software drawing for views that change every frame. For example, if you need an animated vector shape you'll get better performance using CAShapeLayer or OpenGL than drawRect and Core Graphics. But if you draw a shape once and then don't need to change it, it won't make much difference.
Similarly, don't put a drop shadow on an animated view because it will slow down your frame rate. But a shadow on a view that doesn't change from frame to frame won't have much negative impact.
Another thing to watch out for is slowing down the view setup time. For example, suppose you have a page of text with drop shadows on all the text; this will take a very long time to draw initially since both the text and shadows all need to be rendered in software, but once drawn it will be fast. You will therefore want to set up this view in advance when your application loads, and keep a copy of it in memory so that the user doesn't have to wait ages for the view to display when it first appears on screen.
This is probably the reason for the apparent contradiction in the WWDC videos. For large, complex views that don't change every frame, drawing them once in software (after which they are cached and don't need to be redrawn) will yield better performance than having the hardware re-composite them every frame, even though it will be slower to draw the first time.
But for views that must be redrawn constantly, like table cells (the cells are recycled so they must be redrawn each time one cell scrolls offscreen and is re-used as it scrolls back onto the other side as a different row), software drawing may slow things down a lot.
Offscreen-rendering is one of the worst defined topics in iOS rendering, today. When Apple's UIKit engineers refer to offscreen-rendering, it has a very specific meaning, and a ton of third-party iOS dev blogs are getting it wrong.
When you override "drawRect:", you're drawing via the CPU, and spitting out a bitmap. The bitmap is packaged up and sent to separate process that lives in iOS, the render server. Ideally, the render server just displays the data on screen.
If you fiddle with properties on CALayer, like turning on drop shadows, the GPU will perform additional drawing. This additional work is what UIKit engineers mean when they say "off-screen rendering." This is always performed with hardware.
The issue with off-screen drawing isn't necessarily the drawing. The off-screen pass requires a context switch, as the GPU switches its drawing destination. During this switch, the GPU is idle.
While I don't know a full list of properties that trigger an off-screen pass, you can diagnose this with the Core Animation Instrument's "Color Offscreen-rendered layer" toggle. I assume any property other than alpha is performed via an offscreen pass.
With early iOS hardware, it was reasonable to say "do everything in drawRect." Nowadays GPUs are better, and UIKit has features like shouldRasterize. Today, it's a balancing act between the time spent in drawRect, the number of off-screen passes, and the amount of blending. For the full details, watch the 2014 WWDC session 419, "Advanced Graphics and Animation for iOS Apps."
That all said, it's good to understand what's going on behind-the-scenes, and keep it in the back of your head so you don't do anything insane, but you should start from the simplest solution. Then test it on the slowest hardware you support. If you aren't hitting 60FPS, use Instruments to measure things and figure it out. There are a few possible bottlenecks, and if you aren't using data to diagnose things, you're just guessing.
Offscreen rendering / Rendering on the CPU
The biggest bottlenecks to graphics performance are offscreen rendering and blending – they can happen for every frame of the animation and can cause choppy scrolling.
Offscreen rendering (software rendering) happens when it is necessary to do the drawing in software (offscreen) before it can be handed over to the GPU. Hardware does not handle text rendering and advanced compositions with masks and shadows.
The following will trigger offscreen rendering:
Any layer with a mask (layer.mask)
Any layer with layer.masksToBounds / view.clipsToBounds being true
Any layer with layer.allowsGroupOpacity set to YES and layer.opacity is less than 1.0
When does a view (or layer) require offscreen rendering?
Any layer with a drop shadow (layer.shadow*).
Tips on how to fix: https://markpospesel.wordpress.com/tag/performance/
Any layer with layer.shouldRasterize being true
Any layer with layer.cornerRadius, layer.edgeAntialiasingMask, layer.allowsEdgeAntialiasing
Any layer with layer.borderWith and layer.borderColor?
Missing reference / proof
Text (any kind, including UILabel, CATextLayer, Core Text, etc).
Most of the drawings you do with CGContext in drawRect:. Even an empty implementation will be rendered offscreen.
This post covers blending and other things affecting performance: What triggers offscreen rendering, blending and layoutSubviews in iOS?