When cropping a CGImage in Swift 3 (using the .cropping method), the original CGImage is referenced by the cropped version - both according to the documentation, and according to what the Allocations instruments shows me.
I am placing the cropped CGImage objects on an undo stack, so having the original versions retained 'costs' me about 21mb of memory per undo element.
Since there is no obvious way to 'compact' a cropped CGImage and have it made independent from the original, I have currently done something similar to the following (without all the force unwrapping):
let croppedImage = original.cropping(to: rect)!
let data = UIImagePNGRepresentation(UIImage(cgImage: croppedImage))!
let compactedCroppedImage = UIImage(data: data)!.cgImage!
This works perfectly, and now each undo snapshot takes up only the amount of memory that it is supposed to.
My question is: Is there a better / faster way to achieve this?
Your code involves a PNG compression and decompression. This can be avoided. Just create an offscreen bitmap of the target size, draw the original image into it and use it as an image:
UIGraphicsBeginImageContext(rect.size)
let targetRect = CGRect(x: -rect.origin.x, y: -rect.origin.y, width: original.size.width, height: original.size.height)
original.draw(in: targetRect)
let croppedImage = UIGraphicsGetImageFromCurrentImageContext()
UIGraphicsEndImageContext()
Note: The result is slightly different if you don't have integral coordinates.
Related
I'm currently implementing some sort of coloring book and I'm curious about the best way to change pixels in UIImage. Here is my code:
self.context = CGContext(data: nil, width: image.width, height: image.height, bitsPerComponent: 8, bytesPerRow: image.width * 4, space: colorSpace, bitmapInfo: CGBitmapInfo.byteOrder32Little.rawValue | CGImageAlphaInfo.premultipliedFirst.rawValue)!
self.context?.draw(image.cgImage, in: CGRect(x: 0, y: 0, width: CGFloat(image.width), height: CGFloat(image.height)))
let ptr = context.data
self.pixelBuffer = ptr!.bindMemory(to: UInt32.self, capacity: image.width * image.height)
And change pixels using this function:
#inline (__always) func fill(matrixPosition: MatrixPosition, color: UInt32) {
pixelsBuffer?[self.linearIndex(for: matrixPosition)] = color
}
The problem is that every time when I change pixels I have to invoke makeImage on context to generate new image and it takes a lot of time:
func generateImage() -> UIImage {
let cgImage = context.makeImage()!
let uiimage = UIImage(cgImage: cgImage)
return uiimage
}
Does my approach is correct? What are better and faster ways to implement it? Thanks.
Manipulating individual pixels and then copying the entire memory buffer to a CGContext and then creating a UIImage with that context is going to end up being inefficient, as you are discovering.
You can continue to improve and optimize a CoreGraphics canvas approach by being more efficient about what part of your offscreen is copied onto screen. You can detect the pixels that have changed and only copy the minimum bounding rectangle of those pixels onto screen. This approach may be good enough for your use case where you are only filling in areas with colors.
Instead of copying the entire offscreen, copy just the changed area:
self.context?.draw(image.cgImage, in: CGRect(x: diffX, y: diffY, width: diffWidth, height: diffHeight))
It is up to you to determine the changed rectangle and when to update the screen.
Here is an example of a painting app that uses CoreGraphics, CoreImage and CADisplayLink. The code is a bit old, but the concepts are still valid and will serve as a good starting point. You can see how the changes are accumulated and drawn to the screen using a CADisplayLink.
If you want to introduce various types of ink and paint effects, a CoreGraphics approach is going to be more challenging. You will want to look at Apple's Metal API. A good tutorial is here.
I have a UIImageView in a UIScrollView in which can be zoomed in and out. Now, after the user has selected the specific content to be zoomed in, I want to crop that part of image present on the scrollview and get it in the form on UIImage.
For that I am using
extension UIScrollView {
var snapshotVisibleArea: UIImage? {
UIGraphicsBeginImageContext(bounds.size)
UIGraphicsGetCurrentContext()?.translateBy(x: -contentOffset.x, y: -contentOffset.y)
layer.render(in: UIGraphicsGetCurrentContext()!)
let image = UIGraphicsGetImageFromCurrentImageContext()
UIGraphicsEndImageContext()
return image
}
}
But when I implement this, the quality of the image get extremely degraded. Even If I use a 4K image, the final product looks like a 360p resolution.
This logic is just basic capturing of the screen content.
I know there can be a better way but I am not able to find a solution.
Any help is highly appreciated.
You can try this:
let context:CGContext = UIGraphicsGetCurrentContext()!
context.interpolationQuality = .high
Also I'm not sure but image quality could be improve if you initialize image context with this code: UIGraphicsBeginImageContextWithOptions(rect.size, false, 0.0)
After applying a CIFilter to a photo captured with the camera the image taken shrinks and repositions itself.
I was thinking that if I was able to get the original images size and orientation that it would scale accordingly and pin the imageview to the corners of the screen. However nothing is changed with this approach and not aware of a way I can properly get the image to scale to the full size of the screen.
func applyBloom() -> UIImage {
let ciImage = CIImage(image: image) // image is from UIImageView
let filteredImage = ciImage?.applyingFilter("CIBloom",
withInputParameters: [ kCIInputRadiusKey: 8,
kCIInputIntensityKey: 1.00 ])
let originalScale = image.scale
let originalOrientation = image.imageOrientation
if let image = filteredImage {
let image = UIImage(ciImage: image, scale: originalScale, orientation: originalOrientation)
return image
}
return self.image
}
Picture Description:
Photo Captured and screenshot of the image with empty spacing being a result of an image shrink.
Try something like this. Replace:
func applyBloom() -> UIImage {
let ciInputImage = CIImage(image: image) // image is from UIImageView
let ciOutputImage = ciInputImage?.applyingFilter("CIBloom",
withInputParameters: [kCIInputRadiusKey: 8, kCIInputIntensityKey: 1.00 ])
let context = CIContext()
let cgOutputImage = context.createCGImage(ciOutputImage, from: ciInputImage.extent)
return UIImage(cgImage: cgOutputImage!)
}
I remained various variables to help explain what's happening.
Obviously, depending on your code, some tweaking to optionals and unwrapping may be needed.
What's happening is this - take the filtered/output CIImage, and using a CIContext, write a CGImage the size of the input CIImage.
Be aware that a CIContext is expensive. If you already have one created, you should probably use it.
Pretty much, a UIImage size is the same as a CIImage extent. (I say pretty much because some generated CIImages can have infinite extents.)
Depending on your specific needs (and your UIImageView), you may want to use the output CIImage extent instead. Usually though, they are the same.
Last, a suggestion. If you are trying to use a CIFilter to show "near real-time" changes to an image (like a photo editor), consider the major performance improvements you'll get using CIImages and a GLKView over UIImages and a UIImageView. The former uses a devices GPU instead of the CPU.
This could also happen if a CIFilter outputs an image with dimensions different than the input image (e.g. with CIPixellate)
In which case, simply tell the CIContext to render the image in a smaller rectangle:
let cgOutputImage = context.createCGImage(ciOutputImage, from: ciInputImage.extent.insetBy(dx: 20, dy: 20))
I'm working with a CIImage, and while I understand it's not a linear image, it does hold some data.
My question is whether or not a CIImage's extent property returns pixels or points? According to the documentation, which says very little, it's working space coordinates. Does this mean there's no way to get the pixels / points from a CIImage and I must convert to a UIImage to use the .size property to get the points?
I have a UIImage with a certain size, and when I create a CIImage using the UIImage, the extent is shown in points. But if I run a CIImage through a CIFilter that scales it, I sometimes get the extent returned in pixel values.
I'll answer the best I can.
If your source is a UIImage, its size will be the same as the extent. But please, this isn't a UIImageView (which the size is in points). And we're just talking about the source image.
Running something through a CIFilter means you are manipulating things. If all you are doing is manipulating color, its size/extent shouldn't change (the same as creating your own CIColorKernel - it works pixel-by-pixel).
But, depending on the CIFilter, you may well be changing the size/extent. Certain filters create a mask, or tile. These may actually have an extent that is infinite! Others (blurs are a great example) sample surrounding pixels so their extent actually increases because they sample "pixels" beyond the source image's size. (Custom-wise these are a CIWarpKernel.)
Yes, quite a bit. Taking this to a bottom line:
What is the filter doing? Does it need to simply check a pixel's RGB and do something? Then the UIImage size should be the output CIImage extent.
Does the filter produce something that depends on the pixel's surrounding pixels? Then the output CIImage extent is slightly larger. How much may depend on the filter.
There are filters that produce something with no regard to an input. Most of these may have no true extent, as they can be infinite.
Points are what UIKit and CoreGraphics always work with. Pixels? At some point CoreImage does, but it's low-level to a point (unless you want to write your own kernel) you shouldn't care. Extents can usually - but keep in mind the above - be equated to a UIImage size.
EDIT
Many images (particularly RAW ones) can have so large a size as to affect performance. I have an extension for UIImage that resizes an image to a specific rectangle to help maintain consistent CI performance.
extension UIImage {
public func resizeToBoundingSquare(_ boundingSquareSideLength : CGFloat) -> UIImage {
let imgScale = self.size.width > self.size.height ? boundingSquareSideLength / self.size.width : boundingSquareSideLength / self.size.height
let newWidth = self.size.width * imgScale
let newHeight = self.size.height * imgScale
let newSize = CGSize(width: newWidth, height: newHeight)
UIGraphicsBeginImageContext(newSize)
self.draw(in: CGRect(x: 0, y: 0, width: newWidth, height: newHeight))
let resizedImage = UIGraphicsGetImageFromCurrentImageContext()
UIGraphicsEndImageContext();
return resizedImage!
}
}
Usage:
image = image.resizeToBoundingSquare(640)
In this example, an image size of 3200x2000 would be reduced to 640x400. Or an image size or 320x200 would be enlarged to 640x400. I do this to an image before rendering it and before creating a CIImage to use in a CIFilter.
I suggest you think of them as points. There is no scale and no screen (a CIImage is not something that is drawn), so there are no pixels.
A UIImage backed by a CGImage is the basis for drawing, and in addition to the CGImage it has a scale; together with the screen resolution, that gives us our translation from points to pixels.
I created some code to create a UIBezierPath within a UIImage context. Then take the image and create a base64 string. I believe I am supposed to draw the path within the beginning and end of the UIImage. However after many hours, it is not working. I am copying the base64 string to a website to download the image to see if it works. I am writing this in playground:
import Foundation
import UIKit
UIGraphicsBeginImageContextWithOptions(CGSizeMake(200, 200), false, 0.0)
let image = UIGraphicsGetImageFromCurrentImageContext()
UIColor.blackColor().setStroke()
let path = UIBezierPath()
path.lineWidth = 2
path.moveToPoint(CGPointMake(100, 0))
path.addLineToPoint(CGPointMake(200, 40))
path.addLineToPoint(CGPointMake(160, 140))
path.addLineToPoint(CGPointMake(40, 140))
path.addLineToPoint(CGPointMake(0, 40))
path.closePath()
UIGraphicsEndImageContext();
let data = UIImagePNGRepresentation(image)
let b64 = data?.base64EncodedStringWithOptions(NSDataBase64EncodingOptions(rawValue: 0))
print(b64!)
The resulting PNG is 1000x1000 I am guessing because of my retina display. The image itself is completely transparent and nothing is visible. I am using this site to decode base64 and save a file, I've tried 2 others to see if it was the site. http://www.motobit.com/util/base64-decoder-encoder.asp
EDIT
I just tried the following code to see if the issue with with my image or bezier or saving to bas64. This code worked great. So I think it's a problem with my bezier.
import Foundation
import UIKit
let image = UIImage(data: NSData(contentsOfURL: NSURL(string: "http://i.imgur.com/crr4m48.jpg")!)!)!
let data = UIImagePNGRepresentation(image)
let b64 = data?.base64EncodedStringWithOptions(NSDataBase64EncodingOptions(rawValue: 0))
print(b64!)
It looks like you're doing things out of order. For example:
UIGraphicsBeginImageContextWithOptions(CGSizeMake(200, 200), false, 0.0)
let image = UIGraphicsGetImageFromCurrentImageContext()
Why are you getting image before you draw anything into the graphics context? I think you're going to get an empty image -- it's not like the image you get at the outset will change as you draw things in the context. Wait to get the image until after you finish all your drawing. Also, the docs for that function say:
You should call this function only when a bitmap-based graphics context is the current graphics context. If the current context is nil or was not created by a call to UIGraphicsBeginImageContext, this function returns nil.
So, make sure that you've satisfied that requirement and that the image you get back when you do call UIGraphicsBeginImageContext() is not nil.
Next, you're creating a bezier path:
let path = UIBezierPath()
path.lineWidth = 2
path.moveToPoint(CGPointMake(100, 0))
path.addLineToPoint(CGPointMake(200, 40))
//...
but in order for that path to actually be drawn in your context, you have to do something that draws, such as calling path.fill() or path.stroke(). I don't see that anywhere, so even if you fix the first problem above, you're still going to end up with an empty image until you do some drawing.
You need to work with the image context directly. You can try create CGContextRef with current image context like:
UIGraphicsBeginImageContext(CGSizeMake(200, 200))
var context: CGContextRef = UIGraphicsGetCurrentContext()
And then directly add your path to it:
CGContextAddPath(context, path)
Or you can create you curves with context methods like CGContextAddCurveToPoint and then stroke path with:
CGContextStrokePath(context)
And the last action to create image from it:
let image = UIGraphicsGetImageFromCurrentImageContext()
One more Point In addition to #Caleb :-
If you are drawing image.png which result of 1000x1000 pixels then UIGraphicsBeginImageContext parameter must be different as above one :-
//Swift-3 Syntax
UIGraphicsBeginImageContextWithOptions( CGSize(width:500, height:500 ), false , 2.0 )
Reference