I have a 1080x1920 CIImage and wish to apply CIVignette as if the image were square (to mimic a camera lens).
I'm new to CoreImage and am wondering how to temporarily change the extent of my CIImage to 1920x1920. (But I'm not sure if this is needed at all.)
I could copy-paste two narrow slivers of the original image left and right and CICrop afterwards, but this seems hacky.
Any ideas?
You can use a combination of clampedToExtent (which causes the image to repeat its border pixels infinitely) and cropped to make the image square. Then you can apply the vignette and crop the result back to the original extent:
// "crop" to square, placing the image in the middle
let longerSize = max(inputImage.extent.width, inputImage.extent.height)
let xOffset = (longerSize - inputImage.extent.width) / 2.0
let yOffset = (longerSize - inputImage.extent.height) / 2.0
let squared = inputImage.clampedToExtent().cropped(to: CGRect(x: -xOffset, y: -yOffset, width: longerSize, height: longerSize))
// apply vignette
let vignetteFilter = CIFilter(name: "CIVignette")!
vignetteFilter.setValue(squared, forKey: kCIInputImageKey)
vignetteFilter.setValue(1.0, forKey: kCIInputIntensityKey)
vignetteFilter.setValue(1.0, forKey: kCIInputRadiusKey)
let withVignette = vignetteFilter.outputImage!
// crop back to original extent
let output = withVignette.cropped(to: inputImage.extent)
Related
I am currently trying to write a function which takes an image and applies a 3x3 Matrix to filter the vertical edges. For that I am using CoreImage's CIConvolution3X3 and passing the matrix used to detect vertical edges in Sobels edge detection.
Here's the code:
func verticalEdgeFilter() -> UIImage {
let inputUIImage = UIImage(named: imageName)!
let inputCIImage = CIImage(image: inputUIImage)
let context = CIContext()
let weights: [CGFloat] = [1.0, 0.0, -1.0,
2.0, 0.0, -2.0,
1.0, 0.0, -1.0]
let verticalFilter = CIFilter.convolution3X3()
verticalFilter.inputImage = inputCIImage
verticalFilter.weights = CIVector(values: weights, count: 9)
if let output = verticalFilter.outputImage{
if let cgimg = context.createCGImage(output, from: output.extent) {
let processedImage = UIImage(cgImage: cgimg)
return processedImage
}
}
print("returning original")
return inputUIImage
}
Now as a result I always get an almost fully transparent image with a 2 Pixel border like this one:
Original
Screenshot of the result (border on the left side)
Am I missing something obvious because the images are only transparent if the center value of the matrix is 0. But if I try the same kernel on some webpage, it does at least lead to a usable result. Setting a bias also just crashes the whole thing which I don't understand.
I also checked Apples documentation on this, as well as the CIFilter web page but I'm not getting anywhere, so I would really appreciate it if someone could help me with this or tell me an alternative way of doing this in Swift :)
Applying this convolution matrix to a fully opaque image will inevitably produce a fully transparent output. This is because the total sum of kernel values is 0, so after multiplying the 9 neighboring pixels and summing them up you will get 0 in the alpha component of the result. There are two ways to deal with it:
Make output opaque by using settingAlphaOne(in:) CIImage helper method.
Use CIConvolutionRGB3X3 filter that leaves the alpha component alone and applies the kernel to RGB components only.
As far as the 2 pixels border, it's also expected because when the kernel is applied to the pixels at the border it still samples all 9 pixels, and some of them happen to fall outside the image boundary (exactly 2 pixels away from the border on each side). These non existent pixels contribute as transparent black pixels 0x000000.
To get rid of the border:
Clamp image to extent to produce infinite image where the border pixels are repeated to infinity away from the border. You can either use CIClamp filter or the CIImage helper function clampedToExtent()
Apply the convolution filter
Crop resulting image to the input image extent. You can use cropped(to:) CIImage helper function for it.
With these changes here is how your code could look like.
func verticalEdgeFilter() -> UIImage {
let inputUIImage = UIImage(named: imageName)!
let inputCIImage = CIImage(image: inputUIImage)!
let context = CIContext()
let weights: [CGFloat] = [1.0, 0.0, -1.0,
2.0, 0.0, -2.0,
1.0, 0.0, -1.0]
let verticalFilter = CIFilter.convolution3X3()
verticalFilter.inputImage = inputCIImage.clampedToExtent()
verticalFilter.weights = CIVector(values: weights, count: 9)
if var output = verticalFilter.outputImage{
output = output
.cropped(to: inputCIImage.extent)
.settingAlphaOne(in: inputCIImage.extent)
if let cgimg = context.createCGImage(output, from: output.extent) {
let processedImage = UIImage(cgImage: cgimg)
return processedImage
}
}
print("returning original")
return inputUIImage
}
If you use convolutionRGB3X3 instead of convolution3X3 you don't need to do settingAlphaOne.
BTW, if you want to play with convolution filters as well as any other CIFilter out of 250, check this app out that I just published: https://apps.apple.com/us/app/filter-magic/id1594986951
I have some doubts about Core Image coordinate system, way transforms are applied and extent is determined. I couldn't find much in documentation or on internet so I tried the following code to rotate CIImage and display it in UIImageView.
override func viewDidLoad() {
super.viewDidLoad()
// Do any additional setup after loading the view.
imageView.contentMode = .scaleAspectFit
let uiImage = UIImage(contentsOfFile: imagePath)
ciImage = CIImage(cgImage: (uiImage?.cgImage)!)
imageView.image = uiImage
}
private var currentAngle = CGFloat(0)
private var ciImage:CIImage!
private var ciContext = CIContext()
#IBAction func rotateImage() {
let extent = ciImage.extent
let translate = CGAffineTransform(translationX: extent.midX, y: extent.midY)
let uiImage = UIImage(contentsOfFile: imagePath)
currentAngle = currentAngle + CGFloat.pi/10
let rotate = CGAffineTransform(rotationAngle: currentAngle)
let translateBack = CGAffineTransform(translationX: -extent.midX, y: -extent.midY)
let transform = translateBack.concatenating(rotate.concatenating(translate))
ciImage = CIImage(cgImage: (uiImage?.cgImage)!)
ciImage = ciImage.transformed(by: transform)
NSLog("Extent \(ciImage.extent), Angle \(currentAngle)")
let cgImage = ciContext.createCGImage(ciImage, from: ciImage.extent)
imageView.image = UIImage(cgImage: cgImage!)
}
So as I rotate the image every time by the push of a button, the image is rotated by angle pi/10 each time. But I see the image shrinking in UIImageView. The NSLogs show the extent is growing with some rotations with the origin x and y becoming negative.
2021-09-24 14:43:29.280393+0400 CoreImagePrototypes[65817:5175194] Metal API Validation Enabled
2021-09-24 14:43:31.094877+0400 CoreImagePrototypes[65817:5175194] Extent (-105.0, -105.0, 1010.0, 1010.0), Angle 0.3141592653589793
2021-09-24 14:43:41.426371+0400 CoreImagePrototypes[65817:5175194] Extent (-159.0, -159.0, 1118.0, 1118.0), Angle 0.6283185307179586
2021-09-24 14:43:42.244703+0400 CoreImagePrototypes[65817:5175194] Extent (-159.0, -159.0, 1118.0, 1118.0), Angle 0.9424777960769379
2021-09-24 14:43:42.644446+0400 CoreImagePrototypes[65817:5175194] Extent (-105.0, -105.0, 1010.0, 1010.0), Angle 1.2566370614359172
2021-09-24 14:43:43.037312+0400 CoreImagePrototypes[65817:5175194] Extent (0.0, 0.0, 800.0, 800.0), Angle 1.5707963267948966
2021-09-24 14:43:43.478774+0400 CoreImagePrototypes[65817:5175194] Extent (-105.0, -105.0, 1010.0, 1010.0), Angle 1.8849555921538759
2021-09-24 14:43:44.045811+0400 CoreImagePrototypes[65817:5175194] Extent (-159.0, -159.0, 1118.0, 1118.0), Angle 2.199114857512855
My questions:
How exactly do I determine scale factor to rescale the image so that the extent does not cross the original image rectangle?
What exactly does negative extent origin means? Relative to what it is negative? I understand coordinate system in Core Image is relative assuming bottom left corner of image to be (0,0), not with respect to some superview like in UIKit.
It's unclear what the question is, but what you seem to be focussed on is the meaning of the extent. This is like the frame, and, just like the frame, it loses its meaning if you have applied a transform to the CIImage. After a rotation, the extent is now based on the bounding box of the transformed image. So if you have a horizontally wider image and you rotate it a little bit counterclockwise, the extent becomes taller and its top becomes negative.
I have a CIImage, and when I touch a point on the display (Shown in the red box (lets say x: 10, y :20)), I need to crop that part of the image.
Since, it is a CIImage, the coordinates starts from Bottom-Top.
So, my question is when I select a Point on the image (Lets say the Red box shown on the image), How can I crop the image with relation to the CIImage ?
Note: Since the CIImage is reversed I basically want to how to convert my touch point which was x:10, y:20 so it can select the correct point from CIImage. Hope I made the question clear.
Update
let myCropFilter = CIFilter(name: "CICrop")
myCropFilter!.setValue(myInputImage, forKey: kCIInputImageKey)
myCropFilter!.setValue(CIVector(x: 10, y: 20, z: 100, w: 300), forKey: "inputRectangle")
let myOutputImage : CIImage = myCropFilter!.outputImage!
I have a strange issue where after I cropped a photo from my photo library, it cannot be displayed from the App. It gives me this error after I run this code:
self.correctedImageView.image = UIImage(ciImage: correctedImage)
[api] -[CIContext(CIRenderDestination) _startTaskToRender:toDestination:forPrepareRender:error:] The image extent and destination extent do not intersect.
Here is the code I used to crop and display. (inputImage is CIImage)
let imageSize = inputImage.extent.size
let correctedImage = inputImage
.cropped(to: textObvBox.boundingBox.scaled(to: imageSize) )
DispatchQueue.main.async {
self.correctedImageView.image = UIImage(ciImage: correctedImage)
}
More Info: Debug print the extent of the inputImage and correctedImage
Printing description of self.inputImage: CIImage: 0x1c42047a0 extent [0 0 3024 4032]>
crop [430 3955 31 32] extent=[430 3955 31 32]
affine [0 -1 1 0 0 4032] extent=[0 0 3024 4032] opaque
affine [1 0 0 -1 0 3024] extent=[0 0 4032 3024] opaque
colormatch "sRGB IEC61966-2.1"_to_workingspace extent=[0 0 4032 3024] opaque
IOSurface 0x1c4204790(501) seed:1 YCC420f 601 alpha_one extent=[0 0 4032 3024] opaque
Funny thing is that when I put a breakpoint, using Xcode, i was able to preview the cropped image properly. I'm not sure what this extent thing is for CIImage, but UIMageView doesn't like it when I assign the cropped image to it. Any idea what this extent thing does?
I ran into the same problem you describe. Due to some weird behavior in UIKit / CoreImage, I needed to convert the CIImage to a CGImage first. I noticed it only happened when I applied some filters to the CIImage, described below.
let image = /* my CIImage */
let goodImage = UIImage(ciImage: image)
uiImageView.image = goodImage // works great!
let image = /* my CIImage after applying 5-10 filters */
let badImage = UIImage(ciImage: image)
uiImageView.image = badImage // empty view!
This is how I solved it.
let ciContext = CIContext()
let cgImage = self.ciContext.createCGImage(image, from: image.extent)
uiImageView.image = UIImage(cgImage: cgImage) // works great!!!
As other commenters have stated beware creating a CIContext too often; its an expensive operation.
Extent in CIImage can be a bit of a pain, especially when working with cropping and translation filters. Effectively it allows the actual position of the new image to relate directly to the part of the old image it was taken from. If you crop the center out of an image, you'll find the extent's origin is non-zero and you'll get an issue like this. I believe the feature is intended for things such as making an art application that supports layers.
As noted above, you CAN convert the CIImage into a CGImage and from there to a UIImage, but this is slow and wasteful. It works because CGImage doesn't preserve extent, while UIImage can. However, as noted, it requires creating a CIContext which has a lot of overhead.
There is, fortunately, a better way. Simply create a CIAffineTransformFilter and populate it with an affine transform equal to the negative of the extent's origin, thus:
let transformFilter = CIFilter(name: "CIAffineTransform")!
let translate = CGAffineTransform(translationX: -image.extent.minX, y: -image.extent.minY)
let value = NSValue(cgAffineTransform: translate)
transformFilter.setValue(value, forKey: kCIInputTransformKey)
transformFilter.setValue(image, forKey: kCIInputImageKey)
let newImage = transformFilter.outputImage
Now, newImage should be identical to image but with an extent origin of zero. You can then pass this directly to UIView(ciimage:). I timed this and found it to be immensely faster than creating a CIContext and making a CGImage. For the sake of interest:
CGImage Method: 0.135 seconds.
CIFilter Method: 0.00002 seconds.
(Running on iPhone XS Max)
You can use advanced UIGraphicsImageRenderer to draw your image in iOS 10.X or later.
let img:CIImage = /* my ciimage */
let renderer = UIGraphicsImageRenderer(size: img.extent.size)
uiImageView.image = renderer.image { (context) in
UIImage(ciImage: img).draw(in: .init(origin: .zero, size: img.extent.size))
}
After applying a CIFilter to a photo captured with the camera the image taken shrinks and repositions itself.
I was thinking that if I was able to get the original images size and orientation that it would scale accordingly and pin the imageview to the corners of the screen. However nothing is changed with this approach and not aware of a way I can properly get the image to scale to the full size of the screen.
func applyBloom() -> UIImage {
let ciImage = CIImage(image: image) // image is from UIImageView
let filteredImage = ciImage?.applyingFilter("CIBloom",
withInputParameters: [ kCIInputRadiusKey: 8,
kCIInputIntensityKey: 1.00 ])
let originalScale = image.scale
let originalOrientation = image.imageOrientation
if let image = filteredImage {
let image = UIImage(ciImage: image, scale: originalScale, orientation: originalOrientation)
return image
}
return self.image
}
Picture Description:
Photo Captured and screenshot of the image with empty spacing being a result of an image shrink.
Try something like this. Replace:
func applyBloom() -> UIImage {
let ciInputImage = CIImage(image: image) // image is from UIImageView
let ciOutputImage = ciInputImage?.applyingFilter("CIBloom",
withInputParameters: [kCIInputRadiusKey: 8, kCIInputIntensityKey: 1.00 ])
let context = CIContext()
let cgOutputImage = context.createCGImage(ciOutputImage, from: ciInputImage.extent)
return UIImage(cgImage: cgOutputImage!)
}
I remained various variables to help explain what's happening.
Obviously, depending on your code, some tweaking to optionals and unwrapping may be needed.
What's happening is this - take the filtered/output CIImage, and using a CIContext, write a CGImage the size of the input CIImage.
Be aware that a CIContext is expensive. If you already have one created, you should probably use it.
Pretty much, a UIImage size is the same as a CIImage extent. (I say pretty much because some generated CIImages can have infinite extents.)
Depending on your specific needs (and your UIImageView), you may want to use the output CIImage extent instead. Usually though, they are the same.
Last, a suggestion. If you are trying to use a CIFilter to show "near real-time" changes to an image (like a photo editor), consider the major performance improvements you'll get using CIImages and a GLKView over UIImages and a UIImageView. The former uses a devices GPU instead of the CPU.
This could also happen if a CIFilter outputs an image with dimensions different than the input image (e.g. with CIPixellate)
In which case, simply tell the CIContext to render the image in a smaller rectangle:
let cgOutputImage = context.createCGImage(ciOutputImage, from: ciInputImage.extent.insetBy(dx: 20, dy: 20))