captureStillImageAsynchronously Issue - ios

I'm currently having an issue with AVCaptureStillImageOutput where when I try to take a picture the image is currently nil. My current attempts at bug fixing have found that captureStillImageAsynchronously method isn't being called at all and I haven't been able to test whether the sample buffer is nil or not. I'm using this method to feed the camera image into another method that combines the camera image and another image into a single image. The thread fails during that last method. When I try to examine the image from the capture method it is unavailable. What do I need to do to get the camera capture working?
public func capturePhotoOutput()->UIImage
{
var image:UIImage = UIImage()
if let videoConnection = stillImageOutput!.connection(withMediaType: AVMediaTypeVideo)
{
print("Video Connection established ---------------------")
stillImageOutput?.captureStillImageAsynchronously(from: videoConnection, completionHandler: {(sampleBuffer, error) in
if (sampleBuffer != nil)
{
print("Sample Buffer not nil ---------------------")
let imageData = AVCaptureStillImageOutput.jpegStillImageNSDataRepresentation(sampleBuffer)
let dataProvider = CGDataProvider(data: imageData! as CFData)
let cgImageRef = CGImage(jpegDataProviderSource: dataProvider!, decode: nil, shouldInterpolate: true, intent: .defaultIntent)
let camImage = UIImage(cgImage: cgImageRef!, scale: CGFloat(1.0), orientation: UIImageOrientation.right)
image = camImage
}
else
{
print("nil sample buffer ---------------------")
}
})
}
if (stillImageOutput?.isCapturingStillImage)!
{
print("image capture in progress ---------------------")
}
else
{
print("capture not in progress -------------------")
}
return image
}
EDIT: Added below method where the camera image is being used.
func takePicture()-> UIImage
{
/*
videoComponent!.getVideoController().capturePhotoOutput
{ (image) in
//Your code
guard let topImage = image else
{
print("No image")
return
}
}
*/
let topImage = videoComponent!.getVideoController().capturePhotoOutput() //overlay + Camera
let bottomImage = captureTextView() //text
let size = CGSize(width:(topImage.size.width),height:(topImage.size.height)+(bottomImage.size.height))
UIGraphicsBeginImageContextWithOptions(size, false, 0.0)
topImage.draw(in: CGRect(x:0, y:0, width:size.width, height: (topImage.size.height)))
bottomImage.draw(in: CGRect(x:(size.width-bottomImage.size.width)/2, y:(topImage.size.height), width: bottomImage.size.width, height: (bottomImage.size.height)))
let newImage:UIImage = UIGraphicsGetImageFromCurrentImageContext()!
UIGraphicsEndImageContext()
return newImage
}

If you use async method the function will return a wrong value, because the async call is still in progress. You can use a completion block, like that:
public func capturePhotoOutput(completion: (UIImage?) -> ())
{
if let videoConnection = stillImageOutput!.connection(withMediaType: AVMediaTypeVideo)
{
print("Video Connection established ---------------------")
stillImageOutput?.captureStillImageAsynchronously(from: videoConnection, completionHandler: {(sampleBuffer, error) in
if (sampleBuffer != nil)
{
print("Sample Buffer not nil ---------------------")
let imageData = AVCaptureStillImageOutput.jpegStillImageNSDataRepresentation(sampleBuffer)
let dataProvider = CGDataProvider(data: imageData! as CFData)
let cgImageRef = CGImage(jpegDataProviderSource: dataProvider!, decode: nil, shouldInterpolate: true, intent: .defaultIntent)
let camImage = UIImage(cgImage: cgImageRef!, scale: CGFloat(1.0), orientation: UIImageOrientation.right)
completion(camImage)
}
else
{
completion(nil)
}
})
}
else
{
completion(nil)
}
}
How to use it:
capturePhotoOutput
{ (image) in
guard let topImage = image else{
print("No image")
return
}
//Your code
}
Edit:
func takePicture()
{
videoComponent!.getVideoController().capturePhotoOutput
{ (image) in
guard let topImage = image else
{
print("No image")
return
}
let bottomImage = self.captureTextView() //text
let size = CGSize(width:(topImage.size.width),height:(topImage.size.height)+(bottomImage.size.height))
UIGraphicsBeginImageContextWithOptions(size, false, 0.0)
topImage.draw(in: CGRect(x:0, y:0, width:size.width, height: (topImage.size.height)))
bottomImage.draw(in: CGRect(x:(size.width-bottomImage.size.width)/2, y:(topImage.size.height), width: bottomImage.size.width, height: (bottomImage.size.height)))
let newImage:UIImage = UIGraphicsGetImageFromCurrentImageContext()!
UIGraphicsEndImageContext()
self.setPicture(image: newImage)
}
}
func setPicture(image:UIImage)
{
//Your code after takePicture
}

Related

How to make .accurate request for CoreML Segmentation

Im working on a bakground removal app , where Im doing the task using this function , but this is not so accurate . I found that Im not requesting accurate output here . I need to implement two line in my function .
Two line :
var segmentationRequest = VNGeneratePersonSegmentationRequest()
segmentationRequest.qualityLevel = . accurate
My Function :
func removeBackground(image:UIImage) -> UIImage?{
let resizedImage = image.resized(to: CGSize(width: 513, height: 513))
if let pixelBuffer = resizedImage.pixelBuffer(width:
Int(resizedImage.size.width), height: Int(resizedImage.size.height)){
if let outputImage = (try? modelCore.prediction(image:
pixelBuffer))?.semanticPredictions.image(min: 0, max: 1, axes: (0,0,1)),
let outputCIImage = CIImage(image:outputImage){
if let maskImage = removeWhitePixels(image:outputCIImage),
let resizedCIImage = CIImage(image: resizedImage), let compositedImage =
composite(image: resizedCIImage, mask: maskImage){
return UIImage(ciImage: compositedImage).resized(to: CGSize(width:
image.size.width, height: image.size.height))
}
}
}
return nil
}
You Can use .accurate Quality level in your request like this
but if you will use .accurate it will take more time to give you proper output. and I geuss the quality can be used only in non-Custom Models .
func generatePhoto(Image: Image?) {
//generating instance of request
let request = VNGeneratePersonSegmentationRequest()
guard
//Convert Input image to CgImg
let originalImage = Image?.foregroundImage.cgImage else {
print("missing require image")
return
}
//Setting quality level
request.qualityLevel = .accurate
request.revision = VNGeneratePersonSegmentationRequestRevision1
request.outputPixelFormat = kCVPixelFormatType_OneComponent8
//Create reqhandler
let requestHandler = VNImageRequestHandler(cgImage: originalImage, options: [:])
do{
//Process request
try requestHandler.perform([request])
guard let mask = request.results?.first else {
print("error")
return
}
//Convert the pixelbuffer to maskImg to get maskImg
let maskImage = CIImage(cvPixelBuffer: mask.pixelBuffer)
print(maskImage)
self.mskImage = maskImage
let foreground = CIImage(cgImage: originalImage).oriented(.up)
guard let output = blendImages(foreground: foreground, mask: maskImage) else {
print("Error4")
return
}
//update photoOutput
print(output)
//Convert the CIImg to UIImg
if let photoResult = renderAsUIImage(output){
print("output",photoResult.size)
self.outputImage = photoResult
print(outputImage)
}
}
catch {
print("Error processing person segmentation request")
}
}

Converting PDF to Image - Swift iOS

I am trying to convert a PDF file and all its pages to png images.
I have put together the code below filling the example on this thread
How to convert PDF to PNG efficiently?
When I run the code, it crashes on the pdf file source (sourceURL) there is definitely a file there. and when I print sourceURl it prints the URL to the file.
The crash says it found nil - My understanding is that means it could not find the file? even though I can physically see and open the file and also print the URL to the file.
Can someone point out what I'm doing wrong?
Code:
func convertPDFtoPNG() {
let sourceURL = pptURL
print("pptURL:", pptURL!)
let destinationURL = pngURL
let urls = try? convertPDF(at: sourceURL!, to: destinationURL!, fileType: .png, dpi: 200)
}
func convertPDF(at sourceURL: URL, to destinationURL: URL, fileType: ImageFileType, dpi: CGFloat = 200) throws -> [URL] {
let pdfDocument: CGPDFDocument! = CGPDFDocument(sourceURL as CFURL)! //Thread 1: Fatal error: Unexpectedly found nil while unwrapping an Optional value
let colorSpace = CGColorSpaceCreateDeviceRGB()
let bitmapInfo = CGImageAlphaInfo.noneSkipLast.rawValue
var urls = [URL](repeating: URL(fileURLWithPath : "/"), count: pdfDocument.numberOfPages)
DispatchQueue.concurrentPerform(iterations: pdfDocument.numberOfPages) { i in
let pdfPage = pdfDocument.page(at: i + 1)!
let mediaBoxRect = pdfPage.getBoxRect(.mediaBox)
let scale = dpi / 72.0
let width = Int(mediaBoxRect.width * scale)
let height = Int(mediaBoxRect.height * scale)
let context = CGContext(data: nil, width: width, height: height, bitsPerComponent: 8, bytesPerRow: 0, space: colorSpace, bitmapInfo: bitmapInfo)!
context.interpolationQuality = .high
context.fill(CGRect(x: 0, y: 0, width: width, height: height))
context.scaleBy(x: scale, y: scale)
context.drawPDFPage(pdfPage)
let image = context.makeImage()!
let imageName = sourceURL.deletingPathExtension().lastPathComponent
let imageURL = destinationURL.appendingPathComponent("\(imageName)-Page\(i+1).\(fileType.fileExtention)")
let imageDestination = CGImageDestinationCreateWithURL(imageURL as CFURL, fileType.uti, 1, nil)!
CGImageDestinationAddImage(imageDestination, image, nil)
CGImageDestinationFinalize(imageDestination)
urls[i] = imageURL
}
return urls
}
import Foundation
import Photos
// 1: 目前主要用来操作pdf转为图片
// 2: 图片保存到自定义相册中
struct HBPhotosAlbumHelperUtil {
static let shared = HBPhotosAlbumHelperUtil()
// url链接的pdf转为image
// pageNumber :表示pdf的对应的页面,默认为第一页
func drawToImagePDFFromURL(pdfurl url: String?, pageNumber index: Int = 1, scaleX scalex: CGFloat = 1.0, scaleY scaley: CGFloat = -1.0) -> UIImage? {
guard let pdfUrl = url, pdfUrl.count > 0, let formatterUrl = pdfUrl.urlValue else {
return nil
}
guard let document = CGPDFDocument(formatterUrl as CFURL) else {
return nil
}
guard let page = document.page(at: index) else {
return nil
}
let pageRect = page.getBoxRect(.mediaBox)
if #available(iOS 10.0, *) {
let renderGraph = UIGraphicsImageRenderer(size: pageRect.size)
let drawImage = renderGraph.image { context in
UIColor.white.set()
context.fill(pageRect)
context.cgContext.translateBy(x: 0.0, y: pageRect.size.height)
context.cgContext.scaleBy(x: scalex, y: scaley)
context.cgContext.drawPDFPage(page)
}
return drawImage
} else {
UIGraphicsBeginImageContextWithOptions(pageRect.size, false, 1.0)
let context = UIGraphicsGetCurrentContext()
context?.setFillColor(UIColor.white.cgColor)
context?.fill(pageRect)
context?.translateBy(x: 0.0, y: pageRect.size.height)
context?.scaleBy(x: scalex, y: scaley)
context?.drawPDFPage(page)
let pdfImage = UIGraphicsGetImageFromCurrentImageContext()
UIGraphicsEndImageContext()
return pdfImage
}
}
}
// 用来表示保存图片到自定义相册或者系统相册的操作结果
enum HBPhotosAlbumUtilResult {
case success, error, denied
}
extension HBPhotosAlbumHelperUtil {
// 请求获取操作系统相册权限
// 返回true说明已经得到授权
static var photoAlbumAuthorized: Bool {
return PHPhotoLibrary.authorizationStatus() == .authorized || PHPhotoLibrary.authorizationStatus() == .notDetermined
}
// 保存图片到自定义相册中
func saveImageToCustomAlbum(saveImage markImage: UIImage, customAlbumName albumName: String = "丰巢管家电子发票", completion: ((_ result: HBPhotosAlbumUtilResult) -> Void)?) {
guard HBPhotosAlbumHelperUtil.photoAlbumAuthorized else {
completion?(.denied)
return
}
var assetAlbum: PHAssetCollection?
// 如果相册名称为空,则图片默认保存到系统相册里面
if albumName.isEmpty {
let assetCollection = PHAssetCollection.fetchAssetCollections(with: .smartAlbum, subtype: .smartAlbumUserLibrary,
options: nil)
assetAlbum = assetCollection.firstObject
} else {
// 获取指定的相册是否存在
let assetList = PHAssetCollection.fetchAssetCollections(with: .album, subtype: .any, options: nil)
assetList.enumerateObjects { albumOption, _, stop in
let assetCollection = albumOption
if albumName == assetCollection.localizedTitle {
assetAlbum = assetCollection
stop.initialize(to: true)
}
}
// 自定义相册不存在就创建
if assetAlbum == nil {
PHPhotoLibrary.shared().performChanges({
PHAssetCollectionChangeRequest.creationRequestForAssetCollection(withTitle: albumName)
}) { _, _ in
self.saveImageToCustomAlbum(saveImage: markImage, customAlbumName: albumName, completion: completion)
}
}
}
// 保存图片
PHPhotoLibrary.shared().performChanges({
let result = PHAssetChangeRequest.creationRequestForAsset(from: markImage)
if !albumName.isEmpty {
if let assetPlaceHolder = result.placeholderForCreatedAsset,
let lastAssetAlbum = assetAlbum,
let albumChangeRequset = PHAssetCollectionChangeRequest(for:
lastAssetAlbum) {
albumChangeRequset.addAssets([assetPlaceHolder] as NSArray)
}
}
}) { isSuccess, _ in
guard isSuccess else {
completion?(.error)
return
}
completion?(.success)
}
}
}
extension String {
/// URL legalization
public var urlValue: URL? {
if let url = URL(string: self) {
return url
}
var set = CharacterSet()
set.formUnion(.urlHostAllowed)
set.formUnion(.urlPathAllowed)
set.formUnion(.urlQueryAllowed)
set.formUnion(.urlFragmentAllowed)
return self.addingPercentEncoding(withAllowedCharacters: set).flatMap { URL(string: $0) }
}
}
You can use the api like this:
// Use this way to achieve pdf to image
HBPhotosAlbumHelperUtil.shared.drawToImagePDFFromURL(pdfurl: "link to pdf file")
// In this way, you can save pictures to the system custom album.
HBPhotosAlbumHelperUtil.shared.saveImageToCustomAlbum(saveImage: UIImage()) { (result) in
}
Make sure that your pptURL is file url.
URL(string: "path/to/pdf") and URL(fileURLWithPath: "path/to/pdf") are different things and you must use the last one while initiating your url.
The output should start with "file:///" prefix, f.e.
file:///Users/dev/Library/Developer/CoreSimulator/Devices/4FF18699-D82F-4308-88D6-44E3C11C955A/data/Containers/Bundle/Application/8F230041-AC15-45D9-863F-5778B565B12F/myApp.app/example.pdf

Create Animation from Array of images

I am having trouble taking 6 photos and storing them in an object array in order to animate them. I keep getting an error saying:
Array index out of range
Also, I realized that the "image" object isn't recognized outside the if-statement for some reason. What am I doing wrong?
func didPressTakePhoto(){
var picArray: [UIImage] = []
for index in 1...6 {
if let videoConnection = stillImageOutput?.connectionWithMediaType(AVMediaTypeVideo){
videoConnection.videoOrientation = AVCaptureVideoOrientation.Portrait
stillImageOutput?.captureStillImageAsynchronouslyFromConnection(videoConnection, completionHandler: {
(sampleBuffer, error) in
if sampleBuffer != nil {
let imageData = AVCaptureStillImageOutput.jpegStillImageNSDataRepresentation(sampleBuffer)
let dataProvider = CGDataProviderCreateWithCFData(imageData)
let cgImageRef = CGImageCreateWithJPEGDataProvider(dataProvider, nil, true, CGColorRenderingIntent.RenderingIntentDefault)
let image = UIImage(CGImage: cgImageRef!, scale: 1.0, orientation: UIImageOrientation.Right)
self.tempImageView.image = image
self.tempImageView.hidden = false
UIImageWriteToSavedPhotosAlbum(image, nil, nil, nil);
picArray[index] = image;
}
})
}
}
self.gifView.animationImages = picArray;
self.gifView.animationDuration = 1.0
self.gifView.startAnimating()
}
picArray is empty, so you shouldn't use insert method. instead you have to use the append method.
The reason why is picArray is empty you are inserting the values inside the asynchronous block. The for loop completes before inserting images because it doesn't wait for asynchronous blocks to be complete.
You have to wait for asynchronous block to be complete before animating the image view.
You can achieve this using dispatch_group
func didPressTakePhoto(){
var picArray: [UIImage] = []
let dispatchGroup = dispatch_group_create()
for index in 1...6 {
dispatch_group_enter(dispatchGroup)
if let videoConnection = stillImageOutput?.connectionWithMediaType(AVMediaTypeVideo){
videoConnection.videoOrientation = AVCaptureVideoOrientation.Portrait
stillImageOutput?.captureStillImageAsynchronouslyFromConnection(videoConnection, completionHandler: {
(sampleBuffer, error) in
if sampleBuffer != nil {
let imageData = AVCaptureStillImageOutput.jpegStillImageNSDataRepresentation(sampleBuffer)
let dataProvider = CGDataProviderCreateWithCFData(imageData)
let cgImageRef = CGImageCreateWithJPEGDataProvider(dataProvider, nil, true, CGColorRenderingIntent.RenderingIntentDefault)
let image = UIImage(CGImage: cgImageRef!, scale: 1.0, orientation: UIImageOrientation.Right)
self.tempImageView.image = image
self.tempImageView.hidden = false
UIImageWriteToSavedPhotosAlbum(image, nil, nil, nil);
picArray.append(image);
}
dispatch_group_leave(dispatchGroup)
})
} else {
dispatch_group_leave(dispatchGroup)
}
}
dispatch_group_notify(dispatchGroup, dispatch_get_main_queue()) {
self.gifView.animationImages = picArray;
self.gifView.animationDuration = 1.0
self.gifView.startAnimating()
}
}
Hope this helps.

App is crashing silently during custom camera (Swift)

The app is crashing at random points in this function. I believe I need to scale it down but I am not sure. The only requirements I have for the image is that it remains a square and it remains decently sized because I need it to be big enough to take the entire screens width.
Here is an error that sometimes comes along with the crash:
warning: could not load any Objective-C class information. This will significantly reduce the quality of type information available.
#IBAction func didPressTakePhoto(sender: UIButton) {
self.previewLayer?.connection.enabled = false
if let videoConnection = stillImageOutput!.connectionWithMediaType(AVMediaTypeVideo) {
videoConnection.videoOrientation = AVCaptureVideoOrientation.Portrait
stillImageOutput?.captureStillImageAsynchronouslyFromConnection(videoConnection, completionHandler: {(sampleBuffer, error) in
if (sampleBuffer != nil) {
let imageData = AVCaptureStillImageOutput.jpegStillImageNSDataRepresentation(sampleBuffer)
let dataProvider = CGDataProviderCreateWithCFData(imageData)
let cgImageRef = CGImageCreateWithJPEGDataProvider(dataProvider, nil, true, CGColorRenderingIntent.RenderingIntentDefault)
var image = UIImage()
if UIDevice.currentDevice().orientation == .Portrait{
image = UIImage(CGImage: cgImageRef!, scale: 1.0, orientation: UIImageOrientation.Right)
}else if UIDevice.currentDevice().orientation == .LandscapeLeft{
image = UIImage(CGImage: cgImageRef!, scale: 1.0, orientation: UIImageOrientation.Up)
}else if UIDevice.currentDevice().orientation == .LandscapeRight{
image = UIImage(CGImage: cgImageRef!, scale: 1.0, orientation: UIImageOrientation.Down)
}
//Crop the image to a square
let imageSize: CGSize = image.size
let width: CGFloat = imageSize.width
let height: CGFloat = imageSize.height
if width != height {
let newDimension: CGFloat = min(width, height)
let widthOffset: CGFloat = (width - newDimension) / 2
let heightOffset: CGFloat = (height - newDimension) / 2
UIGraphicsBeginImageContextWithOptions(CGSizeMake(newDimension, newDimension), false, 0.0)
image.drawAtPoint(CGPointMake(-widthOffset, -heightOffset), blendMode: .Copy, alpha: 1.0)
image = UIGraphicsGetImageFromCurrentImageContext()
let imageData: NSData = UIImageJPEGRepresentation(image, 0.1)!
UIGraphicsEndImageContext()
self.captImage = UIImage(data: imageData)!
}
}
self.performSegueWithIdentifier("fromCustomCamera", sender: self)
})
}
}
This code is running in my viewDidAppear and stillImageOutput is returning nil when I take a photo.
if self.isRunning == false{
captureSession = AVCaptureSession()
captureSession!.sessionPreset = AVCaptureSessionPresetPhoto
let backCamera = AVCaptureDevice.defaultDeviceWithMediaType(AVMediaTypeVideo)
var error: NSError?
do {
input = try AVCaptureDeviceInput(device: backCamera)
} catch let error1 as NSError {
error = error1
print(error)
input = nil
}
if error == nil && captureSession!.canAddInput(input) {
captureSession!.addInput(input)
stillImageOutput = AVCaptureStillImageOutput()
stillImageOutput!.outputSettings = [AVVideoCodecKey: AVVideoCodecJPEG]
if captureSession!.canAddOutput(stillImageOutput) {
captureSession!.addOutput(stillImageOutput)
previewLayer = AVCaptureVideoPreviewLayer(session: captureSession)
previewLayer!.videoGravity = AVLayerVideoGravityResizeAspectFill
previewLayer!.connection?.videoOrientation = AVCaptureVideoOrientation.Portrait
previewView.layer.addSublayer(previewLayer!)
captureSession!.startRunning()
self.isRunning = true
}
}
}
Fixed it. The reason it was crashing was actually due to my images being way too big. I had to compress them.

How to Circle the image

Hello I am using SDWebImage in my app. This is my code to make the image in circle
extension UIImage {
var circle: UIImage? {
let square = CGSize(width: min(size.width, size.height), height: min(size.width, size.height))
let imageView = UIImageView(frame: CGRect(origin: CGPoint(x: 0, y: 0), size: square))
imageView.contentMode = .ScaleAspectFill
imageView.image = self
imageView.layer.cornerRadius = square.width/2
imageView.layer.masksToBounds = true
UIGraphicsBeginImageContext(imageView.bounds.size)
guard let context = UIGraphicsGetCurrentContext() else { return nil }
imageView.layer.renderInContext(context)
let result = UIGraphicsGetImageFromCurrentImageContext()
UIGraphicsEndImageContext()
return result
}
}
copied it from here
I used to do images in cicle like this
let profilePicture = UIImage(data: NSData(contentsOfURL: NSURL(string:"http://i.stack.imgur.com/Xs4RX.jpg")!)!)!
profilePicture.circle
But Now As I am using SDWebImage its not working
cell.profileImageView.sd_setImageWithURL(UIImage().absoluteURL(profileImageUrl), placeholderImage: UIImage.init(named: "default-profile-icon")?.circle!)
Please let me know how can I make this extension work for SDWebImage
You can use the SDWebImageManager to download the image or take it from the cache and apply the circle in the completion block like this:
SDWebImageManager.sharedManager().downloadWithURL(NSURL(string:"img"), options: [], progress: nil) { (image:UIImage!, error:NSError!, cacheType:SDImageCacheType, finished:Bool) -> Void in
if (image != nil){
let circleImage = image.circle
cell.profileImageView.image = circleImage
}
}
Or you can use the version of the sd_setImageWithURL method that takes a completion block as a parameter
let completionBlock: SDWebImageCompletionBlock! = {(image: UIImage!, error: NSError!, cacheType: SDImageCacheType!, imageURL: NSURL!) -> Void in
if (image != nil){
let circleImage = image.circle
cell.profileImageView.image = circleImage
}
}
cell.profileImageView.sd_setImageWithURL(UIImage().absoluteURL(profileImageUrl), placeholderImage: UIImage.init(named: "default-profile-icon")?.circle!, completed: completionBlock)

Resources