Concurrent image processing task with photo taking in iOS - ios

i need your help with GCD in swift, i just started using this tool and i'm little confused. So let's say i have a function foo() that taking photo and saving it to var, inside this function there is another function that checking if there are faces on that photo and returning true or false. Next, when user taps usePhotoFromPreview button it will save photo through custom delegate whenever there aren't any faces on that photo like this:
var image: UIImage?
var checkingPhotoForFaces: Bool?
func foo(){
let videoConnection = imageOutput?.connection(withMediaType: AVMediaTypeVideo)
imageOutput?.captureStillImageAsynchronously(from: videoConnection, completionHandler: { (imageDataSampleBuffer, error) -> Void in
if let imageData = AVCaptureStillImageOutput.jpegStillImageNSDataRepresentation(imageDataSampleBuffer) {
image = UIImage(data: imageData)
self.checkingPhotoForFaces = self.detect(image: image)
}
})
}
func detect(image: UIImage) -> Bool{
.....
}
func usePhotoFromPreview(){
self.dismiss(animated: true, completion: {
if self.checkingPhotoForFaces == false{
self.delegate?.takenPhoto(photo: self.image!)
print("photo taken")
}else{
self.delegate?.takenPhoto(photo: nil)
print("no photo taken")
}
})
}
So now the tricky part, i'd like to make detect function to be executed asynchronously and only when it's done execute usePhotoFromPreview. detect func i suppose should be on main thread because of CoreImage implementation. I simply made something like this:
func foo(){
...
DispatchQueue.global().async {
self.checkingPhotoForFaces = self.detect(image: stillImage)
}
...
}
but the problem is when user taps too quick on that button it won't work, cuz detect func is still processing, so i think i need some kind of queue but i'm confused which one.
btw. please, do not cling to the lack of some parameters above, i'm writing it on the fly and that's not the point
Thank you for your help

Change detect as commented above:
func detect(image: UIImage, completion: ()->(detectedFaces: Bool)){
//.....
completion(true|false)
}
Then
if let imageData = AVCaptureStillImageOutput.jpegStillImageNSDataRepresentation(imageDataSampleBuffer) {
image = UIImage(data: imageData)
self.detect(image) { detectedFaces in
self.checkingPhotoForFaces = detectedFaces
//Enable button?
}
}

Related

Where to Put DispatchQueue Swift

I am a new developer. I'm using Swift 4.2 and Xcode 10.2.
I'm trying to show a spinner while a photo is being saved. I am simulating a slow connection on my iPhone in Developer Mode to test it. Using the code below the spinner does not show up. The view goes right to the end where the button shows and says the upload is complete (when it isn't). I tried putting it all into a DispatchQueue.global(qos: .userinitiated).async and then showing the button back on the main queue. I also tried putting the showSpinner on a DispatchQueue.main and then savePhoto on a .global(qos: .utility). But I cleary don't understand the GCD processes.
Here's my code:
func savePhoto(image:UIImage) {
// Add a spinner (from the Extensions)
self.showSpinner(onView: self.view)
PhotoService.savePhoto(image: image) { (pct) in
// Can show the loading bar here.
}
// Stop the spinner
self.removeSpinner()
// Show the button.
self.goToPhotosButtonLabel.alpha = 1
self.doneLabel.alpha = 1
}
What types of DispatchQueues should I use and where should I put them?
Here is the savePhoto code:
static func savePhoto(image:UIImage, progressUpdate: #escaping (Double) -> Void) {
// Get data representation of the image
let photoData = image.jpegData(compressionQuality:0.1)
guard photoData != nil else {
print("Couldn't turn the image into data")
return
}
// Get a storage reference
let userid = LocalStorageService.loadCurrentUser()?.userId
let filename = UUID().uuidString
let ref = Storage.storage().reference().child("images/\(String(describing: userid))/\(filename).jpg")
// Upload the photo
let uploadTask = ref.putData(photoData!, metadata: nil) { (metadata, error) in
if error != nil {
// An error during upload occurred
print("There was an error during upload")
}
else {
// Upload was successful, now create a database entry
self.createPhotoDatabaseEntry(ref: ref, filename: filename)
}
}
uploadTask.observe(.progress) { (snapshot) in
let percentage:Double = Double(snapshot.progress!.completedUnitCount /
snapshot.progress!.totalUnitCount) * 100.00
progressUpdate(percentage)
}
}
Since the code that saves the photo is asynchronous , so your current code removes the spinner directly after it's added before the upload is complete
func savePhoto(image:UIImage) {
// Add a spinner (from the Extensions)
self.showSpinner(onView: self.view)
PhotoService.savePhoto(image: image) { (pct) in
// remove spinner when progress is 100.0 = upload complete .
if pct == 100.0 {
// Stop the spinner
self.removeSpinner()
// Show the button.
self.goToPhotosButtonLabel.alpha = 1
self.doneLabel.alpha = 1
}
}
}
Here you don't have to use GCD as firebase upload runs in another background thread so it won't block the main thread

Reload CollectionView data on Button Pressed

I have a camera app that displays thumbnails of the user's photo library in a UICollectionView layered on top of the live camera feed UIView.
I was hoping that I could easily 'refresh' the contents of the UICollectionView every time the user takes a new photo using the reloadData() property:
#IBAction func snapNewPhoto(_ sender: Any) {
takePhotoMethod()
myCollectionView.reloadData()
myCollectionView.collectionViewLayout.invalidateLayout()
}
and this is my takePhoto method:
func takePhotoMethod () {
if let videoConnection = sessionOutput.connection(with: AVMediaType.video) {
sessionOutput.captureStillImageAsynchronously(from: videoConnection, completionHandler: {
buffer, error in
let imageData = AVCaptureStillImageOutput.jpegStillImageNSDataRepresentation(buffer!)
self.imageArray.append(UIImage(data: imageData!)!)
print ("Photo taken", self.imageArray.count)
})
}
}
This does not work, I guess the solution is a bit more complex, do I have to 'reload' the entire View Controller to reload the UICollectionView or is there another more eloquent approach?
func takePhotoMethod () {
if let videoConnection = sessionOutput.connection(with: AVMediaType.video) {
sessionOutput.captureStillImageAsynchronously(from: videoConnection, completionHandler: {
buffer, error in
let imageData = AVCaptureStillImageOutput.jpegStillImageNSDataRepresentation(buffer!)
self.imageArray.append(UIImage(data: imageData!)!)
myCollectionView.reloadData()//reload must be done in here.
print ("Photo taken", self.imageArray.count)
})
}
}

UIDocument not saving to file despite indicating success

I'm trying to open, modify, and save a file in iCloud Drive using UIDocument. When I call save(to:for:completionHandler:) with the file location and using .forOverwriting for the UIDocumentSaveOperation, it completes with a status of success = true. However, the iCloud file (as seen in both desktop and iOS file browser) does not update, and when reopening the file, the changes are not shown. I've verified that contents(forType:) returns the correct (modified) file contents when saving.
(Note: I've already looked at this question, but it wasn't very helpful 😕)
Here are the relevant sections of code:
MainViewController.swift:
var saveFile: SBDocument?
#IBAction func bbiOpen_pressed(_ sender: UIBarButtonItem) {
if saveFile == nil {
let importMenu = UIDocumentMenuViewController(documentTypes: self.UTIs, in: .import)
importMenu.delegate = self
importMenu.popoverPresentationController?.barButtonItem = bbiOpen
self.present(importMenu, animated: true, completion: nil)
} else {
willClose()
}
}
func willClose(_ action: UIAlertAction?) {
if saveFile!.hasUnsavedChanges {
dlgYesNoCancel(self, title: "Save Changes?", message: "Would you like to save the changes to your document before closing?", onYes: doSaveAndClose, onNo: doClose, onCancel: nil)
} else {
doSaveAndClose(action)
}
}
func doSaveAndClose(_ action: UIAlertAction?) {
saveFile?.save(to: saveFileURL!, for: .forOverwriting, completionHandler: { Void in
self.saveFile?.close(completionHandler: self.didClose)
})
}
func documentPicker(_ controller: UIDocumentPickerViewController, didPickDocumentAt url: URL) {
saveFile = SBDocument(fileURL: url)
saveFile!.open(completionHandler: { success in self.finishOpen(didCompleteSuccessfully: success) })
}
func finishOpen(didCompleteSuccessfully result: Bool) {
if result {
print(saveFile!.localizedName)
saveFileURL = saveFile!.fileURL
saveFileName = saveFile!.localizedName
self.navTitleBar.prompt = saveFileName
bbiOpen.title = NSLocalizedString("titleClose", comment: "Close")
bbiOpen.style = .plain
} else {
saveFile = nil
}
}
#IBAction func bbiSave_pressed(_ sender: UIBarButtonItem) {
self.saveFile!.save(to: self.saveFileURL!, for: .forOverwriting, completionHandler: self.didSave)
}
func didSave(_ success: Bool) {
guard success else {
print("Error saving soundboard file to \(String(describing: saveFileURL))")
return
}
print("File saved successfully")
}
SBDocument.swift:
class SBDocument: UIDocument {
override var fileType: String? { get { return "com.whitehatenterprises.SoundBoardFX.sbd" } }
override var savingFileType: String? { get { return "com.whitehatenterprises.SoundBoardFX.sbd" } }
override init(fileURL url: URL) {
super.init(fileURL: url)
}
override func contents(forType typeName: String) throws -> Any {
let arr = NSArray(array: SoundEffects)
let data: NSData = NSKeyedArchiver.archivedData(withRootObject: arr) as NSData
return data
}
}
Update:
I really need help with this, and I've tried everything I can think of to fix this. Any assistance you could give me would be greatly appreciated.
The way the initial file generation works for me is:
let doc = YourUIDocumentClass(fileURL: fileURL)
doc.save(to: fileURL, for: .forCreating) { success in
...
}
Then modify the file and then do:
doc.save(to: fileURL, for: .forOverwriting) { success in
...
}
when done. And subsequent accesses to the file are done by:
doc.open() { success in
...
}
doc.close() { success in
...
}
You might also need to do a:
doc.updateChangeCount(.done)
while the file is open to tell the document there are unsaved changes. Just setting this will cause a save after a few seconds. You don't even need the close to do that.
The ... means that you either have to nest all these or make sure there is enough time between them so they are completed.
In addition to the above answers, another cause of this can be that there's an error during the save process unrelated to contents(forType:).
For example, if you implement fileAttributesToWrite(to:for:) and throw an error, then this can cause a UIDocumentState.savingError even though contents(forType:) returns the correct data.
So according to
https://developer.apple.com/reference/uikit/uidocument
It looks like the save function isn't actually for saving a document. My understanding from reading it is that save is only for creating a new document. I understand that you are using the .forOverwriting to just save over it but there may be something in iCloud that wont let the complete overwrite happen.
In your doSaveAndClose method try calling
self.saveFile?.close(completionHandler: self.didClose)
by itself. You may have to do some type of if query where you check if the file exist. If it doesn't then call the .save(), else call the .close function. It seems that no matter what when the document it closed it saves changes.

How can I display image on fullscreen when user taps on my UIImage in Swift?

I'm displaying a photo in my app and I'm using UIImage for that. So far this is how I'm doing that:
func getDataFromUrl(url:NSURL, completion: ((data: NSData?, response: NSURLResponse?, error: NSError? ) -> Void)) {
NSURLSession.sharedSession().dataTaskWithURL(url) { (data, response, error) in
completion(data: data, response: response, error: error)
}.resume()
}
func downloadImage(url: NSURL){
getDataFromUrl(url) { (data, response, error) in
dispatch_async(dispatch_get_main_queue()) { () -> Void in
guard let data = data where error == nil else { return }
print("Download Finished")
self.requestPhoto.image = UIImage(data: data)
}
}
}
and then in viewDidLoad() I'm doing:
if let checkedUrl = NSURL(string: photo) {
requestPhoto.contentMode = .ScaleAspectFit
downloadImage(checkedUrl)
}
That leaves the UIImage filled with my photo, but it's not clickable and it's the size of the original component. Is there a way of adding some kind of listener or something that will display the photo on fullscreen when user taps the UIImage component?
What you need is to add a UITapGestureRecognizer to your requestPhoto image view. Something like this :
let tapGestureRecognizer = UITapGestureRecognizer(target: self, action: Selector("tapHandler"))
self.requestPhoto.addGestureRecognizer(tapGestureRecognizer)
self.requestPhoto.userInteractionEnabled = true
The last line is needed as UIImageView has it turned of by default, so the touches get canceled.
And then in the same class :
func tapHandler(sender: UITapGestureRecognizer) {
if sender.state == .Ended {
// change the size of the image view so that it fills the whole screen
}
}
The whole thing could also benefit from a flag saying if the view is fullscreen or not, so that when the user taps the fullscreen image you can reverse the process.
As always, you can read more in the docs.
How to enlarge thumbnail photo into fullscreen photo view on iOS?
I resolved this problem by creating new UIImageView object with the same image in it and set the frame of it equal to thumbnail image:
let fullscreenPhoto = UIImageView(frame: thumbnailImage.frame)
Then add image as sublayer into main window:
UIApplication.sharedApplication().windows.first!.addSubview(fullscreenPhoto)
Resizing is made with animation using block:
let windowFrame = UIApplication.sharedApplication().windows.first!.frame
UIView.animateWithDuration(0.4, delay: 0.0, options: .CurveEaseInOut, animations: {
fullscreenPhoto.frame = windowFrame
fullscreenPhoto.alpha = 1
fullscreenPhoto.layoutSubviews()
}, completion: { _ in
})
You can find ready solution here: https://github.com/lukszar/SLImageView
There is provided gesture recognizers for show and hide fullscreen.
You just need to set your UIImageView to SLImageView class in Storyboard and all magic is done.

How to return image from AV Foundation still capture and add it to graphics context for a screenshot?

I have a camera preview which works great. I want to take a screenshot of it and everything on top of it. However, since the usual screenshot approach: CALayer's renderInContext does not render content from the camera, I need to add it separately.
I have this function in my view controller captures the image and saves it to the camera roll.
#IBAction func snapStillImage(sender: AnyObject) {
print("snapStillImage")
(self.previewView.layer as! AVCaptureVideoPreviewLayer).connection.enabled=false;
dispatch_async(self.sessionQueue, {
// Update the orientation on the still image output video connection before capturing.
let videoOrientation = (self.previewView.layer as! AVCaptureVideoPreviewLayer).connection.videoOrientation
self.stillImageOutput!.connectionWithMediaType(AVMediaTypeVideo).videoOrientation = videoOrientation
// Flash set to Auto for Still Capture
ViewController.setFlashMode(AVCaptureFlashMode.Auto, device: self.videoDeviceInput!.device)
self.stillImageOutput!.captureStillImageAsynchronouslyFromConnection(self.stillImageOutput!.connectionWithMediaType(AVMediaTypeVideo), completionHandler: {
(imageDataSampleBuffer: CMSampleBuffer!, error: NSError!) in
if error == nil {
let data:NSData = AVCaptureStillImageOutput.jpegStillImageNSDataRepresentation(imageDataSampleBuffer)
let image:UIImage = UIImage( data: data)!
let libaray:ALAssetsLibrary = ALAssetsLibrary()
let orientation: ALAssetOrientation = ALAssetOrientation(rawValue: image.imageOrientation.rawValue)!
libaray.writeImageToSavedPhotosAlbum(image.CGImage, orientation: orientation, completionBlock: nil)
print("save to album")
}else{
// print("Did not capture still image")
print(error)
}
})
})
}
I want to return the image defined with let image:UIImage = UIImage( data: data)!
so that it is accessible by the following function
#IBAction func downloadButton(sender: UIButton) {
//Create the UIImage
UIGraphicsBeginImageContextWithOptions(previewView.frame.size, false, 0.0)
previewView.layer.renderInContext(UIGraphicsGetCurrentContext()!)
//ALSO ADD image captured with captureStillImageAsynchronouslyFromConnection right here to the context.
let image = UIGraphicsGetImageFromCurrentImageContext()
//Save it to the camera roll
UIImageWriteToSavedPhotosAlbum(image, nil, nil, nil)
}
This would let my screenshot include the recently captured image as well as all the stuff on top that gets added with UIGraphicsGetCurrentContext()!
I have heard that you do this by including the image in the completion handler block already included in the captureStillImageAsynchronouslyFromConnection function but I am not quite sure how to do this. Any suggestions?
You could add the captured still image as a UIImage as a subview of the previewView. Then make the call to renderInContext.
Assuming this code is all going on in the same class, you can make image a property like
class CameraViewController: UIViewController {
var image: UIImage!
...
Then in captureStillImageAsynchronouslyFromConnection
instead of
let image:UIImage = UIImage( data: data)!
use
self.image = UIImage( data: data)!
Then in downloadButton use
var imageView = UIImageView(frame: previewView.frame)
imageView.image = self.image
previewView.addSubview(imageView)
previewView.layer.renderInContext(UIGraphicsGetCurrentContext()!)
You can then remove the imageView if you want
imageView.removeFromSuperview()
You cannot render content from the camera because CALayer's renderInContext doesn't work.
Nonetheless you want to work that, the following link can help you.
Taking a screenshot when camera is on -iOS
https://developer.apple.com/library/ios/qa/qa1702/_index.html#//apple_ref/doc/uid/DTS40010192

Resources