I'm using CoreML and Vision to analyze a photo taken with the camera or imported from the library. Once the photo is obtained I run some code to make sure the photo is valid and if it is it returns true otherwise it returns false. I use it like so:
func photoOutput(_ output: AVCapturePhotoOutput, didFinishProcessingPhoto photo: AVCapturePhoto, error: Error?) {
if let error = error {
// display alert there is a problem
return
}
guard let imageData = photo.fileDataRepresentation(), let previewImage = UIImage(data: imageData) else {
// display alert there is a problem
return
}
if useVisionAndCoreMLToCheckIfIsImageValid(image: previewImage) {
tableData.append(previewImage)
} else {
// display alert image was not valid
}
}
The problem is there are 4 points inside the useVisionAndCoreMLToCheckIfIsImageValid function that can go wrong and I need to return false so I can jump out of the function and if it is valid there is 1 point where it can go right and I need to return true. But since the function returns a Bool I keep getting errors when trying to return true or false at those points:
How can I get rid of the above errors?
func useVisionAndCoreMLToCheckIfIsImageValid(image: UIImage) -> Bool {
if let cgImage = image.cgImage {
let foodModel = CustomFoodModel()
guard let model = try? VNCoreMLModel(for: foodModel.model) else {
return false
}
let request = VNCoreMLRequest(model: model) { [weak self](request, error) in
if let error = error {
// 1st point - if there is an error return false
return false
}
guard let results = request.results as? [VNClassificationObservation], let topResult = results.first else {
// 2nd point - if there is a nil value here return false
return false
}
if topResult.confidence > 0.8 {
// 3rd point - if confidence is greater then 80% return true
return true
} else {
// 4th point - if confidence is less then 80% return false
return false
}
}
let handler = VNImageRequestHandler(cgImage: cgImage, options: [:])
DispatchQueue.global(qos: .userInitiated).async { [weak self] in
do {
try handler.perform([request])
} catch let err as NSError {
// 5th point - if there is a try error return false
return false
}
}
}
// if the cgImage is nil return false
return false
}
The return statements you get errors for are indeed return statements for the closures e.g. - VNCoreMLRequest(model: model) { - return for this block - },
VNCoreMLRequest(model: model) { -return for this block- } and not for the useVisionAndCoreMLToCheckIfIsImageValid function itself.
You need to rework your strategy and understand the async picture in your case.
I strongly recommend to first get some knowledge on the topic rather than copy-paste a solution you don't truly understand.
Some blog post on the async programming:
https://ashfurrow.com/blog/comparative-asynchronous-programming/
I was initially going to use completionHandlers instead of a bool but I thought there was an easier way which there isn't. #vadian sent me a link in the comments to a similar SO question that basically said what I'm trying to do cannot be done because as DenislavaShentova's answer said the return statements with error are for the blocks and not the function itself.
Here's the useVisionAndCoreMLToCheckIfIsImageValid code signature using 2 completionHandlers instead of returning a Bool
func useVisionAndCoreMLToCheckIfIsImageValid(image: UIImage, falseCompletion: #escaping ()->(), trueCompletion: #escaping ()->()) {
if let cgImage = image.cgImage {
let foodModel = CustomFoodModel()
guard let model = try? VNCoreMLModel(for: foodModel.model) else {
falseCompletion()
return
}
let request = VNCoreMLRequest(model: model) { [weak self](request, error) in
if let error = error {
// 1st point - run code for false
falseCompletion()
return
}
guard let results = request.results as? [VNClassificationObservation], let topResult = results.first else {
// 2nd point - run code for false
falseCompletion()
return
}
if topResult.confidence > 0.8 {
// 3rd point - run code for false
trueCompletion()
} else {
// 4th point - run code for false
falseCompletion()
}
}
let handler = VNImageRequestHandler(cgImage: cgImage, options: [:])
DispatchQueue.global(qos: .userInitiated).async { [weak self] in
do {
try handler.perform([request])
} catch let err as NSError {
// 5th point - run code for false
falseCompletion()
}
}
}
// if the cgImage is nil run code for false
falseCompletion()
}
and here is the function is use:
func photoOutput(_ output: AVCapturePhotoOutput, didFinishProcessingPhoto photo: AVCapturePhoto, error: Error?) {
if let error = error {
// display alert there is a problem
return
}
guard let imageData = photo.fileDataRepresentation(), let previewImage = UIImage(data: imageData) else {
// display alert there is a problem
return
}
useVisionAndCoreMLToCheckIfIsImageValid(image: previewImage, falseCompletion: thereIsAProblem, trueCompletion: imageIsValid)
}
func imageIsValid() {
tableData.append(previewImage)
}
func thereIsAProblem() {
// do something
}
Related
I have a PKCanvas, what i am trying to do is to take the pkcanvas drawing that the user has made ad compare to another image.
However i keep getting nil returned section.
Why is this? my drawing and picture elements are returning nil. but my image2 element is returning the users drawing as expected.
in the code below i am just comparing the drawing the user made against itself (just for testing purposes)
Also "nothing here" does get printed.
//function type for ciimage
func featureprintObservationForImage(_ image: CIImage?) -> VNFeaturePrintObservation? {
guard let ciImage = image else {
print("nothing here")
return nil
}
let requestHandler = VNImageRequestHandler(ciImage: ciImage, options: [:])
let request = VNGenerateImageFeaturePrintRequest()
do {
try requestHandler.perform([request])
return request.results?.first as? VNFeaturePrintObservation
} catch {
print("Vision error: \(error)")
return nil
}
}
#IBAction func finishCalculation(_ sender: Any) {
//geting drawing
UIGraphicsBeginImageContextWithOptions(theCanvasView.bounds.size, false, UIScreen.main.scale)
theCanvasView.drawHierarchy(in: theCanvasView.bounds, afterScreenUpdates: true)
let image2 = UIGraphicsGetImageFromCurrentImageContext()
UIGraphicsEndImageContext()
let picture = featureprintObservationForImage(image2?.ciImage)
let drawing = featureprintObservationForImage(image2?.ciImage)
var distance = Float(0)
do {
try picture!.computeDistance(&distance, to: drawing!)
} catch {
print(error)
}
print("picture to drawing")
print(distance)
}
I am working on a similar feature to 'liking/unliking a post'.
I have an MVVM architecture as;
struct MyStructModel {
var isLiked: Bool? = false
}
class MyStructView {
var isLiked: Bool
init(myStructModel: MyStructModel) {
self.isLiked = myStructModel.isLiked ?? false
}
}
I successfully get the value of whether the post is liked or not here;
func isPostLiked(documentID: String, completion: #escaping (Bool) -> Void) {
guard let authID = auth.id else { return }
let query = reference(to: .users).document(authID).collection("liked").document(documentID)
query.getDocument { (snapshot, error) in
if error != nil {
print(error as Any)
return
}
guard let data = snapshot?.data() else { return }
if let value = data["isLiked"] as? Bool {
completion(value)
} else {
completion(false)
}
}
}
func retrieveReviews(completion: #escaping([MyStructModel]) -> ()) {
var posts = [MyStructModel]()
let query = reference(to: .posts).order(by: "createdAt", descending: true)
query.getDocuments { (snapshot, error) in
if error != nil {
print(error as Any)
return
}
guard let snapshotDocuments = snapshot?.documents else { return }
for document in snapshotDocuments {
if var post = try? JSONDecoder().decodeQuery(MyStructModel.self, fromJSONObject: document.decode()) {
// isLiked is nil here...
self.isPostLiked(documentID: post.documentID!) { (isLiked) in
post.isLiked = isLiked
print("MODEL SAYS: \(post.isLiked!)")
// isLiked is correct value here...
}
posts.append(post)
}
completion(posts)
}
}
}
However, when it gets to my cell the value is still nil.
Adding Cell Code:
var post: MyStructView? {
didSet {
guard let post = post else { return }
print(post.isLiked!)
}
}
Your isLiked property is likely nil in your cells because the retrieveReviews function doesn't wait for the isPostLiked function to complete before completing itself.
You could easily solve this issue by using DispatchGroups. This would allow you to make sure all of your Posts have their isLiked value properly set before being inserted in the array, and then simply use the DispatchGroup's notify block to return all the loaded posts via the completion handler:
func retrieveReviews(completion: #escaping([MyStructModel]) -> ()) {
var posts = [MyStructModel]()
let query = reference(to: .posts).order(by: "createdAt", descending: true)
query.getDocuments { [weak self] (snapshot, error) in
guard let self = self else { return }
if error != nil {
return
}
guard let documents = snapshot?.documents else { return }
let dispatchGroup = DispatchGroup()
for document in documents {
dispatchGroup.enter()
if var post = try? JSONDecoder().decodeQuery(MyStructModel.self, fromJSONObject: document.decode()) {
self.isPostLiked(documentID: post.documentID!) { isLiked in
post.isLiked = isLiked
posts.append(post)
dispatchGroup.leave()
}
}
}
dispatchGroup.notify(queue: .main) {
completion(posts)
}
}
}
On a slow connection when I click on back button on a VC it crashes while accessing navigation controller. VC is already deallocated but setNavBarTitle is executed after going back to another view. I understand that function is executing while VC is already deallocated but Im not sure what's the best way to handle such scenario?
override func viewWillAppear(_ animated: Bool){
super.viewWillAppear(animated)
fetchProfile(clientId: clientId) { (result, error) in
if result?.data != nil {
if (result?.success)! {
self.clientProfile = result!.data!
// Avatar
let clientImageView = UIImageView()
if let url = URL(string: result!.data!.pic_url!) {
clientImageView.image = UIImage(named: "System/defaultAvatar")
let task = URLSession.shared.dataTask(with: url) { data, response, error in
guard let data = data, error == nil else { return }
// WARNING: UIImageView.image must be used from main thread only
clientImageView.image = UIImage(data: data)
self.setNavBarTitle(image: clientImageView.image!)
}
task.resume()
}
}
}
}
}
}
private func setNavBarTitle(image: UIImage) {
// Crashes here -> Thread 11: Fatal error: Unexpectedly found nil while unwrapping an Optional value
let navigationBarHeight: CGFloat = self.navigationController!.navigationBar.frame.height
}
You can try using [weak self] in your method callback, so even though the method is called after the VC deallocated it wont cause a crash, since self wont be existing:
override func viewWillAppear(_ animated: Bool){
super.viewWillAppear(animated)
fetchProfile(clientId: clientId) { [weak self] (result, error) in
if result?.data != nil {
if (result?.success)! {
self?.clientProfile = result!.data!
// Avatar
let clientImageView = UIImageView()
if let url = URL(string: result!.data!.pic_url!) {
clientImageView.image = UIImage(named: "System/defaultAvatar")
let task = URLSession.shared.dataTask(with: url) { data, response, error in
guard let data = data, error == nil else { return }
DispatchQueue.main.async {
clientImageView.image = UIImage(data: data)
self?.setNavBarTitle(image: clientImageView.image!)
}
}
task.resume()
}
}
}
}
}
}
private func setNavBarTitle(image: UIImage) {
// Crashes here -> Thread 11: Fatal error: Unexpectedly found nil while unwrapping an Optional value
guard let navigationBarHeight: CGFloat = self.navigationController?.navigationBar.frame.height else {
return
}
}
This question already has an answer here:
Return a string from a web scraping function in swift
(1 answer)
Closed 4 years ago.
How can I return a value within an if let statement to be further returned within a function? Here is the code:
func loadUrl(url:String) -> String {
DispatchQueue.global().async {
do {
let appUrl = URL(string:url)!
let data = try Data(contentsOf:appUrl)
let json = try JSONSerialization.jsonObject(with: data) as! [String:Any]
print("Test from do")
if let results = json["results"] as? [[String:Any]] {
print("Test from if let 1")
if let first = results[0] as? [String:Any] {
print("Test from if let 2")
var cityStateLocation = first["formatted_address"]!
return cityStateLocation
//What needs to be returned
}
}
DispatchQueue.main.async {
print("No Error")
}
} catch {
DispatchQueue.main.async {
print("Cannot connect to the server.")
}
}
}
}
What I would like to be able to do is take cityStateLocation and return it in the func, but because it is a part of an if let statement within an .async method I don't know how to do that. Could someone please explain?
EDIT: I need the return value of cityStateLocation to equal a variable in a separate function. Here is the separate function:
#IBAction func continueButton(_ sender: Any) {
var cityState:String
if locationSwitch.isOn == true {
print(location.latitude)
print(location.longitude)
let url = "https://maps.googleapis.com/maps/api/geocode/json?latlng=\(location.latitude),\(location.longitude)&result_type=locality&key=AIzaSyDI-ZacHyPbLchRhkoaUTDokwj--z_a_jk"
loadUrl(url: url)
cityState = loadUrl(url: url)
} else {
cityState = ""
}
CoreDataHandler.saveObject(locationLocality: cityState)
}
Edit 2: The main reason why the "duplicate answer" is not a duplicate is that my code needs to call the return of this function within a separate function then save it to Core Data. Also, my code is not using an array.
You could modify your function to include a closure. For instance:
func loadUrl(url: String, completionHandler: #escaping (_ location: String?) -> (Void)) {
And then, where you want to return it, you'd pass it in as such.
completionHandler(cityStateLocation)
I made it an optional so that, in your fail paths, you could return nil.
Then, where you call the function would change. Using trailing closure syntax, it could look like this:
loadUrl(url: "someurl.com/filepath.txt") { optionalLocation in
guard let nonOptionalLocation = optionalLocation else {
// Location was nil; Handle error case here
return
}
// Do something with your location here, like setting UI or something
}
This is a fairly common pattern when dealing with asynchronous activity, such as working with network calls.
The simplest (perhaps no the prettiest), way of doing this would simply be to declare and instantiate a variable above the dispatch queue. Then you can set the variable equal to whatever you want, within the dispatch queue, and return it afterwards. You can change the type of ret, so that it suits your needs more directly.
func loadUrl(url:String) -> String {
var ret = NSObject()
DispatchQueue.global().async {
do {
let appUrl = URL(string:url)!
let data = try Data(contentsOf:appUrl)
let json = try JSONSerialization.jsonObject(with: data) as! [String:Any]
print("Test from do")
if let results = json["results"] as? [[String:Any]] {
print("Test from if let 1")
if let first = results[0] as? [String:Any] {
print("Test from if let 2")
var cityStateLocation = first["formatted_address"]!
ret = cityStateLocation
//What needs to be returned
}
}
DispatchQueue.main.async {
print("No Error")
}
} catch {
DispatchQueue.main.async {
print("Cannot connect to the server.")
}
}
}
return ret
}
DispatchQueue.global().async will cause the coded included in the closure to be executed at some point the future, meaning you loadUrl function will return (almost) immediately.
What you need is some kind of callback which can be called when you have a result (AKA closure)
This is just another way to approach the problem, the difference between this and Josh's example is simply, I provide an additional closure to handle the errors
func loadUrl(url:String, complition: #escaping (String?) -> Void, fail: #escaping (Error) -> Void) {
DispatchQueue.global().async {
do {
let appUrl = URL(string:url)!
let data = try Data(contentsOf:appUrl)
let json = try JSONSerialization.jsonObject(with: data) as! [String:Any]
print("Test from do")
if let results = json["results"] as? [[String:Any]], !results.isEmpty {
print("Test from if let 1")
let first = results[0]
print("Test from if let 2")
if let cityStateLocation = first["formatted_address"] as? String {
complition(cityStateLocation)
} else {
complition(nil)
}
} else {
complition(nil)
}
} catch let error {
fail(error)
}
}
}
Which you might call using something like...
loadUrl(url: "your awesome url", complition: { (value) in
guard let value = value else {
// No value
return
}
// process value
}) { (error) in
// Handle error
}
I have some image files I need to get from a drone using the DJI SDK, but the way I've done it doesn't seem the most ideal. I was wondering how I would be able to do this, but inside of a for-loop instead. Here's the code:
func getImages(with file: DJIMediaFile?) {
guard let file = file else { return }
file.fetchPreview { (error) in
if error != nil {
print(String(describing: error?.localizedDescription))
return
}
guard let image = file.preview else { print("No preview"); return }
self.images.append(image)
self.imageView.image = image
self.index += 1
if self.index < self.mediaList.count {
self.getImages(with: self.mediaList[self.index])
} else {
// [...]
}
}
}
}
Any help would be appreciated by someone who's familiar with the DJI SDK and Swift (perhaps I should have used some other API method?). Thanks!
This is mostly a swift question rather than a DJI SDK issue and more related to code review. But its simply
func getNewImages() {
for file in self.mediaList where file != nil {
self.getImages(with: file!) { image, error in //not ideal to force unwarp but i don't know if the where clause supports if let binding, or if file is really nil, you'll have to check yourself in code
guard let newImage = image else { return }
self.cachedImages.append(newImage)
}
}
self.setImage(at: 0)
}
func getImages(with file: DJIMediaFile, completion: (UIImage?, Error?)->()) {
file.fetchPreview { (error) in // is this async? If its' async you may want to use dispatch groups in the for in loop to run the setImage at the correct time
if let newError = error {
print(String(describing: newError.localizedDescription))
completion(nil, error)
return
}
guard let image = file.preview else {
print("No preview")
completion(nil, nil)
return
}
completion(image, nil)
}
}
func setImage(at row: Int) {
guard row < self.cachedImages.count - 1 else { return }
self.imageView.image = self.cachedImages[row]
}
Your recursion may result in errors you don't want. This is kinda what you want to do but it should be straightforward to change based on what I can see here.
Edit: You may also be intending to replace the file as people are loading their images from the drone itself in which case the code is more like this:
func getNewImages() {
for file in self.mediaList where file != nil {
self.getImages(with: file!) { image, error in //not ideal to force unwarp but I don't know if the where clause supports if let binding, or if file is really nil, you'll have to check yourself in code
guard let newImage = image else { return }
self.imageView.image = newImage
}
}
}
func getImages(with file: DJIMediaFile, completion: (UIImage?, Error?)->()) {
file.fetchPreview { (error) in
if let newError = error {
print(String(describing: newError.localizedDescription))
completion(nil, error)
return
}
guard let image = file.preview else {
print("No preview")
completion(nil, nil)
return
}
completion(image, nil)
}
}
edit2: Also note, this is not necessarily the best way to do this, but previously is how you do it in a for loop. I think a better way would be to use map or a forEach on the mediaList array itself like so. (same getImages method, nothing changes). This could be further refined as well between protocols, extensions and what not but without knowing your program, its hard to do better recommendations.
func getNewImages() {
self.mediaList.forEach { file in
guard let newFile = file else { return }
self.getImages(with: newFile) { image, error in
guard let newImage = image else { return }
self.imageView.image = newImage
}
}
}