I have a Google MLKit model for labeling an Image after capturing the image, but everytime I tried to process the Image, it always give me this error:
label process error:: Pipeline failed to fully start: Calculator::Open() for node "ClassifierClientCalculator" failed: #vk The TFLite Model Metadata must not contain label maps when text_label_map_file is used.
Here's my MLKit image labeler configuration code (this code is based on MLKit's documentation):
private func configureModelSource() { // Called in viewDidLoad()
guard let manifestPath = Bundle.main.path(forResource: "filename", ofType: "json") else { return }
guard let localModel = LocalModel(manifestPath: manifestPath) else { return }
let options = CustomImageLabelerOptions(localModel: localModel)
options.confidenceThreshold = NSNumber(value: 0.0)
imageLabeler = ImageLabeler.imageLabeler(options: options)
}
private func processImage(with image: UIImage) { // Called after capturing an Image
guard imageLabeler != nil else { return }
let visionImage = VisionImage(image: image)
visionImage.orientation = image.imageOrientation
imageLabeler?.process(visionImage) { labels, error in
guard error == nil, let labels = labels, !labels.isEmpty else {
print("label process error:: \(error?.localizedDescription ?? "nil")")
return
}
for label in labels {
// Do something...
}
}
}
Is there anyway to solve this? For context, the model.tflite file was updated. The file before the one that gives me this error works as expected. But the new model.tflite file always gives me this error everytime I run my app. Is this a file-related error or did I do something wrong with my code that I have to also update it?
Here's my understanding based on the error message:
Given you are using the LocalModel(manifestPath: manifestPath) API, it is expecting a legacy TFLite model format where the label map is provided through a separate text file and the model.tflite itself does not contain the label map. That's why your file before your model update works.
To use your updated model.tflite (which seems to contain the lab map inside its metadata), I think you can try the following to use the model.tflite file directly with the custom models API without going through the filename.json manifest:
guard let modelPath = Bundle.main.path(forResource: "model", ofType: "tflite") else { return }
guard let localModel = LocalModel(path: modelPath) else { return }
You can check out the documentation about custom models here: https://developers.google.com/ml-kit/vision/image-labeling/custom-models/ios
Related
I am struggling to understand why my event listener that I initialize on a document is not being triggered whenever I update the document within the app in a different UIViewController. If I update it manually in Google firebase console, the listener event gets triggered successfully. I am 100% updating the correct document too because I see it get updated when I update it in the app. What I am trying to accomplish is have a running listener on the current user that is logged in and all of their fields so i can just use 1 global singleton variable throughout my app and it will always be up to date with their most current fields (name, last name, profile pic, bio, etc.). One thing I noticed is when i use setData instead of updateData, the listener event gets triggered. For some reason it doesn't with updateData. But i don't want to use setData because it will wipe all the other fields as if it is a new doc. Is there something else I should be doing?
Below is the code that initializes the Listener at the very beginning of the app after the user logs in.
static func InitalizeWhistleListener() {
let currentUser = Auth.auth().currentUser?.uid
let userDocRef = Firestore.firestore().collection("users").document(currentUser!)
WhistleListener.shared.listener = userDocRef.addSnapshotListener { documentSnapshot, error in
guard let document = documentSnapshot else {
print("Error fetching document: \(error!)")
return
}
guard let data = document.data() else {
print("Document data was empty.")
return
}
print("INSIDE LISTENER")
}
}
Below is the code that update's this same document in a different view controller whenever the user updates their profile pic
func uploadProfilePicture(_ image: UIImage) {
guard let uid = currentUser!.UID else { return }
let filePath = "user/\(uid).jpg"
let storageRef = Storage.storage().reference().child(filePath)
guard let imageData = image.jpegData(compressionQuality: 0.75) else { return }
storageRef.putData(imageData) { metadata, error in
if error == nil && metadata != nil {
self.userProfileDoc!.updateData([
"profilePicURL": filePath
]) { err in
if let err = err {
print("Error updating document: \(err)")
} else {
print("Document successfully updated")
}
}
}
}
}
You can use set data with merge true it doesn't wipe any other property only merge to specific one that you declared as like I am only update the name of the user without wiping the age or address
db.collection("User")
.document(id)
.setData(["name":"Zeeshan"],merge: true)
The answer is pretty obvious (and sad at the same time). I was constantly updating the filepath to be the user's UID therefore, it would always be the same and the snapshot wouldn't recognize a difference in the update. It had been some time since I had looked at this code so i forgot this is what it was doing. I was looking past this and simply thinking an update (no matter if it was different from the last or not) would trigger an event. That is not the case! So what I did was append an additional UUID to the user's UID so that it changed.
I am trying to learn mapbox maps iOS but am struggling with inserting an image layer.
The image is however, failing to load, throwing the error:
Error Domain=NSURLErrorDomain Code=-1002 "unsupported URL"
The docs for for this structure are here
Please help!!!
ViewController.swift:
mapView.mapboxMap.onNext(.mapLoaded) { _ in
self.addImageLayer()
}
internal func addImageLayer() {
var sourceId = "planet-source"
let style = mapView.mapboxMap.style
// Create an `ImageSource`. This will manage the image displayed in the `RasterLayer` as well
// as the location of that image on the map.
var imageSource = ImageSource()
// Set the `coordinates` property to an array of longitude, latitude pairs.
imageSource.coordinates = [
[-80.425, 46.437],
[-71.516, 46.437],
[-71.516, 37.936],
[-80.425, 37.936]
]
// Get the file path for the first radar image, then set the `url` for the `ImageSource` to that path.
let path = Bundle.main.path(forResource: "planet", ofType: "svg")!
imageSource.url = path
// Create a `RasterLayer` that will display the images from the `ImageSource`
var imageLayer = Layer(id: "planet-layer")
imageLayer.source = sourceId
do {
try style.addSource(imageSource, id: sourceId)
try style.addLayer(imageLayer)
} catch {
print("Failed to add the source or layer to style. Error: \(error)")
}
}
I am using Create ML to create a .mlmodel file using a Image Classifier project.
Create ML, creates this file for 2 classes.
After dragging the created ML Model file , and adding it to compile sources, below code is able to find the file and create the model.
But the meta data does not show classes, any reason why? How would I know the classes in the mlmodel file?
let error: NSError! = nil
guard let modelURL = Bundle.main.url(forResource: "ObjectDetector", withExtension: "mlmodelc") else {
return
}
do {
let model = try MLModel(contentsOf: modelURL)
let visionModel = try VNCoreMLModel(for: model)
let metaData = model.modelDescription.metadata[.creatorDefinedKey] as! [String:String]
// ERROR: It could not find metaData["classes"]
let allClasses = metaData["classes"]!.components(separatedBy: ",")
let objectRecognition = VNCoreMLRequest(model: visionModel, completionHandler: { (request, error) in
DispatchQueue.main.async(execute: {
// perform all the UI updates on the main queue
if let results = request.results {
self.drawVisionRequestResults(results)
}
})
})
self.requests = [objectRecognition]
} catch let error as NSError {
print("Model loading went wrong: \(error)")
}
Please note I am not getting "Model class has not been generated yet.".
https://stackoverflow.com/questions/462476…
Clicking on ML Model file shows Automatically generated Swift model class.
The class data is part of the mlmodel file but not in the metadata. You can add it to the metadata if you want, but you'll have to write a short Python script for that.
As of iOS 14, there is MLModelDescription.classLabels that also lets you access the list of class names.
Language : Swift 5
iOS: 13.2
macOS: Catalina 10.15.4
Firebase Storage Rules:
service firebase.storage {
match /b/{bucket}/o {
match /{allPaths=**} {
allow read, write: if request.auth!=null;
}
}
}
The code to upload image and save download URL: (Which works fine, because I can see images uploaded to storage and their respective download URLs stored to real-time database.)
let storageRef = Storage.storage().reference()
//Let's upload all workout pictures
let uploadPicsRef =
storageRef.child("WORKOUTDATA/USERS/"+self.UID!).child("WHITEBOARDWORKOUTS")
let uploadNumberRef = uploadPicsRef.child("\(String(describing: workoutNum))")
let workoutPicturesRef = uploadNumberRef.child("WORKOUTPICTURES")
let workoutPicURLRef = workoutRef.child("WORKOUTPICTURESURL")
var count = 0
var picNumber = 0
//workoutPictures list/array contains images selected from iPhone Gallery, using
//UIImagePickerController
for workoutPic in self.workoutPictures
{
let workoutPicData = workoutPic.jpegData(compressionQuality: 1.0)!
count = count + 1
let pictureName = "Picture\(count).jpg"
// Upload the file to the path in pictureRef
let pictureRef = workoutPicturesRef.child("\(pictureName)")
let metaData = StorageMetadata()
metaData.contentType = "image/jpg"
pictureRef.putData(workoutPicData, metadata: metaData) { (metadata, error) in
if error != nil {
print("Error while uploading image")
}
else
{
pictureRef.downloadURL { (url, err) in
picNumber = picNumber + 1
workoutPicURLRef.child("Picture\(picNumber)").setValue(url?.absoluteString)
}
}
}
}
The code to download image:
let myGroup = DispatchGroup()
let workoutPicUrls = snapshot.childSnapshot(forPath: "WORKOUTPICTURESURL")
for url in workoutPicUrls.children
{
myGroup.enter()
let snap = url as! DataSnapshot
let link = snap.value as? String
let storageRef = Storage.storage().reference()
let pictureRef = storageRef.root().child(link!)
DispatchQueue.main.async {
pictureRef.getData(maxSize: 1*2000000*2000000) { (data, err) in
if (err != nil) {
print(err!)
print(err!.localizedDescription)
} else {
let pic = UIImage(data: data!)
workoutPicsArray.append(pic!)
myGroup.leave()
}
}
}
}
Error:
Error Domain=FIRStorageErrorDomain Code=-13010 "Object https:/firebasestorage.googleapis.com/v0/b/trainer-8cb52.appspot.com/o/WORKOUTDATA%2FUSERS%2F1K7WV1alYIeWPAsFC6YMoJKPFSj1%2FWHITEBOARDWORKOUTS%2F5%2FWORKOUTPICTURES%2FPicture1.jpg?alt=media&token=785ab8c7-1e08-4ad3-a542-c9e6313eb547 does not exist." UserInfo={object=https:/firebasestorage.googleapis.com/v0/b/trainer-8cb52.appspot.com/o/WORKOUTDATA%2FUSERS%2F1K7WV1alYIeWPAsFC6YMoJKPFSj1%2FWHITEBOARDWORKOUTS%2F5%2FWORKOUTPICTURES%2FPicture1.jpg?alt=media&token=785ab8c7-1e08-4ad3-a542-c9e6313eb547, ResponseBody={
"error": {
"code": 404,
"message": "Not Found. Could not get object",
"status": "GET_OBJECT"
}
}, bucket=trainer-8cb52.appspot.com, data={length = 115, bytes = 0x7b0a2020 22657272 6f72223a 207b0a20 ... 54220a20 207d0a7d }, data_content_type=application/json; charset=UTF-8, NSLocalizedDescription=Object https:/firebasestorage.googleapis.com/v0/b/trainer-8cb52.appspot.com/o/WORKOUTDATA%2FUSERS%2F1K7WV1alYIeWPAsFC6YMoJKPFSj1%2FWHITEBOARDWORKOUTS%2F5%2FWORKOUTPICTURES%2FPicture1.jpg?alt=media&token=785ab8c7-1e08-4ad3-a542-c9e6313eb547 does not exist., ResponseErrorDomain=com.google.HTTPStatus, ResponseErrorCode=404}
What I have tried so far:
Checked firebase storage rules.
When I paste the path https:/firebasestorage.googleapis.com/v0/b/trainer8cb52.appspot.com/o/WORKOUTDATA%2FUSERS%2F1K7WV1alYIeWPAsFC6YMoJKPFSj1%2FWHITEBOARDWORKOUTS%2F5%2FWORKOUTPICTURES%2FPicture1.jpg?alt=media&token=785ab8c7-1e08-4ad3-a542-c9e6313eb547 in chrome browser window, the expected image opens.
Set the maxSize to a ridiculously high number 1*2000000*2000000.
Thank you!
Is it possible that you are storing the full https URL in the database and are trying to create a reference by adding the full https url as a child to the storage reference?
I think you should try to either store just the path and name in your database or you change your download code to use the https URL.
// Create a reference from an HTTPS URL
// Note that in the URL, characters are URL escaped!
let httpsReference = storage.reference(forURL: "https://firebasestorage.googleapis.com/b/bucket/o/images%20stars.jpg")
httpsReference.getData(maxSize: ...
Also you're running your getData method inside DispatchQueue.main.async. getData has itself a completion handler and might take some time, when you run that inside of DispatchQueue.main.async it will block your code until the download is done. Only put code that update the UI inside DispatchQueue.main.async. In your case as soon as you do something with your workoutPicsArray or the UIImage to update your view.
Have a look here to see if you can figure out how you are actually trying to get the data. It might be helpful to put a print() after each line to see what you are creating and using at what point.
Download Files on iOS
I try to get metadata of files from Firebase Storage, in particular date of create (because I want to compare date of local file and date of Cloud file and change file, if needed). I use getMetadata { (metadata, error) in ...} method, but I don't get result in my completion. Control doesn't go to completion! My code is below
let metadataURL = Storage.storage().reference().child("tutorials/how_to/use_masking/cover.jpg")
metadataURL.getMetadata { (metadata, error) in
if let error = error {
print(error)
} else {
if let data = metadata {
let dict = data.dictionaryRepresentation()
print(dict)
}
}
}
And I want to say now, that image is really in this path, because I get this image with getData(...) method in the next step.