Issue with Custom Object Detector model from GCP Vision - ios

My full tech stack is:
GCP ML Vision.
Exported Model to tflite format (from the same GCP console).
XCode for iOS development
iPhone 11 pro
I am trying to use a Custom Object detector using MLKit and one model trained in GCP AutoML Vision.
I created the model and exported as tflite file, but when trying to do objectDetector processImage:visionImage, I always get the error:
Error Domain=com.google.visionkit.pipeline.error Code=3 "Pipeline failed to fully start:
CalculatorGraph::Run() failed in Run:
Calculator::Open() for node "BoxClassifierCalculator" failed: #vk Unexpected number of dimensions for output index 0:
got 3D, expected either 2D (BxN with B=1) or 4D (BxHxWxN with B=1, W=1, H=1)."
UserInfo={com.google.visionkit.status=<MLKITvk_VNKStatusWrapper: 0x280841270>, NSLocalizedDescription=Pipeline failed to fully start:
CalculatorGraph::Run() failed in Run:
Calculator::Open() for node "BoxClassifierCalculator" failed: #vk Unexpected number of dimensions for output index 0:
got 3D, expected either 2D (BxN with B=1) or 4D (BxHxWxN with B=1, W=1, H=1).}.
I have downloaded the mlkit examples from https://github.com/googlesamples/mlkitand there is something similar in the vision project, (to detect birds) when I try to replace my own tflite file, it breaks in the exact same way as in my own project.
I presume the tflite is created in a very different way as MLVision does.
Any insight? (Sorry if this is so obvious, but I'm pretty new to TensorFlow and MLVision)
Thanks in advance

The issue is exactly as what the error message says: got 3D, expected either 2D (BxN with B=1) or 4D (BxHxWxN with B=1, W=1, H=1). That means your model is not compatible with ML Kit, as its tensor has incorrect dimension. The model compatibility requirements are specified here.

Related

Inference on openvino model returns only scores

My task is to perform inference for face detection using Intel Movidius and Raspberry Pi. The error is that the model only returns "Scores" -> (1, 3000, 2) and not "Boxes".
Steps:
On my local machine, I trained several models(mb1-ssd, mb1-ssd-lite, vgg16-ssd) from the repository https://github.com/qfgaohao/pytorch-ssd and converted them to onnx. Then, using open vino model optimizer from openvinotoolkit = 2020.1, I obtained the '.bin', '.xml' files for each model.
Then, using the obtained files, I performed the infference on the Rasberry Pi and hit the mentioned error.
Note: The inference works using pretrained face detection models from model zoo, the only difference I found looking at the .xml files and my .xml files is that the last layer, "Detection output" is missing. However, when I visualize the .xml file using netron, the conversion seems to be correct.
Link to repo: https://github.com/cocacola0/bsc_thesis
OpenVINO™ 2020.3 release is the last OpenVINO™ version that supports Intel® Movidius™ Neural Compute Stick powered by the Intel® Movidius™ Myriad™ 2.
Use ssd_mobilenet_v2_coco and ssdlite_mobilenet_v2, alternative models that are available in Open Model Zoo. Both models are working well with your code.

How to Use Custom yolov4 weights in iOS app

I following these steps:
Obtained YOLO v4 weights after training on my custom dataset
Converted weights to TensorFlow checkpoints
Converted Tensorflow checkpoint to tflite
Quantized tflite
I was using tflite samples and imported custom tflite to xcode project
But I'm getting a crash:
Failed to create the interpreter with error: Failed to create the interpreter.
ObjectDetection/ViewController.swift:56: Fatal error: Failed to load model
2022-02-28 13:28:22.850305+0530 ObjectDetection[9505:3901451]
ObjectDetection/ViewController.swift:56: Fatal error: Failed to load model
I tested my tflite weights and its working fine
Follow this space for solution https://github.com/ankits16/CVRecorderFinal/blob/main/README.md

Espresso ANERuntimeEngine Program Inference overflow

I have two CoreML models. One works fine, and the other generates this error message:
[espresso] [Espresso::ANERuntimeEngine::__forward_segment 0] evaluate[RealTime]WithModel returned 0; code=5 err=Error Domain=com.apple.appleneuralengine Code=5 "processRequest:qos:qIndex:error:: 0x3: Program Inference overflow" UserInfo={NSLocalizedDescription=processRequest:qos:qIndex:error:: 0x3: Program Inference overflow}
[espresso] [Espresso::overflow_error] /var/containers/Bundle/Application/E0DE5E08-D2C6-48AF-91B2-B42BA7877E7E/xxx demoapp.app/mpii-hg128.mlmodelc/model.espresso.net:0
Both models are very similar, (Conv2D models). There are generated with the same scripts and versions of PyTorch, ONNX, and onnx-coreml. The model that works has 1036 layers, and the model that generates the error has 599 layers. They both use standard layers - Conv2D, BatchNorm, ReLU, MaxPool, and Upsample (no custom layers and no Functional or Numpy stuff). They both use relatively the same number of features per layer. They follow essentially the same structure, except the erroring model skips a maxpool layer at the start (hence the higher output resolution).
They both take a 256x256 color image as input, and output 16 channels at (working) 64x64 and (erroring) 128x128 pixels.
The app does not crash, but gives garbage results for the erroring model.
Both models train, evaluate, etc. fine in their native formats (PyTorch).
I have no idea what a Code=5 "processRequest:qos:qIndex:error:: 0x3: Program Inference overflow" error is, and google searches are not yielding anything productive, as I gather "Espresso" and "ANERuntimeEngine" are both private Apple Libraries.
What is this error message telling me? How can I fix it?
Can I avoid this error message by not running the model on the bionic chip but on the CPU/GPU?
Any help is appreciated, thanks.
That's a LOT of layers!
Espresso is the C++ library that runs the Core ML models. ANERuntimeEngine is used with the Apple Neural Engine chip.
By passing in an MLModelConfiguration with computeUnits set to .cpuAndGPU when you load the Core ML model, you can tell Core ML to not use the Neural Engine.

Unable to found feed input Error while predicting on Re-trained Inception-V3 in Tensorflow

I'm currently trying to make predictions on re-trained Inception-V3 model in TensorFlow.
When I'm trying to run inference on image with
bazel-bin/tensorflow/examples/label_image/label_image \
--graph=/path/output_graph.pb --labels=/path/output_labels.txt \
--output_layer=final_result \
--image=/path/to/test/image
I'm getting an error
E tensorflow/examples/label_image/main.cc:303] Running model failed: Not found: FeedInputs: unable to find feed output Mul
I used transfer learning to fine tune Inception trained on Imagenet dataset, to train on my own 1000+ classes. Training & evaluation processes were ok. I exported graph with tf.train.write_graph() and freeze it with https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/tools/freeze_graph.py
Did anyone faced this problem??
It seems that in the graph you are using, the feed or "input_layer" node has been renamed and is no longer called "Mul". You need to find the name of the node where inputs should be injected into your saved graph, and pass the node name via the --input_layer flag.
The easiest way to find the node name is just to make sure to set it explicitly to something you know when you build the graph in the first place.

OpenCV : Building a simple 3d model

I Have decided to use OpenCV to build a 3d scene by using a series of 2D Images. I found the example code that came with OpenCV [ build3dmodel.cpp Here ].
I just want to run this once and see what kind of outcome this gives. My knowledge with OpenCV is low, I don't want to understand the whole code, I just want to know how to give inputs to this program (the image set) to see the output.
The line command of this code example requires the following parameters:
build3dmodel -i intrinsics_filename.yml [-d detector] [-de
descriptor_extractor] -m model_name.yml
The first file is the camera matrix which you obtain after the calibration process (there is an especific example with it). Detector and descriptor detector must match with valid FeatureDetector and DescriptorExtractor names. Model name is a bit confusing, it looks like part of the yml file name where data will be saved.
First see some tutorial like introduction to OpenCv or OpenCV tutorial. Also, see input and output with OpenCv.

Resources