CoreMLTools converted Keras Model fails at VNCoreMLTransform - ios

as I'm learning Apple's Vision and CoreML framework but got stuck on how to use my own re-trained models. I tried training a VG16 model with Keras based on this tutorial. Everything looks OK except for some Keras version warnings. Then I tried converting the resulted model with CoreMLTools with the following code
coremlModel = coremltools.converters.keras.convert(
kmodel,
input_names = 'image',
image_input_names = 'image',
output_names = 'classLabelProbs',
class_labels = ['cats', 'dogs'],
)
During the conversion it gave me some version compatible warnings but otherwise it was successful:
WARNING:root:Keras version 2.0.6 detected. Last version known to be fully compatible of Keras is 2.0.4 .
WARNING:root:TensorFlow version 1.2.1 detected. Last version known to be fully compatible is 1.1.1 .
So I loaded this model into Apple's Vision+ML example code, but every time I tried to classify an image it failed with errors
Vision+ML Example[2090:2012481] Error: The VNCoreMLTransform request failed
Vision+ML Example[2090:2012481] Didn't get VNClassificationObservations
Error Domain=com.apple.vis Code=3 "The VNCoreMLTransform request failed" UserInfo={NSLocalizedDescription=The VNCoreMLTransform request failed, NSUnderlyingError=0x1c025d130 {Error Domain=com.apple.CoreML Code=0 "Dimensions of layer 'classLabelProbs' is not the same size as the number of class labels." UserInfo={NSLocalizedDescription=Dimensions of layer 'classLabelProbs' is not the same size as the number of class labels.}}}
I was guessing this is because the pre-trained VGG16 model has already had 1000 categories, so I tried 1000 categories and 1000 + 2 (cats and dogs) categories, but still got the same problem.
Did I miss anything? I'd greatly appreciate any clue and help.

Related

TensorflowJS TFJS error: The dtype of dict

Had tried to run https://glitch.com/~tar-understood-exoplanet
and the model would fail to load and I wouldn't be able to use enable the webcam.
Anyone had the same issue?
While the program is running, in the console I get the following:
tfjs:2 Uncaught (in promise) Error: The dtype of dict['image_tensor'] provided in model.execute(dict) must be int32, but was float32
at Object.b [as assert] (tfjs:2)
at tfjs:2
at Array.forEach (<anonymous>)
at t.checkInputShapeAndType (tfjs:2)
at t.<anonymous> (tfjs:2)
at tfjs:2
at Object.next (tfjs:2)
at tfjs:2
at new Promise (<anonymous>)
at Zv (tfjs:2)
I have a Macbook Pro and some other people on their Windows also had some issues running the model. We also tried it on different browsers, Safari and Chrome.
SUCCESS! after switching to coco-ssd 2.0.2:
I added the version 2.0.2 in line 62 as follows:
<script src="https://cdn.jsdelivr.net/npm/#tensorflow-models/coco-ssd#2.0.2"></script>
This is caused by the warmup run of coco-ssd that uses tf.zeros tensor. The default dtype for tf.zeros is 'float' in the recent release of TFJS.
I have put out a new version with fixes. It should work if you use the latest version of coco-ssd (2.0.2) in the glitch example (index.html) as following.
<!-- Load the coco-ssd model to use to recognize things in images -->
<script src="https://cdn.jsdelivr.net/npm/#tensorflow-models/coco-ssd#2.0.2"></script>
Same error here, just occured since Friday night (04/03/2020)
TFModel works well in past few weeks.
I got the same error.
My Scenerio:
I trained a pre-trained model from tensorflow model zoo using transfer learning using tensorflow api as saved model (model.pb file) and converted it into tfjs format (model.json and shared .bin files).
When I tried running this model.json on the javascript(web), it gives the below error:
Uncaught (in promise) Error: The dtype of dict['input_tensor'] provided in model.execute(dict) must be int32, but was float32
When I tried someone else's working converted model (model.json and shared .bin files) on my javascript(web), it worked.
Conclusion:
There is something wrong with my converted model. I converted it using tensorflowjs_converter. My original model in (model.pb) works accurately in python too.
I'm still trying out to convert my model.pb file with different tensorflowjs_converters as it seems to be the converters versioning issue.

The LSDDetector and some related function couldn't work in Opencv 4.1.0

I want to use the LSDDetector to detect the line features in the image and use the BinaryDescriptorMatcher to describe them so that I can finish the line feature matching. It works properly in opencv version 4.0.x, but the opencv version in my pc updated to 4.1.0 now. Then the code cannot work.
The error notice is as follows:
OpenCV(4.1.0) /tmp/opencv-20190413-55052-1xilkvg/opencv-4.1.0/modules/imgproc/src/lsd.cpp:143: error: (-213:The function/feature is not implemented) Implementation has been removed due original code license issues in function 'LineSegmentDetectorImpl'
def lsdMatching(img1, img2):
img1_gray = cv2.cvtColor(img1,cv2.COLOR_RGB2GRAY)
img2_gray = cv2.cvtColor(img2,cv2.COLOR_RGB2GRAY)
detector = cv2.line_descriptor.LSDDetector_createLSDDetector()
keylines = detector.detect(img1_gray,2,2)
LSD detector had been removed completely from the source tree since 4.1.0 release (see opencv/opencv#3ba49cc). Switch two previous versions such as 3.4.5 works for me.

The OpenCV DNN face detection module can not work with Caffe C++ library

I download the caffe source code from Github and compile it as C++ static library, and I test the OpenCV face detection Caffe model with the static library, but the lib report below error:
[libprotobuf ERROR D:\ThirdPartyLibrary\protobuf\src\google\protobuf\text_format.cc:296] Error parsing text-format caffe.NetParameter: 984:14: Message type "caffe.LayerParameter" has no field named "norm_param".
F0328 02:08:05.225075 24332 upgrade_proto.cpp:88] Check failed: ReadProtoFromTextFile(param_file, param) Failed to parse NetParameter file: D:/DATA/PreTrainedModel/cv_facedet/deploy.prototxt
is it means that norm_param field is implement only by OpenCV, not a standard Caffe field?
norm_param is an argument of Normalization layer from SSD Caffe framework: https://github.com/weiliu89/caffe/blob/8a65ae316b34e7c8fdefa6e18bf08a23b78caa0e/src/caffe/proto/caffe.proto#L523
Origin repo: https://github.com/weiliu89/caffe/tree/ssd
There is no SSD object detection networks in the origin Caffe.
Probably, it has different name in Caffe: normalize_bbox_param. See this discussion.

loading Tensorflow model in Opencv 3.4.1 failed

i'm using opencv 3.4.1 DNN in java in order to load a "LeNet" model trained using keras and tensorflow in python. The model is saved as a tensorflow frozen model ".pb" where i'm using the following line of code to load the it:
Dnn cvDnn = new org.opencv.dnn.Dnn();
Net net = cvDnn.readNetFromTensorflow("C:\\Users\\kr\\Desktop\\Plate_Recognition_frozen.pb");
where the Error says:
OpenCV(3.4.1) Error: Unspecified error (Input layer not found: convolution2d_1_b_1) in cv::dnn::experimental_dnn_v4::`anonymous-namespace'::TFImporter::connect, file C:\build\master_winpack-bindings-win64-vc14-static\opencv\modules\dnn\src\tensorflow\tf_importer.cpp, line 553
Exception in thread "main" CvException [org.opencv.core.CvException: cv::Exception: OpenCV(3.4.1) C:\build\master_winpack-bindings-win64-vc14-static\opencv\modules\dnn\src\tensorflow\tf_importer.cpp:553: error: (-2) Input layer not found: convolution2d_1_b_1 in function cv::dnn::experimental_dnn_v4::`anonymous-namespace'::TFImporter::connect
]
at org.opencv.dnn.Dnn.readNetFromTensorflow_1(Native Method)
at org.opencv.dnn.Dnn.readNetFromTensorflow(Dnn.java:163)
at opencv.Main.main(Main.java:44)
Any help would be appreciated, thanks in advance.

H2O POJO causing javac java.lang.IllegalArgumentException

I have a distributed random forest POJO model using the default model setting except for:
ntrees = 150
max_depth = 50
min_rows = 5
Here are the full settings:
buildModel 'drf', {"model_id":"drf-335270ee-8970-4855-b521-c4fb4ca184f5","training_frame":"frame_0.750","validation_frame":"frame_0.250","nfolds":0,"response_column":"DENIAL","ignored_columns":["tx_match_date"],"ignore_const_cols":true,"ntrees":"150","max_depth":"50","min_rows":"5","nbins":20,"seed":-1,"mtries":-1,"sample_rate":0.6320000290870667,"score_each_iteration":true,"score_tree_interval":0,"balance_classes":false,"nbins_top_level":1024,"nbins_cats":1024,"r2_stopping":1.7976931348623157e+308,"stopping_rounds":0,"stopping_metric":"AUTO","stopping_tolerance":0.001,"max_runtime_secs":0,"checkpoint":"","col_sample_rate_per_tree":1,"min_split_improvement":0.00001,"histogram_type":"AUTO","categorical_encoding":"AUTO","build_tree_one_node":false,"sample_rate_per_class":[],"binomial_double_trees":true,"col_sample_rate_change_per_level":1,"calibrate_model":false}
When I try to compile the pojo with:
$javac -cp "h2o-genmodel.jar" -J-Xmx2g -J-XX:MaxPermSize=128m drf_335270ee_8970_4855_b521_c4fb4ca184f5.java
I get the following error.
An exception has occurred in the compiler (1.8.0_131). Please file a bug against the Java compiler via the Java bug reporting page (http://bugreport.java.com) after checking the Bug Database (http://bugs.java.com) for duplicates. Include your program and the following diagnostic in your report. Thank you.
java.lang.IllegalArgumentException
at java.nio.ByteBuffer.allocate(ByteBuffer.java:334)
at com.sun.tools.javac.util.BaseFileManager$ByteBufferCache.get(BaseFileManager.java:325)
at com.sun.tools.javac.util.BaseFileManager.makeByteBuffer(BaseFileManager.java:294)
at com.sun.tools.javac.file.RegularFileObject.getCharContent(RegularFileObject.java:114)
at com.sun.tools.javac.file.RegularFileObject.getCharContent(RegularFileObject.java:53)
at com.sun.tools.javac.main.JavaCompiler.readSource(JavaCompiler.java:602)
at com.sun.tools.javac.main.JavaCompiler.parse(JavaCompiler.java:665)
at com.sun.tools.javac.main.JavaCompiler.parseFiles(JavaCompiler.java:950)
at com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:857)
at com.sun.tools.javac.main.Main.compile(Main.java:523)
at com.sun.tools.javac.main.Main.compile(Main.java:381)
at com.sun.tools.javac.main.Main.compile(Main.java:370)
at com.sun.tools.javac.main.Main.compile(Main.java:361)
at com.sun.tools.javac.Main.compile(Main.java:56)
at com.sun.tools.javac.Main.main(Main.java:42)
I don't get this error when replacing the DRF model with a deep learning pojo that I have also downloaded from h2o's Flow UI, so I'm thinking it is likely related to the drf_335270ee_8970_4855_b521_c4fb4ca184f5.java file (note that the POJO was too big to preview in H2O's Flow UI). What could be going on here?
Thanks
Instead of trying to compile an H2O random forest POJO, you can download and use a MOJO instead in almost exactly the same way without needing the compile step.
See:
http://docs.h2o.ai/h2o/latest-stable/h2o-genmodel/javadoc/index.html

Resources