I'm trying to use TensorRT to speed up my pre-trained model at inference time. Any object detection pre-trained model would work actually, doesn't have to be Faster R-CNN.
I saw that many layers are not supported by TensorRT yet, what would the procedure be in that case? I read that you can implement those layers yourself, but what does that mean? Would I need to retrain my model after "implementing those layers"?
Thanks
Related
Hi I have trained object detection model using tensorflow 1.14 object detection API, my model is performing well. However, I want to reduce/optimize parameters of model to make it lighter. How can I use pruning on trained model?
Did you check the Pruning guide on the Tensorflow website ?
It has concrete examples on how to prune a model and benchmark the size and performance improvements.
I am trying to use a Pytorch 3D UNet for inference (from here: https://github.com/wolny/pytorch-3dunet) which receives images of size (96, 96, 96). I would like to use it on CPU instances, but I am getting very high memory usages (~18 GB). After researching on the subject I found out that this was due to the way convolutions are implemented on CPU (see https://discuss.pytorch.org/t/pytorch-high-memory-demand/2798/5). I thus have the following questions:
Is there a way to use a more memory-efficient implementation of the convolution in Pytorch?
How can I optimize my model for CPU inference? I saw that some tools like AWS Neo, Intel OpenVINO, etc. exist; could they solve my problem?
Does Tensorflow have a similar problem for using convolutions on CPU?
Any other tip, link on how to deploy such models in an efficient way is welcome!
Thanks!
You could benchmark your model's performance with DNN-Bench and choose the best inference engine for your application and your hardware. You might need to convert your model to ONNX first.
I have a task i need to solve using computer vision, but I can't figure out what kind of feature would be ideal to extract. I could possibly train a CNN network, which could do the classification, is it then somehow possible to learn what kind of features it uses, and what is used to discriminate class A from class b?
And then do the same just using openCV or something similar?
Yes, you can do that. Actually, that is exactly what a CNN does, they learn features that might be the best for your particular data and task, and then you can use a trained network as a feature extractor.
This is pretty common use of a CNN, and OpenCV has some limited support for neural networks in the DNN module.
Is it possible to feed image features, say SIFT features, to a convolutional neural network model in Tensorflow? I am trying a tensorflow implementation of this project in which a grayscale image is coloured. Will image features be a better choice than feeding the images as is to the model?
PS. I am a novice to machine learning and is not familiar with creating neural n/w models
You can feed tensorflow neural net almost anything.
If you have extra features for each pixel, then instead of using one channel (intensity) you would use multiple channels.
If you have extra features, which are about whole image, you can make separate input a merge features at some upper layer.
As for the better performance, you should try both approaches.
General intuition is that, extra features help if you don't have many samples and their effect is diminishing if you have many samples and network can learn features by itself.
Also one more point: If you are novice, I strongly recommend using higher level framework like keras.io (which is layer over tensorflow) instead of tensorflow.
Does it make any sense to perform feature extraction on images using, e.g., OpenCV, then use Caffe for classification of those features?
I am asking this as opposed to the traditional way of passing the images directly to Caffe, and letting Caffe do the extraction and classification procedures.
Yes, it does make sense, but it may not be the first thing you want to try:
If you have already extracted hand-crafted features that are suitable for your domain, there is a good chance you'll get satisfactory results by using an easier-to-use machine learning tool (e.g. libsvm).
Caffe can be used in many different ways with your features. If they are low-level features (e.g. Histogram of Gradients), then several convolutional layers may be able to extract the appropriate mid-level features for your problem. You may also use caffe as an alternative non-linear classifier (instead of SVM). You have the freedom to try (too) many things, but my advice is to first try a machine learning method with a smaller meta-parameter space, especially if you're new to neural nets and caffe.
Caffe is a tool for training and evaluating deep neural networks. It is quite a versatile tool allowing for both deep convolutional nets as well as other architectures.
Of course it can be used to process pre-computed image features.