I have build my own dataset for digit classification and it worked well with convolutional network model developed by lisa lab (Here). I wanted to visualize the weights and i wanted to do it through keras.
Keras documentation tries to load mnist data like this:
(X_train, y_train), (X_test, y_test) = mnist.load_data()
But i want my pickled dataset to load instead of mnist default data. Where does mnist module for keras load it's dataset ? And, how can i pass my own dataset instead of that to use mnist module from keras ?
Thanks in advance.
You can get a lot of info about this method by reading the source: https://github.com/fchollet/keras/blob/master/keras/datasets/mnist.py
In this case, the dataset is a pickle file loaded from an Amazon S3 bucket.
You could write a copy of this function and use it yourself to load up a different pickled dataset.
Related
I have one image. I really want to transform/decode it to become a tensor. Why? Because I want to feed this tensor into my neural network written in keras. The question is, how do I transform this image into a tensor with values, that doesn't give me an error when feeding the neural net ?
So suppose there is a PATH, and this has to be changed into a TENSOR, which can be feed into the keras neural network.
Thank you, very much.
You can use Keras' ImageDataGenerator(), which generates batches of tensor image data. You can then call flow_from_directory() on your ImageDataGenerator() object, which takes a path to the directory where your images are, and generates batches of data from the images themselves. These two videos demonstrate this process with an example:
Image preparation for CNN training with Keras
Create and train a CNN with Keras
In all the tutorials i've seen for tensorflow, they've used the MNIST dataset, i've understood the modelling but how do i load this dataset into tensorflow?
https://www.nist.gov/itl/iad/image-group/emnist-dataset
The EMNIST dataset uses the same binary format as the original MNIST dataset. Therefore you can take the input pipeline code from any tutorial that uses the original MNIST dataset, and point it at the set of files you get from downloading the EMNIST dataset to train on that dataset.
You can load the EMNIST data file in Matlab format with scipy.io.loadmat(). The array has to be rotated after loading. There is a Jupyter Notebook on GitHub which does EMNIST Digits classification.
You could use the EMNIST package that can be found here: https://pypi.org/project/emnist/
To load the dataset you first need to decide which of the six different datasets you would like to work with. Details in this paper: https://arxiv.org/pdf/1702.05373v1.pdf
Let's say we want to use the byclass dataset:
from emnist import extract_training_samples, extract_test_samples
x_train, y_train = extract_training_samples('byclass')
x_test, y_test = extract_test_samples('byclass')
I would like to use an existing VGG16 model trained on imagenet and fine-tune it with a custom dataset for some other classes required by me.
Where can I find the caffemodel, train_val.prototxt and solver.prototxt for the same ?
To fine-tune it with the custom dataset, is the procedure same as
Fine Tuning of GoogLeNet Model
A guide to convert_imageset.cpp
?
However, I want to use the newly-trained weights of the VGG16 model to train a faster RCNN (py-faster-rcnn) https://github.com/rbgirshick/py-faster-rcnn on a custom dataset.
For training faster RCNN on a custom dataset, I was planning on following the steps given here http://sgsai.blogspot.com/2016/02/training-faster-r-cnn-on-custom-dataset.html
Will the caffemodel generated from the VGG16 fine-tuning done earlier work here or some tweaks need to be done ?
I am learning to create a learning model using TensorFlow.
I have successfully run the MNIST tutorial, now would like to test the model with my own images. They are same-size image (224x224) and classified into folders.
Now I would like to use those images as input for my model as in the MNIST example. I tried to open the MNIST data-set but it's unreadable. I guess it has been converted into some binary types. Through the example, I think the MNIST dataset somehow has a structure like this:
mnist
test
images
labels
train
images
labels
How can I make a dataset look like the MNIST data from my own images files?
Thank you very much!
MNIST is not stored in image format. From the mnist web-site (http://yann.lecun.com/exdb/mnist/) you could see that it has specific format which is already close to the tensor or numpy array, which could be used in tensorflow with minimal adjustments. It is a kind of a matrix with numbers.
What you need to work with usual images (.jpg for instance) is to use any python lib for image processing to convert into the np.array. For example PIL will work, like here:
PIL and numpy
Another option is to use a built-in functions from tensorflow to convert your images straight to tensors supported by tensofrlow, check this out:
https://www.tensorflow.org/versions/r0.9/api_docs/python/image.html
I used the same data to train a svm model,the xml model file trained by opencv is only 802 kb,but the model file trained by libsvm is bigger than 17MB; I don't know the difference between those two model file;