I'm using Keras' pre-trained model for feature extraction in two images, however they gave the same outcome (array_equal = True). I've tried other model like VGG16 and Resnet50 but the results are the same. Am I writing the code wrong or is it the limitation of pre-trained model? Is there anything I can do to extract different features? Thanks!
import cv2
from keras.applications.inception_v3 import InceptionV3
from keras.applications.inception_v3 import preprocess_input
model = InceptionV3(weights='imagenet', include_top=False)
def get_img_vector(path):
im = cv2.imread(path)
im = cv2.resize(im,(224,224))
img = preprocess_input(np.expand_dims(im.copy(), axis=0))
resnet_feature = model.predict(img)
return np.array(resnet_feature)
arr1 = get_img_vector('image1.png')
arr2 = get_img_vector('image2.png')
np.array_equal(arr1,arr2)
Below are my two images:
I think that the file format png create the image loading problem. Currently cv2.imread a png file and cv2.imshow it result in a black screen, which make two images identical. Saving the file from png to jpg and trying it again.
If you run the code, you should see some warning like this,
WARNING: TensorFlow:Model was constructed with shape (None, 299, 299, 3)
for input Tensor("input_3:0", shape=(None, 299, 299, 3), dtype=float32),
but it was called on an input with incompatible shape (None, 224, 224, 3).
Change your code to
im = cv2.resize(im,(299,299))
Now about the similar features, pre-trained imagenet can classify 1000 classes and the given picture. If you decode then you'll see that both of them will give you the same output. And you'll see even for the top 5 predictions, confidence is very low, and most similar is to the image of a nematode.
[[('n01930112', 'nematode', 0.11086103), ('n03729826', 'matchstick', 0.08173305), ('n03196217', 'digital_clock', 0.034744), ('n03590841', "jack-o'-lantern", 0.017616412), ('n04286575', 'spotlight', 0.016781498)]]
However, if you want to train a model that can differentiate these two images then you can use the pre-trained models for transfer learning with your own dataset.
Related
I am pretty new to CNN. I am planning to build a classifier where you will be feeding two images as input to the classifier. And it should output whether its a "match" or not .
I am not sure where to start and how to feed two images and train the neural networks. It would of great help if you can post a sample code. Please help
Thank You
You first need to take the two images and put them into an array. So if each image is 26x26 the array shape should then be 2x26x26. Now you must put each of these arrays into you training data array, BUT make sure that you reshape your training data array to 26x26x2 before you hit train. You can do this by typing in numpy.array(your_array_.reshape(-1, 26, 26, 2) to your fit function input.
Here is an example:
import numpy as np
image1 = # put your image array here
image2 = # put other image array here
both_images = [image1, image2]
training_data.append(both_images) # Feel free to add as much training data as you would like
same = 0
labels = [same]
model = create_model() # Make a function to create your model and set your model to a variable
model.fit(np.array(training_data).reshape(-1, 26, 26, 2), np.array(labels), batch_size=32)
This is a "Watson Studio" related question. I've done the following Deep-Learning tutorial/experiment assistant, successfully deployed a generated CNN model to WML(WebService). Cool!
Tutorial: Single convolution layer on MNIST data
Experiment Assistant
Next, I'd like to test if the model could identify my image( MNIST ) in deployed environment, and the questions came to my mind.
What kind of input file( maybe pixel image file ) should I prepare for the model input ? How can I kick the scoring endpoint passing my image? ( I saw python code-snippet on the "Implementation" tab, but it's json example and not sure how can I pass the pixel image...)
payload_scoring = {"fields": [array_of_feature_columns], "values": [array_of_values_to_be_scored, another_array_of_values_to_be_scored]}
Any advice/suggestions highly welcomed. Thx in advance.
The model that was trained accepts an input data that is an array of 4 dimensions i.e [<batchsize>, 28, 28, 1], where 28 refers to the height and width of the image in pixels, 1 refers to the number of channels. Currently the WML online deployment and scoring service requires the payload data in the format that matches the input format of the model. So, to predict any image with this model, you must ...
convert the image to an array of [1, 28, 28, 1] dimension. Converting image to an array is explained in next section.
pre-process the image data as required by the model i.e perform (a) normalize the data (b) convert the type to float
pre-processed data must be be specified in json format with appropriate keys. This json doc will be the input payload for the scoring request for the model.
How to convert image to an array?
There are two ways.. (using python code)
a) keras python library has a MNIST dataset that has MNIST images that are converted to 28 x 28 arrays. Using the python code below, we can use this dataset to create the scoring payload.
import numpy as np
from keras.datasets import mnist
(X, y), (X_test, y_test) = mnist.load_data()
score_payload_data = X_test.reshape(X_test.shape[0], X_test.shape[1], X_test.shape[2], 1)
score_payload_data = score_payload_data.astype("float32")/255
score_payload_data = score_payload_data[2].tolist() ## we are choosing the 2nd image in the list to predict
scoring_payload = {'values': [score_payload_data]}
b) If you have an image of size 28 x 28 pixels, we can create the scoring payload using the code below.
img_file_name = "<image file name with full path>"
from scipy import misc
img = misc.imread(img_file_name)
img_to_predict = img.reshape(img.shape[0], img.shape[1], 1)/255
img_to_predict = img_to_predict.astype("float32").tolist()
scoring_payload = {"values": [img_to_predict]}
This question already has an answer here:
Multi-dimensional input layers in Keras
(1 answer)
Closed 5 years ago.
I'm attempting to train a model of 3 layer Dense Neural Network using Keras with a GPU enabled Tensorflow backend.
The dataset I have is 4 million 20x40px images that I placed in directories with the name of the category they belong to.
Because of the large amount of data I can't just load it all into RAM and feed it to my model so I thought using Keras's ImageDataGenerator, specifically the function flow_from_directory() would do the trick. This yields a tuple of (x, y) where x is the numpy array of the image and y is the label of the image.
I expected the model to know to access the numpy array to be given as input for my model so I setup my input shape to be: (None,20,40,3) where None is the batch size, 20 and 40 are size of the image and 3 are the number of channels in the image. This does not work however as when I try to train my model I keep getting the error:
ValueError: Error when checking target: expected dense_3 to have 4 dimensions, but got array with shape (1024, 2)
I know the cause is that it is getting the tuple from flow_from_directoy and I guess I could change the input shape to match, however, I fear that this would render my model useless as I will be using images to make predictions not a pre-categorized tuple. So my question is, how can I get flow_from_directory to feed the image to my model and only use the tuple to validate it's training? Am I misunderstanding something here?
For reference, here is my code:
from keras.models import Model
from keras.layers import *
from keras.preprocessing.image import ImageDataGenerator
from keras.callbacks import TensorBoard
# Prepare the Image Data Generator.
train_datagen = ImageDataGenerator()
test_datagen = ImageDataGenerator()
train_generator = train_datagen.flow_from_directory(
'/path/to/train_data/',
target_size=(20, 40),
batch_size=1024,
)
test_generator = test_datagen.flow_from_directory(
'/path/to/test_data/',
target_size=(20, 40),
batch_size=1024,
)
# Define input tensor.
input_t = Input(shape=(20,40,3))
# Now create the layers and pass the input tensor to it.
hidden_1 = Dense(units=32, activation='relu')(input_t)
hidden_2 = Dense(units=16)(hidden_1)
prediction = Dense(units=1)(hidden_2)
# Now put it all together and create the model.
model = Model(inputs=input_t, outputs=prediction)
model.compile(optimizer='rmsprop', loss='categorical_crossentropy', metrics=['accuracy'])
# Prepare Tensorboard callback and start training.
tensorboard = TensorBoard(log_dir='./graph', histogram_freq=0, write_graph=True, write_images=True)
print(test_generator)
model.fit_generator(
train_generator,
steps_per_epoch=2000,
epochs=100,
validation_data=test_generator,
validation_steps=800,
callbacks=[tensorboard]
)
# Save trained model.
model.save('trained_model.h5')
Your input shape is wrong for Dense layers.
Dense layers expect inputs in the shape (None,length).
You'll either need to reshape your inputs so that they become vectors:
imageBatch=imageBatch.reshape((imageBatch.shape[0],20*40*3))
Or use convolutional layers, that expect that type of input shape (None,nRows,nCols,nChannels) like in tensorflow.
I trained an image classifier with Keras and I ended up saving the model with the code
model.save('model1.h5')
Now when I'm actually trying to predict another image using this model, I'm using the code
from keras.models import load_model
from keras.preprocessing import image
import numpy as np
# dimensions of our images
img_width, img_height = 231, 172
# load the model we saved
model = load_model('model1.h5')
model.compile(loss='binary_crossentropy',
optimizer='rmsprop',
metrics=['accuracy'])
# predicting images
img = image.load_img('a.png', target_size=(img_width, img_height))
x = image.img_to_array(img)
x = np.expand_dims(x, axis=0)
images = np.vstack([x])
classes = model.predict_classes(images, batch_size=10)
print(classes)
and I get an error saying
ValueError: Error when checking : expected conv2d_1_input to have shape (None, 231, 172, 1) but got array with shape (1, 231, 172, 3)
The images that I use were all in GrayScale mode so I understand that I need to change the 3 to a 1 but I'm not sure how with this code?
Also, when I trained the images, I set
train_datagen = ImageDataGenerator(rescale=1. / 255)
test_datagen = ImageDataGenerator(rescale=1. / 255)
but I'm not exactly sure where I would put the code to rescale it, if it's even needed
Also, I set
batch_size = 16
and I see in the code to predict I put
classes = model.predict_classes(images, batch_size=10)
Should I change that batch_size to 16 as well? Or just leave it at 10?
Thanks again guys!
Firstly, to load an image in grayscale mode just add grayscale=True like this:
img = image.load_img('a.png', target_size=(img_width, img_height), grayscale=True)
Secondly, after having converted img to a numpy array you can rescale it by simply dividing x by 255..
Thirdly, batch_size parameter tells how big are batches in which computation is done. Since you make prediction for a single image it does not really matter what batch_size you define. Here is what Keras documentation says about batch_size for inference:
For inference (evaluate/predict), it is recommended to pick a batch size that is as large as you can afford without going out of memory (since larger batches will usually result in faster evaluating/prediction).
Finally, there is no need to compile the model when you load it for making inference. You can safely remove model.compile(...).
Maybe the error is here.
img = image.load_img('a.png', target_size=(img_width, img_height))
If you have target_size as (64, 64) in the trained model.
The target size during prediction should be (64, 64) and not (img_width, img_height)
I'm trying to create an example using the Keras built in the latest version of TensorFlow from Google. This example should be able to classify a classic image of an elephant. The code looks like this:
# Import a few libraries for use later
from PIL import Image as IMG
from tensorflow.contrib.keras.python.keras.preprocessing import image
from tensorflow.contrib.keras.python.keras.applications.inception_v3 import InceptionV3
from tensorflow.contrib.keras.python.keras.applications.inception_v3 import preprocess_input, decode_predictions
# Get a copy of the Inception model
print('Loading Inception V3...\n')
model = InceptionV3(weights='imagenet', include_top=True)
print ('Inception V3 loaded\n')
# Read the elephant JPG
elephant_img = IMG.open('elephant.jpg')
# Convert the elephant to an array
elephant = image.img_to_array(elephant_img)
elephant = preprocess_input(elephant)
elephant_preds = model.predict(elephant)
print ('Predictions: ', decode_predictions(elephant_preds))
Unfortunately I'm getting an error when trying to evaluate the model with model.predict:
ValueError: Error when checking : expected input_1 to have 4 dimensions, but got array with shape (299, 299, 3)
This code is taken from and based on the excellent example coremltools-keras-inception and will be expanded more when it is figured out.
The reason why this error occured is that model always expects the batch of examples - not a single example. This diverge from a common understanding of models as mathematical functions of their inputs. The reasons why model expects batches are:
Models are computationaly designed to work faster on batches in order to speed up training.
There are algorithms which takes into account the batch nature of input (e.g. Batch Normalization or GAN training tricks).
So four dimensions comes from a first dimension which is a sample / batch dimension and then - the next 3 dimensions are image dims.
Actually I found the answer. Even though the documentation states that if the top layer is included the shape of the input vector is still set to take a batch of images. Thus we need to add this before the code line for the prediction:
elephant = numpy.expand_dims(elephant, axis=0)
Then the tensor is in the right shape and everything works correctly. I am still uncertain why the documentation states that the input vector should be (3x299x299) or (299x299x3) when it clearly wants 4 dimensions.
Be careful!