Related
I'm trying to run a combination of CNN (Convolutional Neural Network) and LSTM (Long Short Term Memory), and didn't find the right reshaping for the data the fits for both. I thought LSTM needs [samples, timesteps, features], but it doesn't work here as input.
I'm receiving an error:
ValueError: Negative dimension size caused by subtracting 3 from 1 for '{{node conv1d/conv1d/Conv2D}} = Conv2D[T=DT_FLOAT, data_format="NHWC", dilations=[1, 1, 1, 1], explicit_paddings=[], padding="VALID", strides=[1, 1, 1, 1], use_cudnn_on_gpu=true](conv1d/conv1d/Reshape, conv1d/conv1d/ExpandDims_1)' with input shapes: [?,1,1,24], [1,3,24,64].
The data is taken from:
https://www.kaggle.com/berkeleyearth/climate-change-earth-surface-temperature-data
It has the shape of:
date
LandAverageTemperature
1750-01-01
1.2
etc..
The full code is:
import tensorflow as tf
def preprocessing(data,n_in=1, n_out=1):
from sklearn.model_selection import train_test_split
def series_to_supervised(df, n_in=1, n_out=1,
dropnan=True):
cols = list()
# input sequence (t-n, ... t-1)
for i in range(n_in, 0, -1):
cols.append(df.shift(i))
# forecast sequence (t, t+1, ... t+n)
for i in range(0, n_out):
cols.append(df.shift(-i))
# put it all together
agg = pd.concat(cols, axis=1)
# drop rows with NaN values
if dropnan:
agg.dropna(inplace=True)
return agg.values
land_temp = pd.DataFrame(data['LandAverageTemperature'].values)
ma_vals = data['LandAverageTemperature'].expanding(min_periods=12).mean()
ma_vals_inter = ma_vals.interpolate(limit_direction='both')
df = series_to_supervised(ma_vals_inter, n_in=n_in, n_out=n_out, dropnan=True)
df = pd.DataFrame(df)
X, y = df.iloc[:, :-n_out], df.iloc[:, -n_out:]
percent = 0.8
if n_out == 1:
y = y.iloc[:, 0]
lim = int(percent * X.shape[0])
X_train, X_test, y_train, y_test = X[:lim], X[lim:], y[:lim], y[ lim:] # train_test_split( X, y, test_size=0.4, random_state=0)
return X_train, X_test, y_train, y_test
def lstm_cnn2(X_train,y_train,config,n_in,n_out=1,batch_size=1,epochs=1000,verbose=0,n_features=1):
input_y = y_train.values.reshape(y_train.shape[0], 1)
n_timesteps, n_features, n_outputs = X_train.shape[0],
X_train.shape[1], input_y.shape[1]
# reshape output into [samples, timesteps, features]
input_x = X_train.values.reshape((X_train.shape[0], 1,
n_features))
# define model
model = Sequential()
model.add(Conv1D(64, 3, activation='relu', input_shape=(n_timesteps,1,n_features)))
model.add(Conv1D(64, 3, activation='relu'))
model.add(MaxPooling1D())
model.add(Flatten())
model.add(RepeatVector(n_outputs))
model.add(LSTM(200, activation='relu',
return_sequences=True))
model.add(TimeDistributed(Dense(100,
activation='relu')))
model.add(TimeDistributed(Dense(1)))
model.compile(loss='mse', optimizer='adam')
# fit network
model.fit(input_x, input_y, epochs=epochs,
batch_size=batch_size, verbose=verbose)
return model
if __name__ == '__main__':
file_location='./GlobalTemperatures.csv'
data = pd.read_csv(file_location)
data['dt'] = pd.to_datetime(data['dt'])
n_out = 1
n_in = 12 * 2
X_train, X_test, y_train, y_test =
preprocessing(data,n_in,n_out)
config = 128, 64, 32, 3 * 48, 48, 24, 100, 20 # lstm
model configuration
verbose, epochs, batch_size = 0, 1, 16
model_lstm = lstm_cnn2(X_train, y_train, config,
n_in,batch_size=batch_size)
X and Y is of shape (89362, 5) and (89362,) repectively.
x_train, x_test, y_train, y_test = train_test_split(X, Y,
test_size = 0.3,
random_state = 1)
x_train.shape, y_train.shape = ((62553, 5), (62553,))
x_test.shape, y_test.shape = ((26809, 5), (26809,))
Reshaped the vectors to:
torch.Size([1, 62553, 5]), torch.Size([1, 62553])
torch.Size([1, 26809, 5]), torch.Size([1, 26809])
The model is defined as
n_steps = 62553
n_features = 5
model = Sequential()
model.add(Conv1D(filters=64, kernel_size=2, activation='relu', input_shape=(n_steps, n_features)))
model.add(MaxPooling1D(pool_size=2))
model.add(Flatten())
model.add(Dense(50, activation='relu'))
model.add(Dense(62553))
model.compile(optimizer='adam', loss='mse')
model.fit(x_train, y_train, epochs=10, verbose=0)
While predicting with x_test, it throws value error
yhat = model.predict(x_test, verbose=0)
print(yhat)
ValueError: Error when checking input: expected conv1d_4_input to have shape (62553, 5) but got array with shape torch.Size([26809, 5])
This is happening because you are specifying a fixed size here:
model.add(Conv1D(filters=64, kernel_size=2, activation='relu', input_shape=(n_steps, n_features)))
Once you pass something else to the model, the model is still expecting that fixed size with dimensions:
n_steps = 62553
n_features = 5
Removing the input_shape parameter should correct this issue:
model.add(Conv1D(filters=64, kernel_size=2, activation='relu'))
I hope that this helps you.
I trained my model of CNN net on images with good val_acc=0.97 and using model.fit_generator.
Here is the output of last epoch, proofing high validation accuracy:
199/200 [============================>.] - ETA: 1s - loss: 0.1563 - acc: 0.9563
200/200 [==============================] - 306s 2s/step - loss: 0.1556 - acc: 0.9565 - val_loss: 0.1402 - val_acc: 0.9691
Epoch 00005: val_acc improved from 0.96701 to 0.96907, saving model to /home/sergorl/cars/color_weights.hdf5
But when I use the same validation data set, which I use during training, but test only one image and for every image in my validation set I always get the wrong predicted label and the predicted probabilities looks like a uniform distribution.
I read this links:
Wrong prediction on images
Why is Keras training well but returning wrong predictions?
Keras Val_acc is good but prediction for same data is poor
But I don't find the solution!
from keras.models import Sequential,Model,load_model
from keras.optimizers import SGD
from keras.layers import BatchNormalization, Lambda, Input, Dense, Convolution2D, MaxPooling2D, AveragePooling2D, ZeroPadding2D, Dropout, Flatten, merge, Reshape, Activation
from keras.layers.merge import Concatenate
from keras.preprocessing.image import ImageDataGenerator
from keras.callbacks import ModelCheckpoint
import os
import cv2
import numpy as np
class CarColorNet:
def __init__(self, numClasses=6, imageWidth=256, imageHeight=256):
self.classes = {}
self.numClasses = numClasses
self.imageWidth = imageWidth
self.imageHeight = imageHeight
input_image = Input(shape=(self.imageWidth, self.imageHeight, 3))
# ------------------------------------ TOP BRANCH ------------------------------------
# first top convolution layer
top_conv1 = Convolution2D(filters=48, kernel_size=(11, 11), strides=(4, 4),
input_shape=(self.imageWidth, self.imageHeight, 3), activation='relu')(input_image)
top_conv1 = BatchNormalization()(top_conv1)
top_conv1 = MaxPooling2D(pool_size=(3, 3), strides=(2, 2))(top_conv1)
# second top convolution layer
# split feature map by half
top_top_conv2 = Lambda(lambda x: x[:, :, :, :24])(top_conv1)
top_bot_conv2 = Lambda(lambda x: x[:, :, :, 24:])(top_conv1)
top_top_conv2 = Convolution2D(filters=64, kernel_size=(3, 3), strides=(1, 1), activation='relu',
padding='same')(top_top_conv2)
top_top_conv2 = BatchNormalization()(top_top_conv2)
top_top_conv2 = MaxPooling2D(pool_size=(3, 3), strides=(2, 2))(top_top_conv2)
top_bot_conv2 = Convolution2D(filters=64, kernel_size=(3, 3), strides=(1, 1), activation='relu',
padding='same')(top_bot_conv2)
top_bot_conv2 = BatchNormalization()(top_bot_conv2)
top_bot_conv2 = MaxPooling2D(pool_size=(3, 3), strides=(2, 2))(top_bot_conv2)
# third top convolution layer
# concat 2 feature map
top_conv3 = Concatenate()([top_top_conv2, top_bot_conv2])
top_conv3 = Convolution2D(filters=192, kernel_size=(3, 3), strides=(1, 1), activation='relu',
padding='same')(top_conv3)
# fourth top convolution layer
# split feature map by half
top_top_conv4 = Lambda(lambda x: x[:, :, :, :96])(top_conv3)
top_bot_conv4 = Lambda(lambda x: x[:, :, :, 96:])(top_conv3)
top_top_conv4 = Convolution2D(filters=96, kernel_size=(3, 3), strides=(1, 1), activation='relu',
padding='same')(top_top_conv4)
top_bot_conv4 = Convolution2D(filters=96, kernel_size=(3, 3), strides=(1, 1), activation='relu',
padding='same')(top_bot_conv4)
# fifth top convolution layer
top_top_conv5 = Convolution2D(filters=64, kernel_size=(3, 3), strides=(1, 1), activation='relu',
padding='same')(top_top_conv4)
top_top_conv5 = MaxPooling2D(pool_size=(3, 3), strides=(2, 2))(top_top_conv5)
top_bot_conv5 = Convolution2D(filters=64, kernel_size=(3, 3), strides=(1, 1), activation='relu',
padding='same')(top_bot_conv4)
top_bot_conv5 = MaxPooling2D(pool_size=(3, 3), strides=(2, 2))(top_bot_conv5)
# ------------------------------------ TOP BOTTOM ------------------------------------
# first bottom convolution layer
bottom_conv1 = Convolution2D(filters=48, kernel_size=(11, 11), strides=(4, 4),
input_shape=(224, 224, 3), activation='relu')(input_image)
bottom_conv1 = BatchNormalization()(bottom_conv1)
bottom_conv1 = MaxPooling2D(pool_size=(3, 3), strides=(2, 2))(bottom_conv1)
# second bottom convolution layer
# split feature map by half
bottom_top_conv2 = Lambda(lambda x: x[:, :, :, :24])(bottom_conv1)
bottom_bot_conv2 = Lambda(lambda x: x[:, :, :, 24:])(bottom_conv1)
bottom_top_conv2 = Convolution2D(filters=64, kernel_size=(3, 3), strides=(1, 1), activation='relu',
padding='same')(bottom_top_conv2)
bottom_top_conv2 = BatchNormalization()(bottom_top_conv2)
bottom_top_conv2 = MaxPooling2D(pool_size=(3, 3), strides=(2, 2))(bottom_top_conv2)
bottom_bot_conv2 = Convolution2D(filters=64, kernel_size=(3, 3), strides=(1, 1), activation='relu',
padding='same')(bottom_bot_conv2)
bottom_bot_conv2 = BatchNormalization()(bottom_bot_conv2)
bottom_bot_conv2 = MaxPooling2D(pool_size=(3, 3), strides=(2, 2))(bottom_bot_conv2)
# third bottom convolution layer
# concat 2 feature map
bottom_conv3 = Concatenate()([bottom_top_conv2, bottom_bot_conv2])
bottom_conv3 = Convolution2D(filters=192, kernel_size=(3, 3), strides=(1, 1), activation='relu',
padding='same')(bottom_conv3)
# fourth bottom convolution layer
# split feature map by half
bottom_top_conv4 = Lambda(lambda x: x[:, :, :, :96])(bottom_conv3)
bottom_bot_conv4 = Lambda(lambda x: x[:, :, :, 96:])(bottom_conv3)
bottom_top_conv4 = Convolution2D(filters=96, kernel_size=(3, 3), strides=(1, 1), activation='relu',
padding='same')(bottom_top_conv4)
bottom_bot_conv4 = Convolution2D(filters=96, kernel_size=(3, 3), strides=(1, 1), activation='relu',
padding='same')(bottom_bot_conv4)
# fifth bottom convolution layer
bottom_top_conv5 = Convolution2D(filters=64, kernel_size=(3, 3), strides=(1, 1), activation='relu',
padding='same')(bottom_top_conv4)
bottom_top_conv5 = MaxPooling2D(pool_size=(3, 3), strides=(2, 2))(bottom_top_conv5)
bottom_bot_conv5 = Convolution2D(filters=64, kernel_size=(3, 3), strides=(1, 1), activation='relu',
padding='same')(bottom_bot_conv4)
bottom_bot_conv5 = MaxPooling2D(pool_size=(3, 3), strides=(2, 2))(bottom_bot_conv5)
# ---------------------------------- CONCATENATE TOP AND BOTTOM BRANCH ------------------------------------
conv_output = Concatenate()([top_top_conv5, top_bot_conv5, bottom_top_conv5, bottom_bot_conv5])
# Flatten
flatten = Flatten()(conv_output)
# Fully-connected layer
FC_1 = Dense(units=4096, activation='relu')(flatten)
FC_1 = Dropout(0.6)(FC_1)
FC_2 = Dense(units=4096, activation='relu')(FC_1)
FC_2 = Dropout(0.6)(FC_2)
output = Dense(units=self.numClasses, activation='softmax')(FC_2)
self.model = Model(inputs=input_image, outputs=output)
sgd = SGD(lr=1e-3, decay=1e-6, momentum=0.9, nesterov=True)
# sgd = SGD(lr=0.01, momentum=0.9, decay=0.0005, nesterov=True)
self.model.compile(optimizer=sgd, loss='categorical_crossentropy', metrics=['accuracy'])
def train(self,
pathToTrainSet,
pathToValidSet,
pathToSaveModel,
epochs=7,
batchSize=32,
stepsPerEpoch=200,
validationSteps=1000):
fileOfWeights = 'color_weights.hdf5'
checkpoint = ModelCheckpoint(os.path.join(pathToSaveModel, fileOfWeights),
monitor='val_acc', verbose=1,
save_best_only=True, mode='max')
checkpoint2 = ModelCheckpoint(os.path.join(pathToSaveModel, fileOfWeights),
monitor='val_loss', verbose=1,
save_best_only=True, mode='max')
trainDataGen = ImageDataGenerator(rescale=1.0/255, shear_range=0.2,
zoom_range=0.3, horizontal_flip=True)
validDataGen = ImageDataGenerator(rescale=1.0/255)
trainSet = trainDataGen.flow_from_directory(
pathToTrainSet,
target_size=(self.imageWidth, self.imageHeight),
batch_size=batchSize,
class_mode='categorical'
)
self.classes = {v: k for k, v in trainSet.class_indices.items()}
np.save(os.path.join(pathToSaveModel, 'class_index.npy'), self.classes)
validSet = validDataGen.flow_from_directory(
pathToValidSet,
target_size=(self.imageWidth, self.imageHeight),
batch_size=batchSize,
class_mode='categorical'
)
self.model.fit_generator(
trainSet,
steps_per_epoch=stepsPerEpoch,
epochs=epochs,
validation_data=validSet,
validation_steps=validationSteps//batchSize,
callbacks=[checkpoint, checkpoint2])
print('============================ Saving is here ============================')
self.model.save(os.path.join(pathToSaveModel, 'car_color_net.h5'))
#staticmethod
def load(pathToModel, pathToClassIndexes):
model = load_model(pathToModel)
layers = model.layers
inputShape, outputShape = layers[0].input_shape, layers[-1].output_shape,
imageWidth, imageHeight = inputShape[1], inputShape[2]
numClasses = outputShape[1]
net = CarColorNet(numClasses, imageWidth, imageHeight)
net.classes = np.load(os.path.join(pathToClassIndexes, 'class_index.npy')).item()
return net
def predictOneImage(self, pathToImage):
frame = cv2.imread(pathToImage)
frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
frame = cv2.resize(frame, (self.imageWidth, self.imageHeight))
frame = np.expand_dims(frame, axis=0)
# cv2.imshow("boxed", frame[0, :, :, :])
# cv2.waitKey(0)
frame = np.asarray(frame, dtype='float32')
img = frame/255
probs = self.model.predict(img)
ind = probs.argmax(axis=-1)[0]
return self.classes[ind]
if __name__ == '__main__':
pathToTrainSet = '/home/sergorl/cars/train'
pathToValidSet = '/home/sergorl/cars/valid'
pathToSaveModel = '/home/sergorl/cars'
## Train net
# net = CarColorNet(numClasses=6)
# net.train(pathToTrainSet, pathToValidSet, pathToSaveModel)
# Test on all images from validSet
net = CarColorNet.load(os.path.join(pathToSaveModel, 'car_color_net.h5'), pathToSaveModel)
count, countTrueLabels = 0, 0
for dirpath, _dirnames, filenames in os.walk(pathToValidSet):
trueLabel = dirpath.split('/')[-1]
for file in filenames:
label = net.predictOneImage(os.path.join(dirpath, file))
print(trueLabel, label)
if label == trueLabel:
countTrueLabels += 1
count += 1
print('rate is {0:.2f}'.format(float(countTrueLabels) / float(count) * 100))
If I have a good val_acc=0.97, I'll expect the same result (or nearly), testing every image in validation set. But always have wrong prediction!
I ran net immediately after train was done and see that learning was good:
if __name__ == '__main__':
pathToTrainSet = '/home/sergorl/cars/train'
pathToValidSet = '/home/sergorl/cars/valid'
pathToSaveModel = '/home/sergorl/cars'
# Train net
net = CarColorNet(numClasses=6)
net.train(pathToTrainSet, pathToValidSet, pathToSaveModel)
# Test on all images from validSet
count, countTrueLabels = 0, 0
for dirpath, _dirnames, filenames in os.walk(pathToValidSet):
trueLabel = dirpath.split('/')[-1]
for file in filenames:
label = net.predictOneImage(os.path.join(dirpath, file))
print(trueLabel, label)
if label == trueLabel:
countTrueLabels += 1
count += 1
print('rate is {0:.2f}'.format(float(countTrueLabels) / float(count) * 100))
So it seems the problem is inside model.save and it looks like saving doesn't work!. I found many related issues on git, for example:
https://github.com/keras-team/keras/issues/4875
https://github.com/keras-team/keras/issues/4904
But I don't know how to fix it with Python 3.7.3 and keras 2.0.0
Can you share more about the issue like what is the output you are getting. From the code i can see that you are training for 6 classes and using categorical cross entropy so ideally you should be getting an array with 6 values with each value bw 0 and 1 and the index of highest value in that array should be the output.
I am facing an error in plotting my confusion matrix. I am giving the test labels and my predicted label in confusion matrix function but it is giving me the value error having the problem in number of samples.
Shape of My data is below.
Trainig Data Shape (4162, 224, 224, 3)
Training Data Labels Shape (4162, 5)
Testing Data Shape (3921, 224, 224, 3)
Testing Data Labels Shape (3921, 5)
Predicted Label is a bit ugly because of only 2 epochs run, I just wanted to plot the confusion matrix first so thats why.
predictingimage = "D:/compCarsThesisData/data/image/78/3/2010/0ba8d018cdc994.jpg" #67/1698/2010/6805eb92ac6c70.jpg"
predictImageRead = mpg.imread(predictingimage)
resizingImage = cv2.cv2.resize(predictImageRead,(224,224))
reshapedFinalImage = np.expand_dims(resizingImage, axis=0)
npimage = np.asarray(reshapedFinalImage)
m = model.predict(npimage)
print(m)
[array([[0.02502811, 0.01959323, 0.6556284 , 0.26472655, 0.03502375]],
dtype=float32), array([[5.8234303e-04, 3.1917400e-04, 9.4957882e-01, 1.8873921e-02,
3.0645736e-02]], dtype=float32), array([[0.02581117, 0.04752538, 0.81816435, 0.04812173, 0.06037736]],
dtype=float32)]
cm = confusion_matrix(train_labels_Encode,m)
plt.imshow(cm)
plt.show()
ERROR
Traceback (most recent call last):
File "d:/ThesisWork/seriouswork/Inception_SVM_CompCarsGoogleNetArchitecture.py", line 299, in <module>
cm = confusion_matrix(train_labels_hotEncode,n)
File "C:\Users\zeele\Miniconda3\lib\site-packages\sklearn\metrics\classification.py", line 253, in confusion_matrix
y_type, y_true, y_pred = _check_targets(y_true, y_pred)
File "C:\Users\zeele\Miniconda3\lib\site-packages\sklearn\metrics\classification.py", line 71, in _check_targets
check_consistent_length(y_true, y_pred)
File "C:\Users\zeele\Miniconda3\lib\site-packages\sklearn\utils\validation.py", line 235, in check_consistent_length
" samples: %r" % [int(l) for l in lengths])
ValueError: Found input variables with inconsistent numbers of samples: [4162, 3]
Classifier Code:
X_train = np.load('D:/Inception_preprocessed_data_Labels_2004/Top5/TrainingData_Top5.npy')#('D:/ThesisWork/S_224_Training_data.npy')#training_images
X_test = np.load('D:/Inception_preprocessed_data_Labels_2004/Top5/TrainingLabels_Top5.npy')#('D:/ThesisWork/S_224_Training_labels.npy')#training_labels
y_train = np.load('D:/Inception_preprocessed_data_Labels_2004/Top5/TestingData_Top5.npy')#('D:/ThesisWork/S_224_Testing_data.npy')#testing_images
y_test = np.load('D:/Inception_preprocessed_data_Labels_2004/Top5/TestingLabels_Top5.npy')#('D:/ThesisWork/S_224_Testing_labels.npy')#testing_labels
print(X_test)
le = preprocessing.LabelEncoder()
le.fit(X_test)
transform_trainLabels = le.transform(X_test)
print(transform_trainLabels)
print(le.inverse_transform(transform_trainLabels))
train_labels_hotEncode = np_utils.to_categorical(transform_trainLabels,len(set(transform_trainLabels)))
shuffle(X_train)
shuffle(train_labels_hotEncode)
le2 = preprocessing.LabelEncoder()
le2.fit(y_test)
transform_testLabels = le2.transform(y_test)
test_labels_hotEncode = np_utils.to_categorical(transform_testLabels,len(set(transform_testLabels)))
print(test_labels_hotEncode.shape)
shuffle(y_train)
shuffle(test_labels_hotEncode)
# print(train_labels_hotEncode[3000])
# exit()
# X_train = np.asarray(X_train / 255.0)
# y_train = np.asarray(y_train / 255.0)
# print("X_Training" ,X_train.shape, X_train)
# print("X_TEST", X_test.shape)
# print("Y_train", y_train.shape)
# print("y_test", y_test.shape)
# exit()
# plt.imshow(X_train[1])
# print(X_test)
# plt.imshow(y_train[1])
# print(y_test)
# plt.show()
print("Trainig Data Shape",X_train.shape)
print("Training Data Labels Shape",train_labels_hotEncode.shape)
print("Testing Data Shape", y_train.shape)
print("Testing Data Labels Shape", test_labels_hotEncode.shape)
# X_train = np.array(X_train).astype(np.float32)
# y_train = np.array(y_train).astype(np.float32)
def inception_module(image,
filters_1x1,
filters_3x3_reduce,
filter_3x3,
filters_5x5_reduce,
filters_5x5,
filters_pool_proj,
name=None):
conv_1x1 = Conv2D(filters_1x1, (1,1), padding='same', activation='relu', kernel_initializer=kernel_init, bias_initializer= bias_init)(image)
conv_3x3 = Conv2D(filters_3x3_reduce, (1,1), padding='same', activation='relu', kernel_initializer=kernel_init, bias_initializer= bias_init)(image)
conv_3x3 = Conv2D(filter_3x3,(3,3), padding='same', activation='relu', kernel_initializer=kernel_init, bias_initializer=bias_init)(conv_3x3)
conv_5x5 = Conv2D(filters_5x5_reduce,(1,1), padding='same', activation='relu',kernel_initializer=kernel_init, bias_initializer= bias_init)(image)
conv_5x5 = Conv2D(filters_5x5, (3,3), padding='same', activation='relu',kernel_initializer=kernel_init, bias_initializer=bias_init)(conv_5x5)
pool_proj = MaxPool2D((3,3), strides=(1,1), padding='same')(image)
pool_proj = Conv2D(filters_pool_proj, (1,1), padding='same', activation='relu', kernel_initializer=kernel_init, bias_initializer= bias_init)(pool_proj)
output = concatenate([conv_1x1, conv_3x3, conv_5x5, pool_proj], axis=3, name=name)
return output
kernel_init = keras.initializers.glorot_uniform()
bias_init = keras.initializers.Constant(value=0.2)
# IMG_SIZE = 64
input_layer = Input(shape=(224,224,3))
image = Conv2D(64,(7,7),padding='same', strides=(2,2), activation='relu', name='conv_1_7x7/2', kernel_initializer=kernel_init, bias_initializer=bias_init)(input_layer)
image = MaxPool2D((3,3), padding='same', strides=(2,2), name='max_pool_1_3x3/2')(image)
image = Conv2D(64, (1,1), padding='same', strides=(1,1), activation='relu', name='conv_2a_3x3/1' )(image)
image = Conv2D(192, (3,3), padding='same', strides=(1,1), activation='relu', name='conv_2b_3x3/1')(image)
image = MaxPool2D((3,3), padding='same', strides=(2,2), name='max_pool_2_3x3/2')(image)
image = inception_module(image,
filters_1x1= 64,
filters_3x3_reduce= 96,
filter_3x3 = 128,
filters_5x5_reduce=16,
filters_5x5= 32,
filters_pool_proj=32,
name='inception_3a')
image = inception_module(image,
filters_1x1=128,
filters_3x3_reduce=128,
filter_3x3=192,
filters_5x5_reduce=32,
filters_5x5=96,
filters_pool_proj=64,
name='inception_3b')
image = MaxPool2D((3,3), padding='same', strides=(2,2), name='max_pool_3_3x3/2')(image)
image = inception_module(image,
filters_1x1=192,
filters_3x3_reduce=96,
filter_3x3=208,
filters_5x5_reduce=16,
filters_5x5=48,
filters_pool_proj=64,
name='inception_4a')
image1 = AveragePooling2D((5,5), strides=3)(image)
image1 = Conv2D(128, (1,1), padding='same', activation='relu')(image1)
image1 = Flatten()(image1)
image1 = Dense(1024, activation='relu')(image1)
image1 = Dropout(0.7)(image1)
image1 = Dense(5, activation='softmax', name='auxilliary_output_1')(image1)
image = inception_module(image,
filters_1x1 = 160,
filters_3x3_reduce= 112,
filter_3x3= 224,
filters_5x5_reduce= 24,
filters_5x5= 64,
filters_pool_proj=64,
name='inception_4b')
image = inception_module(image,
filters_1x1= 128,
filters_3x3_reduce = 128,
filter_3x3= 256,
filters_5x5_reduce= 24,
filters_5x5=64,
filters_pool_proj=64,
name='inception_4c')
image = inception_module(image,
filters_1x1=112,
filters_3x3_reduce=144,
filter_3x3= 288,
filters_5x5_reduce= 32,
filters_5x5=64,
filters_pool_proj=64,
name='inception_4d')
image2 = AveragePooling2D((5,5), strides=3)(image)
image2 = Conv2D(128, (1,1), padding='same', activation='relu')(image2)
image2 = Flatten()(image2)
image2 = Dense(1024, activation='relu')(image2)
image2 = Dropout(0.7)(image2) #Changed from 0.7
image2 = Dense(5, activation='softmax', name='auxilliary_output_2')(image2)
image = inception_module(image,
filters_1x1=256,
filters_3x3_reduce=160,
filter_3x3=320,
filters_5x5_reduce=32,
filters_5x5=128,
filters_pool_proj=128,
name= 'inception_4e')
image = MaxPool2D((3,3), padding='same', strides=(2,2), name='max_pool_4_3x3/2')(image)
image = inception_module(image,
filters_1x1=256,
filters_3x3_reduce=160,
filter_3x3= 320,
filters_5x5_reduce=32,
filters_5x5= 128,
filters_pool_proj=128,
name='inception_5a')
image = inception_module(image,
filters_1x1=384,
filters_3x3_reduce=192,
filter_3x3=384,
filters_5x5_reduce=48,
filters_5x5=128,
filters_pool_proj=128,
name='inception_5b')
image = GlobalAveragePooling2D(name='avg_pool_5_3x3/1')(image)
image = Dropout(0.7)(image)
image = Dense(5, activation='softmax', name='output')(image)
model = Model(input_layer, [image,image1,image2], name='inception_v1')
model.summary()
epochs = 2
initial_lrate = 0.001 # Changed From 0.01
def decay(epoch, steps=100):
initial_lrate = 0.01
drop = 0.96
epochs_drop = 8
lrate = initial_lrate * math.pow(drop,math.floor((1+epoch)/epochs_drop))#
return lrate
sgd = keras.optimizers.SGD(lr=0.01, decay=1e-6, momentum=0.9, nesterov=True)
# nadam = keras.optimizers.Nadam(lr= 0.002, beta_1=0.9, beta_2=0.999, epsilon=None)
# keras
lr_sc = LearningRateScheduler(decay)
# rms = keras.optimizers.RMSprop(lr = initial_lrate, rho=0.9, epsilon=1e-08, decay=0.0)
# ad = keras.optimizers.adam(lr=initial_lrate)
model.compile(loss=['categorical_crossentropy', 'categorical_crossentropy','categorical_crossentropy'],loss_weights=[1,0.3,0.3], optimizer='sgd', metrics=['accuracy'])
# loss = 'categorical_crossentropy', 'categorical_crossentropy','categorical_crossentropy'
history = model.fit(X_train, [train_labels_hotEncode,train_labels_hotEncode,train_labels_hotEncode], validation_split=0.3,shuffle=True,epochs=epochs, batch_size= 32, callbacks=[lr_sc]) # batch size changed from 256 or 64 to 16(y_train,[y_test,y_test,y_test])
# validation_data=(y_train,[test_labels_hotEncode,test_labels_hotEncode,test_labels_hotEncode]), validation_data= (X_train, [train_labels_hotEncode,train_labels_hotEncode,train_labels_hotEncode]),
print(history.history.keys())
plt.plot(history.history['output_acc'])
plt.plot(history.history['val_output_acc'])
plt.title('Model accuracy')
plt.ylabel('accuracy')
plt.xlabel('epoch')
plt.legend(['train', 'test'],loc = 'upper left')
plt.show()
# predictingimage = "D:/compCarsThesisData/data/image/78/3/2010/0ba8d018cdc994.jpg" #67/1698/2010/6805eb92ac6c70.jpg"
predictImageRead = X_train
# resizingImage = cv2.cv2.resize(predictImageRead,(224,224))
# reshapedFinalImage = np.expand_dims(predictImageRead, axis=0)
# print(reshapedFinalImage.shape)
# npimage = np.array(reshapedFinalImage)
m = model.predict(predictImageRead)
print(m)
print(predictImageRead.shape)
print(train_labels_hotEncode)
# print(m.shape)
plt.imshow(predictImageRead[1])
plt.show()
# n = np.argmax(m,axis=-1)
# n = np.array(m)
print(confusion_matrix(X_test,m[0]))
cm = confusion_matrix(X_test,m[0])
plt.imshow(cm)
plt.show()
Please guide me through this.
Thanks!
If you want to compute a confusion matrix of your training data you have to make your moddel predict all your training examples, roughly like this:
m = model.predict(train_data) # train_data should have the shape (4162, 224, 224, 3)
m should then have a length of 4162 and you can plot the confusion matrix like this:
cm = confusion_matrix(train_labels_Encode, m)
plt.imshow(cm)
plt.show()
I am building a image classifier for one-class classification in which i've used autoencoder.
While running this model I am getting this error by the line autoencoder_model.fit:
ValueError: Error when checking target: expected model_2 to have shape (None, 252, 252, 1) but got array with shape (300, 128, 128, 3)
num_of_samples = img_data.shape[0]
labels = np.ones((num_of_samples,),dtype='int64')
labels[0:376]=0
names = ['cats']
input_shape=img_data[0].shape
X_train, X_test = train_test_split(img_data, test_size=0.2, random_state=2)
inputTensor = Input(input_shape)
x = Conv2D(16, (3, 3), activation='relu', padding='same')(inputTensor)
x = Conv2D(8, (3, 3), activation='relu', padding='same')(x)
x = MaxPooling2D((2, 2), padding='same')(x)
x = Conv2D(8, (3, 3), activation='relu', padding='same')(x)
encoded_data = MaxPooling2D((2, 2), padding='same')(x)
encoder_model = Model(inputTensor,encoded_data)
# at this point the representation is (4, 4, 8) i.e. 128-dimensional
encoded_input = Input((4,4,8))
x = Conv2D(8, (3, 3), activation='relu', padding='same')(encoded_input)
x = UpSampling2D((2, 2))(x)
x = Conv2D(8, (3, 3), activation='relu', padding='same')(x)
x = UpSampling2D((2, 2))(x)
x = Conv2D(16, (3, 3), activation='relu',padding='same')(x)
x = UpSampling2D((2, 2))(x)
decoded_data = Conv2D(1, (3, 3), activation='sigmoid', padding='same')(x)
decoder_model = Model(encoded_input,decoded_data)
autoencoder_input = Input(input_shape)
encoded = encoder_model(autoencoder_input)
decoded = decoder_model(encoded)
autoencoder_model = Model(autoencoder_input, decoded)
autoencoder_model.compile(optimizer='adadelta', enter code here`loss='binary_crossentropy')
autoencoder_model.fit(X_train, X_train,
epochs=50,
batch_size=32,
validation_data=(X_test, X_test),
callbacks=[TensorBoard(log_dir='/tmp/autoencoder')])
As the auto-encoder tries to re-create the original images, it seems you are reconstructing an image with different dimensions than the original, due to the fact to have only two MaxPool2D layers in your encoder and three UpSampling2D layers in your decoder.
When the auto-encoder tries to evaluate the loss of the reconstruction, it runs into an error due to a dimension miss-match.
Use this for your encoder and let us know if it works:
inputTensor = Input(input_shape)
x = Conv2D(16, (3, 3), activation='relu', padding='same')(inputTensor)
x = MaxPooling2D((2, 2), padding='same')(x)
x = Conv2D(8, (3, 3), activation='relu', padding='same')(x)
x = MaxPooling2D((2, 2), padding='same')(x)
x = Conv2D(8, (3, 3), activation='relu', padding='same')(x)
encoded_data = MaxPooling2D((2, 2), padding='same')(x)
encoder_model = Model(inputTensor,encoded_data)