Related
I want to store the final gradient vector of a model as a numpy array. Is there an easy and intuitive way to do that using Tensorflow?
I want to store the gradient vectors of Alexnet (in a numpy array) for each iteration,, until convergence.
We can do it as shown below code -
import tensorflow as tf
import numpy as np
print(tf.__version__)
#Define the input tensor
x = tf.constant([3.0,6.0,9.0])
#Define the Gradient Function
with tf.GradientTape() as g:
g.watch(x)
y = x * x
dy_dx = g.gradient(y, x)
#Output Gradient Tensor
print("Output Gradient Tensor:",dy_dx)
#Convert to array
a = np.asarray(dy_dx)
print("Gradient array:",a)
print("Array shape:",a.shape)
print("Output type:",type(a))
The Output of the code is -
2.1.0
Output Gradient Tensor: tf.Tensor([ 6. 12. 18.], shape=(3,), dtype=float32)
Gradient array: [ 6. 12. 18.]
Array shape: (3,)
Output type: <class 'numpy.ndarray'>
Below is the model that resembles Alexnet architecture and capturing gradient for every epoch.
# (1) Importing dependency
import keras
from keras import backend as K
from keras.models import Sequential
from keras.layers import Dense, Activation, Dropout, Flatten, Conv2D, MaxPooling2D
from keras.layers.normalization import BatchNormalization
import numpy as np
np.random.seed(1000)
# (2) Get Data
import tflearn.datasets.oxflower17 as oxflower17
x, y = oxflower17.load_data(one_hot=True)
# (3) Create a sequential model
model = Sequential()
# 1st Convolutional Layer
model.add(Conv2D(filters=96, input_shape=(224,224,3), kernel_size=(11,11), strides=(4,4), padding='valid'))
model.add(Activation('relu'))
# Pooling
model.add(MaxPooling2D(pool_size=(2,2), strides=(2,2), padding='valid'))
# Batch Normalisation before passing it to the next layer
model.add(BatchNormalization())
# 2nd Convolutional Layer
model.add(Conv2D(filters=256, kernel_size=(11,11), strides=(1,1), padding='valid'))
model.add(Activation('relu'))
# Pooling
model.add(MaxPooling2D(pool_size=(2,2), strides=(2,2), padding='valid'))
# Batch Normalisation
model.add(BatchNormalization())
# 3rd Convolutional Layer
model.add(Conv2D(filters=384, kernel_size=(3,3), strides=(1,1), padding='valid'))
model.add(Activation('relu'))
# Batch Normalisation
model.add(BatchNormalization())
# 4th Convolutional Layer
model.add(Conv2D(filters=384, kernel_size=(3,3), strides=(1,1), padding='valid'))
model.add(Activation('relu'))
# Batch Normalisation
model.add(BatchNormalization())
# 5th Convolutional Layer
model.add(Conv2D(filters=256, kernel_size=(3,3), strides=(1,1), padding='valid'))
model.add(Activation('relu'))
# Pooling
model.add(MaxPooling2D(pool_size=(2,2), strides=(2,2), padding='valid'))
# Batch Normalisation
model.add(BatchNormalization())
# Passing it to a dense layer
model.add(Flatten())
# 1st Dense Layer
model.add(Dense(4096, input_shape=(224*224*3,)))
model.add(Activation('relu'))
# Add Dropout to prevent overfitting
model.add(Dropout(0.4))
# Batch Normalisation
model.add(BatchNormalization())
# 2nd Dense Layer
model.add(Dense(4096))
model.add(Activation('relu'))
# Add Dropout
model.add(Dropout(0.4))
# Batch Normalisation
model.add(BatchNormalization())
# 3rd Dense Layer
model.add(Dense(1000))
model.add(Activation('relu'))
# Add Dropout
model.add(Dropout(0.4))
# Batch Normalisation
model.add(BatchNormalization())
# Output Layer
model.add(Dense(17))
model.add(Activation('softmax'))
model.summary()
# (4) Compile
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
# (5) Define Gradient Function
def get_gradient_func(model):
grads = K.gradients(model.total_loss, model.trainable_weights)
inputs = model.model._feed_inputs + model.model._feed_targets + model.model._feed_sample_weights
func = K.function(inputs, grads)
return func
# (6) Train the model such that gradients are captured for every epoch
epoch_gradient = []
for epoch in range(1,5):
model.fit(x, y, batch_size=64, epochs= epoch, initial_epoch = (epoch-1), verbose=1, validation_split=0.2, shuffle=True)
get_gradient = get_gradient_func(model)
grads = get_gradient([x, y, np.ones(len(y))])
epoch_gradient.append(grads)
# (7) Convert to a 2 dimensiaonal array of (epoch, gradients) type
gradient = np.asarray(epoch_gradient)
print("Total number of epochs run:", epoch)
print("Gradient Array has the shape:",gradient.shape)
Output: gradient is the 2 dimensional array that has gradient captured for every epoch that retains the structure of gradient as per the network layers.
Model: "sequential_34"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d_115 (Conv2D) (None, 54, 54, 96) 34944
_________________________________________________________________
activation_213 (Activation) (None, 54, 54, 96) 0
_________________________________________________________________
max_pooling2d_83 (MaxPooling (None, 27, 27, 96) 0
_________________________________________________________________
batch_normalization_180 (Bat (None, 27, 27, 96) 384
_________________________________________________________________
conv2d_116 (Conv2D) (None, 17, 17, 256) 2973952
_________________________________________________________________
activation_214 (Activation) (None, 17, 17, 256) 0
_________________________________________________________________
max_pooling2d_84 (MaxPooling (None, 8, 8, 256) 0
_________________________________________________________________
batch_normalization_181 (Bat (None, 8, 8, 256) 1024
_________________________________________________________________
conv2d_117 (Conv2D) (None, 6, 6, 384) 885120
_________________________________________________________________
activation_215 (Activation) (None, 6, 6, 384) 0
_________________________________________________________________
batch_normalization_182 (Bat (None, 6, 6, 384) 1536
_________________________________________________________________
conv2d_118 (Conv2D) (None, 4, 4, 384) 1327488
_________________________________________________________________
activation_216 (Activation) (None, 4, 4, 384) 0
_________________________________________________________________
batch_normalization_183 (Bat (None, 4, 4, 384) 1536
_________________________________________________________________
conv2d_119 (Conv2D) (None, 2, 2, 256) 884992
_________________________________________________________________
activation_217 (Activation) (None, 2, 2, 256) 0
_________________________________________________________________
max_pooling2d_85 (MaxPooling (None, 1, 1, 256) 0
_________________________________________________________________
batch_normalization_184 (Bat (None, 1, 1, 256) 1024
_________________________________________________________________
flatten_34 (Flatten) (None, 256) 0
_________________________________________________________________
dense_99 (Dense) (None, 4096) 1052672
_________________________________________________________________
activation_218 (Activation) (None, 4096) 0
_________________________________________________________________
dropout_66 (Dropout) (None, 4096) 0
_________________________________________________________________
batch_normalization_185 (Bat (None, 4096) 16384
_________________________________________________________________
dense_100 (Dense) (None, 4096) 16781312
_________________________________________________________________
activation_219 (Activation) (None, 4096) 0
_________________________________________________________________
dropout_67 (Dropout) (None, 4096) 0
_________________________________________________________________
batch_normalization_186 (Bat (None, 4096) 16384
_________________________________________________________________
dense_101 (Dense) (None, 1000) 4097000
_________________________________________________________________
activation_220 (Activation) (None, 1000) 0
_________________________________________________________________
dropout_68 (Dropout) (None, 1000) 0
_________________________________________________________________
batch_normalization_187 (Bat (None, 1000) 4000
_________________________________________________________________
dense_102 (Dense) (None, 17) 17017
_________________________________________________________________
activation_221 (Activation) (None, 17) 0
=================================================================
Total params: 28,096,769
Trainable params: 28,075,633
Non-trainable params: 21,136
_________________________________________________________________
Train on 1088 samples, validate on 272 samples
Epoch 1/1
1088/1088 [==============================] - 22s 20ms/step - loss: 3.1251 - acc: 0.2178 - val_loss: 13.0005 - val_acc: 0.1140
Train on 1088 samples, validate on 272 samples
Epoch 2/2
128/1088 [==>...........................] - ETA: 1s - loss: 2.3913 - acc: 0.2656/usr/local/lib/python3.6/dist-packages/keras/engine/sequential.py:111: UserWarning: `Sequential.model` is deprecated. `Sequential` is a subclass of `Model`, you can just use your `Sequential` instance directly.
warnings.warn('`Sequential.model` is deprecated. '
1088/1088 [==============================] - 2s 2ms/step - loss: 2.2318 - acc: 0.3465 - val_loss: 9.6171 - val_acc: 0.1912
Train on 1088 samples, validate on 272 samples
Epoch 3/3
64/1088 [>.............................] - ETA: 1s - loss: 1.5143 - acc: 0.5000/usr/local/lib/python3.6/dist-packages/keras/engine/sequential.py:111: UserWarning: `Sequential.model` is deprecated. `Sequential` is a subclass of `Model`, you can just use your `Sequential` instance directly.
warnings.warn('`Sequential.model` is deprecated. '
1088/1088 [==============================] - 2s 2ms/step - loss: 1.8109 - acc: 0.4320 - val_loss: 4.3375 - val_acc: 0.3162
Train on 1088 samples, validate on 272 samples
Epoch 4/4
64/1088 [>.............................] - ETA: 1s - loss: 1.7827 - acc: 0.4688/usr/local/lib/python3.6/dist-packages/keras/engine/sequential.py:111: UserWarning: `Sequential.model` is deprecated. `Sequential` is a subclass of `Model`, you can just use your `Sequential` instance directly.
warnings.warn('`Sequential.model` is deprecated. '
1088/1088 [==============================] - 2s 2ms/step - loss: 1.5861 - acc: 0.4871 - val_loss: 3.4091 - val_acc: 0.3787
Total number of epochs run: 4
Gradient Array has the shape: (4, 34)
/usr/local/lib/python3.6/dist-packages/keras/engine/sequential.py:111: UserWarning: `Sequential.model` is deprecated. `Sequential` is a subclass of `Model`, you can just use your `Sequential` instance directly.
warnings.warn('`Sequential.model` is deprecated. '
I built an autoencoder using my own data set of about 32k images. I did a 75/25 split for training/testing, and I was able to get results I'm happy with.
Now I want to be able to extract the feature space and map them to every image in my dataset and to new data that wasn't tested. I couldn't find a tutorial online that delved into using the encoder as a feature space. All I could find was to build the full network.
My code:
> input_img = Input(shape=(200, 200, 1))
# encoder part of the model (increased filter lyaer after each filter)
x = Conv2D(16, (3, 3), activation='relu', padding='same')(input_img)
x = MaxPooling2D((2, 2), padding='same')(x)
x = Conv2D(32, (3, 3), activation='relu', padding='same')(x)
x = MaxPooling2D((2, 2), padding='same')(x)
x = Conv2D(64, (3, 3), activation='relu', padding='same')(x)
x = MaxPooling2D((2, 2), padding='same')(x)
x = Conv2D(128, (3, 3), activation='relu', padding='same')(x)
encoded = MaxPooling2D((2, 2), padding='same')(x)
# decoder part of the model (went backwards from the encoder)
x = Conv2D(128, (3, 3), activation='relu', padding='same')(encoded)
x = UpSampling2D((2, 2))(x)
x = Conv2D(64, (3, 3), activation='relu', padding='same')(x)
x = UpSampling2D((2, 2))(x)
x = Conv2D(32, (3, 3), activation='relu', padding='same')(x)
x = UpSampling2D((2, 2))(x)
x = Conv2D(16, (3, 3), activation='relu', padding='same')(x)
x = UpSampling2D((2, 2))(x)
x = Conv2D(1, (3, 3), activation='sigmoid', padding='same')(x)
decoded = Cropping2D(cropping=((8,0), (8,0)), data_format=None)(x)
autoencoder = Model(input_img, decoded)
autoencoder.compile(optimizer='adam', loss='binary_crossentropy')
autoencoder.summary()
Here's my net setup if anybody is interested:
Model: "model_22"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_23 (InputLayer) (None, 200, 200, 1) 0
_________________________________________________________________
conv2d_186 (Conv2D) (None, 200, 200, 16) 160
_________________________________________________________________
max_pooling2d_83 (MaxPooling (None, 100, 100, 16) 0
_________________________________________________________________
conv2d_187 (Conv2D) (None, 100, 100, 32) 4640
_________________________________________________________________
max_pooling2d_84 (MaxPooling (None, 50, 50, 32) 0
_________________________________________________________________
conv2d_188 (Conv2D) (None, 50, 50, 64) 18496
_________________________________________________________________
max_pooling2d_85 (MaxPooling (None, 25, 25, 64) 0
_________________________________________________________________
conv2d_189 (Conv2D) (None, 25, 25, 128) 73856
_________________________________________________________________
max_pooling2d_86 (MaxPooling (None, 13, 13, 128) 0
_________________________________________________________________
conv2d_190 (Conv2D) (None, 13, 13, 128) 147584
_________________________________________________________________
up_sampling2d_82 (UpSampling (None, 26, 26, 128) 0
_________________________________________________________________
conv2d_191 (Conv2D) (None, 26, 26, 64) 73792
_________________________________________________________________
up_sampling2d_83 (UpSampling (None, 52, 52, 64) 0
_________________________________________________________________
conv2d_192 (Conv2D) (None, 52, 52, 32) 18464
_________________________________________________________________
up_sampling2d_84 (UpSampling (None, 104, 104, 32) 0
_________________________________________________________________
conv2d_193 (Conv2D) (None, 104, 104, 16) 4624
_________________________________________________________________
up_sampling2d_85 (UpSampling (None, 208, 208, 16) 0
_________________________________________________________________
conv2d_194 (Conv2D) (None, 208, 208, 1) 145
_________________________________________________________________
cropping2d_2 (Cropping2D) (None, 200, 200, 1) 0
=================================================================
Total params: 341,761
Trainable params: 341,761
Non-trainable params: 0
Then my training:
autoencoder.fit(train, train,
epochs=3,
batch_size=128,
shuffle=True,
validation_data=(test, test))
My results:
Train on 23412 samples, validate on 7805 samples
Epoch 1/3
23412/23412 [==============================] - 773s 33ms/step - loss: 0.0620 - val_loss: 0.0398
Epoch 2/3
23412/23412 [==============================] - 715s 31ms/step - loss: 0.0349 - val_loss: 0.0349
Epoch 3/3
23412/23412 [==============================] - 753s 32ms/step - loss: 0.0314 - val_loss: 0.0319
Rather not share the images, but they look well reconstructed.
Thank you for all and any help!
Not sure if I fully understand your questions, but do you want to get the resulting feature space for every image you trained on as well as others. Why not just do this?
Name your encoded layer in your autoencoder architecture as 'embedding.' Then create the encoder the following way:
embedding_layer = autoencoder.get_layer(name='embedding').output
encoder = Model(input_img,embedding_layer)
I used Dice Loss and binary_crossentropy whenever I train my model it shows very high train and validation accuracy but always prints out blank images. My masks are black and white binary images where 0 corresponds to black and 1 corresponds to white. In my output image, almost all pixels have value 0 please tell me where am I going wrong.
def train_generator():
while True:
for start in range(0, len(os.listdir('/gdrive/My Drive/Train/img/images/')), 16):
x_batch = np.empty((16,256,512,1),dtype=np.float32)
y_batch = np.empty((16,256,512,1),dtype=np.float32)
end = min(start + 16, len(os.listdir('/gdrive/My Drive/Train/img/images/')))
ids_train_batch_images =os.listdir('/gdrive/My Drive/Train/img/images/')[start:end]
ids_train_batch_mask =os.listdir('/gdrive/My Drive/Train/msk/mask/')[start:end]
for i,id in enumerate(ids_train_batch_images):
x_sample = cv2.imread('/gdrive/My Drive/Train/img/images/'+ids_train_batch_images[i])
y_sample = cv2.imread('/gdrive/My Drive/Train/msk/mask/'+ids_train_batch_mask[i])
x_sample=cv2.resize(x_sample,(512,256),interpolation = cv2.INTER_AREA)
y_sample=cv2.resize(y_sample,(512,256),interpolation = cv2.INTER_AREA)
x_sample=x_sample[:,:,0]
y_sample=y_sample[:,:,0]
x_sample=np.expand_dims(x_sample,axis=-1)
y_sample=np.expand_dims(y_sample,axis=-1)
x_batch[i]=x_sample
y_batch[i]=y_sample.astype(np.bool)
x_batch = np.array(x_batch, np.float32)/255.0
y_batch = np.array(y_batch, np.bool)
yield x_batch, y_batch
def val_generator():
while True:
for start in range(0, len(os.listdir('/gdrive/My Drive/Validation/img/images/')), 16):
x_batch = np.empty((16,256,512,1),dtype=np.float32)
y_batch = np.empty((16,256,512,1),dtype=np.float32)
end = min(start + 16, len(os.listdir('/gdrive/My Drive/Validation/img/images/')))
ids_train_batch_images =os.listdir('/gdrive/My Drive/Validation/img/images/')[start:end]
ids_train_batch_mask =os.listdir('/gdrive/My Drive/Validation/msk/mask/')[start:end]
for i,id in enumerate(ids_train_batch_images):
x_sample = cv2.imread('/gdrive/My Drive/Validation/img/images/'+ids_train_batch_images[i])
y_sample = cv2.imread('/gdrive/My Drive/Validation/msk/mask/'+ids_train_batch_mask[i])
x_sample=cv2.resize(x_sample,(512,256),interpolation = cv2.INTER_AREA)
y_sample=cv2.resize(y_sample,(512,256),interpolation = cv2.INTER_AREA)
x_sample=x_sample[:,:,0]
y_sample=y_sample[:,:,0]
x_sample=np.expand_dims(x_sample,axis=-1)
y_sample=np.expand_dims(y_sample,axis=-1)
x_batch[i]=x_sample
y_batch[i]=y_sample.astype(np.bool)
x_batch = np.array(x_batch, np.float32)/255.0
y_batch = np.array(y_batch, np.bool)
yield x_batch, y_batch
train_gen=train_generator()
val_gen=val_generator()
def unet():
inputs = tf.keras.layers.Input((256,512,1))
s = inputs
c1 = tf.keras.layers.Conv2D(16, (3, 3), activation=tf.keras.activations.elu, kernel_initializer='he_normal',padding='same')(s)
c1 = tf.keras.layers.Dropout(0.3)(c1)
c1 = tf.keras.layers.Conv2D(16, (3, 3), activation=tf.keras.activations.elu, kernel_initializer='he_normal',padding='same')(c1)
p1 = tf.keras.layers.MaxPooling2D((2, 2))(c1)
c2 = tf.keras.layers.Conv2D(32, (3, 3), activation=tf.keras.activations.elu, kernel_initializer='he_normal', padding='same')(p1)
c2 = tf.keras.layers.Dropout(0.3)(c2)
c2 = tf.keras.layers.Conv2D(32, (3, 3), activation=tf.keras.activations.elu, kernel_initializer='he_normal',padding='same')(c2)
p2 = tf.keras.layers.MaxPooling2D((2, 2))(c2)
c3 = tf.keras.layers.Conv2D(64, (3, 3), activation=tf.keras.activations.elu, kernel_initializer='he_normal',padding='same')(p2)
c3 = tf.keras.layers.Dropout(0.3)(c3)
c3 = tf.keras.layers.Conv2D(64, (3, 3), activation=tf.keras.activations.elu, kernel_initializer='he_normal',padding='same')(c3)
p3 = tf.keras.layers.MaxPooling2D((2, 2))(c3)
c4 = tf.keras.layers.Conv2D(128, (3, 3), activation=tf.keras.activations.elu, kernel_initializer='he_normal',padding='same')(p3)
c4 = tf.keras.layers.Dropout(0.3)(c4)
c4 = tf.keras.layers.Conv2D(128, (3, 3), activation=tf.keras.activations.elu, kernel_initializer='he_normal',padding='same')(c4)
p4 = tf.keras.layers.MaxPooling2D(pool_size=(2, 2))(c4)
c6 = tf.keras.layers.Conv2D(256, (3, 3), activation=tf.keras.activations.elu, kernel_initializer='he_normal',padding='same')(p4)
c6 = tf.keras.layers.Dropout(0.3)(c6)
c6 = tf.keras.layers.Conv2D(256, (3, 3), activation=tf.keras.activations.elu, kernel_initializer='he_normal',padding='same')(c6)
p6 = tf.keras.layers.MaxPooling2D(pool_size=(2, 2))(c6)
# c6 = tf.keras.layers.Conv2D(1024, (3, 3), activation=tf.keras.activations.elu, kernel_initializer='he_normal',padding='same')(p5)
# c6 = tf.keras.layers.Dropout(0.1)(c6)
# c6 = tf.keras.layers.Conv2D(1024, (3, 3), activation=tf.keras.activations.elu, kernel_initializer='he_normal',padding='same')(c6)
# p6 = tf.keras.layers.MaxPooling2D(pool_size=(2, 2))(c6)
c7 = tf.keras.layers.Conv2D(512, (3, 3), activation=tf.keras.activations.elu, kernel_initializer='he_normal',padding='same')(p6)
c7 = tf.keras.layers.Dropout(0.3)(c7)
c7 = tf.keras.layers.Conv2D(512, (3, 3), activation=tf.keras.activations.elu, kernel_initializer='he_normal',padding='same')(c7)
# u8 = tf.keras.layers.Conv2DTranspose(1024, (2, 2), strides=(2, 2), padding='same')(c7)
# u8 = tf.keras.layers.concatenate([u8, c6])
# c8 = tf.keras.layers.Conv2D(1024, (3, 3), activation=tf.keras.activations.elu, kernel_initializer='he_normal',padding='same')(u8)
# c8 = tf.keras.layers.Dropout(0.1)(c8)
# c8 = tf.keras.layers.Conv2D(1024, (3, 3), activation=tf.keras.activations.elu, kernel_initializer='he_normal',padding='same')(c8)
u9 = tf.keras.layers.Conv2DTranspose(256, (2, 2), strides=(2, 2), padding='same')(c7)
u9 = tf.keras.layers.concatenate([u9, c6])
c9 = tf.keras.layers.Conv2D(256, (3, 3), activation=tf.keras.activations.elu, kernel_initializer='he_normal',padding='same')(u9)
c9 = tf.keras.layers.Dropout(0.3)(c9)
c9 = tf.keras.layers.Conv2D(256, (3, 3), activation=tf.keras.activations.elu, kernel_initializer='he_normal', padding='same')(c9)
u10 = tf.keras.layers.Conv2DTranspose(128, (2, 2), strides=(2, 2), padding='same')(c9)
u10 = tf.keras.layers.concatenate([u10, c4])
c10 = tf.keras.layers.Conv2D(128, (3, 3), activation=tf.keras.activations.elu, kernel_initializer='he_normal',padding='same')(u10)
c10 = tf.keras.layers.Dropout(0.3)(c10)
c10 = tf.keras.layers.Conv2D(128, (3, 3), activation=tf.keras.activations.elu, kernel_initializer='he_normal',padding='same')(c10)
u11 = tf.keras.layers.Conv2DTranspose(64, (2, 2), strides=(2, 2), padding='same')(c10)
u11 = tf.keras.layers.concatenate([u11, c3], axis=3)
c11 = tf.keras.layers.Conv2D(64, (3, 3), activation=tf.keras.activations.elu, kernel_initializer='he_normal',padding='same')(u11)
c11 = tf.keras.layers.Dropout(0.3)(c11)
c11 = tf.keras.layers.Conv2D(64, (3, 3), activation=tf.keras.activations.elu, kernel_initializer='he_normal',padding='same')(c11)
u12 = tf.keras.layers.Conv2DTranspose(32, (2, 2), strides=(2, 2), padding='same')(c11)
u12 = tf.keras.layers.concatenate([u12, c2], axis=3)
c12 = tf.keras.layers.Conv2D(32, (3, 3), activation=tf.keras.activations.elu, kernel_initializer='he_normal',padding='same')(u12)
c12 = tf.keras.layers.Dropout(0.3)(c12)
c12 = tf.keras.layers.Conv2D(32, (3, 3), activation=tf.keras.activations.elu, kernel_initializer='he_normal',padding='same')(c12)
u13 = tf.keras.layers.Conv2DTranspose(16, (2, 2), strides=(2, 2), padding='same')(c12)
u13 = tf.keras.layers.concatenate([u13, c1], axis=3)
c13 = tf.keras.layers.Conv2D(16, (3, 3), activation=tf.keras.activations.elu, kernel_initializer='he_normal',padding='same')(u13)
c13 = tf.keras.layers.Dropout(0.3)(c13)
c13 = tf.keras.layers.Conv2D(16, (3, 3), activation=tf.keras.activations.elu, kernel_initializer='he_normal',padding='same')(c13)
outputs = tf.keras.layers.Conv2D(1, (1, 1), activation='sigmoid')(c13)
model = tf.keras.Model(inputs=[inputs], outputs=[outputs])
return model
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_4 (InputLayer) [(None, 256, 512, 1) 0
__________________________________________________________________________________________________
conv2d_69 (Conv2D) (None, 256, 512, 16) 160 input_4[0][0]
__________________________________________________________________________________________________
dropout_33 (Dropout) (None, 256, 512, 16) 0 conv2d_69[0][0]
__________________________________________________________________________________________________
conv2d_70 (Conv2D) (None, 256, 512, 16) 2320 dropout_33[0][0]
__________________________________________________________________________________________________
max_pooling2d_15 (MaxPooling2D) (None, 128, 256, 16) 0 conv2d_70[0][0]
__________________________________________________________________________________________________
conv2d_71 (Conv2D) (None, 128, 256, 32) 4640 max_pooling2d_15[0][0]
__________________________________________________________________________________________________
dropout_34 (Dropout) (None, 128, 256, 32) 0 conv2d_71[0][0]
__________________________________________________________________________________________________
conv2d_72 (Conv2D) (None, 128, 256, 32) 9248 dropout_34[0][0]
__________________________________________________________________________________________________
max_pooling2d_16 (MaxPooling2D) (None, 64, 128, 32) 0 conv2d_72[0][0]
__________________________________________________________________________________________________
conv2d_73 (Conv2D) (None, 64, 128, 64) 18496 max_pooling2d_16[0][0]
__________________________________________________________________________________________________
dropout_35 (Dropout) (None, 64, 128, 64) 0 conv2d_73[0][0]
__________________________________________________________________________________________________
conv2d_74 (Conv2D) (None, 64, 128, 64) 36928 dropout_35[0][0]
__________________________________________________________________________________________________
max_pooling2d_17 (MaxPooling2D) (None, 32, 64, 64) 0 conv2d_74[0][0]
__________________________________________________________________________________________________
conv2d_75 (Conv2D) (None, 32, 64, 128) 73856 max_pooling2d_17[0][0]
__________________________________________________________________________________________________
dropout_36 (Dropout) (None, 32, 64, 128) 0 conv2d_75[0][0]
__________________________________________________________________________________________________
conv2d_76 (Conv2D) (None, 32, 64, 128) 147584 dropout_36[0][0]
__________________________________________________________________________________________________
max_pooling2d_18 (MaxPooling2D) (None, 16, 32, 128) 0 conv2d_76[0][0]
__________________________________________________________________________________________________
conv2d_77 (Conv2D) (None, 16, 32, 256) 295168 max_pooling2d_18[0][0]
__________________________________________________________________________________________________
dropout_37 (Dropout) (None, 16, 32, 256) 0 conv2d_77[0][0]
__________________________________________________________________________________________________
conv2d_78 (Conv2D) (None, 16, 32, 256) 590080 dropout_37[0][0]
__________________________________________________________________________________________________
max_pooling2d_19 (MaxPooling2D) (None, 8, 16, 256) 0 conv2d_78[0][0]
__________________________________________________________________________________________________
conv2d_79 (Conv2D) (None, 8, 16, 512) 1180160 max_pooling2d_19[0][0]
__________________________________________________________________________________________________
dropout_38 (Dropout) (None, 8, 16, 512) 0 conv2d_79[0][0]
__________________________________________________________________________________________________
conv2d_80 (Conv2D) (None, 8, 16, 512) 2359808 dropout_38[0][0]
__________________________________________________________________________________________________
conv2d_transpose_15 (Conv2DTran (None, 16, 32, 256) 524544 conv2d_80[0][0]
__________________________________________________________________________________________________
concatenate_15 (Concatenate) (None, 16, 32, 512) 0 conv2d_transpose_15[0][0]
conv2d_78[0][0]
__________________________________________________________________________________________________
conv2d_81 (Conv2D) (None, 16, 32, 256) 1179904 concatenate_15[0][0]
__________________________________________________________________________________________________
dropout_39 (Dropout) (None, 16, 32, 256) 0 conv2d_81[0][0]
__________________________________________________________________________________________________
conv2d_82 (Conv2D) (None, 16, 32, 256) 590080 dropout_39[0][0]
__________________________________________________________________________________________________
conv2d_transpose_16 (Conv2DTran (None, 32, 64, 128) 131200 conv2d_82[0][0]
__________________________________________________________________________________________________
concatenate_16 (Concatenate) (None, 32, 64, 256) 0 conv2d_transpose_16[0][0]
conv2d_76[0][0]
__________________________________________________________________________________________________
conv2d_83 (Conv2D) (None, 32, 64, 128) 295040 concatenate_16[0][0]
__________________________________________________________________________________________________
dropout_40 (Dropout) (None, 32, 64, 128) 0 conv2d_83[0][0]
__________________________________________________________________________________________________
conv2d_84 (Conv2D) (None, 32, 64, 128) 147584 dropout_40[0][0]
__________________________________________________________________________________________________
conv2d_transpose_17 (Conv2DTran (None, 64, 128, 64) 32832 conv2d_84[0][0]
__________________________________________________________________________________________________
concatenate_17 (Concatenate) (None, 64, 128, 128) 0 conv2d_transpose_17[0][0]
conv2d_74[0][0]
__________________________________________________________________________________________________
conv2d_85 (Conv2D) (None, 64, 128, 64) 73792 concatenate_17[0][0]
__________________________________________________________________________________________________
dropout_41 (Dropout) (None, 64, 128, 64) 0 conv2d_85[0][0]
__________________________________________________________________________________________________
conv2d_86 (Conv2D) (None, 64, 128, 64) 36928 dropout_41[0][0]
__________________________________________________________________________________________________
conv2d_transpose_18 (Conv2DTran (None, 128, 256, 32) 8224 conv2d_86[0][0]
__________________________________________________________________________________________________
concatenate_18 (Concatenate) (None, 128, 256, 64) 0 conv2d_transpose_18[0][0]
conv2d_72[0][0]
__________________________________________________________________________________________________
conv2d_87 (Conv2D) (None, 128, 256, 32) 18464 concatenate_18[0][0]
__________________________________________________________________________________________________
dropout_42 (Dropout) (None, 128, 256, 32) 0 conv2d_87[0][0]
__________________________________________________________________________________________________
conv2d_88 (Conv2D) (None, 128, 256, 32) 9248 dropout_42[0][0]
__________________________________________________________________________________________________
conv2d_transpose_19 (Conv2DTran (None, 256, 512, 16) 2064 conv2d_88[0][0]
__________________________________________________________________________________________________
concatenate_19 (Concatenate) (None, 256, 512, 32) 0 conv2d_transpose_19[0][0]
conv2d_70[0][0]
__________________________________________________________________________________________________
conv2d_89 (Conv2D) (None, 256, 512, 16) 4624 concatenate_19[0][0]
__________________________________________________________________________________________________
dropout_43 (Dropout) (None, 256, 512, 16) 0 conv2d_89[0][0]
__________________________________________________________________________________________________
conv2d_90 (Conv2D) (None, 256, 512, 16) 2320 dropout_43[0][0]
__________________________________________________________________________________________________
conv2d_91 (Conv2D) (None, 256, 512, 1) 17 conv2d_90[0][0]
==================================================================================================
Total params: 7,775,313
Trainable params: 7,775,313
Non-trainable params: 0
_________________________________________________________
from keras import backend as K
def dice_coef(y_true, y_pred, smooth=1):
y_true_f = K.flatten(y_true)
y_pred_f = K.flatten(y_pred)
intersection = K.sum(y_true_f * y_pred_f)
return (2. * intersection + smooth) / (K.sum(y_true_f) + K.sum(y_pred_f) + smooth)
def dice_coef_loss(y_true, y_pred):
return 1-dice_coef(y_true, y_pred)
from tensorflow.keras.callbacks import ModelCheckpoint
from tensorflow.keras.callbacks import CSVLogger
from tensorflow.keras.callbacks import EarlyStopping
from tensorflow.keras.optimizers import Adam
NO_OF_TRAINING_IMAGES = len(os.listdir('/gdrive/My Drive/Train/img/images/'))
NO_OF_VAL_IMAGES = len(os.listdir('/gdrive/My Drive/Validation/img/images/'))
NO_OF_EPOCHS = 1
BATCH_SIZE = 32
filepath="weights-improvement-{epoch:02d}-{val_accuracy:.2f}.hdf5"
m = unet()
opt = Adam(lr=1E-5, beta_1=0.9, beta_2=0.999, epsilon=1e-08)
m.compile(optimizer=opt,loss=dice_coef_loss, metrics=[dice_coef])
checkpoint = ModelCheckpoint(filepath, monitor=dice_coef_loss,
verbose=1, save_best_only=True, mode='min')
earlystopping = EarlyStopping(monitor = dice_coef_loss, verbose = 1,
min_delta = 0.01, patience = 1, mode ='min')
callbacks_list = [checkpoint,earlystopping]
results = m.fit_generator(train_gen, epochs=NO_OF_EPOCHS,
steps_per_epoch = (NO_OF_TRAINING_IMAGES//BATCH_SIZE),
validation_data=val_gen,
validation_steps=(NO_OF_VAL_IMAGES//BATCH_SIZE),
use_multiprocessing=False,
workers=1)
m.save('Model.h5')
418/418 [==============================] - 9828s 24s/step - loss: 0.0700 - dice_coef: 0.9300 - val_loss: 0.0299 - val_dice_coef: 0.9701
but wen i take the output everything is just blank. I am scaling up the output by multiplying it by 255 before visualizing and batch normalization is also off
Your output is likely normalized between integer values 0-20~ which requires scaling up these values to 0-255 range prior to visualization.
Furthermore, make sure to turn off batch normalization by indicating that the model is running in inference mode.
Supposing out is your output from the model
img1 = out[0,:,:,:] # select first element from our batch
img1 = img1.permute(1,2,0) # model outputs channel in first dim but to visualize we need it in last dim
matplotlib.imshow( img1 )
I am trying to get a neural network model like that:
input
|
hidden
/ \
hidden output2
|
output1
Here is a simple example in code:
model = Sequential()
model.add(Conv2D(32, kernel_size=(3, 3),
activation='relu',
input_shape=input_shape))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Flatten()) # from here I would like to add a new neural network
model.add(Dense(128, activation='relu'))
model.add(Dense(num_classes, activation='softmax'))
How to get an expected model?
Please sorry if I ask a stupid question, I am a very beginner in artificial intelligence.
You could make use of keras functional APIs instead of sequential APIs to get it done as shown below:
from keras.models import Model
from keras.layers import Input
from keras.layers import Dense
from keras.layers import Flatten
from keras.layers.convolutional import Conv2D
from keras.layers.pooling import MaxPooling2D
num_classes = 10
inp= Input(shape=input_shape)
conv1 = Conv2D(32, kernel_size=(3,3), activation='relu')(inp)
conv2 = Conv2D(64, (3, 3), activation='relu')(conv1)
max_pool = MaxPooling2D(pool_size=(2, 2))(conv2)
flat = Flatten()(max_pool)
hidden1 = Dense(128, activation='relu')(flat)
output1 = Dense(num_classes, activation='softmax')(hidden1)
hidden2 = Dense(10, activation='relu')(flat) #specify the number of hidden units
output2 = Dense(3, activation='softmax')(hidden2) #specify the number of classes
model = Model(inputs=inp, outputs=[output1 ,output2])
your network looks like this:
Model: "model_1"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_7 (InputLayer) (None, 64, 256, 256) 0
__________________________________________________________________________________________________
conv2d_10 (Conv2D) (None, 62, 254, 32) 73760 input_7[0][0]
__________________________________________________________________________________________________
conv2d_11 (Conv2D) (None, 60, 252, 64) 18496 conv2d_10[0][0]
__________________________________________________________________________________________________
max_pooling2d_4 (MaxPooling2D) (None, 30, 126, 64) 0 conv2d_11[0][0]
__________________________________________________________________________________________________
flatten_4 (Flatten) (None, 241920) 0 max_pooling2d_4[0][0]
__________________________________________________________________________________________________
dense_6 (Dense) (None, 128) 30965888 flatten_4[0][0]
__________________________________________________________________________________________________
dense_8 (Dense) (None, 10) 2419210 flatten_4[0][0]
__________________________________________________________________________________________________
dense_7 (Dense) (None, 10) 1290 dense_6[0][0]
__________________________________________________________________________________________________
dense_9 (Dense) (None, 3) 33 dense_8[0][0]
==================================================================================================
Total params: 33,478,677
Trainable params: 33,478,677
Non-trainable params: 0
i'm trying to adapt the 2d convolutional autoencoder example from the keras website: https://blog.keras.io/building-autoencoders-in-keras.html
to my own case where i use 1d inputs:
from keras.layers import Input, Dense, Conv1D, MaxPooling1D, UpSampling1D
from keras.models import Model
from keras import backend as K
import scipy as scipy
import numpy as np
mat = scipy.io.loadmat('edata.mat')
emat = mat['edata']
input_img = Input(shape=(64,1)) # adapt this if using `channels_first` image data format
x = Conv1D(32, (9), activation='relu', padding='same')(input_img)
x = MaxPooling1D((4), padding='same')(x)
x = Conv1D(16, (9), activation='relu', padding='same')(x)
x = MaxPooling1D((4), padding='same')(x)
x = Conv1D(8, (9), activation='relu', padding='same')(x)
encoded = MaxPooling1D(4, padding='same')(x)
x = Conv1D(8, (9), activation='relu', padding='same')(encoded)
x = UpSampling1D((4))(x)
x = Conv1D(16, (9), activation='relu', padding='same')(x)
x = UpSampling1D((4))(x)
x = Conv1D(32, (9), activation='relu')(x)
x = UpSampling1D((4))(x)
decoded = Conv1D(1, (9), activation='sigmoid', padding='same')(x)
autoencoder = Model(input_img, decoded)
autoencoder.compile(optimizer='adadelta', loss='binary_crossentropy')
x_train = emat[:,0:80000]
x_train = np.reshape(x_train, (x_train.shape[1], 64, 1))
x_test = emat[:,80000:120000]
x_test = np.reshape(x_test, (x_test.shape[1], 64, 1))
from keras.callbacks import TensorBoard
autoencoder.fit(x_train, x_train,
epochs=50,
batch_size=128,
shuffle=True,
validation_data=(x_test, x_test),
callbacks=[TensorBoard(log_dir='/tmp/autoencoder')])
however, i receive this error when i try to run the autoencoder.fit():
ValueError: Error when checking target: expected conv1d_165 to have
shape (None, 32, 1) but got array with shape (80000, 64, 1)
i know i'm probably doing something wrong when i set up my layers, i just changed the maxpool and conv2d sizes to a 1d form...i have very little experience with keras or autoencoders, anyone see what i'm doing wrong?
thanks
EDIT:
the error when i run it on a fresh console:
ValueError: Error when checking target: expected conv1d_7 to have
shape (None, 32, 1) but got array with shape (80000, 64, 1)
here is the output of autoencoder.summary()
Layer (type) Output Shape Param #
=================================================================
input_1 (InputLayer) (None, 64, 1) 0
_________________________________________________________________
conv1d_1 (Conv1D) (None, 64, 32) 320
_________________________________________________________________
max_pooling1d_1 (MaxPooling1 (None, 16, 32) 0
_________________________________________________________________
conv1d_2 (Conv1D) (None, 16, 16) 4624
_________________________________________________________________
max_pooling1d_2 (MaxPooling1 (None, 4, 16) 0
_________________________________________________________________
conv1d_3 (Conv1D) (None, 4, 8) 1160
_________________________________________________________________
max_pooling1d_3 (MaxPooling1 (None, 1, 8) 0
_________________________________________________________________
conv1d_4 (Conv1D) (None, 1, 8) 584
_________________________________________________________________
up_sampling1d_1 (UpSampling1 (None, 4, 8) 0
_________________________________________________________________
conv1d_5 (Conv1D) (None, 4, 16) 1168
_________________________________________________________________
up_sampling1d_2 (UpSampling1 (None, 16, 16) 0
_________________________________________________________________
conv1d_6 (Conv1D) (None, 8, 32) 4640
_________________________________________________________________
up_sampling1d_3 (UpSampling1 (None, 32, 32) 0
_________________________________________________________________
conv1d_7 (Conv1D) (None, 32, 1) 289
=================================================================
Total params: 12,785
Trainable params: 12,785
Non-trainable params: 0
_________________________________________________________________
Since the autoencoder output should reconstruct the input, a minimum requirement is that their dimensions should match, right?
Looking at your autoencoder.summary(), it is easy to confirm that this is not the case: your input is of shape (64,1), while the output of your last convolutional layer conv1d_7 is (32,1) (we ignore the None in the first dimension, since they refer to the batch size).
Let's have a look at the example in the Keras blog you link to (it is a 2D autoencoder, but the idea is the same):
from keras.layers import Input, Dense, Conv2D, MaxPooling2D, UpSampling2D
from keras.models import Model
from keras import backend as K
input_img = Input(shape=(28, 28, 1)) # adapt this if using `channels_first` image data format
x = Conv2D(16, (3, 3), activation='relu', padding='same')(input_img)
x = MaxPooling2D((2, 2), padding='same')(x)
x = Conv2D(8, (3, 3), activation='relu', padding='same')(x)
x = MaxPooling2D((2, 2), padding='same')(x)
x = Conv2D(8, (3, 3), activation='relu', padding='same')(x)
encoded = MaxPooling2D((2, 2), padding='same')(x)
# at this point the representation is (4, 4, 8) i.e. 128-dimensional
x = Conv2D(8, (3, 3), activation='relu', padding='same')(encoded)
x = UpSampling2D((2, 2))(x)
x = Conv2D(8, (3, 3), activation='relu', padding='same')(x)
x = UpSampling2D((2, 2))(x)
x = Conv2D(16, (3, 3), activation='relu')(x)
x = UpSampling2D((2, 2))(x)
decoded = Conv2D(1, (3, 3), activation='sigmoid', padding='same')(x)
autoencoder = Model(input_img, decoded)
autoencoder.compile(optimizer='adadelta', loss='binary_crossentropy')
Here is the result of autoencoder.summary() in this case:
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_1 (InputLayer) (None, 28, 28, 1) 0
_________________________________________________________________
conv2d_1 (Conv2D) (None, 28, 28, 16) 160
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 14, 14, 16) 0
_________________________________________________________________
conv2d_2 (Conv2D) (None, 14, 14, 8) 1160
_________________________________________________________________
max_pooling2d_2 (MaxPooling2 (None, 7, 7, 8) 0
_________________________________________________________________
conv2d_3 (Conv2D) (None, 7, 7, 8) 584
_________________________________________________________________
max_pooling2d_3 (MaxPooling2 (None, 4, 4, 8) 0
_________________________________________________________________
conv2d_4 (Conv2D) (None, 4, 4, 8) 584
_________________________________________________________________
up_sampling2d_1 (UpSampling2 (None, 8, 8, 8) 0
_________________________________________________________________
conv2d_5 (Conv2D) (None, 8, 8, 8) 584
_________________________________________________________________
up_sampling2d_2 (UpSampling2 (None, 16, 16, 8) 0
_________________________________________________________________
conv2d_6 (Conv2D) (None, 14, 14, 16) 1168
_________________________________________________________________
up_sampling2d_3 (UpSampling2 (None, 28, 28, 16) 0
_________________________________________________________________
conv2d_7 (Conv2D) (None, 28, 28, 1) 145
=================================================================
Total params: 4,385
Trainable params: 4,385
Non-trainable params: 0
It is easy to confirm that here the dimensions of the input and the output (last convolutional layer conv2d_7) are indeed both (28, 28, 1).
So, the summary() method is your friend when building autoencoders; you should experiment with the parameters until you are sure that you produce an output of the same dimensionality as your input. I managed to do so with your autoencoder simply by changing the size argument of the last UpSampling1D layer from 4 to 8:
input_img = Input(shape=(64,1))
x = Conv1D(32, (9), activation='relu', padding='same')(input_img)
x = MaxPooling1D((4), padding='same')(x)
x = Conv1D(16, (9), activation='relu', padding='same')(x)
x = MaxPooling1D((4), padding='same')(x)
x = Conv1D(8, (9), activation='relu', padding='same')(x)
encoded = MaxPooling1D(4, padding='same')(x)
x = Conv1D(8, (9), activation='relu', padding='same')(encoded)
x = UpSampling1D((4))(x)
x = Conv1D(16, (9), activation='relu', padding='same')(x)
x = UpSampling1D((4))(x)
x = Conv1D(32, (9), activation='relu')(x)
x = UpSampling1D((8))(x) ## <-- change here (was 4)
decoded = Conv1D(1, (9), activation='sigmoid', padding='same')(x)
autoencoder = Model(input_img, decoded)
autoencoder.compile(optimizer='adadelta', loss='binary_crossentropy')
In which case, the autoencoder.summary() becomes:
Layer (type) Output Shape Param #
=================================================================
input_1 (InputLayer) (None, 64, 1) 0
_________________________________________________________________
conv1d_1 (Conv1D) (None, 64, 32) 320
_________________________________________________________________
max_pooling1d_1 (MaxPooling1 (None, 16, 32) 0
_________________________________________________________________
conv1d_2 (Conv1D) (None, 16, 16) 4624
_________________________________________________________________
max_pooling1d_2 (MaxPooling1 (None, 4, 16) 0
_________________________________________________________________
conv1d_3 (Conv1D) (None, 4, 8) 1160
_________________________________________________________________
max_pooling1d_3 (MaxPooling1 (None, 1, 8) 0
_________________________________________________________________
conv1d_4 (Conv1D) (None, 1, 8) 584
_________________________________________________________________
up_sampling1d_1 (UpSampling1 (None, 4, 8) 0
_________________________________________________________________
conv1d_5 (Conv1D) (None, 4, 16) 1168
_________________________________________________________________
up_sampling1d_2 (UpSampling1 (None, 16, 16) 0
_________________________________________________________________
conv1d_6 (Conv1D) (None, 8, 32) 4640
_________________________________________________________________
up_sampling1d_3 (UpSampling1 (None, 64, 32) 0
_________________________________________________________________
conv1d_7 (Conv1D) (None, 64, 1) 289
=================================================================
Total params: 12,785
Trainable params: 12,785
Non-trainable params: 0
with the dimensionality of your input and output matched, as it should be...