I want to concatenate these two models in my project, I am quite new in this field so please don't judge me hard. So here is the code.
model2 = Sequential()
model2.add(Dense(10, input_dim=df2_x.shape[1], activation='relu'))
model2.add(Dense(50, input_dim=df2_x.shape[1], activation='relu'))
model2.add(Dense(10, input_dim=df2_x.shape[1], activation='relu'))
model2.add(Dense(1, kernel_initializer='normal'))
model2.add(Dense(df2_y.shape[1],activation='softmax'))
model2.compile(loss='categorical_crossentropy', optimizer='adam')
monitor2 = EarlyStopping(monitor='val_loss', min_delta=1e-3,
patience=5, verbose=1, mode='auto',
restore_best_weights=True)
model2.fit(df2_x_train,df2_y_train,validation_data=(df2_x_test, df2_y_test),
callbacks=[monitor2],verbose=2,epochs=1000)
model = Sequential()
model.add(Dense(10, input_dim=df_x.shape[1], activation='relu'))
model.add(Dense(50, input_dim=df_x.shape[1], activation='relu'))
model.add(Dense(10, input_dim=df_x.shape[1], activation='relu'))
model.add(Dense(1, kernel_initializer='normal'))
model.add(Dense(df_y.shape[1],activation='softmax'))
model.compile(loss='categorical_crossentropy', optimizer='adam')
monitor = EarlyStopping(monitor='val_loss', min_delta=1e-3,
patience=5, verbose=1, mode='auto',
restore_best_weights=True)
model.fit(df_x_train,df_y_train,validation_data=(df_x_test, df_y_test),
callbacks=[monitor],verbose=2,epochs=1000)
And after the model obtained, I want to make predictions.
So I have two datasets, one for DOS-portmap attacks, and one for DOS-UDP attacks.
If I want to predict something, how can I distinguish between these two?
You can use Keras functional API for this. Model is basically same like a layer, it's callable, and can be included in another model.
For example, this is greatly simplified version of concatenating results from 2 Models:
inputs = keras.Input(input_shape)
y_1 = model1(inputs)
y_2 = model2(inputs)
outputs = tf.concat([y_1, y_2], axis=0)
new_model = keras.Model(inputs, outputs)
Of course, you want to make sure that the outputs is your desired result. Therefore, the key is in the last layer, what kind of operations you want to do with the values you get from y_1 and y_2.
Some use custom layer for this, you can use that as well. You can look for more article about Functional API and Custom Layer in Tensorflow doc.
Related
In regard to this article:
Chen, D., Zhou, R., Pan, Y., & Liu, F. (2022). A Simple Baseline for Adversarial Domain Adaptation-based Unsupervised Flood Forecasting. arXiv preprint arXiv:2206.08105.
The authors describe two models. The first model is a 1D-CNN "encoder" with three layers. The second model is a "prediction head". It is also a 1D-CNN with three layers.
How to implement this?
For example, I'd start by creating two Sequential() models, each with three Conv1D layers and number of filters and kernel size as specified. Next step would be to train the encoder models on the source and target datasets. But what comes next?
For example:
# Encoder model
encoder_model = Sequential()
encoder_model.add(Conv1D(filters=30, kernel_size=2, activation='relu',
input_shape=(n_timesteps, n_features)))
encoder_model.add(Dropout(0.2))
encoder_model.add(Conv1D(filters=30, kernel_size=2, activation='relu'))
encoder_model.add(Dropout(0.2))
encoder_model.add(Conv1D(filters=30, kernel_size=2, activation='relu'))
encoder_model.add(Dropout(0.2))
# Prediction head model
ph_model = Sequential()
ph_model.add(Conv1D(filters=36, kernel_size=2, activation='relu',
input_shape=(n_timesteps, n_features)))
ph_model.add(Conv1D(filters=36, kernel_size=2, activation='relu'))
ph_model.add(Conv1D(filters=1, kernel_size=3))
There is also a "residual connection" in the prediction head. But how to add that?
The article includes this diagram:
How would this look when programmed, for example, in keras?
If I want to train a tensorflow machine learning model and store the model after each training epoch on the hard drive, I can either use the following code (Python):
checkpoint = ModelCheckpoint('model{epoch:08d}.h5', save_freq=1)
history = model.fit(train_it, steps_per_epoch=len(train_it), validation_data=test_it, validation_steps=len(test_it), epochs=numberOfTrainingEpochs, verbose=0, callbacks=checkpoint)
Or, however, I can use a custom, potentially more complex logic which decides when to save the model:
class CustomSaver(Callback):
def on_epoch_end(self, epoch, logs={}):
self.model.save_weights("model_{}.h5".format(epoch))
saver = CustomSaver()
history = model.fit(train_it, steps_per_epoch=len(train_it), validation_data=test_it, validation_steps=len(test_it), epochs=numberOfTrainingEpochs, verbose=0, callbacks=saver)
Both files create .h5 files with the ML model, however, the first one creates file sizes of ca. 100 MB, whereas the second one creates file sizes of ca. 50 MB. What is the difference between those files and what is the cause for it?
Fyi, my model is a relatively simple CNN and defined as follows:
model = Sequential()
model.add(Conv2D(32, (3, 3), activation='relu', kernel_initializer='he_uniform', padding='same', input_shape=(224, 224, 3)))
model.add(MaxPooling2D((2, 2)))
model.add(Conv2D(64, (3, 3), activation='relu', kernel_initializer='he_uniform', padding='same'))
model.add(MaxPooling2D((2, 2)))
model.add(Conv2D(128, (3, 3), activation='relu', kernel_initializer='he_uniform', padding='same'))
model.add(MaxPooling2D((2, 2)))
model.add(Flatten())
model.add(Dense(128, activation='relu', kernel_initializer='he_uniform'))
model.add(Dense(1, activation='sigmoid'))
opt = SGD(lr=0.001, momentum=0.9)
model.compile(optimizer=opt, loss='binary_crossentropy', metrics=['accuracy'])
According to the documentation, the callback ModelCheckpoint saves the full model rather than only its weights by default. This behaviour is controlled by parameter save_weights_only. If you only want to save the weights, you can create the callback with
checkpoint = ModelCheckpoint('model{epoch:08d}.h5', save_freq=1, save_weights_only=True)
I wanna train a CNN using SVM to classify at the last layer. I understand that the categorical_hinge is the best loss function for that . I have 6 classes to classify .
My model is as shown below:
model = Sequential()
model.add(Conv2D(50, 3, 3, activation = 'relu', input_shape = train_data.shape[1:]))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(50, 3, 3, activation = 'relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(50, 3, 3, activation = 'relu'))
model.add(Flatten())
model.add(Dense(400, activation = 'relu'))
model.add(Dropout(0.5))
model.add(Dense(128, activation = 'relu'))
model.add(Dropout(0.5))
model.add(Dense(1, activation = 'sigmoid'))
Is there a problem with the network , data processing , or the loss function?
The model does not learn anything after a point as shown in the image
What should I do?
Your model has a single output neuron, there is no way this will work with 6 classes. The output of your model should have 6 neurons. Also the output of your model should have no activation function in order to produce logits that the categorical hinge can use.
Note that the categorical hinge was added recently (2-3 weeks ago) so its quite new and probably not many people have tested it.
Use hinge loss in and linear activation in last layer.
model.add(Dense(nb_classes), W_regularizer=l2(0.01))
model.add(Activation('linear'))
model.compile(loss='hinge',
optimizer='adadelta',
metrics=['accuracy'])
for more information visit https://github.com/keras-team/keras/issues/6090
I am using Keras on top of Theano to create a MLP which I train and use to predict time series. Independently of the structure and depth of my network I cannot figure out (Keras documentation, StackOverflow, searching the net...) which training algorithm (Backpropagation,...) Keras' model.fit() function is using.
Within Theano (used without Keras before) I could define the way the parameters are adjusted myself with
self.train_step = theano.function(inputs=[u_in, t_in, lrate], outputs=[cost, y],
on_unused_input='warn',
updates=[(p, p - lrate * g) for p, g in zip(self.parameters, self.gradients)],
allow_input_downcast=True)
Not finding any information causes a certain fear that I am missing something essential and that this may be a totally stupid question.
Can anybody help me out here? Thanks a lot in advance.
Look at the example here:
...
model = Sequential()
model.add(Conv2D(32, kernel_size=(3, 3),
activation='relu',
input_shape=input_shape))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(128, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(num_classes, activation='softmax'))
model.compile(loss=keras.losses.categorical_crossentropy,
optimizer=keras.optimizers.Adadelta(),
metrics=['accuracy'])
model.fit(x_train, y_train,
batch_size=batch_size,
epochs=epochs,
verbose=1,
validation_data=(x_test, y_test))
score = model.evaluate(x_test, y_test, verbose=0)
...
model.fit does not use an algorithm to predict the outcome, rather it uses the model you describe. The optimiser algorithm is then specified in model.compile
e.g.
model.compile(loss=keras.losses.categorical_crossentropy,
optimizer=**keras.optimizers.Adadelta()**,
metrics=['accuracy'])
You can find out more about the available optimisers here : https://keras.io/optimizers/
I'm writing a program to classify texts into a few classes. Right now, the program loads the train and test samples of word indices, applies an embedding layer and a convolutional layer, and classifies them into the classes. I'm trying to add handcrafted features for experimentation, as in the following code. The features is a list of two elements, where the first element consists of features for the training data, and the second consists of features for the test data. Each training/test sample will have a corresponding feature vector (i.e. the features are not word features).
model = Sequential()
model.add(Embedding(params.nb_words,
params.embedding_dims,
weights=[embedding_matrix],
input_length=params.maxlen,
trainable=params.trainable))
model.add(Convolution1D(nb_filter=params.nb_filter,
filter_length=params.filter_length,
border_mode='valid',
activation='relu'))
model.add(Dropout(params.dropout_rate))
model.add(GlobalMaxPooling1D())
# Adding hand-picked features
model_features = Sequential()
nb_features = len(features[0][0])
model_features.add(Dense(1,
input_shape=(nb_features,),
init='uniform',
activation='relu'))
model_final = Sequential()
model_final.add(Merge([model, model_features], mode='concat'))
model_final.add(Dense(len(citfunc.funcs), activation='softmax'))
model_final.compile(loss='categorical_crossentropy',
optimizer='adam',
metrics=['accuracy'])
print model_final.summary()
model_final.fit([x_train, features[0]], y_train,
nb_epoch=params.nb_epoch,
batch_size=params.batch_size,
class_weight=data.get_class_weights(x_train, y_train))
y_pred = model_final.predict([x_test, features[1]])
My question is, is this code correct? Is there any conventional way of adding features to each of the text sequences?
Try:
input = Input(shape=(params.maxlen,))
embedding = Embedding(params.nb_words,
params.embedding_dims,
weights=[embedding_matrix],
input_length=params.maxlen,
trainable=params.trainable)(input)
conv = Convolution1D(nb_filter=params.nb_filter,
filter_length=params.filter_length,
border_mode='valid',
activation='relu')(embedding)
drop = Dropout(params.dropout_rate)(conv)
seq_features = GlobalMaxPooling1D()(drop)
# Adding hand-picked features
nb_features = len(features[0][0])
other_features = Input(shape=(nb_features,))
model_final = merge([seq_features , other_features], mode='concat'))
model_final = Dense(len(citfunc.funcs), activation='softmax'))(model_final)
model_final = Model([input, other_features], model_final)
model_final.compile(loss='categorical_crossentropy',
optimizer='adam',
metrics=['accuracy'])
In this case - you are merging features from a sequence analysis with custom features directly - without squashing all custom features to 1 features using Dense.