First time learning keras and I wanted to make a model that can classify pictures between chicken and nature.
train = ImageDataGenerator(rescale=1/255)
validation = ImageDataGenerator(rescale=1/255)
train_data = train.flow_from_directory('./train', target_size=(200, 200), batch_size=3, class_mode='binary')
validation_data = validation.flow_from_directory('./testchickens', target_size=(200, 200), batch_size=3, class_mode='binary')
model = tf.keras.models.Sequential([tf.keras.layers.Conv2D(16,(3,3), activation='relu', input_shape=(200,200,3)),
tf.keras.layers.MaxPool2D(2,2),
tf.keras.layers.Conv2D(32,(3,3), activation='relu'),
tf.keras.layers.MaxPool2D(2,2),
tf.keras.layers.Conv2D(64,(3,3), activation='relu'),
tf.keras.layers.MaxPool2D(2,2),
tf.keras.layers.Flatten(),
tf.keras.layers.Dense(512,activation='relu'),
tf.keras.layers.Dense(1,activation='sigmoid')
])
model.compile(loss='binary_crossentropy', optimizer=RMSprop(learning_rate=0.001), metrics=['accuracy'])
model_fit = model.fit(train_data, steps_per_epoch=3, epochs=10, validation_data = validation_data)
highest score I got was 88.89% in my model. I realized as I was coding the model that I needed 3 separate data sets or files for my images (training, validation, test).
To test my model I downloaded a few pictures off google that weren't in either of my training or validation files but I am not sure how I can test my model on the sample pictures I got. I figured I can convert the image to an array but the output feels off.
sample = glob.glob('./sample/**/*.jpg', recursive=True)
x = sample[0]
img1 = image.load_img(x, target_size=(200, 200, 3))
X = image.img_to_array(img1)
X = np.expand_dims(X, axis=0)
image = np.vstack([X])
model.predict(image)
For some reason my model classifies every single picture as a chicken when clearly I had pictures of trees and forests in my sample or test file. Before the last block of code I ran into a lot of issues with my input not fitting the dimensions/shapes/layers etc so I think if I understood how the structure works on my model maybe I can fix it? If someone can please explain what my issue is or point me to some resource that explains the layering/arrays of keras models I would greatly appreciate.
Related
I am playing with some stocks timeseries data and trying to predict the trend with multivariate features. Below is the sample dataset I have which including different technical indicators including moving average, Parabolic SAR etc for each stocks. From different online sources, most of them are predicting one stock with one feature like "Close" price a time. How can I make use all the stocks' features to predict one output let's say S&P's close price. I know it may not help boosting the prediction accuracy but I am not sure what I am training right now and hope having more insight on LSTM model.
Basically, I put the whole dataset in and do the scaling and training stuffs. How could the prediction being specified on one column?
Code:
scaler = MinMaxScaler(feature_range = (0,1))
scaled_feature_data = scaler.fit_transform(feature_data)
X_train, y_train = training_set[:, :-1], training_set[:, -1]
X_test, y_test = testing_set[:, :-1], testing_set[:, -1]
X_train = X_train.reshape((X_train.shape[0],1,X_train.shape[1]))
X_test = X_test.reshape((X_test.shape[0],1,X_test.shape[1]))
model_lstm.add(LSTM(50, return_sequences = True, input_shape = (X_train.shape[1], X_train.shape[2])))
Model:
model_lstm.add(LSTM(50, return_sequences = True, input_shape = (X_train.shape[1], X_train.shape[2])))
model_lstm.add(Dropout(0.2))
model_lstm.add(LSTM(units=50, return_sequences=True))
model_lstm.add(Dropout(0.2))
model_lstm.add(LSTM(units=50, return_sequences=True))
model_lstm.add(Dropout(0.2))
model_lstm.add(LSTM(units=50))
model_lstm.add(Dropout(0.2))
model_lstm.add(Dense(units=1, activation='relu'))
This is a regression problem. You are predicting one variable. So last layers may look like
model_lstm.add(Dense(1))
model_lstm.compile(loss='mse', optimizer='rmsprop')
Also, you probably don't need to return_sequences for each of input tokens. If return_sequences=true, then an output will be a matrix (actually (batch_size, num_tokens, num_features)), that cannot be automatically flattened to vector (batch_size, num_features) that is expected to be an input of Dense(1) layer. Just use an output of a last LSTM node. For this, set return_sequences=false. Its values depends on previous tokens, so you won't lose much information from them.
The whole model may look like this:
model_lstm = Sequential()
model_lstm.add(LSTM(50, return_sequences=False, input_shape=(X_train.shape[1], X_train.shape[2])))
model_lstm.add(Dropout(0.5))
model_lstm.add(Dense(1))
model_lstm.compile(loss='mse', optimizer='rmsprop')
If you want more layers it will become:
model_lstm = Sequential()
model_lstm.add(LSTM(64, return_sequences=True, dropout=0.5, input_shape=(X_train.shape[1], X_train.shape[-1])))
model_lstm.add(Dropout(0.5))
model_lstm.add(LSTM(32, return_sequences=False, dropout=0.5))
model_lstm.add(Dense(1))
model_lstm.compile(loss='mse', optimizer='rmsprop')
Actually I am wondering how can all features, lets say 10 technical indicator features, can help predict the one price column?
Don't know if i understand you correctly.But thats what a ML does, it tries to find corelations between the features and how they can be used to predict something. So maybe, the "DE30" has (or seems to have) a influence on the price and is therfore helpful. Was that your question?
rom different online sources, most of them are predicting one stock with one feature like "Close" price a time
I guess that for simplification. Therefore they used only one feature
Let my know if that was what you asked for..
I'm new in Keras. I want save model with best weights like as:
model1.compile(loss="mean_squared_error", optimizer="RMSprop")
model1.summary()
mcp_save = ModelCheckpoint('best_model.h5', save_best_only=True, monitor='val_accuracy', mode='auto', verbose=2)
callbacks_list = [mcp_save]
epochs = 5000
batch_size = 50
# fit the model
history = model1.fit(x_train, y_train,
batch_size=batch_size,
epochs=epochs,
callbacks=callbacks_list,
validation_data=(x_test, y_test),
verbose=2)
I couldn't come across warning or error message on Pycharm 2019 Community edition. But I am not able to see 'best_model.h5' on project file folder or somwhere else on my computer after trainig process finished?? Would you give me advices please?? What are my faults??
Your code looks fine to me. I use this callback often. All I can suggest is that you use a full path to designate where to save the model rather than a relative path.
I am attempting to write code to identify data types after loading it in from CSV files. So there are 5 possible labels, and the feature vector contains a list of lists. The feature vector is a list of lists with the following shape:
[slash_count, dash_count, colon_count, letters, dot_count, digits]
I then split my feature and label vectors into training, testing, and validation sets. I found some code on Stackoverflow that someone wrote to do this and I have used the same:
X_train, X_test, y_train, y_test = train_test_split(ml_list, labels, test_size=0.3, random_state=1)
X_train, X_val, y_train, y_val = train_test_split(X_train, y_train, test_size=0.3, random_state=1)
After doing this, I normalize the features in scale [0,1], and then I create the categorical variables for the labels:
min_max_scaler = preprocessing.MinMaxScaler()
X_train_minmax = min_max_scaler.fit_transform(X_train)
X_test_minmax = min_max_scaler.fit_transform(X_test)
X_val_minmax = min_max_scaler.fit_transform(X_val)
from keras.utils import to_categorical
y_train_minmax = to_categorical(y_train)
y_test_minmax = to_categorical(y_test)
y_val_minmax = to_categorical(y_val)
Next, I attempt to find the shape of the newly recoded variables:
print(y_train_minmax.shape) #(91366, 4)
print(X_train_minmax.shape) #(91366, 6)
print(X_test_minmax.shape) #(55939, 6)
print(X_val_minmax.shape) #(39157, 6)
print(y_train_minmax.shape) #(91366, 4)
print(y_test_minmax.shape) #(55939, 4)
print(y_val_minmax.shape) #(39157, 4)
Finally, I build the model and attempt to fit it:
model = models.Sequential()
model.add(layers.Dense(512, activation='relu', input_shape=(91366, 6)))
model.add(layers.Dense(3, activation='softmax'))
model.compile(optimizer='rmsprop', loss='categorical_crossentropy', metrics=['accuracy'])
model.fit(X_train_minmax, y_train_minmax, epochs=5, batch_size=128)
I get this message when I run the code:
ValueError: Error when checking input: expected dense_1_input to have 3 dimensions, but got array with shape (91366, 6)
I believe that the error is in when I create the neural network with the input shape. I am having a hard time understanding where I went wrong. Any help would be great!
You should change this line:
model.add(layers.Dense(512, activation='relu', input_shape=(6,)))
In keras you don't need to directly specify the number of examples you have in your dataset. As input_shape you need to provide only a shape of a single data point.
Another potential error which I spotted in your code snippet is that you should set:
model.add(layers.Dense(4, activation='softmax'))
As your output single data point has a shape of (4,). It's not consistent with what you've said about possible layers so I'd also advise rechecking your data.
Another possible mistake which I spotted is that you are not training separate scalers for train, test and valid datasets - but a single one on a train set - and then scale your other dataset using a trained scaler.
I have trained a constitutional net using transfer learning from ResNet50 in keras as given below.
base_model = applications.ResNet50(weights='imagenet', include_top=False, input_shape=(333, 333, 3))
## set model architechture
x = base_model.output
x = GlobalAveragePooling2D()(x)
x = Dense(1024, activation='relu')(x)
x = Dense(256, activation='relu')(x)
predictions = Dense(y_train.shape[1], activation='softmax')(x)
model = Model(input=base_model.input, output=predictions)
model.compile(loss='categorical_crossentropy', optimizer=optimizers.SGD(lr=1e-4, momentum=0.9),
metrics=['accuracy'])
model.summary()
After training the model as given below I want to save the model.
history = model.fit_generator(
train_datagen.flow(x_train, y_train, batch_size=batch_size),
steps_per_epoch=600,
epochs=epochs,
callbacks=callbacks_list
)
I can't use save_model() function from models of keras as model is of type Model here. I used save() function to save the model. But later when i loaded the model and validated the model it behaved like a untrained model. I think the weights were not saved. What was wrong.? How to save this model properly.?
As per Keras official docs,
If you only need to save the architecture of a model you can use
model_json = model.to_json()
with open("model_arch.json", "w") as json_file:
json_file.write(model_json)
To save weights
model.save_weights("my_model_weights.h5")
You can later load the json file and use
from keras.models import model_from_json
model = model_from_json(json_string)
And similarly, for weights you can use
model.load_weights('my_model_weights.h5')
I am using the same approach and this works perfectly well.
I don't know what happens with my models, but I've never been able to use save_model() and load_model(), there is always an error associated. But these functions exist.
What I usually do is to save and load weights (it's enough for using the model, but may cause a little problem for further training, as the "optimizer" state was not saved, but it was never a big problem, soon a new optimizer finds its way)
model.save_weights(fileName)
model.load_weights(fileName)
Another option us using numpy for saving - this one never failed:
np.save(fileName,model.get_weights())
model.set_weights(np.load(fileName))
For this to work, just create your model again (keep the code you use to create it) and set its weights.
I'm writing a program to classify texts into a few classes. Right now, the program loads the train and test samples of word indices, applies an embedding layer and a convolutional layer, and classifies them into the classes. I'm trying to add handcrafted features for experimentation, as in the following code. The features is a list of two elements, where the first element consists of features for the training data, and the second consists of features for the test data. Each training/test sample will have a corresponding feature vector (i.e. the features are not word features).
model = Sequential()
model.add(Embedding(params.nb_words,
params.embedding_dims,
weights=[embedding_matrix],
input_length=params.maxlen,
trainable=params.trainable))
model.add(Convolution1D(nb_filter=params.nb_filter,
filter_length=params.filter_length,
border_mode='valid',
activation='relu'))
model.add(Dropout(params.dropout_rate))
model.add(GlobalMaxPooling1D())
# Adding hand-picked features
model_features = Sequential()
nb_features = len(features[0][0])
model_features.add(Dense(1,
input_shape=(nb_features,),
init='uniform',
activation='relu'))
model_final = Sequential()
model_final.add(Merge([model, model_features], mode='concat'))
model_final.add(Dense(len(citfunc.funcs), activation='softmax'))
model_final.compile(loss='categorical_crossentropy',
optimizer='adam',
metrics=['accuracy'])
print model_final.summary()
model_final.fit([x_train, features[0]], y_train,
nb_epoch=params.nb_epoch,
batch_size=params.batch_size,
class_weight=data.get_class_weights(x_train, y_train))
y_pred = model_final.predict([x_test, features[1]])
My question is, is this code correct? Is there any conventional way of adding features to each of the text sequences?
Try:
input = Input(shape=(params.maxlen,))
embedding = Embedding(params.nb_words,
params.embedding_dims,
weights=[embedding_matrix],
input_length=params.maxlen,
trainable=params.trainable)(input)
conv = Convolution1D(nb_filter=params.nb_filter,
filter_length=params.filter_length,
border_mode='valid',
activation='relu')(embedding)
drop = Dropout(params.dropout_rate)(conv)
seq_features = GlobalMaxPooling1D()(drop)
# Adding hand-picked features
nb_features = len(features[0][0])
other_features = Input(shape=(nb_features,))
model_final = merge([seq_features , other_features], mode='concat'))
model_final = Dense(len(citfunc.funcs), activation='softmax'))(model_final)
model_final = Model([input, other_features], model_final)
model_final.compile(loss='categorical_crossentropy',
optimizer='adam',
metrics=['accuracy'])
In this case - you are merging features from a sequence analysis with custom features directly - without squashing all custom features to 1 features using Dense.