Understanding how many hidden layer in given LSTM model - machine-learning

Question

I do not able to understand the basic structure of LSTM model.
Here is mine model:
def build_model(train,n_input):
train_x, train_y = to_supervised(train, n_input)
verbose, epochs, batch_size = 1, 60,20
n_timesteps, n_features, n_outputs = train_x.shape[1], train_x.shape[2], train_y.shape[1]
train_y = train_y.reshape((train_y.shape[0], train_y.shape[1], 1))
model = Sequential()
model.add(LSTM(200, activation='relu', input_shape=(n_timesteps, n_features)))
model.add(RepeatVector(n_outputs))
model.add(LSTM(200, activation='relu', return_sequences=True))
model.add(TimeDistributed(Dense(100, activation='relu')))
model.add(TimeDistributed(Dense(1)))
model.compile(loss='mse', optimizer='adam', metrics=['accuracy'])
model.fit(train_x, train_y, epochs=epochs, batch_size=batch_size, verbose=verbose)
return model
Here is my model.summary()
Layer (type) Output Shape Param #
=================================================================
lstm_5 (LSTM) (None, 200) 172000
_________________________________________________________________
repeat_vector_3 (RepeatVecto (None, 7, 200) 0
_________________________________________________________________
lstm_6 (LSTM) (None, 7, 200) 320800
_________________________________________________________________
time_distributed_5 (TimeDist (None, 7, 100) 20100
_________________________________________________________________
time_distributed_6 (TimeDist (None, 7, 1) 101
=================================================================
Total params: 513,001
Trainable params: 513,001
Non-trainable params: 0
_________________________________________________________________
None
From the above summary, i do not understanding what is lstm_5 or lstm_6. Also it don't tell number of hidden layer in the network
Please someone help me understand that in the above model, how many hidden layer are there with neuron.
I basically confuse by add(LSTM(200 ...) and add(TimeDistributed(Dense(100..)
I think 200 and 100 are the number of neuron in hidden layer and there are 4 hidden layer containing all .add() .
Please correct me and clarify my doubts. If possible try to understand by the diagram.

Answer 1

Pictorial representation of the model architecture to understand how outputs of a layer are attached to the next layer in the sequence.
The picture is self explanatory and it matches your model summary. Also note Batch_Size is None in the model summary as it is calculated dynamically. Also note that in LSTM the size of hidden layer is same as the size of the output of the LSTM.

Answer 2

Here you define an LSTM layer with 200 neurons. The 200-dim vector basically represtens the sequence as an interal embedding:
model.add(LSTM(200, activation='relu', input_shape=(n_timesteps, n_features)))
Therfore you get an output of (None, 200)
Here you repeat the vector 7 times:
model.add(RepeatVector(n_outputs))
You get an vector of (None, 7, 200)
You use this vector again as a sequence and you return the state of the 200 neurons at every timestep, unclear why:
model.add(LSTM(200, activation='relu', return_sequences=True))
You get an vector of (None, 7, 200)
You apply a weight-sharing Dense Layer with 100 Neurons on every time step. I actually don´t know, why the weights are shared here, seems weird:
model.add(TimeDistributed(Dense(100, activation='relu')))
You get an vector of (None, 7, 100)
Finally you apply a last neuron for every of these 7 timestep, again with shared weights, making a single value out of the 100-dim vector. The result is a vector of 7 neurons, one for every class:
model.add(TimeDistributed(Dense(1)))