Horrible forecasting KPI scores (RMSE, MAE and MAPE) using RNN with three flavors ("LSTM", "GRU" and "RNN") - time-series

Question

My project is trying to forecast COVID 19 total confirmed case for 2021. This is the overlook of the confirm case data, which I use to train my RNN model.
enter image description here
The confirmed number though doesn't show any repeating pattern. But there has been research studies using RNN and LSTM on the same data (in fact, I use the same data source as them). This is the research study I drew inspiration from:
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8523546/
This is the result I got:
beginning the training of the LSTM RNN:
Epoch 99: 100%|██████████| 29/29 [00:00<00:00, 33.10it/s, loss=0.115, v_num=logs, train_loss=0.0933, val_loss=0.466]
training of the LSTM RNN completed: 105.23 sec
Predicting DataLoader 0: 100%|██████████| 1/1 [00:00<00:00, 6.48it/s]
LSTM :
MAPE : 199.3029
RMSPE : 0.6659
RMSE : 0.6763
-R squared : 15366746993394475597824.0000
se : 0.0871
beginning the training of the GRU RNN:
Epoch 99: 100%|██████████| 29/29 [00:00<00:00, 37.86it/s, loss=0.11, v_num=logs, train_loss=0.0911, val_loss=0.474]
training of the GRU RNN completed: 90.99 sec
Predicting DataLoader 0: 100%|██████████| 1/1 [00:00<00:00, 3.91it/s]
GRU :
MAPE : 209.6602
RMSPE : 0.6771
RMSE : 0.6877
-R squared : 467975998695823638528.0000
se : 0.0871
beginning the training of the Vanilla RNN:
Epoch 99: 100%|██████████| 29/29 [00:00<00:00, 43.54it/s, loss=0.114, v_num=logs, train_loss=0.109, val_loss=0.461]
training of the Vanilla RNN completed: 79.32 sec
Predicting DataLoader 0: 100%|██████████| 1/1 [00:00<00:00, 6.08it/s]
Vanilla :
MAPE : 200.5710
RMSPE : 0.6673
RMSE : 0.6778
-R squared : 115942252717688101559392358367232.0000
se : 0.0871
Also, here is my prediction plot. Three of the plot looks exactly the same, different MAPE score:
enter image description here
For the package i use, i use darts (u8darts[all] when importing the packages).
for methodology, I set the parameters for my model as this. I used this article by Heiko Oinen for methodology (detailed code is below the page):
https://medium.com/towards-data-science/temporal-loops-intro-to-recurrent-neural-networks-for-time-series-forecasting-in-python-b0398963dc1f:
#Set up the models, run the models, plot and evaluate
EPOCH = 100
def run_RNN(flavor, ts, train, val):
#set the model up
model_RNN = RNNModel(
model = flavor,
model_name = flavor + str(" RNN"),
input_chunk_length = 12,
training_length = 20,
hidden_dim = 20,
batch_size = 16,
n_epochs = EPOCH,
dropout = 0,
optimizer_kwargs = {'lr': 1e-3},
log_tensorboard = True,
random_state = 42,
force_reset = True
)
if flavor == "RNN": flavor = "Vanilla"
#fit the model
fit_it(model_RNN, train, val, flavor)
#compute N predictions
pred = model_RNN.predict(n = FC_N,future_covariates = covariates)
#plot predictions vs actual
plot_fitted(pred, ts, flavor)
#print accuracy metrics
res_acc = accuracy_metrics(pred, ts)
I have even tries epoch to 300, but the train loss and loss during training didn't decrease further.
I haven't gotten experience in asking questions here, I'll try to articulate and provide more detail if you have questions. Thank you so much for your help!