Caddy worked until last restart - docker

Question

I am currently working on an already started project, with the current situation (completely new to Caddy, so sorry if asking something basic):
A docker container with postgresSQL -- container called myappdb
A Spring Boot docker application with some microservices -- container called backend
A caddy docker container that reverse proxies to Spring boot container -- container called caddy
The three containers are in a docker network called project_net.
I worked on the spring boot backend and everything worked well. Accidentally I stopped the caddy container and restarted it, and now I cannot make rest calls to https server anymore.
Here the Caddyfile:
https://app.myapp.it {
tls myapp#gmail.com
reverse_proxy /* {
to backend:48795
flush interval -1
}
}
Here the Dockerfile for caddy image:
FROM caddy:2.4.5
COPY Caddyfile /etc/caddy/Caddyfile
ENV ACME_AGREE=true
EXPOSE 443
All is running on an apache application server and I thing everything is set up because everything worked well until yesterday!
Here the log of the caddy container on start:
2022-02-24T00:49:13.077709051Z 2022/02/24 00:49:13.077 INFO using provided configuration {"config_file": "/etc/caddy/Caddyfile", "config_adapter": "caddyfile"}
2022-02-24T00:49:13.080517683Z 2022/02/24 00:49:13.080 WARN input is not formatted with 'caddy fmt' {"adapter": "caddyfile", "file": "/etc/caddy/Caddyfile", "line": 2}
2022-02-24T00:49:13.082483777Z 2022/02/24 00:49:13.082 INFO admin admin endpoint started {"address": "tcp/localhost:2019", "enforce_origin": false, "origins": ["localhost:2019", "[::1]:2019", "127.0.0.1:2019"]}
2022-02-24T00:49:13.083012379Z 2022/02/24 00:49:13.082 INFO http server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS {"server_name": "srv0", "https_port": 443}
2022-02-24T00:49:13.083044007Z 2022/02/24 00:49:13.082 INFO http enabling automatic HTTP->HTTPS redirects {"server_name": "srv0"}
2022-02-24T00:49:13.083262915Z 2022/02/24 00:49:13.082 INFO tls.cache.maintenance started background certificate maintenance {"cache": "0xc0003bdb90"}
2022-02-24T00:49:13.088176927Z 2022/02/24 00:49:13.087 INFO tls cleaning storage unit {"description": "FileStorage:/data/caddy"}
2022-02-24T00:49:13.088214299Z 2022/02/24 00:49:13.087 INFO tls finished cleaning storage units
2022-02-24T00:49:13.088566440Z 2022/02/24 00:49:13.088 INFO http enabling automatic TLS certificate management {"domains": ["app.myapp.it"]}
2022-02-24T00:49:13.089217858Z 2022/02/24 00:49:13.088 INFO autosaved config (load with --resume flag) {"file": "/config/caddy/autosave.json"}
2022-02-24T00:49:13.089255497Z 2022/02/24 00:49:13.088 INFO serving initial configuration
2022-02-24T00:49:13.090255185Z 2022/02/24 00:49:13.089 INFO tls.obtain acquiring lock {"identifier": "app.myapp.it"}
2022-02-24T00:49:13.104037308Z 2022/02/24 00:49:13.103 INFO tls.obtain lock acquired {"identifier": "app.myapp.it"}
2022-02-24T00:49:13.980759033Z 2022/02/24 00:49:13.980 INFO tls.issuance.acme waiting on internal rate limiter {"identifiers": ["app.myapp.it"], "ca": "https://acme-v02.api.letsencrypt.org/directory", "account": "myapp#gmail.com"}
2022-02-24T00:49:13.980807648Z 2022/02/24 00:49:13.980 INFO tls.issuance.acme done waiting on internal rate limiter {"identifiers": ["app.myapp.it"], "ca": "https://acme-v02.api.letsencrypt.org/directory", "account": "myapp#gmail.com"}
2022-02-24T00:49:14.538528714Z 2022/02/24 00:49:14.538 INFO tls.issuance.acme.acme_client trying to solve challenge {"identifier": "app.myapp.it", "challenge_type": "tls-alpn-01", "ca": "https://acme-v02.api.letsencrypt.org/directory"}
2022-02-24T00:49:15.976582736Z 2022/02/24 00:49:15.976 ERROR tls.issuance.acme.acme_client challenge failed {"identifier": "app.myapp.it", "challenge_type": "tls-alpn-01", "status_code": 403, "problem_type": "urn:ietf:params:acme:error:unauthorized", "error": "Cannot negotiate ALPN protocol \"acme-tls/1\" for tls-alpn-01 challenge"}
2022-02-24T00:49:15.976692391Z 2022/02/24 00:49:15.976 ERROR tls.issuance.acme.acme_client validating authorization {"identifier": "app.myapp.it", "error": "authorization failed: HTTP 403 urn:ietf:params:acme:error:unauthorized - Cannot negotiate ALPN protocol \"acme-tls/1\" for tls-alpn-01 challenge", "order": "https://acme-v02.api.letsencrypt.org/acme/order/422657490/66417417610", "attempt": 1, "max_attempts": 3}
2022-02-24T00:49:17.508224302Z 2022/02/24 00:49:17.507 INFO tls.issuance.acme.acme_client trying to solve challenge {"identifier": "app.myapp.it", "challenge_type": "http-01", "ca": "https://acme-v02.api.letsencrypt.org/directory"}
2022-02-24T00:49:18.933967989Z 2022/02/24 00:49:18.933 ERROR tls.issuance.acme.acme_client challenge failed {"identifier": "app.myapp.it", "challenge_type": "http-01", "status_code": 403, "problem_type": "urn:ietf:params:acme:error:unauthorized", "error": "Invalid response from http://app.ripapp.it/.well-known/acme-challenge/QG2yr7WcBg8Wbj9evi8oyk1CzaTFM0Y9bkgkmqq5Iww [91.187.200.219]: \"<html lang=\\\"en\\\" xml:lang=\\\"en\\\" xmlns=\\\"http://www.w3.org/1999/xhtml\\\">\\n<head>\\n <title>Connection denied by Geolocation</title>\\n \""}
2022-02-24T00:49:18.934101729Z 2022/02/24 00:49:18.933 ERROR tls.issuance.acme.acme_client validating authorization {"identifier": "app.myapp.it", "error": "authorization failed: HTTP 403 urn:ietf:params:acme:error:unauthorized - Invalid response from http://app.myapp.it/.well-known/acme-challenge/QG2yr7WcBg8Wbj9evi8oyk1CzaTFM0Y9bkgkmqq5Iww [91.187.200.219]: \"<html lang=\\\"en\\\" xml:lang=\\\"en\\\" xmlns=\\\"http://www.w3.org/1999/xhtml\\\">\\n<head>\\n <title>Connection denied by Geolocation</title>\\n \"", "order": "https://acme-v02.api.letsencrypt.org/acme/order/422657490/66417426840", "attempt": 2, "max_attempts": 3}
2022-02-24T00:49:20.696387362Z 2022/02/24 00:49:20.695 ERROR tls.obtain could not get certificate from issuer {"identifier": "app.myapp.it", "issuer": "acme-v02.api.letsencrypt.org-directory", "error": "[app.myapp.it] solving challenges: app.myapp.it: no solvers available for remaining challenges (configured=[http-01 tls-alpn-01] offered=[http-01 dns-01 tls-alpn-01] remaining=[dns-01]) (order=https://acme-v02.api.letsencrypt.org/acme/order/422657490/66417435240) (ca=https://acme-v02.api.letsencrypt.org/directory)"}
2022-02-24T00:49:21.383148322Z 2022/02/24 00:49:21.382 INFO tls.issuance.zerossl generated EAB credentials {"key_id": "fiNQgkXxmfwTdX1q1gFasg"}
2022-02-24T00:49:24.460492479Z 2022/02/24 00:49:24.459 INFO tls.issuance.acme waiting on internal rate limiter {"identifiers": ["app.myapp.it"], "ca": "https://acme.zerossl.com/v2/DV90", "account": "myapp#gmail.com"}
2022-02-24T00:49:24.460580992Z 2022/02/24 00:49:24.460 INFO tls.issuance.acme done waiting on internal rate limiter {"identifiers": ["app.myapp.it"], "ca": "https://acme.zerossl.com/v2/DV90", "account": "myapp#gmail.com"}
I cannot work without resolving this (on http port is listening current active website, so I cannot test anything over http port).
It seems the problem is that letsencrypt refuses someway the connection. What can I do?
Is there something that I can do to solve? (or also if you need some other files and configurations)
Was thinking about changing to traefik, but the ideal thing is to solve and leave the structure of the project as it is.