Docker Watchtower cannot pull new image from AWS ECR - docker

Question

I've installed Watchtower, and followed the docs for setting up private registry auth with auth helpers. In debug mode, I see it logging a message that auth value was obtained, but then it fails to pull image with "no basic auth credentials." Inspecting auth value, it's just host name from my config, and no credential. I verified that on the host system (Raspbian) I'm able to pull new version using the same docker config, without having to do any custom auth, everything works out of the box, using the same binary.
Here's my docker config:
{
"auths" : {
"0000000000.dkr.ecr.us-east-1.amazonaws.com" : {}
},
"credHelpers": {
"0000000000.dkr.ecr.us-east-1.amazonaws.com": "ecr-login"
}
}
Here's my docker compose:
version: "3"
services:
cavo:
image: 0000000000.dkr.ecr.us-east-1.amazonaws.com/test:1
ports:
- "8080:80"
restart: always
watchtower:
image: containrrr/watchtower
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- /root/.docker/config.json:/config.json
- /usr/bin/docker-credential-ecr-login:/bin/docker-credential-ecr-login
environment:
- AWS_REGION=us-east-1
- AWS_ACCESS_KEY_ID=AAAAAAAAAAAAA
- AWS_SECRET_ACCESS_KEY=aaaaaaaaaaaaaaa
command: --debug --interval 30
restart: always
And when watchtower attempts to check for new image, here is the log file:
watchtower_1 | time="2019-12-25T22:49:34Z" level=debug msg="Pulling 0000000000.dkr.ecr.us-east-1.amazonaws.com/test:1 for /root_test_1"
watchtower_1 | time="2019-12-25T22:49:34Z" level=debug msg="Loaded auth credentials { 0000000000.dkr.ecr.us-east-1.amazonaws.com } from /config.json"
watchtower_1 | time="2019-12-25T22:49:34Z" level=debug msg="Got auth value: eyJzZXJ2ZXJhZGRyZXNzIjoiMDAwMDAwMDAwMC5ka3IuZWNyLnVzLWVhc3QtMS5hbWF6b25hd3MuY29tIn0="
watchtower_1 | time="2019-12-25T22:49:34Z" level=debug msg="Got image name: 0000000000.dkr.ecr.us-east-1.amazonaws.com/test:1"
watchtower_1 | time="2019-12-25T22:49:35Z" level=debug msg="Error pulling image 0000000000.dkr.ecr.us-east-1.amazonaws.com/sump-pump-v2:1, Error response from daemon: Get https://0000000000.dkr.ecr.us-east-1.amazonaws.com/v2/test/manifests/1: no basic auth credentials"
watchtower_1 | time="2019-12-25T22:49:35Z" level=info msg="Unable to update container /root_test_1. Proceeding to next."
watchtower_1 | time="2019-12-25T22:49:35Z" level=debug msg="Error response from daemon: Get https://0000000000.dkr.ecr.us-east-1.amazonaws.com/v2/test/manifests/1: no basic auth credentials"
Unpacking the auth value, it just has the hostname. No repository credential.
I was trying to follow "Credential helpers" documentation, but I'm not sure I understand where the aforementioned Dockerfile belongs.
Any pointers in the right direction would be appreciated. Thanks!

Answer 1

Try the folowing:
Create a docker volume named helper
docker volume create helper
Build de image from de Dockerfile in the docs
docker build -t aws-ecr-dock-cred-helper .
Run the container
docker run -d --rm --name aws-cred-helper --volume helper:/go/bin aws-ecr-dock-cred-helper
The container will start and mount helper volume into /go/bin where is the docker-credential-ecr-login and then stop.
You can check the content of helper volume with
docker run --rm --it -v helper:/go/bin alpine
then do
ls /go/bin
you should see the docker-credential-ecr-login binary.
I didn't use docker compose, but you have to mount the helper volume into the watchtower container on /go/bin and for some reason export your $PATH with /go/bin
like in the docs:
environment:
- HOME=/
- PATH=$PATH:/go/bin