Docker push to AWS ECR hangs immediately and times out - docker

Question

I'm trying to push my first docker image to ECR. I've followed the steps provided by AWS and things seem to be going smoothly until the final push which immediately times out. Specifically, I pass my aws ecr credentials to docker and get a "login succeeded" message. I then tag the image which also works. pushing to the ecr repo I get no error message, just the following:
The push refers to repository [xxxxxxxxxxx.dkr.ecr.ca-central-1.amazonaws.com/reponame]
714c1b96dd83: Retrying in 1 second
d2cdc77dd068: Retrying in 1 second
30aad807caf5: Retrying in 1 second
0559774c4ea2: Retrying in 1 second
285b8616682f: Retrying in 1 second
4aeea0ec2b15: Waiting
1b1312f842d8: Waiting
c310009e0ef3: Waiting
a48777e566d3: Waiting
2a0c9f28029a: Waiting
EOF
It tries a bunch of times and then exits with no message. Any idea what's wrong?

Answer 1

I figured out my issue. I wasn't using the correct credentials. I had a personal AWS account as my default credentials and needed to add my work profile to my credentials.
EDIT
If you have multiple aws profiles, you can mention the profile name at the docker login as below (assuming you have done aws configure --profile someprofile at earlier day),
aws ecr get-login-password --region us-east-1 --profile someprofile | docker login ....

Answer 2

You will get the same behaviour if you forget to create ECR repo before pushing.
Use CloudTrail to get a clue what is wrong.

Answer 3

Also make sure that you have configured correct policy for your user — for example, AmazonEC2ContainerRegistryFullAccess.

Answer 4

Make sure the name of your repository is the same name as your images.
image:latest 756839881602.dkr.ecr.us-east-1.amazonaws.com/image:latest in this case my repository name is image and my image name is image as well. This worked for me.

Answer 5

In my case, the repository I wanted to push to didn't exist (For example, I tried pushing to my-app/backend:latest but only the my-app/cms repository exists). So make sure your repository exists in the AWS ECR Console in the right region. The error returned from AWS CLI (EOF) didn't help at all.

Answer 6

Check your aws permissions. In addition to AmazonEC2ContainerRegistryFullAccess permission, below actions has to be granted for the correct resource. Especially check "arn:aws:ecr:${REGION}:${ACCOUNT_ID}:repository/{$REGISTRY_NAME}" part.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ecr:BatchGetImage",
"ecr:BatchCheckLayerAvailability",
"ecr:CompleteLayerUpload",
"ecr:DescribeImages",
"ecr:DescribeRepositories",
"ecr:GetDownloadUrlForLayer",
"ecr:InitiateLayerUpload",
"ecr:ListImages",
"ecr:PutImage",
"ecr:UploadLayerPart"
],
"Resource": "arn:aws:ecr:${REGION}:${ACCOUNT_ID}:repository/{$REGISTRY_NAME}"
},
{
"Effect": "Allow",
"Action": "ecr:GetAuthorizationToken",
"Resource": "*"
}
]
}

Answer 7

In my case it was related to MFA (Multi-Factor-Authentication).
I had to create a session token. The docker login seemed to be successful, but pushing does not work.
The following script is doing all for you and creates a aws profile "mfa" used to login: get_mfa_credentials.py
After executing, you can login with:
aws ecr get-login-password --region <YOUR_REGION> --profile mfa | docker login --username AWS --password-stdin <Your_REPO>
I do not know who wrote it, but I'm very grateful to this guy.
And thanks to AWS for bad tools that do not help.

Answer 8

Assuming you authenticated successfully to AWS and you have permissions to read, write to ECR, check if the repository does exist
aws ecr describe-repositories --repository-name reponame
If you catch an error RepositoryNotFoundException, then you will create to that repository with the following command
aws ecr create-repository --repository-name reponame
After that, try to push again, it will be fine!

Answer 9

I also was able to login to the registry, yet the pushing of the image would just timeout.
The solution for me was to add AmazonEC2ContainerRegistryFullAccess to my IAM user.
After adding that permission to my IAM user account, I could docker push to the ECS registry just fine.

Answer 10

I have to add for anyone else encountering this problem. Go to IAM and make sure you have put permissions. I don't want to say how long I wasted before figuring that out.
Edit to help #zac's answer:
The policies that need to be attached are AmazonEC2ContainerRegistryFullAccess and AWSAppRunnerServicePolicyForECRAccess

Answer 11

For those who tried the solution above, and it didn't work, make sure the image name your are pushing is the same as the repository name.

Answer 12

Ensure you are using the correct profile and that the repository exists
Command to login with profile: aws ecr get-login-password --region <region> --profile=<profile-name> | docker login --username AWS --password-stdin <aws-account-id>.dkr.ecr.eu-west-1.amazonaws.com
Command to create repo if it does not exists:
aws ecr describe-repositories --repository-names ${REPO_NAME} || aws ecr create-repository --repository-name ${REPO_NAME}(source)

Answer 13

If anyone is still stuck with the issue. I would highly recommend watching this short vid https://www.youtube.com/watch?v=89ZeXaZEf80&ab_channel=IdenticalCloud
Here are the steps I took to fix the issue (if you prefer not to watch the video):
Create a new IAM user with "Access keys" checked
Under permissions, click on "attach existing policies directly" and choose "AmazonEC2ContainerRegistryFullAccess"
Download the CSV file
Run "AWS configure" on your terminal and pass in the credentials from the CSV file
Set the location to the location you created your ECR (mine was us-east-1)
Go to ECR and follow the steps to push the image

Answer 14

For me, I had to delete the stack and re-deploy the stack. Then, I was able to push the docker image to ECR.

Answer 15

Please check cloud trail event logs , this is where all the api issues are clearly highlighted .
In my case it was because i had a - in my image name and hence it was throwing the following error in the cloud trail logs
"The repository with name 'myimage-nginx' does not exist in the registry with id '516583196897'
Please note the - in the image name.
Fixing the image name to remove the - resolved the issue for me.
Commands
docker tag nginx:latest 516583196897.dkr.ecr.ap-south-1.amazonaws.com/myimage:latest
docker push 516583196897.dkr.ecr.ap-south-1.amazonaws.com/myimage:latest

Answer 16

In my case I was creating the repo in us-east-2 and attempting to push to us-east-1, so docker couldn't find it.

Answer 17

Make sure your assumed aws role has the ability to push images to AWS ECR. Easiest is to check the role via the command:
aws sts get-caller-identity --profile=saml

Answer 18

I was following this documentation and hit this error. What addressed the problem was using the repository id instead of the account name.
aws ecs create-repository creates a repo, returning a repositoryUri. Then, the docker login, docker tag and docker push should be done using that repository url instead of the user one.

Answer 19

I had this problem with sam deploy
sam delete --stack-name ...
sam deploy --guided
worked for me