This used to work:
aws ecr get-login-password --region $AWS_DEFAULT_REGION | docker login --username AWS --password-stdin "$AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com"
Now, I get:
Error saving credentials: error storing credentials - err: exit status 1, out: `error storing credentials - err: exit status 1, out: `The user name or passphrase you entered is not correct.``
If I run just aws ecr get-login-password, I get a successful result, which suggests I have the right AWS access:
aws ecr get-login-password --region us-west-2 | base64 -d | jq '.'
{
"payload": "redacted",
"datakey": "redacted",
"version": "2",
"type": "DATA_KEY",
"expiration": redacted
}
This command that is failing is in a script that has been run successfully many times before and hasn't been changed. Has something changed in new versions of Docker that would break this? Or new versions of the Amazon CLI client? One possibility, is my AWS account privileges have changed, but I would have expected the above get-login-password command to have not succeeded in that case.
This login method is documented here, and I seem to be following instructions exactly:
https://docs.aws.amazon.com/cli/latest/reference/ecr/get-login-password.html
Is there anything I can do to troubleshoot this?
How do you get credentials
Docker describe it as getting it from config.json but it is in "credsStore" and how do you get it out from there?
cat ~/.docker/config.json
{
"auths": {
"https://index.docker.io/v1/": {}
},
"credsStore": "desktop",
"experimental": "enabled",
"stackOrchestrator": "swarm"
}
Using docker-credential-osxkeychain get <??>
It is like no matter what I try to "get" is comes back with:
docker-credential-osxkeychain <store|get|erase|list|version>
Short answer
the command docker-credentials-osxkeychain get reads from standard input and expects server url as described here
so one can do something like
$ echo "https://index.docker.io/v1/" | docker-credential-osxkeychain get
You can find the server url using docker-credential-osxkeychain list
$ docker-credential-osxkeychain list
{"https://index.docker.io/v1/":"someuser"}
Long answer
First, you might want to change credsStore to osxkeychain which makes ~/.docker/config.json look like
{
"stackOrchestrator" : "swarm",
"credsStore" : "osxkeychain",
"auths" : {
},
"experimental" : "disabled"
}
Second, login with docker cli using docker login
$ docker login
Login with your Docker ID to push and pull images from Docker Hub. If you don't have a Docker ID, head over to https://hub.docker.com to create one.
Username: someuser
Password:
Login Succeeded
On successful login, docker-credential-osxkeychain has stored the username and password in osxkeychain which can be retrieved using docker-credential-osxkeychain get
$ docker-credential-osxkeychain list
{"https://index.docker.io/v1/":"someuser"}
we can use the url as the key to get our credentials
note: docker-credential-osxkeychain get uses stdin to read the input.
$ echo "https://index.docker.io/v1/" | docker-credential-osxkeychain get
{"ServerURL":"https://index.docker.io/v1/","Username":"someuser","Secret":"your-actual-password"}
Task: pull and push images to the AWS ECR (Private)
Status: works from one Ubuntu 20 host, does not work from another Ubuntu 20 host
Briefly: I have IAM and I am able to both push and pull from a working Ubuntu 20 host. There is another host, I tried both IAM and I even created a pair of credentials for root user, still, no luck. I tried both docker login approach and https://github.com/awslabs/amazon-ecr-credential-helper.
Works from 19.03.8.
Does not work from 19.03.12, updated to Docker 20.10.5, still does not work.
Full explanation.
Using docker login
docker login (zeros in reality contain my acc id)
$ aws ecr get-login-password --region us-east-2 | docker login --username AWS --password-stdin 000000000000.dkr.ecr.us-east-2.amazonaws.com
WARNING! Your password will be stored unencrypted in /home/username/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
$ cat ~/.docker/config.json (zeros in reality contain my acc id)
{
"auths": {
"000000000000.dkr.ecr.us-east-2.amazonaws.com": {
"auth": "QVd[REMOVED A LOT OF CHARACTERS]Q=="
}
}
}
cat ~/.aws/credentials
[default]
aws_access_key_id=A......A
aws_secret_access_key=x.......Q
docker pull (note: (a) image exists (b) I can pull and push from first Ubuntu (c) I replaced the credentials with the root user credentials for the testing purposes; same thing happens with the regular user creds)
$ docker pull 000000000000.dkr.ecr.us-east-2.amazonaws.com/imagename:latest
Error response from daemon: pull access denied
for 000000000000.dkr.ecr.us-east-2.amazonaws.com/imagename:latest,
repository does not exist or may require 'docker login': denied: User: arn:aws:iam::000000000000:root is not authorized to perform:ecr:BatchGetImage on resource: arn:aws:ecr:us-east-2:000000000000:repository/imagename
Using aws-ecr-helper
$ cat ~/.docker/config.json
{
"credsStore": "ecr-login"
}
$cat ~/.ect/cache.json
$ cat cache.json
{
"Registries": {
"us-east-2-Q[REMOVED]s+EJ+-000000000000": {
"AuthorizationToken": "QV[REMOVED LOTS OF CHARACTERS]0=",
"RequestedAt": "2021-03-07T12:26:37.65116332+01:00",
"ExpiresAt": "2021-03-07T23:26:37.605Z",
"ProxyEndpoint": "https://000000000000.dkr.ecr.us-east-2.amazonaws.com",
"Service": "ecr"
}
},
"Version": "1.0"
}
docker pull
$ docker pull 000000000000.dkr.ecr.us-east-2.amazonaws.com/imagename:latest
Error response from daemon: pull access denied
for 000000000000.dkr.ecr.us-east-2.amazonaws.com/imagename:latest,
repository does not exist or may require 'docker login': denied: User: arn:aws:iam::000000000000:root is not authorized to perform:
ecr:BatchGetImage on resource: arn:aws:ecr:us-east-2:000000000000:repository/imagename
Just to clarify. Exactly same config with both regular user AND with root works on another host. Please advise where to go next. Thank you.
UPDATED. Both working and not working hosts are outside of the AWS Ecosystem.
UPDATED #2.
I enabled CloudTrail logs, which allow me to see the requests from Amazon perspective.
So basically I am doing exactly the same thing from both computers. At the same time I receive the result from the first one and I get access denied from the second one.
Basically this means that both requests reach AWS, question is, why the second desktop does not have an access.
Log after connecting from the desktop, which works
(000000000000 is my acc id)
eventVersion "1.08"
userIdentity
type "Root"
principalId "000000000000"
arn "arn:aws:iam::000000000000:root"
accountId "000000000000"
accessKeyId "A[DELETED]Q"
sessionContext
sessionIssuer {}
webIdFederationData {}
attributes
mfaAuthenticated "false"
creationDate "2021-03-07T19:19:57Z"
invokedBy "AWS Internal"
eventTime "2021-03-07T19:20:03Z"
eventSource "ecr.amazonaws.com"
eventName "BatchGetImage"
awsRegion "us-east-2"
sourceIPAddress "AWS Internal"
userAgent "AWS Internal"
requestParameters
registryId "000000000000"
repositoryName "imagename"
imageIds
0
imageTag "latest"
acceptedMediaTypes
0 "application/vnd.docker.distribution.manifest.v2+json"
1 "application/vnd.docker.distribution.manifest.list.v2+json"
2 "application/vnd.oci.image.index.v1+json"
3 "application/vnd.oci.image.manifest.v1+json"
4 "application/vnd.docker.distribution.manifest.v1+prettyjws"
5 "application/json"
responseElements null
requestID "e514746b[DELETED]"
eventID "3e6db33d[DELETED]"
readOnly true
resources
0
accountId "000000000000"
ARN "arn:aws:ecr:us-east-2:000000000000:repository/imagename"
eventType "AwsApiCall"
managementEvent true
eventCategory "Management"
recipientAccountId "000000000000"
And at the same time
Log after connecting from the desktop, which is always denied
eventVersion "1.08"
userIdentity
type "Root"
principalId "000000000000"
arn "arn:aws:iam::000000000000:root"
accountId "000000000000"
accessKeyId "A[DELETED]A"
sessionContext
sessionIssuer {}
webIdFederationData {}
attributes
mfaAuthenticated "false"
creationDate "2021-03-07T17:30:12Z"
invokedBy "AWS Internal"
eventTime "2021-03-07T19:02:50Z"
eventSource "ecr.amazonaws.com"
eventName "BatchGetImage"
awsRegion "us-east-2"
sourceIPAddress "AWS Internal"
userAgent "AWS Internal"
errorCode "AccessDenied"
errorMessage "User: arn:aws:iam::000000000000:root is not authorized to perform: ecr:BatchGetImage on resource: arn:aws:ecr:us-east-2:000000000000:repository/imagename"
requestParameters null
responseElements null
requestID "371354a8[DELETED]"
eventID "2f9f0310[DELETED]"
readOnly true
eventType "AwsApiCall"
managementEvent true
eventCategory "Management"
recipientAccountId "000000000000"
Note for those who thinks about credentials, the access key you see under those logs is not the access key of the user. Those access keys are different on each request (Even each request from the working desktop has different access key).
UPDATED #3. I checked from 3rd Ubuntu 17 host with Docker 18 and fresh aws cli, it works. So this is definitely something about the setup, docker or aws cli, not about the ECR itself.
UPDATED #4. I made a test with shell proxies (tinyproxy + SSH tunnel) and tests to eliminate some kind of shadow IP ban possibility. So basically:
working host keeps working through the proxy based on the non-working host and
non-working host keeps failing through the proxy based on the working host
Which means there is something with the docker on specific host, but removing and installing docker again does not solve it.
As you confirm that it is not an IAM role issue, You can receive this error due to one of the following issues:
Your launch type doesn't have access to the Amazon ECR endpoint.
If your host is in a private subnet then confirm that your subnet has a route to a NAT gateway in the route table.
If your host is in a public subnet, then confirm that the instance has a public IP address.
If you're using AWS PrivateLink for Amazon ECR, then confirm that the security group, associated with the interface VPC endpoints for Amazon ECR,allows inbound traffic over HTTPS (port 443) from within the security group of your VPC CIDR or your host security group.
Confirm that the security group attached to your host allows outbound access for the following: HTTPS on port 443, DNS (UDP and TCP) on port 53, and your subnet's network access control list (network ACL).
Your Amazon ECR repository policy restricts access to repository images.
The following repository policy example allows IAM users to push and pull images:
{
"Version": "2008-10-17",
"Statement": [
{
"Sid": "AllowPushPull",
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::123456789012:user/push-pull-user-1",
"arn:aws:iam::123456789012:user/push-pull-user-2"
]
},
"Action": [
"ecr:GetDownloadUrlForLayer",
"ecr:BatchGetImage",
"ecr:BatchCheckLayerAvailability",
"ecr:PutImage",
"ecr:InitiateLayerUpload",
"ecr:UploadLayerPart",
"ecr:CompleteLayerUpload"
]
}
]
}
When attempting to push image to ECR, I always get 'no basic auth' error. ECR is in us-east-1.
This is login command
aws ecr get-login --region us-east-1 --no-include-email
I get response like this
docker login -u AWS -p eyJwYXl ...
I copy this response and run command, than I got this response
Login Succeeded
I assume that I logged in AWS and docker successfully, so I try to push image to ECR
docker push AWSID.dkr.ecr.us-east-1.amazonaws.com/repositoryname
Than response is like this
ed9f73170eb1: Preparing
f26c0d1885c7: Preparing
254cc70ba305: Preparing
6bfcbc08ecad: Preparing
0cb1addb8efc: Preparing
f9109426e338: Waiting
b7f99d06d826: Waiting
24d803cb9c1a: Waiting
25c4f6422338: Preparing
69b416623121: Waiting
0753f0746a0d: Waiting
a20143cd0986: Waiting
3028f693c3e6: Waiting
514a0f74b55d: Waiting
no basic auth credentials
To figure out what causes error, I checked ~/.docker/config.json. my config.json is like this.
{
"auths": {
"AWSPATH.dkr.ecr.us-east-1.amazonaws.com": {
"auth": "QVd..},
"https://index.docker.io/v1/": {}
},
"HttpHeaders": {
"User-Agent": "Docker-Client/19.03.5 (darwin)"
}
It seems to be an Authentication issue. Reconfigure your AWS-CLI, maybe that will resolve your issue.
aws configure
https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html
try to login this way and push:
eval $(aws ecr get-login --no-include-email | sed ‘s|https://||’)
For me, it was simply missing permissions.
There are a number of permissions related to uploading such as
"ecr:UploadLayerPart",
"ecr:InitiateLayerUpload",
"ecr:CompleteLayerUpload"
"ecr:PutImage"
I have solved it by adding them to my agent's allowed permissions.
The problem was, jenkins server already had aws configured with different team's account. so I added region settings (we uses different region) and the problem solved.
I'm not sure if I have already logged in to a docker registry in cmd line by using cmd: docker login. How can you test or see whether you are logged in or not, without trying to push?
Edit 2020
Referring back to the (closed) github issue, where it is pointed out, there is no actual session or state;
docker login actually isn't creating any sort of persistent session, it is only storing the user's credentials on disk so that when authentication is required it can read them to login
As others have pointed out, an auths entry/node is added to the ~/.docker/config.json file (this also works for private registries) after you succesfully login:
{
"auths": {
"https://index.docker.io/v1/": {}
},
...
When logging out, this entry is then removed:
$ docker logout
Removing login credentials for https://index.docker.io/v1/
Content of docker config.json after:
{
"auths": {},
...
This file can be parsed by your script or code to check your login status.
Alternative method (re-login)
You can login to docker with docker login <repository>
$ docker login
Login with your Docker ID to push and pull images from Docker Hub. If
you don't have a Docker ID, head over to https://hub.docker.com to
create one.
Username:
If you are already logged in, the prompt will look like:
$ docker login
Login with your Docker ID to push and pull images from Docker Hub. If
you don't have a Docker ID, head over to https://hub.docker.com to
create one.
Username (myusername): # <-- "myusername"
For the original explanation for the ~/.docker/config.json, check question: how can I tell if I'm logged into a private docker registry
I use one of the following two ways for this check:
1: View config.json file:
In case you are logged in to "private.registry.com" you will see an entry for the same as following in ~/.docker/config.json:
"auths": {
"private.registry.com": {
"auth": "gibberishgibberishgibberishgibberishgibberishgibberish"
}
}
2: Try docker login once again:
If you are trying to see if you already have an active session with private.registry.com, try to login again:
bash$ docker login private.registry.com
Username (logged-in-user):
If you get an output like the above, it means logged-in-user already had an active session with private.registry.com. If you are just prompted for username instead, that would indicate that there's no active session.
You can do the following command to see the username you are logged in with and the registry used:
docker system info | grep -E 'Username|Registry'
The answers here so far are not so useful:
docker info no longer provides this info
docker logout is a major inconvenience - unless you already know the credentials and can easily re-login
docker login response seems quite unreliable and not so easy to parse by the program
My solution that worked for me builds on #noobuntu's comment: I figured that if I already known the image that I want to pull, but I'm not sure if the user is already logged in, I can do this:
try pulling target image
-> on failure:
try logging in
-> on failure: throw CannotLogInException
-> on success:
try pulling target image
-> on failure: throw CannotPullImageException
-> on success: (continue)
-> on success: (continue)
The docker cli credential scheme is unsurprisingly uncomplicated, just take a look:
cat ~/.docker/config.json
{
"auths": {
"dockerregistry.myregistry.com": {},
"https://index.docker.io/v1/": {}
This exists on Windows (use Get-Content ~\.docker\config.json) and you can also poke around the credential tool which also lists the username ... and I think you can even retrieve the password
. "C:\Program Files\Docker\Docker\resources\bin\docker-credential-wincred.exe" list
{"https://index.docker.io/v1/":"kcd"}
For private registries, nothing is shown in docker info. However, the logout command will tell you if you were logged in:
$ docker logout private.example.com
Not logged in to private.example.com
(Though this will force you to log in again.)
At least in "Docker for Windows" you can see if you are logged in to docker hub over the UI. Just right click the docker icon in the windows notification area:
Just checked, today it looks like this:
$ docker login
Authenticating with existing credentials...
Login Succeeded
NOTE: this is on a macOS with the latest version of Docker CE, docker-credential-helper - both installed with homebrew.
If you want a simple true/false value, you can pipe your docker.json to jq.
is_logged_in() {
cat ~/.docker/config.json | jq -r --arg url "${REPOSITORY_URL}" '.auths | has($url)'
}
if [[ "$(is_logged_in)" == "false" ]]; then
# do stuff, log in
fi
My AWS ECR build-script has:
ECR_HOSTNAME="${ACCOUNT_ID}.dkr.ecr.${REGION}.amazonaws.com"
TOKEN=$(jq -r '.auths["'$ECR_HOSTNAME'"]["auth"]' ~/.docker/config.json)
curl --fail --header "Authorization: Basic $TOKEN" https://$ECR_HOSTNAME/v2/
If accessing ECR fails, a login is done:
aws ecr get-login-password --region ${REGION} | docker login --username AWS --password-stdin https://$ECR_HOSTNAME
For this to work, a proper Docker credential store cannot be used. Default credentials store of ~/.docker/config.json is assumed.
Use command like below:
docker info | grep 'name'
WARNING: No swap limit support
Username: <strong>jonasm2009</strong>
On windows you can inspect the login "authorizations" (auths) by looking at this file:
[USER_HOME_DIR].docker\config.json
Example:
c:\USERS\YOUR_USERANME.docker\config.json
It will look something like this for windows credentials
{
"auths": {
"HOST_NAME_HERE": {},
"https://index.docker.io/v1/": {}
},
"HttpHeaders": {
"User-Agent": "Docker-Client/18.09.0 (windows)"
},
"credsStore": "wincred",
"stackOrchestrator": "swarm"
}
On Linux if you have the secretservice enabled via the credsStore option in your ~/.docker/config.json like below:
"credsStore": "secretservice",
then you will not see the credentials in the config.json. Instead you need to query the credentials using the docker-credential-desktop, see the below answer for more details:
How to know if docker is already logged in to a docker registry server
In Azure Container Registry (ACR) following works as a login-check:
registry="contosoregistry.azurecr.io"
curl -v --header "Authorization: Bearer $access_token" https://$registry/v2/_catalog
If access token has expired, a HTTP/401 will be returned.
Options for getting an access token are from ~/.docker/config.json or requesting one from https://$registry/oauth2/token using a refresh token stored into Docker credStore: echo $registry | docker-credential-desktop get.
More information about refresh tokens and access tokens are at ACR integration docs.
To many answers above is just about how to check login status manually. To do it from command line you can use the command below.
cat ~/.docker/config.json | jq '.auths["<MY_REGISTRY_HOSTNAME>"]' -e > /dev/null && echo "OK" || echo "ERR"
Ensure you have jq command in your local. To test that run jq --version command. If you can't get an version output follow the directions from here to install it https://stedolan.github.io/jq/download/
Replace <MY_REGISTRY_HOSTNAME> with your registry address.
When you run it returns OK if you successfully login already otherwise ERR
NOTE: if you used a credential helper to login (e.g. google cloud auth tool for container registry) replace .auths keyword with .credHelpers
As pointed out by #Christian, best to try operation first then login only if necessary. Problem is that "if necessary" is not that obvious to do robustly. One approach is to compare the stderr of the docker operation with some strings that are known (by trial and error). For example,
try "docker OPERATION"
if it failed:
capture the stderr of "docker OPERATION"
if it ends with "no basic auth credentials":
try docker login
else if it ends with "not found":
fatal error: image name/tag probably incorrect
else if it ends with <other stuff you care to trap>:
...
else:
fatal error: unknown cause
try docker OPERATION again
if this fails: you're SOL!
Here's a powershell powershell command to check if you have previously logged into the registry, making use of the file $HOME/.docker/config.json that others have mentioned:
(Get-Content $HOME/.docker/config.json | ConvertFrom-Json).auths.PSobject.Properties.name -Contains "<registry_url>"
This returns a True / False boolean, so can use as follows:
if ((Get-Content $HOME/.docker/config.json | ConvertFrom-Json).auths.PSobject.Properties.name -Contains "<registry_url>" ) {
Write-Host Already logged into docker registry
} else {
Write-Host Logging into docker registry
docker login
}
If you want it to not fail if the file doesn't exist you need an extra check:
if ( (-Not (Test-Path $HOME/.docker/config.json)) -Or (-Not (Get-Content $HOME/.docker/config.json | ConvertFrom-Json).auths.PSobject.Properties.name -Contains "<registry_url>") )
{
Write-Host Already logged into docker registry
} else {
Write-Host Logging into docker registry
docker login
}
I chose to use the -Not Statements because for some reason when you chain a command after a failed condition with -And instead of -Or the command errors out.