How to force 'docker login' command to ignore existing credentials helper? - docker

I have a system where I'm trying to run the docker logincommand, it is a headless linux system, but unfortunately only the Docker Credentials Helper docker-credential-secretservice is installed.
This means that I get the following error:
Error saving credentials: error storing credentials - err: exit status 1, out: `Cannot autolaunch D-Bus without X11 $DISPLAY`
It makes sense that I get this as:
By default, Docker looks for the native binary on each of the
platforms, i.e. “osxkeychain” on macOS, “wincred” on windows, and
“pass” on Linux. A special case is that on Linux, Docker will fall
back to the “secretservice” binary if it cannot find the “pass”
binary. If none of these binaries are present, it stores the
credentials (i.e. password) in base64 encoding in the config files
described above.
And since secretservice helper uses a GUI credentials store it tries to open a window, which it can't on the headless system.
I've no control over the system, so I can't remove the /usr/bin/docker-credential-secretservice file to force docker login to fall back to the config file rather than using the secretservice helper.
What I can do is create and list files in my user's home folder. I've tried to run the command as such:
docker --config ./docker login -u <user-name> -p <password> <repository>
I was under the impression that the login command would then create a config.json in the ./docker (I've noticed docker login will create the folder if it doesn't exist). This works on a system that doesn't have any helpers installed, but not on the system in question.
I've also tried to create a ~/.docker/config.json with something like:
echo '{"credStore":""}' > ~/.docker/config.json
Hoping that docker login would get the hint not to use any helpers for the credential store.
Is there a way for a non-admin to force docker login to fall back to:
stores the credentials (i.e. password) in base64 encoding in the config files described above.
Without deleting the credentials helper?
(as a side note, I'll of course ask to have the /usr/bin/docker-credential-secretservice removed but, in case it's not possible or for future reference, are there any alternative solutions?)

Logging out the current user, before logging in with a different user name worked for me. Logging out removed the saved docker credentials.
docker logout <reponame>
docker login <reponame>

To avoid using a credsStore and to store a plaintext auth token in your docker config (e.g. ~/.docker/config.json), delete the "credsStore" key from your docker config file and rerun docker login.
When you run docker login, it will give a warning but will save the auth token into the file.
$ docker login
Username: someuser
Password:
WARNING! Your password will be stored unencrypted in ~/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
The resulting docker config file should look like this:
{
"auths": {
"your.docker.registry": {
"auth": "dXNlcm5hbWU6cGFzc3dvcmQK="
}
}
}
The auth token is simply a base64 encoded string of the form username:password.
This worked for Docker Engine versions 19 and 20.

Unfortunately, Docker (as of 18.06) first looks for the docker-credential-* binaries, and if it finds any of them, it will automatically overwrite the "credsStore" value in ~/.docker/config.json.
Your only workaround would be to install docker-credential-pass in your home directory so that Docker will use that instead of docker-credential-secretservice. docker-credential-pass does not require a GUI.
Steps to install docker-credential-pass:
docker login fails on a server with no X11 installed

You can just ignore all the output by sending everything to /dev/null like:
echo $TOKEN|docker login -u=user --password-stdin myregistry.com > /dev/null 2>&1
where $TOKEN will be your previously exported token or password.
This will be useful as well with CI's when using some automation

I've looked through the code and there appears to be no way to generally disable the use of credential helpers. But you can skip the code path on a per-registry basis.
https://github.com/docker/cli/blob/25eee83d6b8c475548254b2decc9c8e0490d229c/cli/config/configfile/file.go#L308 will look up the helper to use (just the suffix after docker-credential-) in credHelpers from the registry host name. If it is "" it will use the normal unencrypted file store. As far as I can tell, this is undocumented and likely unintended behavior.
With that, in order to bypass the credential helpers for a specific registry, do
mkdir ./docker
echo "{\"credHelpers\": {\"$REGISTRY_HOST\": \"\"}}" > ./docker/config.json
docker --config ./docker login -u $USERNAME -p $PASSWORD $REGISTRY_HOST
I believe the cause why {"credsStore":""} isn't working is the omitempty serialization tag on that field. Setting it to empty is the same as not setting it.

Not an answer to the question, but maybe to the problem:
We can launch dbus ourselves then unlock/lock/query the keyring from the cli.
These are the bash functions I use:
function unlock-keyring () {
export $(dbus-launch)
read -rsp "Password: " pass
export $(echo -n "$pass" | gnome-keyring-daemon --unlock)
unset pass
}
function lock-keyring () {
dbus-send --dest=org.gnome.keyring --print-reply /org/freedesktop/secrets org.freedesktop.Secret.Service.LockService
}
function query-keyring-locked () {
busctl --user get-property org.freedesktop.secrets /org/freedesktop/secrets/collection/login org.freedesktop.Secret.Collection Locked
}
https://unix.stackexchange.com/questions/473528/how-do-you-enable-the-secret-tool-command-backed-by-gnome-keyring-libsecret-an
https://superuser.com/questions/700826/how-to-lock-a-unlocked-gnome-keyring
https://superuser.com/questions/1618970/query-status-of-gnome-keyring
https://unix.stackexchange.com/questions/602313/unlock-gnome-keyring-daemon-from-command-line

Simply rename ~/.docker/config.json to something different, or remove it if you won't need it anymore.
mv ~/.docker/config.json ~/.docker/backup-config.json

Related

docker-compose - how to provide credentials or API key in order to pull image from private repository?

I have private repo where I am uploading images outside of the docker.
image: example-registry.com:4000/test
I have that defined in my docker-compose file.
How I can provide credentials or API key in order to pull from that repository? Is it possible to do it without executing "docker login" command or it is required to always execute those commands prior the docker-compose command?
I have API key which I am using for example to do the REST API from PowerShell or any other tool.
Can I use that somehow in order to avoid "docker login" command constantly?
Thank you
docker login creates or updates the ~/.docker/config.json file for you. With just the login part, it look likes
{
"auths": {
"https://index.docker.io/v1/": {
"auth": "REDACTED"
}
}
}
There can be many things in this file, here is the doc
So to answer your question, you can avoid the login command by distributing this file instead. Something like:
Create a dedicated token (you shouldn't have multiple usage by token) here https://hub.docker.com/settings/security
Move your current config elsewhere if it does exist mv ~/.docker/config.json /tmp
Execute docker login -u YOUR-ACCOUNT, using the token as password
Copy the generated ~/.docker/config.json that you can then distribute to your server(s). This file is as much a secret as your password , don't make it public!
Move back your current config mv /tmp/config.json ~/.docker/
Having the file as a secret that you distribute doesn't make much difference than inputing the docker login command though, especially if you've some scripting to do that.

How to properly build a private Go project? [duplicate]

I'm searching for the way to get $ go get work with private repository, after many google try.
The first try:
$ go get -v gitlab.com/secmask/awserver-go
Fetching https://gitlab.com/secmask/awserver-go?go-get=1
https fetch failed.
Fetching http://gitlab.com/secmask/awserver-go?go-get=1
Parsing meta tags from http://gitlab.com/secmask/awserver-go?go-get=1 (status code 200)
import "gitlab.com/secmask/awserver-go": parse http://gitlab.com/secmask/awserver-go?go-get=1: no go-import meta tags
package gitlab.com/secmask/awserver-go: unrecognized import path "gitlab.com/secmask/awserver-go
Yep, it did not see the meta tags because I could not know how to provide login information.
The second try:
Follow https://gist.github.com/shurcooL/6927554. Add config to .gitconfig.
[url "ssh://git#gitlab.com/"]
insteadOf = https://gitlab.com/
$ go get -v gitlab.com/secmask/awserver-go --> not work
$ go get -v gitlab.com/secmask/awserver-go.git --> work but I got src/gitlab.com/secmask/awserer-go.git
Yes it work but with .git extension with my project name, I can rename it to original but do it everytime $ go get is not so good, is there an otherway?
You have one thing to configure. The example is based on GitHub but this shouldn't change the process:
$ git config --global url.git#github.com:.insteadOf https://github.com/
$ cat ~/.gitconfig
[url "git#github.com:"]
insteadOf = https://github.com/
$ go get github.com/private/repo
For Go modules to work (with Go 1.11 or newer), you'll also need to set the GOPRIVATE variable, to avoid using the public servers to fetch the code:
export GOPRIVATE=github.com/private/repo
The proper way is to manually put the repository in the right place. Once the repository is there, you can use go get -u to update the package and go install to install it. A package named
github.com/secmask/awserver-go
goes into
$GOPATH/src/github.com/secmask/awserver-go
The commands you type are:
cd $GOPATH/src/github.com/secmask
git clone git#github.com:secmask/awserver-go.git
I had a problem with go get using private repository on gitlab from our company.
I lost a few minutes trying to find a solution. And I did find this one:
You need to get a private token at:
https://gitlab.mycompany.com/profile/account
Configure you git to add extra header with your private token:
$ git config --global http.extraheader "PRIVATE-TOKEN: YOUR_PRIVATE_TOKEN"
Configure your git to convert requests from http to ssh:
$ git config --global url."git#gitlab.mycompany.com:".insteadOf "https://gitlab.mycompany.com/"
Finally you can use your go get normally:
$ go get gitlab.com/company/private_repo
For people using private GitLabs, here's a snippet that may help: https://gist.github.com/MicahParks/1ba2b19c39d1e5fccc3e892837b10e21
Also pasted below:
Problem
The go command line tool needs to be able to fetch dependencies from your private GitLab, but authenticaiton is required.
This assumes your private GitLab is hosted at privategitlab.company.com.
Environment variables
The following environment variables are recommended:
export GO111MODULE=on
export GOPRIVATE=privategitlab.company.com # this is comma delimited if using multiple private repos
The above lines might fit best in your shell startup, like a ~/.bashrc.
Explanation
GO111MODULE=on tells Golang command line tools you are using modules. I have not tested this with projects not using
Golang modules on a private GitLab.
GOPRIVATE=privategitlab.company.com tells Golang command line tools to not use public internet resources for the hostnames
listed (like the public module proxy).
Get a personal access token from your private GitLab
To future proof these instructions, please follow this guide from the GitLab docs.
I know that the read_api scope is required for Golang command line tools to work, and I may suspect read_repository as
well, but have not confirmed this.
Set up the ~/.netrc
In order for the Golang command line tools to authenticate to GitLab, a ~/.netrc file is best to use.
To create the file if it does not exist, run the following commands:
touch ~/.netrc
chmod 600 ~/.netrc
Now edit the contents of the file to match the following:
machine privategitlab.company.com login USERNAME_HERE password TOKEN_HERE
Where USERNAME_HERE is replaced with your GitLab username and TOKEN_HERE is replaced with the access token aquired in the
previous section.
Common mistakes
Do not set up a global git configuration with something along the lines of this:
git config --global url."git#privategitlab.company.com:".insteadOf "https://privategitlab.company.com"
I beleive at the time of writing this, the SSH git is not fully supported by Golang command line tools and this may cause
conflicts with the ~/.netrc.
Bonus: SSH config file
For regular use of the git tool, not the Golang command line tools, it's convient to have a ~/.ssh/config file set up.
In order to do this, run the following commands:
mkdir ~/.ssh
chmod 700 ~/.ssh
touch ~/.ssh/config
chmod 600 ~/.ssh/config
Please note the permissions on the files and directory above are essentail for SSH to work in it's default configuration on
most Linux systems.
Then, edit the ~/.ssh/config file to match the following:
Host privategitlab.company.com
Hostname privategitlab.company.com
User USERNAME_HERE
IdentityFile ~/.ssh/id_rsa
Please note the spacing in the above file matters and will invalidate the file if it is incorrect.
Where USERNAME_HERE is your GitLab username and ~/.ssh/id_rsa is the path to your SSH private key in your file system.
You've already uploaded its public key to GitLab. Here are some instructions.
All of the above did not work for me. Cloning the repo was working correctly but I was still getting an unrecognized import error.
As it stands for Go v1.13, I found in the doc that we should use the GOPRIVATE env variable like so:
$ GOPRIVATE=github.com/ORGANISATION_OR_USER_NAME go get -u github.com/ORGANISATION_OR_USER_NAME/REPO_NAME
Generate a github oauth token here and export your github token as an environment variable:
export GITHUB_TOKEN=123
Set git config to use the basic auth url:
git config --global url."https://$GITHUB_TOKEN:x-oauth-basic#github.com/".insteadOf "https://github.com/"
Now you can go get your private repo.
If you've already got git using SSH, this answer by Ammar Bandukwala is a simple workaround:
$ go get uses git internally. The following one liners will make git and consequently $ go get clone your package via SSH.
Github:
$ git config --global url."git#github.com:".insteadOf "https://github.com/"
BitBucket:
$ git config --global url."git#bitbucket.org:".insteadOf "https://bitbucket.org/"
I came across .netrc and found it relevant to this.
Create a file ~/.netrc with the following content:
machine github.com
login <github username>
password <github password or Personal access tokens >
Done!
Additionally, for latest GO versions, you might need to add this to the environment variables GOPRIVATE=github.com
(I've added it to my .zshrc)
netrc also makes my development environment setup better as my personal github access for HTTPS is been configured now to be used across the machine (just like my SSH configuration).
Generate GitHub personal access tokens: https://github.com/settings/tokens
See this answer for its use with Git on Windows specifically
Ref: netrc man page
If you want to stick with the SSH authentication, then mask the request to use ssh forcefully
git config --global url."git#github.com:".insteadOf "https://github.com/"
More methods for setting up git access: https://gist.github.com/technoweenie/1072829#gistcomment-2979908
That looks like the GitLab issue 5769.
In GitLab, since the repositories always end in .git, I must specify .git at the end of the repository name to make it work, for example:
import "example.org/myuser/mygorepo.git"
And:
$ go get example.org/myuser/mygorepo.git
Looks like GitHub solves this by appending ".git".
It is supposed to be resolved in “Added support for Go's repository retrieval. #5958”, provided the right meta tags are in place.
Although there is still an issue for Go itself: “cmd/go: go get cannot discover meta tag in HTML5 documents”.
After trying multiple solutions my problem still persisted. The final solution after setting up the ~/.netrc and SSH config file, was to add the following line to my ~/.bash_profile
export GOPRIVATE="github.com/[organization]"
I have created a user specific ssh-config, so my user automatically logs in with the correct credentials and key.
First I needed to generate an key-pair
ssh-keygen -t rsa -b 4096 -C "my#email.here"
and saved it to e.g ~/.ssh/id_my_domain. Note that this is also the keypair (private and public) I've connected to my Github account, so mine is stored in~/.ssh/id_github_com.
I have then created (or altered) a file called ~/.ssh/config with an entry:
Host github.com
HostName github.com
User git
IdentityFile ~/.ssh/id_github_com
On another server, the "ssh-url" is admin#domain.com:username/private-repo.git and the entry for this server would have been:
Host domain.com
HostName domain.com
User admin
IdentityFile ~/.ssh/id_domain_com
Just to clarify that you need ensure that the User, Host and HostName is set correctly.
Now I can just browse into the go path and then go get <package>, e.g go get main where the file main/main.go includes the package (from last example above) domain.com:username/private-repo.git.
For me, the solutions offered by others still gave the following error during go get
git#gl.nimi24.com: Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
What this solution required
As stated by others:
git config --global url."git#github.com:".insteadOf "https://github.com/"
Removing the passphrase from my ./ssh/id_rsa key which was used for authenticating the connection to the repository. This can be done by entering an empty password when prompted as a response to:
ssh-keygen -p
Why this works
This is not a pretty workaround as it is always better to have a passphrase on your private key, but it was causing issues somewhere inside OpenSSH.
go get uses internally git, which uses openssh to open the connection. OpenSSH takes the certs necessary for authentication from .ssh/id_rsa. When executing git commands from the command line an agent can take care of opening the id_rsa file for you so that you do not have to specify the passphrase every time, but when executed in the belly of go get, this did not work somewhy in my case. OpenSSH wants to prompt you then for a password but since it is not possible due to how it was called, it prints to its debug log:
read_passphrase: can't open /dev/tty: No such device or address
And just fails. If you remove the passphrase from the key file, OpenSSH will get to your key without that prompt and it works
This might be caused by Go fetching modules concurrently and opening multiple SSH connections to Github at the same time (as described in this article). This is somewhat supported by the fact that OpenSSH debug log showed the initial connection to the repository succeed, but later tried it again for some reason and this time opted to ask for a passphrase.
However the solution of using SSH connection multiplexing as put forward in the mentioned article did not work for me. For the record, the author suggested adding the collowing conf to the ssh config file for the affected host:
ControlMaster auto
ControlPersist 3600
ControlPath ~/.ssh/%r#%h:%p
But as stated, for me it did not work, maybe I did it wrong
After setting up GOPRIVATE and git config ...
People may still meeting problems like this when fetching private source:
https fetch: Get "https://private/user/repo?go-get=1": EOF
They can't use private repo without .git extension.
The reason is the go tool has no idea about the VCS protocol of this repo, git or svn or any other, unlike github.com or golang.org them are hardcoded into go's source.
Then the go tool will do a https query before fetching your private repo:
https://private/user/repo?go-get=1
If your private repo has no support for https request, please use replace to tell it directly :
require private/user/repo v1.0.0
...
replace private/user/repo => private.server/user/repo.git v1.0.0
https://golang.org/cmd/go/#hdr-Remote_import_paths
first I tried
[url "ssh://git#github.com/"]
insteadOf = https://github.com/
but it didn't worked for my local.
I tried
ssh -t git#github.com
and it shows my ssh is fine.
finally, I fix the problem to tell the go get to consider all as private and use ssh instead of HTTPS.
adding export GOPRIVATE=*
Make sure you remove your previous gitconfigs, I had the same issue.
Previously I executed gitconfig whose token was expired, when you execute the command next time with new token make sure to delete previous one.
For standalone/final repos, an as a quick fix, why don't just to name the module within the go.mod as a package using your company's domain ... ?
module go.yourcompany.tld/the_repo
go.yourcompany.tld don't even have to exist as a valid (sub)domain...
Also, in the same go.mod you can use replacement block/lines to use private repos previously cloned the same way (within a respective folder cloned also in $GOPATH/src/go.yourcompany.tld) (why do we have to depend too much in GitHub?)
Edit
Needless to say that a private repo usually shall be a private repo, typically a standard git repo, right? With that, why not just to git clone and then go get within the cloned folder?
It's Hard Code In Go Get. Not The Git Reason. So Modify Go Source.
Reason:
repoRootForImportDynamic Will Request: https://....go-get
// RepoRootForImportPath analyzes importPath to determine the
// version control system, and code repository to use.
func RepoRootForImportPath(importPath string, mod ModuleMode, security web.SecurityMode) (*RepoRoot, error) {
rr, err := repoRootFromVCSPaths(importPath, security, vcsPaths)
if err == errUnknownSite {
rr, err = repoRootForImportDynamic(importPath, mod, security)
if err != nil {
err = importErrorf(importPath, "unrecognized import path %q: %v", importPath, err)
}
}
if err != nil {
rr1, err1 := repoRootFromVCSPaths(importPath, security, vcsPathsAfterDynamic)
if err1 == nil {
rr = rr1
err = nil
}
}
So add gitlab domain to vcsPaths will ok.
Download go source code:
vi ./src/cmd/go/internal/vcs/vcs.go
Look for code below:
var vcsPaths = []*vcsPath{
// GitHub
{
pathPrefix: "github.com",
regexp: lazyregexp.New(`^(?P<root>github\.com/[A-Za-z0-9_.\-]+/[A-Za-z0-9_.\-]+)(/[A-Za-z0-9_.\-]+)*$`),
vcs: "git",
repo: "https://{root}",
check: noVCSSuffix,
},
add Code As Follow,XXXX Is Your Domain:
// GitLab
{
pathPrefix: "gitlab.xxxx.com",
regexp: lazyregexp.New(`^(?P<root>gitlab.xxxx\.com/[A-Za-z0-9_.\-]+/[A-Za-z0-9_.\-]+)(/[A-Za-z0-9_.\-]+)*$`),
vcs: "git",
repo: "https://{root}",
check: noVCSSuffix,
},
compile and replace go.

Is there any quick way to switch between docker hub accounts in command line?

Tired of typing in login and password. So the registry is the same, but accounts are different. Should I just make a script that replaces ~/.docker/config.json?
Unfortunately, no. The config.json can only hold one credentials value for each remote registry. Moreover, there is an open issue to handle multiple logins to dockerhub.
However, you can easily solve the problem using bash aliases. Edit your ~/.bashrc file and add the following lines:
alias dl1='docker login -u <user1> -p <password1>'
alias dl2='docker login -u <user2> -p <password2>'
Now you can do dl1 to login to account1 and dl2 to switch to account2
You can basically also do something similar if you are on MAC or Windows.

How can I tell if I'm logged in to a private Docker registry from a script?

How can I tell whether or not I'm logged in to a private Docker registry server from a script? In other words, has docker login some.registry.com been run successfully (and is still valid)?
Note: I'm asking about an arbitrary private registry, not the docker.io registry.
if docker login worked, you will find a .docker folder on your home directory (~/.docker/), with a config.json file with credentials in it.
otherwise you would get an error login in.
Note: docker determine what credentials to use by looking at the registry name:
if you do
docker pull myregistry.com/myimage:tag
docker will look if you're logged in, and if not will check if you have the credentials for the registry myregistry.com and login with those.
If not you will get a permission error
This is a bit hacky, but it works in most of the cases for me:
if ! grep -q "my.private.registry.com" ~/.docker/config.json ; then
docker login "my.private.registry.com"
fi
Basically, you search if there is a record of "my.private.registry.com" in ~/.docker/config.json. However, if the session is expired, this check won't catch it.
I believe the error message will vary by registry implementation. However, my own technique is to pull an image that doesn't exist and parse any error message:
$!/bin/sh
repo="$1"
msg=$(docker pull ${repo}/missing:missing 2>&1)
case "$msg" in
*"requested access to the resource is denied"*|*"pull access denied"*)
echo "Logged out";;
*"manifest unknown"*|*"not found"*)
echo "Logged in, or read access for anonymous allowed";;
*"Pulling from"*)
echo "Missing image was not so missing after all?";;
*)
echo "Unknown message: $msg";;
esac
This has been tested with the docker standalone registry and docker_auth. You'll want to test with registries that you may encounter.
If your registry server allows anonymous pulls and you want to verify a push is possible, you can create a dummy empty image and replace the pull with a push of that image. E.g.
#!/bin/sh
repo=$1
# note, I do not like the "." here, better to change it to an empty directory
# see "mktemp" for an option if you cannot make your own empty directory somewhere
docker build -t "${repo}/empty:test" -f - . <<EODF
FROM scratch
EODF
msg=$(docker push "${repo}/empty:test" 2>&1)
rc=$?
if [ "$rc" = "0" ]; then
echo "Access granted to push"
else
case "$msg" in
*"requested access to the resource is denied"*)
echo "Access denied";;
*)
echo "Unknown error message: $msg";;
esac
fi
If you are constrained to examining your local system, it's impossible to know!
... The only way to be sure the credentials docker has stored are still valid is to perform an operation that will cause them to be presented to the registry, and to see if the registry accepts them.
If you want to use the docker CLI to get an answer, then you could use #matanper suggestion of "login again" which will complete automatically if you still have valid credentials.
Another way is to try to pull an image known not to exist, which will show different error message when logged in or not e.g.
# NO VALID LOGIN:
$ docker pull 999999999999.dkr.ecr.us-west-2.amazonaws.com/this/image:does_not_exist
Error response from daemon: pull access denied for 999999999999.dkr.ecr.us-west-2.amazonaws.com/this/image, repository does not exist or may require 'docker login'
versus
# WITH VALID LOGIN:
$ docker pull 999999999999.dkr.ecr.us-west-2.amazonaws.com/this/image:does_not_exist
Error response from daemon: manifest for 999999999999.dkr.ecr.us-west-2.amazonaws.com/this/image:does_not_exist not found
(presume that you didn't want to pull because you don't want any delay or large data tranfser, so the above method is still 'ok')
In the past, when docker always stored credentials in ~/.docker/config.json (or equivalent for your OS), you could parse that file to get the currently stored credentials and then run a simple list operation using curl or similar. However, recent docker versions store the credentials in host OS specific stores (e.g. the keychain on Mac OS X) so that is no longer a portable methodology. If portability is not important, you could still try something like that - the hash in config.json is just the base64 encoded username & password, separated by a colon, as is standard for HTTP basic auth e.g. on linux, with jq to parse JSON, and base64 to decode base64:
$ cat ~/.docker/config.json | jq -r '.auths["registry.example.com"].auth' | base64 -d
username:password
So, completing that with a registry list operation using curl:
REGISTRY="registry.example.com"
CREDENTIALS="$(cat ~/.docker/config.json | jq -r ".auths[\"${REGISTRY}\"].auth" | base64 -d)"
curl -sSf --max-time 3 --user "${CREDENTIALS}" "https://${REGISTRY}/v2/_catalog"
will return exit code zero, and a JSON response if the CREDENTIALS are good; or a non-zero exit code if not
{
"repositories": [
"jamesjj/test-image",
"jamesjj/other-image",
...
...
}
NOTE: When parsing the JSON, the registry address key may or may not include the schema https://, depending on how the original login was performed, so cat ~/.docker/config.json | jq -r ".auths[\"${REGISTRY}\"].auth" | base64 -d)"
... may need to be:
cat ~/.docker/config.json | jq -r ".auths[\"https://${REGISTRY}\"].auth" | base64 -d)"
This is a little hacky, I think until docker will have a command to check login, there won't be any good solution.
You can in your bash script try to login with timeout of x seconds, if you aren't logged in the command will try to prompt for username and then it will timeout with status 124. If you are indeed logged in, it will just log you in again using the save credentials and continue with status 0
#!/bin/bash
timeout -s SIGKILL 3s docker login some.registry.com >/dev/null 2>&1
if [ $? -eq 0 ]
then
echo Logged In!
else
echo Not logged in...
fi
This is somewhat old, but I needed the same thing today. I found
this question first, and from that, I derived this, which I think answers the OP's question directly:
# try to login. If you are logged in, it exits happily. If you aren't, it prompts
# for credentials. By redirecting stdin to /dev/null, we guarantee this fails
docker login ${MY_REG} < /dev/null >& /dev/null
if [ $? -eq 0]; then
echo "Already logged in"
else
echo "Login required"
fi
If MY_REG is unset, it will check against docker.io. Otherwise, against the registry you set.
I do understand that there is a difference between storing credentials and authenticating with the registry, and perhaps that's an important nuance in the OP's use case. In my case, I just wanted to know the former. If the latter is important, then by all means, pull an image.
You can parse .docker/config.json and try to manually connect to each registry specified in the file. The file contains the registry address and encoded username and password so you can script this process. You can do that using a library like docker-registry-client.
pip install docker-registry-client
And then:
import base64
import docker_registry_client
import json
import os.path
def get_authed_registries():
result = []
config_path = os.path.expanduser("~/.docker/config.json")
if not os.path.isfile(config_path):
print("No docker config")
return []
docker_config = json.load(open(config_path))
for registry, auth in docker_config.get("auths", {}).items():
username, password = base64.b64decode(auth["auth"]).decode("utf-8").split(":", 1)
if not registry:
registry = "https://index.docker.io/v1/"
if not registry.startswith("http"):
registry = "https://" + registry
try:
rc = docker_registry_client.DockerRegistryClient(registry, username=username, password=password)
result.append(registry)
except Exception, e:
print(registry, "failed:", e)
return result
get_authed_registries()
A few caveats:
This may fail if you're using a credential store.
Tested on Python 2.7. Might need small adjustments for Python 3.
This code works for authentication, but any further actions on the registry fail. You will need to strip away API version (/v1, /v2, etc.) from the host name for that.
The code assumes all registries are HTTPS (unless specified otherwise in config.json)
An even more correct version will probably strip away anything but the hostname and try both v1 and v2.
That said, I was able to get a list of logged-in registries and it correctly ignored expired ECR login.
When you do the
Docker login <private registry> -u <user> -p <password>
command from your terminal, you will have a response: (stored in $?)
0
Login Succeeded
if you were successful.
In your shell script, you could just look at the response you're receiving, if it does not equal 0, you've failed to login.
sudo docker login <registry> -u <uname> -p <password>
if [ $? -ne 0 ]; then
echo Login failed!
else
echo Login OK!
fi

How to "Unlock Jenkins"?

I am installing Jenkins 2 on windows,after installing,a page is opened,URL is:
http://localhost:8080/login?from=%2F
content of the page is like this:
Question:
How to "Unlock Jenkins"?
PS:I have looked for the answer in documentation and google.
Starting from version 2.0 of Jenkins you may use
-Djenkins.install.runSetupWizard=false
to prevent this screen.
Accroding to documentation
jenkins.install.runSetupWizard - Set to false to skip install wizard. Note that this leaves Jenkins unsecured by default.
Development-mode only: Set to true to not skip showing the setup wizard during Jenkins development.
More details about Jenkins properties can be found on this Jenkins Wiki page.
Check https://wiki.jenkins-ci.org/display/JENKINS/Logging to see where Jenkins is logging its files.
e.g. for Linux, use the command: less /var/log/jenkins/jenkins.log
And scroll down to the part: "Jenkins initial setup is required. An admin user has been created ... to proceed to installation:
[randompasswordhere] <--- Copy and paste
Linux
By default logs should be made available in /var/log/jenkins/jenkins.log, unless customized in /etc/default/jenkins (for *.deb) or via /etc/sysconfig/jenkins (for */rpm)
Windows
By default logs should be at %JENKINS_HOME%/jenkins.out and %JENKINS_HOME%/jenkins.err, unless customized in %JENKINS_HOME%/jenkins.xml
Mac OS X
Log files should be at /var/log/jenkins/jenkins.log, unless customized in org.jenkins-ci.plist
open file: e:\Program Files (x86)\Jenkins\secrets\initialAdminPassword
copy content file: 47c5d4f760014e54a6bffc27bd95c077
paste in input: http://localhost:8080/login?from=%2F
DONE
Some of the above instructions seem to have gone out of date. As of the released version 2.0, creating the following file will cause Jenkins to skip the unlock screen:
${JENKINS_HOME}/jenkins.install.InstallUtil.lastExecVersion
This file must contain the string 2.0 without any line terminators. I'm not sure if this is required but Jenkins also sets the owner/group to be the same as the Jenkins server, so that's probably a good thing to mimic as well.
I did not need to create the upgraded or .last_exec_version files.
I assume you were running jenkins.war manually with java -jar jenkins.war, then all logging information by default is output to standard out, just type the token to unlock jenkins2.0.
If you were not running jenkins with java -jar jenkins.war, then you can always follow this Official Document to find the correct log location.
Open your terminal and type code below to find all the containers.
docker container list -a
You will find jenkinsci/blueocean and/or docker:dind if not than
docker container run --name jenkins-docker --rm --detach ^
--privileged --network jenkins --network-alias docker ^
--env DOCKER_TLS_CERTDIR=/certs ^
--volume jenkins-docker-certs:/certs/client ^
--volume jenkins-data:/var/jenkins_home ^
--volume "%HOMEDRIVE%%HOMEPATH%":/home ^
docker:dind
and
docker container run --name jenkins-blueocean --rm --detach ^
--network jenkins --env DOCKER_HOST=tcp://docker:2376 ^
--env DOCKER_CERT_PATH=/certs/client --env DOCKER_TLS_VERIFY=1 ^
--volume jenkins-data:/var/jenkins_home ^
--volume jenkins-docker-certs:/certs/client:ro ^
--volume "%HOMEDRIVE%%HOMEPATH%":/home ^
--publish 8080:8080 --publish 50000:50000 jenkinsci/blueocean
run command
docker run jenkinsci/blueocean
or
docker run docker:dind
Copy and Paste the secret key.
One method to prevent the installation wizard is to do the following in $JENKINS_HOME:
Create an empty file named .last_exec_version
Create a file named upgraded
If left empty, a banner will prompt you to "upgrade" to 2.0 (which just means install a bunch of new plugins like Pipeline)
If the contents of that file is 2.0, you'll receive no banner and it will act like an regular old Jenkins install
Remember, this wizard is in place to prevent unauthorized access to Jenkins during setup. However, bypassing this wizard can be useful if, for example, you want to deploy automated installations of Jenkins with something like Ansible/Puppet/etc.
This was tested against Jenkins 2.0-beta-1 – so these instructions may not work in future beta or stable releases.
In the mac use:
sudo more /Users/Shared/Jenkins/Home/secrets/initialAdminPassword
I have seen a lot of response to the question, most of them were actually solution to it but they solve the problem when jenkins is actually run as a standalone CI without Application container using the command:
java -jar jenkins.war
But when running on Tomcat as it is the case in this scenario, Jenkins logs are displayed on the catalina logs since the software is running on a container.
So you need to go to the logs folder:
C:\Program Files\tomcat_folder\Tomcat 8.5\logs\catalina.log
in my own case. Just scroll almost to the middle to look for a generated password which is essentially a token and copy and paste it to unlock jenkins.
I hope this fix your problem.
Step 1: Open the terminal on your mac
Step 2: Concatenate or Paste
sudo cat **/Users/Shared/Jenkins/Home/secrets/initialAdminPassword**
Step 3: It will ask for password, type your mac password and enter
Step 4: A key would be generated.
Step 5: Copy and paste the security token in Jenkins
Step 6: Click continue
I found the token in the following file in the installation dir:
<jenkins install dir>\users\admin\config.xml
and the element
<jenkins.install.SetupWizard_-AuthenticationKey>
<key> THE KEY </key>
</jenkins.install.SetupWizard_-AuthenticationKey>
You might see it in the catalina.out. I installed Jenkins war in tomcat and I can see this in the catalina.out
The below method does not work anymore on 2.42.2
Create an empty file named .last_exec_version
Create a file named upgraded
If left empty, a banner will prompt you to "upgrade" to 2.0 (which just means install a bunch of new plugins like Pipeline)
If the contents of that file is 2.0, you'll receive no banner and it will act like an regular old Jenkins install
mostly jenkins will show you the path for initialAdminPassword if you dont find it there, then you have to check jenkins logs
in log you will see
05-May-2017 01:01:41.854 INFO [Jenkins initialization thread] jenkins.install.SetupWizard.init
Jenkins initial setup is required. An admin user has been created and a password generated.
Please use the following password to proceed to installation:
7c249e4ed93c4596972f57e55f7ff32e
This may also be found at: /opt/tomcat/.jenkins/secrets/initialAdminPassword
Use a lil shortcut to get to the folder:
cmd + shift + g
then insert /Users/Shared/Jenkins
there u can see the secrets folder - probably shows that it's locked.
to unlock it: right click on the folder and click info + click on the lock at the bottom. now u can change the rights shown at the bottom of the window
hope that helped :)
Greetings, Stefanie ^__^
If unable to find Jenkins password in the location C:\Windows\System32\config\systemprofile\.jenkins\secrets\initialAdminPassword
by installing Jenkins generic war on Tomcat server, try below
Solution:
Set environmental variable JENKINS_HOME to your jenkins path say
JENKINS_HOME ="C:/users/username/apachetomcat/webapps/jenkins"
Place Jenkins.war in the webapp folder of Tomcat and start Tomcat,
initial admin password gets generated in the path
C:\Program Files (x86)\Apache Software Foundation\Tomcat 9.0\webapps\jenkins\secrets\initialAdminPassword
Yet another way to bypass the unlock screen is to copy the UpgradeWizard state to the InstallUtil last execution version, add an install.runSetupWizard file with the contents 'false', and update the config.xml installStateName from NEW to RUNNING.
docker exec -it jenkins bash
sed -i s/NEW/RUNNING/ /var/jenkins_home/config.xml
echo 'false' > /var/jenkins_home/jenkins.install.runSetupWizard
cp /var/jenkins_home/jenkins.install.UpgradeWizard.state /var/jenkins_home/jenkins.install.InstallUtil.lastExecVersion
exit
docker restart jenkins
For reference, this is the command I use to run jenkins:
docker run --rm --name jenkins --network host -u root -d -v jenkins:/var/jenkins_home -v /var/run/docker.sock:/var/run/docker.sock jenkinsci/blueocean:1.16.0
You will also want to update the config with the Root URL:
echo "<?xml version='1.1' encoding='UTF-8'?><jenkins.model.JenkinsLocationConfiguration><jenkinsUrl>http://<IP>:8080/</jenkinsUrl></jenkins.model.JenkinsLocationConfiguration>" > jenkins.model.JenkinsLocationConfiguration.xml
exit
docker restart jenkins
In case, if you installed/upgraded new versions of jenkins and unable to find admin credentials on server then, ...
if you are using old version of jenkins and on the top of it you are trying to reinstall/upgrade new version of jenkins then,
the files under "JENKINS_HOME", namely -
${JENKINS_HOME}/jenkins.install.InstallUtil.lastExecVersion
${JENKINS_HOME}/jenkins.install.UpgradeWizard.state
will cause jenkins to skip the unlock (or admin credentials screen) and webpage directly ask you for username and password. even on server you will not able to find "${JENKINS_HOME}/secrets/initialAdminPassword".
In such case, don't get panic. just try to use old admin user creds in newly installed/upgraded jenkins server.
In simple language, if you have admin creds as admin/admin in old version of jenkins server then, after upgrading jenkins server, the new server won't ask you set password for admin user again. in fact it will use old creds only.
I have found the password in C:\Program Files\Jenkins\jenkins.err. Open jenkins.err text file and scroll down, and you will find the password.
Go to C:\Program Files (x86)\Jenkins\secrets
then with notepad ++ open file initail Admin Password and paste its content.
thats it
-->if you are using linux machine, then login as root user: sudo su
-->then go to the below path: cd /var/lib/jenkins/secrets
-->just view the IntialAdminPassword file ,you can see the secret key.
-->paste the secret key into jenkins window,it will be unlocked.
https://issues.jenkins-ci.org/browse/JENKINS-35981
Try this %2Fjenkins%2F instead of %2Fjenkins in the browser
Open the terminal on your mac and open new window (command+T)
Paste sudo cat /Users/Shared/Jenkins/Home/secrets/initialAdminPassword
It will ask for password, type your password(i gave my mac password, i haven't check if any other password would work) and enter
A key would be generated.
Copy the key and paste it where it asks you to enter admin password
click continue
The problem can be fixed in latest version: mine is 2.4. The error comes because of %2fjenkins%2f in URL. The previous version was coming with %2fjenkins and the same error used to come. They have resolved the issue, but the URL has been changed from %2fjenkins to %2fjenkins%. So as a summary in the URL currently %2fjenkins% is coming. before passing the admin password change it to %2fjenkins. Along with that add a .last_exec_version empty file.
If someone chooses running Jenkins as a Docker container, may face the same problem with me.
Because accessing-the-jenkins-blue-ocean-docker-container is quite different,
Common problem is /var/lib/jenkins/secrets: No such file or directory
You need to access through Docker, the link Jenkins provide is quite helpful.
Except <docker-container-name> maybe not specified, then you may need to use the container ID.
After
docker exec -it jenkins-blueocean bash
or
docker exec -it YOUR_JENKINS_CONTAINER_ID bash
The /var/lib/jenkins/secrets/initialAdminPassword would be accessible.
The password would be there.
I have setup Jenkins using Brew, But when I restarted Mac Jenkins was asking for initialAdminPassword(The screenshot attached in question)
And the problem was it was not generated under sercret directory.
So I'd found the Jenkins process which was running on port: 8080 using: $ sudo lsof -i -n -P | grep TCP and killed it using $ sudo kill 66(66 was process id).
Then I downloaded the latest jenkins .war file from: https://jenkins.io/download/
And executed command: $ java -jar jenkins.war (Make sure you are in jenkins.war directory).
And that's it everything is working fine.
This works well when you are stuck with Docker on Windows and are using Git-Bash
Presuming something like:
# docker run --detach --publish 8080:8080 --volume jenkins_home:/var/jenkins_home --name jenkins jenkins/jenkins:lts
Execute to get the Container ID, for example "d56686cb700d"
# docker ps -l
Now tell Docker to return the password written in the logs for that Container ID:
# docker logs d56686cb700d 2>&1 | grep -A5 -B5 Admin
2>&1 redirects stderr to stdout
-A5 includes 5 lines AFTER the line with "Admin" in it
-B5 includes 5 lines BEFORE the line with "Admin" in it
Output example:
Jenkins initial setup is required. An admin user has been created and a password generated.
Please use the following password to proceed to installation:
47647383733f4387a0d53c873334b707
This may also be found at: /var/jenkins_home/secrets/initialAdminPassword
*************************************************************
*************************************************************
*************************************************************
I found it under below directory. Full issue detail https://github.com/jenkinsci/ibm-security-appscansource-scanner-plugin/issues/2
C:\Windows\SysWOW64\config\systemprofile\AppData\Local\Jenkins\.jenkins
Open jenkins.err file in C:\Program Files\Jenkins\.
In that file check for a hash key after this line
Jenkins initial setup is required. An admin user has been created and a password generated.
Please use the following password to proceed to installation:
And paste it there in the jenkins prompt. Worked for me.
To solve this problem for docker container in Ubuntu 18.04.5 LTS (Bionic Beaver) - Ubuntu Releases
1- connect to your docker server or ubuntu server witch ssh or other method
2- run sudo docker ps
3- copy the container name parameter ("NAMES")
4- run sudo docker logs "your_parameters_NAMES_VALUES"
5- Find the folowing sentence "Jenkins initial setup is required. An admin user has been created and a password generated.
Please use the following password to proceed to installation:" and copy the password

Resources