.docker/config.json vs .dockercfg - docker

When I do a docker login to a private repository using docker 1.10.1, an entry is created in my ~/.docker/config.json file. Is this file in the same format as what I see being called a .dockercfg file? Is the config.json file interchangeable with a .dockercfg file?

I assume config.json is the new .dockercfg file.
See "docker/cliconfig/config.go"
// ConfigFileName is the name of config file
ConfigFileName = "config.json"
oldConfigfile = ".dockercfg"
The new config file is now documented under man/config-json.5.md
That was introduced in commit 18c9b6c in docker 1.7.0 (April 2015)
Add .docker/config.json and support for HTTP Headers
This PR does the following:
migrated ~/.dockerfg to ~/.docker/config.json.
The data is migrated but the old file remains in case its needed.
Note: since 2016:
config-json.5 to docker-config-json.5 (commit a596d3d, docker docs-v1.12.0-2016-07-28 )
the documentation was moved to docker/cli (commit b5579a4, Docker v17.07.0-ce-rc1)
It differs from docker-daemon.8.md which uses by default /etc/docker/daemon.json, the daemon configuration file introduced with Docker v1.10.
So:
config.json is for the docker CLI
daemon.json is for the dockerd (daemon) CLI
But the config.json (which applies to all containers) does not include docker run network settings: --net=host could not be specified in that config file.
Update 2021 (since 2017), as noted by slm in the comments:
The credentials are now stored in whatever your OS makes use of for managing secrets.
On MacOS they get stored under Keychain, you can find them by looking for "Docker Credentials" under All Items
You can see the project docker/docker-credential-helpers: a suite of programs to use native stores to keep Docker credentials safe.
See docker login/Credentials Store for more.

Related

Can I specify a custom location for docker temporary files?

I'm trying to run docker in a partially locked-down environment, with /etc on a read-only mount point and a "/data" folder in a read/write mount point. I've added an /etc/docker/daemon.json file:
{
"data-root": "/data/docker"
}
but dockerd is failing on startup with this error:
failed to start daemon: Error saving key file: open /etc/docker/.tmp-key.json128868007: read-only file system
Can I stop dockerd from trying to write into /etc? Are there best practices for running docker on a host with read-only mounts?
EDIT: Turns out there was only one file being written: /etc/docker/key.json which is talked about in detail here. The .tmp-key.json bit is likely a part of some atomic file write code.
Looks like only the "key.json" file is written to /etc. After some digging, I found this PR which talks about making it configurable. As of docker 19.03.6, the option is still available for use in the daemon.json file as "deprecated-key-path": "/path/to/file".

Security of a Docker image

I am considering to package a Rust application into a Docker container.
The current version of that application contains various credential files used to register to Discord API or Google API through a service account key.
Would these files be accessibles if I package my application as such?
[EDIT: added Dockerfile]
FROM rust:1.28.0
WORKDIR /usr/src/<application>
COPY . .
RUN cargo install --force --path .
CMD ["<application>"]
Never put actual credentials into anything that might not be accessed by you and only you.
You basically have two options:
1) Have your application pull the required credentials from its environment, then set these variables when you start the container. see docs
2) Have your application read the credentials from a config file, that doesn't get pulled into the docker image. Then, when running the container, mount that file into it, see docs
You could actually do both: Have an environment variable that tells your application whether it should look for a config file ( maybe in production) and if that variable is unset, check the environment (for development).
Edit: It's best practice to create a .dockerignore File in your build-context, containing the name (or path) of the file holding the credentials.

Change "hosts" / "-h" Docker for Windows in daemon.json

I'm trying to change daemon.json on Docker Desktop for Windows (Windows 10 Aniversary latest updates installed) 1.13.0-rc5 so I can change the "hosts": [] setting like this:
{
"hosts": [
"tcp://0.0.0.0",
"http://0.0.0.0"
]
}
However, after change the settings using the settings app I got this error:
Docker daemon failed with message: unable to configure the Docker
daemon with file C:\ProgramData\docker\config\daemon.json: the
following directives are specified both as a flag and in the
configuration file: hosts: (from flag:
[npipe:////./pipe/docker_engine_windows], from file: [tcp://0.0.0.0
http://0.0.0.0])
Looks like the daemon is already started with -H flag and the json config isn't merged with it.
So, how can we change those settings by either json file or change the dockerd startup parameters?
You have a similar case with issue 22339:
This is expected; you cannot specify options both as a flag and in the configuration file (daemon.json).
If you change your DOCKER_OPTS to DOCKER_OPTS="" and restart, then it should work. We explicitly don't "merge" these configurations.
Or add in docker.conf
[Service]
ExecStart=
ExecStart=/path/to/dockerd
# or
ExecStart=/path/to/dockerd daemon
But the official stance remains:
There's no bug in the systemd configuration, to override defaults in a systemd unit file, you can use a drop-in file, as described in "Custom Docker daemon options".
Producing an error if both a flag and an option in daemon.json are provided was a design decision when implementing that (in general, flags should always have precedence over configuration files); automatically merging options was not an option, as this would lead to unexpected results (was the intent to override an option, or to add to an option?)
PR 27473 was rejected, for issue 21559.

How to "Unlock Jenkins"?

I am installing Jenkins 2 on windows,after installing,a page is opened,URL is:
http://localhost:8080/login?from=%2F
content of the page is like this:
Question:
How to "Unlock Jenkins"?
PS:I have looked for the answer in documentation and google.
Starting from version 2.0 of Jenkins you may use
-Djenkins.install.runSetupWizard=false
to prevent this screen.
Accroding to documentation
jenkins.install.runSetupWizard - Set to false to skip install wizard. Note that this leaves Jenkins unsecured by default.
Development-mode only: Set to true to not skip showing the setup wizard during Jenkins development.
More details about Jenkins properties can be found on this Jenkins Wiki page.
Check https://wiki.jenkins-ci.org/display/JENKINS/Logging to see where Jenkins is logging its files.
e.g. for Linux, use the command: less /var/log/jenkins/jenkins.log
And scroll down to the part: "Jenkins initial setup is required. An admin user has been created ... to proceed to installation:
[randompasswordhere] <--- Copy and paste
Linux
By default logs should be made available in /var/log/jenkins/jenkins.log, unless customized in /etc/default/jenkins (for *.deb) or via /etc/sysconfig/jenkins (for */rpm)
Windows
By default logs should be at %JENKINS_HOME%/jenkins.out and %JENKINS_HOME%/jenkins.err, unless customized in %JENKINS_HOME%/jenkins.xml
Mac OS X
Log files should be at /var/log/jenkins/jenkins.log, unless customized in org.jenkins-ci.plist
open file: e:\Program Files (x86)\Jenkins\secrets\initialAdminPassword
copy content file: 47c5d4f760014e54a6bffc27bd95c077
paste in input: http://localhost:8080/login?from=%2F
DONE
Some of the above instructions seem to have gone out of date. As of the released version 2.0, creating the following file will cause Jenkins to skip the unlock screen:
${JENKINS_HOME}/jenkins.install.InstallUtil.lastExecVersion
This file must contain the string 2.0 without any line terminators. I'm not sure if this is required but Jenkins also sets the owner/group to be the same as the Jenkins server, so that's probably a good thing to mimic as well.
I did not need to create the upgraded or .last_exec_version files.
I assume you were running jenkins.war manually with java -jar jenkins.war, then all logging information by default is output to standard out, just type the token to unlock jenkins2.0.
If you were not running jenkins with java -jar jenkins.war, then you can always follow this Official Document to find the correct log location.
Open your terminal and type code below to find all the containers.
docker container list -a
You will find jenkinsci/blueocean and/or docker:dind if not than
docker container run --name jenkins-docker --rm --detach ^
--privileged --network jenkins --network-alias docker ^
--env DOCKER_TLS_CERTDIR=/certs ^
--volume jenkins-docker-certs:/certs/client ^
--volume jenkins-data:/var/jenkins_home ^
--volume "%HOMEDRIVE%%HOMEPATH%":/home ^
docker:dind
and
docker container run --name jenkins-blueocean --rm --detach ^
--network jenkins --env DOCKER_HOST=tcp://docker:2376 ^
--env DOCKER_CERT_PATH=/certs/client --env DOCKER_TLS_VERIFY=1 ^
--volume jenkins-data:/var/jenkins_home ^
--volume jenkins-docker-certs:/certs/client:ro ^
--volume "%HOMEDRIVE%%HOMEPATH%":/home ^
--publish 8080:8080 --publish 50000:50000 jenkinsci/blueocean
run command
docker run jenkinsci/blueocean
or
docker run docker:dind
Copy and Paste the secret key.
One method to prevent the installation wizard is to do the following in $JENKINS_HOME:
Create an empty file named .last_exec_version
Create a file named upgraded
If left empty, a banner will prompt you to "upgrade" to 2.0 (which just means install a bunch of new plugins like Pipeline)
If the contents of that file is 2.0, you'll receive no banner and it will act like an regular old Jenkins install
Remember, this wizard is in place to prevent unauthorized access to Jenkins during setup. However, bypassing this wizard can be useful if, for example, you want to deploy automated installations of Jenkins with something like Ansible/Puppet/etc.
This was tested against Jenkins 2.0-beta-1 – so these instructions may not work in future beta or stable releases.
In the mac use:
sudo more /Users/Shared/Jenkins/Home/secrets/initialAdminPassword
I have seen a lot of response to the question, most of them were actually solution to it but they solve the problem when jenkins is actually run as a standalone CI without Application container using the command:
java -jar jenkins.war
But when running on Tomcat as it is the case in this scenario, Jenkins logs are displayed on the catalina logs since the software is running on a container.
So you need to go to the logs folder:
C:\Program Files\tomcat_folder\Tomcat 8.5\logs\catalina.log
in my own case. Just scroll almost to the middle to look for a generated password which is essentially a token and copy and paste it to unlock jenkins.
I hope this fix your problem.
Step 1: Open the terminal on your mac
Step 2: Concatenate or Paste
sudo cat **/Users/Shared/Jenkins/Home/secrets/initialAdminPassword**
Step 3: It will ask for password, type your mac password and enter
Step 4: A key would be generated.
Step 5: Copy and paste the security token in Jenkins
Step 6: Click continue
I found the token in the following file in the installation dir:
<jenkins install dir>\users\admin\config.xml
and the element
<jenkins.install.SetupWizard_-AuthenticationKey>
<key> THE KEY </key>
</jenkins.install.SetupWizard_-AuthenticationKey>
You might see it in the catalina.out. I installed Jenkins war in tomcat and I can see this in the catalina.out
The below method does not work anymore on 2.42.2
Create an empty file named .last_exec_version
Create a file named upgraded
If left empty, a banner will prompt you to "upgrade" to 2.0 (which just means install a bunch of new plugins like Pipeline)
If the contents of that file is 2.0, you'll receive no banner and it will act like an regular old Jenkins install
mostly jenkins will show you the path for initialAdminPassword if you dont find it there, then you have to check jenkins logs
in log you will see
05-May-2017 01:01:41.854 INFO [Jenkins initialization thread] jenkins.install.SetupWizard.init
Jenkins initial setup is required. An admin user has been created and a password generated.
Please use the following password to proceed to installation:
7c249e4ed93c4596972f57e55f7ff32e
This may also be found at: /opt/tomcat/.jenkins/secrets/initialAdminPassword
Use a lil shortcut to get to the folder:
cmd + shift + g
then insert /Users/Shared/Jenkins
there u can see the secrets folder - probably shows that it's locked.
to unlock it: right click on the folder and click info + click on the lock at the bottom. now u can change the rights shown at the bottom of the window
hope that helped :)
Greetings, Stefanie ^__^
If unable to find Jenkins password in the location C:\Windows\System32\config\systemprofile\.jenkins\secrets\initialAdminPassword
by installing Jenkins generic war on Tomcat server, try below
Solution:
Set environmental variable JENKINS_HOME to your jenkins path say
JENKINS_HOME ="C:/users/username/apachetomcat/webapps/jenkins"
Place Jenkins.war in the webapp folder of Tomcat and start Tomcat,
initial admin password gets generated in the path
C:\Program Files (x86)\Apache Software Foundation\Tomcat 9.0\webapps\jenkins\secrets\initialAdminPassword
Yet another way to bypass the unlock screen is to copy the UpgradeWizard state to the InstallUtil last execution version, add an install.runSetupWizard file with the contents 'false', and update the config.xml installStateName from NEW to RUNNING.
docker exec -it jenkins bash
sed -i s/NEW/RUNNING/ /var/jenkins_home/config.xml
echo 'false' > /var/jenkins_home/jenkins.install.runSetupWizard
cp /var/jenkins_home/jenkins.install.UpgradeWizard.state /var/jenkins_home/jenkins.install.InstallUtil.lastExecVersion
exit
docker restart jenkins
For reference, this is the command I use to run jenkins:
docker run --rm --name jenkins --network host -u root -d -v jenkins:/var/jenkins_home -v /var/run/docker.sock:/var/run/docker.sock jenkinsci/blueocean:1.16.0
You will also want to update the config with the Root URL:
echo "<?xml version='1.1' encoding='UTF-8'?><jenkins.model.JenkinsLocationConfiguration><jenkinsUrl>http://<IP>:8080/</jenkinsUrl></jenkins.model.JenkinsLocationConfiguration>" > jenkins.model.JenkinsLocationConfiguration.xml
exit
docker restart jenkins
In case, if you installed/upgraded new versions of jenkins and unable to find admin credentials on server then, ...
if you are using old version of jenkins and on the top of it you are trying to reinstall/upgrade new version of jenkins then,
the files under "JENKINS_HOME", namely -
${JENKINS_HOME}/jenkins.install.InstallUtil.lastExecVersion
${JENKINS_HOME}/jenkins.install.UpgradeWizard.state
will cause jenkins to skip the unlock (or admin credentials screen) and webpage directly ask you for username and password. even on server you will not able to find "${JENKINS_HOME}/secrets/initialAdminPassword".
In such case, don't get panic. just try to use old admin user creds in newly installed/upgraded jenkins server.
In simple language, if you have admin creds as admin/admin in old version of jenkins server then, after upgrading jenkins server, the new server won't ask you set password for admin user again. in fact it will use old creds only.
I have found the password in C:\Program Files\Jenkins\jenkins.err. Open jenkins.err text file and scroll down, and you will find the password.
Go to C:\Program Files (x86)\Jenkins\secrets
then with notepad ++ open file initail Admin Password and paste its content.
thats it
-->if you are using linux machine, then login as root user: sudo su
-->then go to the below path: cd /var/lib/jenkins/secrets
-->just view the IntialAdminPassword file ,you can see the secret key.
-->paste the secret key into jenkins window,it will be unlocked.
https://issues.jenkins-ci.org/browse/JENKINS-35981
Try this %2Fjenkins%2F instead of %2Fjenkins in the browser
Open the terminal on your mac and open new window (command+T)
Paste sudo cat /Users/Shared/Jenkins/Home/secrets/initialAdminPassword
It will ask for password, type your password(i gave my mac password, i haven't check if any other password would work) and enter
A key would be generated.
Copy the key and paste it where it asks you to enter admin password
click continue
The problem can be fixed in latest version: mine is 2.4. The error comes because of %2fjenkins%2f in URL. The previous version was coming with %2fjenkins and the same error used to come. They have resolved the issue, but the URL has been changed from %2fjenkins to %2fjenkins%. So as a summary in the URL currently %2fjenkins% is coming. before passing the admin password change it to %2fjenkins. Along with that add a .last_exec_version empty file.
If someone chooses running Jenkins as a Docker container, may face the same problem with me.
Because accessing-the-jenkins-blue-ocean-docker-container is quite different,
Common problem is /var/lib/jenkins/secrets: No such file or directory
You need to access through Docker, the link Jenkins provide is quite helpful.
Except <docker-container-name> maybe not specified, then you may need to use the container ID.
After
docker exec -it jenkins-blueocean bash
or
docker exec -it YOUR_JENKINS_CONTAINER_ID bash
The /var/lib/jenkins/secrets/initialAdminPassword would be accessible.
The password would be there.
I have setup Jenkins using Brew, But when I restarted Mac Jenkins was asking for initialAdminPassword(The screenshot attached in question)
And the problem was it was not generated under sercret directory.
So I'd found the Jenkins process which was running on port: 8080 using: $ sudo lsof -i -n -P | grep TCP and killed it using $ sudo kill 66(66 was process id).
Then I downloaded the latest jenkins .war file from: https://jenkins.io/download/
And executed command: $ java -jar jenkins.war (Make sure you are in jenkins.war directory).
And that's it everything is working fine.
This works well when you are stuck with Docker on Windows and are using Git-Bash
Presuming something like:
# docker run --detach --publish 8080:8080 --volume jenkins_home:/var/jenkins_home --name jenkins jenkins/jenkins:lts
Execute to get the Container ID, for example "d56686cb700d"
# docker ps -l
Now tell Docker to return the password written in the logs for that Container ID:
# docker logs d56686cb700d 2>&1 | grep -A5 -B5 Admin
2>&1 redirects stderr to stdout
-A5 includes 5 lines AFTER the line with "Admin" in it
-B5 includes 5 lines BEFORE the line with "Admin" in it
Output example:
Jenkins initial setup is required. An admin user has been created and a password generated.
Please use the following password to proceed to installation:
47647383733f4387a0d53c873334b707
This may also be found at: /var/jenkins_home/secrets/initialAdminPassword
*************************************************************
*************************************************************
*************************************************************
I found it under below directory. Full issue detail https://github.com/jenkinsci/ibm-security-appscansource-scanner-plugin/issues/2
C:\Windows\SysWOW64\config\systemprofile\AppData\Local\Jenkins\.jenkins
Open jenkins.err file in C:\Program Files\Jenkins\.
In that file check for a hash key after this line
Jenkins initial setup is required. An admin user has been created and a password generated.
Please use the following password to proceed to installation:
And paste it there in the jenkins prompt. Worked for me.
To solve this problem for docker container in Ubuntu 18.04.5 LTS (Bionic Beaver) - Ubuntu Releases
1- connect to your docker server or ubuntu server witch ssh or other method
2- run sudo docker ps
3- copy the container name parameter ("NAMES")
4- run sudo docker logs "your_parameters_NAMES_VALUES"
5- Find the folowing sentence "Jenkins initial setup is required. An admin user has been created and a password generated.
Please use the following password to proceed to installation:" and copy the password

How to set volume for dokku-persistent-storage

I am trying to use dokku-persistent-storage so my uploads for my rails app stay on the server, but I don't quite understand how to build the path since I am new to Dokku and Docker.
(I am running this on an Ubuntu droplet on Digital Ocean)
I'm not sure if it should be something like this:
[SERVER IP ADDRESS]/home/dokku/myapp/public_folder
or
/home/dokku/myapp/public_folder
or if i'm way off and it should be something completely different.
This is what the github section says about it:
In your applications folder (/home/dokku/app_name) create a file called PERSISTENT_STORAGE.
Inside this file list one volume-map/volume per line to mount. For example:
/host/path:/container/path
/another/container/path
The above example will result in the following arguments being passed to docker during deploy and docker run:
-v /host/path:/container/path -v /another/container/path
Move information on docker volumes can be found here: http://docs.docker.io/en/latest/use/working_with_volumes/
I am not into Ruby or dokku, but if I understood correctly, you want your docker to have a persistent storage on the host machine.
PERSISTENT_STORAGE file, as to the documentation that you've quoted, contains mappings from host file-system directories to your container file-system directories (translated to -v arguments of the CLI).
Therefore, you should map the directory of your uploads in the container, to the desired directory on the host.
For example, if your app's uploads are saved to this dir (inside the docker container):
/home/dokku/myapp/public_folder
and you'd like them to be kept in your host at:
/home/some/dir
then, as I understand, the content of PERSISTENT_STORAGE file should be:
/home/some/dir:/home/dokku/myapp/public_folder
I hope I got you right.
Use Dokku's storage:mount option.
You'll need to SSH into your dokku host:
ssh dokku#host
Run the following command to link the storage directory for that app to the app/public/uploads folder, for example:
storage:mount <app> /var/lib/dokku/data/storage:/app/public/uploads
The Dokku docs cover this well at: at http://dokku.viewdocs.io/dokku/advanced-usage/persistent-storage/

Resources