I'm looking for a way to reduce the output generated by docker compose up.
When running in CI all the "interactive" output for download and extract progress is completely useless and generate lots of useless text.
docker has --quiet but I don't see the same for docker compose.
There is a --quiet-pull option that lets you reduce the output generated docker compose up and docker compose run
docker compose up --quiet-pull
You can always run the docker compose in a detached mode with the -d parameter and then check logs of the service/container you want with docker logs --follow <container>
There was an option to set the log-level with --log-level [DEBUG, INFO, WARNING, ERROR, CRITICAL] but it is deprecated from version 2.0.
I have a simple code for which I have created a docker container and the status shows it running fine. Inside the code I have used some print() commands to print the data. I wanted to see that print command output.
For this I have seen docker logs . But it seems not to be working as it shows no logs. How to check logs.?
$ sudo docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
a3b3fd261b94 myfirstdocker "python3 ./my_script…" 22 minutes ago Up 22 minutes elegant_darwin
$ sudo docker logs a3b3fd261b94
<shows nothing>
The first point you need to print your logs to stdout.
To check docker logs just use the following command:
docker logs --help
Usage: docker logs [OPTIONS] CONTAINER
Fetch the logs of a container
Options:
--details Show extra details provided to logs
-f, --follow Follow log output
--help Print usage
--since string Show logs since timestamp
--tail string Number of lines to show from the end of the logs (default "all")
-t, --timestamps Show timestamps
Some example:
docker logs --since=1h <container_id>
4
Let's try using that docker create start and then logs command again and see what happens.
sudo docker create busybox echo hi there
output of the command
now I will take the ID and run a docker start and paste the ID that starts up the container it executes echo high there inside of it and then immediately exits.
Now I want to go back to that stopped container and get all the logs that have been emitted inside of it.
To do so I can run at docker logs and then paste the ID in and I will see that when the container had been running it had printed out the string Hi there.
One thing to be really clear about is that by running docker logs I am not re-running or restarting the container to in any way shape or form, I am just getting a record of all the logs that have been emitted from that container.
docker logs container_id
If there's not so much supposed output (e.g. script just tries to print few bytes), I'd suspect python is buffering it.
Try adding more data to the output to be sure that buffer is flushed, and also using PYTHONUNBUFFERED=1 (although, python3 still may do some buffering despite of this setting).
So today I noticed a interesting "crond" process taking up 100% of the cpu.
The strange thing is, I don't have cron installed.
find / -name "crond"
/var/lib/docker/devicemapper/mnt/d359c68dd07e2defb573e3d6f5c20f9984a3796d1fbdd92dd2d48923bf49ea8f/rootfs/usr/sbin/crond
Not really sure what else I can do besides kill the process. Is there any way I could diagnose the cause of this issue?
The crond is running within the container.
You can attach to the container, i.e. with docker exec -it <container> bash then navigate to /var/logs and inspect logs for further analysis.
In case you have multiple running containers you need to find which one is generating the issue: i.e. you can enumerate them with docker ps then run
docker inspect <container> | grep d359c68dd07e2defb573e3d6f5c20f9984a3796d1fbdd92dd2d48923bf49ea8f on each.
When you have a hit you have found the container to analyze further.
I use docker logs [container-name] to see the logs of a specific container.
Is there an elegant way to clear these logs?
First the bad answer. From this question there's a one-liner that you can run:
echo "" > $(docker inspect --format='{{.LogPath}}' <container_name_or_id>)
instead of echo, there's the simpler:
: > $(docker inspect --format='{{.LogPath}}' <container_name_or_id>)
or there's the truncate command:
truncate -s 0 $(docker inspect --format='{{.LogPath}}' <container_name_or_id>)
I'm not a big fan of either of those since they modify Docker's files directly. The external log deletion could happen while docker is writing json formatted data to the file, resulting in a partial line, and breaking the ability to read any logs from the docker logs cli. For an example of that happening, see this comment on duketwo's answer:
after emptying the logfile, I get this error: error from daemon in stream: Error grabbing logs: invalid character '\x00' looking for beginning of value
Instead, you can have Docker automatically rotate the logs for you. This is done with additional flags to dockerd if you are using the default JSON logging driver:
dockerd ... --log-opt max-size=10m --log-opt max-file=3
You can also set this as part of your daemon.json file instead of modifying your startup scripts:
{
"log-driver": "json-file",
"log-opts": {"max-size": "10m", "max-file": "3"}
}
These options need to be configured with root access. Make sure to run a systemctl reload docker after changing this file to have the settings applied. This setting will then be the default for any newly created containers. Note, existing containers need to be deleted and recreated to receive the new log limits.
Similar log options can be passed to individual containers to override these defaults, allowing you to save more or fewer logs on individual containers. From docker run this looks like:
docker run --log-driver json-file --log-opt max-size=10m --log-opt max-file=3 ...
or in a compose file:
version: '3.7'
services:
app:
image: ...
logging:
options:
max-size: "10m"
max-file: "3"
For additional space savings, you can switch from the json log driver to the "local" log driver. It takes the same max-size and max-file options, but instead of storing in json it uses a binary syntax that is faster and smaller. This allows you to store more logs in the same sized file. The daemon.json entry for that looks like:
{
"log-driver": "local",
"log-opts": {"max-size": "10m", "max-file": "3"}
}
The downside of the local driver is external log parsers/forwarders that depended on direct access to the json logs will no longer work. So if you use a tool like filebeat to send to Elastic, or Splunk's universal forwarder, I'd avoid the "local" driver.
I've got a bit more on this in my Tips and Tricks presentation.
Use:
truncate -s 0 /var/lib/docker/containers/**/*-json.log
You may need sudo
sudo sh -c "truncate -s 0 /var/lib/docker/containers/**/*-json.log"
ref. Jeff S. Docker: How to clear the logs properly for a Docker container?
Reference: Truncating a file while it's being used (Linux)
On Docker for Windows and Mac, and probably others too, it is possible to use the tail option. For example:
docker logs -f --tail 100
This way, only the last 100 lines are shown, and you don't have first to scroll through 1M lines...
(And thus, deleting the log is probably unnecessary)
sudo sh -c "truncate -s 0 /var/lib/docker/containers/*/*-json.log"
You can set up logrotate to clear the logs periodically.
Example file in /etc/logrotate.d/docker-logs
/var/lib/docker/containers/*/*.log {
rotate 7
daily
compress
size=50M
missingok
delaycompress
copytruncate
}
You can also supply the log-opts parameters on the docker run command line, like this:
docker run --log-opt max-size=10m --log-opt max-file=5 my-app:latest
or in a docker-compose.yml like this
my-app:
image: my-app:latest
logging:
driver: "json-file"
options:
max-size: "10m"
max-file: "5"
Credits: https://medium.com/#Quigley_Ja/rotating-docker-logs-keeping-your-overlay-folder-small-40cfa2155412 (James Quigley)
Docker4Mac, a 2018 solution:
LOGPATH=$(docker inspect --format='{{.LogPath}}' <container_name_or_id>)
docker run -it --rm --privileged --pid=host alpine:latest nsenter -t 1 -m -u -n -i -- truncate -s0 $LOGPATH
The first line gets the log file path, similar to the accepted answer.
The second line uses nsenter that allows you to run commands in the xhyve VM that servers as the host for all the docker containers under Docker4Mac. The command we run is the familiar truncate -s0 $LOGPATH from non-Mac answers.
If you're using docker-compose, the first line becomes:
local LOGPATH=$(docker inspect --format='{{.LogPath}}' $(docker-compose ps -q <service>))
and <service> is the service name from your docker-compose.yml file.
Thanks to https://github.com/justincormack/nsenter1 for the nsenter trick.
You can't do this directly through a Docker command.
You can either limit the log's size, or use a script to delete logs related to a container. You can find scripts examples here (read from the bottom): Feature: Ability to clear log history #1083
Check out the logging section of the docker-compose file reference, where you can specify options (such as log rotation and log size limit) for some logging drivers.
Here is a cross platform solution to clearing docker container logs:
docker run --rm -v /var/lib/docker:/var/lib/docker alpine sh -c "echo '' > $(docker inspect --format='{{.LogPath}}' CONTAINER_NAME)"
Paste this into your terminal and change CONTAINER_NAME to desired container name or id.
As a root user, try to run the following:
> /var/lib/docker/containers/*/*-json.log
or
cat /dev/null > /var/lib/docker/containers/*/*-json.log
or
echo "" > /var/lib/docker/containers/*/*-json.log
On my Ubuntu servers even as sudo I would get Cannot open ‘/var/lib/docker/containers/*/*-json.log’ for writing: No such file or directory
But combing the docker inspect and truncate answers worked :
sudo truncate -s 0 `docker inspect --format='{{.LogPath}}' <container>`
I do prefer this one (from solutions above):
truncate -s 0 /var/lib/docker/containers/*/*-json.log
However I'm running several systems (Ubuntu 18.x Bionic for example), where this path does not work as expected. Docker is installed through Snap, so the path to containers is more like:
truncate -s 0 /var/snap/docker/common/var-lib-docker/containers/*/*-json.log
This will delete all logfiles for all containers:
sudo find /var/lib/docker/containers/ -type f -name "*.log" -delete
Thanks to answer by #BMitch, I've just wrote a shell script to clean logs of all the containers:
#!/bin/bash
ids=$(docker ps -a --format='{{.ID}}')
for id in $ids
do
echo $(docker ps -a --format='{{.ID}} ### {{.Names}} ### {{.Image}}' | fgrep $id)
truncate -s 0 $(docker inspect --format='{{.LogPath}}' $id)
ls -llh $(docker inspect --format='{{.LogPath}}' $id)
done
Not sure if this is helpful for you, but removing the container always helps.
So, if you use docker-compose for your setup, you can simply use docker-compose down && docker-compose up -d instead of docker-compose restart. With a proper setup (make sure to use volume mounts for persistent data), you don't lose any data this way.
Sure, this is more than the OP requested. But there are various situations where the other answers cannot help (if using a remote docker server or working on a Windows machine, accessing the underlying filesystem is proprietary and difficult)
Linux/Ubuntu:
If you have several containers and you want to remove just one log but not others.
(If you have issues like "Permission denied" do first sudo su.)
List all containers: docker ps -a
Look for the container you desire and copy the CONTAINER ID. Example: E1X2A3M4P5L6.
Containers folders and real names are longer than E1X2A3M4P5L6 but first 12 characters are those resulted in docker ps -a.
Remove just that log:
> /var/lib/docker/containers/E1X2A3M4P5L6*/E1X2A3M4P5L6*-json.log (Replace E1X2A3M4P5L6 for your result !! )
As you can see, inside /containers are the containers, and logs has the same name but with -json.log at the end. You just need to know that first 12 characters, because * means "anything".
Docker for Mac users, here is the solution:
Find log file path by:
$ docker inspect | grep log
SSH into the docker machine( suppose the name is default, if not, run docker-machine ls to find out):
$ docker-machine ssh default
Change to root user(reference):
$ sudo -i
Delete the log file content:
$ echo "" > log_file_path_from_step1
I needed something I could run as one command, instead of having to write docker ps and copying over each Container ID and running the command multiple times. I've adapted BMitch's answer and thought I'd share in case someone else may find this useful.
Mixing xargs seems to pull off what I need here:
docker ps --format='{{.ID}}' | \
xargs -I {} sh -c 'echo > $(docker inspect --format="{{.LogPath}}" {})'
This grabs each Container ID listed by docker ps (will erase your logs for any container on that list!), pipes it into xargs and then echoes a blank string to replace the log path of the container.
To remove/clear docker container logs we can use below command
$(docker inspect container_id|grep "LogPath"|cut -d """ -f4)
or
$(docker inspect container_name|grep "LogPath"|cut -d """ -f4)
If you need to store a backup of the log files before deleting them, I have created a script that performs the following actions (you have to run it with sudo) for a specified container:
Creates a folder to store compressed log files as backup.
Looks for the running container's id (specified by the container's name).
Copy the container's log file to a new location (folder in step 1) using a random name.
Compress the previous log file (to save space).
Truncates the container's log file by certain size that you can define.
Notes:
It uses the shuf command. Make sure your linux distribution has it or change it to another bash-supported random generator.
Before use, change the variable CONTAINER_NAME to match your running container; it can be a partial name (doesn't have to be the exact matching name).
By default it truncates the log file to 10M (10 megabytes), but you can change this size by modifying the variable SIZE_TO_TRUNCATE.
It creates a folder in the path: /opt/your-container-name/logs, if you want to store the compressed logs somewhere else, just change the variable LOG_FOLDER.
Run some tests before running it in production.
#!/bin/bash
set -ex
############################# Main Variables Definition:
CONTAINER_NAME="your-container-name"
SIZE_TO_TRUNCATE="10M"
############################# Other Variables Definition:
CURRENT_DATE=$(date "+%d-%b-%Y-%H-%M-%S")
RANDOM_VALUE=$(shuf -i 1-1000000 -n 1)
LOG_FOLDER="/opt/${CONTAINER_NAME}/logs"
CN=$(docker ps --no-trunc -f name=${CONTAINER_NAME} | awk '{print $1}' | tail -n +2)
LOG_DOCKER_FILE="$(docker inspect --format='{{.LogPath}}' ${CN})"
LOG_FILE_NAME="${CURRENT_DATE}-${RANDOM_VALUE}"
############################# Procedure:
mkdir -p "${LOG_FOLDER}"
cp ${LOG_DOCKER_FILE} "${LOG_FOLDER}/${LOG_FILE_NAME}.log"
cd ${LOG_FOLDER}
tar -cvzf "${LOG_FILE_NAME}.tar.gz" "${LOG_FILE_NAME}.log"
rm -rf "${LOG_FILE_NAME}.log"
truncate -s ${SIZE_TO_TRUNCATE} ${LOG_DOCKER_FILE}
You can create a cronjob to run the previous script every month. First run:
sudo crontab -e
Type a in your keyboard to enter edit mode. Then add the following line:
0 0 1 * * /your-script-path/script.sh
Hit the escape key to exit Edit mode. Save the file by typing :wq and hitting enter. Make sure the script.sh file has execution permissions.
On computers with docker desktop we use:
truncate -s 0 //wsl.localhost/docker-desktop-data/data/docker/containers/*/*-json.log
For linux distributions you can use this it works for me with this path:
truncate -s 0 /var/lib/docker/containers/*/*-json.log
docker system prune
run this command in command prompt
I currently use docker for my backend, and when I first start them up with
docker-compose up
I get log outputs of all 4 dockers at once, so I can see how they are interacting with each other when a request comes in. Looking like this, one request going from nginx to couchdb
The issue is now that I am running on GCE with load balancing, when a new VM spins up, it auto starts the dockers and runs normally, I would like to be able to access a load balanced VM and view the live logs, but I can not get docker to allow me this style, when I use logs, it gives me normal all white font with no label of where it came from.
Using
docker events
does nothing, it won't return any info.
tldr; what is the best way to obtain a view, same as the log output you get when running "docker-compose up"
If using docker-compose, you use
docker-compose logs --tail=0 --follow
instead of
docker logs --tail=0 --follow
This will get the output I was originally looking for.
You can see the logs for all running containers with
docker ps -q | xargs -L 1 docker logs
In theory this might work for the --follow too if xargs is ran with -P <count>, where the count is higher than the number of running containers.
I use a variation of this to live tail (--follow) all logs and indicate which log is tailing at the time. This bash includes both stdout and stderr. Note you may need to purge the /tmp dir of *.{log,err} afterwards.
for c in $(docker ps -a --format="{{.Names}}")
do
docker logs -f $c > /tmp/$c.log 2> /tmp/$c.err &
done
tail -f /tmp/*.{log,err}
Hope this helps. Logging has become so problematic these days, and other get-off-my-lawn old man rants...
Try "watch"
Here's a quick and dirty multitail/xtail for docker containers.
watch 'docker ps --format "{{.Names}}" | sort | xargs --verbose --max-args=1 -- docker logs --tail=8 --timestamps'
How this works:
watch to run every few seconds
docker ps --format "{{.Names}}" to get the names of all running containers
sort to sort them
xargs to give these names to docker logs:
docker logs to print the actual logs
Adjust parameter "--tail=8" as needed so that everything still fits on one screen.
The "xargs" methods listed above (in another user's answer) will stop working as containers are stopped and restarted. This "watch" method here does not have that problem. (But it's not great either.)
If you are using Docker Swarm, you can find your services by
docker service ls
Grap the id, and then run
docker service logs $ID -f
if the service is defined with tty: true, then you must run with the --raw flag. Notice, this wont tell you which container is giving the outputted log entry.