How to view log output using docker-compose run? - docker

When I use docker-compose up I can see logs for all containers in my docker-compose.yml file.
However, when I use docker-compose run app I only see console output for app but none of the services that app depends on. How can see log output for the other services?

At the time of writing this the docker-compose run command does not provide a switch to see the logs of other services, hence you need to use the docker-compose logs command to see the logs you want.
Update June 10th 2022
As commented by #Sandburg docker compose is now integrated into docker. As described here most of the docker compose commands can be called the same way just without the dash:
The new Compose V2, which supports the compose command as part of the Docker CLI, is now available.
Compose V2 integrates compose functions into the Docker platform, continuing to support most of the previous docker-compose features and flags. You can run Compose V2 by replacing the hyphen (-) with a space, using docker compose, instead of docker-compose.
Update July 1st 2019
docker-compose logs <name-of-service>
for all services
docker-compose logs
Use the following options from the documentation:
Usage: logs [options] [SERVICE...]
Options:
--no-color Produce monochrome output.
-f, --follow Follow log output.
-t, --timestamps Show timestamps.
--tail="all" Number of lines to show from the end of the logs
for each container.
See docker logs
You can start Docker compose in detached mode and attach yourself to the logs of all container later. If you're done watching logs you can detach yourself from the logs output without shutting down your services.
Use docker-compose up -d to start all services in detached mode (-d) (you won't see any logs in detached mode)
Use docker-compose logs -f -t to attach yourself to the logs of all running services, whereas -f means you follow the log output and the -t option gives you timestamps (See Docker reference)
Use Ctrl + z or Ctrl + c to detach yourself from the log output without shutting down your running containers
If you're interested in logs of a single container you can use the docker keyword instead:
Use docker logs -t -f <name-of-service>
Save the output
To save the output to a file you add the following to your logs command:
docker-compose logs -f -t >> myDockerCompose.log

If you want to see output logs from all the services in your terminal.
docker-compose logs -t -f --tail <no of lines>
Eg.:
Say you would like to log output of last 5 lines from all service
docker-compose logs -t -f --tail 5
If you wish to log output from specific services then it can be done as below:
docker-compose logs -t -f --tail <no of lines> <name-of-service1> <name-of-service2> ... <name-of-service N>
Usage:
Eg. say you have API and portal services then you can do
something like below :
docker-compose logs -t -f --tail 5 portal api
Where 5 represents last 5 lines from both logs.
Ref: https://docs.docker.com/v17.09/engine/admin/logging/view_container_logs/

use the command to start containers in detached mode:
docker-compose up -d
to view the containers use: docker ps
to view logs for a container: docker logs <containerid>

Unfortunately we need to run docker-compose logs separately from docker-compose run. In order to get this to work reliably we need to suppress the docker-compose run exit status then redirect the log and exit with the right status.
#!/bin/bash
set -euo pipefail
docker-compose run app | tee app.log || failed=yes
docker-compose logs --no-color > docker-compose.log
[[ -z "${failed:-}" ]] || exit 1

Related

How to reduce output in `docker-compose` up command

I'm looking for a way to reduce the output generated by docker compose up.
When running in CI all the "interactive" output for download and extract progress is completely useless and generate lots of useless text.
docker has --quiet but I don't see the same for docker compose.
There is a --quiet-pull option that lets you reduce the output generated docker compose up and docker compose run
docker compose up --quiet-pull
You can always run the docker compose in a detached mode with the -d parameter and then check logs of the service/container you want with docker logs --follow <container>
There was an option to set the log-level with --log-level [DEBUG, INFO, WARNING, ERROR, CRITICAL] but it is deprecated from version 2.0.

Docker-compose not show logs

I'm runing my api of express and mongo with docker-compose using the command docker-compose up, all fine but when i try show the logs have the next output error:
With docker-compose logs you need to use the name of the service in the docker-compose.yaml not the name of the container.
You ran docker-compose logs -f backend_api_1, which is the name of the container. If your docker-compose file does not contain any special renaming, the following should work: docker-compose logs -f backend_api (assuming the service is called backend_api)
This is a common confusion point with docker-compose orchestration. Docker compose deals with services, which then can start one or more containers for a service.
You can clarify this for yourself by looking at the manual page for whatever command you plan to use, as it will tell you whether it requires a service name or a container name.
For docker-compose logs the manual shows:
Usage: logs [options] [SERVICE...]
Since we don't have your docker-compose.yaml to refer to, we can only infer that you may have named the service backend_api. I'm just repeating the answer provided by Dennis van de Hoef, which is a reasonable guess based on how docker will name containers for you.
docker-compose logs -f backend_api
The docker logs command can be used to look at the logs of a container.
docker logs -f backend_api_1

How to get the output of a docker image

when i do
docker run consoleapp
I get the output "Hello World"
but when i want to run this program in a docker-compose with -d then nothing happens:
Starting AA... done
Starting BB ... done
Starting consoleapp ... done
Is there a way to see the output?
Try running compose without -d, it will show you the stdout of the containers on the screen, with -d it daemonizes the containers -
$ docker-compose up
In case you want to daemonize it, you can use service name defined in docker compose to get the container logs like -
$ docker-compose up -d ; docker-compose logs consoleapp
Ref - https://docs.docker.com/compose/reference/logs/

How to see the logs of a docker container

I have a simple code for which I have created a docker container and the status shows it running fine. Inside the code I have used some print() commands to print the data. I wanted to see that print command output.
For this I have seen docker logs . But it seems not to be working as it shows no logs. How to check logs.?
$ sudo docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
a3b3fd261b94 myfirstdocker "python3 ./my_script…" 22 minutes ago Up 22 minutes elegant_darwin
$ sudo docker logs a3b3fd261b94
<shows nothing>
The first point you need to print your logs to stdout.
To check docker logs just use the following command:
docker logs --help
Usage: docker logs [OPTIONS] CONTAINER
Fetch the logs of a container
Options:
--details Show extra details provided to logs
-f, --follow Follow log output
--help Print usage
--since string Show logs since timestamp
--tail string Number of lines to show from the end of the logs (default "all")
-t, --timestamps Show timestamps
Some example:
docker logs --since=1h <container_id>
4
Let's try using that docker create start and then logs command again and see what happens.
sudo docker create busybox echo hi there
output of the command
now I will take the ID and run a docker start and paste the ID that starts up the container it executes echo high there inside of it and then immediately exits.
Now I want to go back to that stopped container and get all the logs that have been emitted inside of it.
To do so I can run at docker logs and then paste the ID in and I will see that when the container had been running it had printed out the string Hi there.
One thing to be really clear about is that by running docker logs I am not re-running or restarting the container to in any way shape or form, I am just getting a record of all the logs that have been emitted from that container.
docker logs container_id
If there's not so much supposed output (e.g. script just tries to print few bytes), I'd suspect python is buffering it.
Try adding more data to the output to be sure that buffer is flushed, and also using PYTHONUNBUFFERED=1 (although, python3 still may do some buffering despite of this setting).

View logs for all docker containers simultaneously

I currently use docker for my backend, and when I first start them up with
docker-compose up
I get log outputs of all 4 dockers at once, so I can see how they are interacting with each other when a request comes in. Looking like this, one request going from nginx to couchdb
The issue is now that I am running on GCE with load balancing, when a new VM spins up, it auto starts the dockers and runs normally, I would like to be able to access a load balanced VM and view the live logs, but I can not get docker to allow me this style, when I use logs, it gives me normal all white font with no label of where it came from.
Using
docker events
does nothing, it won't return any info.
tldr; what is the best way to obtain a view, same as the log output you get when running "docker-compose up"
If using docker-compose, you use
docker-compose logs --tail=0 --follow
instead of
docker logs --tail=0 --follow
This will get the output I was originally looking for.
You can see the logs for all running containers with
docker ps -q | xargs -L 1 docker logs
In theory this might work for the --follow too if xargs is ran with -P <count>, where the count is higher than the number of running containers.
I use a variation of this to live tail (--follow) all logs and indicate which log is tailing at the time. This bash includes both stdout and stderr. Note you may need to purge the /tmp dir of *.{log,err} afterwards.
for c in $(docker ps -a --format="{{.Names}}")
do
docker logs -f $c > /tmp/$c.log 2> /tmp/$c.err &
done
tail -f /tmp/*.{log,err}
Hope this helps. Logging has become so problematic these days, and other get-off-my-lawn old man rants...
Try "watch"
Here's a quick and dirty multitail/xtail for docker containers.
watch 'docker ps --format "{{.Names}}" | sort | xargs --verbose --max-args=1 -- docker logs --tail=8 --timestamps'
How this works:
watch to run every few seconds
docker ps --format "{{.Names}}" to get the names of all running containers
sort to sort them
xargs to give these names to docker logs:
docker logs to print the actual logs
Adjust parameter "--tail=8" as needed so that everything still fits on one screen.
The "xargs" methods listed above (in another user's answer) will stop working as containers are stopped and restarted. This "watch" method here does not have that problem. (But it's not great either.)
If you are using Docker Swarm, you can find your services by
docker service ls
Grap the id, and then run
docker service logs $ID -f
if the service is defined with tty: true, then you must run with the --raw flag. Notice, this wont tell you which container is giving the outputted log entry.

Resources