I have a strange behavior when searching with grep in docker logs which I don't understand.
As an example, I start a jupyter notebook container with following command.
docker run -it -d --rm --name test -p 8888:8888 jupyter/minimal-notebook
With that running Container I can now display the containers log and grep for a part of interest (eg. URL with token of the running jupyter server). This is done with the following Command and shows me the expected result.
docker logs test --tail 5 | grep -ozE "http://127.*"
But when the Container is started without the -it option this will not work and it just prints the whole log message.
Can someone explain this behavior?
The pipe between commands only pipes stdout. If the application inside the container outputs to stderr, this will be displayed bypassing the pipe and grep. You can adjust this by sending stderr to stdout before the pipe with a 2>&1:
docker logs test --tail 5 2>&1 | grep -ozE "http://127.*"
Related
I'm executing the following command to mount Hadoop-spark-pig-hive docker container using port mapping,
docker run -p 8088:8088 -p 50070:50070 --name hadoop-spark-pig-hive -v C:\Users\Mr.Semicolon\Desktop\iit:/resource -d suhothayan/hadoop-spark-pig-hive:2.9.2
Just to confirm it is up and running I executed the command docker ps
And it works for a few seconds only and docker container exit without any message. Please can someone suggest why this is happening and how can I solve the issue?
Note: I'm using Windows 10 Home environment and docker version 10.03.13 build 4484c46d9d
As Julien B requested I executed docker logs and got the following log message but still have no idea how to solve this,
/etc/bootstrap.sh: line 9: /usr/local/spark/conf/spark-env.sh: Permission denied
/
* Starting OpenBSD Secure Shell server sshd
...done.
Waiting for hdfs to exit from safemode
Safe mode is OFF
Started
I followed a tutorial and Following command worked for me,
docker run -it -p 8088:8088 -p 50070:50070 --name hadoop-spark-pig-hive -v C:\Users\Mr.Semicolon\Desktop\iit:/resource -d suhothayan/hadoop-spark-pig-hive:2.9.2
According to this notebook,
The -it flag tells docker that it should open an interactive container instance.
How to have live log from a docker container?
After many sleepless nights and many many internet searches an answer finely comes to me in a dream I would say - Hey! just detach the dumb container!
The easiest way if you have the container running and sending logs to stdout, is to use docker attach , here is an example of running a container and attaching to it to see the output
$ docker run -d --name topdemo ubuntu /usr/bin/top -b
$ docker attach topdemo
basically this is the syntax of the command docker attach [OPTIONS] <CONTAINER>
After searching quite a lot(!),
I am using it like this:
NUM=`docker run -d --user 1013830000 -p 4567:4567 container_test:v1`
docker logs -f $NUM
docker stop $NUM
I'm running the command docker logs <container-id> | tail -10 and still, docker shows the entire log history. I know docker logs --tail 10 <container-id> is a valid command and serves the purpose. But, why doesn't the former command works as it does in case of a file?
If you want your everything your program writes to either stdout or stderr to go through the pipeline to tail, redirect stderr to stdout:
docker logs "$container_id" 2>&1 | tail -10
in case someone may want to tail -f docker logs
here you may try this:
docker logs -f --tail 0 "$container_id"
Is there any way I can see the log of a container that has exited?
I can get the container id of the exited container using docker ps -a but I want to know what happened when it was running.
Use docker logs. It also works for stopped containers and captures the entire STDOUT and STDERR streams of the container's main process:
$ docker run -d --name test debian echo "Hello World"
02a279c37d5533ecde76976d7f9d1ca986b5e3ec03fac31a38e3dbed5ea65def
$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
49daa9d41a24 debian "echo test" 2 minutes ago Exited (0) 2 minutes ago test
$ docker logs -t test
2016-04-16T15:47:58.988748693Z Hello World
docker logs --tail=50 <container id> for the last fifty lines - useful when your container has been running for a long time.
You can use below command to copy logs even from an exited container :
docker cp container_name:path_of_file_in_container destination_path_locally
Eg:
docker cp sample_container:/tmp/report /root/mylog
To directly view the logfile of an exited container in less, scrolled to the end of the file, I use:
docker inspect $1 | grep 'LogPath' | sed -n "s/^.*\(\/var.*\)\",$/\1/p" | xargs sudo less +G
run as ./viewLogs.sh CONTAINERNAME
This method has the benefit over docker logs based approaches, that the file is directly opened, instead of streamed.
sudo is necessary, as the LogPath/File usually is under root-owned
#icyerasor comment above actually helped me solve the issue. In my particular situation the container that has stopped running had no container name only container id.
Steps that found the logs also listed in this post
Find the stopped container via docker ps -a
grab the container id of the failed container
Substitute it in this command cat /var/lib/docker/containers/<container id>/<container id>-json.log
The question may be a bit of newbie.
I run docker exec -it mycontainer bash to enter into a daemon container(postgresSQL ),
and echo something.
now I exit it , and use docker logs mycontainer so as to see my echos.
According to
The docker logs command batch-retrieves logs present at the time of execution.
The docker logs --follow command will continue streaming the new output from the container's STDOUT and STDERR.
The docker logs listen STDOUT of the container, why I don't see my string just echoed inside it?
Docker engine only stores stdout from process with ID 0 (i.e. the process launched by the CMD directive of your Dockerfile).
By the way, on your Docker host, you can inspect the content of container's logs by viewing the file /var/lib/docker/containers/<ID of your container>/<ID of your container>-json.log.
This file stores logs in JSON format.
I assume logging only occurs for the main process in a container. As exec creates a new process, it won't get logged.
Note that docker logs works for processes given in the run command e.g:
$ ID=$(docker run -d debian sh -c "while true; do echo "hello"; sleep 1; done;")
$ docker logs $ID
hello
hello
hello
hello
hello