I have a simple question: what will happen if I delete a Docker container log file?
As we know Docker stores container log under /var/lib/docker/containers/*/*-json.log.
If by chance I deleted that <container_name>-json.log then what will happen? Docker will create a new log file or it will stop writing logs?
Related
JVMStabilityInspector.java:196 - Exiting due to error while processing commit log during initialization.
org.apache.cassandra.db.commitlog.CommitLogReadHandler$CommitLogReadException: \
Could not read commit log descriptor in file /opt/cassandra/data/commitlog/CommitLog-7-1676434400779.log
I ran the Cassandra container in Docker, and the above error appears and stops.
It worked well before, but it doesn't seem to work well after deleting and recreating the Cassandra container.
I think we need to clear the /opt/cassandra/data/commitlog/CommitLog-7-1676434400779.log file.
However, I am not used to using dockers.
How do I erase this file?
I'm not sure if erasing the file will fix the error.
I also asked about this problem in chatgpt. However, after asking a lot of questions for an hour, they told me to try again next time, so I haven't solved it yet. So I'm going to post on Stack Overflow.
So this error likely means that the commitlog file specified is corrupted. I would definitely try deleting it.
If it's on a running docker container, you could try something like this:
Run a docker ps to get the container ID.
Remove the file using docker exec. If my container ID is f6b29860bbe5:
docker exec f6b29860bbe5 rm -rf /opt/cassandra/data/commitlog/CommitLog-7-1676434400779.log
Your question is missing a lot crucial information such as which Docker image you're running, the full Docker command you ran to start the container, and other relevant settings you've configured so I'm going to make several assumptions.
The official Cassandra Docker image (see the Quickstart Guide on the Cassandra website) that we (the Cassandra project) publish stores the commit logs in /var/lib/cassandra/commitlog/ but your deployment stores it somewhere else:
Could not read commit log descriptor in file /opt/cassandra/data/commitlog/CommitLog-7-1676434400779.log
Assuming that you're using the official image, it indicates to me that you have possibly mounted the container directories on a persistent volume on the host. If so, you will need to do a manual cleanup of all the Cassandra directories when you delete the container and recreate it.
The list of directories you need to empty include:
data/
commitlog/
saved_caches/
In your case, it might be just as easy to delete the contents of /opt/cassandra/.
If those directories are not persisted on the Docker host then you can open an interactive bash session into the Cassandra container. For example if you've named your container cassandra:
$ bash exec -it cassandra bash
For details, see the docker exec manual on the Docker Docs website. Cheers!
I have been using docker-compose to setup some docker containers.
I am aware that the logs can be viewed using docker logs <container-name>.
All logs are being printed to STDOUT and STDERR when the containers are run, there is no log 'file' being generated in the containers.
But these logs (obtained from docker logs command) are removed when their respective containers are removed by commands like docker-compose down or docker-compose rm.
When the containers are created and started again there is a fresh set of logs. No logs from the previous 'run' is present.
I am curious if there is a way to somehow prevent the logs from being removed along with their containers.
Ideally i would like to keep all my previous logs even when the container is removed.
I believe you have two ways you can go:
Make containers log into file
You can reconfigure the applications inside the container to write into logfiles rather than stdout/stderr. As you put it, you'd like to keep the logs even when the container is removed. Therefore ensure the files are stored in a (bind) mounted volume.
Reconfigure docker to store logs
Reconfigure docker to use a different logging driver. This can be especially helpful as it prevents you from changing each and every container.
Is there any way to use Docker's --rm option that auto-removes the container once it exits but allow the container's logs to persist?
I have an application that creates containers to process jobs, and then once all jobs are complete, the container exits and is deleted to conserve space. However, in case a bug caused the container's process to exit prematurely, I'd like to persist the log files so I can confirm it exited cleanly or diagnose a faulty exit.
However, the --rm option appears to remove the container's logs along with the container.
Log to somewhere outside of the container.
You could mount a directory of the host in your container, so logs will be written to the host directory and kept after rm.
Or you can mount a volume on your container; which will be persisted after rm
Or you can setup rsyslog - or some similar log collection agent - to export your logs to a remote service. See https://www.simulmedia.com/blog/2016/02/19/centralized-docker-logging-with-rsyslog/ for more on this solution.
The first 2 are hacks but easier to get up and running on your workstation/server. If this is all cloud hosted there might be a decent log offloading option (Cloudwatch on AWS) which saves you the hassle of configuring rsyslog
I am new to docker. I want to run tinyproxy within docker. Here is the image I used to create a docker container: "https://hub.docker.com/r/dtgilles/tinyproxy/".
For some unknown reason, when I mount the log file to the host machine, I can see the .conf file, but I can't see log file and the proxy server seems doesn't work.
Here is the command I tried:
docker run -v $(pwd):/logs -p 8888:8888 -d --name tiny
dtgilles/tinyproxy
If I didn't mount the file, then every time when run a container, I need to change its config file inside container.
Does anyone has any ideas about saving the changes in container?
Question
How to save a change committed by/into a container?
Answer
The command docker commit creates a new image from a container's changes (from the man page).
Best Practice
You actually should not do this to save a configuration file. A Docker image is supposed to be immutable. This increases sharing, and image customization through mounted volume.
What you should do is create the configuration file on the host and share it at through parameters with docker run. This is done by using the option -v|--volume. Check the man page, you'll then be able to share files (or directories) between host and containers allowing to persists the data through different runs.
have an application running inside docker container. the application writes log messages into local log files. how can i make the log file persistent in case the docker container stops or crashes?
Since the container are run time entity ,when i stop the image my logs/data is gone.
Thanks,
Sohan
You can do this using docker volumes:
https://docs.docker.com/userguide/dockervolumes/
For example:
docker run -v /var/log/docker:/var/log your-image
will mount the log directory on your local file system. You can also get much fancier, creating containers just for data. It's all explained in the above link.