I have an app that is dynamically creating docker containers and I can't intercept the way it is created.
I want to see logs from all the machines that are up. no matter if it was via docker-compose or just docker command line. I need to see all the logs.
Is it possible?
right no I need to run docker ps, see all the created machines and run docker log container.
I can't really monitor what is going inside.
Thanks
An approach is to use a dedicated logging container that can gather log events from other containers, aggregate them, then store or forward the events to a third-party service, this approach eliminates the dependencies on a host.
Further, dedicated logging containers can automatically collect, monitor, and analyze log events, It can scale your log events automatically without configuration. It can retrieve logs through multiple streams of log events, stats, and Docker API data.
You can check this link also for some help.
Docker Logging Best Practices
Related
I am going to use Azures Computer Vision (Read 3.2) and run it on-premise as a container, and therefore using Docker Desktop.
However, I have not been able to figure out if Docker collects any data that is being processed by containers running on Docker.
In https://www.docker.com/legal/security-and-privacy-guidelines under the header 'Data Privacy and Security' Docker writes:
"In general, Docker does not collect or store personal data and the use of Docker products does not result in personal data being collected or stored."
Now, to me, this sounds ambiguous. We are using Azure's on-premise container in order to stay compliant and that part works since Azure does not collect any data from the container. However, if Docker itself collects data then that is a show stopper. FYI, I am a beginner to Docker and I might be completely off.
EDIT: So my question is, does Docker collect any of the input or the output going in and out from the container?
Thankful for any answers or wisdom you might be able to share.
Regards
As you saw, the Docker privacy policy "applies to Docker websites, products and services offered by Docker". I do not think running a Docker image as a container would be considered under those terms, and so I do not think Docker collect any information produced by the container as 'output' - i.e. standard output/error streams or the like.
Docker Desktop may collect statistics, metrics and information on the images/containers run directly under Docker desktop where they have access to that information, but also many docker-built images will be run under non-docker environments (such as Kubernetes) where they could not have access to the such information.
As an aside, I think all the image themselves be built from scratch and you (or other interested parties) have access to the layers within the image so you can see what has been added and what the effect of the layer is. Thus you could also verify that Docker (or other parties) are not harvesting data from a running container.
I want to make a faster deployment process than before. Always too much time spent in this step.
But I can't find any way to see detailed docker logs such as Downloading, Pulling Images, Starting Containers, ... etc. I want to see it in the machine; I want to debug it. How to check this?
These will be in various places.
docker events will show you each action the scheduler takes, and any actions on the node you've run that command on. You'll need to run this on all potential nodes while creating/updating a service to get a full accounting of manager and worker events.
On the node that's been assigned a task to create a container, the docker debug flag may give you more insight.
If I want to know where and when a Docker image in my container registry has been run (e.g., for audit purposes, to see what images are being used the most, or to see if an image is truly stale before deleting it), what are the best tools for getting that information?
(For example, for a VM analogy on AWS: I could check the log of API calls via AWS CloudTrail for when EC2 instances have started and stopped, get the instance IDs, and then join that against the VM image that was running on those images.)
Docker images are downloaded from registry onto hosts, so you would not know if someone starts an image pulled from the registry: it is already downloaded.
There is in fact no way for you to know that an image has started on a host, except if you implement a proper reporting on bootstrap/entrypoint.
Cluster orchestrators can of course provide you adequate reporting on when are started pods/containers, but you should refer to the respective documentation for this.
You could attach to each docker daemon to listen to its Events:
https://docs.docker.com/engine/reference/commandline/events/
Also you can filter them by any criteria.
Docker images report the following events:
delete
import
load
pull
push
save
tag
untag
I am running docker with kubernetes.
I need to find out when the last time docker container is used by a user.
I am creating one container per user.I have to kill that container if the user has not interacted with the container for a specific amount of time.
Currently, I am running a daemon inside docker container which checks last modified files and sends the info.
Is there any docker/kubernetes API for the same?
I thinks there's no API for that as "usage" is something which is hard to measure. One way would be to check whether systems stopped logging at some point back in time.
The other option would be to use the metrics which are exposed by Kubernetes and bring up monitoring and alerting systems like Prometheus to tell you once a Deployment/Pod is not used anymore. "Usage" could then be determined through the exposed network metrics e.g. like this:
max_over_time(
container_network_receive_bytes_total{kubernetes_pod_name=~"^yourdeployment.*$"}[1h]
)
If that's below a certain threshold you could trigger and alert and perform further actions.
Say I have a container that has everything I need to run my web application (such as https://github.com/grigio/docker-stringer for example). How would I go about inspecting the logs for the different services (web server, application server, database server)? With all of the tutorials so far I have only been able to view the logs for the specific command run when starting the container.
One method would be to configure your logs to write to stdout and to use docker logs to retrieve them.
Another option would be to use a bindmount and link to your host file system.