My log4j 1x version is sending logs from a docker container to syslog server. In my rsyslog.conf I am storing these logs under /var/log/rsyslog/appname/%hostname% .
Ideally if I am sending the logs from my log4j1x docker container, rsyslog should identify the container id as %hostname% but it identifies the underlying server as hostname. This is not the case when my nginx docker container sends logs to rsyslog. It correct identifies the container id as hostname.
Can someone assist please?
Related
I am working on Log monitoring, I have requirement of getting tomcat application server logs(example: catalina.log) running in the docker container(container is running in atomic host) and passing it to Logstash server using rsyslog.
I am able to get the docker container related logs, but not able to get the tomcat server and application logs from the docker container where tomcat is running.
Any suggestion on this is appreciated.
Thanks,
Praveen
You could mount the local host when initializing the container. You just pass -v flag for mounting to the local host.
docker run -v /tmp may/hello-world.py
This will allow you to mount the directory of the tomcat logs to the local host.
Here is the Docker Docs that talks about it.
https://docs.docker.com/engine/userguide/containers/dockervolumes/
-Bruce
I want to run a task in some docker containers on different hosts. And I have written a manager app to manage the containers(start task, stop task, get status, etc...) . Once a container is started, it will send an http request to the manager with its address and port, so the manager will know how to manage the container.
Since there may be more than one containers running on a same host, they would be mapped to different ports. To register a container on my manager, I have to know which port each container is mapped to.
How can I get the mapped port inside a docker container?
There's an solution here How do I know mapped port of host from docker container? . But it's not applicable if I run container with -P. Since this question is asked more than 1 year ago, I'm wondering maybe there's a new feature added to docker to solve this problem.
You can also you docker port container_id
The doc
https://docs.docker.com/engine/reference/commandline/port/
examples from the doc
$ docker port test
7890/tcp -> 0.0.0.0:4321
9876/tcp -> 0.0.0.0:1234
$ docker port test 7890/tcp
0.0.0.0:4321
$ docker port test 7890/udp
2014/06/24 11:53:36 Error: No public port '7890/udp' published for test
$ docker port test 7890
0.0.0.0:4321
i share /var/run/docker.sock to container and get self info
docker run --rm -it -v /var/run/docker.sock:/var/run/docker.sock alpine:latest sh
in container shell
env //get HOSTNAME
curl --unix-socket /var/run/docker.sock http://localhost/containers/3c6b9e44a622/json
the 3c6b9e44a622 is your HOSTNAME
Once a container is started, it will send an http request to the manager with its address and port
This isn't going to be working. From inside a container you cannot figure out to which docker host port a container port is mapped to.
What I can think about which would work and be the closest to what you describe is making the container open a websocket connection to the manager. Such a connection would allow two ways communication between your manager and container while still being over HTTP.
What you are trying to achieve is called service discovery. There are already tools for service discovery that work with Docker. You should pick one of them instead of trying to make your own.
See for instance:
etcd
consul
zookeeper
If you really want to implement your service discovery system, one way to go is to have your manager use the docker event command (or one of the docker client librairies). This would enable your manager to get notified of containers creations/deletions with nothing to do on the container side.
Then query the docker host to figure out the ports that are mapped to your containers with docker port.
Can logs in a docker container ... say logs located in /var/log/syslog get shipped to logstash without using any additional components such as lumberjack and logspout?
Just wondering because I set up an environment and tried to make it work with syslog (so syslog ships the logs from docker container to logstash) but for now it's not working .. just wondering if there's something wrong with my logic.
There's no way for messages in /var/log/syslog to magically route to logstash without something configured to forward messages. Something must send the logs to logstash. You have a few options:
Configure your app to send log messages to stdout rather than to /var/log/syslog, and run logspout to collect stdout from all the running containers and send messages to your logstash endpoint.
Run rsyslog inside your container and configure a syslog daemon such as rsyslog to send messages to your logstash endpoint
Bind mount /dev/log from the host to your container by passing -v /dev/log:/dev/log to docker run when starting your container. On the host, configure your syslog daemon to send messages to logstash.
You could use the docker syslog driver to send docker logs straight from docker containers to logstash. Just have to add some parameters when you run your container
https://docs.docker.com/engine/admin/logging/overview/#supported-logging-drivers
I have created docker container for Couchbase. The service is up and running but with the following command,
couchbase-cli cluster-init -c $CB_SERVER_IPADDR:8091 --cluster-init-username=admin --cluster-init-password=couchbase --cluster-init-ramsize=2048
I get a server node in couchbase with the name as its internal IP address - 172.x.x.x. Then when i create bucket using the following command,
couchbase-cli bucket-create -c $CB_SERVER_IPADDR:8091 -u admin -p couchbase --bucket=heartbeat --bucket-type=couchbase --bucket-ramsize=500 --bucket-replica=0 --wait
buckets gets created. But when my web application which is in a different container, communicates with couchbase with the IP address of the docker host where the Couchbase server is running, i get the following error message
2014-08-05 21:49:00.106 INFO com.couchbase.client.CouchbaseConnection: Reconnecting due to exception on {QA sa=172.x.x.x/172.x.x.x:11210, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0}
java.net.NoRouteToHostException: No route to host
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:485)
at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:322)
at com.couchbase.client.CouchbaseConnection.run(CouchbaseConnection.java:288)
I could see that my application is using the server node name which is the internal IP address to communicate with the couchbase and thus the error. Any help appreciated
Your web application docker container is not able to resolve the host name of the couchbase docker container.
You would need to link the web application container and couchbase container together.
Below will create a name "couchbase" for your couchbase instance, this name will later be referenced by web application docker container
sudo docker run -d --name couchbase <your couchbase image name>
Below will add an entry "$your_couchbase_docker_container_IP couchbase" into /etc/hosts on your web application docker container
sudo docker run -d -P --link couchbase:couchbase <your web application image name>
Then in your web application, you can use couchbase:8091 to access the couchbase docker container
More detail on "docker link" is available at https://docs.docker.com/userguide/dockerlinks/
I have a ubuntu machine which is a VM where I have installed docker in it. I am using this machine from my local windows machine and doing ssh , opening the terminal to the ubuntu machine.
Now , I am going to take a docker image which contains all the necessary softwares for eg: apache installed in it. Later I am going to deploy a sample appication(which is a web applicationP on to it and save it .
Now , I am in a confused mode as in how to check the deployed application if its running properly. i.e., what would be the address of the container which containds the deployed application.
for eg:- If I type http://127.x.x.x which is the address of the ubuntu machine , I am just getting time out .
Can anyone tell me how to verify the deployed application . Also, the printing the output of the program on the console works seemlessly fine , as the output gets printed , only thing I have some doubts is regarding the web application.
There are some possibilities to check whether your app is running.
Remote API
As JimiDini said, one possibility is the Docker remote API. You can use it to see all running containers (which would be your use case, right?), inspect a certain container or start and stop containers. The API is a REST-API with several binding for programming languages (at https://docs.docker.io/reference/api/remote_api_client_libraries/). Some of them are very outdated. To use the Docker remote API from another machine, I needed to open it explicitly:
docker -H tcp://0.0.0.0:4243 -H unix:///var/run/docker.sock -d &
Note that the API is open to the world now! In a real scenario you would need to secure it in some way (e.g. see the example at http://java.dzone.com/articles/securing-docker%E2%80%99s-remote-api).
Docker PS
To see all running containers run docker ps on your host. This will list all running containers. If you do not see your app, it is not running. It also shows you the ports your app is exposing. You can also do this via the remote API.
Logs
You can also check the logs. You can run docker attach <container id> to attach to a certain container an see its stdout. You can run also run docker logs <container id> to receive the Docker logs. What I prefer is to write the logs to a certain directory, e.g. all logs to /var/log and mount this folder to my host machine. Then all your logs will end up in /home/ubuntu/docker-logs on your host.
docker run -p 80:8080 -v /home/ubuntu/docker-logs:/var/log:rw my/application
One word to ports and IP
Every container will get its own IP address. You can check this IP address via the remote API or via Docker on the host machine directly. You can also specify a certain host name for the container (by passing the --hostname="test42" to the run command). However, you mostly did not need that.
To access the application in the container, you need to open the port in the container and bind to a port on the host.
In your Dockerfile you need to EXPOSE the port your app runs on:
FROM ubuntu
...
EXPOSE 8080
CMD run-my-app.sh
When you start your container, you need to bind this port to a port of the host:
docker run -p 80:8080 my/application
Now you can access your app on http://localhost:80 or http://127.0.0.1:80.
If you app does not response, check if the container is running by typing docker ps or the remote API. If it is not running, check the logs for the reason.
(Note: If you run your Ubuntu VM in something like VirtualBox and you try to access it from your Windows machine, make sure you opened the ports in VirtualBox too!).
Docker container has a separate IP address. By default it is private (accessible only from the host-machine).
Docker provides all metadata (including IP address) via its API:
https://docs.docker.io/reference/api/docker_remote_api_v1.10/#inspect-a-container
https://docs.docker.io/reference/api/docker_remote_api_v1.10/#monitor-docker-s-events
You can also take a look at a little tool called docker-gen for inspiration. It monitors docker-events and created configuration-files on host machine using templates.
To obtain the ip address of a docker container, if you know its id (a long hex string) or if you named it:
docker inspect --format '{{ .NetworkSettings.IPAddress }}' <container-id-or-name>
Docker is running its own network and to get information about it you can run the following commands:
docker network ls
docker network inspect <network name>
docker inspect <container id>
In the output, you should be able to find the IP.
But there is also a couple of things you need to be aware of, regarding Dockerfile and docker run command:
when you EXPOSE a port in Dockerfile, the service in the container is not accessible from outside Docker, but from inside other Docker containers
and when you EXPOSE and use docker run -p ... flag, the service in the container is accessible from anywhere, even outside Docker
So for example, if your apache is running on port 8080 you should expose it in Dockerfile and then you can run it as:
docker run -d -p 8080:8080 <image name> and you should be able to access it from your host on HTTP://localhost:8080.
It is an old question/answer but it might help somebody else ;)
working as of 2020
docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' container_name_or_id