I am trying to get JMX monitoring working to monitor a test kafka instance.
I have kafka (ches/kafka) running in docker via boot2docker but I can not get JMX monitoring configured properly. I have done a bunch of troubleshooting and I know the kafka instance is running properly (consumers and producers work). The issue arises when I try simple JMX tools (jconsole and jvisualvm) and both can not connect (insecure connection error, connection failed).
Configuration items of note: I connect to 192.168.59.103 (virtualbox ip found when running 'boot2docker ip') and the ches/kafka docker/kafka instance uses the port 7203 as the JMX_PORT (confirmed in the kafka startup logs). Using jconsole, I connect to 192.168.59.103:7203 and that is when the errors occur.
Any help is appreciated.
For completeness, here is the solution that works:
I ran ches/kafka docker image as follows -- note that the JMX_PORT (7203) is now published appropriately:
$ docker run --hostname localhost --name kafka --publish 9092:9092 --publish 7203:7203 --env EXPOSED_HOST=192.168.59.103 --env ZOOKEEPER_IP=192.168.59.103 ches/kafka
Also, the following environment is set in the kafka-run-class.sh (.bat for windows)
-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false
But I needed to add one additional item (thanks to one of the commenters for pointing this out):
-Dcom.sun.management.jmxremote.rmi.port=7203
Now, to run the ches/docker image in boot2docker you just need to set one of the recognized environment variables (KAFKA_JMX_OPTS or KAKFA_OPTS) to add the additional item and it now works.
Thanks for the help!
There's no reason the kafka docker port would bind to the same port in the boot2docker VM except if you specify it.
Try running it with -p 7203:7203 to force the 1:1 forwarding of the port.
Related
I have a macbook and I have locally running zookeeper and kafka instances on port 2181 and 9092 respectively. I am able to bring up kowl locally using the following command mentioned in the git repo
kowl % docker run -p 8081:8081 -e KAFKA_BROKERS=host.docker.internal:19092 quay.io/cloudhut/kowl:master
I see that the app is running successfully on docker, please find the logs attached below
{"level":"info","msg":"config filepath is not set, proceeding with
options set from env variables and flags"}
{"level":"info","ts":"2022-03-23T17:48:25.475Z","msg":"started
Kowl","version":"master","git_sha":"5ff6e3c4dea98737b661186519eb310f2a898d06","built":"2022-03-09T15:34:53Z"}
{"level":"info","ts":"2022-03-23T17:48:25.478Z","msg":"connecting to
Kafka seed brokers, trying to fetch cluster metadata"}
{"level":"info","ts":"2022-03-23T17:48:25.489Z","msg":"successfully
connected to kafka
cluster","advertised_broker_count":1,"topic_count":2,"controller_id":0,"kafka_version":"v3.0"}
{"level":"info","ts":"2022-03-23T17:48:25.489Z","msg":"creating Kafka
connect HTTP clients and testing connectivity to all clusters"}
{"level":"info","ts":"2022-03-23T17:48:25.489Z","msg":"tested Kafka
connect cluster
connectivity","successful_clusters":0,"failed_clusters":0}
{"level":"info","ts":"2022-03-23T17:48:25.490Z","msg":"successfully
create Kafka connect service"}
{"level":"info","ts":"2022-03-23T17:48:25.549Z","msg":"Server
listening on address","address":"[::]:8080","port":8080}
I am unable to access the application from the browser at https://localhost:8081. I tried accessing this via the container IP using docker inspect as well as host.docker.internal ip but to no avail. I recently moved to a mac, is it a known issue ? Any solutions would be greatly appreciated.
the following command worked, I had the ports mixed up
docker run -p 8081:8080 -e KAFKA_BROKERS=host.docker.internal:19092 quay.io/cloudhut/kowl:master
now I am able to access the ui at http://localhost:8081/
I have 2 docker containers running on my Mac host - container 1 is Jenkins from Docker Hub and container 2 is SonarQube from Docker Hub. I have both containers running successfully. I can access Jenkins from my host by going to http://localhost:8080/ and I can access my SonarQube by going to http://localhost:9000/.
The Jenkins container was started like this:
docker run -d -p 8080:8080 -p 50000:50000 jenkins/jenkins:latest
The SonarQube container was started like this:
docker run -d -p 9000:9000 sonarqube
Now I want to have each container communicate with each other so I need to provide the IP address of the other container to each container.
I got the IP address of each container by executing this:
docker inspect --format '{{ .NetworkSettings.IPAddress }}' container_name_or_id
This returns an IP address of 172.17.0.2 for the Jenkins container and 172.17.0.3 for the SonarQube container. But when I try and access the Jenkins container from my host by going to http://172.17.0.2:8080 I get a request timeout. The same thing happens when I try and access the SonarQube container from my host by going to http://172.17.0.3:9000
Is this normal behavior?
Shouldn't I be able to access each container from my host by their internal IP address?
And how can I test that one container (e.g. Jenkins) can access the other container (e.g. SonarQube) by IP address?
Is this normal behavior? Shouldn't I be able to access each container from my host by their internal IP address?
What you describe is normal behavior: you can't directly reach the Docker-internal IP addresses from a MacOS host. See "Per-container IP addressing is not possible" in the Docker for Mac docs.
How can I test that one container (e.g. Jenkins) can access the other container (e.g. SonarQube) by IP address?
This isn't something I normally "test" per se. Start up both processes and have them make their normal (HTTP) connections; if it works you'll see appropriate log messages, and if it doesn't work you'll see complaints. (Getting a root shell in a container to send ICMP packets from one container to another seems to be a popular option but doesn't prove much.)
Also: don't make this connection by explicit IP address. As you've noticed already the Docker-internal IP addresses aren't usable in some contexts, and they'll change whenever you restart containers. Instead, Docker provides an internal DNS service that can resolve host names when communicating between containers, but you need to explicitly set up a non-default bridge network. That setup would look like:
docker network create jenkinsnet
docker run --name sonarqube -d --net jenkinsnet \
-p 9000:9000 \
sonarqube
docker run --name jenkins -d --net jenkinsnet \
-p 8080:8080 -p 50000:50000 \
-e SONARQUBE_URL=http://sonarqube:9000 \
jenkins/jenkins:latest
So I've explicitly created a network; started both containers connected to it; and told the client container (via an environment variable) where the server container is. You don't have to publish ports with docker run -p to reach them this way; whether you do or not, use the port the server process is listening on (the second port number in the docker run -p option).
From the host, your only (portable, reliable) path to reach the container is via its published ports.
Looks like you are using default bridge network model. Internal IPs are meant for each container to talk to each other under bridge networking. You cannot access them from host.
There are multiple options for you.
You can configure http://172.17.0.3:9000 as your sonar endpoint in Jenkins.
You can configure http://172.17.0.2:8000 as your jenkins endpoint in sonar.
If you don't want to hard code above Ips then both of your containers can talk to each using Docker Default GatewayIp(172.17.0.1) and their internal port. so essentially you can configure http://172.17.0.1 as well.
Note - Default Gateway Ip change change if you define user defined bridge network.
https://docs.docker.com/v17.09/engine/userguide/networking/#the-default-bridge-network
https://docs.docker.com/network/network-tutorial-standalone/
If you want to spin up both containers using docker-compose, then you can link both containers using service name. Just follow Networking in Compose.
The accepted answer (https://stackoverflow.com/a/53992787/7730554) already provides valid options of which I personally usually prefer using docker compose.
But as you are running Docker on Mac you could also use host.docker.internal in combination with the defined forwarding host port. So Docker will take care that host.docker.internal is resolved to the corresponding IP even if your Host IP changes.
See https://docs.docker.com/desktop/mac/networking/.
Note that this is for development mode only and works when you use Docker Desktop.
I am running docker selenium hub on windows. The container is up as docker log shows. However I am confused at what is the actual hostname that the hub uses. When i use conventional Selenium hub, I can use http://localhost:4444/grid/console to check that it's running correctly. But in this case of docker, I am confused at the printing of Docker log.
I tried 0.0.0.0, localhost, 172.17.0.2 as hostname to open /grid/console in browser. None of them works.
Also when I tried to use 'netstat -a' to list all tcp connections, I do not see any of them has port 4444.
Could anyone let me know what went wrong?
You are running docker on windows, there are two things you can do.
Map your port 4444 to host when starting the container.
On windows, your container is ruining in Boot2Docker virtual machine.
So instead of http://localhost:4444/grid/console use
http://{ip of boot2docker vm}:4444/grid/console
To get IP on boot2docker machine try the following command:
docker-machine ip default
Port expose docker example:
docker run -d -p 4444:4444 --name <container-name> <image-name>
I'm new to docker and maybe this is something I don't fully understand yet, but what I'm trying to do is connect to an open port in a running docker container. I've pulled and run the rabbitmq container from hub (https://hub.docker.com/_/rabbitmq/). The rabbitmq container should uses port 5672 for clients to connect to.
After running the container (as instructed in the hub page):
$ docker run -d --hostname my-rabbit --name some-rabbit rabbitmq:3
Now what I want to do is telnet into the open post (it is possible on a regular rabbitmq installation and should be on a container as well).
I've (at least I think I did) gotten the container IP address using the following command:
docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' container_name_or_id
And the result I got was 172.17.0.2. When I try to access using telnet 172.17.0.2 5672 it's unsuccessful.
The address 172.17.0.2 seems strange to me because if I run ipconfig on my machine I don't see any interface using 172.17.0.x address. I do see Ethernet adapter vEthernet (DockerNAT) using the following ip: 10.0.75.1. Is this how it is supposed to be?
If I do port binding (adding -p 5672:5672) then I can telnet into this port using telnet localhost 5672 and immidiatly connect.
What am I missing here?
As you pointed out, you need port binding in order to achieve the result you need because you are running the application over the default bridge network (on Windows i guess).
From the official docker doc
Containers connected to the same user-defined bridge network automatically expose all ports to each other, and no ports to the outside world. [...]
If you run the same application stack on the default bridge network, you need to open both the web port and the database port, using the -p or --publish flag for each. This means the Docker host needs to block access to the database port by other means.
Later in the rabbitmq hub there is a reference to a Management Plugin which is run by executing the command
docker run -d --hostname my-rabbit --name some-rabbit -p 8080:15672 rabbitmq:3-management
Which exposes the port 8080 used for management which I think is what you may need.
You should also notice that they talk about clusters and nodes there, maybe they meant the container to be run as a service in a swarm (hence using the overlay network and not the bridge one).
Hope I could help somehow :)
In short - Can I run Elasticsearch & Dropwizard app in separate docker containers and allow them to see each other?
I am running Elasticsearch 6.2.2 from Docker (on mac). using the command..
docker run -p 9200:9200 -p 9300:9300 -e "network.host=0.0.0.0" \
-e "http.port=9200" -e "discovery.type=single-node" \
docker.elastic.co/elasticsearch/elasticsearch:6.2.2
I can access Elasticsearch (POST & GET) fine using Postman directly on mac e.g
localhost:9200/testindex/_search
However when running a Dropwizard application from a different docker image which accesses the docker Elasticsearch instance, I get connection refused using same host and port (localhost 9200).
I have no problems at all when running the Dropwizard app direct from an IDE, only when its running from a docker image and accessing ES from a different image.
docker image -p 8080:8080 -p 8081:8081 testapp
Has anyone else had similar issues or solved this in the past?
I'm assuming it 'network' related and that connecting to localhost from one docker image will not be map to the other docker image
The issue you are facing is in the url you pass to the dropwizard container. As a container by default has its own networking, a value of localhost means the dropwizard container itself, not what you see as your local host from outside the container.
Please have a look at docker networking, how you can link two containers by name. I would suggest to check out docker-compose for multi-container setups on a local machine.
What would also work (but is not good practice) is to pass the dropwizard container the ip of your machine as elasticsearch host because you created a port mapping from your host into the elasticsearch container. But better have a look at compose to do it as it is supposed to be done.
For details how to use compose please have a look at this answer with a similar example.