I have an error whith docker swarm new services when I'm added somes services with a port like :
docker service create --name=elasticSearch --network=swarm-net -e xpack.security.enabled=false -e xpack.monitoring.enabled=false -e xpack.graph.enabled=false -e xpack.watcher.enabled=false -p 9200:9200 -p 9300:9300 -d docker.elastic.co/elasticsearch/elasticsearch:5.3.2
When I don't publish any ports the service works well but when I'm added it, the process stay at state "new" not running.
Did you have an idea ?
Regards
Related
I just tried to create two containers for Elastic Search and Kibana.
docker network create esnetwork
docker run --name myes --net esnetwork -p 127.0.0.1:9200:9200 -p 127.0.0.1:9300:9300 -e "discovery.type=single-node" elasticsearch:7.9.3
and Elastic Search works when I use http://localhost:9200 or http://internal-ip:9200
But when I use http://myes:9200, it just can't resolve the container name.
Thus when I run
docker run --name mykib --net esnetwork -p 5601:5601 -e “ELASTICSEARCH_HOSTS=http://myes:9200” docker.elastic.co/kibana/kibana:7.9.3
It couldn't be created because it cannot resolve myes:9200
I also tried to replace "ELASTICSEARCH_HOSTS=http://myes:9200" with localhost:9200 or internal IP instead. but nothing works.
So I think my question should be how to make the container's DNS works?
How are you resolving 'myes'?
Is it mapped in hostname file and resolving to 127.0.0.1?
Also, use 127.0.0.1 wherever possible as localhost could be pointing to something else and not getting resolved.
It seems this problem doesn't arise from DNS. Both Elastic search and Kibana containers should use the fix name "elasticsearch" . so the docker command will be:
$docker network create esnetwork
$sudo vi /etc/sysctl.d/max_map_count.conf
vm.max_map_count=262144
$docker run --name elasticsearch --net esnetwork -p 127.0.0.1:9200:9200 -p 127.0.0.1:9300:9300 -e
$docker run --name kib01-test --net esnetwork -p 5601:5601 -e “ELASTICSEARCH_HOSTS=http://elasticsearch:9200” docker.elastic.co/kibana/kibana:7.9.3
Then if the terminals that run installations could be ended automatically, just close them. And restart containers from the docker desktop manager. Then everything will go smoothly.
My environment is Fedora 36, docker 20.10.18
So i have mysqlrouter docker container that is load balancing request to my mysql innodb cluster and every thing is working perfectly.
Now being that having a single mysqlrouter in my setup also means that the single router is a single-point-of-failure, i want to achieve HA with spinning another docker container - mysqlrouter-bkup. Also, i want to spin 2 keepalived docker containers monitoring the mysqlrouter cluster so as to cause failover (i.e make the backup instance a master when the master instance dies).
What i have managed to do so far (without any proper documented guidance on how to achieve this using docker containers online) is:
Spin up the first mysqlrouter with the following command:
docker run --name mysqlrouter -p 6446:6446 -p 4667:4667 --net=smartdev-pro_internalnet -e MYSQL_HOST=mysql1 -e MYSQL_PORT=3306 -e MYSQL_USER=clusterAdmin -e MYSQL_PASSWORD=password -e MYSQL_INNODB_CLUSTER_MEMBERS=3 -e MYSQL_CREATE_ROUTER_USER=0 mysql/mysql-router
Spin up the second mysqlrouter with the following command:
docker run --name mysqlrouter-bkup -p 6448:6446 -p 4669:4667 --net=smartdev-pro_internalnet -e MYSQL_HOST=mysql1 -e MYSQL_PORT=3306 -e MYSQL_USER=clusterAdmin -e MYSQL_PASSWORD=password -e MYSQL_INNODB_CLUSTER_MEMBERS=3 -e MYSQL_CREATE_ROUTER_USER=0 mysql/mysql-router
Spin up the first keepalived docker container:
docker run --name keepalived1 --env KEEPALIVED_STATE="MASTER" --env KEEPALIVED_INTERFACE="eth0" --env KEEPALIVED_UNICAST_PEERS="#PYTHON2BASH:['172.18.0.9', '172.18.0.10']" --env KEEPALIVED_ROUTER_ID="100" --env KEEPALIVED_PRIORITY="200" --env KEEPALIVED_PASSWORD="password" --env KEEPALIVED_VIRTUAL_IPS="172.18.0.100" --cap-add=NET_ADMIN --cap-add=NET_BROADCAST --cap-add=NET_RAW --net=host osixia/keepalived:2.0.20
Spin up the second keepalived docker container:
docker run --name keepalived2 --env KEEPALIVED_STATE="BACKUP" --env KEEPALIVED_INTERFACE="eth0" --env KEEPALIVED_UNICAST_PEERS="#PYTHON2BASH:['172.18.0.9', '172.18.0.10']" --env KEEPALIVED_ROUTER_ID="101" --env KEEPALIVED_PRIORITY="100" --env KEEPALIVED_PASSWORD="password" --env KEEPALIVED_VIRTUAL_IPS="172.18.0.100" --cap-add=NET_ADMIN --cap-add=NET_BROADCAST --cap-add=NET_RAW --net=host osixia/keepalived:2.0.20
172.18.0.9 is mysqlrouter ip address and 172.18.0.10 is mysqlrouter-bkup ip address.
So all containers start successfully and the complete logs for both keepalived services are posted here: click me because i couldn't post more than 30,000 characters here.
Now in my nodejs mysql connection string, 172.18.0.100 (which is the virtual ip of keepalived) is what i supply and the host.
const connection = mysql.createPool({
host: 172.18.0.100,
user: process.env.USER,
password: process.env.PASSWORD,
database: process.env.DATABASE,
port: 6446,
connectionLimit: 20
});
So with this setup, everything works fine until i stop the mysqlrouter container to see if it will failover to mysqlrouter-bkup, it doesn't.
Please can someone point out to me what i am doing wrong.
Extra Info: i am running this containers on docker desktop(with wsl2), windows 11.
And please if there is a better way of achieving my purpose using docker on windows, i am open to it.
Well, the set up is simple, there should be two containers: one of them for the mysql database and the other one for web application.
What I do to run the containers,
the first one for database and the second for the app:
docker run --name mysql-container -e MYSQL_ROOT_PASSWORD=root -e MYSQL_DATABASE=db -p 3306:3306 -d mysql
docker run -p 8081:8081 myrepo/myapp
The application tries to connect to database using localhost:3306, but as I found out the issue is that each container has its own localhost.
One of the solution I found was to add the same network for containers using --net and the docker commands happend to be like the following:
docker network create my-network
docker run --name mysql-container -e MYSQL_ROOT_PASSWORD=root -e MYSQL_DATABASE=db -p 3306:3306 -d
--net my-network mysql
docker run --net my-network -p 8081:8081 myrepo/myapp
Though, the web application still is not able to connect to the database. What am I doing wrong and what is the proper flow to connect application to database when they are both inside containers?
You could use the name of the container (i.e. mysql-container) to connect to mysql. Example:
Run the mysql container:
docker run --name mysql-container -e MYSQL_ROOT_PASSWORD=root -e MYSQL_DATABASE=db -p 3306:3306 -d --net my-network mysql
Connect from another container using the mysql client:
docker run --net my-network -it mysql mysql -u root -p db -h mysql-container
In your application you should replace in the database URL, whatever IP you have with mysql-container.
Well, after additional research, I successfully managed to connect to the database.
The approach I used is the following:
On my host I grabbed the IP address of the docker itself but not the specific container:
sudo ip addr show | grep docker0
The IP address of the docker0 I added to the database connection URL inside my application and thus application managed to connect to the database (note: with this flow I don't add the --net keyword when start container)
What definitely strange is that even adding shared network like --net=my-nework for both the container didn't work. Moreover I did try to use --net=host to share the host network with container's one, still it was unsuccessful. If there's any who can explain why it couldn't work, please - share your knowledge.
I'm trying to connect to the Kafka using a KafkaTool. I got an error:
Error connecting to the cluster. failed create new KafkaAdminClient
Kafka and Zookeeper is hosting in the Docker. I run next commands
docker network create kafka
docker run --network=kafka -d --name zookeeper -e ZOOKEEPER_CLIENT_PORT=2181 confluentinc/cp-zookeeper:latest
docker run --network=kafka -d -p 9092:9092 --name kafka -e KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181 -e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://kafka:9092 -e KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=1 confluentinc/cp-kafka:latest
Settings for KafkaTool
Why does KafkaTool not connect to the Kafka that is hosting in the Docker?
I'm assuming this GUI is not coming from a Docker container. Therefore, your host machine doesn't know what zookeeper or kafka are, only the Docker network does.
In the GUI, you will want to use localhost for both, then in your Kafka run command, leave all the other variables alone but change -e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://localhost:9092
Zookeeper run command is fine, but add -p 2181:2181 to expose the port out to the host so that the GUI can connect
I want to install elasticsearch and kibana, on dockers, on my host machine:
$sudo docker run -dit --name elasticsearch -h elasticsearch --net host -p 9200:9200 -p 9300:9300 -v $(pwd)/elasticsearch/data/:/usr/share/elasticsearch/data/ -e "discovery.type=single-node" elasticsearch:6.6.1
WARNING: Published ports are discarded when using host network mode
$sudo docker run -dit --name kibana -h kibana --net host -p 5601:5601 kibana:6.6.1
WARNING: Published ports are discarded when using host network mode
and I get the following errors on kibana:
log [14:32:26.655] [warning][admin][elasticsearch] Unable to revive connection: http://elasticsearch:9200/
log [14:32:26.656] [warning][admin][elasticsearch] No living connections
But If I don't use host machine:
sudo docker network create mynetwork
sudo docker run -dit --name elasticsearch -h elasticsearch --net mynetwork -p 9200:9200 -p 9300:9300 -v $(pwd)/elasticsearch/data/:/usr/share/elasticsearch/data/ -e "discovery.type=single-node" elasticsearch:6.6.1
sudo docker run -dit --name kibana -h kibana --net mynetwork -p 5601:5601 kibana:6.6.1
all working fine. What is the problem?
--net host disables most of the Docker networking stack. Basic features like communicating between containers using their container name as a host name don’t work. Except in very unusual circumstances it’s never necessary.
Your second setup that uses standard Docker networking and publishes selected ports through the host is a best practice.