I have two docker containers:
database
app that consumes the database
I run my database container like this:
docker run --name my-db -p 127.0.0.1:3306:3306 my-db-image
And my app container like this:
docker run --name my-app --network host -it my-app-image
This works fine on Linux. I can access the DB from both the host system and the app container. Perfect.
However --network host does not work on Mac and Windows:
The host networking driver only works on Linux hosts, and is not supported on Docker for Mac, Docker for Windows, or Docker EE for Windows Server.
(source: https://docs.docker.com/network/host/)
I can still access the database via 127.0.0.1:3306 from the main host, but I cannot access it from the app container.
How can I solve this issue? How can I let the app container connect to the database (and keep accessing also to the DB from the main host using 127.0.0.1:3306)?
I've tried using host.docker.internal and gateway.docker.internal but it doesn't work.
I've also tried to launch both containers using --network my-network after creating my-network with docker network create my-network but it doesn't work.
I can't figure out how to solve this issue.
For multiple services, it can often be easier to create a docker-compose.yml file that will launch all the services and any networks needed to connect them.
version: '3'
services:
my-db:
image: my-db-image
ports:
- "3306:3306"
networks:
- mynetwork
my-app:
image: my-app-image
ports:
- "8000:80"
networks:
- mynetwork
networks:
mynetwork:
From the project folder, you run docker-compose up or docker-compose up -d to make the services run in the background.
In this scenario, the magic of Docker provisions a network with hostname "mynetwork". It should expose default ports to other services on that network. If you want to remap the ports, the pattern is target:source.
I don't know that you need the 'ports' config here. But I'm trying to map your config to the compose file. Also I'm assuming you need to expose the app on some port; using 8000 as it's pretty common setup.
What are the parameters here? Docker-compose reference
Related
I am trying to connect some app that is a docker-compose service to a MongoDB running in a separate docker container using its own network on the same host machine. What URL should be used by the app to connect to that external network?
My steps...
Created a network and started MongoDB on that network:
docker network create my_app_mongo_db
docker run --name db-mongo -d --network=my_app_mongo_db -p 27017:27017 mongo
Created a docker-compose.yaml like so:
version: "3"
services:
my_app:
image: my_app
container_name: my_app
networks:
- default
- my_app_mongo_db
networks:
default:
my_app_mongo_db:
external: true
The docker-compose -up starts fine and docker network inspect my_app_mongo_db shows that the service is connected to the external network.
Next I am trying to connect to MongoDB using this URL, similarly how I would connect to the DB if it was running as a service:
mongodb://my_app_mongo_db:27017
Yet this approach doesn't work with an external network. What does work is using host.docker.internal:
mongodb://host.docker.internal:27017
But according to this Docker doc:
This is for development purpose and will not work in a production environment outside of Docker Desktop for Mac.
Any ideas how to resolve this when running on a server?
you need to connect using the Container name not the network , try this:
mongodb://db-mongo:27017
I have two services defined for docker-compose
version: '3'
services:
celery:
build:
context: .
dockerfile: ./docker/celery/Dockerfile
command: celery -A api.tasks worker -l info
rabbitmq:
image: "rabbitmq:3-management"
ports:
- "5672:5672"
- "15672:15672"
hostname: "0.0.0.0"
I can start the first service
docker-compose run --service-ports rabbitmq
And everything works well. I can ping and connect to port 5672 for communication from host os.
$ curl 0.0.0.0:5672
AMQP
However, the second service cannot see that port. The following command errors because it cannot connect to 0.0.0.0:5672.
docker-compose run --service-ports celery
How do I setup two docker containers, such that they can see each other?
From inside the Docker container, the loopback address 0.0.0.0 refers to the container itself. Use your host's IP address to reach the other container.
Here's an extensive explanation on how to reach your host from inside a container and various network modes that Docker offers: https://stackoverflow.com/a/24326540/417866
Another approach would be to create a Docker Network, connect both your containers to it, and then they'll be able to resolve each other on that network. Details here: https://docs.docker.com/engine/reference/commandline/network_create/
So the easy answer is to refer to each other by name. In your compose file you reference two services:
rabbitmq
celery
if you use docker-compose up -d (or just docker-compose up) it will create the new containers on a newly created network they share. Docker compose then registers both services to the DNS service for that network via an automatic alias.
So from celery, you could ping rabbitmq via:
ping rabbitmq and on rabbitmq you could ping celery via
ping celery
This applies to all network communications as it's just name resolution. You can accomplish this all manually by creating a new network, assigning them to the hosts, and then registering aliases, but docker-compose does all the hard work.
I am running single node zookeeper docker image using below command
docker run -it -p 2181:2181 --name zookeeper zookeeper.
Inorder to run kafka docker image, I need to share the IP address of the zookeper docker image as ENV variable and update kafka server.properties file.
Using docker inspect, I am able to fetch the IP address of zookeeper image and pass it during docker run of kafka.
But is there any way to automatically detect and share the IP address between docker containers.
I could see some example using --link, but in the latest docker documentation it seems deprecated.
Appreciate any suggestion/help
Thanks
Docker Compose is a perfect solution for your case.
By default Compose sets up a single network for your app. Each
container for a service joins the default network and is both
reachable by other containers on that network, and discoverable by
them at a hostname identical to the container name.
The docker-compose.yaml is something like below one for your case:
version: '3'
services:
zookeeper:
image: <zookeeper_image_name>
restart: always
ports:
- 2181:2181
kafka:
image: <kafka_image_name>
restart: always
Kafka container should have zookeeper as a zookeeper hostname in server.properties file.
I created two docker containers based on two different images. One of db and another for webserver. Both containers are running on my mac osx.
I can access db container from host machine and same way can access webserver from host machine.
However, how do I access db connection from webserver?
The way I started db container is
docker run --name oracle-db -p 1521:1521 -p 5501:5500 oracle/database:12.1.0.2-ee
I started wls container as
docker run --name oracle-wls -p 7001:7001 wls-image:latest
I can access db on host by connecting to
sqlplus scott/welcome1#//localhost:1521/ORCLCDB
I can access wls on host as
http://localhost:7001/console
It's easy.
If you have two or more running container, complete next steps:
docker network create myNetwork
docker network connect myNetwork web1
docker network connect myNetwork web2
Now you connect from web1 to web2 container or the other way round.
Use the internal network IP addresses which you can find by running:
docker network inspect myNetwork
Note that only internal IP addresses and ports are accessible to the containers connected by the network bridge.
So for example assuming that web1 container was started with: docker run -p 80:8888 web1 (meaning that its server is running on port 8888 internally), and inspecting myNetwork shows that web1's IP is 172.0.0.2, you can connect from web2 to web1 using curl 172.0.0.2:8888).
Easiest way is to use --link, however the newer versions of docker are moving away from that and in fact that switch will be removed soon.
The link below offers a nice how too, on connecting two containers. You can skip the attach portion, since that is just a useful how to on adding items to images.
https://web.archive.org/web/20160310072132/https://deis.com/blog/2016/connecting-docker-containers-1/
The part you are interested in is the communication between two containers. The easiest way, is to refer to the DB container by name from the webserver container.
Example:
you named the db container db1 and the webserver container web0. The containers should both be on the bridge network, which means the web container should be able to connect to the DB container by referring to its name.
So if you have a web config file for your app, then for DB host you will use the name db1.
if you are using an older version of docker, then you should use --link.
Example:
Step 1: docker run --name db1 oracle/database:12.1.0.2-ee
then when you start the web app. use:
Step 2: docker run --name web0 --link db1 webapp/webapp:3.0
and the web app will be linked to the DB. However, as I said the --link switch will be removed soon.
I'd use docker compose instead, which will build a network for you. However; you will need to download docker compose for your system. https://docs.docker.com/compose/install/#prerequisites
an example setup is like this:
file name is base.yml
version: "2"
services:
webserver:
image: moodlehq/moodle-php-apache:7.1
depends_on:
- db
volumes:
- "/var/www/html:/var/www/html"
- "/home/some_user/web/apache2_faildumps.conf:/etc/apache2/conf-enabled/apache2_faildumps.conf"
environment:
MOODLE_DOCKER_DBTYPE: pgsql
MOODLE_DOCKER_DBNAME: moodle
MOODLE_DOCKER_DBUSER: moodle
MOODLE_DOCKER_DBPASS: "m#0dl3ing"
HTTP_PROXY: "${HTTP_PROXY}"
HTTPS_PROXY: "${HTTPS_PROXY}"
NO_PROXY: "${NO_PROXY}"
db:
image: postgres:9
environment:
POSTGRES_USER: moodle
POSTGRES_PASSWORD: "m#0dl3ing"
POSTGRES_DB: moodle
HTTP_PROXY: "${HTTP_PROXY}"
HTTPS_PROXY: "${HTTPS_PROXY}"
NO_PROXY: "${NO_PROXY}"
this will name the network a generic name, I can't remember off the top of my head what that name is, unless you use the --name switch.
IE docker-compose --name setup1 up base.yml
NOTE: if you use the --name switch, you will need to use it when ever calling docker compose, so docker-compose --name setup1 down this is so you can have more then one instance of webserver and db, and in this case, so docker compose knows what instance you want to run commands against; and also so you can have more then one running at once. Great for CI/CD, if you are running test in parallel on the same server.
Docker compose also has the same commands as docker so docker-compose --name setup1 exec webserver do_some_command
best part is, if you want to change db's or something like that for unit test you can include an additional .yml file to the up command and it will overwrite any items with similar names, I think of it as a key=>value replacement.
Example:
db.yml
version: "2"
services:
webserver:
environment:
MOODLE_DOCKER_DBTYPE: oci
MOODLE_DOCKER_DBNAME: XE
db:
image: moodlehq/moodle-db-oracle
Then call docker-compose --name setup1 up base.yml db.yml
This will overwrite the db. with a different setup. When needing to connect to these services from each container, you use the name set under service, in this case, webserver and db.
I think this might actually be a more useful setup in your case. Since you can set all the variables you need in the yml files and just run the command for docker compose when you need them started. So a more start it and forget it setup.
NOTE: I did not use the --port command, since exposing the ports is not needed for container->container communication. It is needed only if you want the host to connect to the container, or application from outside of the host. If you expose the port, then the port is open to all communication that the host allows. So exposing web on port 80 is the same as starting a webserver on the physical host and will allow outside connections, if the host allows it. Also, if you are wanting to run more then one web app at once, for whatever reason, then exposing port 80 will prevent you from running additional webapps if you try exposing on that port as well. So, for CI/CD it is best to not expose ports at all, and if using docker compose with the --name switch, all containers will be on their own network so they wont collide. So you will pretty much have a container of containers.
UPDATE: After using features further and seeing how others have done it for CICD programs like Jenkins. Network is also a viable solution.
Example:
docker network create test_network
The above command will create a "test_network" which you can attach other containers too. Which is made easy with the --network switch operator.
Example:
docker run \
--detach \
--name db1 \
--network test_network \
-e MYSQL_ROOT_PASSWORD="${DBPASS}" \
-e MYSQL_DATABASE="${DBNAME}" \
-e MYSQL_USER="${DBUSER}" \
-e MYSQL_PASSWORD="${DBPASS}" \
--tmpfs /var/lib/mysql:rw \
mysql:5
Of course, if you have proxy network settings you should still pass those into the containers using the "-e" or "--env-file" switch statements. So the container can communicate with the internet. Docker says the proxy settings should be absorbed by the container in the newer versions of docker; however, I still pass them in as an act of habit. This is the replacement for the "--link" switch which is going away. Once the containers are attached to the network you created you can still refer to those containers from other containers using the 'name' of the container. Per the example above that would be db1. You just have to make sure all containers are connected to the same network, and you are good to go.
For a detailed example of using network in a cicd pipeline, you can refer to this link:
https://git.in.moodle.com/integration/nightlyscripts/blob/master/runner/master/run.sh
Which is the script that is ran in Jenkins for a huge integration tests for Moodle, but the idea/example can be used anywhere. I hope this helps others.
You will have to access db through the ip of host machine, or if you want to access it via localhost:1521, then run webserver like -
docker run --net=host --name oracle-wls wls-image:latest
See here
Using docker-compose, services are exposed to each other by name by default. Docs.
You could also specify an alias like;
version: '2.1'
services:
mongo:
image: mongo:3.2.11
redis:
image: redis:3.2.10
api:
image: some-image
depends_on:
- mongo
- solr
links:
- "mongo:mongo.openconceptlab.org"
- "solr:solr.openconceptlab.org"
- "some-service:some-alias"
And then access the service using the specified alias as a host name, e.g mongo.openconceptlab.org for mongo in this case.
Environment: Windows 10, Docker Desktop version 4.5.1.
Use hostname host.docker.internal to access services running on your host machine from inside a container.
See: https://docs.docker.com/desktop/windows/networking/#use-cases-and-workarounds
I run PostgreSQL in one container and my app in a separate container.
I configure the app database connection to use host.docker.internal as the hostname and it just works.
Consider Example
We Create two containers here PostgreSQL server and pgadmin(for accessing servers like PHPMyAdmin, SQL studio, workbench).
Exposed port
PostgreSql --->5436
Pgadmin --->5050
After adding a server in pgadmin hostname as localhost.It will show a connection error. Because Docker container pgadmin getting localhost as their system instead we need PostgreSQL IP to solve the problem.
docker network create con
docker network connect con app1
docker network connect con app2
This command gets connected container IP address and other details.
docker network inspect con
Now you can see the IP address shown in the network inspect. Choose the Postgres container IP. You can access other exposed ports through this IP. Here postgre 5432 is only exposed.Now set hostname as the container ip and it will work.
You can use the default docker network. If you don't want to go through any docker networking, you can do this:
Copy the ip address in Docker subnet in Resources>Network in Docker Preferences in Mac:
Docker preferences screenshot
As you can see from the screenshot link the ip address is
192.168.65.0
You just need to replace “localhost” in your containers config file with “192.168.65.1" (i.e. IP address picked + 1 ).
You can start your containers and should be set for local development/testing.
For some more details, you can see my article:
Connect Docker containers the easy way
In my case, the host connection in the application to a container from an other container by the IP provide by the bridge didn't work.
But it works with the name of the container (see my screenshot).
So you can replace the IP by the name of the container.
Here's the result when I type docker ps :
I have 3 docker containers: webapps, redis and rabbitmq. I want to link container webapps to container redis and rabbitmq container. In non docker apps, mywebapps can send message to rabbitmq and write/read redis.
I tried using command like this
docker run --name rabbitmq -p 8080:80 --link webapps:nimmis/apache-php7 -d rabbitmq
but it does not work.
Here is my config.php on webapps where I am trying to send messages via rabbitmq:
define('HOST', 'localhost');
define('PORT', 5672);
I tried to change localhost with hostname
define('HOST', 'rabbitmq');
define('PORT', 5672);
Error message says connection refused.
It seems that in my three containers needs to be configured in the same network namespace.
Linking is a legacy feature. Please use "user defined networks":
sudo docker network create mynetwork
Then rerun your containers using this network:
sudo docker run --name rabbitmq -p 8080:80 -d --network mynetwork rabbitmq
Do the same for other containers that you want connected with each other.
Using "user defined networks", you have an "internal name resolution" at your disposal (somewhat like domain name resolution when visiting websites). You can use the names of the container that you want to refer to, in order to resolve the IP addresses of containers, as long as they are running on the same "user defined network". With this, you can resolve the IP address of the rabbitmq container with its name, within other containers, on the same network.
All containters on the same "user defined network" will have network connectivity. There is no need for "legacy linking".
For inter-container dependencies and links, you'll want to use docker-compose where you can define the links between containers.
In your root directory where you store your Docker files, just make a new file called docker-compose.yml and here you can define your containers as services which rely on each other like this:
version: '2'
services:
webapps:
build: .
links:
- "rabbitmq:rabmq"
- "redis"
rabbitmq:
image: rabbitmq
redis:
image: redis
so here in the definition of the webapps service, you see it links the other two services rabbitmq and redis. What this means is that when the webapps container is build, an entry to it's hosts file is made such that the domain name redis is translated to the IP and port number of the actual container.
You have the option to change the name of how this container is address by using the service:alias notation, like how I defined the rabbitmq to
use the alias rabmq inside the container webapps.
To now build and start your containers using docker-compose just type:
docker-compose up -d
So connecting to another container is as simple as using this alias as the name of the host.
Since you are using docker-compose in this case, it creates a docker network automatically to connect all the containers so you shouldn't have to worry about that. But for more information have a look at the docs:
https://docs.docker.com/compose/networking/#/specifying-custom-networks
You need to link rabbitmq and redis to your webapps container and not the other way arround.
#run redis container
docker run --name some-redis -d redis
#run rabbitmq container
docker run -d --hostname my-rabbit --name some-rabbit rabbitmq
#run webapps container
docker run --name webapps -p 8080:80 --link some-redis:redis --link some-rabbit:rabbitmq nimmis/apache-php7
First run redis and rabbitmq containers.
Then run webapps container with links to the 2 containers.
Now, to configure redis host in the webapps - its easy. You can simply use env variable 'REDIS_PORT_6379_TCP_ADDR'. Because once a container is linked you get its env variables. and redis exports that variable.
Regarding the rabbitmq host - you can get the ip after the rabbit container is up by:
RABBITMQ_IP=$(docker inspect --format '{{ .NetworkSettings.IPAddress }}' some-rabbit)
and then pass it in --env when you run the webapps container.
In my experience working with declaratives such as docker-compose.yml is okay, but simply you can use
docker run -d -P -link nimmis/apache-php7 rabbitmq redis
You can define your services to use a user-defined network in your docker-compose.yml
version: "3"
services:
webapps:
image: nimmis/apache-php7
ports:
- "80:8080"
networks:
- my-network
rabbitmq:
image: rabbitmq
networks:
- my-network
redis:
image: redis
networks:
- my-network
networks:
my-network:
driver: overlay
Then do:
docker swarm init
docker stack deploy -c docker-compose.yml my-stack
Check out the full example at https://docs.docker.com/get-started/part3/
You could access the IP Address of your Redis Container.
Start rabbitmq and get the internal IP Adress:
docker inspect -f '{{range.NetworkSettings.Networks}}{{.IPAddress}}{{end}}' rabbitmq > .rabbitmq.ip
Now, you can add an Apache configuration and add the internal IP Address for rabbitmq while starting the webapps container. Or simply add an entry in the Apache container's /etc/hosts like:
// the dynamic internal IP of rabbitmq is known once rabbitmq starts:
172.30.20.10 rabbitmq.redis.local