I have mysql DB running in a container and web app running in another container. My use case is once the DB container is up and running app container has to insert some initial data to DB using Liquibase and start the app. My docker yml looks like below.
db:
build: kdb
user: "1000:50"
volumes:
- /data/mysql:/var/lib/mysql
container_name: kdb
environment:
- MYSQL_ALLOW_EMPTY_PASSWORD=yes
image: kdb
ports:
- "3307:3306"
k-api:
container_name: k_api
hostname: k-api
domainname: i.com
image: k_api
volumes:
- /Users/agu/work:/data
build:
context: ./api
args:
KB_API_WAR: k-web-1.2.9.war
KB_API_URL: https://artifactory.b-aws.i.com
ports:
- "8097:8080"
depends_on:
- db
#command: [/usr/local/bin/wait-for-it.sh, "db:3306","-s","-t","0","--","/bin/sh" "wait_for_liquibase.sh"]
links:
- "db:kdb_docker_host"
And in my Dockerfile for api i have entry point for a shell script called "wait_for_liquibase.sh"
CMD ["wait_for_liquibase.sh"]
wait_for_liquibase.sh:
#!/bin/sh
set -e
#RUN liquibase
mvn clean install -X -PdropAll -Dcontexts=test -Dliquibase.user=XX -Dliquibase.pass=XX -Dliquibase.host=db -Dliquibase.port=3306 -Dliquibase.schema=knowledgebasedb -DpromptOnNonLocalDatabase=false -Dcontexts=test -f k/k-liquibase
/usr/local/tomcat/bin/catalina.sh run
The issue is once the DB container is up and running app container is not able to reach the DB server to perform liquibase setup for database. I see the below error.
Communication failure: Unknown database host -Dliquibase.host=db.
I am assuming you are using version 1.
You giving an alias to your "db" service, you will need to use that alias, kdb_docker_host
Also, the ports are mapping to the host machine, to expose ports between containers yuo will need to use the expose property.
expose:
- 3306
I used this in Docker file RUN apt-get update
RUN apt-get install netcat -y
ADD wait-for-base.sh /wait-for-base.sh
CMD ["/wait-for-base.sh"]
and in wait-for-base.sh :
#!/bin/bash
while ! nc -z db 3306; do sleep 3; done
[my command to run]
In your case /usr/local/tomcat/bin/catalina.sh run
Related
When building the docker file, I have the command:
CMD ["/app/database/updateLocalDocker.sh"]
The shell script should connect to the postgres service using liquibase but fails with the error connection refused...
When i comment out the above CMD and run the same script directory from the container via docker exec -t -i f42c4bbcd95d /bin/bash, it works fine.
The URL i'm trying to connect to is: jdbc:postgresql://localhost:5432/service_x"
I have a feeling that it's related to either the service not being started or a network issue, when trying to execute the CMD during the docker-compose build stage.
Any guidance would be much appreicated.
docker-compose.yml:
version: "3.8"
services:
db:
image: local.db
build:
context: .
ports:
- 15432:5432
environment:
POSTGRES_PASSWORD: password
networks:
- a
networks:
a:
name: a
external: true
To access your database from your localhost you need to use the port 15432 instead of 5432.
services:
db:
image: local.db
build:
context: .
ports:
- 15432:5432 <--- Here
environment:
POSTGRES_PASSWORD: password
networks:
- a
The first port is your host and the second is the port used in your container.
You can also access it with the container name and the port used in it.
Docker port mapping documentation :
https://docs.docker.com/config/containers/container-networking/
Instead of putting the command in the Dockerfile, you can directly put the command in the docker-compose file and remove CMD ["/app/database/updateLocalDocker.sh"].
docker-compose.yml
version: "3.8"
services:
db:
image: local.db
build:
context: .
command: sh -c "<Enter-your-command>"
ports:
- 15432:5432
environment:
POSTGRES_PASSWORD: password
networks:
- a
networks:
a:
name: a
external: true
If you have one command execute
command: <command>
OR
If you have more than one command, it should be separated by &&.
Syntax:
sh -c "<command-1> && <command-2> && <command-3>"
I'm building 2 docker containers, "app" and "db", via a docker-compose file.
The app server just installs java/tomcat via a Dockerfile which is what docker-compose uses to build.
The db server uses an MS SQL image.
When I run:
docker-compose up
I follow that with a build process of software I need to load which deploys a war to the tomcat directory in the app server and builds the database in the database server.
My problem is: The build process can reference localhost:8080 to install/patch the software to the app server and reference localhost:1433 to install/patch the database portion of the software to the database server. However, when I start Tomcat the system doesn't come online because the app server can't connect to the database server via "localhost:1433" so it requires me to jump in and update the properties file after the build to the docker internal IP address and THEN it works.
My question is: How am I able to get my localhost and my app container to reference the DB in the same manner in a database url?
Dockerfile for app server:
FROM centos:centos7
COPY apache-tomcat-9.0.20.tar.gz /tmp/
WORKDIR /tmp/
RUN yum -y update
RUN yum -y install java-11-openjdk-devel
RUN tar -xf apache-tomcat-9.0.20.tar.gz
RUN mv apache-tomcat-9.0.20 /opt/tomcat/
RUN export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.212.b04-0.el7_6.x86_64/
RUN export PATH=$PATH:$JAVA_HOME/jre/bin
RUN export CATALINA_HOME=/opt/tomcat/
RUN export PATH=$PATH:$CATALINA_HOME/bin
WORKDIR /opt/tomcat/webapps
RUN mkdir testapp
enter code here
enter code here
Docker-Compose File:
version: '3.3'
services:
db:
image: "mcr.microsoft.com/mssql/server:2017-latest"
restart: always
volumes:
- db_data:/var/lib/mssql
environment:
- ACCEPT_EULA=Y
- SA_PASSWORD=Test123
network_mode: bridge
hostname: db
ports:
- "1433:1433"
app:
build: './testapp'
volumes:
- './system/build:/opt/tomcat/webapps/testapp/'
ports:
- "8080:8080"
- "8009:8009"
network_mode: bridge
tty: true
depends_on:
- db
volumes:
db_data:
Bring your service to the same network and target the service by service name. For that you need to define a docker network like below. For the following example I can access DB with http://mongo:27017.
mongo:
image: mongo:latest
ports:
- "27017:27017"
volumes:
- ./data/db:/data/db
networks:
- my-net
spring:
depends_on:
- mongo
image: docker-spring-http-alpine
ports:
- "8080:8080"
networks:
- my-net
networks:
my-net:
I have created a docker-compose file it has two services with Go and Mysql. It creates container for go and mysql. Now i am running code which try to connect to mysql database which is running as a docker container. but i get error.
docker-compose.yml
version: "2"
services:
app:
container_name: golang
restart: always
build: .
ports:
- "49160:8800"
links:
- "mysql"
depends_on:
- "mysql"
mysql:
image: mysql
container_name: mysql
volumes:
- dbdata:/var/lib/mysql
restart: always
environment:
- MYSQL_ROOT_PASSWORD=root
- MYSQL_DATABASE=testDB
- MYSQL_USER=root
- MYSQL_PASSWORD=root
ports:
- "3307:3306"
volumes:
dbdata:
Error while connecting to mysql database
golang | 2019/02/28 11:33:05 dial tcp 127.0.0.1:3306: connect: connection refused
golang | 2019/02/28 11:33:05 http: panic serving 172.24.0.1:49066: dial tcp 127.0.0.1:3306: connect: connection refused
golang | goroutine 19 [running]:
Connection with MySql Database
func DB() *gorm.DB {
db, err := gorm.Open("mysql", "root:root#tcp(mysql:3306)/testDB?charset=utf8&parseTime=True&loc=Local")
if err != nil {
log.Panic(err)
}
log.Println("Connection Established")
return db
}
EDIT:Updated docker file
FROM golang:latest
RUN go get -u github.com/gorilla/mux
RUN go get -u github.com/jinzhu/gorm
RUN go get -u github.com/go-sql-driver/mysql
COPY ./wait-for-it.sh .
RUN chmod +x /wait-for-it.sh
WORKDIR /go/src/app
ADD . src
EXPOSE 8800
CMD ["go", "run", "src/main.go"]
I am using gorm package which lets me connet to the database
depends_on is not a verification that MySQL is actually ready to receive connections. It will start the second container once the database container is running regardless it was ready for connections or not which could lead to such an issue with your application as it expects the database to be ready which might not be true.
Quoted from the documentation:
depends_on does not wait for db and redis to be “ready” before starting web - only until they have been started.
There are many tools/scripts that can be used to solve this issue like wait-for which sh compatible in case your image based on Alpine for example (You can use wait-for-it if you have bash in your image)
All you have to do is to add the script to your image through Dockerfile then use this command in docker-compose.yml for the service that you want to make it wait for the database.
What comes after -- is the command that you would normally use to start your application
version: "2"
services:
app:
container_name: golang
...
command: ["./wait-for", "mysql:3306", "--", "go", "run", "myapplication"]
links:
- "mysql"
depends_on:
- "mysql"
mysql:
image: mysql
...
I have removed some parts from the docker-compose for easier readability.
Modify this part go run myapplication with the CMD of your golang image.
See Controlling startup order for more on this problem and strategies for solving it.
Another issue that will rise after you solve the connection issue will be as the following:
Setting MYSQL_USER with root value will cause a failure in MySQL with this error message:
ERROR 1396 (HY000) at line 1: Operation CREATE USER failed for 'root'#'%'
This is because this user already exist in the database and it tries to create another. if you need to use the root user itself you can use only this variable MYSQL_ROOT_PASSWORD or change the value of MYSQL_USER so you can securely use it in your application instead of the root user.
Update: In case you are getting not found and the path was correct, you might need to write the command as below:
command: sh -c "./wait-for mysql:3306 -- go run myapplication"
First, if you are using latest version of docker compose you don't need the link argument in you app service. I quote the docker compose documentation Warning: The --link flag is a legacy feature of Docker. It may eventually be removed. Unless you absolutely need to continue using it, https://docs.docker.com/compose/compose-file/#links
I think the solution is to use the networks argument. This create a docker network and add each service to it.
Try this
version: "2"
services:
app:
container_name: golang
restart: always
build: .
ports:
- "49160:8800"
networks:
- my_network
depends_on:
- "mysql"
mysql:
image: mysql
container_name: mysql
volumes:
- dbdata:/var/lib/mysql
restart: always
networks:
- my_network
environment:
- MYSQL_ROOT_PASSWORD=root
- MYSQL_DATABASE=testDB
- MYSQL_USER=root
- MYSQL_PASSWORD=root
ports:
- "3307:3306"
volumes:
dbdata:
networks:
my_network:
driver: bridge
By the way, if you only connect to Mysql from your app service you don't need to expose the mysql port. If the containers runs in the same network they can reach all ports inside this network.
If my example doesn't works try this
run the docker compose and next go into the app container using
docker container exec -it CONTAINER_NAME bash
Install ping in order to test connection and then run ping mysql.
I am quite new to docker but am trying to use docker compose to run automation tests against my application.
I have managed to get docker compose to run my application and run my automation tests, however, at the moment my application is running on localhost when I need it to run against a specific domain example.com.
From research into docker it seems you should be able to hit the application on the hostname by setting it within links, but I still don't seem to be able to.
Below is the code for my docker compose files...
docker-compose.yml
abc:
build: ./
command: run container-dev
ports:
- "443:443"
expose:
- "443"
docker-compose.automation.yml
tests:
build: test/integration/
dockerfile: DockerfileUIAuto
command: sh -c "Xvfb :1 -screen 0 1024x768x16 &>xvfb.log && sleep 20 && DISPLAY=:1.0 && ENVIRONMENT=qa BASE_URL=https://example.com npm run automation"
links:
- abc:example.com
volumes:
- /tmp:/tmp/
and am using the following command to run...
docker-compose -p tests -f docker-compose.yml -f docker-compose.automation.yml up --build
Is there something I'm missing to map example.com to localhost?
If the two containers are on the same Docker internal network, Docker will provide a DNS service where one can talk to the other by just its container name. As you show this with two separate docker-compose.yml files it's a little tricky, because Docker Compose wants to isolate each file into its own separate mini-Docker world.
The first step is to explicitly declare a network in the "first" docker-compose.yml file. By default Docker Compose will automatically create a network for you, but you need to control its name so that you can refer to it from elsewhere. This means you need a top-level networks: block, and also to attach the container to the network.
version: '3'
networks:
abc:
name: abc
services:
abc:
build: ./
command: run container-dev
ports:
- "443:443"
networks:
abc:
aliases:
- example.com
Then in your test file, you can import that as an external network.
version: 3
networks:
abc:
external: true
name: abc
services:
tests:
build: test/integration/
dockerfile: DockerfileUIAuto
command: sh -c "Xvfb :1 -screen 0 1024x768x16 &>xvfb.log && sleep 20 && npm run automation"
environment:
DISPLAY: "1.0"
ENVIRONMENT: qa
BASE_URL: "https://example.com"
networks:
- abc
Given the complexity of what you're showing for the "test" container, I would strongly consider running it not in Docker, or else writing a shell script that launches the X server, checks that it actually started, and then runs the test. The docker-compose.yml file isn't the only tool you have here.
I'm trying to dockerize my Laravel app. In my local machine i have Mysql and MongoDB.
I have a script that run mysql restore and mongorestore to restore a production db.
In local environment i don't have problems because mysql and mongodb are installed locally.
Now, i created a docker-compose.yml with the build instructions:
version: '3'
services:
app:
build:
context: .
dockerfile: .docker/Dockerfile.dev
image: laravel-docker
ports:
- 8080:80
volumes:
- /srv/app/vendor
- .:/srv/app
links:
- mariadb
- redis
redis:
image: redis:latest
mariadb:
image: mariadb:latest
environment:
- MYSQL_ALLOW_EMPTY_PASSWORD=yes
- MYSQL_USER=root
mongo:
image: mongo
All works fine but i need to run mysql command from app container (that should call mysql command in mariadb container)
Just install a MySQL client in your app container.
Add something like this in your Dockerfile :
apt-get install mysql-client
Then you should be able to connect to your MySql server from your app container using the service name if they belong to the same network :
mysql -u USERNAME -p PASSWORD -h mariadb