Wordpress can't see linked mysql container on IBM Bluemix with Docker compose - docker

I have a simple docker-compose.yml (wp image is based on ibmjstart/wp-bluemix-container, db image is mariadb)
db:
image: registry.eu-gb.bluemix.net/foo/db
environment:
MYSQL_ROOT_PASSWORD: examplepass
ports:
- 3306:3306
volumes:
- /var/lib/mysql
wp:
image: registry.eu-gb.bluemix.net/foo/wp
links:
- db:mysql
ports:
- 80:80
after executing docker compose up -d I get
error: missing WORDPRESS_DB_HOST and MYSQL_PORT_3306_TCP environment variables
Did you forget to --link some_mysql_container:mysql or set an external db
with -e WORDPRESS_DB_HOST=hostname:port?
As you can see, the db container is linked.
When I do the same without docker-compose, using
$ cf ic run -v mysql-vol:/var/lib/mysql --name wpdb -d registry.eu-gb.bluemix.net/foo/db
$ cf ic run -e MYSQL_ROOT_PASSWORD=my-secret-pw -v web-files:/var/www/html/ --link wpdb:mysql -d registry.eu-gb.bluemix.net/foo/wp
Everything works well.
I do export docker variables after cf ic login
More info:
root#vps:~/test/compose# docker-compose --version
docker-compose version 1.7.0, build 0d7bf73
root#vps:~/test/compose# docker --version
Docker version 1.10.3, build 20f81dd
root#vps:~/test/compose# cf --version
cf version 6.15.0+fa1bfe2-2016-01-13
root#vps:~/test/compose# cf ic --version
Docker version 1.10.3, build 20f81dd
UPDATE: As I understand, this problem is caused by the naming:
This docker-compose.yml throws an error
db:
image: registry.eu-gb.bluemix.net/foo/db
environment:
MYSQL_ROOT_PASSWORD: examplepass
container_name:
wpdb
ports:
- 3306:3306
volumes:
- /var/lib/mysql
wp:
image: registry.eu-gb.bluemix.net/foo/wp
links:
- wpdb:mysql
ports:
- 80:80
ERROR: Service "wp" has a link to service "wpdb" which does not exist.
However, if you name the service and container the same, the syntax is ok.
db:
image: registry.eu-gb.bluemix.net/foo/db
environment:
MYSQL_ROOT_PASSWORD: examplepass
container_name:
db
ports:
- 3306:3306
volumes:
- /var/lib/mysql
wp:
image: registry.eu-gb.bluemix.net/foo/wp
links:
- db:mysql
ports:
- 80:80
Although the syntax is OK and the container is linked, the wordpress container logs this
Warning: mysqli::mysqli(): (HY000/2002): php_network_getaddresses: getaddrinfo failed: Name or service not known in - on line 10
MySQL Connection Error: (2002) php_network_getaddresses: getaddrinfo failed: Name or service not known
Is this a bug in Bluemix? Looks like a /etc/hosts/ related problem
Sorry for a long post :)

#bartimar Yes, the problem is related to the /etc/hosts file. It needs to have an entry for the db container, but it is not creating it.
I can recreate your problem in the prod-lon02-vizio1 environment, but it works fine in the prod-lon02-kraken1 environment.
My recommendation if for you to manually migrate to the prod-lon02-kraken1 environment to use docker-compose.yml with IBM containers. All environments will be automatically migrated on May 25th anyway.
To migrate simply run the following command:
$ cf ic reprovision
Please note that your images are migrated to new environment, but all your running containers are deleted and you will have to recreate them in the new environment. So use this option with caution.
For more details check the link below:
https://developer.ibm.com/bluemix/2016/03/24/new-deployment-architecture-for-containers/?linkId=22660520

Related

Test the connection between docker containers within docker-composed environment

We are using docker-compose to set up the services for our app:
version: "3"
services:
db:
container_name: db
image: postgres:11.1
environment:
POSTGRES_USER: xxx
POSTGRES_PASSWORD: xxx
POSTGRES_DB: xxx
PGPASSWORD: xxx
volumes:
- pgdata:/var/lib/postgresql/data
- ./data/dbdump:/dbdump
networks:
- zenet
ports:
- "5432:5432"
# The React web application
web:
container_name: web
build:
context: .
dockerfile: devenv/web/Dockerfile
volumes:
- ./src/client-app:/usr/local/abc
- /usr/local/abc/node_modules
networks:
- zenet
ports:
- "3000:3000"
command: npm run startindocker
# The Django Rest Framework API
api:
container_name: api
build:
context: .
dockerfile: devenv/api/Dockerfile
environment:
DJANGO_SETTINGS_MODULE: abc.settings.dev
PYTHONSTARTUP: /root/pythonstartup.sh
PYTHONIOENCODING: UTF-8
volumes:
- .:/usr/local/borrow-a-boat
- ./devenv/api/pythonrc.py:/root/pythonstartup.sh
networks:
- zenet
depends_on:
- "db"
ports:
- "9000:9000"
command:
python3 /usr/local/borrow-a-boat/src/django/abc/manage.py runserver 0.0.0.0:9000
tty: true
volumes:
pgdata:
customboatdata:
networks:
zenet:
(sensitive info has been replaced)
My colleagues have the setup running fine. I setup the app & the volumes & containers are up & running. I can hit the service api at port 9000 fine from browser & confirm that the db is populated. However, my web service is unable to get the data from the api. How can I confirm that the above assertion is correct & that the web really cannot communicate with the api service.
And how can I fix this & get the web to receive the data from api. Apologies for the newbie question.
EDIT:
When I run ping api from within the web container using docker exec -it [containerID] /bin/sh, I am recieving a response in the form of :
64 bytes from 172.18.0.4: seq=139 ttl=64 time=0.084 ms
So, clearly, my assertion is incorrect. Why is web service unable to get a response from api service. When I load the web app in browser, I do not get any log display in the terminal of the api being hit.
EDIT-2 :
As per #runwuf question & my response, clearly, the 'web' is able to communicate with the 'api' service. So, something else is wrong. Here are the steps, we follow to setup the stack on our systems. I use a Linux Mint 19.2 OS, while the team uses Macs. The commands are:
docker kill $(docker_container_names)
docker rm -v $(docker_container_names)
docker volume rm abc_pgdata
docker image rm abc_api
docker image rm abc_web
docker-compose build
docker-compose up -d db api web
ssh abc#abc.com 'pg_dump abc | gzip' | gunzip | docker-compose run --rm db psql --host db --username abc
docker-compose run --rm db psql --host db --username abc -c "update core_photo set image_base = 'sample.jpg'"
docker-compose run --rm db psql --host db --username abc -c "update core_experienceimage set image_base = 'sample.jpg'"
In the end, it was a case of env variable not accessible within web service. All it took was to read the console logs in the browser which showed the undefined variable.
The lesson for me is to when it comes to problem solving, no matter how new the technology, don't forget to use the tools you are familiar with.

Docker-compose setting problem about Domjudge server

I want to build a domjudge server with mriadb, phpmyadmin, judgehost in the docker base on Debian9,
I've install the docker and docker compose
here is the docker-compose.yml code below.
and I use docker-compose up -d and there are some WARNING and ERROR pop out.
here is the entire docker-compose.yml file code
http://codepad.org/souBFdFz
WARNING and ERROR messages:
WARNING: some networks were defined but are not used by any service: phpmyadmin, dj-judgedameons_1, dj-judgedameons_2
ERROR: dor domjudge_dj-judgedameons_2_1 Cannot start service dj-judgedameons_1 : OCI runtime create failed: container_linux.go:345: starting container process caused "process_linux.go:311:getting Starting domjudge_dj-judgedameons_1_1
...and a lots of error messages that I cant even read(binary code or address i think)
Please help me fix it or if there is a easy way to set up domjudge server with mariadb, phpmyadmin, judgehost
THANKS!
Update
I've tried this file several times and it has a drifferent result but it still can't connect to the server (domjudge & phpmyadmin).
here is the message
https://i.stack.imgur.com/qDcDd.jpg
Unfortunately what you want to do is not really possible because of how the application is built: containers need to wait for each other and some of them need manual actions.
However, this is a sequence of actions that works and will bring all containers up and running.
NOTE: I removed the networks declarations because they don't add any value.
version: '3'
services:
dj-mariadb:
image: mariadb
environment:
- MYSQL_ROOT_PASSWORD=rootpw
- MYSQL_DATABASE=domjudge
- MYSQL_USER=domjudge
- MYSQL_PASSWORD=djpw
command:
--max-connections=1000
dj-domserver:
image: domjudge/domserver:latest
volumes:
- /sys/fs/cgroup:/sys/fs/cgroup:ro
environment:
- CONTAINER_TIMEZONE=Asia/Taipei
- MYSQL_ROOT_PASSWORD=rootpw
- MYSQL_DATABASE=domjudge
- MYSQL_USER=domjudge
- MYSQL_PASSWORD=djpw
ports:
- 9090:80
links:
- dj-mariadb:mariadb
dj-judgehost:
image: domjudge/judgehost:latest
privileged: true
hostname: judgedaemon-0
volumes:
- /sys/fs/cgroup:/sys/fs/cgroup:ro
environment:
- DAEMON_ID=0
- JUDGEDAEMON_PASSWORD=domjudge
links:
- dj-domserver:domserver
dj-judgehost_1:
image: domjudge/judgehost:latest
privileged: true
hostname: judgedaemon-1
volumes:
- /sys/fs/cgroup:/sys/fs/cgroup:ro
environment:
- DAEMON_ID=1
- JUDGEDAEMON_PASSWORD=domjudge
links:
- dj-domserver:domserver
dj-judgehost_2:
image: domjudge/judgehost:latest
privileged: true
hostname: judgedaemon-2
volumes:
- /sys/fs/cgroup:/sys/fs/cgroup:ro
environment:
- DAEMON_ID=2
- JUDGEDAEMON_PASSWORD=domjudge
links:
- dj-domserver:domserver
phpmyadmin:
image: phpmyadmin/phpmyadmin
container_name: myadmin
ports:
- 8888:80
environment:
- PMA_ARBITRARY=1
- PMA_HOST=dj-mariadb
links:
- dj-mariadb:db
Start the database and wait for it to initialize (otherwise the server will exit because it cannot find the schema it needs)
docker-compose up -d dj-mariadb
Start the server:
docker-compose up -d dj-domserver
Get the admin password from the logs:
docker-compose logs dj-domserver
Look for the line saying: Initial admin password is .... and save the password.
Set the judgehost password in the web interface: open http://localhost:9090 and login with user admin and the password you saved from the previous step. Go to Users and click on judgehost user. In there change the password to domjudge (according to what you set in the docker-compose.yml for JUDGEDAEMON_PASSWORD. Save the data.
Start the rest of the containers:
docker-compose up -d
Verify that all containers are up and running:
docker-compose ps
Output should look similar to this:
Name Command State Ports
---------------------------------------------------------------------------------------------------
domjudge_dj-domserver_1 /scripts/start.sh Up 0.0.0.0:9090->80/tcp
domjudge_dj-judgehost_1 /scripts/start.sh Up
domjudge_dj-judgehost_1_1 /scripts/start.sh Up
domjudge_dj-judgehost_2_1 /scripts/start.sh Up
domjudge_dj-mariadb_1 docker-entrypoint.sh --max ... Up 3306/tcp
myadmin /run.sh supervisord -n -j ... Up 0.0.0.0:8888->80/tcp, 9000/tcp

What is the docker run -e equivalent in docker-compose

I can't get environmental variables in a docker-compose file written directly in it to work. A similar configuration with the command line work just fine like this:
docker run --name container_name -d --network=my-net --mount type=bind,src=/Users/t2wu/Documents/Work/Dodo/Intron-Exon_expression/DockerCompose/intronexon_db/mnt_mysql,dst=/var/lib/mysql -e MYSQL_DATABASE=db_name -e MYSQL_USER=username -e MYSQL_PASSWORD=passwd mysql/mysql-server:8.0.13
This is an MySQL instance which sets three environmental variables: MYSQL_DATABASE, MYSQL_USER and MYSQL_PASSWORD. I'm later able to launch bash into it docker exec -it container_name bash and launch the client mysql -u username -p and connects just fine.
However when I write it in a docker-compose.yml:
version: "3.7"
services:
intronexon_db:
image: mysql/mysql-server:8.0.13
volumes:
- type: bind
source: ./intronexon_db/mnt_mysql
target: /var/lib/mysql
environment:
MYSQL_DATABASE: db_name
MYSQL_USER: username
MYSQL_PASSWORD: passwd
networks:
- my-net
networks:
my-net:
driver: bridge
Then when I use the mysql client, it's as if the user doesn't exist. How do I set it so that it is equivalent to the -e flag during docker run?
EDIT
docker-compose --version shows docker-compose version 1.24.1, build 4667896b
EDIT 2
The environmental flag did work. But I run into problem because:
Part of the problem was that it takes MySQL sometime to get the database, username and password setup ready. And I was checking it way too early.
I need to specify localhost for some reason: mysql --host=localhost -u user -p. Specifying 127.0.0.1 will not work.
For some unknown reason the example stack.yml from the official docker image did not have to specify --host when the adminer container is run. If I wipe out the adminer, then --host flag needs to be given.
Sometimes MySQL daemon will stop. It might has to do with my mount target /var/lib/mysql but I'm not certain.
command: --default-authentication-plugin=mysql_native_password is actually significant. I don't know why when I did docker run I didn't need to do anything about this.
docker-compose accept both types of ENVs either an array or a dictionary, better to double or try both approaches.
environment
Add environment variables. You can use either an array or a
dictionary. Any boolean values; true, false, yes no, need to be
enclosed in quotes to ensure they are not converted to True or False
by the YML parser.
Environment variables with only a key are resolved to their values on
the machine Compose is running on, which can be helpful for secret or
host-specific values.
environment:
RACK_ENV: development
SHOW: 'true'
SESSION_SECRET:
or
environment:
- RACK_ENV=development
- SHOW=true
- SESSION_SECRET
Might be something with docker-compose version as it working fine with 3.1. as the offical image suggested, so Better to try offical image docker-compose.yml
version: '3.1'
services:
db:
image: mysql
command: --default-authentication-plugin=mysql_native_password
restart: always
environment:
MYSQL_ROOT_PASSWORD: example
Also, better to debug such cases where everything seems correct but some minor syntax is missing. you can test it before working with DB.
version: "3.7"
services:
intronexon_db:
image: alpine
environment:
MYSQL_DATABASE: myDb
command: tail -f /dev/null
run docker-compose up
Now test and debug in testing enviroment.
docker exec -it composeenv_intronexon_db_1 ash -c "printenv"
the environment params in your yml need the - in front of them could be the likely culprit
version: "3.7"
services:
intronexon_db:
image: mysql/mysql-server:8.0.13
volumes:
- ./intronexon_db/mnt_mysql:/var/lib/mysql
environment:
- MYSQL_DATABASE: db_name
- MYSQL_USER: username
- MYSQL_PASSWORD: passwd
networks:
- my-net
networks:
my-net:
driver: bridge

Unable to connect mysql from docker container?

I have created a docker-compose file it has two services with Go and Mysql. It creates container for go and mysql. Now i am running code which try to connect to mysql database which is running as a docker container. but i get error.
docker-compose.yml
version: "2"
services:
app:
container_name: golang
restart: always
build: .
ports:
- "49160:8800"
links:
- "mysql"
depends_on:
- "mysql"
mysql:
image: mysql
container_name: mysql
volumes:
- dbdata:/var/lib/mysql
restart: always
environment:
- MYSQL_ROOT_PASSWORD=root
- MYSQL_DATABASE=testDB
- MYSQL_USER=root
- MYSQL_PASSWORD=root
ports:
- "3307:3306"
volumes:
dbdata:
Error while connecting to mysql database
golang | 2019/02/28 11:33:05 dial tcp 127.0.0.1:3306: connect: connection refused
golang | 2019/02/28 11:33:05 http: panic serving 172.24.0.1:49066: dial tcp 127.0.0.1:3306: connect: connection refused
golang | goroutine 19 [running]:
Connection with MySql Database
func DB() *gorm.DB {
db, err := gorm.Open("mysql", "root:root#tcp(mysql:3306)/testDB?charset=utf8&parseTime=True&loc=Local")
if err != nil {
log.Panic(err)
}
log.Println("Connection Established")
return db
}
EDIT:Updated docker file
FROM golang:latest
RUN go get -u github.com/gorilla/mux
RUN go get -u github.com/jinzhu/gorm
RUN go get -u github.com/go-sql-driver/mysql
COPY ./wait-for-it.sh .
RUN chmod +x /wait-for-it.sh
WORKDIR /go/src/app
ADD . src
EXPOSE 8800
CMD ["go", "run", "src/main.go"]
I am using gorm package which lets me connet to the database
depends_on is not a verification that MySQL is actually ready to receive connections. It will start the second container once the database container is running regardless it was ready for connections or not which could lead to such an issue with your application as it expects the database to be ready which might not be true.
Quoted from the documentation:
depends_on does not wait for db and redis to be “ready” before starting web - only until they have been started.
There are many tools/scripts that can be used to solve this issue like wait-for which sh compatible in case your image based on Alpine for example (You can use wait-for-it if you have bash in your image)
All you have to do is to add the script to your image through Dockerfile then use this command in docker-compose.yml for the service that you want to make it wait for the database.
What comes after -- is the command that you would normally use to start your application
version: "2"
services:
app:
container_name: golang
...
command: ["./wait-for", "mysql:3306", "--", "go", "run", "myapplication"]
links:
- "mysql"
depends_on:
- "mysql"
mysql:
image: mysql
...
I have removed some parts from the docker-compose for easier readability.
Modify this part go run myapplication with the CMD of your golang image.
See Controlling startup order for more on this problem and strategies for solving it.
Another issue that will rise after you solve the connection issue will be as the following:
Setting MYSQL_USER with root value will cause a failure in MySQL with this error message:
ERROR 1396 (HY000) at line 1: Operation CREATE USER failed for 'root'#'%'
This is because this user already exist in the database and it tries to create another. if you need to use the root user itself you can use only this variable MYSQL_ROOT_PASSWORD or change the value of MYSQL_USER so you can securely use it in your application instead of the root user.
Update: In case you are getting not found and the path was correct, you might need to write the command as below:
command: sh -c "./wait-for mysql:3306 -- go run myapplication"
First, if you are using latest version of docker compose you don't need the link argument in you app service. I quote the docker compose documentation Warning: The --link flag is a legacy feature of Docker. It may eventually be removed. Unless you absolutely need to continue using it, https://docs.docker.com/compose/compose-file/#links
I think the solution is to use the networks argument. This create a docker network and add each service to it.
Try this
version: "2"
services:
app:
container_name: golang
restart: always
build: .
ports:
- "49160:8800"
networks:
- my_network
depends_on:
- "mysql"
mysql:
image: mysql
container_name: mysql
volumes:
- dbdata:/var/lib/mysql
restart: always
networks:
- my_network
environment:
- MYSQL_ROOT_PASSWORD=root
- MYSQL_DATABASE=testDB
- MYSQL_USER=root
- MYSQL_PASSWORD=root
ports:
- "3307:3306"
volumes:
dbdata:
networks:
my_network:
driver: bridge
By the way, if you only connect to Mysql from your app service you don't need to expose the mysql port. If the containers runs in the same network they can reach all ports inside this network.
If my example doesn't works try this
run the docker compose and next go into the app container using
docker container exec -it CONTAINER_NAME bash
Install ping in order to test connection and then run ping mysql.

docker-compose + external container

I have started a docker container with the following command
docker run --name mysql --restart always -p 3306:3306 -v /var/lib/mysql:/var/lib/mysql -e MYSQL_ROOT_PASSWORD=password -d mysql:5.7.14
and then would like to connect a wordpress site with the following docker-compose.yml file
version: '2'
services:
wordpress:
image: wordpress
external_links:
- mysql:mysql
ports:
- 80:80
environment:
WORDPRESS_DB_USER: root
WORDPRESS_DB_PASSWORD: password
volumes:
- /var/www/somesite.com:/var/www/html
But I keep getting the following error
Starting somesitecom_wordpress_1
Attaching to somesitecom_wordpress_1
wordpress_1 |
wordpress_1 | Warning: mysqli::mysqli(): (HY000/2002): Connection refused in - on line 19
wordpress_1 |
wordpress_1 | MySQL Connection Error: (2002) Connection refused
It seems like the external_links isn't working.
Any idea what I am doing wrong?
Your link is working, but you're on separate networks inside of Docker. From the docker-compose.yml docs:
Note: If you’re using the version 2 file format, the externally-created containers must be connected to at least one of the same networks as the service which is linking to them.
To solve this, you can create your own network:
docker network create dbnet
docker network connect dbnet mysql
Then configure your docker-compose.yml with:
version: '2'
networks:
dbnet:
external:
name: dbnet
services:
wordpress:
image: wordpress
ports:
- 80:80
environment:
WORDPRESS_DB_USER: root
WORDPRESS_DB_PASSWORD: password
volumes:
- /var/www/somesite.com:/var/www/html
networks:
- dbnet
Note with recent versions of Docker, you shouldn't need to link the containers, the DNS service should do the name resolution for you.

Resources