I have the follow docker compose file:
version: '3.9'
services:
db-production:
container_name: mysql-production
image: mysql:latest
restart: always
environment:
MYSQL_HOST: localhost
MYSQL_DATABASE: dota2learning-db
MYSQL_ROOT_PASSWORD: toor
ports:
- "3306:3306"
volumes:
- ./data/db-production:/home/db-production
db-testing:
container_name: mysql-testing
image: mysql:latest
restart: always
environment:
MYSQL_HOST: localhost
MYSQL_ROOT_PASSWORD: toor
ports:
- "3307:3306"
volumes:
- ./data/db-testing:/home/db-testing
volumes:
data:
I also have a sql script to dump my database. The problem that's docker take a long time to start mysql and the script don't work.
I tried add the follow command on docker compose file:
command: mysql --user=root --password=toor dota2learning-db < /home/db-production/dumb-db-production.sql
This command does not work because it tries to run before the mysql server is working.
I know because as soon as I created the container I got into it and tried to log into mysql and it wasn't available yet:
sudo docker exec -it mysql-production bash
on container:
mysql --user=root --password=toor
ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)
I also tried to start MySQL manually:
root#4c91b5407561:/# mysqld start
2022-06-20T14:56:18.448123Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.29) starting as process 97
2022-06-20T14:56:18.451281Z 0 [ERROR] [MY-010123] [Server] Fatal error: Please read "Security" section of the manual to find out how to run mysqld as root!
2022-06-20T14:56:18.451346Z 0 [ERROR] [MY-010119] [Server] Aborting
2022-06-20T14:56:18.451514Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.29) MySQL Community Server - GPL.
That's add the follow command on docker compose don't work:
command: mysqld start
NOTE:
But I know that if I wait 1 or 2 minutes mysql will be available to run the script though, I want to run this script automatically and not manually.
When I add the commands on docker compose the docker container keeps restarting forever, because it keeps trying to execute the commands and mysql is not available yet.
Related
I'm using a MariaDB docker image and keep getting the Warning:
[Warning] Access denied for user 'root'#'localhost' (using password: NO)
This is my docker file used to create the image:
FROM mariadb:10.6.4
ENV MYSQL_ROOT_PASSWORD pw
ENV MARIADB_DATABASE db1
ENV MARIADB_USER user1
ENV MARIADB_PASSWORD user1pw
I start the container in docker-compose :
mariadb:
restart: always
image: mariadb_image
container_name: mariadb_container
build: topcat_mariadb/.
ports:
- 2306:3306
and I access the container from another container using:
mysql+pymysql://user1:user1pw#mariadb_container:3306/db1
Most of the solutions google found was running some kind of SQL statement, but as I'm running a docker container, I need an ephemeral solution.
Anyone know how to fix this in docker?
-----EDIT-----
I had forgotten to add the health check section to the dockercompose file:
mariadb:
restart: always
image: mariadb_image
container_name: mariadb_container
build: topcat_mariadb/.
ports:
- 2306:3306
healthcheck:
test: ["CMD-SHELL", 'mysqladmin ping']
interval: 10s
timeout: 2s
retries: 10
Update
Based on the update to your question, you're trying to run the mysqladmin ping command inside the container. mysqladmin is attempting to connect as the root user, but authenticating to your database server requires a password.
You can provide a password to mysqladmin by:
Using the -p command line option
Using the MYSQL_PWD environment variable
Creating a credentials file
If we move the root password out of your image, and instead set it at runtime, we can write your docker-compose.yml file like this:
version: "3"
services:
mariadb:
restart: always
image: mariadb_image
container_name: mariadb_container
build: topcat_mariadb/.
environment:
- "MYSQL_ROOT_PASSWORD=$MYSQL_ROOT_PASSWORD"
- "MYSQL_PWD=$MYSQL_ROOT_PASSWORD"
healthcheck:
test: ["CMD-SHELL", 'mysqladmin ping']
interval: 10s
timeout: 2s
retries: 10
And then in our .env file we can set:
MYSQL_ROOT_PASSWORD=pw1
Now after the container starts up we see that the container is
healthy:
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
c1c9c9f787e6 mariadb_image "docker-entrypoint.s…" 28 seconds ago Up 27 seconds (healthy) 3306/tcp mariadb_container
As a side note, it's not clear from this example why you're bothering to build a custom image: it's better to set the environment variables at runtime, rather than creating an image with "baked-in" credentials.
Previous answer
I can't reproduce your problem when using the mysql client or Python code. Given the following docker-compose.yml:
version: "3"
services:
mariadb:
restart: always
image: mariadb_image
container_name: mariadb_container
build: topcat_mariadb/.
shell:
image: mariadb:10.6.4
command: sleep inf
(The directory topcat_mariadb contains the Dockerfile from your
question.)
If I exec into the shell container:
docker-compose exec shell bash
And run mysql like this:
mysql -h mariadb_container -u user1 -p db1
It works just fine:
root#4fad8e8435df:/# mysql -h mariadb_container -u user1 -p db1
Enter password:
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MariaDB connection id is 6
Server version: 10.6.4-MariaDB-1:10.6.4+maria~focal mariadb.org binary distribution
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MariaDB [db1]>
It looks like you may be using sqlalchemy. If I add a Python container
to the mix:
version: "3"
services:
mariadb:
restart: always
image: mariadb_image
container_name: mariadb_container
build: topcat_mariadb/.
shell:
image: mariadb:10.6.4
command: sleep inf
python:
image: python:3.9
command: sleep inf
And then run the following Python code in the python container:
>>> import sqlalchemy
>>> e = sqlalchemy.engine.create_engine('mysql+pymysql://user1:user1pw#mariadb_container:3306/db1')
>>> res = e.execute('select 1')
>>> res.fetchall()
[(1,)]
It also seems to work without a problem.
I am trying to run the docker-compose command on Jenkins slave but it fails while running the command pytest tests/integration.
The command run integration tests with backend as postgres.
Dockerfile is
version: "3.4"
services:
test:
build:
context: ../..
dockerfile: Dockerfile
depends_on:
- postgres_db
environment:
PG_UNITTEST_DB: "postgresql://testuser:testpassword#postgres_db/testdb"
command: pytest tests/integration
postgres_db:
image: postgis/postgis
ports:
- "5432:5432"
environment:
POSTGRES_PASSWORD: testpassword
POSTGRES_USER: testuser
POSTGRES_DB: testdb
And the error I am getting is
psycopg2.OperationalError: could not connect to server: Connection refused
Is the server running on host "postgres_db" (172.19.0.2) and accepting
TCP/IP connections on port 5432?
I tried exposing port 5432 in docker-compose file in postgres_db section but didn't help. The same code works fine locally. The command I run is
docker-compose -f tests/integration/docker-compose.yml up --build --exit-code-from test
Postgres takes a moment to start up before it can start servicing requests. It is likely that your code in the test container is attempting to connect before Postgres is ready.
Your best option is probably to add some retry logic to your integration test. E.g., add something to yoursetup method that loops until it is able to establish a successful database connection.
You need to define the order of starting the service so that your postgres container is up before the test container. For detailed info, you can refer to the docs: https://docs.docker.com/compose/startup-order/
I am trying to docker-compose up -d the docker-compose.yml below.
The first container is a mariadb with an sql script to init the database schema.
The second container is a nestjs application.
When I run docker-compose up -d the first container is completely ignored, and it starts trying to boot the second container up, which eventually leads to the startup of the nestjs app failing due to the missing database connection.
I'm failing to understand why docker-compose goes straight to initializing the server container, since if I manually start server-db container it does start up.
The actual logs from the docker-compose up -d
Deploying 'Compose: integration'...
"C:\Program Files\Docker\Docker\resources\bin\docker-compose.exe" -f D:\dev\projects\integration\docker-compose.yml up -d --build
Building server
I am on Docker Engine v20.10.5 for windows.
version: '3.3'
services:
server-db:
image: mariadb:10.3
restart: always
environment:
MYSQL_DATABASE: 'serverdb'
MYSQL_USER: 'serverdb'
MYSQL_PASSWORD: 'serverdb'
MYSQL_ROOT_PASSWORD: 'root'
command: --init-file /data/application/init.sql
ports:
- '3306:3306'
volumes:
- ./serverdb/database/init.sql:/data/application/init.sql
server:
build: ./server
ports:
- '8080:3000'
depends_on:
- server-db
I want to build a domjudge server with mriadb, phpmyadmin, judgehost in the docker base on Debian9,
I've install the docker and docker compose
here is the docker-compose.yml code below.
and I use docker-compose up -d and there are some WARNING and ERROR pop out.
here is the entire docker-compose.yml file code
http://codepad.org/souBFdFz
WARNING and ERROR messages:
WARNING: some networks were defined but are not used by any service: phpmyadmin, dj-judgedameons_1, dj-judgedameons_2
ERROR: dor domjudge_dj-judgedameons_2_1 Cannot start service dj-judgedameons_1 : OCI runtime create failed: container_linux.go:345: starting container process caused "process_linux.go:311:getting Starting domjudge_dj-judgedameons_1_1
...and a lots of error messages that I cant even read(binary code or address i think)
Please help me fix it or if there is a easy way to set up domjudge server with mariadb, phpmyadmin, judgehost
THANKS!
Update
I've tried this file several times and it has a drifferent result but it still can't connect to the server (domjudge & phpmyadmin).
here is the message
https://i.stack.imgur.com/qDcDd.jpg
Unfortunately what you want to do is not really possible because of how the application is built: containers need to wait for each other and some of them need manual actions.
However, this is a sequence of actions that works and will bring all containers up and running.
NOTE: I removed the networks declarations because they don't add any value.
version: '3'
services:
dj-mariadb:
image: mariadb
environment:
- MYSQL_ROOT_PASSWORD=rootpw
- MYSQL_DATABASE=domjudge
- MYSQL_USER=domjudge
- MYSQL_PASSWORD=djpw
command:
--max-connections=1000
dj-domserver:
image: domjudge/domserver:latest
volumes:
- /sys/fs/cgroup:/sys/fs/cgroup:ro
environment:
- CONTAINER_TIMEZONE=Asia/Taipei
- MYSQL_ROOT_PASSWORD=rootpw
- MYSQL_DATABASE=domjudge
- MYSQL_USER=domjudge
- MYSQL_PASSWORD=djpw
ports:
- 9090:80
links:
- dj-mariadb:mariadb
dj-judgehost:
image: domjudge/judgehost:latest
privileged: true
hostname: judgedaemon-0
volumes:
- /sys/fs/cgroup:/sys/fs/cgroup:ro
environment:
- DAEMON_ID=0
- JUDGEDAEMON_PASSWORD=domjudge
links:
- dj-domserver:domserver
dj-judgehost_1:
image: domjudge/judgehost:latest
privileged: true
hostname: judgedaemon-1
volumes:
- /sys/fs/cgroup:/sys/fs/cgroup:ro
environment:
- DAEMON_ID=1
- JUDGEDAEMON_PASSWORD=domjudge
links:
- dj-domserver:domserver
dj-judgehost_2:
image: domjudge/judgehost:latest
privileged: true
hostname: judgedaemon-2
volumes:
- /sys/fs/cgroup:/sys/fs/cgroup:ro
environment:
- DAEMON_ID=2
- JUDGEDAEMON_PASSWORD=domjudge
links:
- dj-domserver:domserver
phpmyadmin:
image: phpmyadmin/phpmyadmin
container_name: myadmin
ports:
- 8888:80
environment:
- PMA_ARBITRARY=1
- PMA_HOST=dj-mariadb
links:
- dj-mariadb:db
Start the database and wait for it to initialize (otherwise the server will exit because it cannot find the schema it needs)
docker-compose up -d dj-mariadb
Start the server:
docker-compose up -d dj-domserver
Get the admin password from the logs:
docker-compose logs dj-domserver
Look for the line saying: Initial admin password is .... and save the password.
Set the judgehost password in the web interface: open http://localhost:9090 and login with user admin and the password you saved from the previous step. Go to Users and click on judgehost user. In there change the password to domjudge (according to what you set in the docker-compose.yml for JUDGEDAEMON_PASSWORD. Save the data.
Start the rest of the containers:
docker-compose up -d
Verify that all containers are up and running:
docker-compose ps
Output should look similar to this:
Name Command State Ports
---------------------------------------------------------------------------------------------------
domjudge_dj-domserver_1 /scripts/start.sh Up 0.0.0.0:9090->80/tcp
domjudge_dj-judgehost_1 /scripts/start.sh Up
domjudge_dj-judgehost_1_1 /scripts/start.sh Up
domjudge_dj-judgehost_2_1 /scripts/start.sh Up
domjudge_dj-mariadb_1 docker-entrypoint.sh --max ... Up 3306/tcp
myadmin /run.sh supervisord -n -j ... Up 0.0.0.0:8888->80/tcp, 9000/tcp
I have created a docker-compose file it has two services with Go and Mysql. It creates container for go and mysql. Now i am running code which try to connect to mysql database which is running as a docker container. but i get error.
docker-compose.yml
version: "2"
services:
app:
container_name: golang
restart: always
build: .
ports:
- "49160:8800"
links:
- "mysql"
depends_on:
- "mysql"
mysql:
image: mysql
container_name: mysql
volumes:
- dbdata:/var/lib/mysql
restart: always
environment:
- MYSQL_ROOT_PASSWORD=root
- MYSQL_DATABASE=testDB
- MYSQL_USER=root
- MYSQL_PASSWORD=root
ports:
- "3307:3306"
volumes:
dbdata:
Error while connecting to mysql database
golang | 2019/02/28 11:33:05 dial tcp 127.0.0.1:3306: connect: connection refused
golang | 2019/02/28 11:33:05 http: panic serving 172.24.0.1:49066: dial tcp 127.0.0.1:3306: connect: connection refused
golang | goroutine 19 [running]:
Connection with MySql Database
func DB() *gorm.DB {
db, err := gorm.Open("mysql", "root:root#tcp(mysql:3306)/testDB?charset=utf8&parseTime=True&loc=Local")
if err != nil {
log.Panic(err)
}
log.Println("Connection Established")
return db
}
EDIT:Updated docker file
FROM golang:latest
RUN go get -u github.com/gorilla/mux
RUN go get -u github.com/jinzhu/gorm
RUN go get -u github.com/go-sql-driver/mysql
COPY ./wait-for-it.sh .
RUN chmod +x /wait-for-it.sh
WORKDIR /go/src/app
ADD . src
EXPOSE 8800
CMD ["go", "run", "src/main.go"]
I am using gorm package which lets me connet to the database
depends_on is not a verification that MySQL is actually ready to receive connections. It will start the second container once the database container is running regardless it was ready for connections or not which could lead to such an issue with your application as it expects the database to be ready which might not be true.
Quoted from the documentation:
depends_on does not wait for db and redis to be “ready” before starting web - only until they have been started.
There are many tools/scripts that can be used to solve this issue like wait-for which sh compatible in case your image based on Alpine for example (You can use wait-for-it if you have bash in your image)
All you have to do is to add the script to your image through Dockerfile then use this command in docker-compose.yml for the service that you want to make it wait for the database.
What comes after -- is the command that you would normally use to start your application
version: "2"
services:
app:
container_name: golang
...
command: ["./wait-for", "mysql:3306", "--", "go", "run", "myapplication"]
links:
- "mysql"
depends_on:
- "mysql"
mysql:
image: mysql
...
I have removed some parts from the docker-compose for easier readability.
Modify this part go run myapplication with the CMD of your golang image.
See Controlling startup order for more on this problem and strategies for solving it.
Another issue that will rise after you solve the connection issue will be as the following:
Setting MYSQL_USER with root value will cause a failure in MySQL with this error message:
ERROR 1396 (HY000) at line 1: Operation CREATE USER failed for 'root'#'%'
This is because this user already exist in the database and it tries to create another. if you need to use the root user itself you can use only this variable MYSQL_ROOT_PASSWORD or change the value of MYSQL_USER so you can securely use it in your application instead of the root user.
Update: In case you are getting not found and the path was correct, you might need to write the command as below:
command: sh -c "./wait-for mysql:3306 -- go run myapplication"
First, if you are using latest version of docker compose you don't need the link argument in you app service. I quote the docker compose documentation Warning: The --link flag is a legacy feature of Docker. It may eventually be removed. Unless you absolutely need to continue using it, https://docs.docker.com/compose/compose-file/#links
I think the solution is to use the networks argument. This create a docker network and add each service to it.
Try this
version: "2"
services:
app:
container_name: golang
restart: always
build: .
ports:
- "49160:8800"
networks:
- my_network
depends_on:
- "mysql"
mysql:
image: mysql
container_name: mysql
volumes:
- dbdata:/var/lib/mysql
restart: always
networks:
- my_network
environment:
- MYSQL_ROOT_PASSWORD=root
- MYSQL_DATABASE=testDB
- MYSQL_USER=root
- MYSQL_PASSWORD=root
ports:
- "3307:3306"
volumes:
dbdata:
networks:
my_network:
driver: bridge
By the way, if you only connect to Mysql from your app service you don't need to expose the mysql port. If the containers runs in the same network they can reach all ports inside this network.
If my example doesn't works try this
run the docker compose and next go into the app container using
docker container exec -it CONTAINER_NAME bash
Install ping in order to test connection and then run ping mysql.