Connecting to MySQL from Flask Application using docker-compose - docker

I have an application using Flask and MySQL. The application does not connect to MySQL container from the Flask Application but it can be accessed using Sequel Pro with the same credentials.
Docker Compose File
version: '2'
services:
web:
build: flask-app
ports:
- "5000:5000"
volumes:
- .:/code
mysql:
build: mysql-server
environment:
MYSQL_DATABASE: test
MYSQL_ROOT_PASSWORD: root
MYSQL_ROOT_HOST: 0.0.0.0
MYSQL_USER: testing
MYSQL_PASSWORD: testing
ports:
- "3306:3306"
Docker file for MySQL
The docker file for MySQL will add schema from test.dump file.
FROM mysql/mysql-server
ADD test.sql /docker-entrypoint-initdb.d
Docker file for Flask
FROM python:latest
COPY . /app
WORKDIR /app
RUN pip install -r requirements.txt
ENTRYPOINT ["python"]
CMD ["app.py"]
Starting point app.py
from flask import Flask, request, jsonify, Response
import json
import mysql.connector
from flask_cors import CORS, cross_origin
app = Flask(__name__)
def getMysqlConnection():
return mysql.connector.connect(user='testing', host='0.0.0.0', port='3306', password='testing', database='test')
#app.route("/")
def hello():
return "Flask inside Docker!!"
#app.route('/api/getMonths', methods=['GET'])
#cross_origin() # allow all origins all methods.
def get_months():
db = getMysqlConnection()
print(db)
try:
sqlstr = "SELECT * from retail_table"
print(sqlstr)
cur = db.cursor()
cur.execute(sqlstr)
output_json = cur.fetchall()
except Exception as e:
print("Error in SQL:\n", e)
finally:
db.close()
return jsonify(results=output_json)
if __name__ == "__main__":
app.run(debug=True,host='0.0.0.0')
When I do a GET request on http://localhost:5000/ using REST Client I get a valid response.
A GET request on http://localhost:5000/api/getMonths gives error message:
mysql.connector.errors.InterfaceError: 2003: Can't connect to MySQL server on '0.0.0.0:3306' (111 Connection refused)
When the same credentials were used on Sequel Pro, I was able to access the database.
Please advice me on how to connect the MySQL container from the Flask Application. This is my first time suing Docker and do forgive me if this is a silly mistake from my part.

Change this
return mysql.connector.connect(user='testing', host='0.0.0.0', port='3306', password='testing', database='test')
to
return mysql.connector.connect(user='testing', host='mysql', port='3306', password='testing', database='test')
Your code is running inside the container and not on your host. So you need to provide it a address where it can reach within container network. For docker-compose each service is reachable using its name. So in your it is mysql as that is name you have used for the service

For others who encounter similar issue, if you are mapping different ports from host to container for the MySQL service, make sure that container that needs to connect to the MySQL service is using the port for the container not for the host.
Here is an example of a docker compose file. Here you can see that my application (which is running in a container) will be using port 3306 to connect to the MySQL service (which is also running in a container on port 3306). Anyone connecting to this MySQL service from the outside of the "backend" network which is basically anything that does not run in a container with the same network will need to use port 3308 to connect to this MySQL service.
version: "3"
services:
redis:
image: redis:alpine
command: redis-server --requirepass imroot
ports:
- "6379:6379"
networks:
- frontend
mysql:
image: mariadb:10.5
command: --default-authentication-plugin=mysql_native_password
ports:
- "3308:3306"
volumes:
- mysql-data:/var/lib/mysql/data
networks:
- backend
environment:
MYSQL_ROOT_PASSWORD: imroot
MYSQL_DATABASE: test_junkie_hq
MYSQL_HOST: 127.0.0.1
test-junkie-hq:
depends_on:
- mysql
- redis
image: test-junkie-hq:latest
ports:
- "80:5000"
networks:
- backend
- frontend
environment:
TJ_MYSQL_PASSWORD: imroot
TJ_MYSQL_HOST: mysql
TJ_MYSQL_DATABASE: test_junkie_hq
TJ_MYSQL_PORT: 3306
TJ_APPLICATION_PORT: 5000
TJ_APPLICATION_HOST: 0.0.0.0
networks:
backend:
frontend:
volumes:
mysql-data:

Related

Docker compose for nginx, streamlit, and mariadb

I am not exactly sure how to go about this. I have an instance in AWS Lightsail that has a static IP for which the IP department granted access to read from MariaDB database. I am using streamlit for my app and have stored my database credentials in a .env file. After, I have copied the file code and dockerized it (running the following command):
docker-compose up --build -d
It is built successfully, but when I use the static IP to look at the web page I get the following error:
OperationalError: (2003, "Can't connect to MySQL server on 'localhost' ([Errno 99] Cannot assign requested address)")
Is there something I have to do either in docker or with MariaDB? Thank you in advance.
File of docker-compose.yml:
version: '3'
services:
app:
container_name: app
restart: always
build: ./app
ports:
- "8501:8501"
command: streamlit run Main.py
database:
image: mariadb:latest
volumes:
- /data/mysql:/var/lib/mysql
restart: always
env_file: .env
nginx:
container_name: nginx
restart: always
build: ./nginx
ports:
- "80:80"
depends_on:
- app
- database
I am not sure how the streamlit app is connected with mariadb here.

Go backend to redis connection refused after docker compose up

I'm currently trying to introduce docker compose to my project. It includes a golang backend using the redis in-memory database.
version: "3.9"
services:
frontend:
...
backend:
build:
context: ./backend
ports:
- "8080:8080"
environment:
- NODE_ENV=production
env_file:
- ./backend/.env
redis:
image: "redis"
ports:
- "6379:6379"
FROM golang:1.16-alpine
RUN mkdir -p /usr/src/app
ENV PORT 8080
WORKDIR /usr/src/app
COPY go.mod /usr/src/app
COPY . /usr/src/app
RUN go build -o main .
EXPOSE 8080
CMD [ "./main" ]
The build runs successfully, but after starting the services, the go backend immediately exits throwing following error:
Error trying to ping redis: dial tcp 127.0.0.1:6379: connect: connection refused
Error being catched here:
_, err = client.Ping(ctx).Result()
if err != nil {
log.Fatalf("Error trying to ping redis: %v", err)
}
How come the backend docker service isn't able to connect to redis? Important note: when the redis service is running and I start my backend manually using go run *.go, there's no error and the backend starts successfully.
When you run your Go application inside a docker container, the localhost IP 127.0.0.1 is referring to this container. You should use the hostname of your Redis container to connect from your Go container, so your connection string would be:
redis://redis
I found I was having this same issue. Simply changing (in redis.NewClient(&redis.Options{...}) Addr: "localhost:6379"to Addr: "redis:6379" worked.
Faced similar issue with Golang and redis.
version: '3.0'
services:
redisdb:
image: redis:6.0
restart: always
ports:
- "6379:6379"
container_name: redisdb-container
command: ["redis-server", "--bind", "redisdb", "--port", "6379"]
urlshortnerservice:
depends_on:
- redisdb
ports:
- "7777:7777"
restart: always
container_name: url-shortner-container
image: url-shortner-service
In redis configuration use
redisClient := redis.NewClient(&redis.Options{
Addr: "redisdb:6379",
Password: "",
DB: 0,
})

Unable to connect mysql from docker container?

I have created a docker-compose file it has two services with Go and Mysql. It creates container for go and mysql. Now i am running code which try to connect to mysql database which is running as a docker container. but i get error.
docker-compose.yml
version: "2"
services:
app:
container_name: golang
restart: always
build: .
ports:
- "49160:8800"
links:
- "mysql"
depends_on:
- "mysql"
mysql:
image: mysql
container_name: mysql
volumes:
- dbdata:/var/lib/mysql
restart: always
environment:
- MYSQL_ROOT_PASSWORD=root
- MYSQL_DATABASE=testDB
- MYSQL_USER=root
- MYSQL_PASSWORD=root
ports:
- "3307:3306"
volumes:
dbdata:
Error while connecting to mysql database
golang | 2019/02/28 11:33:05 dial tcp 127.0.0.1:3306: connect: connection refused
golang | 2019/02/28 11:33:05 http: panic serving 172.24.0.1:49066: dial tcp 127.0.0.1:3306: connect: connection refused
golang | goroutine 19 [running]:
Connection with MySql Database
func DB() *gorm.DB {
db, err := gorm.Open("mysql", "root:root#tcp(mysql:3306)/testDB?charset=utf8&parseTime=True&loc=Local")
if err != nil {
log.Panic(err)
}
log.Println("Connection Established")
return db
}
EDIT:Updated docker file
FROM golang:latest
RUN go get -u github.com/gorilla/mux
RUN go get -u github.com/jinzhu/gorm
RUN go get -u github.com/go-sql-driver/mysql
COPY ./wait-for-it.sh .
RUN chmod +x /wait-for-it.sh
WORKDIR /go/src/app
ADD . src
EXPOSE 8800
CMD ["go", "run", "src/main.go"]
I am using gorm package which lets me connet to the database
depends_on is not a verification that MySQL is actually ready to receive connections. It will start the second container once the database container is running regardless it was ready for connections or not which could lead to such an issue with your application as it expects the database to be ready which might not be true.
Quoted from the documentation:
depends_on does not wait for db and redis to be “ready” before starting web - only until they have been started.
There are many tools/scripts that can be used to solve this issue like wait-for which sh compatible in case your image based on Alpine for example (You can use wait-for-it if you have bash in your image)
All you have to do is to add the script to your image through Dockerfile then use this command in docker-compose.yml for the service that you want to make it wait for the database.
What comes after -- is the command that you would normally use to start your application
version: "2"
services:
app:
container_name: golang
...
command: ["./wait-for", "mysql:3306", "--", "go", "run", "myapplication"]
links:
- "mysql"
depends_on:
- "mysql"
mysql:
image: mysql
...
I have removed some parts from the docker-compose for easier readability.
Modify this part go run myapplication with the CMD of your golang image.
See Controlling startup order for more on this problem and strategies for solving it.
Another issue that will rise after you solve the connection issue will be as the following:
Setting MYSQL_USER with root value will cause a failure in MySQL with this error message:
ERROR 1396 (HY000) at line 1: Operation CREATE USER failed for 'root'#'%'
This is because this user already exist in the database and it tries to create another. if you need to use the root user itself you can use only this variable MYSQL_ROOT_PASSWORD or change the value of MYSQL_USER so you can securely use it in your application instead of the root user.
Update: In case you are getting not found and the path was correct, you might need to write the command as below:
command: sh -c "./wait-for mysql:3306 -- go run myapplication"
First, if you are using latest version of docker compose you don't need the link argument in you app service. I quote the docker compose documentation Warning: The --link flag is a legacy feature of Docker. It may eventually be removed. Unless you absolutely need to continue using it, https://docs.docker.com/compose/compose-file/#links
I think the solution is to use the networks argument. This create a docker network and add each service to it.
Try this
version: "2"
services:
app:
container_name: golang
restart: always
build: .
ports:
- "49160:8800"
networks:
- my_network
depends_on:
- "mysql"
mysql:
image: mysql
container_name: mysql
volumes:
- dbdata:/var/lib/mysql
restart: always
networks:
- my_network
environment:
- MYSQL_ROOT_PASSWORD=root
- MYSQL_DATABASE=testDB
- MYSQL_USER=root
- MYSQL_PASSWORD=root
ports:
- "3307:3306"
volumes:
dbdata:
networks:
my_network:
driver: bridge
By the way, if you only connect to Mysql from your app service you don't need to expose the mysql port. If the containers runs in the same network they can reach all ports inside this network.
If my example doesn't works try this
run the docker compose and next go into the app container using
docker container exec -it CONTAINER_NAME bash
Install ping in order to test connection and then run ping mysql.

Docker-compose port forwarding

I have a website hosted on shared hosting on production. The website connects to the database via localhost in the code. In my docker-compose I have a php:5.6-apache and mysql:5.6 instance.
Is there any way to tell docker-compose to have port 3306 on the web container port forwarded to 3306 on the db container, so that when the web container tries to connect to localhost on 3306 it gets sent to db on 3306 and also share port 80 on the web container to the outside world?
Current docker-compose.yml:
version: "3"
services:
web:
build: .
#image: php:5.6-apache
ports:
- "8080:80"
environment:
- "APP_LOG=php://stderr"
- "LOG_LEVEL=debug"
volumes:
- .:/var/www/html
network_mode: service:db # See https://stackoverflow.com/a/45458460/95195
# networks:
# - internal
working_dir: /var/www
db:
image: mysql:5.6
ports:
- "3306:3306"
environment:
- "MYSQL_XXXXX=*****"
volumes:
- ./provision/mysql/docker-entrypoint-initdb.d:/docker-entrypoint-initdb.d
# networks:
# - internal
networks:
internal:
driver: bridge
Current error:
ERROR: for web Cannot create container for service web: conflicting options: port publishing and the container type network mode
Yes it is possible. You need to use the network_mode option. See the below example
version: '2'
services:
db:
image: mysql
environment:
MYSQL_ROOT_PASSWORD: root
ports:
- "80:80"
- "3306:3306"
app:
image: ubuntu:16.04
command: bash -c "apt update && apt install -y telnet && sleep 10 && telnet localhost 3306"
network_mode: service:db
outputs
app_1 | Trying 127.0.0.1...
app_1 | Connected to localhost.
app_1 | Escape character is '^]'.
app_1 | Connection closed by foreign host.
network_mode: service:db instructs docker to not assign the app services it own private network. Instead let it join the network of db service. So any port mapping that you need to do, needs to happen on the db service itself.
The way I usually use it is different, I create a base service which runs a infinite loop and the db and app service both are launched on base service network. All ports mapping need to happen at the base service.

How to connect a flask container with a mysql container

I have already a mysql container named "mysqlDemoStorage" running, exposing port 3306 to 0.0.0.0:3306. I also have a flask app which provides a login page and table-displaying page. The flask app works quite well in host. The login page connects to "user" table in the mysql container and the table-displaying page connects to another table holding all the data to display.
The docker-compose file I used to create the mysql container is as follows:
version: '3'
services:
mysql:
container_name: mysqlDemoStorage
environment:
MYSQL_ROOT_PASSWORD: "demo"
command:
--character-set-server=utf8
ports:
- 3306:3306
image: "docker.io/mysql:latest"
restart: always
Now I want to dockerize the flask app so that I can still view the app from host. The mysql container detail is as followed:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
c48955b3589e mysql:latest "docker-entrypoint.s…" 13 days ago Up 49 minutes 0.0.0.0:3306->3306/tcp, 33060/tcp mysqlDemoStorage
The dockerfile of the flask app I wrote is as follows:
FROM python:latest
WORKDIR /storage_flask
ADD . /storage_flask
RUN pip install -r requirements.txt
EXPOSE 5000
ENTRYPOINT ["python","run.py"]
The flask image can be successfuly built, but when I run the image, I fail to load the page. One point I think that causes the problem is the init.py file to initiate the flask app, which is as follows:
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
from flask_bcrypt import Bcrypt
from flask_login import LoginManager
app = Flask(__name__)
app.config['SECRET_KEY'] = 'aafa4f8047ce31126011638be8530da6'
app.config['SQLALCHEMY_DATABASE_URI'] = 'mysql+pymysql://root:demo#localhost:3306/storage'
db = SQLAlchemy(app)
bcrypt = Bcrypt(app)
login_manager = LoginManager(app)
login_manager.login_view = "login"
login_manager.login_message_category = 'info'
from storage_flask import routes
I was thinking passing the IP of the mysql container to the flask container as the config string for DB connection. But I'm not sure how to do it.
Could someone help to solve the problem? Thank you
change this line
app.config['SQLALCHEMY_DATABASE_URI'] = 'mysql+pymysql://root:demo#localhost:3306/storage'
to
app.config['SQLALCHEMY_DATABASE_URI'] = 'mysql+pymysql://root:demo#mysql:3306/storage'
You also need to make sure that both containers are connected to the same network, for that you need to update your docker-compose file to be something like the below file
version: '3.7'
networks:
my_network_name:
name: my_network_name
external: false
services:
mysql:
container_name: mysqlDemoStorage
environment:
MYSQL_ROOT_PASSWORD: "demo"
command:
--character-set-server=utf8
ports:
- 3306:3306
image: "docker.io/mysql:latest"
restart: always
networks:
- my_network_name
second file
version: '3.7'
networks:
my_network_name:
name: my_network_name
external: true
services:
python_app:
container_name: pythonDemoStorage
ports:
- 5000:5000
image: "Myimage"
restart: always
networks:
- my_network_name

Resources