I'm working on building docker containers for a Ruby-on-Rails project I'm currently working on, so that I can develop this project using the remote feature of Visual Studio Code. This project should still continue to work without using docker containers, so I cannot make breaking changes to the existing code that would compromise this.
The application server (Rails) needs to connect to a MySQL database that's running in a separate container. The database container is named db, and I can connect from the application container to this container by using the db hostname.
The database.yml config file for rails defines how to connect to the database, but this is where my problem is situated. I don't want to change the host to db instead of localhost as this would mean that regular users (that do not use Docker containers) will no longer be able to connect to the database without changing this file. How can I somehow start or change my docker config so that db is accessible as localhost instead of db inside of the application container?
database.yml:
default: &default
adapter: mysql2
username: ****
password: ****
pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
development:
<<: *default
database: ****
# setup local port forwarding for this to work
host: db
port: 3306
docker-compose.yml:
version: '3.7'
services:
db:
build: ./unipept-db
environment:
MYSQL_ROOT_PASSWORD: ****
MYSQL_DATABASE: ****
MYSQL_USER: ****
MYSQL_PASSWORD: ****
restart: always
ports:
- "3306:3306"
hostname: mysql
phpmyadmin:
depends_on:
- db
image: phpmyadmin/phpmyadmin
ports:
- '8080:80'
environment:
PMA_HOST: db
MYSQL_ROOT_PASSWORD: ****
restart: always
app:
depends_on:
- db
build: ./unipept-application
command: sleep infinity
ports:
- '5000:5000'
volumes:
- ~/.gitconfig:/root/.gitconfig
- ..:/workspace
user network_mode: "host"in your APP config then you can call the DBfrom your APP using localhost:PORT
phpmyadmin:
depends_on:
- db
image: phpmyadmin/phpmyadmin
ports:
- '8080:80'
environment:
PMA_HOST: db
MYSQL_ROOT_PASSWORD: ****
restart: always
network_mode: "host"
app:
depends_on:
- db
build: ./unipept-application
command: sleep infinity
ports:
- '5000:5000'
network_mode: "host"
volumes:
- ~/.gitconfig:/root/.gitconfig
- ..:/workspace
PS: Published ports are discarded when using host network mode
If you make it an environment variable in your database.yml.erb file, it will be configurable. You even already have an example of this. You can set:
host: <%= ENV.fetch('DB_HOST', 'localhost') %>
In development, just don't set the environment variable, and it will use localhost. In a Docker environment, do set it, and it will use that hostname.
version: '3'
services:
db:
image: mysql
app:
build: .
environment:
DB_HOST: db
ports:
- '5000:5000'
You should also pass things like database credentials the same way.
Related
I am relatively new to dev in general, to the Docker universe and to Rails in particular, apologize in advance if it sounds like a silly question.
I am trying to run an application in a monorepo composed of 4 services (2 websites and 2 APIs) + Postgresql, with the help of Docker Compose. The final goal is to run it on a VPS with Traefik (once I get the current app to work locally).
Here are the different services :
Postgres (through the Postgres image available in Dockerhub)
a B2C website (NextJS)
an admin website (React with create Vite)
an API (Rails). It should be linked to the Postgres database
a Strapi API (for the content of the B2C website). Strapi has its own SQLite database. Only the B2C website requires the data coming from Strapi.
When I run the docker compose up -d command, it seems to be working (see pic below)
but when I go to one of the websites (except for the Strapi that seems to be correctly working) (https://localhost:3009, or 3008 or 3001), I get nothing (see below).
However, I don't see any error in the logs of any apps. For instance the Rails API logs below:
I assume that I have mistakes in my config, especially in the database.yml config of the Rails api and the docker-compose.yml file.
database.yml :
default: &default
adapter: postgresql
encoding: unicode
pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
host: pg
development:
<<: *default
database: chana_api_v2_development
test:
<<: *default
database: chana_api_v2_test
production:
<<: *default
database: chana_api_v2_production
username: chana
password: <%= ENV["CHANA_DATABASE_PASSWORD"] %>
docker-compose.yml
version: '3'
services:
# ----------------POSTGRES -----------------
pg:
image: postgres:14.6
container_name: pg
networks:
- chana_postgres_network
ports:
- "5432:5432"
environment:
POSTGRES_DB: chana_development
POSTGRES_USER: chana
POSTGRES_PASSWORD: chana
volumes:
- ./data:/var/lib/postgresql/data
# ----------------- RAILS API -----------------
api:
build: ./api
container_name: api
networks:
- chana_postgres_network
- api_network
volumes:
- ./api:/chana_api
ports:
- "3001:3000"
depends_on:
- pg
# ----------------- STRAPI -----------------
strapi:
build:
context: ./strapi
args:
BASE_VERSION: latest
STRAPI_VERSION: 4.5.0
container_name: chana-strapi
restart: unless-stopped
env_file: .env
environment:
NODE_ENV: ${NODE_ENV}
HOST: ${HOST}
PORT: ${PORT}
volumes:
- ./strapi:/srv/app
- strapi_node_modules:/srv/app/node_modules
ports:
- "1337:1337"
# ----------------- B2C website -----------------
public-front:
build: ./public-front
container_name: public-front
restart: always
command: yarn dev
ports:
- "3009:3000"
networks:
- api_network
- chana_postgres_network
depends_on:
- api
- strapi
volumes:
- ./public-front:/app
- /app/node_modules
- /app/.next
# ----------------- ADMIN website -----------------
admin-front:
build: ./admin-front
container_name: admin-front
restart: always
command: yarn dev
ports:
- "3008:3000"
networks:
- api_network
- chana_postgres_network
depends_on:
- api
volumes:
- ./admin-front:/app
- /app/node_modules
- /app/.next
volumes:
strapi_node_modules:
networks:
api_network:
chana_postgres_network:
Do you have any idea why I cannot see anything on the website pages?
I tried to change the code of the different files that are relevant, especially database.yml, docker-compose.yml, and the dockerfiles of each app.
Also, I tried to look into the api container (Rails) with the command docker exec -it api /bin/sh to check the database through the Rails console, and I get this error message:
activeRecord::ConnectionNotEstablished could not connect to server: No such file or directory. Is the server running locally and accepting connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
instead of localhost press ctrl and click on the url that it passes sometimes does not open on the localhost port of your website
looks like your host in docker is 127.0.0.1:3000 normally this is fine but in docker when you want to expose the app to your host machine you need to change the app to run on 0.0.0.0:3000 and then docker will be able to pass the app through to your host machine. Without specific Dockerfiles this is the best I can do. I have run into this issue with strapi and some other apps before so hopefully it helps.
It will still be localhost:3000 on the host machine if i wasn't clear.
OK so I am trying to deploy a Rails app to a docker container (host machine is a mac). I was thinking to deploy in development first to check everything is working.
I have setup a phpmyadmin service and I can connect to the server by typing in server name moviedb_mariamovie_1 with user root and corresponding PW.
But whatever I put into my database.yml for Rails doesn't work: I tried localhost, I tried 127.0.0.1, I tried "mariamovie" and I tried "moviedb_mariamovie_1", and it always says "host not found" when I tried rails db:create (or anything actually that involves the DB).
I am totally confused by this. I read the database section of the docker manuals and I seem to be too stupid for that.
(I have other problems with this but one after the other :)
docker-compose.yml:
version: "3.7"
services:
moviedb:
image: tkhobbes/moviedb
restart: unless-stopped
ports:
- 3001:3000
depends_on:
- mariamovie
environment:
MYSQL_ROOT_PASSWORD: redacted
RAILS_ENV: development
volumes:
- /Users/thomas/Documents/Production/moviedb/storage:/opt/activestorage
mariamovie:
image: mariadb
restart: unless-stopped
ports:
- 3333:3306
environment:
MYSQL_ROOT_PASSWORD: redacted
phpmymaria:
image: phpmyadmin
restart: unless-stopped
ports:
- 8021:80
depends_on:
- mariamovie
environment:
PMA_PORT: 3333
PMA_ARBITRARY: 1
image: nginx:1.21-alpine
volumes:
- /Users/thomas/Documents/Production/moviedb/vendor/nginx:/etc/nginx/user.conf.d:ro
ports:
- 8020:8020
depends_on:
- moviedb
restart: unless-stopped
database.yml:
default: &default
adapter: mysql2
encoding: utf8mb4
pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
host: 127.0.0.1
port: 3333
username: redacted
password: redacted
development:
<<: *default
database: newmovie_development
...
You're inside your docker "network". Your database should be accessible from your Rails app (which is inside too) via mariamovie:3306.
I am a totally beginner at Shopware and I want to use PhpMyAdmin for my local Shopware 6 setup.
For the download I used the official Shopware 6 Development repository https://github.com/shopware/development
I've already seen that the docker-compose.yml has implemented the following:
app_mysql:
build: dev-ops/docker/containers/mysql
environment:
MYSQL_ROOT_PASSWORD: root
MYSQL_USER: app
MYSQL_PASSWORD: app
networks:
shopware:
aliases:
- mysql
and now I want to implement phpmyadmin. I tried the following:
phpmyadmin:
image: phpmyadmin/phpmyadmin
links:
- app_mysql:mysql
depends_on:
- app_mysql
ports:
- 8181:80
environment:
PMA_HOST: app_mysql
MYSQL_ROOT_PASSWORD: root
MYSQL_USER: app
MYSQL_PASSWORD: app
phpmyadmin is visible on localhost:8181 but when I try to login I get the following errors:
mysqli::real_connect(): php_network_getaddresses: getaddrinfo failed: Name or service not known
mysqli::real_connect(): (HY000/2002): php_network_getaddresses: getaddrinfo failed: Name or service not known
How can I solve it?
Usually phpmyadmin should be in the same network as the database.
Service names are resolved to the IP addresses of the containers, therefore it's recommended to use names allowed by RFC1035 to avoid additional problems.
I removed links:, aliases, depends_on that are deprecated/not required and ended up with this docker-compose.yml.
version: '3.7'
services:
app-mysql:
#build: dev-ops/docker/containers/mysql
image: mysql
environment:
MYSQL_ROOT_PASSWORD: root
MYSQL_USER: app
MYSQL_PASSWORD: app
networks:
- shopware
phpmyadmin:
image: phpmyadmin/phpmyadmin
ports:
- 8181:80
environment:
PMA_HOST: app-mysql
PMA_PORT: 3306
PMA_ARBITRARY: 1
networks:
- shopware
networks:
shopware
Run the containers:
docker-compose up
Open http://localhost:8181/index.php in a browser.
Use
Server: app-mysql
Username: root
Password: root
Enjoy:
You must use as host mysql or app_mysql
It is not an answer, but try
http://localhost:8001/
it is not phpMyAdmin but it is another tool Adminer :)
You should link phpmyadmin and mysql using
links:
- app_mysql:mysql
use it as :
links:
- mysql
Add environment:
`PMA_HOST`: mysql
Each time when I try to access the phpmyadmin from browser I receive this error:
"Cannot log in to the MySQL server"
I tried to change networks, restart docker.
version: '3'
services:
web:
image: nginx:alpine
ports:
- 80:80
volumes:
- ./public_html:/public_html
- ./conf.d:/etc/nginx/conf.d
networks:
- nginxphp
php:
image: php:7.1.11-fpm-alpine
volumes:
- ./public_html:/public_html
expose:
- 9000
networks:
- nginxphp
db:
image: mysql
environment:
MYSQL_ROOT_PASSWORD: root
ports:
- "3306:3306"
phpmyadmin:
image: phpmyadmin/phpmyadmin
environment:
MYSQL_ROOT_PASSWORD: root
MYSQL_USER: root
MYSQL_PASSWORD: root
depends_on:
- db
ports:
- "8080:80"
networks:
nginxphp:
Cannot log in to the MySQL server
mysqli_real_connect(): The server requested authentication method unknown to the client [caching_sha2_password]
mysqli_real_connect(): (HY000/2054): The server requested authentication method unknown to the client
Disclaimer: I'm not a Docker user!
Your didn't mention if you're using the browser on the same computer as the sever or remotely. You'll need access to the mysql server (mysqld) through a terminal (command prompt). If this is a new install, it must be on the computer that is running mysql server.
In the Docker mysql page:
"The default configuration for MySQL can be found in /etc/mysql/my.cnf, which may include additional directories such as /etc/mysql/conf.d or /etc/mysql/mysql.conf.d."
Try looking in the /etc/mysql/my.cnf file on the server you're trying access. first. You're looking for:
bind-address = x.x.x.x
This is the address that the mysql server will talk to ("bound to"). Its typically "localhost" or "127.0.0.1".
To eliminate the error message like you see, I had to do two things:
1) change 'bind-address to 0.0.0.0'
this allows the server to connect to any computer. However, THIS IS A SECURITY RISK. Once you get it working, go read about bind addresses on the mysql website and set it appropriately.
2) Create an new account user#ipaddr where user is the new username and IPAddress is the ip4 address of the computer your trying to connect from. i.e.:
CREATE USER 'user'#'192.168.1.68' IDENTIFIED BY 'password';
GRANT ALL PRIVILEGES ON *.* TO 'user'#'192.168.1.68';
Now try accessing mysql through the terminal program using the new username on the computer with the ip you entered above by typing:
mysql -uuser -ppassword -hx.x.x.x
Hopefully, this will help point you in the right direction. There's a ton of information about bind addresses and security on the web.
Because the phpmyadmin container has connected to the default host localhost. But your mysql server is located in other container (it means you cannot connect to mysql server by using localhost). So in the phpmyadmin service, you have to set PMA_HOST=db. See full env variables: https://hub.docker.com/r/phpmyadmin/phpmyadmin/
Full docker-compose.yml:
version: '3'
services:
web:
image: nginx:alpine
ports:
- 80:80
volumes:
- ./public_html:/public_html
- ./conf.d:/etc/nginx/conf.d
networks:
- nginxphp
php:
image: php:7.1.11-fpm-alpine
volumes:
- ./public_html:/public_html
expose:
- 9000
networks:
- nginxphp
db:
image: mysql
environment:
MYSQL_ROOT_PASSWORD: root
ports:
- "3306:3306"
phpmyadmin:
image: phpmyadmin/phpmyadmin
environment:
PMA_HOST: db
MYSQL_ROOT_PASSWORD: root
MYSQL_USER: root
MYSQL_PASSWORD: root
depends_on:
- db
ports:
- "8080:80"
networks:
nginxphp:
If you are using phpmyadmin with latest mysql version you will have some login issues:
Cannot log in to the MySQL server mysqli_real_connect():
The server requested authentication method unknown to the client [caching_sha2_password] mysqli_real_connect(): (HY000/2054):
The server requested authentication method unknown to the client
The solution is to downgrade to mysql 5.7 or another.
I tested with mysql 5.7 and it works for me.
If you want to can test with another versions and let the community know.
Below is the docker-compose file that builds and run Nginx + php 7.1 + mysql 5.7 +phpmyadmin
version: '3'
services:
web:
image: nginx:alpine
ports:
- 80:80
volumes:
- ./public_html:/public_html
- ./conf.d:/etc/nginx/conf.d
networks:
- nginxphp
php:
image: php:7.1.11-fpm-alpine
volumes:
- ./public_html:/public_html
expose:
- 9000
networks:
- nginxphp
db:
image: mysql:5.7
ports:
- "3306:3306"
environment:
MYSQL_ROOT_PASSWORD: root
phpmyadmin:
image: phpmyadmin/phpmyadmin
links:
- db
environment:
PMA_HOST: db
PMA_PORT: 3306
MYSQL_USER: root
MYSQL_PASSWORD: root
MYSQL_ROOT_PASSWORD: root
depends_on:
- db
ports:
- "8080:80"
networks:
nginxphp:
Hope this will save some time to someone!
Instead of below mysql container DB link, I want to link AWS Mysql RDS in docker yml file. Is it possible?
mysql_db:
image: mysql:5.6
container_name: shishir_db
environment:
MYSQL_ROOT_PASSWORD: "xxxxxxxx"
#MYSQL_USER: "shishir"
#MYSQL_DATABASE: "shishir1"
MYSQL_PASSWORD: "xxxxxxxx"
ports:
- "3306:3306"
There are a couple of ways that you can link to a AWS RDS MySQL instance from your docker-compose.yml.
The first, and perhaps simplest way is to set environment variables on the containers that need access to the RDS MySQL instance. So for example, you could update your OperationEngine service definition to look something like:
OperationEngine:
image: shishir/operationengine:${RELEASE_OTA_VERSION}
container_name: operation_engine
ports:
- "8080:8080"
environment:
- DOCKER_HOST_IP: ${DOCKER_HOST_IP}
- JAVA_OPTS: ${JAVA_OPTS}
- MYSQL_HOST: "your-mysql-cname.rds.amazon.com"
- MYSQL_USER: "username"
- MYSQL_PASSWORD: "password"
volumes:
- ${HOME}/operationengine/logs/:/usr/local/tomcat/logs/
You can then update the configuration in that service to read database connection details from the environment e.g ${MYSQL_HOST}.
The obvious downside to this approach is that you have connection details stored as plain text in your docker-compose.yml, which is not great, but may be acceptable depending on your requirements.
The 2nd approach (and the one I tend to favour) is to bind mount the database configuration into the running container.
Most applications support reading database connection details from a properties file. As an example: lets say that on start-up your application read from /config/database.properties and required the following properties to connect to the database:
config.db.host=your-mysql-cname.rds.amazon.com
config.db.user=foo
config.db.password=bar
I would setup my environment so that at runtime, I bind mount a properties file that provides all of the required values to the container:
OperationEngine:
volumes:
- /secure/config/database.properties:/config/database.properties
The /secure/config directory is part of the filesystem on your Docker host. How that directory gets created and populated is up-to-you. Typically I approach this by having my environment setup scripts make the directory and then clone a private Git repository into this directory which contains the correct configuration for that environment. Naturally only those with the required permission levels can view the Git repositories that contain sensitive configuration details i.e. for production system.
Hope that helps.
No, i have not created networking in my yml file. Below is the entirecontent of my yml file, so what changes would be required?
mysql_db:
image: mysql:5.6
container_name: shishir_db
environment:
MYSQL_ROOT_PASSWORD: "xxxxxxxx"
#MYSQL_USER: "shishir"
#MYSQL_DATABASE: "shishir1"
MYSQL_PASSWORD: "xxxxxxxx"
ports:
- "3306:3306"
zookeeper:
image: wurstmeister/zookeeper
container_name: zookeeper
ports:
- "2181:2181"
kafka:
image: shishir/kafkaengine:0.10
container_name: kafka
links:
- zookeeper
ports:
- "9092:9092"
environment:
KAFKA_ADVERTISED_HOST_NAME: ${DOCKER_HOST_IP}
KAFKA_ZOOKEEPER_CONNECT: ${DOCKER_HOST_IP}:2181
KAFKA_ADVERTISED_PORT: 9092
KAFKA_CREATE_TOPICS: ${KAFKA_TOPICS}
KAFKA_ZOOKEEPER_SESSION_TIMEOUT_MS: 12000
KAFKA_ZOOKEEPER_CONNECTION_TIMEOUT_MS: 12000
flywaydb:
image: shishir/flywaydb
container_name: flywaydb
links:
- mysql_db
OperationEngine:
image: shishir/operationengine:${RELEASE_OTA_VERSION}
container_name: operation_engine
links:
- mysql_db
ports:
- "8080:8080"
environment:
DOCKER_HOST_IP: ${DOCKER_HOST_IP}
JAVA_OPTS: ${JAVA_OPTS}
volumes:
- ${HOME}/operationengine/logs/:/usr/local/tomcat/logs/