I am trying to develop an automatic Drupal installer which creates an already configured Drupal docker container ready to be used with one single command execution.
In order to achieve this, I have this docker-compose.yml:
version: "3"
services:
# Database
drupal_db:
image: mysql:8.0
command: --default-authentication-plugin=mysql_native_password
container_name: drupal_db
ports:
- "33061:3306"
restart: unless-stopped
volumes:
- drupal_db_data:/var/lib/mysql
environment:
MYSQL_ROOT_PASSWORD: drupal
MYSQL_DATABASE: drupal
MYSQL_USER: drupal
MYSQL_PASSWORD: drupal
networks:
- drupal
# Drupal
drupal:
image: drupal:9-php7.4-apache
container_name: drupal
ports:
- "8080:80"
restart: unless-stopped
volumes:
- ./drupal/d_modules:/var/www/html/modules
- ./drupal/d_profiles:/var/www/html/profiles
- ./drupal/d_sites:/var/www/html/sites
- ./drupal/d_sites/default/files/translations:/var/www/html/sites/default/files/translations
- ./drupal/d_themes:/var/www/html/themes
- ./scripts/drush:/opt/drupal/scripts
depends_on:
- drupal_db
env_file:
- drupal-install.env
links:
- drupal_db:mysql
networks:
- drupal
volumes:
drupal_db_data: {}
drupal_data: {}
networks:
drupal:
Together with this Makefile:
clear:
docker-compose down -v
autoinstall:
docker-compose up -d
docker exec drupal composer require drush/drush
docker exec drupal bash -c '/opt/drupal/scripts/autoinstall.sh'
autoinstall.sh is an script which is mounted via one of Drupal's container volumes, which runs this:
#!/bin/bash
drush site-install ${DRUPAL_PROFILE} \
--locale=${LOCALE} \
--db-url=${DB_URL} \
--site-name=${SITE_NAME} \
--site-mail=${SITE_MAIL} \
--account-name=${ACCOUNT_NAME} \
--account-mail=${ACCOUNT_MAIL} \
--account-pass=${ACCOUNT_PASS} \
--yes
This uses environment variables, which are specified at docker-compose.yml through the env-file drupal-install.env:
HOST=drupal_db:33061
DBASE=drupal
USER=drupal
PASS=drupal
DATABASE_HOST=drupal_db:33061
DRUPAL_PROFILE=standard
LOCALE=en
DB_URL=mysql://drupal:drupal#drupal_db:3306/drupal
SITE_NAME=NewSite
SITE_MAIL=newsite#test.com
ACCOUNT_NAME=admin
ACCOUNT_MAIL=admin#test.com
ACCOUNT_PASS=admin
However, when running the make autoinstall command, first two lines run with no issues, but the last one throws this error:
Database settings:<br /><br />Resolve all issues below to continue the installation. For help
configuring your database server, see the <a href="https://www.drupal.org/docs/8/install">in
stallation handbook</a>, or contact your hosting provider.<div class="item-list"><ul><li>Fail
ed to connect to your database server. The server reports the following message: <em class="p
laceholder">SQLSTATE[HY000] [2002] Connection refused</em>.<ul><li>Is the database server run
ning?</li><li>Does the database exist or does the database user have sufficient privileges to
create the database?</li><li>Have you entered the correct database name?</li><li>Have you en
tered the correct username and password?</li><li>Have you entered the correct database hostna
me and port number?</li></ul></li></ul></div>
If I manually run:
docker-compose up -d
docker exec -it drupal bash
composer require drush/drush
/opt/drupal/scripts/autoinstall.sh
Everything works perfectly, but the makefile script doesn't work.
Something really weird happens because if I run make autoinstall twice, the first time will throw this error, but it actually works when I run it a second time. It is really strange and I can't find a solution, but I would like to not to have to run the command twice.
Related
We are using docker-compose to set up the services for our app:
version: "3"
services:
db:
container_name: db
image: postgres:11.1
environment:
POSTGRES_USER: xxx
POSTGRES_PASSWORD: xxx
POSTGRES_DB: xxx
PGPASSWORD: xxx
volumes:
- pgdata:/var/lib/postgresql/data
- ./data/dbdump:/dbdump
networks:
- zenet
ports:
- "5432:5432"
# The React web application
web:
container_name: web
build:
context: .
dockerfile: devenv/web/Dockerfile
volumes:
- ./src/client-app:/usr/local/abc
- /usr/local/abc/node_modules
networks:
- zenet
ports:
- "3000:3000"
command: npm run startindocker
# The Django Rest Framework API
api:
container_name: api
build:
context: .
dockerfile: devenv/api/Dockerfile
environment:
DJANGO_SETTINGS_MODULE: abc.settings.dev
PYTHONSTARTUP: /root/pythonstartup.sh
PYTHONIOENCODING: UTF-8
volumes:
- .:/usr/local/borrow-a-boat
- ./devenv/api/pythonrc.py:/root/pythonstartup.sh
networks:
- zenet
depends_on:
- "db"
ports:
- "9000:9000"
command:
python3 /usr/local/borrow-a-boat/src/django/abc/manage.py runserver 0.0.0.0:9000
tty: true
volumes:
pgdata:
customboatdata:
networks:
zenet:
(sensitive info has been replaced)
My colleagues have the setup running fine. I setup the app & the volumes & containers are up & running. I can hit the service api at port 9000 fine from browser & confirm that the db is populated. However, my web service is unable to get the data from the api. How can I confirm that the above assertion is correct & that the web really cannot communicate with the api service.
And how can I fix this & get the web to receive the data from api. Apologies for the newbie question.
EDIT:
When I run ping api from within the web container using docker exec -it [containerID] /bin/sh, I am recieving a response in the form of :
64 bytes from 172.18.0.4: seq=139 ttl=64 time=0.084 ms
So, clearly, my assertion is incorrect. Why is web service unable to get a response from api service. When I load the web app in browser, I do not get any log display in the terminal of the api being hit.
EDIT-2 :
As per #runwuf question & my response, clearly, the 'web' is able to communicate with the 'api' service. So, something else is wrong. Here are the steps, we follow to setup the stack on our systems. I use a Linux Mint 19.2 OS, while the team uses Macs. The commands are:
docker kill $(docker_container_names)
docker rm -v $(docker_container_names)
docker volume rm abc_pgdata
docker image rm abc_api
docker image rm abc_web
docker-compose build
docker-compose up -d db api web
ssh abc#abc.com 'pg_dump abc | gzip' | gunzip | docker-compose run --rm db psql --host db --username abc
docker-compose run --rm db psql --host db --username abc -c "update core_photo set image_base = 'sample.jpg'"
docker-compose run --rm db psql --host db --username abc -c "update core_experienceimage set image_base = 'sample.jpg'"
In the end, it was a case of env variable not accessible within web service. All it took was to read the console logs in the browser which showed the undefined variable.
The lesson for me is to when it comes to problem solving, no matter how new the technology, don't forget to use the tools you are familiar with.
I can't get environmental variables in a docker-compose file written directly in it to work. A similar configuration with the command line work just fine like this:
docker run --name container_name -d --network=my-net --mount type=bind,src=/Users/t2wu/Documents/Work/Dodo/Intron-Exon_expression/DockerCompose/intronexon_db/mnt_mysql,dst=/var/lib/mysql -e MYSQL_DATABASE=db_name -e MYSQL_USER=username -e MYSQL_PASSWORD=passwd mysql/mysql-server:8.0.13
This is an MySQL instance which sets three environmental variables: MYSQL_DATABASE, MYSQL_USER and MYSQL_PASSWORD. I'm later able to launch bash into it docker exec -it container_name bash and launch the client mysql -u username -p and connects just fine.
However when I write it in a docker-compose.yml:
version: "3.7"
services:
intronexon_db:
image: mysql/mysql-server:8.0.13
volumes:
- type: bind
source: ./intronexon_db/mnt_mysql
target: /var/lib/mysql
environment:
MYSQL_DATABASE: db_name
MYSQL_USER: username
MYSQL_PASSWORD: passwd
networks:
- my-net
networks:
my-net:
driver: bridge
Then when I use the mysql client, it's as if the user doesn't exist. How do I set it so that it is equivalent to the -e flag during docker run?
EDIT
docker-compose --version shows docker-compose version 1.24.1, build 4667896b
EDIT 2
The environmental flag did work. But I run into problem because:
Part of the problem was that it takes MySQL sometime to get the database, username and password setup ready. And I was checking it way too early.
I need to specify localhost for some reason: mysql --host=localhost -u user -p. Specifying 127.0.0.1 will not work.
For some unknown reason the example stack.yml from the official docker image did not have to specify --host when the adminer container is run. If I wipe out the adminer, then --host flag needs to be given.
Sometimes MySQL daemon will stop. It might has to do with my mount target /var/lib/mysql but I'm not certain.
command: --default-authentication-plugin=mysql_native_password is actually significant. I don't know why when I did docker run I didn't need to do anything about this.
docker-compose accept both types of ENVs either an array or a dictionary, better to double or try both approaches.
environment
Add environment variables. You can use either an array or a
dictionary. Any boolean values; true, false, yes no, need to be
enclosed in quotes to ensure they are not converted to True or False
by the YML parser.
Environment variables with only a key are resolved to their values on
the machine Compose is running on, which can be helpful for secret or
host-specific values.
environment:
RACK_ENV: development
SHOW: 'true'
SESSION_SECRET:
or
environment:
- RACK_ENV=development
- SHOW=true
- SESSION_SECRET
Might be something with docker-compose version as it working fine with 3.1. as the offical image suggested, so Better to try offical image docker-compose.yml
version: '3.1'
services:
db:
image: mysql
command: --default-authentication-plugin=mysql_native_password
restart: always
environment:
MYSQL_ROOT_PASSWORD: example
Also, better to debug such cases where everything seems correct but some minor syntax is missing. you can test it before working with DB.
version: "3.7"
services:
intronexon_db:
image: alpine
environment:
MYSQL_DATABASE: myDb
command: tail -f /dev/null
run docker-compose up
Now test and debug in testing enviroment.
docker exec -it composeenv_intronexon_db_1 ash -c "printenv"
the environment params in your yml need the - in front of them could be the likely culprit
version: "3.7"
services:
intronexon_db:
image: mysql/mysql-server:8.0.13
volumes:
- ./intronexon_db/mnt_mysql:/var/lib/mysql
environment:
- MYSQL_DATABASE: db_name
- MYSQL_USER: username
- MYSQL_PASSWORD: passwd
networks:
- my-net
networks:
my-net:
driver: bridge
I'm trying to dockerize an existing Rails app that uses Postgresql 9.5 as its database. In my docker-compose.yml. After a successful "docker-compose build" I can run the "docker-compose up" command and see the connection but when I navigate to localhost I get the following error.
PG::ConnectionBad
could not connect to server: No such file or directory Is the server running >locally and accepting connections on Unix domain socket >"/var/run/postgresql/.s.PGSQL.5432"?
Here is what is in my docker-compose.yml
version: '2'
services:
db:
image: postgres:9.5
restart: always
volumes:
- ./tmp/db:/var/lib/postgresql/9.5/main
environment:
POSTGRES_USER: username
POSTGRES_PASSWORD: password
POSTGRES_DB: hardware_development
web:
build: .
command: bash -c "rm -f tmp/pids/server.pid && bundle exec rails s -p 3000 -b '0.0.0.0'"
volumes:
- .:/myapp
ports:
- "3000:3000"
depends_on:
- db
From what I've seen I need to do some specification somewhere in my dockerfile or the docker-compose.yml but I either don't see a change or end up back at the same error.
I've been able to use Docker's own docs to use Docker Compose to create a new rails app with postgres where I see the "yay you're on rails!" page but now with my own code I can't see anything. Running the app outside of docker shows me the test page as well so its not the code within my rails app or the Postgres evnironment outside of Docker.
Your db docker-compose entry isn't exposing any ports. It needs to expose 5432. Add a ports line for that just like you have for web.
Edit: also I don't know why you added restart: always to your database container, but I wouldn't recommend that for rails or pretty much anything.
I have below docker-compose.yml
version: "2"
services:
api:
build:
context: .
dockerfile: ./build/dev/Dockerfile
container_name: "project-api"
volumes:
# 1. mount your workdir path
- .:/app
depends_on:
- mongodb
links:
- mongodb
- mysql
nginx:
image: nginx:1.10.3
container_name: "project-nginx"
ports:
- 80:80
restart: always
volumes:
- ./build/dev/nginx.conf:/etc/nginx/conf.d/default.conf
- .:/app
links:
- api
depends_on:
- api
mongodb:
container_name: "project-mongodb"
image: mongo:latest
environment:
- MONGO_DATA_DIR=/data/db
- MONGO_LOG_DIR=/dev/null
ports:
- "27018:27017"
command: mongod --smallfiles --logpath=/dev/null # --quiet
mysql:
container_name: "gamestore-mysql"
image: mysql:5.7.23
ports:
- "3306:3306"
environment:
MYSQL_DATABASE: project_test
MYSQL_USER: user
MYSQL_PASSWORD: user
MYSQL_ROOT_PASSWORD: root
And below .gitlab-ci.yml
test:
stage: test
image: docker:latest
services:
- docker:dind
variables:
DOCKER_DRIVER: overlay2
before_script:
- apk add --no-cache py-pip
- pip install docker-compose
script:
- docker-compose up -d
- docker-compose exec -T api ls -la
- docker-compose exec -T api composer install
- docker-compose exec -T api php core/init --env=Development --overwrite=y
- docker-compose exec -T api vendor/bin/codecept -c core/common run
- docker-compose exec -T api vendor/bin/codecept -c core/rest run
When i running my gitlab pipeline it's become field because i think gitlab can't work with services runned by docker-compose.
The error says that mysql refuse the connection.
I need this connection because my test written by codeception will test my models and api actions.
I want test my branches every time any one push in them and if pass just in develop deploy into test server and in master deploy on production server.
What is best way to run my test in gitlab ci/cd and then deploy them in my server?
You should use GitLab CI services instead of docker-compose.
You have to pick one image as your main, in which your commands will be run, and other containers just as services.
Sadly CI services cannot have mounted files in gitlab, you have to be able to configure them with env variables, or you need to create you own image with files in it (you can do that CI stage)
I would suggest you to don't use nginx, and use built-in php server for tests. It that's not possible (you have spicifix nginx config), you will need to build yourself nginx image with copied files inside it.
Also for PHP (the api service in docker-compose.yaml i assume), you need to either build the image ahed or copy command from your dockerfile to script.
So the result should be something like:
test:
stage: test
image: custom-php-image #build from ./build/dev/Dockerfile
services:
- name: mysql:5.7.23
alias: gamestore-mysql
- name: mongo:latest
alias: project-mongodb
command: mongod --smallfiles --logpath=/dev/null
variables:
MYSQL_DATABASE: project_test
MYSQL_USER: user
MYSQL_PASSWORD: user
MYSQL_ROOT_PASSWORD: root
MONGO_DATA_DIR: /data/db
MONGO_LOG_DIR: /dev/null
script:
- api ls -la
- composer install
- php core/init --env=Development --overwrite=y
- php -S localhost:8000 # You need to configure your built-in php server probably here
- vendor/bin/codecept -c core/common run
- vendor/bin/codecept -c core/rest run
I don't know your app, so you will probably have to made some tweaks.
More on that:
https://docs.gitlab.com/ee/ci/docker/using_docker_images.html#define-image-and-services-from-gitlab-ciyml
https://docs.gitlab.com/ee/ci/services/
http://php.net/manual/en/features.commandline.webserver.php
I have mysql DB running in a container and web app running in another container. My use case is once the DB container is up and running app container has to insert some initial data to DB using Liquibase and start the app. My docker yml looks like below.
db:
build: kdb
user: "1000:50"
volumes:
- /data/mysql:/var/lib/mysql
container_name: kdb
environment:
- MYSQL_ALLOW_EMPTY_PASSWORD=yes
image: kdb
ports:
- "3307:3306"
k-api:
container_name: k_api
hostname: k-api
domainname: i.com
image: k_api
volumes:
- /Users/agu/work:/data
build:
context: ./api
args:
KB_API_WAR: k-web-1.2.9.war
KB_API_URL: https://artifactory.b-aws.i.com
ports:
- "8097:8080"
depends_on:
- db
#command: [/usr/local/bin/wait-for-it.sh, "db:3306","-s","-t","0","--","/bin/sh" "wait_for_liquibase.sh"]
links:
- "db:kdb_docker_host"
And in my Dockerfile for api i have entry point for a shell script called "wait_for_liquibase.sh"
CMD ["wait_for_liquibase.sh"]
wait_for_liquibase.sh:
#!/bin/sh
set -e
#RUN liquibase
mvn clean install -X -PdropAll -Dcontexts=test -Dliquibase.user=XX -Dliquibase.pass=XX -Dliquibase.host=db -Dliquibase.port=3306 -Dliquibase.schema=knowledgebasedb -DpromptOnNonLocalDatabase=false -Dcontexts=test -f k/k-liquibase
/usr/local/tomcat/bin/catalina.sh run
The issue is once the DB container is up and running app container is not able to reach the DB server to perform liquibase setup for database. I see the below error.
Communication failure: Unknown database host -Dliquibase.host=db.
I am assuming you are using version 1.
You giving an alias to your "db" service, you will need to use that alias, kdb_docker_host
Also, the ports are mapping to the host machine, to expose ports between containers yuo will need to use the expose property.
expose:
- 3306
I used this in Docker file RUN apt-get update
RUN apt-get install netcat -y
ADD wait-for-base.sh /wait-for-base.sh
CMD ["/wait-for-base.sh"]
and in wait-for-base.sh :
#!/bin/bash
while ! nc -z db 3306; do sleep 3; done
[my command to run]
In your case /usr/local/tomcat/bin/catalina.sh run