I am stuck trying to configure docker volumes to share files between my host and make able in my container to use this files. let me explain.
I have a rails docker app with puma as a web server, I want to make able to puma to view and use the ssl .key and .crt files, so for this project also I am using docker-compose in "production mode", but I do not know how to make this work.
My setup is this:
Ubuntu 18.04 server host for production has the ssl files inside /home/ubuntu/my_app_keys, the containers are also in my host.
/home/ubuntu/docker-compose.yml
version: '3'
services:
postgres:
image: postgres:10.5
environment:
POSTGRES_DB: my_app_production
env_file:
-~/production.env
redis:
image: redis:4.0.11
web:
image: my_app:latest
command: bundle exec rails server -p 3000 -b 'ssl://127.0.0.1:3000?key=/home/ubuntu/my_app_keys/server.key&cert=/home/ubuntu/my_app_keys/server.crt' -e production
ports:
- '3000:3000'
volumes:
- /home/ubuntu/my_app_keys
depends_on:
- postgres
- redis
env_file:
- ~/production.env
restart: always
sidekiq:
image: my_app_sidekiq:latest
command: bundle exec sidekiq -C config/sidekiq.yml
depends_on:
- postgres
- redis
env_file:
- ~/production.env
restart: always
so, as you can see: command: bundle exec rails server -p 3000 -b 'ssl://127.0.0.1:3000?key=/home/ubuntu/my_app_keys/server.key&cert=/home/ubuntu/my_app_keys/server.crt' is looking for ssl files in /home/ubuntu/my_app_keys, when I execute docker-compose up puma can not find the ssl files and exits with:
/usr/local/bundle/gems/puma-3.9.1/lib/puma/minissl.rb:180:in `key=': No such key file '/home/ubuntu/my_app_keys/server.key' (ArgumentError)
I think is because key=/home/ubuntu/my_app_keys/server.key&cert=/home/ubuntu/my_app_keys/server.crt are pointing in the container context but I have the cert and key in my host context
so, I include in docker compose volume in order to bind-mount the files:
volumes:
- /home/ubuntu/my_app_keys
but without luck, same error.
In the container context my app lives in /var/www/my_app directory, so I tried to specify an absolute path (for some reason I imagined that it was because the ssl files were not in the same directory where my app lived could not be shared), so I add as compose-file docs say:
volumes:
- /home/ubuntu/my_app_keys:/var/www/my_app
and change in compose file:
command: bundle exec rails server -p 3000 -b 'ssl://127.0.0.1:3000?key=server.key&cert=server.crt' -e
when I execute the compose up my web service exit with error:
web | Could not locate Gemfile or .bundle/ directory
only way that web service run is (but no ssl files exist):
volumes:
- /home/ubuntu/my_app_keys
so, I do not know what to do now. any help?
When your Docker Compose YAML file says:
volumes:
- /home/ubuntu/my_app_keys
It means, "make /home/ubuntu/my_app_keys in container space persist across restarts of the container; it will start off empty unless the Dockerfile did something special; it's not connected to any specific host content".
When you say:
volumes:
- /home/ubuntu/my_app_keys:/var/www/my_app
It means, "totally replace the contents of /var/www/my_app in container space with the contents of /home/ubuntu/my_app_keys on the host". (The path names in host and container space don't need to be the same.)
As a bonus question, when you say:
rails server -b 'ssl://127.0.0.1:3000?...'
It means, "only listen for inbound connections on port 3000 initiated from within this Docker container; don't accept any connections from outside the container at all, whether from the same physical host, other containers, or elsewhere."
Related
I have a running server with nginx and with some multiple personal apps. I want to add my new app which is built on docker with docker-compose. Now i want to add it in the existing nginx(which is not built in docker) but not sure how.
current docker-compose.prod file
version: '1'
services:
app:
build: .
command: "sh scripts/wait-for 127.0.0.1:3306 -- scripts/start.sh"
network_mode: "host"
environment:
- RAILS_ENV=production
- RAILS_SERVE_STATIC_FILES=true # NOTE: this is for LOCAL PROD. to be removed on the actual server; nginx should handle static assets
- MYSQL_USER=${DATABASE_USER}
- MYSQL_PASSWORD=${DATABASE_PASSWORD}
env_file:
- .env.prod
ports:
- "3051:3000" # random port
I'm a beginner in both nginx and docker. Could you give out some basic steps on how to configure it to run with nginx
I'm trying to dockerize an existing Rails app that uses Postgresql 9.5 as its database. In my docker-compose.yml. After a successful "docker-compose build" I can run the "docker-compose up" command and see the connection but when I navigate to localhost I get the following error.
PG::ConnectionBad
could not connect to server: No such file or directory Is the server running >locally and accepting connections on Unix domain socket >"/var/run/postgresql/.s.PGSQL.5432"?
Here is what is in my docker-compose.yml
version: '2'
services:
db:
image: postgres:9.5
restart: always
volumes:
- ./tmp/db:/var/lib/postgresql/9.5/main
environment:
POSTGRES_USER: username
POSTGRES_PASSWORD: password
POSTGRES_DB: hardware_development
web:
build: .
command: bash -c "rm -f tmp/pids/server.pid && bundle exec rails s -p 3000 -b '0.0.0.0'"
volumes:
- .:/myapp
ports:
- "3000:3000"
depends_on:
- db
From what I've seen I need to do some specification somewhere in my dockerfile or the docker-compose.yml but I either don't see a change or end up back at the same error.
I've been able to use Docker's own docs to use Docker Compose to create a new rails app with postgres where I see the "yay you're on rails!" page but now with my own code I can't see anything. Running the app outside of docker shows me the test page as well so its not the code within my rails app or the Postgres evnironment outside of Docker.
Your db docker-compose entry isn't exposing any ports. It needs to expose 5432. Add a ports line for that just like you have for web.
Edit: also I don't know why you added restart: always to your database container, but I wouldn't recommend that for rails or pretty much anything.
[https://github.com/gtriggiano/ngrok-tunnel ] runs ngrok inside a container. Ngrok is required to run in the container to avert security risks. But am facing problems after running the scripts, which generates the url
$ docker pull gtriggiano/ngrok-tunnel
$ docker run -it -e "TARGET_HOST=localhost" -e "TARGET_PORT=3000" -p 4040 gtriggiano/ngrok-tunnel
am running my rails app on localhost:3000
is it my problem or can it be fixed by altering the scripts(inside the repo)?
I couldn't get this working but switched to https://github.com/shkoliar/docker-ngrok and it works brilliantly.
In my case I added it to my docker-compose.yml file:
ngrok:
image: shkoliar/ngrok:latest
ports:
- 4551:4551
links:
- web
environment:
- PARAMS=http -region=eu -authtoken=${NGROK_AUTH_TOKEN} localdev.docker:80
networks:
dev_net:
ipv4_address: 10.5.0.10
And it's started with everything else when I do docker-compose up -d
Then there's a web UI at http://localhost:4551/ for you to see the status, requests, the ngrok URLs, etc.
The Github page does have examples of running it manually from the command line too though, rather than via docker-compose:
Command-line Example The example below assumes that you have running
web server docker container named dev_web_1 with exposed port 80.
docker run --rm -it --link dev_web_1 shkoliar/ngrok ngrok http dev_web_1:80
With command line usage, ngrok session is active until it
won't be terminated by Ctrl+C combination.
No. if you execute -p with single number it's container port - host port is randomly assigned.
Using -p, --publish ip:[hostPort]:containerPort at docker run can specify the the host port with the container port.
as of now the 4040 of container is exposed. Not sure if your service listens by default on it.
To get localhost port execute
docker ps
you'll see the actual port it's not listening on.
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
1aaaeffe789d gtriggiano/ngrok-tunnel "npm start" About a minute ago Up About a minute 0.0.0.0:32768->4040/tcp wizardly_poincare
here it's listening on localhost:32768
this composer works for me. Note that in the entrypoint command for ngrok you have to reference the other service by name
version: '3'
services:
yourwebserver:
build:
context: ./
dockerfile: ...
target: ...
container_name: yourwebserver
volumes:
- ...
ports:
- ...
extra_hosts:
- 'host.docker.internal:host-gateway'
depends_on:
- ngrok
ngrok:
image: ngrok/ngrok:alpine
environment:
NGROK_AUTHTOKEN: '...'
command: 'http yourwebserver:80'
ports:
- '4040:4040'
expose:
- '4040'
I'm not sure if you have already solved this but when I was getting this error I could only solve it like this:
# docker-compose.yml
networks:
- development
I also needed to expose the 3000 port of my web container because it still wasn't exposed.
# docker.compose.yml
web:
expose:
- "3000"
My container for the server running on development is also under the development network. The only parameters, I believe, you should pass for the container to execute are image, ports, environment with DOMAIN and PORT for the server container, a link, and an expose on your web container:
# docker-compose.yml
ngrok:
image: shkoliar/ngrok
ports:
- 4551:4551
links:
- web
networks:
- development
environment:
- DOMAIN=squad_web
- PORT=3000
Actually to make ngrok work with your docker container you can install it outside of your project just like the manual on their website says. And then add
nginx:
labels:
- "traefik.http.routers.${PROJECT_NAME}_nginx.rule=Host(`${PROJECT_BASE_URL}`, `aaa-abc-xxx-140-177.eu.ngrok.io`)"
This particular example is for docker4drupal docker-compose file and traefik mapped as 80:80
I have the following docker-compose.yml to start a webserver with PHP.
version: "2.0"
services:
nginx:
image: nginx
ports:
- "8000:80"
volumes:
- ./web:/web
- ./nginx/default.conf:/etc/nginx/conf.d/default.conf
php:
image: php:${PHP_VERSION}-fpm
volumes:
- ./web:/web
After running docker-compose up, I can access my website perfectly at http://localhost:8000. But if I then access the nginx container, with:
$ docker-compose run nginx bash
and within the container I run:
$ service nginx stop
I still can see the website http://localhost:8000 being displayed in the browser.
How can it be that after stopping the server in the container, the website is still being delivered?
The docker-compose run command starts a new container, you're stopping nginx in that new container, what you want is docker attach nginx
The documentation is located here.
I am running a Java app inside a Docker container which is supposed to connect MySQL inside the other container. Trying multiple options suggested in the forms, nothing really works. Here is my Docker Compose file:
version: "3"
services:
app:
build:
context: ./
dockerfile: /src/main/docker/Dockerfile
image: app1
environment:
- DB_HOST=Imrans-MacBook-Pro.local
- DB_PORT=3306
ports:
- 8080:8080
networks:
- backend
depends_on:
- mysql
mysql:
image: mysql:5.7.20
hostname: mysql
environment:
- MYSQL_USER=root
- MYSQL_ALLOW_EMPTY_PASSWORD=yes
- MYSQL_DATABASE=app1
ports:
- 3306:3306
command: mysqld --lower_case_table_names=1 --skip-ssl --character_set_server=utf8 --explicit_defaults_for_timestamp
networks:
- backend
networks:
backend:
driver: bridge
Where DB_HOST=Imrans-MacBook-Pro.local is my laptop's name. This did not work. Some suggest that the container name can be used so tried DB_HOST= mysql, never worked.
The only thing works from times to time when I pass the laptop's IP address, which is not I want to do. So, what is a good way to create communication between those containers?
The mysql is running in the container so there are two things that you should consider here:
If the mysql is running in the container then you will need to link the app container to the mysql container. This will allow them to talk to
each other using docker's inter container communication. The containers talk to each other using hostnames to resolve their respective internal IP addresses. See later in my answer I will show you how to get the two containers to communicate with each other using a compose file.
The mysql container should make use of a docker volume to store the database. This will allow you to store the database and related files on the file system of the host (server or machine where the containers are running on). The docker volume will then be mounted as a directory in the container. Thus the container can now read and write to a directory on the machine where the docker containers are running on. This means that even if the containers are all deleted or removed you will still have the database data persist. Here is a nice beginner friendly article on docker volumes and using them with MySQL:
https://severalnines.com/blog/mysql-docker-containers-understanding-basics
Container communication using only docker without compose:
You have container "app" and "mysql", you want to be able to access "app" on localhost and you want "app" to be able to connect to mysql. How are you gonna do this?
1. You need to expose a port for container "app" so we can access it on localhost. The docker containers have their own internal network and it is closed to you unless you expose some ports with docker.
You need to link the "mysql" container to "app" without exposing "mysql" 's ports to the rest of the world.
This config should work for what you want to achieve:
version: "2"
services:
app:
build:
context: ./
dockerfile: /src/main/docker/Dockerfile
image: app1:latest
links:
- mysql
environment:
- DB_HOST=mysql
# This is the hostname that app will reach the mysql container on.
# If you do with app container:
# docker exec -it <app container id> bash
# # apt-get update -y && apt-get install iputils-ping -y
#
# Then you should be able to ping mysql container with:
#
# # ping -c 2 mysql
- DB_PORT=3306
ports:
- 8080:8080
# You will access "app" on localhost:8080 in your browser. If this is running on your own machine.
mysql: #hostname actually gets set here so no need to set it later
image: mysql:5.7.20
environment:
- MYSQL_USER=root
- MYSQL_ALLOW_EMPTY_PASSWORD=yes
- MYSQL_DATABASE=app1
# Remember to use a volume if you would like this container's data to persist or if you would like
# to restore a database backup.
command: mysqld --lower_case_table_names=1 --skip-ssl --character_set_server=utf8 --explicit_defaults_for_timestamp
Now you can just start it up with:
$ docker-compose up
If you ran this before then just make sure to run this first before running docker-compose up:
$ docker-compose down
Let me know if that helps.
I have, in the past, gotten this to work without explicitly setting the host networking part in Docker Compose. Because Docker images inside a Docker Compose File are put into a Docker Network with each other, you really shouldn't have to do anything to get this to work: by default you should be able to attach into the container for your Spring app and be able to ping mysql and have it work out.
DB host should be localhost or 127.0.0.1