I am running docker Server Version: 18.06.0-ce on centos 7.5.
I have a docker-compose file running db2 server with the following sample definition:
The docker-compose file has the following options:
version: "3.7"
services:
db2exp:
image: db2
ports:
- "50000:50000"
networks:
- lmnet
ipc: host
cap_add:
- IPC_LOCK
- IPC_OWNER
environment:
- DB2INSTANCE=db2inst1
- DB2PASSWD=db2inst1
- LICENSE=accept
volumes:
- db2data:/home
When using docker-compose up, I do not have problems with starting the db2 service. However when I try to use docker stack, I get the following message:
docker stack deploy test --compose-file docker-compose.yml
Ignoring unsupported options: cap_add, ipc
This renders db2start to return SQL1042C An unexpected system error occurred.
It would be ideal if what runs in compose runs in stack. What, if any, can be done so that the db2 container can be used in a docker stack environment and not just docker-compose?
If it matters, I have docker-compose version 1.23.0-rc1, build 320e4819.
Thanks in advance.
This is not supported by swarm mode currently as the error message you've show and documentation identify. Personally I'd question whether you really want to have your database running in swarm mode. Docker does not migrate the volume for you, so you wouldn't see your data if rescheduled on another node.
You can follow the progress of getting this added to Swarm Mode in the github issues, there are several, including:
https://github.com/moby/moby/issues/24862
https://github.com/moby/moby/issues/25885
The hacky solution I've seen if you really need this run from swarm mode is to schedule a container with the docker socket mounted and docker binaries in the image, which then executes a docker run command directly against the local engine. E.g.:
version: "3.7"
services:
db2exp-wrapper:
image: docker:stable
volumes:
- /var/run/docker.sock:/var/run/docker.sock
command: docker run --rm --cap-add IPC_LOCK --cap-add IPC_OWNER -p 50000:50000 ... db2
I don't really recommend the above solution, sticking with docker-compose would likely be a better implementation for your use case. Downsides of this solution include only publishing the port on the single host, and potential security risks of anyone else with access to that docker socket.
Related
We have written our own logger that writes about 10 different log files: skl log, http request log, a separate log for each client, etc.
If you run the service through docker, is it possible to tell him to write these logs not inside himself, but in an external folder?
From what I've read, I've only realized so far that docker only logs output to the console, and in one shared file.
You can run a docker container with volumes where you can mention your expected log directory, settings file, app resource, etc.
Here is the straightforward way to create the docker container with volumes
docker create --name YOUR_SERVICE_NAME -p 80:80 -v /APP_DIR_OUT_SIDE_OF_CONTAINER/settings/appsettings.json:/app/appsettings.json -v /APP_DIR_OUT_SIDE_OF_CONTAINER/Logs:/app/Logs:z YOUR_DOCKER_IMAGE_REPO_URL:IMAGE_TAG
Below is the docker-compose sample that should be running a container with volumes.
version: '3.4'
services:
YOUR_SERVICE_NAME:
image: IMAGE_URL
container_name: CONTAINER_NAME
ports:
- "80:80"
volumes:
- /APP_DIR_OUT_SIDE_OF_CONTAINER/config/appsettings.json:/app/appsettings.json
- /APP_DIR_OUT_SIDE_OF_CONTAINER/logs:/app/Logs:z
restart: always
Also, you can get all possible ways to run the docker container with volumes from Use volumes
Given compose file
version: '3.8'
services:
whoami1:
image: containous/whoami
depends_on:
- whoami2
whoami2:
image: containous/whoami
when deployed to docker swarm docker stack deploy -c docker-compose.yaml test
services whoami1 and whoami2 seem to start in random order and ignore depends_on condition.
docker stack deploy -c docker-compose.yaml test
Creating network test_default
Creating service test_whoami1
Creating service test_whoami2
Does docker swarm support service startup sequencing via dependencies?
No, at least not built in.
Even with depends_on the whoami2 may not yet be ready to interact with whoami1 because it may need time to boot itself:
However, for startup Compose does not wait until a container is “ready” (whatever that means for your particular application) - only until it’s running. There’s a good reason for this.
https://docs.docker.com/compose/startup-order/
They hint at two possibilites to check if whoami2 is ready.
Use a tool such as wait-for-it, dockerize, or sh-compatible wait-for. These are small wrapper scripts which you can include in your application’s image to poll a given host and port until it’s accepting TCP connections.
And depends_on is indeed ignored for docker swarm:
There are several things to be aware of when using depends_on:
(...)
The depends_on option is ignored when deploying a stack in swarm mode with a version 3 Compose file.
https://docs.docker.com/compose/compose-file/#depends_on
I have a docker-compose.yml file which works with docker-compose up --build. My app works and everything is fine.
version: '3'
services:
myapp:
container_name: myapp
restart: always
build: ./myapp
ports:
- "8000:8000"
command: /usr/local/bin/gunicorn -w 2 -b :8000 flaskplot:app
nginx:
container_name: nginx
restart: always
build: ./nginx
ports:
- "80:80"
depends_on:
- myapp
But when I use docker stack deploy -c docker-compose.yml myapp, I get the following error:
Ignoring unsupported options: build, restart
Ignoring deprecated options:
container_name: Setting the container name is not supported.
Creating network myapp_default
Creating service myapp_myapp
failed to create service myapp_myapp: Error response from daemon: rpc error: code = InvalidArgument desc = ContainerSpec: image reference must be provided
any hints how I should "translate" the docker-compose.yml file to make it compatible with docker stack deploy?
To run containers in swarm mode, you do not build them on each swarm node individually. Instead you build the image once, typically on a CI server, push to a registry server (often locally hosted, or you can use docker hub), and specify the image name inside your compose file with an "image" section for each service.
Doing that will get rid of the hard error. You'll likely remove the build section of the compose file since it no longer applies.
Specifying "container_name" is unsupported because it would break the ability to scale or perform updates (a container name must be unique within the docker engine). Let swarm name the containers and reference your app on the docker network by it's service name.
Specifying "depends_on" is not supported because containers may be started on different nodes, and rolling updates/failure recovery may remove some containers providing a service after the app started. Docker can retry the failing app until the other service starts up, or preferably you configure an entrypoint that waits for the dependencies to become available with some kind of ping for a minute or two.
Without seeing your Dockerfile, I'd also recommend setting up a healthcheck on each image. Swarm mode uses this to control rolling updates and recover from application failures.
Lastly, consider adding a "deploy" section to your compose file. This tells swarm mode how to deploy and update your service, including how many replicas, constraints on where to run, memory and CPU limits and requirements, and how fast to update the service. You can define a restart policy here as well but I recommend against it since I've seen docker engines restarting containers that conflict with swarm mode deploying containers on other nodes, or even a new container on the same node.
You can see the full compose file documentation with all of these options here: https://docs.docker.com/compose/compose-file/
I've just set up a new Windows 10 development machine and so as to minimise the hassle of installs I've got various dev dependencies (Oracle, MongoDB, RabbitMQ, HAProxy, etc.) running under Docker using a docker-compose script.
I'd like to automatically start these containers on Windows logon but as yet I haven't figured out a way to do this; a simple script that executes docker-compose up -d in the correct directory should do it, but if it executes immediately on logon Docker hasn't yet started up so the script fails. Does anyone know how to programatically wait until docker is running?
To further elaborate on my comment i have done a little test with a webserver service, but it should work for any service, as long as you configure it the way you want it to behave.
Its quite easy to set this up using the following commands:
docker swarm init
Then for example a webserver
docker service create --name webserver --publish 80:80 httpd
Or even a database
docker service create --replicas 1 --name database --publish 1433:1433 -e "ACCEPT_EULA=y" -e "SA_PASSWORD=test" microsoft/mssql-server-linux
These will restart after a reboot and on fatal crashes automatically because of the requested amount of replicas (1 by default) that Docker swarm keeps alive for you.
Hopefully this can be of some help!
Turns out this is really easy to achieve via docker-compose using restart! Have changed out compose file as follows:
version: '2'
services:
rabbitmq:
image: rabbitmq:3.6-management
ports:
- "5672:5672"
- "15672:15672"
volumes:
- /var/lib/rabbitmq
restart: unless-stopped
This extra restart directive means that unless the container has been explicitly stopped it will start up with docker on logon/reboot. Tested and working!
I'm developing a server and its client simultaneously and I'm designing them in Docker containers. I'm using Docker Compose to link them up and it works just fine for production but I can't figure out how to make it work with a development workflow in which I've got a shell running for each one.
My docker-compose-devel.yml:
server:
image: node:0.10
client:
image: node:0.10
links:
- server
I can do docker-compose up client or even docker-compose run client but what I want is a shell running for both server and client so I can make rapid changes to both as I develop iteratively.
I want to be able to do docker-compose run server bash in one window and docker-compose run --no-deps client bash in another window. The problem with this is that no address for the server is added to /etc/hosts on the client because I'm using docker-compose run instead of up.
The only solution I can figure out is to use docker run and give up on Docker Compose for development. Is there a better way?
Here's a solution I came up with that's hackish; please let me know if you can do better.
docker-compose-devel.yml:
server:
image: node:0.10
command: sleep infinity
client:
image: node:0.10
links:
- server
In window 1:
docker-compose --file docker-compose-dev.yml up -d server
docker exec --interactive --tty $(docker-compose --file docker-compose-dev.yml ps -q server) bash
In window 2:
docker-compose --file docker-compose-dev.yml run client bash
I guess your main problem is about restarting the application when there are changes in the code.
Personnaly, I launch my applications in development containers using forever.
forever -w -o log/out.log -e log/err.log app.js
The w option restarts the server when there is a change in the code.
I use a .foreverignore file to exclude the changes on some files:
**/.tmp/**
**/views/**
**/assets/**
**/log/**
If needed, I can also launch a shell in a running container:
docker exec -it my-container-name bash
This way, your two applications could restart independently without the need to launch the commands yourself. And you have the possibility to open a shell to do whatever you want.
Edit: New proposition considering that you need two interactive shells and not simply the possibility to relaunch the apps on code changes.
Having two distinct applications, you could have a docker-compose configuration for each one.
The docker-compose.yml from the "server" app could contain this kind of information (I added different kind of configurations for the example):
server:
image: node:0.10
links:
- db
ports:
- "8080:80"
volumes:
- ./src:/src
db:
image: postgres
environment:
POSTGRES_USER: dev
POSTGRES_PASSWORD: dev
The docker-compose.yml from the "client" app could use external_links to be able to connect to the server.
client:
image: node:0.10
external_links:
- project_server_1:server # Use "docker ps" to know the name of the server's container
ports:
- "80:80"
volumes:
- ./src:/src
Then, use docker-compose run --service-ports service-name bash to launch each configuration with an interactive shell.
Alternatively, the extra-hosts key may also do the trick by calling the server app threw a port exposed on the host machine.
With this solution, each docker-compose.yml file could be commited in the repository of the related app.
First thing to mention, for development environment you want to utilize volumes from docker-compose to mount your app to the container when it's started (at the runtime). Sorry if you're already doing it and I mention this, but it's not clear from your definition of docker-compose.yml
To answer your specific question - start your containers normally, then when doing docker-compose ps, you'll see a name of your container. For example 'web_server' and 'web_client' (where web is the directory of your docker-compose.yml file or name of the project).
When you got name of the container you want to connect to, you can run this command to run bash exactly in the container that's running your server:
docker exec -it web_server bash.
If you want to learn more about setting up development environment for reasonably complex app, checkout this article on development with docker-compose