I have tests which run in a docker container. For this, I use docker-dind service, my .gitlab-ci:
image: "docker:17"
variables:
DOCKER_DRIVER: overlay2
services:
- docker:dind
- name: docker.elastic.co/elasticsearch/elasticsearch:5.5.2
alias: elasticsearch
command: [ "bin/elasticsearch", "-Expack.security.enabled=false", "-Ediscovery.type=single-node" ]
stages:
- test
before_script:
- apk --update add py2-pip python3 bash zip ansible openssh git docker-py curl
- pip3 install docker-compose
- docker info
- docker-compose --version
# Login to registry.gitlab.com
- docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN registry.gitlab.com
test:
script:
- curl "http://elasticsearch:9200" # this works
- docker-compose docker-compose.test.yml build --pull
- docker-compose docker-compose.test.yml run app
stage: test
My tests use ES for this I added ES service but I can't connect to ES cluster from my container where I run tests.
On my machine with runner when CI works I have:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
f5d9a64bbe8e 1ee5a99eba5f "gitlab-runner-build" 1 second ago Up Less than a second runner-b7dcaf73-project-2199705-concurrent-1-predefined-0
c85c49d35946 ca27036dd5e7 "bin/elasticsearch -…" 16 seconds ago Up 15 seconds 9200/tcp, 9300/tcp runner-b7dcaf73-project-2199705-concurrent-1-docker.elastic.co__elasticsearch__elasticsearch-1
57472d0300ad 85e924caedbd "dockerd-entrypoint.…" 17 seconds ago Up 16 seconds 2375/tcp runner-b7dcaf73-project-2199705-concurrent-1-docker-0
598c019aa28c - a container with a runner, I can enter to this container run curl "http://elasticsearch:9200" and it works
57472d0300ad - dind container, right? I can enter to this container but curl "http://elasticsearch:9200" doesn't work, docker ps shows:
/ # docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
252f0588a41d backend_app "make _inside_docker…" 6 seconds ago Up 5 seconds 8080/tcp backend_app_run_116d12907320
cd0ebb2f1d2d postgres:9.6 "docker-entrypoint.s…" 7 seconds ago Up 6 seconds 5432/tcp backend_postgresql_1
How I can connect from my container with tests (252f0588a41d) to container with ES?
Thanks.
Related
In Docker Swarm mode, how can I restart a single global service? Is it even possible? I know you can scale replicated services to zero then back to 1+, but there doesn't appear to be any documentation on how to have the same effect with global services.
I am updating my SSL certificate so would like to just restart our reverse proxy instead of restarting our entire app (via restarting the docker service).
The docs just mention you cannot scale global services:
The scale command enables you to scale one or more replicated services either up or down to the desired number of replicas. This command cannot be applied on services which are global mode.
You can force a rolling update of a service, either globally scheduled or replicated using docker service update --force ${service_name}. Here's an example compose file:
version: '3'
services:
busybox-global:
image: busybox
command: tail -f /dev/null
deploy:
mode: global
busybox-replicated:
image: busybox
command: tail -f /dev/null
deploy:
replicas: 2
Verify it has started:
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
917aefdc910b busybox:latest "tail -f /dev/null" 50 seconds ago Up 31 seconds sched_busybox-global.q44zx0s2lvu1fdduk800e5ini.hzn6jnzh7x539timamphzzw8a
7187fbbde0da busybox:latest "tail -f /dev/null" About a minute ago Up 31 seconds sched_busybox-replicated.1.i4nm7lpr1spmf0aorh1dtcqrc
f04a0062b088 busybox:latest "tail -f /dev/null" About a minute ago Up 31 seconds sched_busybox-replicated.2.oc6zn0ziqg9wyzofokek8eb24
$ docker service ls
ID NAME MODE REPLICAS IMAGE PORTS
gto0d5a6betb sched_busybox-global global 1/1 busybox:latest
yfq5mne0qhtj sched_busybox-replicated replicated 2/2 busybox:latest
$ docker service ps sched_busybox-global
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
hzn6jnzh7x53 sched_busybox-global.q44zx0s2lvu1fdduk800e5ini busybox:latest bmitch-asusr556l Running Running 49 seconds ago
Force the rolling update:
$ docker service update --force sched_busybox-global
sched_busybox-global
overall progress: 1 out of 1 tasks
q44zx0s2lvu1: running [==================================================>]
verify: Service converged
$ docker service ps sched_busybox-global
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
zcfocrfjvvux sched_busybox-global.q44zx0s2lvu1fdduk800e5ini busybox:latest bmitch-asusr556l Running Running 7 seconds ago
hzn6jnzh7x53 \_ sched_busybox-global.q44zx0s2lvu1fdduk800e5ini busybox:latest bmitch-asusr556l Shutdown Shutdown 10 seconds ago
$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
3c5fe0f79e3d busybox:latest "tail -f /dev/null" About a minute ago Up About a minute sched_busybox-global.q44zx0s2lvu1fdduk800e5ini.zcfocrfjvvuxz6tkge0pn0bq2
917aefdc910b busybox:latest "tail -f /dev/null" 3 minutes ago Exited (137) About a minute ago sched_busybox-global.q44zx0s2lvu1fdduk800e5ini.hzn6jnzh7x539timamphzzw8a
7187fbbde0da busybox:latest "tail -f /dev/null" 3 minutes ago Up 2 minutes sched_busybox-replicated.1.i4nm7lpr1spmf0aorh1dtcqrc
f04a0062b088 busybox:latest "tail -f /dev/null" 3 minutes ago Up 2 minutes sched_busybox-replicated.2.oc6zn0ziqg9wyzofokek8eb24
The same would have worked if I forced an update to the replicated service.
I have this YML file
docker-compose-testing.yml
with docker compose configuration:
version: '3'
services:
nginx_testing:
image: MY_SERVER_IP:5000/lens/nginx_testing:${VERSION}
volumes:
- certs:/etc/letsencrypt
- certs-data:/data/letsencrypt
ports:
- 80:80
- 443:443
depends_on:
- ws_server
- translator
- auth
ws_server:
image: MY_SERVER_IP:5000/lens/ws_server:${VERSION}
worker:
image: MY_SERVER_IP:5000/lens/worker:${VERSION}
depends_on:
- ws_server
translator:
image: MY_SERVER_IP:5000/lens/translator:${VERSION}
auth:
image: MY_SERVER_IP:5000/lens/auth:${VERSION}
volumes:
- auth-data:/usr/src/app/data
volumes:
certs:
certs-data:
auth-data:
Normally, I use this command to apply the above configuration:
export VERSION=578d8de && envsubst < docker-compose-testing.yml | docker-compose -f - pull && envsubst < docker-compose-testing.yml | docker-compose -f - -p PROJECT_NAME up -d --no-build --scale worker=5
Now, when I execute this command (above) the console show this error:
Pulling translator (MY_SERVER_IP:5000/lens/translator:578d8de)...
ERROR: manifest for MY_SERVER_IP:5000/lens/translator:578d8de not found
The response for one similar question below says that the tag do not exists:
Error response from daemon: manifest for ibmblockchain/fabric-peer:latest not found
But when I list the images with command:
docker images | grep 578d8de
Console show this output proving that the tag exists:
MY_SERVER_IP:5000/lens/auth 578d8de 8103c4d63870 2 hours ago 195MB
MY_SERVER_IP:5000/lens/nginx_testing 578d8de 578d8dead150 4 hours ago 235MB
MY_SERVER_IP:5000/lens/translator 578d8de e9eb25fa0aef 5 hours ago 185MB
MY_SERVER_IP:5000/lens/ws_server 578d8de 92b1d1a4cee9 5 hours ago 177MB
MY_SERVER_IP:5000/lens/worker 578d8de 22a935deba5c 7 days ago 175MB
Some extra details:
The server (MY_SERVER_IP) has a docker registry listening on 5000 port.
The image with version tag 578d8de was uploaded to the server not with registry, but with the "docker save" and "scp" command on the dev machine, and "docker load" on the server.
Any idea why this error is occurring?
This error is happenning when I send image with scp command to server and load the image to docker.
How I don't use docker push to the registry, the image don't exists in registry.
So when docker-compose pull execute, don't find image on registry and dispatch the error.
I am trying to run container with host network mode and it does not work:
version: '2'
services:
tests:
container_name: my_container
build: .
network_mode: "host"
I got 'Connection refused' when do curl -v http://127.0.0.1:7878
(this is my server which is up and running locally)
Docker version 17.09.0-ce, build afdb6d4
docker-compose version 1.16.1, build 6d1ac219
Update
docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES d1e3ac55cfaa app_cont "/bin/sh -c 'sh -c..." 17 seconds ago Up 18 seconds my_container
sudo netstat -plant
netstat: lant: unknown or uninstrumented protocol
Host OS: OSX 10.12
I"m following the tutorial from https://docs.docker.com/get-started/part3/ on a Mac. This tutorial is to setup a Docker service using swarm node. However, I'm getting Empty reply from server if I goto http://localhost. I have verified that port 80 is with a Docker process, and the Docker container are running as well.
Mac-Machine: docker stack deploy -c docker-compose.yml getstartedlab
Creating network getstartedlab_webnet
Creating service getstartedlab_web
Mac-Machine:docker user1$ lsof -i tcp:80
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
com.docke 7592 user1 44u IPv4 0xfc021b13bc914389 0t0 TCP *:http (LISTEN)
Mac-Machine:docker user1$ curl http://localhost
curl: (52) Empty reply from server
Mac-Machine:docker user1$ docker service ls
ID NAME MODE REPLICAS IMAGE
w4dghr7jcpca getstartedlab_web replicated 5/5 dockhub-user1/get-started:part1
Mac-Machine:docker user1$ docker service ps w4dghr7jcpca
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
4oykrq8ge8yl getstartedlab_web.1 dockhub-user1/get-started:part1 moby Running Running about a minute ago
ba1n3m1pis2f getstartedlab_web.2 dockhub-user1/get-started:part1 moby Running Running about a minute ago
kmy8n4tm0n44 getstartedlab_web.3 dockhub-user1/get-started:part1 moby Running Running about a minute ago
cyeyozw6u8x7 getstartedlab_web.4 dockhub-user1/get-started:part1 moby Running Running about a minute ago
0evm9skw7p44 getstartedlab_web.5 dockhub-user1/get-started:part1 moby Running Running about a minute ago
Mac-Machine:docker user1$ docker container ls
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
5223c52b2014 dockhub-user1/get-started#sha256:2d3934a04a4aecc453652678489b2d96ce8d3dc5457aa8afdaeb71dbeff236ff "python app.py" 2 minutes ago Up About a minute 80/tcp getstartedlab_web.1.4oykrq8ge8ylw3ilbufxdp4t0
910b7b7521b3 dockhub-user1/get-started#sha256:2d3934a04a4aecc453652678489b2d96ce8d3dc5457aa8afdaeb71dbeff236ff "python app.py" 2 minutes ago Up About a minute 80/tcp getstartedlab_web.4.cyeyozw6u8x7j1zy1k82dugrn
d3ebd24cfe9a dockhub-user1/get-started#sha256:2d3934a04a4aecc453652678489b2d96ce8d3dc5457aa8afdaeb71dbeff236ff "python app.py" 2 minutes ago Up 2 minutes 80/tcp getstartedlab_web.5.0evm9skw7p44npujg6nbhckmy
ba29ffbdf2ce dockhub-user1/get-started#sha256:2d3934a04a4aecc453652678489b2d96ce8d3dc5457aa8afdaeb71dbeff236ff "python app.py" 2 minutes ago Up About a minute 80/tcp getstartedlab_web.2.ba1n3m1pis2flttytx87nucvb
6d8af1744b75 dockhub-user1/get-started#sha256:2d3934a04a4aecc453652678489b2d96ce8d3dc5457aa8afdaeb71dbeff236ff "python app.py" 2 minutes ago Up About a minute 80/tcp getstartedlab_web.3.kmy8n4tm0n44jgb1tc34qgeww
Here is the docker-compose.yml
version: "3"
services:
web:
# replace username/repo:tag with your name and image details
image: dockhub-user1/get-started:part1
deploy:
replicas: 5
resources:
limits:
cpus: "0.1"
memory: 50M
restart_policy:
condition: on-failure
ports:
- "80:80"
networks:
- webnet
networks:
webnet:
Image dockhub-user1/get-started:part1 was created using the following Docker file.
# Use an official Python runtime as a parent image
FROM python:2.7-slim
# Set the working directory to /app
WORKDIR /app
# Copy the current directory contents into the container at /app
ADD . /app
# Install any needed packages specified in requirements.txt
RUN pip install -r requirements.txt
# Make port 80 available to the world outside this container
EXPOSE 80
# Define environment variable
ENV NAME World
# Run app.py when the container launches
CMD ["python", "app.py"]
Here is the app.py file
from flask import Flask
from redis import Redis, RedisError
import os
import socket
# Connect to Redis
redis = Redis(host="redis", db=0, socket_connect_timeout=2, socket_timeout=2)
app = Flask(__name__)
#app.route("/")
def hello():
try:
visits = redis.incr("counter")
except RedisError:
visits = "<i>cannot connect to Redis, counter disabled</i>"
html = "<h3>Hello {name}!</h3>" \
"<b>Hostname:</b> {hostname}<br/>" \
"<b>Visits:</b> {visits}"
return html.format(name=os.getenv("NAME", "world"), hostname=socket.gethostname(), visits=visits)
if __name__ == "__main__":
app.run(host='0.0.0.0', port=80)
After spending a whole afternoon debugging the issue, I found out that it was caused by Kubernetes. Stopping it from Docker Desktop resolved the issue.
There are two tasks: run app container, run almost the same deploy-app container. The differences for them, for example, that deploy container does not have port sharing.
So, I made configs for this tasks...
./dockerfiles/base.yml:
app:
net: docker_internal_net
environment:
APPLICATION_SERVER: "docker"
./dockerfiles/base.run.yml:
app:
container_name: project-app
# set the build context to the project root
build: ..
volumes:
- /var/log/project/nginx:/var/log/nginx
- /var/log/project/php-fpm:/var/log/php5-fpm
- ..:/var/www/project
./dockerfiles/dev/run.yml:
app:
dockerfile: ./dockerfiles/dev/run-app/Dockerfile
ports:
- "80:80"
- "22:22"
environment:
DEV_SSH_PUBKEY: "$SSH_PUBLIC_KEY"
APPLICATION_PLATFORM: "dev"
./dockerfiles/dev/build.yml:
app:
container_name: project-app-deploy
# set the build context to the project root
build: ../..
dockerfile: ./dockerfiles/dev/build-app/Dockerfile
environment:
APPLICATION_PLATFORM: "dev"
volumes:
- ../..:/var/www/project
So I can tun the app container like this:
$ docker-compose -f ./dockerfiles/base.yml -f ./dockerfiles/base.run.yml -f ./dockerfiles/dev/run.yml up -d app
Creating project-app
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
dae45f3f2c42 dockerfiles_app "/sbin/my_init" 2 seconds ago Up 1 seconds 0.0.0.0:2223->22/tcp, 0.0.0.0:8081->80/tcp project-app
Everything okay. But if then I trying to run deploy-app container, I will receive this message:
$ docker-compose -f ./dockerfiles/base.yml -f ./dockerfiles/dev/build.yml up -d app
Recreating project-app
WARNING: Service "app" is using volume "/var/www/project" from the previous container. Host mapping ".." has no effect. Remove the existing containers (with `docker-compose rm app`) to use the host volume mapping.
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
53059702c09b dockerfiles_app "/sbin/my_init" 6 seconds ago Up 4 seconds 22/tcp, 80/tcp project-app-deploy
This is because both of them are shared one local directory? But why I can run deploy-app container manually without docker-compose?
$ docker run -d --net docker_internal_net -e APPLICATION_SERVER=docker -e APPLICATION_PLATFORM=dev --name project-app-deploy -v ..:/var/www/project mybaseimage
86439874b8df561f529fde0d1e31824d70dc7e2a2377cd529331a2d7fcb00467
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
86439874b8df mybaseimage "/sbin/my_init" 4 seconds ago Up 3 seconds 22/tcp, 80/tcp project-app-deploy
40641f02a09b dockerfiles_app "/sbin/my_init" 2 minutes ago Up 2 minutes 0.0.0.0:2223->22/tcp, 0.0.0.0:8081->80/tcp project-app
I've solved my problem with the extend command in that way:
1) making changes into my ./dockerfiles/dev/build.yml file:
deploy-app:
extends:
file: ../base.yml
service: app
container_name: project-app-deploy
# set the build context to the project root
build: ../..
dockerfile: ./dockerfiles/dev/build-app/Dockerfile
environment:
APPLICATION_PLATFORM: "dev"
volumes:
- ../..:/var/www/project
2) run my deploy app container so:
$ docker-compose -f ./dockerfiles/dev/build.yml up -d deploy-app
Building deploy-app
...
Successfully built 74750fe274c6
Creating lovetime-app-deploy
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
9bb2af79ffaa dev_deploy-app "/sbin/my_init" 5 seconds ago Up 4 seconds 22/tcp, 80/tcp project-app-deploy
812b8824f1f0 dockerfiles_app "/sbin/my_init" 3 minutes ago Up 3 minutes 0.0.0.0:2223->22/tcp, 0.0.0.0:8081->80/tcp project-app
$ docker inspect -f '{{ .Mounts }}' project-app-deploy
[{ ...... /var/www/project rw true}]
Update:
According to documentation, this command is not supported in newer compose versions:
The extends keyword is supported in earlier Compose file formats up to Compose file version 2.1 (see extends in v1 and extends in v2), but is not supported in Compose version 3.x.