My container uses Vieux/sshfs to mount an external directory, but the transport endpoint is not connected error will appear after a period of time. How can I solve this problem? Here is my docker compose file
volumes:
airflow:
name: airflow_airflow
driver: vieux/sshfs
driver_opts:
allow_other: ""
sshcmd: root#xxxx:/root/airflow/dags
password: xxxxxxx
ServerAliveInterval: 5
ServerAliveCountMax: 3
Related
I ve a service running under a stack swarm :
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
de74ba4d48c1 myregistry/myApi:1.0 "java -Dfile.encodin…" 3 minutes ago Up 3 minutes 8300/tcp myApiCtn
As you can see , my service is running on the 8300 port.
The probleme is that when i run curl ; it seems to not reply:
[user#server home]$ curl http://localhost:8300/api/elk/batch
curl: (52) Empty reply from server
In another side if i ran my container manually (without stack and without swarm services )
(docker run ...)
-> curl works well
My docker-compose is the following :
---
version: '3.4'
services:
api-batch:
image: myRegistry/myImageApi
networks:
- net_common
- default
stdin_open: true
volumes:
- /opt/application/current/logs:/opt/application/current/logs
- /var/opt/data/flat/flf/:/var/opt/data/flat/flf/
tty: true
ports:
- target: 8300
published: 8300
protocol: tcp
deploy:
mode: global
resources:
limits:
memory: 1024M
placement:
constraints:
- node.labels.type == test
healthcheck:
disable: true
networks:
net_common:
external: true
Where my networks list is the following :
NETWORK ID NAME DRIVER SCOPE
17795bfee9ca bridge bridge local
0faecb070730 docker_gwbridge bridge local
51c34d251495 host host local
j2nnf26asn3k ingress overlay swarm
3all3tmn3qn9 net_common overlay swarm
b7alw2yi5fk9 srcd-current_default overlay swarm
Any suggestion to make it work under swarm service ?
I want to run filebeat as a sidecar container next to my main application container to collect application logs. I'm using docker-compose to start both services together, filebeat depending on the application container.
This is all working fine. I'm using a shared volume for the application logs.
However I would like to collect docker container logs (stdout JSON driver) as well in filebeat.
Filebeat provides a docker/container input module for this purpose. Here is my configuration. First part is to get the application logs. Second part should get docker logs:
filebeat.inputs:
- type: log
paths:
- /path/to/my/application/*.log.json
exclude_lines: ['DEBUG']
- type: docker
containers.ids: '*'
json.message_key: message
json.keys_under_root: true
json.add_error_key: true
json.overwrite_keys: true
tags: ["docker"]
What I don't like it the containers.ids: '*'. Here I would want to point filebeat to the direct application container, ignoring all others.
Since I don't know the container ID before I run docker-compose up starting both containers, I was wondering if there is a easy way to get the container ID from my application container in my filebeat container (via docker-comnpose?) to filter on this ID?
I think you may work around the problem:
first set all the logs from the contianer to a syslog:
driver: "syslog"
options:
syslog-address: "tcp://localhost:9000"
then configure filebeat to get the logs from that syslog server like this:
filebeat.inputs:
- type: syslog
protocol.udp:
host: "localhost:9000"
This is also not really answering the question, but should work as a solution as well.
The main idea is to use label within the filebeat autodiscovery filter.
Taken from this post: https://discuss.elastic.co/t/filebeat-autodiscovery-filtering-by-container-labels/120201/5
filebeat.yml
filebeat.autodiscover:
providers:
- type: docker
templates:
- condition:
contains:
docker.container.labels.somelabel: "somevalue"
config:
- type: docker
containers.ids:
- "${data.docker.container.id}"
output.console:
pretty: true
docker-compose.yml:
version: '3'
services:
filebeat:
image: docker.elastic.co/beats/filebeat:6.2.1
command: "--strict.perms=false -v -e -d autodiscover,docker"
user: root
volumes:
- ./filebeat.yml:/usr/share/filebeat/filebeat.yml
- /var/lib/docker/containers:/var/lib/docker/containers
- /var/run/docker.sock:/var/run/docker.sock
test:
image: alpine
command: "sh -c 'while true; do echo test; sleep 1; done'"
depends_on:
- filebeat
labels:
somelabel: "somevalue"
Upon logging in into my GitLab Registry, I receive the following output:
$ docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN $CI_REGISTRY
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
Error response from daemon: Get https://gitlab.MYDOMAIN:4501/v2/: denied: access forbidden
On my linux machine, using docker login with my private account details results in the same error.
Setup
docker Docker version 18.03.1-ce, build 9ee9f40
traefik for my GitLab docker container
opened port 4501 for GitLab Registry & passed docker variable GITLAB_OMNIBUS_CONFIG: | registry_external_url 'https://${GITLAB_DOMAIN}:${GITLAB_REGISTRY_PORT}'
registry url is reachable from outside (can call it from my browser with the default response UNAUTHORIZED - authentication required
.gitlab-ci.yml
image: docker
services:
- name: docker:dind
command: ["--insecure-registry=gitlab.MYDOMAIN:4501"]
before_script:
- docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
stages:
- build
build:
stage: build
script:
- docker build -t $CI_REGISTRY_IMAGE:$CI_COMMIT_REF_SLUG .
- docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_REF_SLUG
only:
- master
GitLab Runner configuration
concurrent = 1
check_interval = 0
[[runners]]
name = "olaf"
url = "https://gitlab.MYDOMAIN"
token = "xxxxxxxxxxxxx"
executor = "docker"
[runners.docker]
tls_verify = false
image = "ruby:2.1"
privileged = true
disable_cache = false
volumes = ["/cache"]
shm_size = 0
[runners.cache]
inside the CI Pipeline with docker info
$ docker info
Containers: 0
Running: 0
Paused: 0
Stopped: 0
Images: 0
Server Version: 18.05.0-ce
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: xxxxxxxxxxxxxxxx
runc version: xxxxxxxxxxxxxxx
init version: xxxxxxxxx
Security Options:
seccomp
Profile: default
Kernel Version: 4.9.0-6-amd64
Operating System: Alpine Linux v3.7 (containerized)
OSType: linux
Architecture: x86_64
CPUs: 8
Total Memory: 31.29GiB
Name: xxxxxxxxxxxxx
ID: xxxxxxxxxxxxxx
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
gitlab.MYDOMAIN:4501
127.0.0.0/8
Live Restore Enabled: false
Complete output from CI Pipeline
Running with gitlab-runner 10.8.0 (079aad9e)
on olaf 2467327f
Using Docker executor with image docker ...
Starting service docker:dind ...
Pulling docker image docker:dind ...
Using docker image sha256:1f44348b3ad523d5dc4ae7d53bd873879e06e0df2d686e9029a666945443ef42 for docker:dind ...
Waiting for services to be up and running...
Pulling docker image docker ...
Using docker image sha256:2232c0bbbb8cc9238eefc10721db5662156a2624bc7405dc1cade624dde9aaec for docker ...
Running on runner-2467327f-project-17-concurrent-0 via 4ba803c01c0b...
Fetching changes...
HEAD is now at c8dff7b Update .gitlab-ci.yml
warning: redirecting to https://gitlab.MYDOMAIN:443/kwinkel/imagetest.git/
From http://gitlab.MYDOMAIN/kwinkel/imagetest
c8dff7b..dc1b150 master -> origin/master
Checking out dc1b1501 as master...
Skipping Git submodules setup
$ docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN $CI_REGISTRY
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
Error response from daemon: Get https://gitlab.MYDOMAIN:4501/v2/: denied: access forbidden
ERROR: Job failed: exit code 1
inside the gitlab container /var/log/gitlab/registry/current
2018-06-02_19:27:03.50891 time="2018-06-02T19:27:03.50886204Z" level=warning msg="error authorizing context: authorization token required"
environment=production go.version=go1.9.2 http.request.host="registry.gitlab.MYDOMAIN:4567"
http.request.id=336c98a1-743a-47a5-9760-c20f5b77116a http.request.method=GET http.request.remoteaddr=- http.request.uri="/v2/"
http.request.useragent="docker/18.03.1-ce go/go1.9.5 git-commit/9ee9f40 kernel/4.9.0-6-amd64 os/linux arch/amd64
UpstreamClient(Docker-Client/18.03.1-ce \\(linux\\))" instance.id=1024a4ad-7a80-49c9-92c6-77cbcff85bf6 service=registry version=v2.6.2-2-g91c17ef
So, I found the answer myself.
The trick was to
set the external url to https://
set the omnibus nginx port to 80
disable https for omnibus
docker compose file / omnibus configuration
external_url 'https://${GITLAB_DOMAIN}'
nginx['listen_port'] = '80'
nginx['listen_https'] = false
and do the same for the registry...
registry_external_url 'https://registry.${GITLAB_DOMAIN}'
registry_nginx['listen_port'] = '80'
registry_nginx['listen_https'] = false
For solution: please take a look at https://github.com/kwinkel/Dockerfiles/tree/master/gitlab. Maybe it's helpful for you :)
I have been playing around with docker-in-docker (dind) setups and am running into a weird problem.
If I run a docker container separately inside dind and expose a port then I could connect to the port without any problems. For example, using the docker swarm visualizer inside dind:
/home/dockremap # docker run -d -p 8080:8080 dockersamples/visualizer:stable
/home/dockremap # wget localhost:8080
Connecting to localhost:8080 (127.0.0.1:8080)
index.html 100% |*********************** ....
However, if I run the same inside a swarm by deploying from a compose file it doesn't work.
Here is what my compose file looks like:
version: "3"
services:
visualizer:
image: dockersamples/visualizer:stable
ports:
- "8080:8080"
volumes:
- "/var/run/docker.sock:/var/run/docker.sock"
deploy:
placement:
constraints: [node.role == manager]
networks:
- webnet
networks:
webnet:
and the commands I run:
/home/dockremap # docker swarm init
/home/dockremap # docker stack deploy -c compose.yaml test
now when I do wget I get connection refused error:
/home/dockremap # wget localhost:8080
Connecting to localhost:8080 (127.0.0.1:8080)
wget: can't connect to remote host (127.0.0.1): Connection refused
Should doing this sort of thing in dind be able to work by default, or is there something I need to configure? I am using docker 17.03.1-ce on Windows and here is what I get when I run docker info in dind:
Containers: 2
Running: 1
Paused: 0
Stopped: 1
Images: 1
Server Version: 17.05.0-ce
Storage Driver: vfs
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Swarm: active
NodeID: wz2r6iuyqztg3ivyk9fwsn976
Is Manager: true
ClusterID: mshadtrs0b1oayva2vrquf67d
Managers: 1
Nodes: 1
Orchestration:
Task History Retention Limit: 5
Raft:
Snapshot Interval: 10000
Number of Old Snapshots to Retain: 0
Heartbeat Tick: 1
Election Tick: 3
Dispatcher:
Heartbeat Period: 5 seconds
CA Configuration:
Expiry Duration: 3 months
Node Address: 172.17.0.2
Manager Addresses:
172.17.0.2:2377
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 9048e5e50717ea4497b757314bad98ea3763c145
runc version: 9c2d8d184e5da67c95d601382adf14862e4f2228
init version: 949e6fa
Security Options:
seccomp
Profile: default
Kernel Version: 4.4.59-boot2docker
Operating System: Alpine Linux v3.5 (containerized)
OSType: linux
Architecture: x86_64
CPUs: 1
Total Memory: 987.1MiB
Name: 7e480e7313ae
ID: EL7P:NI2I:TOR4:I7IW:DPAB:WKYU:6A6J:NCC7:3K3E:6YVH:PYVB:2L2W
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
WARNING: bridge-nf-call-iptables is disabled
WARNING: bridge-nf-call-ip6tables is disabled
I have deployed two docker containers which hosts two REST services deployed in Jetty.
Container 1 hosts service 1 and it Listens to 7070
Container 2 hosts service 2 and it Listens to 9090
Endpoints:-
service1:
/ping
/service1/{param}
service2:
/ping
/service2/callService1
curl -X GET http://localhost:7070/ping [Works]
curl -X GET http://localhost:7070/service1/hello [Works]
curl -X GET http://localhost:9090/ping [Works]
I have configured the containers in such a way that:
http://localhost:9090/serivce2/callService1
calls
http://localhost:7070/service1/hello
This throws a connection refused exception. Here's the configuration I have.
docker-compose.yml
------------------
service1:
build: microservice/
ports:
- "7070:7070"
expose:
- "7070"
service2:
build: microservice_link/
ports:
- "9090:9090"
expose:
- "9090"
links:
- service1
service1 Dockerfile
-------------------
FROM localhost:5000/java:7
COPY ./target/service1.jar /opt
WORKDIR /opt
ENTRYPOINT ["java", "-jar", "service1.jar","7070"]
CMD [""]
service2 Dockerfile
-------------------
FROM localhost:5000/java:7
COPY ./target/service2.jar /opt
WORKDIR /opt
ENTRYPOINT ["java", "-jar", "service2.jar","9090"]
CMD [""]
docker info
-----------
root#LT-NB-108U:~# docker info
Containers: 3
Running: 2
Paused: 0
Stopped: 1
Images: 12
Server Version: 1.10.1
Storage Driver: aufs
Root Dir: /var/lib/docker/aufs
Backing Filesystem: extfs
Dirs: 28
Dirperm1 Supported: false
Execution Driver: native-0.2
Logging Driver: json-file
Plugins:
Volume: local
Network: null host bridge
Kernel Version: 3.13.0-48-generic
Operating System: Ubuntu precise (12.04.5 LTS)
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 3.47 GiB
Name: LT-NB-108U
ID: BS52:XURM:3SD7:TC3R:7YVA:ZBZK:CCL2:7AVC:RNZV:RBGW:2X2T:7C46
WARNING: No swap limit support
root#LT-NB-108U:~#
Question:-
I am trying to access the endpoint deployed in Container 1 from Container 2. However, I get a connection refused exception.
I tried exposing port 7070 in container 2. That didn't work.
curl http://service1:7070/
use - host1_name:inner_port_of_host1
That host is called "service1" in container2. Use that as the host name and the port is the inner port listener in service1's container.
If you have an express server on service1, listen on port 7070.