I am trying to run a few docker containers on GCP VMs. As of today, when I run a docker-compose, my ssh connection breaks. This was not happening as of this morning. The only thing that changed was I did a sudo apt update && sudo apt upgrade.
An example of a docker-compose file that breaks the connection:
version: '3'
services:
mongo:
container_name: mongo
image: mongo
expose:
- "27017"
ports:
- "27017:27017"
I'm on Ubuntu 5.13.0-1030-gcp #36~20.04.1-Ubuntu SMP Fri Jun 3 15:33:42 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux.
docker-compose version 1.25.0
Docker version 20.10.17, build 100c701
and I'm using BuildKit for my Docker and docker-compose builds.
I'm not sure how to debug this issue, any suggestions would be appreciated. Will update the post with more info if needed.
Check your dmesg to see if the kernel crashed.
Probably related to https://www.mail-archive.com/kernel-packages#lists.launchpad.net/msg482001.html
in our case it was an internal ip address space exhaustion
because docker switches afterwards to the 192.168.. network
and blocks external traffic
to fix - shutdown all docker containers
purge all docker networks
add or append to docker daemon.json
e.g. vi /etc/docker/daemon.json for 512 networks
{
"default-address-pools":[
{"base":"172.20.0.0/16","size":24},
{"base":"172.21.0.0/16","size":24}
]
}
Do not update your kernel,new kernel will influence docker process
Related
I'm trying to establish a Jenkins Pipeline, that's able to build docker images. But I ran into the problem docker: not found after executing the pipeline. The Jenkinsfile has the following content:
pipeline {
agent { dockerfile true }
stages {
stage('Test') {
steps {
sh 'docker --version '
}
}
}
}
It's a simple script to get things started. But it seems that the dockerized Jenkins installation can't find a suitable docker installation to use.
The required plugins (Docker and Docker pipeline) are installed and a global docker installation configuration is present. But the error keeps going.
Jenkins setup is done by using this docker-compose:
version: '3.1'
networks:
docker:
volumes:
jenkins-data:
jenkins-docker-certs:
services:
jenkins:
image: jenkins/jenkins:lts
restart: always
networks:
- docker
ports:
- 8090:8080
- 50000:50000
tty: true
volumes:
- jenkins-data:/var/jenkins_home
- jenkins-docker-certs:/certs/client:ro
- $HOME:/home
environment:
- DOCKER_HOST=tcp://docker:2376
- DOCKER_CERT_PATH=/certs/client
- DOCKER_TLS_VERIFY=1
dind:
image: docker:dind
privileged: true
restart: always
networks:
docker:
aliases:
- docker
ports:
- 2376:2376
tty: true
volumes:
- jenkins-data:/var/jenkins_home
- jenkins-docker-certs:/certs/client
- $HOME:/home
environment:
- DOCKER_TLS_CERTDIR=/certs
After reading some more posts about that issue and following the official Jenkins doc, I thought that for this purpose docker:dind is used. Maybe I miss some important configurations here? When launching the docker:dind container, the log states the following warning message: could not change group /var/run/docker.sock to docker: group docker not found, but the group exists and I'm able to run docker commands without specifying sudo. (Followed the official docker post-installation steps)
Another problematic point right now is, that Jenkins can't persist configuration data in general or pipeline related stuff. After restarting the machine I have to go through the wizard every single time and I don't know why.
Did someone suffer similar problems?
Many thanks in advice!
Your docker-compose file is correct, you just need to add a volume in the jenkins container :
- /usr/bin/docker:/usr/bin/docker
You have also a lot of configuration not required, you can check this link to see others possible configurations. You use actually the Solution 3 and you can switch to this docker-compose file.
For volumes, they should be persisted since they are declared in the volume section. You can try to use external volumes if needed.
Fast forward one year and I've run into analogous problem, only with mismatched GLIBC versions, as described here.
I solved it by upgrading GLIBC version in the Jenkins container to 2.35 (as shipped with Ubuntu Jammy on the host). To achieve this I had to build my own Jenkins container based on ubuntu:jammy and JDK 17, using a template from the official Debian-based one (sourced from here). Now GLIBC versions agree, and docker-in-docker Jenkins builds can be made using docker installed on a host with Ubuntu Jammy:
$ ldd --version
ldd (Ubuntu GLIBC 2.35-0ubuntu3.1) 2.35
# vs.
$ docker run --rm -it mirekphd/jenkins-jdk17-on-ubuntu-2204:2.374 ldd --version
ldd (Ubuntu GLIBC 2.35-0ubuntu3.1) 2.35
Feel free to use this container (best served with the latest tag), as I will have to maintain it for our own in-house use, setting its builds as one of... Jenkins pipelines (bootstrap problem notwithstanding). It will be a Docker-in-Docker Jenkins-in-Jenkins pipeline:)
I have been using a docker compose file to run a development environment for a microservices architecture that in production is deployed to kubernetes, so I use links to map k8s provided domain names to the compose service names. e.g
links:
- "kafka:kafkadc-c44b-44.default.svc.cluster.local"
All works well when using docker-compose. But when using the new docker compose (note no hyphen), the containers are unable to comminicate with eachother. Kafka connections fail. HTTP requests fail.
Are there any changes between the docker cli version of compose, and the old docker-compose command. I cann't see a version of docker compose without the hypen, but with I have the following version information:
docker-compose version 1.29.0, build 07737305
docker-py version: 5.0.0
CPython version: 3.9.0
OpenSSL version: OpenSSL 1.1.1h 22 Sep 2020
You can use hostname to specify a hostname for the container within its network or you can use an alias if you need a hostname per network.
e.g.
db:
image: mysql:5.7
container_name: mycontainername
hostname: myhostname
networks:
default:
aliases:
myalias
I am getting this error when docker-compose up on one of the containers only.
exec: "com.docker.cli": executable file not found in $PATH
The terminal process "/bin/zsh '-c', 'docker logs -f f6557b5dd19d9b2bc5a63a840464bc2b879d375fe72bc037d82a5358d4913119'" failed to launch (exit code: 1).
I uninstalled and reinstalled docker desktop#2.3.0.5 on Mac
docker-compose build from scratch
other containers are running
I get the above error.
It used to be running. I am not sure why this is happening. I know that I upgraded docker from I think 2.3
also I think I received an update on my mac
Dockerfile
FROM tiangolo/uvicorn-gunicorn:python3.8
COPY requirements.txt /app/
RUN pip install -r requirements.txt
COPY ./app /app/app
#COPY config.py /app/app/
docker-compose.yml
version: "3"
services:
postgresql:
container_name: postgresql
image: postgres:12
ports:
- "5433:5432"
environment:
- POSTGRES_USER=${POSTGRES_USER}
- POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
- POSTGRES_DB=${POSTGRES_DB}
volumes:
- ./postgres-data:/var/lib/postgresql/data
fastapi:
build:
context: ./fastapi/
dockerfile: Dockerfile
volumes:
- ./fastapi/app/imgs:/app/app/imgs
ports:
- "1001:80"
depends_on:
- postgresql
env_file:
- .env
pgadmin:
container_name: pgadmin
image: dpage/pgadmin4
environment:
- PGADMIN_DEFAULT_EMAIL=pgadmin4#pgadmin.org
- PGADMIN_DEFAULT_PASSWORD=admin
ports:
- "5050:80"
depends_on:
- postgresql
solr:
build:
context: ./solr/
dockerfile: Dockerfile
restart: always
ports:
- "8983:8983"
volumes:
- data:/var/solr
volumes:
data:
update:
It worked when I downgraded to docker desktop 2.3.0.4
Updated Answer:
Since VSCode Docker 1.14.0 you can now set the Docker executable path in the settings, which should help in most cases.
Old Answer (Option was removed from Docker Desktop):
The Desktop Docker Version 2.4.0.0 is working for me after I did deactivate the feature Enable cloud experience. You can find it under Preferences --> Command Line.
If you are still experience the problem, you may try a clean remove and install of Docker and also make sure that Docker is actually running, see other possible solution(s) here.
History of GitHub Issues:
https://github.com/docker/for-mac/issues/4956
https://github.com/microsoft/vscode-docker/issues/2366
https://github.com/microsoft/vscode-docker/issues/2578
https://github.com/microsoft/vscode-docker/issues/2894
Status (2021-06-22): VSCode Version 1.57.0 seems to have fixed the issue again.
You might get the following error message simply because you did not start Docker just yet
exec: "com.docker.cli": executable file not found in $PATH
In my case the problem was I had installed and then crudely removed the docker compose cli. This resulted in the above error to start popping up.
I got the compose CLI back using instructions from https://docs.docker.com/cloud/ecs-integration/#install-the-docker-compose-cli-on-linux and running (as root):
curl -L https://raw.githubusercontent.com/docker/compose-cli/main/scripts/install/install_linux.sh | sh
This fixed it for me.
Note: I would not recommend installing docker-compose cli to fix this issue, but to share some insights in case this is applicable to you as well.
Ensure that docker CLI is installed not just docker desktop on Linux. YOu can install it using:
sudo apt install docker.io
Update: The "cloud experience" no longer exists even as an experimental feature in Docker Desktop v3.2.1. This should no longer be an issue.
If you continue to see this problem on a newer version, you will need to downgrade to Docker v3.1.0, disable the cloud experience feature, then upgrade to the newest version.
Had the exact same issue. Was fixed after starting the upgraded docker first, then running this command.
dostarr#DOSTARR-M-38LF ~ % docker run busybox
exec: "com.docker.cli": executable file not found in $PATH
<started docker>
dostarr#DOSTARR-M-38LF ~ % docker run busybox
dostarr#DOSTARR-M-38LF ~ %
I had the same problem when trying to run minikube tunnel, and since I didn't want to re-install anything, I ended up running it from the docker bin path (on Windows it's in 'C:\Program Files\Docker\Docker\resources\bin') and it worked.
An alternative to Docker Desktop is colima, container runtimes on macOS (and Linux) with minimal setup.
# Homebrew
brew install colima docker
colima start
Now, you can use the docker commands as before.
For docker compose commands, you have to install:
brew install docker-compose
if already have installed docker, it may not have started. So type in terminal,"docker run -d -p 80:80 docker/getting-started" and it should solve the issue.
Could you advise how to make run freeradius using dockercompose?
Compose file here which is stop automatically in a sec.
version: '3'
services:
freeradius:
image: freeradius/freeradius-server
restart: always
volumes:
- ./freeradius:/etc/freeradius
ports:
- "1812-1813:1812-1813/udp"
volumes:
freeradius:
But when I run it with docker directly, then it runs
docker run --name my-radius -i -t freeradius/freeradius-server /bin/bash
In here, it display configuration file,
root#945f7bcb3520:/# ls /etc/freeradius
README.rst clients.conf experimental.conf huntgroups mods-config panic.gdb
proxy.conf sites-available templates.conf users
certs dictionary hints mods-available mods-enabled policy.d
radiusd.conf sites-enabled trigger.conf
but then volume folder, ./freeradius don't include any conf file.
So, how can make it work properly in general?
I have gotten a similar setup up and running with my config being loaded. All my configuration has been done according to the docker hub documentation. Here is my docker-compose.yml and Dockerfile for reference.
(I am aware that I could probably avoid the Dockerfile completely, but the advantage of this is that the Dockerfile is basically 1:1 to the official documentation..)
run docker-compose up -d to run it. Both files should be in the parent directory of raddb
Dockerfile
FROM freeradius/freeradius-server:latest
COPY raddb/ /etc/raddb/
EXPOSE 1812 1813
docker-compose.yml
version: '2.2'
services:
freeradius:
build:
context: .
container_name: freeradius
ports:
- "1812-1813:1812-1813/udp"
restart: always
You don't display your Dockerfile here. But I can guess that you are running a command in the Dockerfile that doesn't persist. It works from the command line, because /bin/bash will persist until you exit.
I have had this problem a couple times recently.
Your command to run the container directly
docker run --name my-radius -i -t freeradius/freeradius-server /bin/bash
is not equivalent to your dockercompose setup.
You are not mounting the config directory (also you are not publishing the container ports to the host - this will prevent you from accessing freeradius from outside a container).
I assume if you run your docker container mounting the volume
docker run --name my-radius -v ./freeradius:/etc/freeradius -i -t freeradius/freeradius-server /bin/bash
it will not work either.
For me, it didn't work when I tried to replace the whole config directory with a volume mount.
I had to mount components of the configuration individually. E.g.
-v ./freeradius/clients.conf:/etc/freeradius/clients.conf
Apparently, when you replace the whole directory something fails when starting freeradius. Excerpt from radius.log when mounting the whole config directory:
Fri Jan 13 10:49:22 2023 : Info: Debug state unknown (cap_sys_ptrace capability not set)
Fri Jan 13 10:49:22 2023 : Info: systemd watchdog is disabled
Fri Jan 13 10:49:22 2023 : Error: rlm_preprocess: Error reading /etc/freeradius/mods-config/preprocess/huntgroups
Fri Jan 13 10:49:22 2023 : Error: /etc/freeradius/mods-enabled/preprocess[13]: Instantiation failed for module "preprocess"
I'm trying to docker-compose an existing application. Now I struggle on centos on a basic task.
Given docker-compose.yml:
version: '3.8'
services:
nginx:
image: nginx:latest
ports:
- "1024:80"
When run with docker-compose up localhost:1024 is not accessible. When I add the started container to the default bridge network (docker network connect <id of bridge> <id of container>), localhost:1024 is accessible.
When run with docker run -p 1024:80 nginx:latest localhost:1024 is accessible.
docker-compose version 1.26.2, build eefe0d31
Docker version 19.03.12, build 48a66213fe
CentOS Linux release 7.6.1810
When attempting to docker-compose up the same file on macOS localhost:1024 is accessible.
EDIT #1
docker ps PORTS output: 0.0.0.0:1024->80/tcp
new interface added ifconfig: br-da0abb61196d
Same behavior on different ports e.g. 8080