I'm using docker context to run container on remote host. When I'm running the following command I'm expecting to have my container get rebuild and run on remote host:
docker-compose --context REMOTE up --build -d
The behaviour I see is that container is rebuild locally and on remote host the old version of docker image is running. When I'm checking docker image ls on remote host I see that image wasn't rebuilt and outdated version is running.
When I'm explicitly removing container on remote machine it's get rebuild, although it seems it get rebuild twice, once on local machine and once on remote.
Is there any way to make docker-compose to always rebuild image on remote machine?
I was using an old version of docker-compose. In the latest version the command looks like DOCKER_CONTEXT=REMOTE docker-compose up --build -d and working as expected.
Related
I get this error when trying to start docker-compose by Docker Desktop v4.12.0.
docker-compose up runs in terminal normally.
Settings->General->Enable Docker Compose V1/V2 compatibility mode (check and restart Docker desktop)
I have a linux vm on which I installed docker. I have several docker containers with the different programs I have to use. Here's my architecture:
Everything is working fine except for the red box.
What I am trying to do is to dynamically provide a jenkins docker-in-docker agent with the cloud functionality in order to build my docker images and push them to the docker registry I set up.
I have been looking for documentation to create a docker in docker container and I found this:
https://jpetazzo.github.io/2015/09/03/do-not-use-docker-in-docker-for-ci/
This article states that in order to avoid problems with my main docker installation I have to create a volume:
-v /var/run/docker.sock:/var/run/docker.sock
I tested my image locally and I have no problem to run
docker run -d -v --name test /var/run/docker.sock:/var/run/docker.sock
docker exec -it test /bin/bash
docker run hello-world
The container is using the linux vm docker installation to build and run the docker images so everything is fine.
However, I face problems when it comes to the jenkins docker cloud configuration.
From what I gather, since the #826 build, the docker jenkins plugin has change its syntax for volumes.
This is the configuration I tried:
And the error message I have when trying to launch the agent:
Reason: Template provisioning failed.
com.github.dockerjava.api.exception.BadRequestException: {"message":"create
/var/run/docker.sock: \"/var/run/docker.sock\" includes invalid characters for a local
volume name, only \"[a-zA-Z0-9][a-zA-Z0-9_.-]\" are allowed. If you intended to pass a
host directory, use absolute path"}
I also tried that configuration:
Reason: Template provisioning failed.
com.github.dockerjava.api.exception.BadRequestException: {"message":"invalid mount config for type \"volume\": invalid mount path: './var/run/docker.sock' mount path must be absolute"}
I do not get what that means as on my linux vm the docker.sock absolute path is /var/run/docker.sock, and it is the same path inside the docker in docker I ran locally...
I tried to check the source code to find what I did wrong but it's unclear what the code is doing for me (https://github.com/jenkinsci/docker-plugin/blob/master/src/main/java/com/nirima/jenkins/plugins/docker/DockerTemplateBase.java, from row 884 onward), I also tried with backslashes, etc. Nothing worked.
Has anyone any idea what is the expected syntax in that configuration panel for setting up a simple volume?
Change the configuration to this:
type=bind,source=/var/run/docker.sock,destination=/var/run/docker.sock
it is not a volume, it is a bind type.
This worked for me
type=bind,source=/sys/fs/cgroup,target=/sys/fs/cgroup,readonly
I am getting while running local docker registry on centos system. I am explaining the error below.
docker: Error response from daemon: lstat /var/lib/docker/overlay2/3202584ed599bad99c7896e0363ac9bb80a0385910844ce13e9c5e8849494d07: no such file or directory.
I am setting of the local registry like below.
vi /etc/docker/daemon.json:
{ "insecure-registries":["ip:5000"] }
I have the registry image installed my system and I am running using the below command.
docker run -dit -p 5000:5000 --name registry bundle/tools:registry_3.0.0-521
I have cleaned all volume as per some suggestion from google but still same issue. Can anybody help me to resolve this error.
The error is not related to the registry and is happening in the client side because of local caching (or some other docker-related issue) in your system.
I've seen this error a lot in the docker community and the most suggested approach to solve this error is to clean up the whole /var/lib/docker directory.
On your local client system, if you don't care about your current containers, images, and caches, try stopping the docker daemon, removing the whole /var/lib/docker directory, and starting it again:
Note that sometimes it gets fixed by only restarting the daemon, so it worths trying it first:
sudo service docker restart
If a simple restart can't solve the problem, go ahead and destroy it:
sudo service docker stop
sudo rm -rf /var/lib/docker
sudo service docker start
(I'm not sure about if these systemd commands will work on your CentOS too)
We were trying to build and run docker-compose project on remote host. I tried using:
docker-compose -H 'ssh://remote_address' up --build
And got
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
So we tried:
docker-compose -H 'ssh://remote_address' build
docker-compose -H 'ssh://remote_address' up
Which worked fine. My problem is I can't find evidence in docs for this to be correct behaviour. Is this a bug in docker-compose, a feature, or a bug in my environment?
I'm not sure of the error you got for the first command, I mean docker-compose -H 'ssh://ip' up --build as it may be really a but, but the three mentioned commands have surely differences. I'll try to explain in my simple way:
First command is docker-compose up --build.
This command finds docker-compose file and rebuilds the image then make it running. Suppose you have made some changes into your docker-compose file, so when you only run docker-compose, you'll get a warning that image is not rebuilt, you should run docker-compose up --build to rebuild it and make everything be built again (despite something done before and present in cache).
Second command is docker-compose build.
This command only builds your image based on docker-compose, but does not run it. You can see the built image by docker image ls or docker images. Also executing docker ps -a should not see your recent built image running.
Third and the last command is docker-compose up.
If this command is entered for the first time, it tries to run everything in Dockerfile if exists and download base image, etc. Then makes the image and runs the container.
If the image has been built before, it just runs it.
Unlike the first command, the third one only runs the latest build of that image, but would not build it again.
I've run through the initial Overview of Docker Compose exactly as written and it works just fine locally with boot2docker. However, if I try to do a docker-compose up on a remote host, it does not add the code to the remote container.
To reproduce:
Run through the initial Overview of Docker Compose exactly as written.
Install Docker Machine and start a Dockerized VM on any cloud provider.
docker-machine create --driver my-favourite-cloud composetest
eval "$(docker-machine env composetest)"
Now that you're working with a remote host, run docker-compose up on the original code.
composetest $ docker-compose up
Redis runs fine but the Flask app does not.
composetest $ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
794c90928b97 composetest_web "/bin/sh -c 'python About a minute ago Exited (2) About a minute ago composetest_web_1
2c70bd687dfc redis "/entrypoint.sh redi About a minute ago Up About a minute 6379/tcp composetest_redis_1
What went wrong?
composetest $ docker logs 794c90928b97
python: can't open file 'app.py': [Errno 2] No such file or directory
Can we confirm it?
composetest $ docker-compose run -d web sleep 60
Starting composetest_redis_1...
composetest_web_run_3
composetest $ docker exec -it a4 /bin/bash
root#a4a73c6dd159:/code# ls -a
. ..
Nothing there. Can we fix it?
Comment out volumes in docker-compose.yml
web:
build: .
ports:
- "5000:5000"
# volumes:
# - .:/code
links:
- redis
redis:
image: redis
Then just docker-compose up and it works!
Let's try again on boot2docker.
composetest $ eval "$(boot2docker shellinit)"
composetest $ docker-compose up
Recreating composetest_redis_1...
Recreating composetest_web_1...
Attaching to composetest_redis_1, composetest_web_1
...
The Flask app does work but it has a serious problem. If you change app.py, the Flask dev server doesn't reload and those changes aren't automatically seen. Even if you stop the container and docker-compose up again, the changes still aren't seen. I realize we lose this essential feature because the volume is no longer mounted. But not mounting the volume is the only way I've been able to get docker-compose to work with a remote host. We should be able to get both local and remote hosts to work using the same docker-compose.yml and Dockerfile.
How do I develop interactively with a local VM and deploy to a remote VM without having to change the Dockerfile or docker-compose.yml?
Versions:
Docker 1.7.0
Docker Compose 1.3.0
Docker Machine 0.3.0
Compose just does pretty much the same thing as you can do with the regular command line interface under the hood. So your command is roughly equivalent to:
$ docker run --name web -p 5000:5000 -v $(pwd):/code --link redis:redis web
The issue is that the volume is relative to the docker host, not the client. So it will mount the working directory on the remote VM, not the client. In your case, this directory is empty.
If you want to develop interactively with a remote VM, you will have to check out the source and edit the files on the VM.
UPDATE: It seems that you actually want to develop and test locally, then deploy a production version to a remote VM. (Apologies if I still misunderstand). To do this, I suggest you have a separate Compose file for development where you mount the local volume, then rebuild and deploy the image for production. By rebuilding the image, it will pick up the latest version of the code. Mounting a volume in production breaks because you've hidden the code in the image with an empty directory.
It's also worth pointing out that Docker don't advise using Compose in production currently.
What I'm really looking for is found in this Using Compose in production doc. By Extending services in Compose you're able to develop interactively with a local VM and deploy to a remote VM without having to change the Dockerfile or docker-compose.yml.
I ran into this "can't open file 'app.py'" problem following the Getting Started tutorials, for me it was because I'm running Docker in Windows. I needed to make sure that I'd shared the drive containing my project directory in Docker settings.
Source: see the "Shared Drive" section of Docker Settings in the docs