This may seem like duplicate question? But I think it's not, and also. I have tried google.
This is the situation, I want to build a CI pipeline with Gogs and Drone, with their nice documents, I have build it up. But now, I am stucking on: How to copy files from container to the host in the container. below is my drone yml config.
pipeline:
build:
image: node:7
commands:
- cd client
- npm config set registry https://registry.npm.taobao.org
- npm install --no-optional
- yarn run build
- sudo docker cp $(sudo docker ps -alq):$PWD/build /var/www/react/
The CI ends with error:
/bin/sh: sudo: not found
after try without sudo, error continues:
/bin/sh: docker: not found
The answers I found are all about copy files to host from container which the shell runs in host computer. but now the commands I tried to run is in the container, so What should i do? or I miss something.
you could mount a volume on the container - a volume is a directory mapping from the host to the container - a shared folder between the two
In the container, you would then copy the files you want to the shared folder and the host could access them in the shared folder.
Don't copy the files into the shared folder before mounting has finished. If you do, the files will not be accessible.
Here's the docs on volumes:
https://docs.docker.com/storage/volumes/#create-and-manage-volumes
Related
We have mounted a folder in a Linux machine to our docker container application using (docker-compose)
volumes:
- /mnt/share:/mnt/share
The /mnt/share is a mounted folder in the machine(Not a real folder in the machine, its our file server). IF for some reason that mount is lost and then remounted again.
The application running in the docker container is not having access to the mounted folder until the container is restarted.
You might want to use to use a Volume Driver instead of bind-mounting a local filesystem.
See Share data among machines
Without knowing more about your environment it is impossible to give a more detailed answer. It would be helpful to know if your container runs in a AWS data center or if you use nfsv3, nfsv4 or cifs for mounting.
The following solution helped me to continue.
I wrote a script to check whether the folder exists.
The script is then called a command in the docker-compose file.
version:"3"
services:
flowable-task-handler:
build: flowable-task-handler
ports:
- "8085:8085"
command: bash -c "/wait_for_file_mount.sh /mnt/share/fileshares/ && java -jar /app.jar"
wait_for_file_mount.sh
#!/bin/sh
# Used to check whether the mount folder is ready for flowable to use
mountedfolder="$1"
until [ -d "$mountedfolder" ];
do sleep 2;
echo error "Mounted folder not found : $mountedfolder";
done;
Its a spring boot application. I have removed the entrypoint in the DockerFile and application is started using the command in docker compose(java -jar /app.jar")
defining the mount propagation as ":shared" should fix this:
-v /autofs:/autofs:shared \
not sure about docker-compose - I don't really use that. but you can define a docker volume with mount propagation and put this into your DC file.
I use docker-compose for a simple keycloak container and I've been trying to install a new theme for keycloak.
However I've been unable to copy even a single file to the container usnig a Dockerfile. The Dockerfile and docker-compose.yml are in the same directory
Neither of these commands work or cause any events or warnings in the logs.
COPY test /tmp/
COPY /home/adm/workspace/docker/keycloak-cluster/docker/kctheme/theme/login/. /opt/jboss/keycloak/themes/keycloak/login/
Copying manually with
sudo docker cp test docker_keycloak_1:/tmp
works without any issues.
Quick understanding on Docker.
docker build: Create an image from a Dockerfile
docker run: Create a container from an image.
(you can create yourself the image or use a existing image from docker hub )
Based on what you said, you have 2 options.
Create a new docker image based on the existing one and add the theme.
something like
# Dockerfile
FROM jboss/keycloak
COPY test /tmp/
COPY /home/adm/workspace/docker/keycloak-cluster/docker/kctheme/theme/login/. /opt/jboss/keycloak/themes/keycloak/login/
and then use docker build to create your new image
Volume the theme in the correct directory
using docker-compose volume
version: '3'
services:
keycloak:
image: quay.io/keycloak/keycloak:latest
volumes:
- "./docker/kctheme/theme/login:/opt/jboss/keycloak/themes/keycloak/login"
Files have to be in the same directory or a subdirectory of the directory with your Dockerfile build file if you use COPY, and have to be present at build time. No absolute paths.
/tmp as destination is also a bit tricky, because the startup process of the container might have a /tmp cleanout, which means that you would never see that file in a running container.
I'm in Docker Desktop for Windows. I am trying to use docker-compose as a build container, where it builds my code and then the code is in my local build folder. The build processes are definitely succeeding; when I exec into my container, the files are there. However, nothing happens with my local folder -- no build folder is created.
docker-compose.yml
version: '3'
services:
front_end_build:
image: webapp-build
build:
context: .
dockerfile: Dockerfile
ports:
- 5000:5000
volumes:
- "./build:/srv/build"
Dockerfile
FROM node:8.10.0-alpine
EXPOSE 5000
# add files from local to container
ADD . /srv
# navigate to the directory
WORKDIR /srv
# install dependencies
RUN npm install --pure-lockfile --silent
# build code (to-do: get this code somewhere where we can use it)
RUN npm run build
# install 'serve' and launch server.
# note: this is just to keep container running
# (so we can exec into it and check for the files).
# once we know that everything is working, we should delete this.
RUN npx serve -s -l tcp://0.0.0.0:5000 build
I also tried removing the final line that serves the folder. Then I actually did get a build folder, but that folder was empty.
UPDATE:
I've also tried a multi-stage build:
FROM node:12.13.0-alpine AS builder
WORKDIR /app
COPY . .
RUN yarn
RUN yarn run build
FROM node:12.13.0-alpine
RUN yarn global add serve
WORKDIR /app
COPY --from=builder /app/build .
CMD ["serve", "-p", "80", "-s", "."]
When my volumes aren't set (or are set to, say, some nonexistent source directory like ./build:/nonexistent), the app is served correctly, and I get an empty build folder on my local machine (empty because the source folder doesn't exist).
However when I set my volumes to - "./build:/app" (the correct source for the built files), I not only wind up with an empty build folder on my local machine, the app folder in the container is also empty!
It appears that what's happening is something like
1. Container is built, which builds the files in the builder.
2. Files are copied from builder to second container.
3. Volumes are linked, and then because my local build folder is empty, its linked folder on the container also becomes empty!
I've tried resetting my shared drives credentials, to no avail.
How do I do this?!?!
I believe you are misunderstanding how host volumes work. The volume definition:
./build:/srv/build
In the compose file will mount ./build from the host at /srv/build inside the container. This happens at run time, not during your image build, so after the Dockerfile instructions have been performed. Nothing from the image is copied out to the host, and no files in the directory being mounted in top of will be visible (this is standard behavior of the Linux mount command).
If you need files copied back out of the container to the host, there are various options.
You can perform your steps to populate the build folder as part of the container running. This is common for development. To do this, your CMD likely becomes a script of several commands to run, with the last step being an exec to run your app.
You can switch to a named volume. Docker will initialize these with the contents of the image. It's even possible to create a named bind mount to a folder on your host, which is almost the same as a host mount. There's an example of a named bind mount in my presentation here.
Your container entrypoint can copy the files to the host mount on startup. This is commonly seen on images that will run in unknown situations, e.g. the Jenkins image does this. I also do this in my save/load volume scripts in my example base image.
tl;dr; Volumes aren't mounted during the build stage, only while running a container. You can run the command docker run <image id> -v ./build/:/srv/build cp -R /app /srv/build to copy the data to your local disk
While Docker is building the image it is doing all actions in ephemeral containers, each command that you have in your Dockerfile is run in a separate container, each making a layer that eventually becomes the final image.
The result of this is that the data flow during the build is unidirectional, you are unable to mount a volume from the host into the container. When you run a build you will see Sending build context to Docker daemon, because your local Docker CLI is sending the context (the path you specified after the docker build, ususally . which represents the current directory) to the Docker daemon (the process that actually does the work). One key point to remember is that the Docker CLI (docker) doesn't actually do any work, it just sends commands to the Docker Daemon dockerd. The build stages shouldn't change anything on your local system, the container is designed to encapsulate the changes only into the container image, and give you a snapshot of the build that you can reuse consistently, knowing that the contents are the same.
I cannot seem to run composer install in Dockerfile but I can in the container after building an image and running the container.
Here's the command from Dockerfile:
RUN composer require drupal/video_embed_field:1.5
RUN composer install --no-autoloader --no-scripts --no-progress
The output is:
Loading composer repositories with package information
Installing dependencies (including require-dev) from lock file
Nothing to install or update
But after running the container with docker-compose:
...
drupal:
image: docker_image
container_name: container
ports:
- 8081:80
volumes:
- ./container/modules:/var/www/html/web/modules
links:
# Link the DB container:
- db
running docker exec composer install will install the packages correctly:
Loading composer repositories with package information
Installing dependencies (including require-dev) from lock file
Package operations: 1 installs, 0 updates, 0 removals
...
Generating autoload files
I am assuming that the composer.json and composer.lock files are correct because I can run the composer install command in the container without any further effort, but only after the container is running.
Update
Tried combining the composer commands with:
RUN composer require drupal/video_embed_field:1.5 && composer install
Same issue, "Nothing to install or update". Ultimately I would like to continue using seperate RUN statements in Dockerfile to take advantage of docker caching.
Your issue is coming from the fact that, docker-compose is meant to orchestrate multiple docker container build and run at the same time and it somehow is not really showing easily what it does behind the scene to people starting with docker.
Behind a docker-compose up there are four steps:
docker-compose build if needed, and if there is no existing image(s) yet, create the image(s)
docker-compose create if needed, and if there is no container(s) existing yet, create the container(s)
docker-compose start start existing container(s)
docker-compose logs logs stderr and stdout of the containers
So what you have to spot on there, is the fact that action contained into you Dockerfile are executed at the image creation step.
When mounting folders is executed at start of containers step.
So when you try to use a RUN command, part of the image creation step, on files like composer.lock and composer.json that are mounted on starting step, you end up having nothing to install via composer because your files are not mounted anywhere yet.
If you do a COPY of those files that may actual get you somewhere, because you will then have the composer files as part of your image.
This said, be careful that the mounted source folder will totally override the mounting point so you could end up expecting a vendor folder and not have it.
What you should ideally do is to have it as the ENTRYPOINT, this one happens at the last step of the container booting.
Here is for a little developing comparison: a docker image is to a docker container what a class is to an instance of an class — an object.
Your container are all created from images built possibly long time before.
Most of the steps in your Dockerfile happens at image creation and not at container boot time.
While most of the instruction of docker-compose are aimed at the automatisation of the container build, which include the mounting of folders.
Just noting a docker-compose.yml approach to the issue when the volume mount overwrites the composer files inside the container:
docker-compose.yml
environment:
PROJECT_DIR: /var/www/html
volumes:
- ./php/docker/entrypoint/90-composer-install.sh:/docker-entrypoint-init.d/90-composer-install.sh
composer-install.sh
#!/usr/bin/env bash
cd ${PROJECT_DIR}
composer install
This runs composer install after the build, using the docker-entrypoint-init.d shell script
I would like to know if there's any way that I can share a directory from my host machine to the docker container (Shared Volume) using Dockerfile.
I understand that we can do that using volumes (-v option) while using docker run. But I couldn't find any way by which I can do that while using an Instruction of Dockerfile.
I already tried VOLUME instruction in Dockerfile but couldn't succeed.
Here's some details about my environment:
[me#myHost new]$ tree -L 1
.
|-- docker-compose.yml
|-- Dockerfile
|-- Shared // This is the directory I wish to share with my containers.
`
I was using docker-compose.yml file to mount this directory till now:
volumes:
- ./Shared:/shared # "Relative Path at the host":"Absolute Path at the container"
But now, due to some reasons, I need to mount it in Dockerfile. I already tried the following but couldn't succeed (It is creating a new empty volume at /shared.):
VOLUME ./Shared:/shared
I could use docker run and save the image by making manual changes in the container, but I wished if I could do that in Dockerfile itself.
Thanks.
You can't mount a local directory using commands in a Dockerfile. You must do this with docker run, or a proxy to docker run like docker-compose.