I've got some Python app which used bind mounts to mount code into container, so I don't have to build container on each code change, like this:
app:
volumes:
- type: bind
source: ./findface
target: /app/findface
And it was working fine. But now I also want to bind my startup script, which is invoked from Dockerfile:
app:
volumes:
- type: bind
source: ./findface
target: /app/findface
- type: bind
source: ./startup.sh
target: /app
And this one just doesn't bind. There is an actual file in the host filesystem, but when I build the container it can't find it:
Step 7/8 : RUN chmod +x startup.sh
---> Running in ecaae384b6e5
chmod: cannot access 'startup.sh': No such file or directory
ERROR: Service 'app' failed to build: The command '/bin/sh -c chmod +x startup.sh' returned a non-zero code: 1
What am I doing wrong?
Dockerfile itself:
FROM jjanzic/docker-python3-opencv:latest
ENV PYTHONUNBUFFERED=1
RUN pip install --no-cache-dir gunicorn[eventlet]
WORKDIR /app
COPY ./requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
RUN chmod +x startup.sh
CMD "./startup.sh"
Base on your comment, then you do not need set permission at build stage as the file is not exist at build time, if you do not want to copy at build stage, set permission on the host to make executable and then bind with docker run command or as per the yml config that you mentioned.
CMD "/app/startup.sh"
for example
docker run -it --rm -v $PWD/startup.sh:/app/startup.sh my_image
Related
I'm running K8s deployment and trying to harden the security of one of my pod and because of that I started using the following docker image:
nginxinc/nginx-unprivileged:alpine
The problem is that I need to create a symlink and cannot get it done.
Here is the structure of my dockerfile
FROM nginxinc/nginx-unprivileged:alpine
ARG name
ARG ver
USER root
COPY ./outbox/${name}-${ver}.tgz ./
COPY ./nginx.conf /etc/nginx/nginx.conf
COPY ./mime.types /etc/nginx/mime.types
COPY ./about.md ./
RUN mv /${name}-${ver}.tgz /usr/share/nginx/html
WORKDIR /usr/share/nginx/html
RUN tar -zxf ${name}-${ver}.tgz \
&& mv ngdist/* . \
&& mv /about.md ./assets \
&& rm -fr ngdist web-ui-${ver}.tgz \
&& mkdir -p /tmp/reports
RUN chown -R 1001 /usr/share/nginx/html/
COPY ./entrypoint.sh.${name} /bin/entrypoint.sh
RUN chown 1001 /bin/entrypoint.sh
USER 1001
EXPOSE 8080
CMD [ "/bin/entrypoint.sh" ]
and here my entrypoint.sh
#!/bin/sh
ln -s /tmp/reports /usr/share/nginx/html/reports
and here is my container in the pod deployment yaml file
containers:
- name: web-ui
image: "myimage"
imagePullPolicy: Always
ports:
- containerPort: 8080
name: web-ui
volumeMounts:
- name: myvolume
mountPath: /tmp/reports
I tried to set the entrypoint under the root execution but that did not help either, the error i'm getting is this:
Error: failed to start container "web-ui": Error response from daemon:
OCI runtime create failed: container_linux.go:380: starting container
process caused: exec: "/bin/entrypoint.sh": permission denied: unknown
Like other Linux commands, a Docker container's main CMD can't run if the program it names isn't executable.
Most source-control systems will track whether or not a file is executable, and Docker COPY will preserve that permission bit. So the best way to address this is to make the scripts executable on the host:
chmod +x entrypoint.sh.*
git add entrypoint.sh.*
git commit -m 'make entrypoint scripts executable'
docker-compose build
docker-compose up -d
If that's not an option, you can fix this up in the Dockerfile too.
COPY ./entrypoint.sh.${name} /bin/entrypoint.sh
RUN chmod 0755 /bin/entrypoint.sh
Like other things in /bin, the script should usually be owned by root, executable by everyone, and writable only by its owner; you do not generally want the application to have the ability to overwrite its own code.
I am using docker context to deploy my local container to my debian webserver. I use Docker Desktop for Windows on Windows 10. The app is written using Flask.
At some point I tried “docker-compose up --build” after “docker context use remote” and I was getting the following error:
Error response from daemon: invalid volume specification: ‘C:\Users\user\fin:/fin:rw’
Locally everything works fine when I try to deploy it to the production server the error pops up.
The Dockerfile looks like the following:
FROM python:3.8-slim-buster
ENV INSTALL_PATH /app
RUN mkdir -p $INSTALL_PATH
WORKDIR $INSTALL_PATH
ENV PATH="/home/user/.local/bin:${PATH}"
COPY . ./
# set environment variables
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
RUN useradd -ms /bin/bash user && chown -R user $INSTALL_PATH
USER user
COPY requirements.txt requirements.txt
RUN pip install -r requirements.txt
RUN pip install --upgrade pip
CMD gunicorn -c "python:config.gunicorn" "fin.app:create_app()"
while an excerpt of the docker-compose.yml look like the following:
version: '3.8'
services:
flask-app:
container_name: flask-app
restart: always
build: .
command: >
gunicorn -c "python:config.gunicorn" "fin.app:create_app()"
environment:
PYTHONUNBUFFERED: 'true'
volumes:
- '.:/fin'
ports:
- 8000:8000
env_file:
- '.env'
In the .env file the option
COMPOSE_CONVERT_WINDOWS_PATHS=1 is set.
At some point I tried the same procedure using WSL2 with Ubuntu installed, which led to the following message:
Error response from daemon: create \\wsl.localhost\Ubuntu-20.04\home\user\fin: "\\\\wsl.localhost\\Ubuntu-20.04\\home\\user\\fin" includes invalid characters for a local volume name, only "[a-zA-Z0-9][a-zA-Z0-9_.-]" are allowed. If you intended to pass a host directory, use absolute path
Based on this message I changed the Dockerfile to:
FROM python:3.8-slim-buster
ENV INSTALL_PATH=/usr/src/app
RUN mkdir -p $INSTALL_PATH
WORKDIR $INSTALL_PATH
ENV PATH=/home/user/.local/bin:${PATH}
COPY . /usr/src/app/
# set environment variables
ENV PYTHONDONTWRITEBYTECODE=1
ENV PYTHONUNBUFFERED=1
#ENV COMPOSE_CONVERT_WINDOWS_PATHS=1
RUN useradd -ms /bin/bash user && chown -R user $INSTALL_PATH
USER user
COPY requirements.txt /usr/src/app/requirements.txt
RUN pip install -r requirements.txt
RUN pip install --upgrade pip
CMD gunicorn -c "python:config.gunicorn" "fin.app:create_app()"
But still the error remains, and I have to clue how to solve it.
Thank you in advance for your help.
You are getting invalid volume specification: ‘C:\Users\user\fin:/fin:rw’ in your production environment is because, the host path C:\Users\user\fin isn't available. You can remove it when you are deploying or change it to an absolute path which is available in your production environment as below.
volumes:
- '/root:/fin:rw'
where /root is a directory available in my production environment.
/path:/path/in/container mounts the host directory, /path at the /path/in/container
path:/path/in/container creates a volume named path with no relationship to the host.
Note the slash at the beginning. if / is present it will be considered as a host directory, else it will be considered as a volume
use this (without quotes and with a slash so it knows you mean this folder):
volumes:
- ./:/fin
Tried a lot already, but doesn't want to run docker-compose with entrypoint...
I'm new to docker and trying to figure it out, but I can't seem to run the program through entrypoint.sh
Dockerfile
FROM python:3.7-alpine
WORKDIR /app
RUN apk update && apk add postgresql-dev gcc python3-dev musl-dev
ENV FLASK_APP=app.py
ENV FLASK_RUN_HOST=0.0.0.0
RUN /usr/local/bin/python3 -m pip install --upgrade pip
COPY requirements.txt requirements.txt
RUN pip install -r requirements.txt
COPY /web .
COPY entrypoint.sh entrypoint.sh
RUN chmod u+x ./entrypoint.sh
docker-compose.yml
version: "3.9"
services:
web:
build: .
ports:
- "5000:5000"
volumes:
- ./web:/app
environment:
PG_HOST: pg
PG_USER: ${POSTGRES_USER}
PG_PASSWORD: ${POSTGRES_PASSWORD}
PG_DB: ${POSTGRES_DB}
depends_on:
- pg
entrypoint: entrypoint.sh
pg:
image: "postgres:alpine"
environment:
POSTGRES_USER: ${POSTGRES_USER}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
POSTGRES_DB: ${POSTGRES_DB}
# PGDATA: /var/lib/postgresql/data/pgdata
# volumes:
# - ./data/pgdata:/var/lib/postgresql/data/pgdata
ports:
- 5432:5432
entrypoint.sh
#python3 manage.py db upgrade
flask run --host=0.0.0.0
ERROR:
Step 1/11 : FROM python:3.7-alpine
---> c051b8594107
Step 2/11 : WORKDIR /app
---> Using cache
---> d19c508ff35c
Step 3/11 : RUN apk update && apk add postgresql-dev gcc python3-dev musl-dev
---> Using cache
---> 19b5e197621a
Step 4/11 : ENV FLASK_APP=app.py
---> Using cache
---> af02528555c4
Step 5/11 : ENV FLASK_RUN_HOST=0.0.0.0
---> Using cache
---> 4029b777d985
Step 6/11 : RUN /usr/local/bin/python3 -m pip install --upgrade pip
---> Using cache
---> e94f1f70106b
Step 7/11 : COPY requirements.txt requirements.txt
---> Using cache
---> fb4ad7239fa2
Step 8/11 : RUN pip install -r requirements.txt
---> Using cache
---> 1f912edd8219
Step 9/11 : COPY /web .
---> Using cache
---> 32966afa52ee
Step 10/11 : COPY entrypoint.sh entrypoint.sh
---> Using cache
---> 11525954905f
Step 11/11 : RUN chmod u+x ./entrypoint.sh
---> Using cache
---> 575c41093e1f
...
Successfully built 89b11d4d5f71
Successfully tagged rodauth_web:latest
rodauth_pg_1 is up-to-date
Recreating 1e4f3f2f6898_rodauth_web_1 ...
Recreating 1e4f3f2f6898_rodauth_web_1 ... error
ERROR: for 1e4f3f2f6898_rodauth_web_1 Cannot start service web: OCI runtime create failed: container_linux.go:370: starting container process caused: exec: "entrypoint.sh": executable file not found in $PATH: unknown
ERROR: for web Cannot start service web: OCI runtime create failed: container_linux.go:370: starting container process caused: exec: "entrypoint.sh": executable file not found in $PATH: unknown
Encountered errors while bringing up the project.
tried adding command RUN chmod u+x ./entrypoint.sh but
the problem is not gone.
Added the build log for COPY entrypoint.sh entrypoint.sh part.
Your entrypoint.sh file get removed from the container when you mount ./web to /app folder as a volume in docker-compose file. so copy entrypoint to root
COPY entrypoint.sh /entrypoint.sh
RUN chmod u+x /entrypoint.sh
And set entrypoint in docker-compose as
entrypoint: /entrypoint.sh
instead of COPY /web ., just do COPY . /app where you are copying the same structure into /app. as you mentioned, workdir /app you are already in that app folder.
or you can just change the last line, COPY entrypoint.sh .
simple docker example : https://github.com/cerofrais/baiscs/tree/master/basic_docker
First, if you want the script to be the normal entrypoint for the image, I'd move that into the Dockerfile with the following:
ENTRYPOINT ["./entrypoint.sh"]
As for why you'd see a file not found error on a script, several potential reasons:
The first line, e.g. #!/bin/bash, points to a command that does not exist in the image you are using. You need to update your script to use the right shell, or change base images to one that includes your shell.
The script is formatted with Windows linefeeds that include a CR before the LF. That turns /bin/sh into /bin/sh^M (sometimes shown as /bin/sh\r) which doesn't exist in the filesystem. This can be fixed by changing the line feed format in your editor, fixing Git to change how linefeeds are automatically changed, or using a tool like dos2unix.
If you run an image for a different platform, and have qemu binfmt_misc installed but without the --fix-binary option, it will look for the interpreter for your platform inside the image rather that from the host. Often this can be fixed with a newer version of binfmt_misc.
The exec format error mentioned in the comments indicates you are running an image designed for another platform on your host. Docker will attempt to pull the proper platform if you have a multi-platform image. However if the image is only built for one platform, and you try to run it on a different platform without something like qemu's binfmt_misc properly configured, you'll get errors. And the fix for that is to change base images to one that supports that CPU architecture and binary format where you are running the image.
This is how I customized Ravisha Hesh's answer and it worked for me.
COPY entrypoint.sh /entrypoint.sh
RUN chmod u+x /entrypoint.sh
CMD ["/entrypoint.sh"]
I want to create an empty DB file using touch by Dockerfile or docker-compose.yml and volume it. Actually, I'm able to create it manually within the container as follows:
docker exec -it <container-name> bash
# touch /app/model/modbus.db
Whereas, when I use the following procedure it throws exited with code 0 and stops:
version: '3'
services:
collector:
build: .
image: collector:2.0.0
command: bash -c "touch /app/model/modbus.db" # Note
# command: bash /app/bashes/create_an_empty_db.sh
volumes:
- "./model/modbus.db:/app/model/modbus.db:rw"
tty: true
As well as this, I tried that via Dockerfile without any success either:
FROM python:3.6-slim
ENV PYTHONUNBUFFERED 1
RUN mkdir /app
WORKDIR /
ADD . /app
RUN touch /app/model/modbus.db # Note
CMD python app
[NOTE]:
Also without the command: bash -c "touch /app/model/modbus.db" in the docker-compose.yml which was the cause of exited with code 0; a directory will be created named modbus.db instead of a file due to the following section:
volumes:
- "./model/modbus.db:/app/model/modbus.db:rw"
TL;DR:
How to send a new file from the container to the host which does not exist in the host? (In other words, it is done inside of the container, not in the host)
I am not sure about the docker-compose.yml but the dockerfile that you have seems to be working for me.
The Dockerfile looks like this,
FROM python:3.6-slim
RUN mkdir /app
WORKDIR /
RUN touch /app/modbus.db
Build the dockerfile,
docker build -t test .
Sending build context to Docker daemon 2.048kB
Step 1/4 : FROM python:3.6-slim
---> 903e8a0f0681
Step 2/4 : RUN mkdir /app
---> Using cache
---> c039967bf463
Step 3/4 : WORKDIR /
---> Using cache
---> c8c81ac01f50
Step 4/4 : RUN touch /app/modbus.db
---> Using cache
---> 785916fe4cea
Successfully built 785916fe4cea
Successfully tagged test:latest
Build the container,
docker run -dit test
52cde500cda015f170140ae9e7174a0367b29265a49a3742173946b686179fb3
I ssh'ed into the container and was able to find the file.
docker exec -it 52cde500cda015f170140ae9e7174a0367b29265a49a3742173946b686179fb3 /bin/bash
root#52cde500cda0:/# ls
app bin boot dev etc home lib lib64 media mnt opt proc root run sbin srv sys tmp usr var
root#52cde500cda0:/# cd app
root#52cde500cda0:/app# ls
modbus.db
Put the following in your docker-compose.yml
volumes:
- "./model:/app/model"
This will create a /app/model folder inside of your container. Its contents (which you will create inside the container) will be available on ./model on your host.
If you put the touch command in the CMD of your Dockerfile, that file will be created after starting the container when the volume is also initialized. So the following Dockerfile should work:
FROM python:3.6-slim
ENV PYTHONUNBUFFERED 1
RUN mkdir /app
WORKDIR /
ADD . /app
CMD touch /app/model/modbus.db && python app
Been stuck on this for the last 3 days. I'm building an image in a docker and
copy command fails due to not finding the right directory.
FROM python:3.6.7-alpine
WORKDIR /usr/src/app
COPY ./requirements.txt /usr/src/app/requirements.txt
RUN pip3 install -r requirements.txt
COPY . /usr/src/app
CMD python3 manage.py run -h 0.0.0.0
which is run by this docker-dev file:
version: '3.7'
services:
users:
build:
context: ./services/users
dockerfile: Dockerfile-dev
volumes:
- './services/users:/usr/src/app'
ports:
- 5001:5000
environment:
- FLASK_APP=project/__init__.py
- FLASK_ENV=development
and getting this error:
Building users
Step 1/6 : FROM python:3.6.7-alpine
---> cb04a359db13
Step 2/6 : WORKDIR /usr/src/app
---> Using cache
---> 06bb39a49444
Step 3/6 : COPY ./requirements.txt /usr/src/app/requirements.txt
ERROR: Service 'users' failed to build: COPY failed: stat /var/snap/docker/common/var-lib-docker/tmp/docker-builder353668631/requirements.txt: no such file or directory
I don't even know where to start with debugging this. When I tried to access the directory it gave me permission error. So I tried to run the command with sudo which didn't help. Any thoughts ?
Little late to reply, but second COPY command COPY . /usr/src/app replaces the /usr/src/app content generated by RUN pip3 install -r requirements.txt.
Try
FROM python:3.6.7-alpine
WORKDIR /usr/src/app
# install in temp directory
RUN mkdir /dependencies
COPY ./requirements.txt /dependencies/requirements.txt
RUN cd /dependencies && pip3 install -r requirements.txt
COPY . /usr/src/app
# copy generated dependencies
RUN cp -r /dependencies/* /usr/src/app/
CMD python3 manage.py run -h 0.0.0.0
As larsks suggests in his comment, you need the file in the services/users directory. To understand why, an understanding of the "context" is useful.
Docker does not build on the client, it does not see your current directory, or other files on your filesystem. Instead, the last argument to the build command is passed as the build context. With docker-compose, this context defaults to the current directory, which you will often see as . in a docker build command, but you can override that as you've done here with ./services/users as your context. When you run a build, the very first step is to send that build context from the docker client to the server. Even when the client and server are on the same host (a common default, especially for desktop environments), this same process happens. Files listed in .dockerignore, and files in parent directories to the build context are not sent to the docker server.
When you run a COPY or ADD command, the first argument (or all but the last argument when you have multiple) refer to files from the build context, and the last argument is the destination file or directory inside the image.
Therefore, when you put together this compose file entry:
build:
context: ./services/users
dockerfile: Dockerfile-dev
with this COPY command:
COPY ./requirements.txt /usr/src/app/requirements.txt
the COPY will try to copy the requirements.txt file from the build context generated from ./services/users, meaning ./services/users/requirements.txt needs to exist, and not be excluded by a .dockerignore file in ./services/users.
I had a similar problem building an image with beryllium, and I solved this deleting it into the .dockerignore
$ sudo docker build -t apache .
Sending build context to Docker daemon
10.55MB Step 1/4 : FROM centos ---> 9f38484d220f Step 2/4 :
RUN yum install httpd -y
---> Using cache ---> ccdafc4ae476 Step 3/4 :
**COPY ./**beryllium** /var/www/html COPY failed: stat /var/snap/docker/common/var-lib-docker/tmp/docker-builder04301**
$nano .dockerignore
startbootstrap-freelancer-master
run.sh
pro
fruit
beryllium
Bell.zip
remove beryllium from that file
$ sudo docker build -t apache .
Sending build context to Docker daemon 12.92MB
Step 1/4 : FROM centos
---> 9f38484d220f
Step 2/4 : RUN yum install httpd -y
---> Using cache
---> ccdafc4ae476
Step 3/4 : COPY ./beryllium /var/www/HTML
---> 40ebc02992a9
Step 4/4 : CMD apachectl -DFOREGROUND
---> Running in dab0a406c89e
Removing intermediate container dab0a406c89e
---> 1bea741cfb65
Successfully built 1bea741cfb65
Successfully tagged apache:latest