With docker I would like to offer a vm to each client to compile and execute a C program in only one file.
For that, I share a folder with the docker and the host thanks to a dockerfile and the command "ADD".
My folder is like that:
folder/id_user/script.sh
folder/id_user/code.c
In script.sh:
gcc ./compil/code.c -o ./compil/code && ./compil/code
My problem is in the doc we can read this for ADD:
All new files and directories are created with mode 0755, uid and gid 0.
But when I launch "ls" on the file I have:
ls -l compil/8f41dacd-8775-483e-8093-09a8712e82b1/
total 8
-rw-r--r-- 1 1000 1000 51 Feb 11 10:52 code.c
-rw-r--r-- 1 1000 1000 54 Feb 11 10:52 script.sh
So I can't execute the script.sh. Do you know why?
Maybe you wonder why proceed like that.
It's because if I do:
sudo docker run ubuntu/C pwd && pwd
result:
/
/srv/website
So we can see the first command is in the VM but not the second. I understand it might be normal for docker.
If you have any suggestion I'm pleased to listen it.
Thanks !
You can set up the correct mode by RUN command with chmod:
# Dockerfile
...
ADD script.sh /root/script.sh
RUN chmod +x /root/script.sh
...
The second question, you should use CMD command - && approach does work in Dockerfile, try to put this line at the end of your Dockerfile:
CMD pwd && pwd
then docker build . and you will see:
root#test:/home/test/# docker run <image>
/
/
Either that your you can do:
RUN /bin/sh /root/script.sh
to achieve the same result
Related
I am having a problem adding a file to an image and setting ownership via --chown flag. Specifically, here is a dockerfile adding a simple text file:
FROM fedora:24
ARG user_name=slave
ARG user_uid=1000
ARG user_home=/home/$user_name/
RUN useradd -l -u ${user_uid} -ms /bin/bash $user_name
WORKDIR ${user_home}
USER ${user_name}
ADD --chown=1397765041:1397765041 test.txt ./
CMD ls -l
This results in expected ownership of text.txt as can be seen:
$ docker run --rm -it bm/tmp:latest
total 4
-rw-r--r-- 1 some_user 1397765041 6 Oct 21 20:00 test.txt
Cool. Now if I change test.txt to a tar file (for example boost_1_57_0.tar.bz2), and rebuild, this is what I get:
$ docker run --rm -it bm/tmp:latest
total 4
drwx------ 8 501 root 4096 Oct 31 2014 boost_1_57_0
Here is how I am building (probably doesn't matter tho):
docker build -t bm/tmp --build-arg user_name=some_user --build-arg user_uid=1397765041 .
As we can see, ownership is NOT as expected in this case. It seems the behavior of --chown differs from the two cases shown above. I know that ADD automatically extracts tars. I don't know how the ownership is being set in the case where the file is a tar file. Anyone?
Unfortunately, ADD --chown only works for regular files. ADD with a tarball uses the ownership and permissions listed inside in tarball.
Workarounds:
Run tar yourself with --owner/--owner-map/--group/--group-map.
chown -R after the ADD.
Recently, we decided to move one of our services to docker container. The service is product of another company and they have provided us the docker image. However, we need to do some extra configuration steps in the container entrypoint.
The first thing I tried, was to create a DockerFile from the base image and then add commands to do the extra steps, like this:
From baseimage:tag
RUN chmod a+w /path/to/entrypoint_creates_this_file
But, it failed, because these extra steps must be run after running the base container entrypoint.
Is there any way to extend entrypoint of a base image? if not, what is the correct way to do this?
Thanks
I finally ended up calling the original entrypoint bash script in my new entrypoint bash script, before doing other extra configuration steps.
You do not need to even create a new Dockerfile. To modify the entrypoint you can just run the image using the command such as below:
docker run --entrypoint new-entry-point-cmd baseimage:tag <optional-args-to-entrypoint>
create your custom entry-point file
-> add this to image
-> specify this as your entrypoint file
FROM image:base
COPY /path/to/my-entry-point.sh /my-entry-point.sh
// do sth here
ENTRYPOINT ["/my-entry-point.sh"]
Let me take an example with certbot. Using the excellent answer from Anoop, we can get an interactive shell (-ti) into a temporary container (--rm) with this image like so:
$ docker run --rm -ti --entrypoint /bin/sh certbot/certbot:latest
But what if we want to run a command after the original entry point, like the OP requested? We could run a shell and join the commands as in the following example:
$ docker run --rm --entrypoint /bin/sh certbot/certbot:latest \
-c "certbot --version && touch i-can-do-nice-things-here && ls -lah"
certbot 1.30.0
total 28K
drwxr-xr-x 1 root root 4.0K Oct 5 15:10 .
drwxr-xr-x 1 root root 4.0K Sep 7 18:10 ..
-rw-r--r-- 1 root root 0 Oct 5 15:10 i-can-do-nice-things-here
drwxr-xr-x 1 root root 4.0K Sep 7 18:10 src
drwxr-xr-x 1 root root 4.0K Sep 7 18:10 tools
Background
If I run it with the original entrypoint I will get this:
$ docker run --rm certbot/certbot:latest
Saving debug log to /var/log/letsencrypt/letsencrypt.log Certbot
doesn't know how to automatically configure the web server on this
system. However, it can still get a certificate for you. Please run
"certbot certonly" to do so. You'll need to manually configure your
web server to use the resulting certificate.
Or:
$ docker run --rm certbot/certbot:latest --version
certbot 1.30.0
I can see the entrypoint with docker inspect:
$ docker inspect certbot/certbot:latest | grep -i entry -C 2
},
"WorkingDir": "/opt/certbot",
"Entrypoint": null,
"OnBuild": null,
"Labels": null
--
},
"WorkingDir": "/opt/certbot",
"Entrypoint": [
"certbot"
],
If /bin/sh doesn't work in your container, try /bin/bash.
I use supervisor to run cron and nginx, the problem is when i try to COPY or VOLUME mount my cron files, it does not run my cron files in /etc/cron.d
But when I exec -it <container_id> bash into the container and create the exact same cron file from inside, it is immediately recognized and runs as it should.
Dockerfile :
FROM phusion/baseimage:latest
ENV TERM xterm
ENV HOME /root
RUN apt-get update && apt-get install -y \
nginx \
supervisor \
curl \
nano \
net-tools
RUN rm -rf /etc/nginx/*
COPY nginx_conf /etc/nginx
COPY supervisor_conf /etc/supervisor/
RUN mkdir -p /var/log/supervisor
COPY crontabs /etc/cron.d/
RUN chmod -R 644 /etc/cron.d/
CMD /usr/bin/supervisord
The cron itself
* * * * * root curl --silent http://127.0.0.1/cronjob/cron_test_docker.php >> /var/www/html/log/docker_test.log 2>&1
cron and nginx run through supervisor
[supervisord]
nodaemon = true
[program:nginx]
command = /usr/sbin/nginx -g "daemon off;"
autostart = true
[program:cron]
command = /usr/sbin/cron -f
autostart = true
The logs inside /var/log/supervisor/ relating to cron for stdout and stderr are empty.
I also tried stripping out supervisor and running cron on its own through phusion and CMD cron -f but got the same issue of it not working when the source is external(COPY or VOLUME) and magically works when created inside the container.
Initially believed it to be a permissions issue and tried chmod 644 (as this was the permission a file created in the container had) on all files that were the result of COPY into.
RUN chmod 644 /etc/cron.d/
After which tried every possible combination of permissions with rwx to no avail.
Also, tried to append the line of the cronjob into /etc/crontab but it is not recognized in crontab -l.
COPY crontab /tmp/crontab
RUN cat crontab >> /etc/crontab
It would be really handy if it worked just when it was created through COPY or VOLUME as it is a hassle to create it manually in the container everytime.
Any help would be greatly appreciated!
Edit 1 :
Some additional information about the file permissions after COPY or VOLUME.
When I perform
COPY crontabs /etc/cron.d/
RUN chmod -R 644 /etc/cron.d/
Inside the container running ls -l inside /etc/cron.d/ shows
-rw-r--r-- 1 root root 118 Jul 20 11:03 wwwcron-cron-docker_test
When I mount the folder through my docker-compose through VOLUME
volumes:
- ./server/crontabs:/etc/cron.d
ls -l shows
-rwxrwxrwx 1 1000 staff 118 Jul 20 11:03 wwwcron-cron-docker_test
In addition if I manually create the cron file in the container it looks like this and this works
-rw-r--r-- 1 root root 118 Jul 22 15:50 wwwcron-cron-docker_test_inside_docker
Clearly there are very different permissions and ownership when making COPY or VOLUME. But making a COPY with exact permissions does not work but seems to work when created in the container.
Thanks to #BMitch was able to find the issue which was related to line endings since my host machine was windows and the cron file origin was windows as well there was a disparity in the line endings thereby cron did not pick it up automatically.
I added this line to my Dockerfile and it works like a charm
RUN find /etc/cron.d/ -type f -print0 | xargs -0 dos2unix
And iterating on that the size of the file is indeed 1 byte smaller when a dos2unix is run, so you can verify if this operation indeed occurred.
-rw-r--r-- 1 root root 117 Jul 25 08:33 wwwcron-cron-docker_test
Have you tried installing the crontab as a separate command in the Dockerfile?
i.e.
...
COPY crontabs /path/to/crontab.txt
RUN crontab -u myUser /path/to/crontab.txt
...
I am learning Docker which is completely new to me. I already was able to create an jboss/wildfly image, and then i was able to start jboss with my application using this Dockerfile:
FROM jboss/wildfly
CMD ["/opt/jboss/wildfly/bin/standalone.sh", "-c", "standalone-full.xml", "-b", "0.0.0.0"]
ADD mywebapp-web/target/mywebapp-1.0.war /opt/jboss/wildfly/standalone/deployments/mywebapp-1.0.war
Now i would like to add support for a MySQL Database by adding a datasource to the standalone and the mysql connector. For that i am following this example:
https://github.com/arun-gupta/docker-images/tree/master/wildfly-mysql-javaee7
Following is my dockerfile and my execute.sh script
Dockerfile:
FROM jboss/wildfly:latest
ADD customization /opt/jboss/wildfly/customization/
CMD ["/opt/jboss/wildfly/customization/execute.sh"]
execute script code:
#!/bin/bash
# Usage: execute.sh [WildFly mode] [configuration file]
#
# The default mode is 'standalone' and default configuration is based on the
# mode. It can be 'standalone.xml' or 'domain.xml'.
echo "=> Executing Customization script"
JBOSS_HOME=/opt/jboss/wildfly
JBOSS_CLI=$JBOSS_HOME/bin/jboss-cli.sh
JBOSS_MODE=${1:-"standalone"}
JBOSS_CONFIG=${2:-"$JBOSS_MODE.xml"}
function wait_for_server() {
until `$JBOSS_CLI -c ":read-attribute(name=server-state)" 2> /dev/null | grep -q running`; do
sleep 1
done
}
echo "=> Starting WildFly server"
echo "JBOSS_HOME : " $JBOSS_HOME
echo "JBOSS_CLI : " $JBOSS_CLI
echo "JBOSS_MODE : " $JBOSS_MODE
echo "JBOSS_CONFIG: " $JBOSS_CONFIG
echo $JBOSS_HOME/bin/$JBOSS_MODE.sh -b 0.0.0.0 -c $JBOSS_CONFIG &
$JBOSS_HOME/bin/$JBOSS_MODE.sh -b 0.0.0.0 -c $JBOSS_CONFIG &
echo "=> Waiting for the server to boot"
wait_for_server
echo "=> Executing the commands"
$JBOSS_CLI -c --file=`dirname "$0"`/commands.cli
# Add MySQL module
module add --name=com.mysql --resources=/opt/jboss/wildfly/customization/mysql-connector-java-5.1.39-bin.jar --dependencies=javax.api,javax.transaction.api
# Add MySQL driver
/subsystem=datasources/jdbc-driver=mysql:add(driver-name=mysql,driver-module-name=com.mysql,driver-xa-datasource-class-name=com.mysql.jdbc.jdbc2.optional.MysqlXADataSource)
# Deploy the WAR
#cp /opt/jboss/wildfly/customization/leadservice-1.0.war $JBOSS_HOME/$JBOSS_MODE/deployments/leadservice-1.0.war
echo "=> Shutting down WildFly"
if [ "$JBOSS_MODE" = "standalone" ]; then
$JBOSS_CLI -c ":shutdown"
else
$JBOSS_CLI -c "/host=*:shutdown"
fi
echo "=> Restarting WildFly"
$JBOSS_HOME/bin/$JBOSS_MODE.sh -b 0.0.0.0 -c $JBOSS_CONFIG
But I get a error when i run the image complaining that a file or directory is not found:
Building Image
$ docker build -t mpssantos/leadservice:latest .
Sending build context to Docker daemon 19.37 MB
Step 1 : FROM jboss/wildfly:latest
---> b8279b641e82
Step 2 : ADD customization /opt/jboss/wildfly/customization/
---> aea03d4f2819
Removing intermediate container 0920e2cd97fd
Step 3 : CMD /opt/jboss/wildfly/customization/execute.sh
---> Running in 8a0dbcb01855
---> 10335320b89d
Removing intermediate container 8a0dbcb01855
Successfully built 10335320b89d
SECURITY WARNING: You are building a Docker image from Windows against a non-Windows Docker host. All files and directories added to build context will have '-rwxr-xr-x' permissions. It is recommended to double check and reset permissions for sensitive files and directories.
Running image
$ docker run mpssantos/leadservice
no such file or directory
Error response from daemon: Cannot start container 5d3357ba17afa36e81d8794f2b0cd45cc00dde955b2b2054282c4ef17dd4f265: [8] System error: no such file or directory
Can someone let me know how can i access the filesystem so i can check which file or directory is complaining? Is there a better way to debug this?
I believe that is something related with the bash which is referred on first line of the script because the following echo is not printed
Thank you so much
I made it to ssh the container to check whats inside.
1) ssh to the docker machine: docker-machine ssh default
2) checked the container id with the command: docker ps -a
3) ssh to the container with the command: sudo docker exec -i -t 665b4a1e17b6 /bin/bash
4) i can check that the "/opt/jboss/wildfly/customization/" directory exists with the expected files
The customization dir have the following permissions and is listed like this:
drwxr-xr-x 2 root root 4096 Jun 12 23:44 customization
drwxr-xr-x 10 jboss jboss 4096 Jun 14 00:15 standalone
and the files inside the customization dir
drwxr-xr-x 2 root root 4096 Jun 12 23:44 .
drwxr-xr-x 12 jboss jboss 4096 Jun 14 00:15 ..
-rwxr-xr-x 1 root root 1755 Jun 12 20:06 execute.sh
-rwxr-xr-x 1 root root 989497 May 4 11:11 mysql-connector-java-5.1.39-bin.jar
if i try to execute the file i get this error
[jboss#d68190e4f0d8 customization]$ ./execute.sh
bash: ./execute.sh: /bin/bash^M: bad interpreter: No such file or directory
Does this bring light to anything?
Thank you so much again
I found the issue. The execute.sh file was with windows eof. I converted to UNIX And start to work.
I believe the execute.sh is not found. You can verify by running the following and finding the result is an empty directory:
docker run mpssantos/leadservice ls -al /opt/jboss/wildfly/customization/
The reason for this is you are doing your build on a different (virtual) machine than your local system, so it's pulling the "customization" folder from that VM. I'd run the build within the VM and place the files you want to import on that VM where the build can find it.
I am using a very simple .travis.yml to compile a cpp program via docker in Travis-CI. (My motivation is to experiment running docker in Travis CI.)
sudo: required
services:
- docker
before_install:
- docker pull glot/clang
script:
- sudo docker run --rm -v "$(pwd)":/app -w /app glot/clang g++ main.cpp
But the build is failing with following error:
/usr/bin/ld: cannot open output file a.out: Permission denied. This is regardless of whether I use sudo or not. Can someone help me out figuring out the root cause and help fix this? Thanks.
I would suggest you to set mounting path explicitly rather then doing it with $(pwd). Then you need to check the permissions from inside the container. Try something like that:
sudo docker run --rm -v "$(pwd)":/app -w /app glot/clang stat /app
This will show folder permissions. Probably noone is able to write into this folder.
Also you should avoid building your software using root permissions, it's not secure. Create non-priveleged user and use them when you running the compiler.
UPD:
I cannot reproduce this issue with docker 1.6.0, probably it's caused by some filesystem settings persisted by Travis-CI virtual machine. This is what I have on my localhost:
➜ /tmp mkdir /tmp/code
➜ /tmp echo "int main(){}" > /tmp/code/main.cpp
➜ /tmp echo "g++ main.cpp && ls -l" > /tmp/code/build.sh
➜ /tmp docker run --rm -v /tmp/code:/app -w /app glot/clang bash /app/build.sh
total 20
-rwxr-xr-x 1 glot glot 8462 Dec 30 10:19 a.out
-rwxrwxr-x 1 glot glot 22 Dec 30 10:17 build.sh
-rw-rw-r-- 1 glot glot 13 Dec 30 10:10 main.cpp
As you see, the resulting binary appears in /app folder