I hope this question finds you all well.
I am currently trying to create a Docker container image and I am facing a problem.
My original idea for the Dockerfile was the following:
FROM centos:7
ENV container docker
RUN (cd /lib/systemd/system/sysinit.target.wants/; for i in *; do [ $i == \
systemd-tmpfiles-setup.service ] || rm -f $i; done); \
rm -f /lib/systemd/system/multi-user.target.wants/*;\
rm -f /etc/systemd/system/*.wants/*;\
rm -f /lib/systemd/system/local-fs.target.wants/*; \
rm -f /lib/systemd/system/sockets.target.wants/*udev*; \
rm -f /lib/systemd/system/sockets.target.wants/*initctl*; \
rm -f /lib/systemd/system/basic.target.wants/*;\
rm -f /lib/systemd/system/anaconda.target.wants/*;
VOLUME [ "/sys/fs/cgroup" ]
CMD ["/usr/sbin/init"]
And saving this as a base image using docker build -t baseimage .. So far so good.
The application's installer is a .run which among other things executes something like systemctl start xxx.service (I cannot change that and the installation will fail if this part fails).
I have already tried some things for the second Dockerfile, such as:
FROM baseimage
...
COPY xxx.run xxx.run
RUN ./xxx.run # This return an error like "Failed to get D-Bus connection: Operation not permitted"
...
and changing the original CMD to run this script:
#! /bin/bash
/usr/sbin/init # With and without & after this one
./xxx.run
Any ideas?
The case of using an installer for a non-dockerized application in a docker container ... is one of the achievements that work with the docker-systemctl-replacement.
Your build is trying to do a privileged operation. The issue is, --privileged flag is unavailable at build time. Here's the issue, which makes an interesting reading : https://github.com/moby/moby/issues/1916
TL;DR there seems to be two possibilities with docker itself
Enable experimental mode then RUN --security=insecure ./xxx.run, as documented here https://github.com/moby/buildkit/blob/master/frontend/dockerfile/docs/experimental.md#run---securityinsecuresandbox
Install buildx cli plugin, then docker buildx build --allow security.insecure, as documented here https://github.com/docker/buildx/blob/master/README.md#--allowentitlement
Other solution outside docker would be to use buildah, but that's a longer shot by far. But an interesting one, especially if you works in the RedHat ecosystem, since they want to get rid of Docker in favor of Buildah/Podman. Here's the get started. Though I'm not clear about how it solve your issue, it's been highlighted as being able to deal with this kind of issues.
Related
I'm on the part of the tutorial where it talks about data persistence.
First, I run this command to put a random number into a text file within an ubuntu image:
docker run -d ubuntu bash -c "shuf -i 1-10000 -n 1 -o /data.txt && tail -f /dev/null"
I think I understand this line pretty well.
Next, the instructions ask me to start a new container (the same image) and I will see that the file is not the same:
docker run -it ubuntu ls /
However, when I run the above command, I get the following error:
/ ls: cannot access 'C:/Program Files/Git/': No such file or directory
I'm running Windows 10 using Git Bash, and this is being done through VS Code.
For now, I've gotten around this issue by re-running the exact command (docker run -d ubuntu bash -c "shuf -i 1-10000 -n 1 -o /data.txt && tail -f /dev/null"), but I would like to know why the docker run -it ubuntu ls / instructions failed, and what the solution is?
I managed to solve the issue so I am posting the solution here in case people come across the same issue in the future: git bash changes absolute paths so it is something that should be disabled.
Put this into .bashrc to correct the way paths are handled:
# Workaround for Docker for Windows in Git Bash.
docker()
{
(export MSYS_NO_PATHCONV=1; "docker.exe" "$#")
}
Unfortunately, this doesn't work in scenarios where docker run is called from npm scripts, etc. Volume mapping will still break.
See here to continue exploring the issue and seeing possible workarounds
We are trying to store the container names in my Makefile but I see below error when executing the build, someone please advise. Thanks.
.PHONY: metadata
metadata: .env1
docker pull IMAGE_NAME
docker run $IMAGE_NAME;
ID:= $(shell docker ps --format '{{.Names}}')
#echo ${ID}
docker cp ${ID}:/app/.env .env2
Container names are not shown in below "ID" Variable when executing the makefile from Jenkins
ID:=
/bin/sh: ID:=: command not found
There are a couple of things you can do in terms of pure Docker mechanics to simplify this.
You can specify an alternate command when you docker run an image: anything after the image name is taken as the image to run. For instance, you can cat the file as the main container command, and replace everything you have above as:
.PHONY: getmetadata
getmetadata: .env2
.env2: .env1
docker run --rm \
-e "ARTIFACTORY_USER=${ARTIFACTORY_CREDENTIALS_USR}" \
-e "ARTIFACTORY_PASSWORD=${ARTIFACTORY_CREDENTIALS_PSW}" \
--env-file .env1 \
"${ARTIFACTDATA_IMAGE_NAME}" \
cat /app/.env \
> $#
(It is usually better to avoid docker cp, docker exec, and other imperative-type commands; it is fairly inexpensive and better practice to run a new container when you need to.)
If you can't do this, you can docker run --name your choice of names, and then use that container name in the docker cp option.
.PHONY: getmetadata
getmetadata: .env2
.env2: .env1
docker run --name getmetadata ...
docker cp getmetadata:/app/.env $#
docker stop getmetadata
docker rm getmetadata
If you really can't avoid this at all, each line of the Makefile runs in a separate shell. On the one hand this means you need to join together lines if you want variables from one line to be visible in a later line; on the other, it means you have normal shell functionality available and don't need to use the GNU Make $(shell ...) extension (which evaluates when the Makefile is loaded and not when you're running the command).
.PHONY: getmetadata
getmetadata: .env2
.env2: .env1
# Note here:
# $$ escapes $ for the shell
# Multiple shell commands joined together with && \
# Beyond that, pure Bourne shell syntax
ID=$$(docker run -d ...) && \
echo "$$ID" && \
docker cp "$$ID:/app/.env" "$#"
I'm trying to get Jenkins to run Docker that runs SystemD.
So far I've been able to run systemd inside docker locally without Jenkins. Here are the steps to run it locally without jenkins:
# pull unop/fedora-systemd and create and run the container for it
sudo docker run --cap-add=SYS_ADMIN -e container=docker --tmpfs /run --tmpfs /tmp -v /sys/fs/cgroup:/sys/fs/cgroup:ro -t -i unop/fedora-systemd
# on a different terminal window, I can:
# get the container id of the "unop/fedora-systemd" image
sudo docker ps
# then exec bash on it
sudo docker container exec -t -i a98aa2bcd19e bash # where a98aa2bcd19e is the container id found above
# once inside the container, I can run systemd without any problems. examples:
systemctl status
systemctl start dbus.service
systemctl status dbus.service
The above works locally and I am able to run systemd inside the docker container.
The problem I get is when I try the same thing, but inside Jenkins.
I've tried to tweak Jenkinsfile several times, but not of my previous tries seemed to work. I always get an error when running under Jenkins similar to this:
+ systemctl status
System has not been booted with systemd as init system (PID 1). Can't operate.
Failed to connect to bus: Host is down
This is my latest Jenkinsfile that I've tried
pipeline {
agent {
docker {
image 'unop/fedora-systemd'
args '--cap-add=SYS_ADMIN -e container=docker --tmpfs /run --tmpfs /tmp -v /sys/fs/cgroup:/sys/fs/cgroup:ro -t -i'
}
}
stages {
stage('test') {
steps {
sh "echo hello world"
sh "systemctl status"
sh "systemctl start dbus.service"
sh "systemctl dbus.service"
}
}
}
}
On previous iterations of the Jenkinsfile, I've tried to replace -cap-add=SYS_ADMIN -e container=docker for --privileged, but that didn't help, I still got the same errors
Anyone have an idea of how can I get this to work? Why does the above work locally, but not on Jenkins? what am I missing here?
Note: Jenkins version: 2.150.2 and this is the Dockerfile used by unop/fedora-systemd
FROM fedora:rawhide
MAINTAINER http://fedoraproject.org/wiki/Cloud
ENV container docker
RUN dnf -y update && dnf clean all
RUN dnf -y install systemd && dnf clean all && \
(cd /lib/systemd/system/sysinit.target.wants/; for i in *; do [ $i == systemd-tmpfiles-setup.service ] || rm -f $i; done); \
rm -f /lib/systemd/system/multi-user.target.wants/*;\
rm -f /etc/systemd/system/*.wants/*;\
rm -f /lib/systemd/system/local-fs.target.wants/*; \
rm -f /lib/systemd/system/sockets.target.wants/*udev*; \
rm -f /lib/systemd/system/sockets.target.wants/*initctl*; \
rm -f /lib/systemd/system/basic.target.wants/*;\
rm -f /lib/systemd/system/anaconda.target.wants/*;
VOLUME [ "/sys/fs/cgroup", "/tmp", "/run" ]
CMD ["/usr/sbin/init"]
PS: I've seen a related question, but what they were asking is different
I did not know about the related question. Let me point out again that you do not need to run a systemd daemon in a systemd controlled container if it is just about running multiple services in it. Simply overwrite /usr/bin/systemctl with the docker-systemctl-replacement script. Then go to register it with CMD ["/usr/bin/systemctl"] as the init process of the container.
That's it. Now you can run any systemctl-start process from the operating system. It works to the extent that even provisioning with ansible/puppet scripts have no problem at all. An specficially, I am using that to provision Jenkins images with the operating system that the developers like to have as a basis. No priviledged mode required.
You may try an image that has Fedora with System D already active with this command:
docker run -d --name systemd-fedora --privileged -v /sys/fs/cgroup:/sys/fs/cgroup:ro jrei/systemd-fedora
Then you just need to run:
docker exec -it systemd-fedora /bin/bash
and there you can just install, start and restart any service you need.
I have created a docker container that runs a command line tool. The container is supposed to be interactive. Am I somehow able to specify in the Dockerfile that the container is always started in interactive mode?
For reference this is the dockerfile:
FROM ubuntu:latest
RUN apt-get update && apt-get -y install curl
RUN mkdir adr-tools && \
cd adr-tools && \
curl -L https://github.com/npryce/adr-tools/archive/2.2.0.tar.gz --output adr-tools.tar.gz && \
tar -xvzf adr-tools.tar.gz && \
cp */src/* /usr/bin && \
rm -rf adr-tools
CMD ["/bin/bash"]
EDIT:
I know of the -it options for the run command. I'm explicitly asking for a way to do this in the docker file.
EDIT2:
This is not a duplicate of Interactive command in Dockerfile since my question addresses an issue with how arguments specified to docker run can be avoided in favor of specifying them in the Dockerfile whereas the supposed duplicate addresses an issue of interactive input during the build of the image by docker itself.
Many of the docker run options can only be specified at the command line or via higher-level wrappers (shell scripts, Docker Compose, Kubernetes, &c.). Along with port mappings and network settings, the “interactive” and “tty” options can only be set at run time, and you can’t force these in the Dockerfile.
You can use the docker run command.
docker build -t curly .
docker run -it curly curl https://stackoverflow.com
The convention is:
docker run -it IMAGE_NAME [COMMAND] [ARG...]
Where [COMMAND] is curl and [ARG...] are the curl arguments, which is https://stackoverflow.com in my example.
-i enables interactive process mode. You can't specify this in the Dockerfile.
-t allocates a pseudo-TTY for the container.
Are you looking for the -it option?
From the Docker documentation:
For interactive processes (like a shell), you must use -i -t together
in order to allocate a tty for the container process.
So, for example you can run it like:
docker run -it IMAGE_NAME [COMMAND] [ARG...]
Actually, in Ubuntu, I am running Apache Server in the background.
But for you, Try with below command and you should be able to go inside docker container.
docker exec -i -t your_container_name bash
I want to install docker inside a running docker container.
docker run -it centos:centos7
My base container is using centos, I can login to running container using docker exec. But when I try to install docker inside it using yum install -y docker it installs.
But somehow I can't start the docker service with docker -d &, it gives me error as:
INFO[0000] Option DefaultNetwork: bridge
WARN[0000] Running modprobe bridge nf_nat br_netfilter failed with message: , error: exit status 1
FATA[0000] Error starting daemon: Error initializing network controller: Error initializing bridge driver: Setup IP forwarding failed: open /proc/sys/net/ipv4/ip_forward: read-only file system
Is there a way I can install docker inside docker container or build image already having running docker? I have already seen these examples but none works for me.
The output of uname -r on the host machine:
[fedora# ~]$ uname -r
4.2.6-200.fc22.x86_64
Any help would be appreciated.
Thanks in advance
Update
Thanks to https://stackoverflow.com/a/38016704/372019 I want to show another approach.
Instead of mounting the host's docker binary, you should copy or install a container specific release of the docker binary. Since you're only using it in a client mode, you won't need to install it as a system service. You still need to mount the Docker socket into the container so that you can easily communicate with the host's Docker engine.
Assuming that you got a base image with a working Docker binary (e.g. the official docker image), the example now looks like this:
docker run\
-v /var/run/docker.sock:/var/run/docker.sock\
docker:1.12 docker info
Without actually answering your question I'd suggest you to read Using Docker-in-Docker for your CI or testing environment? Think twice.
It explains why running docker-in-docker should be replaced with a setup where Docker containers run as siblings of the "outer" or "base" container. The article also links to the original https://github.com/jpetazzo/dind project where you can find working examples how to run Docker in Docker - in case you still want to have docker-in-docker.
An example how to enable a container to access the host's Docker daemon look like this:
docker run\
-v /var/run/docker.sock:/var/run/docker.sock\
-v /usr/bin/docker:/usr/bin/docker\
busybox:latest /usr/bin/docker info
If you are on Mac with Docker toolbox.
The below command WON’T WORK
docker run\
-v /var/run/docker.sock:/var/run/docker.sock\
-v /usr/bin/docker:/usr/bin/docker\
busybox:latest /usr/bin/docker info
Because /var/run/docker.sock will not be on your OSX filesystem
the Docker daemon is running inside the boot2docker VM - and that's where the unix socket is.
So you have to run the container from boot2docker VM
$ docker-machine ssh default
$ docker run\
-v /var/run/docker.sock:/var/run/docker.sock\
-v $(which docker):/usr/bin/docker\
busybox:latest /usr/bin/docker info
$ exit
This looks like Docker-in-Docker, feels like Docker-in-Docker, but it’s not Docker-in-Docker, when this container will create more containers, those containers will be created in the top-level Docker.
You need the --privileged parameter.
By default, Docker containers are “unprivileged” and cannot, for
example, run a Docker daemon inside a Docker container.
Source
Run your base image with the command docker run --privileged -it centos:centos7 bash. Then you may install and run another docker container inside that container.
I`ve a similar problems in my vms.
I`ve solve the problem with change the storage file system from image to vfs(in daemon.json file)
like the image bellow
For image works first create a base image, in my case with centos7
FROM centos:7
ENV container docker
RUN (cd /lib/systemd/system/sysinit.target.wants/; for i in *; do [ $i == \
systemd-tmpfiles-setup.service ] || rm -f $i; done); \
rm -f /lib/systemd/system/multi-user.target.wants/*;\
rm -f /etc/systemd/system/*.wants/*;\
rm -f /lib/systemd/system/local-fs.target.wants/*; \
rm -f /lib/systemd/system/sockets.target.wants/*udev*; \
rm -f /lib/systemd/system/sockets.target.wants/*initctl*; \
rm -f /lib/systemd/system/basic.target.wants/*;\
rm -f /lib/systemd/system/anaconda.target.wants/*;
VOLUME [ "/sys/fs/cgroup" ]
CMD ["/usr/sbin/init"]
with this image builded (in my case i called local/c7-systemd) create a second image, installing docker and moving daemon.json to inside.
FROM local/c7-systemd
RUN yum install -y yum-utils
RUN yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
RUN yum install -y docker-ce docker-ce-cli containerd.io
RUN curl -L "https://github.com/docker/compose/releases/download/1.28.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
RUN chmod +x /usr/local/bin/docker-compose
RUN ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose
COPY daemon.json /etc/docker/daemon.json
RUN yum install -y nano
RUN systemctl enable docker
EXPOSE 80
EXPOSE 8080
EXPOSE 8161
EXPOSE 6379
EXPOSE 8761
CMD ["/usr/sbin/init"]
enjoy!