There is any "Podman Compose"? - docker

I recently found out about Podman (https://podman.io). Having a way to use Linux fork processes instead of a Daemon and not having to run using root just got my attention.
But I'm very used to orchestrate the containers running on my machine (in production we use kubernetes) using docker-compose. And I truly like it.
So I'm trying to replace docker-compose. I will try to keep docker-compose and using podman as an alias to docker as Podman uses the same syntax as docker:
alias docker=podman
Will it work? Can you suggest any other tool? I really intend to keep my docker-compose.yml file, if possible.

Yes, that is doable now, check podman-compose, this is one way of doing it, another way is to convert the docker-compose yaml file to a kubernetes deployment using Kompose. there is a blog post from Jérôme Petazzoni #jpetazzo: from docker-compose to kubernetes deployment

Update 6 May 2022 : Podman now supports Docker Compose v2.2 and higher (see Podman 4.1.0 release notes)
Old answer:
Running docker-compose with Podman as a normal user (rootless)
Requirement: Podman version >= 3.2.1 (released in June 2021)
Install the executable docker-compose
curl -sL -o ~/docker-compose https://github.com/docker/compose/releases/latest/download/docker-compose-$(uname -s)-$(uname -m)
chmod 755 ~/docker-compose
Alternatively you could also run docker-compose in a container image (see below).
Run
systemctl --user start podman.socket
Set the environment variable DOCKER_HOST
export DOCKER_HOST=unix://$XDG_RUNTIME_DIR/podman/podman.sock
Run
~/docker-compose up -d
Running docker-compose with Podman as root
Requirement: Podman version >= 3.0 (released in February 2021)
Follow the same procedure but remove the flag --user
systemctl start podman.socket
Running docker-compose in a container image
Use the container image docker.io/docker/compose to run
docker-compose
podman \
run \
--rm \
--detach \
--env DOCKER_HOST=unix://$XDG_RUNTIME_DIR/podman/podman.sock \
--security-opt label=disable \
--volume $XDG_RUNTIME_DIR/podman/podman.sock:$XDG_RUNTIME_DIR/podman/podman.sock \
--volume $(pwd):$(pwd) \
--workdir $(pwd) \
docker.io/docker/compose \
--verbose \
up -d
(the flag --verbose is optional)
The same command with short command-line options on a single line:
podman run --rm -d -e DOCKER_HOST=unix://$XDG_RUNTIME_DIR/podman/podman.sock --security-opt label=disable -v $XDG_RUNTIME_DIR/podman/podman.sock:$XDG_RUNTIME_DIR/podman/podman.sock -v $(pwd):$(pwd) -w $(pwd) docker.io/docker/compose --verbose up -d
Regarding SELINUX: Runnng Podman with SELINUX is preferable from a security point-of-view, but I didn't get it to work on a Fedora 34 computer so I disabled SELINUX by adding the command-line option
--security-opt label=disable
Troubleshooting tips
Test the Docker REST API
A minimal check to see that the Docker REST API is working:
$ curl -H "Content-Type: application/json" \
--unix-socket $XDG_RUNTIME_DIR/podman/podman.sock \
http://localhost/_ping
OK$
Avoid short container image names
If any of your docker-compose.yaml or Dockerfile files contain a short container image name, for instance
$ grep image: docker-compose.yaml
image: mysql:8.0.19
$
$ grep FROM Dockerfile
FROM python:3.9
$
edit the files to use the whole container image name instead
$ grep image: docker-compose.yaml
image: docker.io/library/mysql:8.0.19
$
$ grep FROM Dockerfile
FROM docker.io/library/python:3.9
$
Most often short names have been used to reference DockerHub Official Images
(a catalogue) so a good guess would be to prepend the container image name with docker.io/library/
There are currently many different container image registries, not just DockerHub (docker.io). Writing the whole container image name is thus a good practice. Podman might complain otherwise depending on how Podman is configured.
Rootless users can't bind to ports below 1024
If for instance
$ grep -A1 ports: docker-compose.yml
ports:
- 80:80
$
edit docker-compose.yaml so that the host port number >= 1024, for instance 8080
$ grep -A1 ports: docker-compose.yml
ports:
- 8080:80
$
An alternative solution is to adjust net.ipv4.ip_unprivileged_port_start with sysctl (see Shortcomings of Rootless Podman)
In case Systemd is missing
Most Linux distributions use Systemd where you would preferably start the Podman service (providing the REST API) by "starting" the Podman socket
systemctl --user start podman.socket
or
systemctl start podman.socket
but in case Systemd is missing you could also start the Podman service directly
podman system service --time 0 unix:/some/path/podman.sock
Systemd gives the extra benefit that the Podman service is started on demand with Systemd socket activation and stops after some time of inactivity.
Caveat: Swarm functionality is missing
A difference to Docker is that the functionality relating to Swarm is not supported when using docker-compose with Podman.
References:
https://www.redhat.com/sysadmin/podman-docker-compose
https://github.com/containers/podman/discussions/10644#discussioncomment-857897

Ensure Podman is installed on your machine.
You can install Podman Compose in a terminal with the following command:
pip3 install https://github.com/containers/podman-compose/archive/devel.tar.gz
cd into the directory your docker-compose file is located in
Run podman-compose up
See the following link for a decent introduction.

Related

How an app in docker container access DB in windows?

OS: Windows server 2016
I have an App wrote in Go and put in a docker container. The App has to access "D:\test.db". How can I do that?
Using docker volumes and by using the -v or --mount flag when you start your container.
A modified example from the Docker docs:
$ docker run -d \
--mount source=myvol2,target=/app \
nginx:latest
you just need to replace nginx:latext with your image name and adapt source and target as you need.
Another example (also from the docs) using -v and mounting in read-only mode:
$ docker run -d \
-v nginx-vol:/usr/share/nginx/html:ro \
nginx:latest

Run bitcoind with bitcoind.conf in docker

I know docker, but less about bitcoind.
Now I want to use this docker image to start my own test environment:
The description tells me:
docker volume create --name=bitcoind-data
docker run -v bitcoind-data:/bitcoin --name=bitcoind-node -d \
-p 8333:8333 \
-p 127.0.0.1:8332:8332 \
kylemanna/bitcoind
Now I want to now how I have to add my bitcoind.conf?
This isn't provided anywere? Can I use it at container startup or docker exec?
The repository contains a documentation file dedicated to your issue: https://github.com/kylemanna/docker-bitcoind/blob/master/docs/config.md

Unable to mount cifs filesystem in Docker container

I'm on Docker 17.06.0-ce and I'm attempting to mount a CIFS share in a container and only having some luck. If I use --privileged, it works, but that's not desirable for me. I've tried using --cap-add as well as suggested in this answer (even trying with --cap-add ALL with no success.
The same mount command works fine on the host system as well.
Here's a simple docker file I've tried playing with
FROM alpine:latest
RUN apk add --no-cache cifs-utils
Run with many different permutations, all with the same result below:
Works:
docker run --rm -it --privileged cifs-test /bin/sh
Doesn't Work:
docker run --rm -it --cap-add SYS_ADMIN --cap-add DAC_READ_SEARCH cifs-test /bin/sh
Doesn't Work:
docker run --rm -it --cap-add SYS_ADMIN --cap-add DAC_READ_SEARCH --cap-add NET_ADMIN cifs-test /bin/sh
Doesn't Work:
docker run --rm -it --cap-add ALL cifs-test /bin/sh
And the command:
mkdir /test && mount.cifs //myserver/testpath /test -o user=auser,password=somepass,domain=mydomain
And the result from each run command above except the first:
mount error(13): Permission denied
Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)
Has something changed in Docker that requires --privileged all the time for these types of mounts now? Or is there something else I'm missing?
I started using docker-volume-netshare so far with good success. There are some minor problems, like volumes created with docker volume create not being persistent, but nevertheless it looks like this volume driver is quite usable. One advantage is that special caps/privileged mode are not necessary. Here are some hints on how to use it.
Install (Ubuntu/Debian)
$ curl -L -o /tmp/docker-volume-netshare_0.34_amd64.deb https://github.com/ContainX/docker-volume-netshare/releases/download/v0.34/docker-volume-netshare_0.34_amd64.deb
$ sudo dpkg -i /tmp/docker-volume-netshare_0.34_amd64.deb
$ rm /tmp/docker-volume-netshare_0.34_amd64.deb
Configure
$ sudo vi /etc/default/docker-volume-netshare
enter as single setting
DKV_NETSHARE_OPTS="cifs --netrc=/root/"
then
$ sudo vi /root/.netrc
enter the following settings per host:
machine <host>
username <user>
password <password>
domain <domain>
Note that <host> must be a host name or an IP address followed by a colon (e.g. 10.20.30.4:)
Enable the volume driver as a systemd service
Note: if your OS does not support systemd, another method to install it as a service is necessary.
$ sudo systemctl enable docker-volume-netshare
Use a volume in docker run and docker service create
$ sudo docker run -it --rm --mount type=volume,volume-driver=cifs,source=<myvol>,destination=<absolute-path-in-container>,volume-opt=share=<ip>:/<share> ubuntu:zesty bash
$ sudo docker service create --name <name> --mount type=volume,volume-driver=cifs,source=<myvol>,destination=<absolute-path-in-container>,volume-opt=share=<host>/<share> <image>
Obviously it is not necessary to use the identical volume in multiple containers, because the volumes only map to a cifs share which in turn is shared among containers mounting it. As mentioned above, don't use docker volume create with this volume driver, as volumes are lost as soon as docker-volume-netshare is stopped and/or restarted (and hence on reboot).
Get help
$ docker-volume-netshare --help
$ docker-volume-netshare cifs --help
Logs
Hint: for debugging use DKV_NETSHARE_OPTS="cifs --netrc=/root/ --verbose" in /etc/default/docker-volume-netshare or stop the service and start docker-volume-netshare cifs --netrc=/root/ --verbose in a shell)
$ dmesg | tail
$ tail -50 /var/log/docker-volume-netshare.log
Resources
github
project

Docker Compose to CoreOS

I'm currently learning Docker, and have made a nice and simple Docker Compose setup. 3 containers, all with their own Dockerfile setup. How could I go about converting this to work on CoreOS so I can setup up a cluster later on?
web:
build: ./app
ports:
- "3030:3000"
links:
- "redis"
newrelic:
build: ./newrelic
links:
- "redis"
redis:
build: ./redis
ports:
- "6379:6379"
volumes:
- /data/redis:/data
taken from https://docs.docker.com/compose/install/
the only thing is that /usr is read only, but /opt/bin is writable and in the path, so:
sd-xx~ # mkdir /opt/
sd-xx~ # mkdir /opt/bin
sd-xx~ # curl -L https://github.com/docker/compose/releases/download/1.3.3/docker-compose-`uname -s`-`uname -m` > /opt/bin/docker-compose
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 403 0 403 0 0 1076 0 --:--:-- --:--:-- --:--:-- 1080
100 7990k 100 7990k 0 0 2137k 0 0:00:03 0:00:03 --:--:-- 3176k
sd-xx~ # chmod +x /opt/bin/docker-compose
sd-xx~ # docker-compose
Define and run multi-container applications with Docker.
Usage:
docker-compose [options] [COMMAND] [ARGS...]
docker-compose -h|--help
Options:
-f, --file FILE Specify an alternate compose file (default: docker-compose.yml)
-p, --project-name NAME Specify an alternate project name (default: directory name)
--verbose Show more output
-v, --version Print version and exit
Commands:
build Build or rebuild services
help Get help on a command
kill Kill containers
logs View output from containers
port Print the public port for a port binding
ps List containers
pull Pulls service images
restart Restart services
rm Remove stopped containers
run Run a one-off command
scale Set number of containers for a service
start Start services
stop Stop services
up Create and start containers
migrate-to-labels Recreate containers to add labels
I've created simple script for installing latest Docker Compose on CoreOS:
https://gist.github.com/marszall87/ee7c5ea6f6da9f8968dd
#!/bin/bash
mkdir -p /opt/bin
curl -L `curl -s https://api.github.com/repos/docker/compose/releases/latest | jq -r '.assets[].browser_download_url | select(contains("Linux") and contains("x86_64"))'` > /opt/bin/docker-compose
chmod +x /opt/bin/docker-compose
Just run it with sudo
The proper way to install or run really anything on CoreOS is either
Install it as a unit
Run in a separate docker container
For docker-compose you probably want to install it as a unit, just like you have docker as a unit. See Digital Ocean's excellent guides on CoreOS and the systemd units chapter to learn more.
Locate your cloud config based on your cloud provider or custom installation, see https://coreos.com/os/docs/latest/cloud-config-locations.html for locations.
Install docker-compose by adding it as a unit
#cloud-config
coreos:
units:
- name: install-docker-compose.service
command: start
content: |
[Unit]
Description=Install docker-compose
ConditionPathExists=!/opt/bin/docker-compose
[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/usr/bin/mkdir -p /opt/bin/
ExecStart=/usr/bin/curl -o /opt/bin/docker-compose -sL "https://github.com/docker/compose/releases/download/1.9.0/docker-compose-linux-x86_64"
ExecStart=/usr/bin/chmod +x /opt/bin/docker-compose
Note that I couldn't get the uname -s and uname -m expansions to work in the curl statement so I just replaced them with their expanded values.
Validate your config file with
coreos-cloudinit -validate --from-file path-to-cloud-config
It should output something like
myhost core # coreos-cloudinit -validate --from-file path-to-cloudconfig
2016/12/12 12:45:03 Checking availability of "local-file"
2016/12/12 12:45:03 Fetching user-data from datasource of type "local-file"
myhost core #
Note that coreos-cloudinit doesn't validate the contents-blocks in your cloud-config. Restart CoreOS when you're finished, and you're ready to go.
Update: As #Wolfgang comments, you can run coreos-cloudinit --from-file path-to-cloud-config instead of restarting CoreOS.
I would also suggest docker-compose in a docker container like the one from dduportal.
For the sake of usability I extended my cloud-config.yml as follows:
write_files:
- path: "/etc/profile.d/aliases.sh"
content: |
alias docker-compose="docker run -v \"\$(pwd)\":\"\$(pwd)\" -v /var/run/docker.sock:/var/run/docker.sock -e COMPOSE_PROJECT_NAME=\$(basename \"\$(pwd)\") -ti --rm --workdir=\"\$(pwd)\" dduportal/docker-compose:latest"
After updating the cloud-config via sudo coreos-cloudinit -from-url http-path-to/cloud-config.yml and a system reboot, you are able to use the docker-compose command like you are used to on every other machine.
CentruyLabs created a rubygem called fig2coreos
It translates fig.yml to .service files
fig is deprecated since docker-compose was created but the syntax seems to be the same so that it could probably work.
Simple 3 Steps:
sudo mkdir -p /opt/bin
Grab the command in the official website https://docs.docker.com/compose/install/ and change the output path from /usr/local/bin/docker-compose to /opt/bin :
sudo curl -L "https://github.com/docker/compose/releases/download/1.9.0/docker-compose-$(uname -s)-$(uname -m)" -o /opt/bin/docker-compose
Make executable:
sudo chmod +x /opt/bin/docker-compose
Now you have docker-compose :)
here it is,
the best way I found:
core#london-1 ~ $ docker pull dduportal/docker-compose
core#london-1 ~ $ cd /dir/where-it-is-your/docker-compose.yml
core#london-1 ~ $ docker run -v "$(pwd)":/app \
-v /var/run/docker.sock:/var/run/docker.sock \
-e COMPOSE_PROJECT_NAME=$(basename "$(pwd)")\
-ti --rm \
dduportal/docker-compose:latest up
done!
well, coreOS supports docker but it is bare bone linux with clustering suppport so you need to include a base image for all your containers ( use FROM and in Dockerfile you might also need to do RUN yum -y install bzip2 gnupg etc., ) that has the bins and libs that are needed by you app and redis ( better take some ubuntu base image )
Here you can put all of them in one container/docker or seperate if you do it seperate then you need to link the containers and optionally volume mount - docker has some good notes about it (https://docs.docker.com/userguide/dockervolumes/)
Atlast, you need to write cloud config which specifies the systemd units . In your case you will have 3 units that will be started by systemd ( systemd replaces the good old init system in coreOS) and feed it to coreos-cloudinit ( tip: coreos-cloudinit -from-file=./cloud-config -validate=false ), You also need to provide this cloud-config on the linux bootcmd for persistency.
Currently, the easiest way to use docker-compose agains a CoreOS Vagrant VM. You just need to make sure to forward Docker port.
If you are not particularly attached to using docker-compose, you can try CoreOS running Kubernetes. There are multiple options and I have implemented one of those for Azure.
For using docker-compose with Fedora CoreOS you may run into issues with python, however running docker-compose from a container works perfectly.
There is a handy bash wrapper script and it is documented in the official documentation here: https://docs.docker.com/compose/install/#alternative-install-options under the "Install as a container" section.

Docker Not Linking Containers

I'm following the userguide on dockerlinks
I followed these steps as follows :
sudo docker run -d --name db training/postgres
sudo docker run -d -P --name web --link db:db training/webapp python app.py
sudo docker inspect -f "{{ .HostConfig.Links }}" web
In this last step, the guide asserts the return value of [/db:/web/db]
What I'm receiving is <no value>
Why are the containers not linking?
The apt-get repositories include docker version 1.0.1, but HostConfig.Links doesn't show up in the documentation until version 1.3 (See the version switcher under the "Search the Docs").
You can verify that this is the problem by running docker --version.
If you want the latest version of docker (1.4), you should uninstall docker.io and follow the directions here: http://docs.docker.com/installation/ubuntulinux/#docker-maintained-package-installation . In particular see the note, as you can just run:
curl -sSL https://get.docker.com/ubuntu/ | sudo sh
That will also give you some other nice features that are missing from the version in the Ubuntu repository, such as exec.

Resources