Docker RabbitMQ persistency - docker

RabbitMQ in docker lost data after remove container without volume.
My Dockerfile:
FROM rabbitmq:3-management
ENV RABBITMQ_HIPE_COMPILE 1
ENV RABBITMQ_ERLANG_COOKIE "123456"
ENV RABBITMQ_DEFAULT_VHOST "123456"
My run script:
IMAGE_NAME="service-rabbitmq"
TAG="${REGISTRY_ADDRESS}/${IMAGE_NAME}:${VERSION}"
echo $TAG
docker rm -f $IMAGE_NAME
docker run \
-itd \
-v "rabbitmq_log:/var/log/rabbitmq" \
-v "rabbitmq_data:/var/lib/rabbitmq" \
--name "service-rabbitmq" \
--dns=8.8.8.8 \
-p 8080:15672 \
$TAG
After removing the container, all data are lost.
How do I configure RabbitMQ in docker with persistent data?

Rabbitmq uses the hostname as part of the folder name in the mnesia
directory. Maybe add a --hostname some-rabbit to your docker run?
I had the same issue and I found the answer here.

TL;DR
Didn't do too much digging on this, but it appears that the simplest way to do this is to change the hostname as Pedro mentions above.
MORE INFO:
Using RABBITMQ_NODENAME
If you want to edit the RABBITMQ_NODENAME variable via Docker, it looks like you need to add a hostname as well since the Docker hostnames are generated as random hashes.
If you change the RABBITMQ_NODENAME var to something static like my-rabbit, RabbitMQ will throw something like an "nxdomain not found" error because it's looking for something likemy-rabbit#<docker_hostname_hash>. If you know the Docker hostname and can automate pulling it into your RABBITMQ_NODENAME value like so, my-rabbit#<docker_hostname_hash> I believe it would work.
UPDATE
I previously said,
If you know the Docker hostname and can automate pulling it into your RABBITMQ_NODENAME value like so, my-rabbit#<docker_hostname_hash> I believe it would work.
This would not work as described precisely because the default docker host name is randomly generated at launch, if it is not assigned explicitly. The hurdle would actually be to make sure you use the EXACT SAME <docker_hostname_hash> as your originating run so that the data directory gets picked up correctly. This would be a pain to implement dynamically/robustly. It would be easiest to use an explicit hostname as described below.
The alternative would be to set the hostname to a value you choose -- say, app-messaging -- AND ALSO set the RABBITMQ_NODENAME var to something like rabbit#app-messaging. This way you are controlling the full node name that will be used in the data directory.
Using Hostname
(Recommended)
That said, unless you have a reason NOT to change the hostname, changing the hostname alone is the simplest way to ensure that your data will be mounted to and from the same point every time.
I'm using the following Docker Compose file to successfully persist my setup between launches.
version: '3'
services:
rabbitmq:
hostname: 'mabbit'
image: "${ARTIFACTORY}/rabbitmq:3-management"
ports:
- "15672:15672"
- "5672:5672"
volumes:
- "./data:/var/lib/rabbitmq/mnesia/"
networks:
- rabbitmq
networks:
rabbitmq:
driver: bridge
This creates a data directory next to my compose file and persists the RabbitMQ setup like so:
./data/
rabbit#mabbit/
rabbit#mabbit-plugins-expand/
rabbit#mabbit.pid
rabbit#mabbit-feature_flags

Related

docker-compose in gitlab-ci: expose ports

I like to set up a gitlab repository and a gitlab-ci with docker-compose for integration tests.
I finally managed to start some containers with docker-compose.
image: docker/compose:1.29.2
variables:
DOCKER_HOST: tcp://docker:2375/
DOCKER_HOSTNAME: myhost
DOCKER_DRIVER: overlay2
DOCKER_TLS_CERTDIR: "" #TODO
services:
- name: docker:dind
alias: docker
command: [
"--registry-mirror=https://artifactory.mycorp.net"
]
Now I am faced with the problem, that I need network interaction with some services that run on a windows server with (for the tests irrelevant) UI and stuff - so I can not dockerize them with adequate effort.
The gitlab-runner runs exclusively on the server for this one project only!
My idea is, I need to get the docker:dind-service to be on the host-network, so the docker-containers that are spawned inside that service will be available through their explicitly exposed ports. However I have no clue on how I might achieve that.
Any other way to solve that problem is also welcome!
I figured a solution that seems to run for now:
Create a network, e.g docker network create gitlab-runner (bridge-mode)
Configure the gitlab-runner to use that network by setting the network_mode to afore created networks name (e.g. "gitlab-runner").
Start the dind-container manually, connected to afore created network, exposing the necessary port(ranges)
Don't create the dind-container as service in the .gitlab-ci.yml
So far it seems to work for a minimal example starting up zookeeper, kafka and a kafka-restproxy.
For the full project I still have some errors, but I assume they are unrelated to this issue. If it turns out to be wrong, I'll keep you updated.
Actually the errors are related to this issue: With this method, the checked out project is available in the docker/compose-container. When starting the containers, this is done from the context of the dind-container, in which the files are not present.
The files could be copied with a "build"-step in a new docker container first or made available through a shared volume.
You gotta make sure that both services are within the same docker network to achieve what you want. Generally when creating images they are assume different networks but this can be configured by u to allow them share the same network.
Below is an example of what I mean. Do the same for ur containers
docker network create db_network docker run -d \ --name mysql-spi1 \ --publish 3306 \ --network db_network \ --restart unless-stopped \ --env MYSQL_ROOT_PASSWORD=ebsbduabdc \ --volume mysqlspi-datadir:/var/lib/mysql \ mysql:8 \ --default-authentication-plugin=mysql_native_password

Sharing data between docker containers without making data persistent

Let's say I have a docker-compose file with two containers:
version: "3"
services:
app:
image: someimage:fpm-alpine
volumes:
- myvolume:/var/www/html
web:
image: nginx:alpine
volumes:
- myvolume:/var/www/html
volumes:
myvolume:
The app container contains the application code in the /var/www/html directory which gets updated with each version of the image, so I don't want this directory to be persistent.
Yet I need to share the data with the nginx container. If I use a volume or a host bind the data is persistent and doesn't get updated with a new version. Maybe there is a way to automatically delete a volume whenever I pull a new image? Or a way to share an anonymous volume?
i think its better for you to use anonymous volume
volumes:
- ./:/var/www/html
You would have to be willing to drop back to docker-compose version 2 and use data containers with the volumes_from directive.
Which is equivalent to --volumes-from on a docker run command.
This should work fine. The problem isn't with docker. You can use volumes to communicate in this way. If you run docker-compose up in a directory with the following compose file:
version: "3"
services:
one:
image: ubuntu
command: sleep 100000
volumes:
- vol:/vol
two:
image: ubuntu
command: sleep 100000
volumes:
- vol:/vol
volumes:
vol:
Then, in a 2nd terminal docker exec -it so_one_1 bash (you might have to do a docker ps to find the exact name of the container, it can change). You'll find yourself in a bash container. Change to the /vol directory cd /vol and then echo "wobble" > wibble.txt", then exit` the shell (ctrl-d).
In the same terminal you can then type docker exec -it so_two_1 bash (again, check the names). Just like last time you can cd /vol and type ls -gAlFh you'll see the wibble.txt file we created in the other container. You can even cat wibble.txt to see the contents. It'll be there.
So if the problem isn't docker, what can it be? I think the problem is that nginx isn't seeing the changes on the filesystem. For that, I believe that setting expires -1; inside a location block in the config will actually disable caching completely and may solve the problem (dev only).

Docker-compose redis: start with fixture?

I am using docker-compose to create a redis container. However, I need it to start with some default key values. Is this possible?
You need to modify your DockerCompose file, You can also add from some file which contains key value but here is the simplest example that adds and get key in DockerCompose file.
version: '2'
services:
redis:
image: 'bitnami/redis:latest'
environment:
- ALLOW_EMPTY_PASSWORD=yes
ports:
- '6379:6379'
command:
- /bin/sh
- -c
- |
nohup redis-server &
sleep 5
echo "adding some default key value"
redis-cli SET docker awesome
echo "Get docker key value"
redis-cli GET docker
# this will keep container running
tail -f /dev/null
There are several approaches but be aware that, by default, services start in an arbitrary order using Docker Compose and, even if you use depends_on this only checks that containers are running (e.g. redis) and not that they've completed some initialization process.
1. Easiest: Pre-create
See the option to run the redis image with persistent storage:
https://hub.docker.com/_/redis/
Using this approach, you'd either mount a local directory into the container's /data directory or create a (data) volume and use that. Then, you'd pre-populate the redis server by running the redis-cli against it.
One hack to doing this is to your planned docker-compose.yml file but docker-compose --file=/path/to/docker-compost.yaml up redis where redis is the name of the redis service too. You'll need to ensure the redis service is accessible from the host --ports: 6379:6379 perhaps so that the external redis-cli can access it.
This approach works well for local-only use but does not facilitate deploying the solution elsewhere.
2. Resilient: Test for keys
Docker Compose -- to my knowledge -- doesn't offer an elegant equivalent to Kubernetes' init containers which are run before the dependent container.
With Docker Compose, you could include an initialization (run once) redis-cli to populate the server but you must then augment any clients to check that this has completed or for the existence of this data before starting (successfully).
The simplest solution for this is for the redis clients to fail and restart: always if the redis keys aren't present.
A more advanced solution would be to define a healthcheck for the existence of the redis keys and then depends_upon: ... condition: service_healthy (see link)
See also startup order in Docker Compose described here

Advantage of using docker-compose file version 3 over a shellscript?

My initial reason for creating a docker-compose.yml, was to take advantage of features such as build: and depends-on: to make a single file that builds all my images and runs them in containers. However, I noticed version
3 depreciates most of these functions, and I'm curious why I would use this over building a shellscript.
This is currently my shellscript that runs all my containers (I assume this is what the version 3 docker-compose file would replace if I were to use it):
echo "Creating docker network net1"
docker network create net1
echo "Running api as a container with port 5000 exposed on net1"
docker run --name api_cntr --net net1 -d -p 5000:5000 api_img
echo "Running redis service with port 6379 exposed on net1"
docker run --name message_service --net net1 -p 6379:6379 -d redis
echo "Running celery worker on net1"
docker run --name celery_worker1 --net net1 -d celery_worker_img
echo "Running flower HUD on net1 with port 5555 exposed"
docker run --name flower_hud --net net1 -d -p 5555:5555 flower_hud_img
Does docker-swarm rely on using stacks? If so then I can see a use for docker-compose and stacks, but I couldn't seem to find an answer online. I would use version 3 because it is compatible with swarm, unlike version 2 if what I've read it true. Maybe I am missing the point of docker-compose completely, but as of right I'm a bit confused as to what it brings to the table.
Readability
Compare your sample shell script to a YAML version of same:
services:
api_cntr:
image: api_img
network: net1
ports:
- 5000:5000
message_service:
image: redis
network: net1
ports:
- 6379:6379
celery_worker1:
image: celery_worker_img
network: net1
flower_hud:
image: flower_hud_img
network: net1
ports:
- 5555:5555
To my eye at least, it is much easier to determine the overall architecture of the application from reading the YAML than from reading the shell commands.
Cleanup
If you use docker-compose, then running docker-compose down will stop and clean up everything, remove the network, etc. To do that in your shell script, you'd have to separately write a remove section to stop and remove all the containers and the network.
Multiple inheriting YAML files
In some cases, such as for dev & testing, you might want to have a main YAML file and another that overrides certain values for dev/test work.
For instance, I have an application where I have a docker-compose.yml as well as docker-compose.dev.yml. The first contains all of the production settings for my app. But the "dev" version has a more limited set of things. It uses the same service names, but with a few differences.
Adds a mount of my code directory into the container, overriding the version of the code that was built into the image
Exposes the postgres port externally (so I can connect to it for debugging purposes) - this is not exposed in production
Uses another mount to fake a user database so I can easily have some test users without wiring things up to my real authentication server just for development
Normally the service only uses docker-compose.yml (in production). But when I am doing development work, I run it like this:
docker-compose -f docker-compose.yml -f docker-compose.dev.yml up -d
It will load the normal parameters from docker-compose.yml first, then read docker-compose.dev.yml second, and override only the parameters found in the dev file. The other parameters are all preserved from the production version. But I don't require two completely separate YAML files where I might need to change the same parameters in both.
Ease of maintenance
Everything I described in the last few paragraphs can be done using shell scripts. It's just more work to do it that way, and probably more difficult to maintain, and more prone to mistakes.
You could make it easier by having your shell scripts read a config file and such... but at some point you have to ask if you are just reimplementing your own version of docker-compose, and whether that is worthwhile to you.

Export RabbitMQ Docker image with vhost and queues

I have a rabbitMQ docker container that I started using the following command:
docker run -d --name myrabbit1 -p 15672:15672 rabbitmq:3-management
I then loggin to the management plugin and create users, vhosts, queues, etc.
I want to save all those settings so they can be loaded up again. To do that I tried committing to a new image:
docker commit myrabbit1 vbrabbit:withVhostAndQueues
I then start up my new container (after stopping the old one):
docker run -d --name vbrabbit2 -p 15672:15672 -p 5672:5672 vbrabbit:withVhostAndQueues
I expect that all the queues, vhosts, etc would be saved, but they are not.
What am I missing?
Result from docker ps -a:
I want to save all those settings so they can be loaded up again
are you needing to create a copy of the container, with the same settings?
or are you just looking to docker stop myrabbit1 and then later docker start myrabbit to run the same container, again?
TL;DR
The RabbitMQ instance within the container is looking for data in a different place. The default configuration changes the data storage/load location per container creation. Thus the OPs data existed in the created "final" image but rabbitmq wasn't loading it.
To fix statically set RABBITMQ_NODENAME which likewise might requiring adding another line to /etc/hosts for RabbitMQ to affirm the node is active.
Details
This happened to me with docker rabbit:3.8.12-management
This is caused by RabbitMQ's default configuration impacting how it does data storage. By default RabbitMQ starts a node on UNIX system with a name of rabbit#$HOSTNAME (see RABBITMQ_NODENAME on config docs). In Docker the $HOSTNAME changes per container run it defaults to the container id (e.g. something like dd84759287560).
In #jhilden's case is when the vbrabbit:withVhostAndQueues image is booted as a new container the RABBITMQ_NODENAME becomes a different value then what was used to create and store the original vhosts, user, queues, etc. And as RabbitMQ stores data inside a directory named after the RABBITMQ_NODENAME the existing data isn't loaded on boot of vbrabbit:withVhostAndQueues. As when the $HOSTNAME changes the RABBITMQ_NODENAME changes. Thus the booting RabbitMQ instance cannot find any existing data. (e.g. the existing data is there in the image but for a different RABBITMQ_NODENAME and isn't loaded).
Note: I've only looked into solving this for a local development single instance cluster. If you're using RabbitMQ docker for a production deployment you'd probably need to look into customized hostnames
To fix this issue we set a static RABBITMQ_NODENAME for the container.
In docker-compose v3 file we updated from:
# Before fix
rabbitmq:
image: "rabbitmq:$RABBITMQ_VERSION"
container_name: rabbitmq
ports:
- "5672:5672"
- "15672:15672"
- "61613:61613"
volumes:
- "./etc/rabbit-plugins:/etc/rabbitmq/enabled_plugins"
- type: volume
source: rabbitmq-data
target: /var/lib/rabbitmq
Into after fix:
rabbitmq:
image: "rabbitmq:$RABBITMQ_VERSION"
container_name: rabbitmq
# Why do we set NODENAME and extra_hosts?
#
# It codifies that we're using the same RabbitMQ instance between container rebuilds.
# If NODENAME is not set it defaults to "rabbit#$HOST" and because $HOST is dynamically
# created in docker it changes per container deployment. Why is a changing host an issue?
# Well because under the hood Rabbit stores data on a per node basis. Thus without the
# static RABBITMQ_NODENAME the directory the data is stored within changes per restart.
# Going from "rabbit#7745942c559e" to "rabbit#036834720485" the next. Okay, but why do we
# need extra_hosts? We'll Rabbit wants to resolve itself to affirm it's management UI is
# functioning post deployment and does that with an HTTP call. Thus to resolve the static
# host from RABBITMQ_NODENAME we need to add it to the containers /etc/hosts file.
environment:
RABBITMQ_NODENAME: "rabbit#staticrabbit"
extra_hosts:
- "staticrabbit:127.0.0.1"
ports:
- "5672:5672"
- "15672:15672"
- "61613:61613"
volumes:
- "./etc/rabbit-plugins:/etc/rabbitmq/enabled_plugins"
- type: volume
source: rabbitmq-data
target: /var/lib/rabbitmq

Resources