How to add a RabbitMQ user while RabbitMQ is not live - docker

I run a RabbitMQ service and Celery in a Docker container for my server. Workers are celery instances which connect to the server via RabbitMQ.
I set up and run RabbitMQ like this:
sudo service rabbitmq-server start
rabbitmqctl add_user bunny password
rabbitmqctl add_vhost bunny_host
rabbitmqctl set_permissions -p bunny_host bunny ".*" ".*" ".*"
This has a problem: if a worker tries to connect between the service being started and the bunny user being created and given permissions, then the worker's celery instance will terminate.
I tried adding this to the Dockerfile for my server to add the user before "live" startup:
RUN sudo service rabbitmq-server start && \
rabbitmqctl add_user bunny password && \
rabbitmqctl add_vhost bunny_host && \
rabbitmqctl set_permissions -p bunny_host bunny ".*" ".*" ".*" && \
sudo service rabbitmq-server stop
But when I restarted the rabbitmq-server service within the container, the user bunny did not exist.
(If I try to use rabbitmqctl to add a user when the service is not running, it errors out.)
Any help would be much appreciated.

You can not run rabbitmqctl during the build time. Instead, you can achieve the same by using the bootstrap files. you need to update your files like the below snippets
Dockerfile
FROM rabbitmq:3.6.11-management-alpine
ADD rabbitmq.config /etc/rabbitmq/
ADD definitions.json /etc/rabbitmq/
RUN chmod 666 /etc/rabbitmq/*
rabbitmq.config
[
{
rabbit,
[
{ loopback_users, [] }
]
},
{
rabbitmq_management,
[
{ load_definitions, "/etc/rabbitmq/definitions.json" }
]
}
].
definitions.json
{
"rabbit_version": "3.6.14",
"users": [
{
"name": "user",
"password_hash": "0xZBvBD2JOGWrVO84nZ62EJuQIRehcILEiPVFB9mD4zhFcAo",
"hashing_algorithm": "rabbit_password_hashing_sha256",
"tags": "administrator"
}
],
"vhosts": [
{
"name": "/"
}
],
"permissions": [
{
"user": "community",
"vhost": "/",
"configure": ".*",
"write": ".*",
"read": ".*"
}
],
"parameters": [],
"global_parameters": [
{
"name": "cluster_name",
"value": "rabbit#rabbitmq"
}
],
"policies": [],
"queues": [],
"exchanges": [],
"bindings": []
}

Related

AWS ECS EC2 ECR not updating files after deployment with docker volume nginx

I am being stuck on issue with my volume and ECS.
I would like to attach volume so i can store there .env files etc so i dont have to recreate this manually after every deployment.
The problem is, the way I have it set up it does not update(or overwrite) files, which are pushed to ECR. So If i do code change and push it to git, it does following:
Creates new image and pushes it to ECR
It Creates new containers with image pushed to ECR (it dynamically assigns tag to the image)
when I do docker ps on EC2 I see new containers, and container with code changes is built from correct image which has just been pushed to ECR. So it seems all is working fine until this point.
But the code changes dont appear when i refresh browser nor after clearing caches.
I am attaching volume to the folder /var/www/html where sits my app, so from my understanding this code should get replaced during deployment. But the problem is, it does not replaces the code.
When I remove the volume, I can see the code changes everytime deployment finishes but I also always have to create manually .env file + run couple of commands.
PS: I have another container (mysql) which is setting volume exactly the same way and changes I do in database are persistent even after new container is created.
Please see my Docker file and taskDefinition.json to see how I deal with volumes.
Dockerfile:
FROM alpine:${ALPINE_VERSION}
# Setup document root
WORKDIR /var/www/html
# Install packages and remove default server definition
RUN apk add --no-cache \
curl \
nginx \
php8 \
php8-ctype \
php8-curl \
php8-dom \
php8-fpm \
php8-gd \
php8-intl \
php8-json \
php8-mbstring \
php8-mysqli \
php8-pdo \
php8-opcache \
php8-openssl \
php8-phar \
php8-session \
php8-xml \
php8-xmlreader \
php8-zlib \
php8-tokenizer \
php8-fileinfo \
php8-json \
php8-xml \
php8-xmlwriter \
php8-simplexml \
php8-dom \
php8-pdo_mysql \
php8-pdo_sqlite \
php8-tokenizer \
php8-pecl-redis \
php8-bcmath \
php8-exif \
supervisor \
nano \
sudo
# Create symlink so programs depending on `php` still function
RUN ln -s /usr/bin/php8 /usr/bin/php
# Configure nginx
COPY tools/docker/config/nginx.conf /etc/nginx/nginx.conf
# Configure PHP-FPM
COPY tools/docker/config/fpm-pool.conf /etc/php8/php-fpm.d/www.conf
COPY tools/docker/config/php.ini /etc/php8/conf.d/custom.ini
# Configure supervisord
COPY tools/docker/config/supervisord.conf /etc/supervisor/conf.d/supervisord.conf
# Make sure files/folders needed by the processes are accessable when they run under the nobody user
RUN chown -R nobody.nobody /var/www/html /run /var/lib/nginx /var/log/nginx
# Install composer
RUN curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer
RUN apk update && apk add bash
# Install node npm
RUN apk add --update nodejs npm \
&& npm config set --global loglevel warn \
&& npm install --global marked \
&& npm install --global node-gyp \
&& npm install --global yarn \
# Install node-sass's linux bindings
&& npm rebuild node-sass
# Switch to use a non-root user from here on
USER nobody
# Add application
COPY --chown=nobody ./ /var/www/html/
RUN cat /var/www/html/resources/js/Components/Sections/About.vue
RUN composer install --optimize-autoloader --no-interaction --no-progress --ignore-platform-req=ext-zip --ignore-platform-req=ext-zip
USER root
RUN yarn && yarn run production
USER nobody
VOLUME /var/www/html
# Expose the port nginx is reachable on
EXPOSE 8080
# Let supervisord start nginx & php-fpm
CMD ["/usr/bin/supervisord", "-c", "/etc/supervisor/conf.d/supervisord.conf"]
# Configure a healthcheck to validate that everything is up&running
HEALTHCHECK --timeout=10s CMD curl --silent --fail http://127.0.0.1:8080/fpm-ping
taskDefinition.json
{
"containerDefinitions": [
{
"name": "fooweb-nginx-php",
"cpu": 100,
"memory": 512,
"links": [
"mysql"
],
"portMappings": [
{
"containerPort": 8080,
"hostPort": 80,
"protocol": "tcp"
}
],
"essential": true,
"environment": [],
"mountPoints": [
{
"sourceVolume": "fooweb-storage-web",
"containerPath": "/var/www/html"
}
]
},
{
"name": "mysql",
"image": "mysql",
"cpu": 50,
"memory": 512,
"portMappings": [
{
"containerPort": 3306,
"hostPort": 4306,
"protocol": "tcp"
}
],
"essential": true,
"environment": [
{
"name": "MYSQL_DATABASE",
"value": "123"
},
{
"name": "MYSQL_PASSWORD",
"value": "123"
},
{
"name": "MYSQL_USER",
"value": "123"
},
{
"name": "MYSQL_ROOT_PASSWORD",
"value": "123"
}
],
"mountPoints": [
{
"sourceVolume": "fooweb-storage-mysql",
"containerPath": "/var/lib/mysql"
}
]
}
],
"family": "art_web_task_definition",
"taskRoleArn": "arn:aws:iam::123:role/ecs-task-execution-role",
"executionRoleArn": "arn:aws:iam::123:role/ecs-task-execution-role",
"networkMode": "bridge",
"volumes": [
{
"name": "fooweb-storage-mysql",
"dockerVolumeConfiguration": {
"scope": "shared",
"autoprovision": true,
"driver": "local"
}
},
{
"name": "fooweb-storage-web",
"dockerVolumeConfiguration": {
"scope": "shared",
"autoprovision": true,
"driver": "local"
}
}
],
"placementConstraints": [],
"requiresCompatibilities": [
"EC2"
],
"cpu": "1536",
"memory": "1536",
"tags": []
}
So I believe there will be some problem with the way I have set the volume or maybe there could be some permission issue ?
Many thanks !
"I am attaching volume to the folder /var/www/html where sits my app,
so from my understanding this code should get replaced during
deployment."
That's the opposite of how docker volumes work.
It is going to ignore anything in /var/www/html inside the docker image, and instead reuse whatever you have in the mounted volume. Mounted volumes are primarily for persisting files between container restarts and image changes. If there is updated code in /var/www/html inside the image you are building, and you want that updated code to be active when your application is deployed, then you can't mount that as a volume.
If you are specifying a VOLUME instruction in your Dockerfile, then the very first time you ran your container in it would have "initialized" the volume with the files that are inside the docker container, as part of the process of creating the volume. After that, the files in the volume on the host server are persisted across container restarts/deployments, and any new updates to that path inside the new docker images are ignored.

ECS Fargate NGINX container not showing errors in CloudWatch logs

My nginx Dockerfile:
FROM nginx:1.15.12-alpine
RUN rm /etc/nginx/conf.d/default.conf
COPY ./nginx/nginx.conf /etc/nginx/conf.d
# Forward request logs to Docker log collector
RUN ln -sf /dev/stdout /var/log/nginx/access.log \
&& ln -sf /dev/stderr /var/log/nginx/error.log
EXPOSE 80
ENTRYPOINT ["nginx", "-g", "daemon off;"]
My container from my task definition for ECS:
[
{
"name": "nginx",
"image": "<ECR REPO HERE>",
"networkMode": "awsvpc",
"essential": true,
"portMappings": [
{
"containerPort": 80,
"protocol": "http"
}
],
"logConfiguration": {
"logDriver": "awslogs",
"options": {
"awslogs-group": "mygroup",
"awslogs-region": "us-east-1",
"awslogs-stream-prefix": "nginx"
}
},
"essential": true
}
]
Yet when the task is deployed, it fails, and in CloudWatch I see the following:
I'm very new to ECS / Cloudwatch. How can I see the NGINX errors from the container failing?
you should check ECS_Execution_Role_Policy. it should contains logs permission. like :
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ecr:GetAuthorizationToken",
"ecr:BatchCheckLayerAvailability",
"ecr:GetDownloadUrlForLayer",
"ecr:BatchGetImage",
"logs:CreateLogStream",
"logs:PutLogEvents"
],
"Resource": "*"
}
]
}
you should configure ecs_agent's config for awslogs driver.
this config file path is /etc/ecs/ecs.config in host. this file should be like :
ECS_CLUSTER=test_ecs_cluster
ECS_AVAILABLE_LOGGING_DRIVERS=["awslogs","json-file"]
See :
Here's a document

Docker, dangling container?

I cant start my docker-compose environment, because the port is allocated by something.
ERROR: for ***** Cannot start service *****: driver failed programming external connectivity on endpoint ***** (4314ec13837d41ca8ef1b7e1d8446ab8cfa96136539a75ac763ba1cf538ffdc1): Bind for 0.0.0.0:8881 failed: port is already allocated
Encountered errors while bringing up the project.
However, there is not such a container
$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
But when I inspect the docker-compose network, I can see a container in the network
$ docker network inspect *****
[
{
"Name": "*****",
"Id": "56d09f51e8fe5a9ad11dd6cdaff7e89983f519fd1ecff08116c2e48a3b34ff32",
"Created": "2018-08-01T08:17:30.704352821Z",
"Scope": "local",
"Driver": "bridge",
"EnableIPv6": false,
"IPAM": {
"Driver": "default",
"Options": null,
"Config": [
{
"Subnet": "172.18.0.0/16",
"Gateway": "172.18.0.1"
}
]
},
"Internal": false,
"Attachable": true,
"Ingress": false,
"ConfigFrom": {
"Network": ""
},
"ConfigOnly": false,
"Containers": {
"08a5defa5ceae7a5a5ac7ec85f5ecd0924ecb4e5eacca4f5752049e32a0f190c": {
"Name": "08a5defa5cea_*****",
"EndpointID": "5d081a19b5ffe4e893965a4306668059900812532cc4995e168ce017f2765e12",
"MacAddress": "02:42:ac:12:00:06",
"IPv4Address": "172.18.0.6/16",
"IPv6Address": ""
}
},
"Options": {},
"Labels": {
"com.docker.compose.network": "*****",
"com.docker.compose.project": "docker"
}
}
]
Yet it is impossible to remove this container
docker rm -f *****
Error: No such container: *****
How to proceed?
I run this script to shut down all stuck containers which sometimes linger
#!/bin/bash
# ... bring down all containers
force_it="" # normal
# force_it=" -f " # when you see "device or resource busy"
for curr_container_id in $( docker ps -q ); do
echo curr_container_id $curr_container_id
echo
echo "docker stop $curr_container_id "
docker stop $curr_container_id
echo
echo "docker rm $force_it $curr_container_id "
docker rm $force_it $curr_container_id
done
for curr_container_id in $( docker ps -q -a ); do
echo curr_container_id $curr_container_id
echo
echo "docker stop $curr_container_id "
docker stop $curr_container_id
echo
echo "docker rm $force_it $curr_container_id "
docker rm $force_it $curr_container_id
done
it works 99% of the time ... if not near the top you can toggle the force parm and re-run it ... this allows you to avoid having to bounce any docker server

How to configure rabbitmq.config inside Docker containers?

I'm using the official RabbitMQ Docker image (https://hub.docker.com/_/rabbitmq/)
I've tried editing the rabbitmq.config file inside the container after running
docker exec -it <container-id> /bin/bash
However, this seems to have no effect on the rabbitmq server running in the container. Restarting the container obviously didn't help either since Docker starts a completely new instance.
So I assumed that the only way to configure rabbitmq.config for a Docker container was to set it up before the container starts running, which I was able to partly do using the image's supported environment variables.
Unfortunately, not all configuration options are supported by environment variables. For instance, I want to set {auth_mechanisms, ['PLAIN', 'AMQPLAIN', 'EXTERNAL']} in rabbitmq.config.
I then found the RABBITMQ_CONFIG_FILE environment variable, which should allow me to point to the file I want to use as my conifg file. However, I've tried the following with no luck:
docker service create --name rabbitmq --network rabbitnet \
-e RABBITMQ_ERLANG_COOKIE='mycookie' --hostname = "{{Service.Name}}{{.Task.Slot}}" \
--mount type=bind,source=/root/mounted,destination=/root \
-e RABBITMQ_CONFIG_FILE=/root/rabbitmq.config rabbitmq
The default rabbitmq.config file containing:
[ { rabbit, [ { loopback_users, [ ] } ] } ]
is what's in the container once it starts
What's the best way to configure rabbitmq.config inside Docker containers?
the config file lives in /etc/rabbitmq/rabbitmq.config so if you mount your own config file with something like this (I'm using docker-compose here to setup the image)
volumes:
- ./conf/myrabbit.conf:/etc/rabbitmq/rabbitmq.config
that should do it.
In case you are having issues that the configuration file get's created as directory, try absolute paths.
I'm able to run RabbitMQ with a mounted config using the following bash script:
#RabbitMQ props
env=dev
rabbitmq_name=dev_rabbitmq
rabbitmq_port=5672
#RabbitMQ container
if [ "$(docker ps -aq -f name=${rabbitmq_name})" ]; then
echo Cleanup the existed ${rabbitmq_name} container
docker stop ${rabbitmq_name} && docker rm ${rabbitmq_name}
echo Create and start new ${rabbitmq_name} container
docker run --name ${rabbitmq_name} -d -p ${rabbitmq_port}:15672 -v $PWD/rabbitmq/${env}/data:/var/lib/rabbitmq:rw -v $PWD/rabbitmq/${env}/definitions.json:/opt/definitions.json:ro -v $PWD/rabbitmq/${env}/rabbitmq.config:/etc/rabbitmq/rabbitmq.config:ro rabbitmq:3-management
else
echo Create and start new ${rabbitmq_name} container
docker run --name ${rabbitmq_name} -d -p ${rabbitmq_port}:15672 -v $PWD/rabbitmq/${env}/data:/var/lib/rabbitmq:rw -v $PWD/rabbitmq/${env}/definitions.json:/opt/definitions.json:ro -v $PWD/rabbitmq/${env}/rabbitmq.config:/etc/rabbitmq/rabbitmq.config:ro rabbitmq:3-management
fi
I also have the following config files in my rabbitmq/dev dir
definitions.json
{
"rabbit_version": "3.7.3",
"users": [{
"name": "welib",
"password_hash": "su55YoHBYdenGuMVUvMERIyUAqJoBKeknxYsGcixXf/C4rMp",
"hashing_algorithm": "rabbit_password_hashing_sha256",
"tags": ""
}, {
"name": "admin",
"password_hash": "x5RW/n1lq35QfY7jbJaUI+lgJsZp2Ioh6P8CGkPgW3sM2/86",
"hashing_algorithm": "rabbit_password_hashing_sha256",
"tags": "administrator"
}],
"vhosts": [{
"name": "/"
}, {
"name": "dev"
}],
"permissions": [{
"user": "welib",
"vhost": "dev",
"configure": ".*",
"write": ".*",
"read": ".*"
}, {
"user": "admin",
"vhost": "/",
"configure": ".*",
"write": ".*",
"read": ".*"
}],
"topic_permissions": [],
"parameters": [],
"global_parameters": [{
"name": "cluster_name",
"value": "rabbit#98c821300e49"
}],
"policies": [],
"queues": [],
"exchanges": [],
"bindings": []
}
rabbitmq.config
[
{rabbit, [
{loopback_users, []},
{vm_memory_high_watermark, 0.7},
{vm_memory_high_watermark_paging_ratio, 0.8},
{log_levels, [{channel, warning}, {connection, warning}, {federation, warning}, {mirroring, info}]},
{heartbeat, 10}
]},
{rabbitmq_management, [
{load_definitions, "/opt/definitions.json"}
]}
].

Docker container communication on the same net not work

I have 3 different docker container (on windows 10) on the same network (core_net), but when use curl on backend
-curl localhost:7000
the response is:
"curl: (7) Failed to connect to localhost port 7000: Connection refused"
Why?
Docker commands:
Frontend:
docker run -itd --name dam --net=core_net -p 3000:3000 DAM
Backend:
docker run -itd --name core --net=core_net -p 6000:6000 -p 7000:7000 -p 8000:8000 -p 9000:9000 SG
Database:
docker run --name mongodb -p 27017:27017 -d mongo:3
These is the dockerfile:
Frontend:
FROM node:4.5.0
# Create app directory
RUN mkdir -p /DAM
WORKDIR /DAM
# Install app dependencies
COPY package.json /DAM
RUN npm install
RUN npm install gulp -g
RUN echo '{ "allow_root": true }' > /root/.bowerrc
RUN npm install bower -g
# Bundle app source
COPY . /DAM
ENV PORT 3000 3001
EXPOSE $PORT
CMD ["gulp", "serve"]
and
Backend:
FROM node:4.5.0
RUN npm install nodemon -g
# Create app directory
RUN mkdir -p /SG
WORKDIR /SG
# Install app dependencies
COPY package.json /SG
RUN npm install
# Bundle app source
COPY . /SG
ENV PORT 6000 7000 8000 9000
EXPOSE $PORT
CMD ["npm", "start"]
Inside container the ping works and the inspect is this:
$ docker network inspect core_net
{
"Name": "core_net",
"Id": "1f9e5426abe397d520360c05c95fee46fe08c98fe5c474c8b52764e491ea23e7",
"Scope": "local",
"Driver": "bridge",
"EnableIPv6": false,
"IPAM": {
"Driver": "default",
"Options": {},
"Config": [
{
"Subnet": "172.18.0.0/16",
"Gateway": "172.18.0.1"
}
]
},
"Internal": false,
"Containers": {
"3d3b8780fba2090b1c2feaddf2e035624529cf5474ad4e6332fe7071c0acbd25": {
"Name": "core",
"EndpointID": "f0a6882e690cf5a7deedfe57ac9b941d239867e3cd58cbdf0ca8a8ee216d53a9",
"MacAddress": "02:42:ac:12:00:04",
"IPv4Address": "172.18.0.4/16",
"IPv6Address": ""
},
"bb6a6642b3a7ab778969f2e00759d3709bdca643cc03f5321beb9b547b574466": {
"Name": "dam",
"EndpointID": "b42b802e219441f833d24971f1e1ea74e093f56e28126a3472a44750c847daa4",
"MacAddress": "02:42:ac:12:00:02",
"IPv4Address": "172.18.0.2/16",
"IPv6Address": ""
},
"cf8dd2018f58987443ff93b1e84fc54b06443b17c7636c7f3b4685948961ba3f": {
"Name": "mongodb",
"EndpointID": "be02d784cbd46261b7a53d642102887cafa0f880c8fe08086b9cc026971ea1be",
"MacAddress": "02:42:ac:12:00:03",
"IPv4Address": "172.18.0.3/16",
"IPv6Address": ""
}
},
"Options": {},
"Labels": {}
}
Commnication between mongodb and core work but between dam and core not work.
what's the problem?
To connect to another container with in an network you can not use localhost, but cou can use the name of the conatiner you want to reach. e.g. curl core:7000
To use localhost the conatiners have to share their network stack. You can do that with --network container:core
And if you dont have to reach the backend from outside of docker, it is enought to only expose the ports, and not to publish them

Resources