Not able to extend the parent's entry point - docker

I am trying to extend the parent image's entry point by following this tutorial
This is the content of the child's entry point shell file new-initfile.sh
#!/bin/sh
exec /usr/local/bin/docker-entrypoint.sh "$#"
exec flyway migrate
So here basically I am executing the parent images entry point and then adding my own command to it.
Here is my docker file ↓
FROM postgres:alpine
RUN apk --no-cache add wget su-exec
RUN wget -qO- https://repo1.maven.org/maven2/org/flywaydb/flyway-commandline/7.3.2/flyway-commandline-7.3.2-linux-x64.tar.gz | tar xvz && su-exec sh ln -s `pwd`/flyway-7.3.2/flyway /usr/local/bin
RUN mv /flyway-7.3.2/conf/flyway.conf /flyway-7.3.2/conf/flyway.conf.orig
COPY flyway.conf /flyway-7.3.2/conf/flyway.conf
COPY new-initfile.sh /new-initfile.sh
RUN ["chmod", "+x", "/new-initfile.sh"]
ENTRYPOINT [ "/new-initfile.sh" ]
This is my docker-compose.yml file
version: "3.7"
services:
db:
build: database
container_name: db
environment:
POSTGRES_USER: ${POSTGRES_USER}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
POSTGRES_DB: ${POSTGRES_DB}
volumes:
- ./pgdata:/var/lib/postgressql/data
- ./database/migrations:/migrations
ports:
- "5432:5432"
When I do docker-compose up it goes to the last step and exits with code 0. If I comment out the last line in my dockerfile, it starts absolutely fine but then my extension commands are not run.
My end objective is that it should first run the parent's entry point command and then run mine.

That is the way Docker works - it keeps a container up only while the process started by initial arguments is running. new-initfile.sh completes all the lines and exits normally and so does the container.
In general you need an endless task at the end of your entrypoint script so that it never exits unless an error or a stop signal happens. In your case, I would have delegated migrations to the application rather that the database. It is common to run migrations before starting an application and it's more convenient when you add new migrations (you don't have to mess with two images).
If you still wish the database to do migrations, here are two options:
Postgres Docker image supports extension scripts but they only run at the first launch. That is when Postgres creates a database. It will not run extension scripts on consecutive launches. This is the best way to load a dump or something like that. To utilise the feature place the script in /docker-entrypoint-initdb.d/ inside the container. The script must have *.sql, *.sql.gz, or *.sh extension. Read more about initialization scripts here.
Run the following command and examine the output: docker run --rm postgres:alpine cat /usr/local/bin/docker-entrypoint.sh. This is the default entrypoint script in this image. There are a lot of code in it but look at this first:
if ! _is_sourced; then
_main "$#"
fi
It won't do much if it is sourced. So the idea is: you source the default entrypoint into your entrypoint script:
#!/bin/sh
. /usr/local/bin/docker-entrypoint.sh
Then you copy the contents of the _main function into your script by the way adding there flyway migrate where you think it fits. And this way you should have a proper entrypoint script.

Related

docker-compose and docker: need to intialize with dummy data only first time. whats the best way to do this

I have a django docker image and using docker-compose to start it along with postgresql.
# docker-compose -p torsha-single -f ./github/docker-compose.yml --project-directory ./FINAL up --build --force-recreate --remove-orphans
# docker-compose -p torsha-single -f ./github/docker-compose.yml --project-directory ./FINAL exec fastapi /bin/bash
# My version of docker = 18.09.4-ce
# Compose file format supported till version 18.06.0+ is 3.7
version: "3.7"
services:
postgresql:
image: "postgres:13-alpine"
restart: always
volumes:
- type: bind
source: ./DO_NOT_DELETE_postgres_data
target: /var/lib/postgresql/data
environment:
POSTGRES_DB: project
POSTGRES_USER: postgres
POSTGRES_PASSWORD: abc123
PGDATA: "/var/lib/postgresql/data/pgdata"
networks:
- postgresql_network
webapp:
image: "django_image"
depends_on:
- postgresql
ports:
- 8000:8000
networks:
- postgresql_network
networks:
postgresql_network:
driver: bridge
Now when after I do docker-compose up for first time I have to create dummy data using
docker-compose exec webapp sh -c 'python manage.py migrate';
docker-compose exec webapp sh -c 'python manage.py shell < useful_scripts/intialize_dummy_data.py'
After this anytime later I dont need to do the above.
Where to place this script so it checks if its first time then run these commands.
One of the Django documentation's suggestions for Providing initial data for models is to write it as a data migration. That will automatically load the data when you run manage.py migrate. Like other migrations, Django records the fact that it has run in the database itself, so it won't re-run it a second time.
This then reduces the problem to needing to run migrations when your application starts. You can write a shell script that first runs migrations, then runs some other command that's passed as arguments:
#!/bin/sh
python manage.py migrate
exec "$#"
This is exactly the form required for a Docker ENTRYPOINT script. In your Dockerfile, COPY this script in with the rest of your application, set the ENTRYPOINT to run this script, and set the CMD to run the application as before.
COPY . . # probably already there
ENTRYPOINT ["./entrypoint.sh"] # must be JSON-array form
CMD python manage.py runserver 0.0.0.0:8000 # unchanged
(If you already have an entrypoint wrapper script, add the migration line there. If your Dockerfile somehow splits ENTRYPOINT and CMD, combine them into a single CMD.)
Having done this, the container will run migrations itself whenever it starts. If this is the first time the container runs, it will also load the seed data. You don't need any manual intervention.
docker-compose run is designed for this type of problem. Pair it with the -rm flag to remove the container when the command is complete. Common examples are doing the sort of migrations and initialization you are trying to accomplish.
This is right out of the manual page for the docker-compose run command:
docker-compose run --rm web python manage.py db upgrade
You can think of this as a sort of disposable container, that does one job, and exits. This technique can also be used for scheduled jobs with cron.

docker-compose named volume with one file: ERROR: Cannot create container for service, source is not directory

I am trying to make the binary file /bin/wkhtmltopdf from the container wkhtmltopdf available in the web container. I try to achieve this with a named volume.
I have the following docker container setup in my docker-compose.yml:
services:
web:
image: php:7.4-apache
command: sh -c "mkdir -p /usr/local/bin && touch /usr/local/bin/wkhtmltopdf"
entrypoint: sh -c "exec 'apache2-foreground'"
volumes:
- wkhtmltopdfvol:/usr/local/bin/wkhtmltopdf
wkhtmltopdf:
image: madnight/docker-alpine-wkhtmltopdf
command: sh -c "touch /bin/wkhtmltopdf"
entrypoint: sh -c "tail -f /dev/null" # workaround to keep container running
volumes:
- wkhtmltopdfvol:/bin/wkhtmltopdf
volumes:
wkhtmltopdfvol:
However, I get the following error when running docker-compose up:
ERROR: for wkhtmltopdf Cannot create container for service wkhtmltopdf:
source /var/lib/docker/overlay2/42e7082b8024ae4ebb13a4f0003a9e17bc18b33ef0677431dd002da3c21dde88/merged/bin/wkhtmltopdf is not directory
.../bin/wkhtmltopdf is not directory
Does that mean that I can't share one file between containers but only directories through a named volume? How do I achieve this?
Edit: I also noticed that /usr/local/bin/wkhtmltopdf inside the web container is a directory and not a file as I expected.
It can be tricky to share binaries between containers like this. Volumes probably aren't the mechanism you're looking for.
If you look at the Docker Hub page for the php image you can see that php:7.4-apache is an alias for (currently) php:7.4.15-apache-buster, where "Buster" is the name of a Debian release. You can then search on https://packages.debian.org/ to discover that Debian has a prepackaged wkhtmltopdf package. You can install this using a custom Dockerfile:
FROM php:7.4-apache
RUN apt-get update \
&& DEBIAN_FRONTEND=noninteractive \
apt-get install --assume-yes --no-install-recommends \
wkhtmltopdf
# COPY ...
# Base image provides EXPOSE, CMD
Then your docker-compose.yml file needs to build this image:
version: '3.8'
services:
web:
build: .
# no image:, volumes:, or command: override
Just in terms of the mechanics of sharing binaries like this, you can run into trouble where a binary needs a shared library that's not present in the target container. The apt-get install mechanism handles this for you. There are also potential troubles if a container has a different shared-library ecosystem (especially Alpine-based containers), or using host binaries from a different operating system.
The Compose file you show mixes several concepts in a way that doesn't really work. A named volume is always a directory, so trying to mount that over the /bin/wkhtmltopdf file in the second container causes the error you see. There's a dependency issue for which container starts up first and gets to create the volume. A container only runs a single command, and if you have both entrypoint: and command: then the command gets passed as extra arguments to the entrypoint (and if the entrypoint is an sh -c ... invocation, effectively ignored).
If you really wanted to try this approach, you should make web: {depends_on: [wkhtmltopdf]} to force the dependency order. The second container should mount the volume somewhere else, it probably shouldn't have an entrypoint:, and it should do something like command: cp -a /bin/wkhtmltopdf /export. (It will exit immediately once this cp finishes, but that shouldn't matter.) The first container can then mount the volume on, say, /usr/local/bin, and not specially set command: or entrypoint:. There will still be a minor race condition (you're not guaranteed the cp command will complete before Apache starts) but it probably wouldn't be a practical problem.

Docker app supposed to output json to a dir on a volume, when running no data show

I am working on a docker app. The purpose of this repo is to output some json into a volume. I am using a Dockerfile, docker-compose and a Makefile. I'll show the contents of each file below. Goal/desired outcome is that when I run using make up that the container runs and outputs the json.
Directory looks like this:
docker-compose.yaml
Dockerfile
Makefile
main/ # a directory
Here are the contents of directory Main:
example.R
Not sure the best order to show these files. Throughout my setup I refer to a variable $PROJECTS_DIR which is a global on the host / local:
echo $PROJECTS_DIR
/home/doug/Projects
Here are my files:
docker-compose.yaml:
version: "3.5"
services:
nextzen_ga_extract_marketing:
build:
context: .
environment:
start_date: "2020-11-18"
start_date: "2020-11-19"
volumes:
- ${PROJECTS_DIR}/Zen/nextzen_google_analytics_extract_pipeline:/home/rstudio/Projects/nextzen_google_analytics_extract_pipeline
Dockerfile:
FROM rocker/tidyverse:latest
ADD main main
WORKDIR "/main"
RUN apt-get update && apt-get install -y \
less \
vim
ENTRYPOINT ["Rscript", "example.R"]
Makefile:
.PHONY: build
build:
docker-compose build
.PHONY: up
up:
docker-compose pull
docker-compose up -d
.PHONY: restart
restart:
docker-compose restart
.PHONY: down
down:
docker-compose down
Here is the contents of the 'main' file of the Docker app, example.R:
library(jsonlite)
unlink("../output_data", recursive = TRUE) # delete any existing data from previous runs
dir.create('../output_data')
write(toJSON(mtcars), '../output_data/ga_tables.json')
If I navigate into ${PROJECTS_DIR}/Zen/nextzen_google_analytics_extract_pipeline/main and then run sudo Rscript example.R then the file runs and outputs the json in '../output_data/ga_tables.json as expected.
I am struggling to get this to happen when running the container. If I navigate into ${PROJECTS_DIR}/Zen/nextzen_google_analytics_extract_pipeline/ and then in the terminal run make up for:
docker-compose pull
docker-compose up -d
I then see:
make up
docker-compose pull
docker-compose up -d
Creating network "nextzengoogleanalyticsextractpipeline_default" with the default driver
Creating nextzengoogleanalyticsextractpipeline_nextzen_ga_extract_marketing_1 ...
Creating nextzengoogleanalyticsextractpipeline_nextzen_ga_extract_marketing_1 .
It 'looks' like everything ran as expected with no errors. Except no output appears in directory output_data as expected?
I guess I'm misunderstanding or misusing ENTRYPOINT in the Dockerfile with ENTRYPOINT ["Rscript", "example.R"]. My goal is that this file would run when the container is run.
How can I 'run' (if that's the correct terminology) my app so that it outputs json into /output_data/ga_tables.json?
Not sure what other info to provide? Any help much appreciated, I'm still getting to grips with docker.
If you run your application from /main and its output is supposed to go into ../output_data (so effectively /output_data), you need to bind mount this directory to have this output available on host. Therefore I would update your docker-compose.yaml to read something like this:
volumes:
- /path/to/output_data/on/host:/output_data
Bear in mind however that your script will not be able to remove /output_data when bind-mounted this way, so you might want to change your step to removing directory contents and not directory itself.
In my case, I got this working when I used full paths as opposed to relative paths.

docker-compose subfolders doesn't appear in volume folder

Dockerfile:
FROM golang:latest
RUN mkdir /app/
RUN mkdir /app/subfolder1
RUN mkdir /app/subfolder2
VOLUME /app/
docker-compose.yml
version: '3.3'
services:
my_test:
build: .
volumes:
- ./app:/app
I watched (in mysql Dockerfile) how the database mysql files are shared, I decided to do the same. I expect that the first time start docker-compose up, two subfolders from outside will be created in the /app folder. But during running docker-compose up, only one folder /app is created without subfolders inside. What am I doing wrong?
Please tell me how can I achieve the same behavior as with the MySQL container, when at the first start my external folder is filled with files and folders, and then it’s just used:
version: '3'
services:
mysql:
image: mysql:5.7
volumes:
- ./data/db:/var/lib/mysql
Example above works, but my first example doesn't work
The mysql image has an involved entrypoint script that does the first-time setup. That specifically checks to see whether the data directory exists or not:
if [ -d "$DATADIR/mysql" ]; then
DATABASE_ALREADY_EXISTS='true'
fi
if [ -z "$DATABASE_ALREADY_EXISTS" ]; then
docker_init_database_dir "$#"
...
fi
Note that this does not rely on any built-in Docker functionality, and does not copy any content out of the original image; it runs a fairly involved sequence of steps to populate the initial database setup, configure users, and run the contents in the /docker-entrypoint-initdb.d directory.
If you want to copy some sort of seed data into a mounted volume, your container generally needs to handle this itself. You could write an entrypoint script like:
#!/bin/sh
# If the data directory doesn't have content, copy it
if ! [ -d /data/content ]; then
cp -a /app/data/content /data
fi
# Run whatever the container's main command is
exec "$#"
(There is a case where Docker will populate named volumes from image content. This has some severe limitations: it only works on named volumes and not bind-mounted host directories; it doesn't work on Kubernetes, if that's in your future; if the image content is updated, the volume will not be changed. Writing out the setup code explicitly at startup will give you more predictable behavior.)

Recommended way to handle empty vs existing DB in Dockerfile

I want to run M/Monit (https://mmonit.com/) in a docker container and found this Dockerfile: https://github.com/mlebee/docker-mmonit/blob/master/Dockerfile
I'm using it with a simple docker-compose.yml in my test environment:
version: '3'
services:
mmonit:
build: .
ports:
- "8080:8080"
#volumes:
#- ./db/:/opt/mmonit/db/
It does work, but I want to extend the Dockerfile so that the path /opt/mmonit/db/ is exported as a volume. I'm struggling to implement the following behaviour:
When the volume mapped to /opt/mmonit/db/ is empty (for example on first setup) the files from the install archive should be written to the volume. The db folder is part of the archive.
When the database file /opt/mmonit/db/mmonit.db already exists in the volume, it should not be overwritten in any circumstances.
I do have an idea how to script the required operations / checks in bash, but I'm not even sure if it would be better to replace the ENTRYPOINT with a custom start script or if it should be done by modifying the Dockerfile only.
That's why I ask for the recommended way.
In general the strategy you lay out is the correct path; it's essentially what the standard Docker Hub database images do.
The image you link to is a community image, so you shouldn't feel particularly bound to that image's decisions. Given the lack of any sort of license file in the GitHub repository you may not be able to copy it as-is, but it's also not especially complex.
Docker supports two "halves" of the command to run, the ENTRYPOINT and CMD. CMD is easy to provide on the Docker command line, and if you have both, Docker combines them together into a single command. So a very typical pattern is to put the actual command to run (mmmonit -i) as the CMD, and have the ENTRYPOINT be a wrapper script that does the required setup and then exec "$#".
#!/bin/sh
# I am the Docker entrypoint script
# Create the database, but only if it does not already exist:
if ! test -f /opt/mmonit/db/mmonit.db; then
cp -a /opt/monnit/db_base /opt/monnit/db
fi
# Replace this script with the CMD
exec "$#"
In your Dockerfile, then, you'd specify both the CMD and ENTRYPOINT:
# ... do all of the installation ...
# Make a backup copy of the preinstalled data
RUN cp -a db db_base
# Install the custom entrypoint script
COPY entrypoint.sh /opt/monit/bin
RUN chmod +x entrypoint.sh
# Standard runtime metadata
USER monit
EXPOSE 8080
# Important: this must use JSON-array syntax
ENTRYPOINT ["/opt/monit/bin/entrypoint.sh"]
# Can be either JSON-array or bare-string syntax
CMD /opt/monit/bin/mmonit -i
I would definitely make these kind of changes in a Dockerfile, either starting FROM that community image or building your own.

Resources