How can I change my / Prometheus/ Prometheus.yml on the container itself
I want it to track
1) my appserver - an Node application in a docker container
2) my postgres db
3) my apached and nginx web server
I do know that one has to change the Prometheus.yml file and add targets
Generic mechanisms to change docker images are
Mount your configuration file at the desired path.
Create a new image by copying the co fig file in the new Dockerfile. Not recommended if you have to use different configs for different environments/apps
Change the file on the running container if the application (peomerheus in this case) supports it. I know that some of the apps like Kibana do this. Good for debugging, not recommended for production environments.
It's hard to be precise with an answer given the lack of details but in general, you place your modified prometheus.yml file within the Docker context and modify your Dockerfile to add the instruction
COPY prometheus.yml /path/to/prometheus.yml
Related
I created one Docker Image for a custom application which needs some license files (some files and one directory) to run, so I'm using the COPY command in the Dockerfile to copy license file to the image:
# Base image: Application installed on Debian 10 but unlicensed
FROM me/application-unlicensed
# Copy license files
COPY *.license /opt/application/
COPY application-license-dir /var/lib/application-license-dir
I use this Dockerfile to build a new image with the license for a single container. As I have 5 different licenses I created 5 different images each with one specific license file and directory.
The licenses are also fixed to a MAC Address, so when I run one of the five container I specify its own MAC Address with the --mac-address parameter:
docker run --rm --mac-address AB:CD:EF:12:34:56 me/application-license1
This work, but I wish to have a better and smarter way to manage this:
As with docker-compose is it possible to specify the container MAC Address, could I just use the unlicensed base image and copy license files and the license directory when I build the 5 containers with docker-compose?
Edit: let me better explain the structure of license files
The application is deployed into /opt/application directory into the Docker image.
License files (*.license) are into the /opt/application at the same level of the application itself, to they cannot be saved into a Docker volume unless I create some symlinks (but I have to check if the application will work this way).
The directory application-license-dir needs to be at /var/lib/application-license-dir, so it can be mounted into a Docker volume (I have to check if the application will work but I think so).
Both *.license files and the files into the application-license-dir are binary, so I cannot script or create them at runtime.
So:
can docker-compose create a local directory on the Docker host
server before binding and mounting it to a Docker volume?
can docker-compose copy my licenses file and my license directory from
the GIT repository (locally cloned) to the local directory created
during the step 1?
can docker-compose create some symlinks into the
container's /opt/application directory for the *.license files
stored into the volume?
For things that are different every time you run a container or when you run a container on a different system, you generally don't want to specify these in a Dockerfile. This includes the license files you show above; things like user IDs also match this pattern; depending on how fixed your configuration files are they can also count. (For things that are the same every time you run the container, you do want these in your image; especially this is the application source code.)
You can use a Docker bind mount to inject files into a container at run time. There is Compose syntax for bind mounts using the volumes: directive.
This would give you a Compose file roughly like:
version: '3'
services:
app1:
image: me/application-unlicensed
volumes:
- './app1.license:/opt/application/app.license'
- './application-license-dir:/var/lib/application-license-dir'
mac_address: 'AB:CD:EF:12:34:56'
Bind mounts like this are a good match for pushing configuration files into containers. They can provide an empty host directory into which log files can be written, but aren't otherwise a mechanism for copying data out of an image. They're also useful as a place to store data that needs to outlive a container, if your application can't store all of its state in an external database.
According to this commit docker-compose has mac_address support.
Mounting license files with -v could be an option.
You can set mac_address for the different containers as
mac_address: AB:CD:EF:12:34:12. For documentation reference see this
For creating multiple instances from the same image, you will have to copy paste each app block 5 times in your docker-compose file and in each you can set a different mac_adddress
I'm new to docker and now I want to use docker-compose. I would like to provide the docker-compose.yml script the config file with host/port, maybe credentials and other cassandra configuration.
DockerHub link.
How to specify it in the compose script:
cassandra:
image: bitnami/cassandra:latest
You can do it using Docker compose environment variables(https://docs.docker.com/compose/environment-variables/#substituting-environment-variables-in-compose-files). You can also specify a separate environment file with environment variables.
Apparently, including a file outside of the scope of the image build, such as a config file, has been a major point of discussion (and perhaps some tension) between the community and the folks at Docker. A lot of the developers in that linked issue recommended adding config files in a folder as a volume (unfortunately, you cannot add an individual file).
It would look something like:
volumes:
- ./config_folder:./config_folder
Where config_folder would be at the root of your project, at the same level as the Dockerfile.
If Docker compose environment cannot solved your problem, you should create a new image from the original image with COPY to override the file.
Dockerfile:
From orginal_image:xxx
COPY local_conffile in_image_dir/conf_dir/
I'm building an ELK (elasticsearch/logstash/kibana) stack using docker-compose/docker-machine. The plan is to deploy it to a digitalocean droplet and, if needed, use Swarm to scale it.
It works really well, but I'm a bit confused where I should store configuration for the services (e.g. configuration files for logstash, or the SSL certs for nginx).
At first, I just mounted a host directory as volume. The problem with that is that all the configuration files have to be available on the docker host, so I have to sync them to the digitalocean droplet.
Then I thought I had a very smart idea: create a data container with all the configuration, and let the other services access it using volumes_from:
config:
volumes:
- /conf
build:
context: .
# this just copies the conf folder into the image
dockerfile: /dockerfiles/config/Dockerfile
logstash:
image: logstash:2.2
volumes_from:
- config
The problem with this approach became obvious quite fast: every time I change any configuration, I need to stop all containers that are linked to the config container, recreate the config image and container, and then start up the services again. Not great for uptime :(.
So, what's the best way? Ideally, the configuration files would be inside a container, so I can just ship it to wherever.
One common solution to this problem is to put a load balancer in front of the services. That way when you want to change the configuration you can start a new container and the load balancer will pick it up, then stop the old container. No downtime, and it lets you reload the config.
Another option might be to use a named volume. Then you can just modify the contents of the named volume and any containers using it will see the new files. However if you are using multiple nodes with swarm, you'll need to use a volume driver that supports multi-host volumes.
Did you consider to use the extension mechanism and override a settings file? Put a second docker-compose.override.yml in the same directory as the main compose file, or use explicit extension within the compose file. See
https://docs.docker.com/compose/extends/
That way you could integrate a configuration file in a transparent way, or control the parameters you want to change via environment variables that are different in the overriding composition.
Maybe I'm missing this when reading the docs, but is there a way to overwrite files on the container's file system when issuing a docker run command?
Something akin to the Dockerfile COPY command? The key desire here is to be able to take a particular Docker image, and spin several of the same image up, but with different configuration files. (I'd prefer to do this with environment variables, but the application that I'm Dockerizing is not partial to that.)
You have a few options. Using something like docker-compose, you could automatically build a unique image for each container using your base image as a template. For example, if you had a docker-compose.yml that look liked:
container0:
build: container0
container1:
build: container1
And then inside container0/Dockerfile you had:
FROM larsks/thttpd
COPY index.html /index.html
And inside container0/index.html you had whatever content you
wanted, then running docker-compose build would generate unique
images for each entry (and running docker-compose up would start
everything up).
I've put together an example of the above
here.
Using just the Docker command line, you can use host volume mounts,
which allow you to mount files into a container as well as
directories. Using my thttpd as an example again, you could use the
following -v argument to override /index.html in the container
with the content of your choice:
docker run -v index.html:/index.html larsks/thttpd
And you could accomplish the same thing with docker-compose via the
volume entry:
container0:
image: larsks/thttpd
volumes:
- ./container0/index.html:/index.html
container1:
image: larsks/thttpd
volumes:
- ./container1/index.html:/index.html
I would suggest that using the build mechanism makes more sense if you are trying to override many files, while using volumes is fine for one or two files.
A key difference between the two mechanisms is that when building images, each container will have a copy of the files, while using volume mounts, changes made to the file within the image will be reflected on the host filesystem.
There seems to be sparse conflicting information around on this subject. Im new to Docker and need some help. I have several docker containers to run an application, some require different config files for local development as they do for production. I don't seem to be able to find a neat way to automate this with Docker.
My containers that include custom config are Nginx, Freeradius and my code/data container is Laravel therefore requires a .env.php file (L4.2 at the moment).
I have tried Dockers environment variables in docker compose:
docker-compose.yml:
freeradius:
env_file: ./env/freeradius.env
./env/freeradius.env
DB_HOST=123.56.12.123
DB_DATABASE=my_database
DB_USER=me
DB_PASS=itsasecret
Except I can't pick those variables up in /etc/freeradius/mods-enabled/sql where they need to be.
How can I get Docker to run as a 'local' container with local config, or as a 'production' container with production config without having to actually build different containers, and without having to attach to each container to manually config them. I need it automated as this is to eventually be used on quite a large production environment which will have a large cluster of servers with many instances.
Happy to learn Ansible if this is how people achieve this.
If you can't use environment variables to configure the application (which is my understanding of the problem), then the other option is to use volumes to provide the config files.
You can use either "data volume containers" (which are containers with the sole purpose of sharing files and directories) with volumes_from, or you can use a named volume.
Data Volume container
If the go with the "data volume container" route, you would create a container with all the environment configuration files. Every service that needs a file uses volumes_from: - config. In dev you'd have something like:
configs:
build: dev-configs/
freeradius:
volumes_from:
- configs
The dev-configs directory will need a Dockerfile to build the image, which will have a bunch of VOLUME directives for all the config paths.
For production (and other environments) you can create an override file which replaces the configs service with a different container:
docker-compose.prod.yml:
configs:
build: prod-configs/
You'll probably have other settings you want to change between dev and prod, which can go into this file as well. Then you run compose with the override file:
docker-compose -f docker-compose.yml -f docker-compose.prod.yml up -d
You can learn more about this here: http://docs.docker.com/compose/extends/#multiple-compose-files
Named Volume
If you go with the "named volume" route, it's a bit easier to configure. On dev you create a volume with docker volume create thename and put some files into it. In your config you use it directly:
freeradius:
volumes:
- thename:/etc/freeradius/mods-enabled/sql
In production you'll either need to create that named volume on every host, or use a volume driver plugin that supports multihost (I believe flocker is one example of this).
Runtime configs using Dockerize
Finally, another option that doesn't involve volumes is to use https://github.com/jwilder/dockerize which lets you generate the configs at runtime from environment variables.