Given the following configuration:
mysql:
environment:
MY_MYSQL_PORT:
image: mysql
ports:
- "${MY_MYSQL_PORT}:3306"
There's a way to provide a fallback value for MY_MYSQL_PORT without relying on wrapper scripts? I already tested like bash ${MY_MYSQL_PORT-3306} but it doesn't work.
They implemented that feature with compose 1.9 release:
Added support for shell-style inline defaults in variable interpolation.
The supported forms are ${FOO-default} (fall back if FOO is unset) and ${FOO:-default} (fall back if FOO is unset or empty).
Release Notes Docker Compose 1.9
An explanation in the Dockerfile documentation can be found at:
https://docs.docker.com/engine/reference/builder/#environment-replacement
To quote from there:
The ${variable_name} syntax also supports a few of the standard bash modifiers as specified below:
${variable:-word} indicates that if variable is set then the result will be that value. If variable is not set then word will be the result.
${variable:+word} indicates that if variable is set then word will be the result, otherwise the result is the empty string.
In all cases, word can be any string, including additional environment variables.
Escaping is possible by adding a \ before the variable: \$foo or \${foo}, for example, will translate to $foo and ${foo} literals respectively.
Related
I have a YAML scalar that is throwing the following error when I try to evaluate my docker-compose.yml file:
ERROR: Invalid interpolation format for "environment" option in
service "time_service": "${Time.now}"
YAML:
---
version: '2'
services:
time_service:
build: "."
environment:
TIME: "${Time.now}"
How can I maintain the same string output as written, but avoid having the docker-compose interpret it as faulty string interpolation?
You can use a $$ (double-dollar sign) when your configuration needs
a literal dollar sign.
You are hitting the docker-compose variable substitution, which is well documented here:
Both $VARIABLE and ${VARIABLE} syntax are supported. Extended shell-style features, such as ${VARIABLE-default} and ${VARIABLE/foo/bar}, are not supported.
You can use a $$ (double-dollar sign) when your configuration needs a literal dollar sign. This also prevents Compose from interpolating a value, so a $$ allows you to refer to environment variables that you don’t want processed by Compose.
docker-compose is written in Python, as you see on github, the doubling mechanism to get the original meaning of special characters can be found in many programs, I needed to use this myself, while programming, as far back in 1984.
Found the answer by copying the suggestion for % characters in this post
It requires a double dollar sign $$.
So I needed "$${Time.now}", which evaluates to "${Time.now}"
Does Pandoc expose variables set on the command line (pandoc -V foo=bar) to scripts running inside the built in lua filter environment? In other words if I run:
pandoc -V foo=bar --lua-filter=myfilter.lua
...what can I put in myfilter.lua to access foo?
(Since: pandoc 2.17)
PANDOC_WRITER_OPTIONS.variables["foo"]
See https://pandoc.org/lua-filters.html#type-writeroptions
This is documented a bit in the description for the --metadata:
Like --variable, --metadata causes template variables to be set.
But unlike --variable, --metadata affects the metadata of the
underlying document (which is accessible from filters and may be
printed in some output formats) and metadata values will be escaped
when inserted into the template.
So, I think that using -M to set a variable will give you access to the variable inside your lua filter.
How can I use variable substitution for a list, map, or array value in a docker-compose.yml file.
For example:
graylog:
image: graylog2/server2
extra_hosts: ${EXTRA_HOSTS}
and
export EXTRA_HOSTS="['host1:10.10.10.1','host2:10.10.10.2']"
gives the following error: graylog.extra_hosts must be a mapping
I've tried different variations of the above with no luck.
I do see that there's an open issue about this here: https://github.com/docker/compose/issues/4249
Is it just not possible? Does anyone know of a workaround?
At least as of this time (June 2018), Docker still doesn't support this. I was able to work around the issue utilizing envsubst.
envsubst is part of gettext and it can be used to replace only environment variables you tell it to.
Tweak the docker-compose.yml value to look like an array or map (either brackets or curly braces) but have the value be an environment variable.
For example
graylog:
image: graylog2/server2
extra_hosts: [ ${EXTRA_HOSTS} ]
Then, define your environment variable without brackets or curly braces.
For example:
export EXTRA_HOSTS="'host1:10.10.10.1','host2:10.10.10.2'"
Then utilize envsubst
envsubst '${EXTRA_HOSTS}' < docker-compose.yml > docker-compose.subst.yml && docker stack deploy -c docker-compose.subst.yaml foobar
Notice that you pass '${EXTRA_HOSTS}' to envsubst. This tells it to only replace this environment variable. This ensures it doesn't accidentally replace some other variable that's utilizing the variable substitution syntax of Docker compose files.
I have got a docker-compose.yml file and there I define:
extra_hosts:
- "localhost:${MY_MACHINE_IP}"
It works if I define MY_MACHINE_IP as environment var earlier.
What I want to achieve is to perform action like:
extra_hosts:
- "localhost:<get MY_MACHINE_IP from env if it exists, if not set MY_MACHINE_IP env variable with value <docker-machine-ip>>"
In other words: I want to define it in extra_hosts section, if MY_MACHINE_IP is already specified, get it, if not - set this env. variable with value = my docker machine ip.
Is it possible?
Yes in according to docker documentation
docker-compose run SERVICE env
So i think the variables are not global as you may think. You have to pass them as parameters.
Read this.
You can use the package ruamel.dcw for that (dcw for Docker Compose Wrapper, disclaimer: I am the author of that package). It allows you to create a section with key user-data in your docker-compose.yaml file, which is stripped out before handing the file to the normal docker-compose. That section can look like:
user-data:
author: Your Name <your-name#youremail.com>
description: container for postfix/submission
env-defaults:
PORT: 587 # override during development
NAME: submission
DOCKER_BASE: /data0/DATA
and then you can use {PORT}, {NAME} and {DOCKER_BASE} in the rest of the file, with the option of overriding these default values with environment variables.
The utility also write out a file .dcw_env_vars.inc which you can copy into your container and source to get the appropriate values into scripts you RUN from within the Dockerfile
Using typesafe config, how do I override the reference configuration with an environment variable? For example, lets say I have the following configuration:
foo: "bar"
I want it to be overriden with the environment variable FOO if one exists.
If I correctly understood your question, the answer is here.
You can do
foo: "bar"
foo: ${?FOO}
The official doc now describes it very clearly and supports multiple options for this. Here is a brief summary...
Most common way is to use this form:
basedir = "/whatever/whatever"
basedir = ${?FORCED_BASEDIR}
If env variable is set, then it will override your default value, otherwise it will be left intact.
A more convenient way is to use JVM property -Dconfig.override_with_env_vars=true to override any config variable. In this case you don't have to create duplicate declarations. You env variables will have to be named with prefix CONFIG_FORCE_. See how env var to config name mapping works in the docs. As an example: CONFIG_FORCE_a_b__c___d will be mapped to a.b-c_d.
Finally, if you want to roll out your own mapping, which is similar to the option described above without using override_with_env_vars you can use some shell hacking as described below.
If you have to use environment variables and if their naming is consistent with config names you can use a bash script like this to automatically convert from your environment vars to JVM cmd args. These -D JVM args will override Typesafe Config values. Example:
# export my_PROP1=1
# export my_PROP2=2
#
# props=$(env | grep my_ | awk '{print "-D"$_}' ORS=' ')
#
# echo "JVM executable command is: java $props some.jar"
JVM executable command is: java -Dmy_PROP2=2 -Dmy_PROP1=1 some.jar
Convert upper to lower case, do substring operations on env vars as you please if they don't directly map to your config values.
I am using the System property -Dconfig.override_with_env_vars=true. With it all properties are automatically overridden via environment variables.