Override json file values with environment variables docker - docker

Question

Assume that I've a complex json file that is used to configurate my project.
Like the json below:
{
"apis": {
"payment": {
"base_url": "https://example.com/"
},
"order": {
"base_url": "https://example.com/"
},
},
"features": {
"authentication": {
"authProviders": true,
"registration": false
}
},
"availableLocales": [
"en",
"es"
]
}
With .Net there's a feature that allows us to override the values based on environment variables.
If I wanted to override the value of apis.payment.base_url I could pass an environment variable: APIS__PAYMENT__BASE_URL and the value would be replaced.
Since I'm currently not using .Net is there any alternatives?
This is what I'm using right now, but this does not fit my needs
FROM code as prepare-build
ENV JQ_VERSION=1.6
RUN wget --no-check-certificate \
https://github.com/stedolan/jq/releases/download/jq-${JQ_VERSION}/jq-linux64 \
-O /tmp/jq-linux64
RUN cp /tmp/jq-linux64 /usr/bin/jq
RUN chmod +x /usr/bin/jq
WORKDIR /code/public
RUN jq 'reduce path(recurse | scalars) as $p (.;setpath($p; "$" + ($p | join("_"))))' \
./configurations/settings.json > ./configurations/settings.temp.json && \
yez | cp ./configurations/settings.temp.json ./configurations/settings.json
WORKDIR /code/deploy
RUN echo "#!/usr/bin/env sh" | tee -a /code/deploy/start.sh > /dev/null && \
echo 'export EXISTING_VARS=$(printenv | awk -F= '\''{print $1}'\'' | sed '\''s/^/\$/g'\'' | paste -sd,);' | tee -a /code/deploy/start.sh > /dev/null && \
echo 'for file in $CONFIGURATIONS_FOLDER;' | tee -a /code/deploy/start.sh > /dev/null && \
echo 'do' | tee -a /code/deploy/start.sh > /dev/null && \
echo ' cat $file | envsubst $EXISTING_VARS | tee $file' | tee -a /code/deploy/start.sh > /dev/null && \
echo 'done' | tee -a /code/deploy/start.sh > /dev/null && \
echo 'nginx -g '\''daemon off;'\''' | tee -a /code/deploy/start.sh > /dev/null
WORKDIR /code
This was I have a problem that, I need to pass all the json paths as environment variables, to override it correctly. If not, the variables will be replaced with the path of it, only.
I think the best approach would be:
Read the environment variables and create a json file with their values, then override the existing json file with the values of the created one.
Does anyone have any thing that could help me achieve this?
To summarize.
In order to make easy to identify which environment variables I should use, let's assume it will have a prefix of SETTINGS.
Example of how I would override values.
JSON PATH
EQUIVALENT ENVIRONMENT VARIABLE
APIS.PAYMENT.BASE_URL
SETTINGS__APIS__PAYMENT__BASE_URL
AVAILABLELOCALES[0]
SETTINGS__AVAILABLELOCALES__0

Answer 1

The task can be solved using jq.
The version is robust against settings that do not match a path in the document.
Variables
SETTINGS__APIS__PAYMENT__BASE_URL=https://example2.com
SETTINGS__AVAILABLELOCALES__0=cs
SETTINGS__UNAVAILABLE__PATH=1
Code
jq 'def settings:
def prepareVariables:
[$ENV | to_entries[] | select(.key | startswith("SETTINGS__"))] # select all variables that starts with "SETTINGS__"
| map(.key |= (. / "__" | map(tonumber? // .))[1:]); # convert variable names to path arrays
[paths(scalars) | [., map(ascii_upcase? // .)]] | # collect all leaf paths from input file and add uppercase path
reduce .[] as $leafPath # add leaf paths to corresponding settings
(prepareVariables; map(select($leafPath[1] == .key) |= . + {path: $leafPath[0]})) |
map(select(has("path"))); # drop settings for unknown paths
. as $input |
reduce settings[] as $setting # apply new settings from variables to input file
($input; . | setpath($setting["path"]; $setting["value"]))
' input.json
Output
{
"apis": {
"payment": {
"base_url": "https://example2.com"
},
"order": {
"base_url": "https://example.com/"
}
},
"features": {
"authentication": {
"authProviders": true,
"registration": false
}
},
"availableLocales": [
"cs",
"es"
]
}

Answer 2

I'm a jq novice, and I'd be very interested in a better jq script, but here's one way to use environment variables to modify a settings.json file.
$ cat settings.json
{
"apis": {
"payment": {
"base_url": "https://example.com/"
},
"order": {
"base_url": "https://example.com/"
}
},
"features": {
"authentication": {
"authProviders": true,
"registration": false
}
},
"availableLocales": [
"en",
"es"
]
}
$ printenv|grep SETTINGS__
SETTINGS__APIS__PAYMENT__BASE_URL=https://example2.com
SETTINGS__AVAILABLELOCALES__0=cs
$ jq -n '
inputs as $i
| [ $i
| ..
| keys_unsorted?
| .[]
| strings
]
| unique as $allKeys
|
def fixCase:
. as $w
| reduce ($allKeys[]|select(length == ($w|length))) as $k
("";. + $k|match($w;"i").string)
;
def envpaths:
[
$ENV
| to_entries[]
| select(.key | startswith("SETTINGS__"))
| [[ (.key|split("__"))[1:][]
| if test("^[0-9]+$") then tonumber else fixCase end
],
.value
]
]
;
reduce envpaths[] as $p ($i; .|setpath($p[0];$p[1]))' settings.json
# the output
{
"apis": {
"payment": {
"base_url": "https://example2.com"
},
"order": {
"base_url": "https://example.com/"
}
},
"features": {
"authentication": {
"authProviders": true,
"registration": false
}
},
"availableLocales": [
"cs",
"es"
]
}
See it work on jqplay.org.