Dockerfile ENV not replacing in sed - docker

I have set up dockerfile with making changes to config files with sed.
The image is a simple rtmp restreamer.
The lines in dockerfile go like:
RUN sed -i 's/ytkey/${YOUTUBE_KEY}/g' /etc/nginx/nginx.conf &&\
sed -i 's/fbkey/${FACEBOOK_KEY}/g' /etc/nginx/nginx.conf
i set up ENV at the beginning
ENV YOUTUBE_KEY=default FACEBOOK_KEY=default
but after building the image it doesn't replace ytkey or fbkey as default but inputs a string ${YOUTUBE_KEY}.
I have tried running
RUN sed -i 's/ytkey/\${YOUTUBE_KEY}/g' /etc/nginx/nginx.conf &&\
sed -i 's/fbkey/\${FACEBOOK_KEY}/g' /etc/nginx/nginx.conf
But had the same result. Is it a problem with sed (having the env in '') or something else? Is there an alternative to sed that works with dockerfile?
Or maybe i'm doing things completely the wrong way?

You can use double quotation marks around environment variables like this:
RUN sed -i 's/user/'"${YOUTUBE_KEY}"'/g' /etc/nginx/nginx.conf &&\
sed -i 's/pid/'"${FACEBOOK_KEY}"'/g' /etc/nginx/nginx.conf

I realise the question is specifically about sed, but you might also consider using envsubst to do this, especially if you start to use more variables.
It processes stdin to stdout, so in your example you would need to use a temp file strategy, but personally I prefer that approach:
envsubst < /etc/nginx/nginx.conf > /etc/nginx/nginx.conf.tmp
mv /etc/nginx/nginx.conf.tmp /etc/nginx/nginx.conf
For more complex configs, your templates would often be in a different directory anyway, so this wouldn't be necessary.

Related

Docker file RUN multiple sed commands

I am trying to change several configuration items in a single file that is part of an existing Docker image using the following RUN command:
RUN sed --in-place '/start_rpc:/ s/false/true/g' /etc/cassandra/cassandra.yaml &&\
sed --in-place '/broadcast_rpc_address:/ s/172.17.0.2/192.168.1.248/g' /etc/cassandra/cassandra.yaml &&\
sed --in-place '/listen_address:/ s/172.17.0.2/192.168.1.248/g' /etc/cassandra/cassandra.yaml &&\
sed --in-place '/seeds:/ s/172.17.0.2/192.168.1.248/g' /etc/cassandra/cassandra.yaml
I am sure the multiple sed commands are fine as I can run them on the host's console and achieve the 4 changes I am trying to execute.
The docker build does not show any errors and, in fact, it generates a new image. However, when I run the container and login to it, I can see that only the first of the four replacements took place.
I have already tried several combinations, including a compound sed such as:
RUN sed --in-place '/start_rpc:/ s/false/true/g; /broadcast_rpc_address:/ s/172.17.0.2/192.168.1.248/g; /listen_address:/ s/172.17.0.2/192.168.1.248/g; /seeds:/ s/172.17.0.2/192.168.1.248/g' /etc/cassandra/cassandra.yaml
And also four individual RUN commands. In all cases the result was the same: start_rpc is changed to true but not one of the other IP addresses gets changed.
Does anyone have any idea what could be wrong here and, if so, why docker build does not report something useful (and preferably fails)?
The answer to these problems can be found here but, in a nutshell, once the container starts, additional scripts are run to complete the set up of Cassandra and IP addresses are the typical type of configuration that can only be discovered once the container starts.

Echo dynamic sed to file inside Dockerfile

I am working on a Dockerfile, inside of which I want to dynamically create a sed expression based on the input argument variable, and write this expression to a file.
Here's part of the Dockerfile:
FROM ubuntu
ARG VERSION
RUN echo $VERSION > /usr/local/testfile
RUN echo '#!/bin/sh \n\
sed -i "s/\"version\"/\${VERSION}/g" file' > /usr/local/foo.sh
the image builds fine.
When I start a container from that image, and inspect the files:
# cat /usr/local/testfile
0.0.1
# cat /usr/local/foo.sh
#!/bin/sh
sed -i "s/\"version\"/\${VERSION}/g" file
I notice that the $VERSION was not replaced correctly in the sed command. What am I missing here? I've tried a few different things (e.g. "$VERSION") but none of them worked.
I ended up breaking down the command. I created a variable for the sed command by using string concatenation and then I echoed that to the file separately:
FROM ubuntu
ARG VERSION
ENV command="sed -i s/\"version\"/""$VERSION""/g"
RUN echo '#!/bin/sh' > /usr/local/foo.sh
RUN echo $command >> usr/local/foo.sh
# cat /usr/local/foo.sh
#!/bin/sh
sed -i s/"version"/0.0.1/g

Dockerfile RUN shell-script not running during docker build

I try to build a custom image for the EMQ MQTT server. But the script update_config.sh is not executed by during docker copmose up.
Dockerfile:
FROM emqttd-docker-v2.3.5
# change configuration file
ADD update_config.sh /opt/emqttd/etc/update_config.sh
ADD ./certs/MyEMQ1.key /opt/emqttd/etc/certs/MyEMQ1.key
ADD ./certs/MyEMQ1.pem /opt/emqttd/etc/certs/MyEMQ1.pem
ADD ./certs/MyRootCA.pem /opt/emqttd/etc/certs/MyRootCA.pem
WORKDIR /opt/emqttd/etc/
#update the emqtt config file
RUN /bin/ash -c /opt/emqttd/etc/update_config.sh
update_config.sh
#!/bin/ash
cd /opt/emqttd/etc
cp ./emq.conf ./emq.conf.bak
sed -i 's|.*listener.ssl.external.keyfile.*|listener.ssl.external.keyfile = etc/certs/MyEMQ1.key|g' ./emq.conf
sed -i 's|.*listener.ssl.external.certfile.*|listener.ssl.external.certfile = etc/certs/MyEMQ1.pem|g' ./emq.conf
sed -i 's|.*listener.ssl.external.cacertfile.*|listener.ssl.external.cacertfile = etc/certs/MyRootCA.pem|g' ./emq.conf
sed -i 's|.*listener.ssl.external.verify.*|listener.ssl.external.verify = verify_peer|g' ./emq.conf
I use docker-compose to build the image.
The update_config.sh script is copied to the image but not executed.
What I tried so far:
Used COPY instead of ADD to copy the file
Tried the RUN /bin/ash -c /opt/emqttd/etc/update_config.sh in the following
flavors:
RUN /bin/ash -c /opt/emqttd/etc/update_config.sh
RUN /opt/emqttd/etc/update_config.sh
RUN ./update_config.sh
Tried to add RUN chmod +x /opt/emqttd/etc/update_config.sh before the line RUN /bin/ash -c /opt/emqttd/etc/update_config.sh which results in the error chmod: /opt/emqttd/etc/update_config.sh: Operation not permitted during build
Can anyone help me? Thanks.
Just add ENTRYPOINT ["/bin/bash", "update_config.sh" ] this as your last line.
And also update_config.sh file to start your application and make your container in infinite loop.
Example update_config.sh:
#!/bin/ash
cd /opt/emqttd/etc
cp ./emq.conf ./emq.conf.bak
sed -i 's|.*listener.ssl.external.keyfile.*|listener.ssl.external.keyfile = etc/certs/MyEMQ1.key|g' ./emq.conf
sed -i 's|.*listener.ssl.external.certfile.*|listener.ssl.external.certfile = etc/certs/MyEMQ1.pem|g' ./emq.conf
sed -i 's|.*listener.ssl.external.cacertfile.*|listener.ssl.external.cacertfile = etc/certs/MyRootCA.pem|g' ./emq.conf
sed -i 's|.*listener.ssl.external.verify.*|listener.ssl.external.verify = verify_peer|g' ./emq.conf
sh start_your_app.sh
touch 1.txt;tail -f 1.txt #This will make your container in running infinite so that even after all the steps of this script has been executed your container will continue running. until you kill tail -f 1.txt command.
Hope this will help.
Thank you!
ash - is one of the smallest shells. This command interpreter has 24 built-in commands and 10 different command-line options.
ash hasn't all commands which you need. You should use /bin/bash

How to Search for a number and replace it with other number

I have a file server.xml with below line-
Server port="8007" shutdown="SHUTDOWN"
I want to search for 8007 and replace it with other number eg.: 8010.
How can I do it?
try this :
sed 's/8007/8010/g' yourfile
To make replacements you can use sed. I suppose that the new "port" is stored in a variable. As a result both solutions bellow works ok with GNU Sed:
$ a=$'Server port="8007" shutdown="SHUTDOWN"';echo "$a"
Server port="8007" shutdown="SHUTDOWN"
$ newport="8010"
$ sed -r "s/([0-9]+)/$newport/g" <<<"$a"
Server port="8010" shutdown="SHUTDOWN"
# Alternative:
$ sed -r 's/(port=")(.*)(" .*)/\1'"$newport"'\3/g' <<<"$a"
sed -r (or sed -E) enables extended regex groups.
If you need to apply replacements in-place (on the file) you need to use also -i switch in sed (sed -r -i.bak "....")

Editing Files from dockerfile

I need to add several lines to /etc/sysctl.conf in a Docker image.
Is there an idempotent way to do this via a Dockerfile rather than editing manually and using the docker commit approach?
I would use the following approach in the Dockerfile
RUN echo "Some line to add to a file" >> /etc/sysctl.conf
That should do the trick. If you wish to replace some characters or similar you can work this out with sed by using e.g. the following:
RUN sed -i "s|some-original-string|the-new-string |g" /etc/sysctl.conf
However, if your problem lies in simply getting the settings to "bite" this question might be of help.
sed work pretty well to replace stuff, if you need to append, you can user double redirect
sed -i 's/origin text/new text/g' /etc/sysctl.conf
bash -c 'echo hello world' >> /etc/sysctl.conf
-i is a non-standard option of GNU sed for inline editing (alleviating the need for dealing with temporary files).
The s is the substitute command of sed for find and replace
The g means global replace i.e. find all occurrences of origin text and replace with new text using sed

Resources