I have an environment variable on my server (local raspberry pi) that stores a token that I need. When I run the docker container myself, it doesn't seem to have any problems getting the token from the environment variable. The variable is exported in my .bashrc and can be seen with echo.
When running a workflow through github actions with the same steps, the application cannot find the token.
The environment variable is consistently not there when I check for it. After thinking maybe it was having trouble accessing my .bashrc file, I made a github secret and have been trying to pass the value referencing that instead as you can see below.
I have these lines in my Dockerfile:
ARG MY_TOKEN
ENV MY_TOKEN=$MY_TOKEN
And this is my workflow yaml:
docker build --build-arg MY_TOKEN=${{ secrets.MY_TOKEN }} -t my_img ~/my_project
docker run -de MY_TOKEN --restart=always --name=my_container my_img
Any guidance will be greatly appreciated, I feel like I could get this to work by passing system arguments but
I'm unsure if that's good practice
I'd feel better if someone could point out my bonehead mistake thats holding me back before doing something else
Related
Is there a way to get all environment variables that a Docker image accepts? Including authentications and all possible ones to make the best out of that image?
For example, I've run a redis:7.0.8 container and I want to use every possible feature this image offers.
First I used docker inspect and saw this:
"Env": [
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
"GOSU_VERSION=1.16",
"REDIS_VERSION=7.0.8",
"REDIS_DOWNLOAD_URL=http://download.redis.io/releases/redis-7.0.8.tar.gz",
"REDIS_DOWNLOAD_SHA=06a339e491306783dcf55b97f15a5dbcbdc01ccbde6dc23027c475cab735e914"
],
I also tried docker exec -it my-container env which just showed me the same thing. I know there are more variables, for example this doesn't include the following:
REDIS_PASSWORD
REDIS_ACLS
REDIS_TLS_CERT_FILE
Absent documentation, this is pretty much impossible.
Let's start by repeating #jonrsharpe's comment:
They accept any env var at all, but they won't respond to all of them.
Consider this Python code, for example:
import os
def get_environ(d, name):
d.get(name, 'absent')
foo = os.environ.get('FOO', 'default_foo')
star_foo = get_environ(os.environ, foo)
print(star_foo)
This fragment looks up an environment variable $FOO. You could probably figure that out, if you knew the main process was in Python and recognized os.environ. But then it passes that value and the standard environment to a helper function, which looks up that environment variable by name. You'd need detailed static analysis to understand this is actually also an environment-variable lookup.
$ ./test.py
absent
$ default_foo=bar ./test.py
bar
$ FOO=BAR BAR=quux ./test.py
quux
$ I=3 ./test.py
absent
(A fair bit of the code I work with accesses environment variables rather haphazardly; it's not just "find the main function" but "find every ENV reference in every file in every library". Some frameworks like Spring Boot make it possible to set hundreds of configuration options via environment variables, and even if it were possible to get every possible setting here, the output would be prohibitive.)
"What environment variables are there" isn't standard container metadata. You'd have to identify the language the main container process runs, and do this sort of analysis on it, including compiled languages. That doesn't seem like a solvable problem.
In my ~/.bashrc, I have set GOOGLE_APPLICATION_CREDENTIALS=~/.gc/credential_file_name.json.
My source code is located in (and I'm working from here) ~/repos/github_repo/ where I have a Dockerfile with its working directory set to /usr/src/app.
If I copy ~/.gc/credential_file_name.json to ~/repos/github_repo/credential_file_name.json and run the docker container with
docker run -t \
-e GOOGLE_APPLICATION_CREDENTIALS=/usr/src/app/credential_file_name.json \
...
the credential file gets picked up and subsequent code runs ok.
But, ideally, I don't want to copy the credential to my github repository, as that risks possibly pushing it on github (even when I add it to .gitignore, it's still not safe).
Additionally, instead of having to explicitly give then full path -e GOOGLE_APPLICATION_CREDENTIALS=/usr/src/app/credential_file_name.json, I would like to do something like -e GOOGLE_APPLICATION_CREDENTIALS=${GOOGLE_APPLICATION_CREDENTIALS} where ${GOOGLE_APPLICATION_CREDENTIALS} gets picked up from my ~/.bashrc.
But obviously, ${GOOGLE_APPLICATION_CREDENTIALS} will point to a path on my computer, which different directory structure than the docker container.
What is the best way to resolve this? I'm new to this and I came across direnv and .envrc, but don't quite understand.
I'm using Makefile to run the docker commands. I will try to avoid docker-compose, but if it solves this problem, please let me know.
Thanks for help!
I have seen some similar questions, but none of them appear to solve my problem. I want to add a user to a docker container and in my Dockerfile, I define the username with:
ARG USERNAME="some_user"
Instead, I want the username to be the current user's computer username, as obtained by running the command whoami in the local terminal.
So what I would like to have is something like
ARG USERNAME=$(whoami)
.
This $(whoami) should be obtained from the local system environment, and not from the docker container.
Is there a way to do this for dockerfiles? I have thought of .env and docker-compose solutions but these also require each user to set their own username according to my knowledge.
There is no integrated way to execute arbitrary commands on the host directly outside of a container using just docker build / docker-compose build.
So to execute an arbitrary command to get/generate the required information you'll need to provide a custom script / use another build system to call docker/docker-compose with the respective flags or maybe generate the .env file from a template / interactively.
If you only need the current user name you may want to use the $USER / $LOGNAME environment variables that are set by the system in many default configurations. But since these are just normal environment variables their values may be incorrect / empty / manually changed by the user, see this question.
I am trying to pass on environment variables to be read from an XML file inside a docker container running wildly app service and hosted inside REHL 7 image.
What I've done so far:
I've created an environment file as key value pair, for example: FILESERVICE_MAX_POOL_SIZE=5
I am running docker by referencing the environment file: docker run -d --env-file ./ENV_VARIABLES <myImage>
In the Dockerfile I copy the xml template I need: COPY dockerfiles/standalone.xml /opt/wildfly/standalone/configuration/standalone.xml
Inside the XML template I'm trying to reference the environment variable: <max-pool-size>${env.FILESERVICE_MAX_POOL_SIZE}</max-pool-size>
I can see those environment variables inside the running container as root but not as the wildly user which needs them. How can I make an attribute visible to a specific user other than root ?
Clearly I'm doing something fundamentally wrong here just not sure what ?
Thanks in advance for your help.
Problem solved: wildfly couldn't see the attributes because in my startup script I didn't add the -E flag for sudo to preserve environment variables.
I am struggling with the same problem mentioned by Gavin on
this question.
Specifically in with new
docker build secret information
What is the right way to use it that feature?
Looking around on the internet I only found some variations of the same example in docker documentation mentioned above, which prints the secret on build time. Maybe I didn't fully understand the example, so please help me.
If there is no way to get the secret in build time an use in another part of a Dockerfile (e.g. an ARG variable or RUN command), when and how that new feature can be used to truly keep my secret safe and also do the work?
My go is to use this new feature in build time and also keep my secret information safe in case someone get my image file and execute a history on it.
For example, ff I have a Dockerfile like this:
FROM influxdb:2.0
ENV DOCKER_INFLUXDB_INIT_MODE=setup
ENV DOCKER_INFLUXDB_INIT_USERNAME=admin
ENV DOCKER_INFLUXDB_INIT_PASSWORD="mypassword"
How can I use that new feature mentioned in docker documentation to set my variable (DOCKER_INFLUXDB_INIT_PASSWORD), for example, in a way that it will not logged in image history?
Thanks in advance
How can I use that new feature mentioned in docker documentation to
set my variable (DOCKER_INFLUXDB_INIT_PASSWORD), for example, in a way
that it will not logged in image history?
It depends on whether you need the secret only at build time, or
whether you actually want to use it at runtime. The latter situation
is probably more common, and there's already a canonical solution:
If you want to keep DOCKER_INFLUXDB_INIT_PASSWORD out of your image
history, just don't set it during the build process. Require it to be
set a runtime, e.g., via the -e VAR=VALUE argument to docker run
(or the --env-file option):
docker run -e DOCKER_INFLUXDB_INIT_PASSWORD=mypassword myimage
You could have an ENTRYPOINT script that checks for the variable at
runtime and exits with a helpful error message if it's not set.
The official Docker images for things like MySQL and PostgreSQL work
this way.
In contrast, a build secret is meant for secrets that are only
required at build time. For example, let's say your build process
needs to do something like this:
RUN curl -o /data/mydataset -u username:password \
https://example.com/dataset
You'd like to share your Dockerfile and associated sources with
other people, but you don't want to share your username and password.
This is where build secrets come in. You would instead stick your
authentication information in a file, and modify your Dockerfile to
read that information from a secret:
RUN --mount=type=secret,id=mysecret \
curl -o /data/mydataset -u $(cat /run/secrets/mysecret) \
https://example.com/dataset
In this example, once we've copied the remote file into the image,
we're done with the secret: we don't need it in order to run a
container from the image; it was only required during the build
process.
NB: The above assumes that you're providing the secret at build time as described in the documentation, so your build command might look something like:
DOCKER_BUILDKIT=1 docker build --secret id=mysecret,src=mysecret.txt -t myimage .