If I use a environment variable the circle.yml bellow, fails, But if I statically type the machine name it will work.
How can I properly reference environment variables in CircleCI?
version: 2
executorType: machine
stages:
build:
workDir: ~/app
enviroment:
- IMAGE_NAME: "nginx-ks8-circleci-hello-world"
# - AWS_REGISTER: "096957576271.dkr.ecr.us-east-1.amazonaws.com"
steps:
- type: checkout
- type: shell
name: Build the Docker image
shell: /bin/bash
command: |
docker build --rm=false -t $IMAGE_NAME .
I check your syntax with this example of circleci docs https://circleci.com/docs/2.0/language-python/#config-walkthrough so you have to remove the hiphen
enviroment:
IMAGE_NAME: "nginx-ks8-circleci-hello-world"
Thats for the environment variable inside the docker image for CircleCi 2.0.
Circle runs each command in a subshell so there isn't a way to set environment variables for the CircleCi build from the build itself.
Instead use the actual CircleCi environment variables:
https://circleci.com/gh/{yourOrganization}/{yourRepo}/edit#env-vars
Related
We have used the technique detailed here to expose host environment variables to Docker build in a secured fashion.
# syntax=docker/dockerfile:1.2
FROM golang:1.18 AS builder
# move secrets out of the build process (and docker history)
RUN --mount=type=secret,id=github_token,dst=/app/secret_github_token,required=true,uid=10001 \
export GITHUB_TOKEN=$(cat /app/secret_github_token) && \
<nice command that uses $GITHUB_TOKEN>
And this command to build the image:
export DOCKER_BUILDKIT=1
docker build --secret id=github_token,env=GITHUB_TOKEN -t cool-image-bro .
The above works perfectly.
Now we also have a docker-compose file running in CI that needs to be modified. However, even if I confirmed that the ENV vars are present in that job, I do not know how to assign the environment variable to the github_token named secret ID.
In other words, what is the equivalent docker-compose command (up --build, or build) that can accept a mapping of an environment variable with a secret ID?
Turns out I was a bit ahead of the times. docker compose v.2.5.0 brings support for secrets.
After having modified the Dockerfile as explained above, we must then update the docker-compose to defined secrets.
docker-compose.yml
services:
my-cool-app:
build:
context: .
secrets:
- github_user
- github_token
...
secrets:
github_user:
file: secrets_github_user
github_token:
file: secrets_github_token
But where are those files secrets_github_user and secrets_github_token coming from? In your CI you also need to export the environment variable and save it to the default secrets file location. In our project we are using Tasks so we added these too lines.
Note that we are running this task from our CI, so you could do it differently without Tasks for example.
- printenv GITHUB_USER > /root/project/secrets_github_user
- printenv GITHUB_TOKEN > /root/project/secrets_github_token
We then update the CircleCI config and add two environment variable to our job:
.config.yml
name-of-our-job:
environment:
DOCKER_BUILDKIT: 1
COMPOSE_DOCKER_CLI_BUILD: 1
You might also need a more recent Docker version, I think they introduced it in a late 19 release or early 20. I have used this and it works:
steps:
- setup_remote_docker:
version: 20.10.11
Now when running your docker-compose based commands, the secrets should be successfully mounted through docker-compose and available to correctly build or run your Dockerfile instructions!
I am trying to build and run a docker container using ansible, but I am unable to pass the environment variable to the Dockerfile in the build state.
Below is my ansible file dev.yml
---
- name: setup docker
tasks:
- name: build dockerfile
community.docker.docker_container:
name: test
tag: v0
path: .
nocache: yes
env:
TEST_ENV: "SOME_TESTS_VARIABLE"
SSH_KEY: "{{LOCAL_SSH_KEY}}"
I am running ansible-playbook,
ansible-playbook -i hosts dev.yml -e "LOCAL_SSH_KEY='$(cat ~/.ssh/id_rsa)'"
I have figured out looking at the doc & the errors that community.docker.docker_container does not support nocache, path, tag and to build the container I should rather use docker_image which then does not supports env
Is there a way to build docker containers using the environment variables.
In other words how can I pass my ssh keys to the docker build step?
I have looked at other answers but those don't work for me, maybe because of ansible version 2.10 which I am using.
For example I've setup name: FOO value: 'bar'.
I've validated that the key value works. Because what does work is:
jobs:
build:
docker:
- image: circleci/node:10.17.0
steps:
- run: |
node something $FOO
However, the following does not work:
Now when I deploy and try to use it in my application it returns undefined:
console.log(process.env.FOO); // returns undefined
I tried setting it under the 'environment' key in the config.yml file:
jobs:
build:
docker:
- image: circleci/node:10.17.0
environment:
- FOO # note, don't use $FOO
steps:
- run: |
node something $FOO
ssh $MACHINE -- 'cd /home/ && docker build -t myApp . docker restart myApp'
But still no change.
Should I perhaps pass the variables to the build script in the ssh command?
Any ideas?
update based on Delena's tip
Kept ./circle-ci/config.yml as:
jobs:
build:
docker:
- image: circleci/node:10.17.0
environment:
FOO: $FOO
Then in the docker-compose file:
myApp:
environment:
- FOO
Will accept the answer when the build is green
It looks like you're trying to access the environment variable from an app that runs in a Docker container, but you're not setting the environment variable in the container.
If that's the case, you can check out How to set an environment variable in a running docker container, but it looks like you'll have to stop the container and restart it again with the environment variable.
You could do something like:
ssh $MACHINE -- 'cd /home/ && docker build -t myApp && docker stop myApp && docker run -e "FOO=$FOO"'
Also check out the ENV (environment variables) section in the docker run docs.
I am new to docker.I want to pass gitlab-ci variable to Dockerfile. Tried lot of things but nothing works. Below is my gitlab-ci.yml
api-tests:
image: test-img
stage: test
services:
- docker:abc
variables:
privileged: "true"
DOCKER_HOST: tcp://localhost:2375
script:
- apk add dialog && apk add bind-tools
- docker login -u $ci_account -p $ci_token $REGISTRY_HOST
- git clone --single-branch --depth 1 --recurse-submodules --branch master ssh://git#git.easygroup.co:1234/test/code.git test && cd test
- cd -
- make test-project
- TEST_PATH=first make test; r=$?
after_script:
- docker ps -a
- docker logs --tail=50 test-project
Thanks
You need to create environment variables using .gitlab-ci.yml variables sections. Once environment variables are created then can be accessed by script or other process.
Git lab docs:
Creating a custom environment variable
Assume you have something you want to repeat through your scripts in GitLab CI/CD’s configuration file. To keep this example simple, let’s say you want to output HELLO WORLD for a TEST variable.
You can either set the variable directly in the .gitlab-ci.yml file or through the UI.
Via .gitlab-ci.yml
To create a new custom env_var variable via .gitlab-ci.yml, define their variable/value pair under variables:
variables:
TEST: "HELLO WORLD"
For a deeper look into them, see .gitlab-ci.yml defined variables.
More info here->https://docs.gitlab.com/ee/ci/variables/#gitlab-ciyml-defined-variables
So I have a CloudBuild trigger that builds my cloudbuild.yaml file and this is all fine and dandy. I also use the gcloud builder to run docker commands to pass ENV variables to my Dockerfile. for example:
steps:
- name: 'gcr.io/$PROJECT_ID/swift:4.2'
args: ['test']
id: 'Running unit tests'
- name: 'gcr.io/cloud-builders/docker'
args: ['build','--build-arg', 'PROJECT=$PROJECT_ID','-t', 'us.gcr.io/$PROJECT_ID/$BRANCH_NAME:$SHORT_SHA', '.']
id: 'Building docker image'
- name: 'gcr.io/cloud-builders/docker'
args: ["push", "us.gcr.io/$PROJECT_ID/$BRANCH_NAME:$SHORT_SHA"]
id: 'Pushing built image to registry'
- name: 'gcr.io/cloud-builders/gcloud'
args: ['app', 'deploy']
id: 'Deploying to AppEngine'
timeout: 1800s # 30 minute timeout
As you can see I, I'm using the ENV variables that all GCP resources have by default.($PROJECT_ID for example). And in the docker command I'm passing it as an argument so I can use the ARG command in the dockerfile:
ARG PROJECT
FROM gcr.io/${PROJECT}/swift:4.2 as builder
WORKDIR /App
#Other commands....
Now all of this works fine and I'm able to build my image etc. now I want to deploy to app engine in the final step.
Only problem is that I'm using the same Dockerfile to uses the swift:4.2 base image that's only located in my GoogleContainerRegistry so I need the $PROJECT_ID for my project to pull that.
My question is: Is there any way to have AppEngine build environment pass arguments to the docker build that builds my image when deploying? I have an app.yaml file and I know there's an env_variables: property and I know I'd be able to use the docker ARG or ENV command (can't remember which one) to get my $PROJECT_ID inside my Dockerfile. But the only problem is AppEngine doesn't have that Property defined as far as I know. The only other thing I can think of is to echo the $PROJECT_ID from Cloud Builder step to the end of the app.yaml file. But if there's a cleaner approach I'd love to hear about it. Thanks!
I think I've found a solution for my needs.
gcloud app deploy has a flag image-url that can specify an already built image rather than rebuilding the Dockerfile. So I went with this as my final cloudbuild.yaml
- name: 'gcr.io/cloud-builders/gcloud'
args: ['app', 'deploy', '--image-url', 'gcr.io/$PROJECT_ID/$BRANCH_NAME:$SHORT_SHA']
Basically point to the image I just built and pushed to my container registry.