What does this mean in docker-compose.yml directive?
${{ variable }}
And how is it used?
In a pipeline, template expression variables (${{ variables.var }}) get processed at compile time, before runtime starts. Macro syntax variables ($(var)) get processed during runtime before a task runs. Runtime expressions ($[variables.var]) also get processed during runtime but were designed for use with conditions and expressions. When you use a runtime expression, it must take up the entire right side of a definition.
In the following example, you can see that the template expression still has the initial value of the variable after the variable is updated. The value of the macro syntax variable updates. The template expression value does not change because all template expression variables get processed at compile time before tasks run. In contrast, macro syntax variables are evaluated before each task runs.
variables:
- name: one
value: initialValue
steps:
- script: |
echo ${{ variables.one }} # outputs initialValue
echo $(one)
displayName: First variable pass
- bash: echo "##vso[task.setvariable variable=one]secondValue"
displayName: Set new variable value
- script: |
echo ${{ variables.one }} # outputs initialValue
echo $(one) # outputs secondValue
displayName: Second variable pass
Related
Right now, if I want to pass a environment variable to a reusable workflow I have to do something like this:
name: Reusable workflow
on:
workflow_call:
inputs:
my_env_var:
required: false
type: string
env:
my_env_var: ${{ inputs.my_env_var }}
However, for this I first need to define as an input each environment variable I want to pass. This works, but having to hard code the environment variables makes my reusable workflows less generic. Is there a way to pass envs without having to define them one by one? I was thinking on something like this:
name: Calling reusable workflow
on:
workflow_dispatch:
jobs:
push-image-dev:
uses: ./.github/workflows/my-reusable-workflow.yml
with:
input1: ...
input2: ...
env:
env1: ...
env2: ...
However, I have been reading some documentation and I don't think that exists. Is there any other way of doing it, as inheriting env variables or creating a single input which is a variable dictionary, which is later parsed and sets all the env vars in the reusable workflow?
I created this simple logic which allows me to pass all the environment variables I want on a 1:N relationship with respect to inputs. I created an input which expects a list of environment variables, formatted as "env=value", and which is later converted to environment variables in a step inside the workflow.
Calling the workflow:
name: Calling reusable workflow
on:
workflow_dispatch:
jobs:
my-job:
uses: ./.github/workflows/my-reusable-workflow.yml
with:
env_vars: |
env1=value1
env2=value2
env3=value3
Workflow definition:
name: My reusable workflow
on:
workflow_call:
inputs:
env_vars:
description: List of environment variables to set up, given in env=value format.
required: false
type: string
jobs:
my-job:
runs-on: ubuntu-latest
steps:
- name: Set environment variables
if: ${{ inputs.env_vars }}
run: |
for i in "${{ inputs.env_vars }}"
do
printf "%s\n" $i >> $GITHUB_ENV
done
Please I'm trying to run some steps in the CircleCI Pipeline with conditions happened in the previous step. I tried a lot of tricks like exposing the value from Step 1 to global vars and pickup it in Step 2, I can see and print the variables in Step 2 but using WHEN BLOCK forever evaluated with Empty. I searched a lot and I knew that logical conditions already evaluated before running the jobs, Please I need alternative way to execute steps in second job in case a condition happened in Step 1?
I pasted here the example that I'm trying to fix
version: 2.1
orbs:
workflows:
test-and-deploy:
jobs:
- set-data:
context: my-context
- read-data:
context: my-context
requires:
- set-data
definitions:
node_image: &node-image
docker:
- image: cimg/node:14.15.5
executors:
base-12-14-0:
description: |
Single Docker container with Node 12.14.0 and Cypress dependencies
see https://github.com/cypress-io/cypress-docker-images/tree/master/base.
Use example: `executor: cypress/base-12-14-0`.
docker:
- image: cypress/base:12.14.0
jobs:
set-data:
<<: *node-image
description: Sets the data
steps:
- run: echo "VAR=app" > global-vars
- persist_to_workspace:
root: .
paths:
- global-vars
read-data:
<<: *node-image
description: read the data
steps:
- attach_workspace:
at: .
- run: ls
- run: cat global-vars // I COULD HERE SEE THE CORRECT VAR inside global-vars
- run: cat global-vars >> $BASH_ENV
- run: echo "Test $VAR" // Successfully Printed
- when:
condition:
matches: {
pattern: "app",
value: $VAR
}
steps:
- run: echo "Condition Executed"
It's not possible to use environment variables in logic statements. The reason is that logic statements are evaluated at configuration compilation time, whereas environment variables are interpolated at run time.
The only workaround I know of is to use the CircleCI dynamic configuration functionality to set pipeline parameters' values in the "setup workflow" that you then pass to the "continuation" workflow.
And by the way, you're not using $BASH_ENV correctly (https://circleci.com/docs/env-vars#setting-an-environment-variable-in-a-shell-command). But again, even if you did, you wouldn't be able to use an environment variable in a logic statement.
I have a yml file which calls a templated stage:
- template: pipeline-templates/iac/deploy-template.yml
parameters:
envName: dev
varGroup: dev-variables-group-library
In the template I have made use of the template parameter:
parameters:
envName: ''
varGroup: ''
stages:
- stage: Deploy_for_${{ parameters.envName }}
I want to use the envName variable from the variable group, instead of the template parameters. I have tried template code like this:
parameters:
envName: ''
varGroup: ''
stages:
- stage: Deploy_for_$( envName )
When I do this, the pipeline throws an unrelated error, for example stating that the stage name cannot be duplicate. This is clearly because the variable envName is not being read.
There are other places in my template where the pipeline variable is read and used successfully, so what am I doing wrong in this particular part of the template script?
So if wanted to use a variable group need to define the group to be read in:
- group: ${{parameters.varGroup}}
After doing this should be able to reference by $(envName) or if compile time is alright can also specify ${{ varaibles.envName }}
Microsoft Documentation
I'm trying to create dynamic Jenkins job pipeline stages based on an array of values but I can't seem to get the loop functioning as expected, it complains about the syntax I'm using but I can't figure it out, is this a Groovy issue?
Approach
uat_nodes:
- 'node1'
- 'node2'
dsl: |
stage('Update UAT dist') {{
build job: '{key}-{module}-DP-BuildNamedDist-UAT'
}}
def UAT_NODES = {uat_nodes}
UAT_NODES.each { UAT_NODE ->
stage('Deploy code to UAT node: ' . ${{UAT_NODE}}) {{
build job: '{key}-{module}-DP-UAT-Nodes', parameters: [
string(name: 'LIMIT', value: '${{UAT_NODE}}'),
string(name: 'PLAYBOOK', value: '{playbook}')
]
}}
}
Error
WorkflowScript: 8: Ambiguous expression could be either a parameterless closure expression or an isolated open code block;
solution: Add an explicit closure parameter list, e.g. {it -> ...}, or force it to be treated as an open block by giving it a label, e.g. L:{...} # line 8, column 56.
e to UAT node: ' . ${{UAT_NODE}}) {{
As the error states, there is a problem with this piece of code: . ${{UAT_NODE}}
If strings would have a $ method, that would call it with a closure inside a closure, that returns UAT_NODE.
I can only assume, that you want to concat strings here similar to perl or php. This is not how it works in groovy.
Use: "Deploy code to UAT node: ${UAT_NODE}". Note the double quotes "! Single quotes ' won't give you replacement (this is every other string you are using in your code).
I am having these lines in my Jenkinsfile:
parameters {
string(name: 'DATABASE', defaultValue: 'jenkinsdatabase',
description: 'The name of the database')
}
(...)
Now I want to use the value of ${params.DATABASE} in a step of a stage e.g.
sh 'mysql --user ${USER} -p${PASSWORD} --host ${HOST} -e "DROP DATABASE IF EXISTS ${params.DATABASE};CREATE DATABASE ${params.DATABASE} DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; commit;";export GRADLE_OPTS="-Xms1536m -Xmx1536m"'
But this ends with a exeception: Bad substitution
Can anybody help me?
Your sh script body uses single quotes around it.
sh '...'
Since these are single quotes, string interpolation will not occur. This means that all of your $ strings will use the shell variable substitution instead of the Groovy substitution. You are getting a Bad substitution in the shell because the shell won't be able to substitute ${params.DATABASE} because that is an invalid shell substitution.
params is a Groovy variable, so switching your single quotes to double quotes will perform string interpolation of the variables.
sh "..."