I'm upgrading to CircleCI version 2.1
version: 2.1
I would like to declare a parameter, but give it a value during the run.
The reason for this is to utilize the when clause.
This would look something like:
parameters: my-param:
type: string
default: ""
jobs:
build:
steps:
- checkout
- run:
name: Set the param
command: << parameters.my-param >> = my-value
The reason for this is I do not know the value until I run a command on the commit that triggered the build. I did something similar in 2.0, so that I could skip certain jobs based on the input, but I've run into limitations of 2.1.
Is this possible?
Any ideas for a workaround that achieve what I'm trying to do would be greatly appreciated!
Related
Somehow I feel this should be very simple, and I cannot seem to find a way to do it with the CircleCI config.
I have a simple task—take one value and map it to another based on a predefined map.
Something like this:
buckets:
staging:
dashboard: staging-dashboard.example.com
referrer: referrer-stage.example.com
YAML allows me to define this without problems. My issue is I cannot figure out how to use it in CircleCI config:
deploy:
executor: gcp/google
parameters: *build_parameters
steps:
- attach_workspace:
root: .
- gcp/install
- configure_google_sdk
- gcp/initialize:
gcloud-service-key: GCLOUD_SERVICE_KEY_DECODED
- get_bucket: <<< HOW_TO_DO_THIS >>>
environment: <<parameters.environment>>
dashboard_type: <<parameters.dashboard_type>>
- gcp-storage/upload:
source_path: dist/apps/<<parameters.dashboard_type>>-admin/**
destination_bucket: $BUCKET_NAME
My only solution so far has been to write a custom bash script which would map things inside code and set $BUCKET_NAME environment variable. It seems like a massive overkill for such a simple thing, and the last thing I want is to store a mapping in some cryptic bash script.
Any better ideas?
With GitHub Actions I can read from JSON files directly. Maybe something like this exists for CircleCI?
Following the docs, I set my environment variable in the console ($CLIENT_ID).
In the console I added the echo command to try and insert the variable into a .env.
The error I keep getting is There was an issue connecting to your repo provider. When I remove the echo line the build passes. I've tried single/double quotes and putting the line above/below the other lines under the build commands phase.
Here's the backend section for the build process.
backend:
phases:
build:
commands:
- echo 'CLIENT_ID=$CLIENT_ID' >> backend/.env
- '# Execute Amplify CLI with the helper script'
- amplifyPush --simple
I wrote an comment but to make it easier, I quote from answers from this question
build:
commands:
- npm run build
- VARIABLE_NAME_1=$VARIABLE_NAME_1 # it works like this
- VARIABLE_NAME_2=${VARIABLE_NAME_2} # it also works this way
Please thumb up on the original answers, and flag this question as duplicated.
Seems this is a feature request:
https://github.com/aws-amplify/amplify-cli/issues/4347
I have a circle config which includes the following custom command:
remove-circle-ip:
description: "remove current Circle CI box IP from inbound security group rules for DB"
steps:
- aws-white-list-circleci-ip/remove:
tag-key: circleci
tag-value: whitelistmeplease
port: 5432
which I use in my job as follows:
jobs:
test:
docker:
- image: nikolaik/python-nodejs:python3.8-nodejs12
environment:
AWS_DEFAULT_REGION: us-east-2
steps:
- setup
- install-python-deps
- add-circle-ip
- run:
name: run tests
command: |
poetry run coverage run --source='.' manage.py test
- run:
name: remove circle IP
command: remove-circle-ip
when: always
I'd like the step for remove circle IP to run even if the tests which run before it fail. I can't seem to figure out the syntax for this. Previously, I had just used - remove-circle-ip to run the command rather than putting a run block, i.e.:
jobs:
test:
docker:
...
steps:
- setup
- ...
- add-circle-ip
- ...
- remove-circle-ip
but couldn't figure out how to specify when: always if I did it that way.
But now, when switching to calling my command as part of a run block, it fails with "remove-circle-ip: command not found"
So how can I make this command always run even if steps before fail?
I'm fairly new to CircleCI so there may be a better way to do this, or maybe this shouldn't be done at all, however something similar was done (before I joined) to a project I'm working on. It was achieved by making every step report success, whether it actually succeeded or failed, which allows the command at the end to always run. The commands are all terminal commands, so they just have || true at the end. I'm not sure how you would achieve that with a more complex command or using a builtin command.
In our case the steps that can fail are optional and we don't care if they actually fail or not. However if you want to report the failure I think that you should be able to store the failure from a previous step somewhere, and add a final step that reports it.
I am using opensource Jenkins 2.235.3 version on Centos 7.7 machine with Active Choices Plug-in Version 2.4
The problem I am running into is very strange
I am using a groovy script to dynamically populate parameters and I am using Active Choices Reactive Parameter. In that plugin I have written a following groovy script
Groovy Code:
this.class.classLoader.addURL(new URL("file:///var/lib/jenkins/.groovy/grapes/org.yaml/snakeyaml/jars/snakeyaml-1.17.jar"))
def url = 'git#xxx.xxx.com:csd/deployment.git'
def envlist = ['/bin/bash', '-c', ' ssh-add /var/lib/jenkins/.ssh/id_rsa_git; cd /var/lib/jenkins/services/; git archive --remote='+ url +' master inventory | tar xvf -' ].execute().
envlist.waitFor()
def yaml = Class.forName("org.yaml.snakeyaml.Yaml").newInstance()
def config = yaml.load(new FileReader("/var/lib/jenkins/services/inventory/${SERVICES}.yml"))
return config.all.children."${SERVICES}".children.keySet()
NOTE: SERVICES is another defined parameter
Whereas when I use Active choice reactive reference parameter with same code:
Same groovy code works and prints out the following result
"[functional, staging_stable, pilot_stable, prod_stable]"
When I use script console in manage jenkins, it runs fine and prints me the same result
Result:
[functional, staging_stable, pilot_stable, prod_stable]
When I check web console logs, I get
"Calling Java server code to update HTML elements... unochoice.js:154:17
Values returned from server: [[],[]] unochoice.js:157:21
How can I proceed with debugging, I want to only use Active Choices Reactive Parameter?
I have attached the screenshot to show how one is populated and one is not. Not enough to go through and debug this problem over the internet.
Can somebody please help?
I created my first pipeline yesterday and I wanted to replace a placeholder in my bundle.gradle file with the CIRCLE_BUILD_NUM environment variable. The only method I found find was writing my own ‘sed’ command and executing the regex in a run statement. This worked fine to get up and running, since there was only one variable to replace, however this method obviously won’t scale, down the road. Is there a CircleCI feature/orb or other method to do a more comprehensive placeholder/envar swap throughout my project?
- run:
name: Increment build id
command: sed "s/_buildNum/${CIRCLE_BUILD_NUM}/g" -i build.gradle
EDIT
Looking for a utility/tools/orb/CircleCI best practice similar to what they have in Azure DevOps (Jenkins performs a similar feature as well): simply replace all placeholders in specified files with environment variables matching the same name.
https://marketplace.visualstudio.com/items?itemName=qetza.replacetokens
There is envtpl tool with myriad of implementation in various languages.
It allows for interpolating variables in templates with values set in environment variables.
The following described command installs an implementation in Rust.
commands:
replace-vars-from-env:
description: Replace variables in file from environment variables.
parameters:
filename:
type: string
steps:
- run:
name: Replace variables in build.gradle file
command: |
if ! [ -x /usr/local/bin/envtpl ]; then
curl -L https://github.com/niquola/envtpl/releases/download/0.0.3/envtpl.linux > /usr/local/bin/envtpl
chmod +x /usr/local/bin/envtpl
fi
mv <<parameters.filename>> <<parameters.filename>>.tpl
cat <<parameters.filename>>.tpl | envtpl > <<parameters.filename>>
rm <<parameters.filename>>
and use that in other commands or as a part of your jobs. For example,
executors:
linux:
machine:
image: ubuntu-1604:201903-01
jobs:
build:
executor: linux
steps:
- replace-vars-from-env:
filename: build.gradle
You could use envsubst which provides that basically out of the box.
Depending on your primary container you can install envsubst on top of alpine/your distro, or use some image that has that already, like datasailors/envsubst.
In that case, you would just need to run configure like:
- run:
name: Increment build id
command: envsubst < build.gradle.template > build.gradle
And in your template file you can have ${CIRCLE_BUILD_NUM}, as many other variables directly.