I am new to Jenkins pipeline can you please provide below out by using groovy declarative pipeline and -D should be appended for every key.
Input request should be string '''name= ram id = 123 role = IT''' and output needed as -Dname=ram -Did=123 -Drole=IT
pipeline{
agent any
parameters {
text defaultValue: '''name = ram id = 123 role = IT]''', description: 'employee details', name : 'details'
}
environment {
emp_details = "${env.details}"
}
stage('build') {
steps{
script{
echo "dislay details, ${emp_details }"
for(e in emp_details){
print e + ":" + emp_details[e])
}
}
}
}
}
Final output: -Dname=ram -Did=123 -Drole=IT
error: No Such property: [ for class: java.lang.String
if passing input is not right, how can we pass it?
You are getting a String not an array, so first, you need to create an iterable list and modify each key by iterating.
pipeline {
agent any
parameters {
text defaultValue: '''[name = ram, id = 123, role = IT]''', description: 'employee details', name : 'details'
}
environment {
emp_details = "${env.details}"
}
stages {
stage('Hello') {
steps{
script{
echo "dislay details, ${params.details}"
def appendedString = ""
// SInce it's a string we need get rid of [ and ] characters and then the spaces.
def lst = params.details.replaceAll("[^0-9a-zA-Z=,]+","").split(',')
println lst
for(e in lst){
print(e)
appendedString += "-D" + e + " ";
}
println "Final String: " + appendedString;
}
}
}
}
}
Update
Update the split logic to support the new string. '''name = ram id = 123 role = IT'''
def lst = params.details.replaceAll("\\s=\\s", "=").split("\\s")
Related
I'm trying to convert my jenkins pipeline to a shared library since it can be reusable on most of the application. As part of that i have created groovy file in vars folder and kept pipeline in jenkins file in github and able to call that in jenkins successfully
As part of improving this i want to pass params, variables, node labels through a file so that we should not touch jenkins pipeline and if we want to modify any vars, params, we have to do that in git repo itself
pipeline {
agent
{
node
{
label 'jks_deployment'
}
}
environment{
ENV_CONFIG_ID = 'jenkins-prod'
ENV_CONFIG_FILE = 'test.groovy'
ENV_PLAYBOOK_NAME = 'test.tar.gz'
}
parameters {
string (
defaultValue: 'test.x86_64',
description: 'Enter app version',
name: 'app_version'
)
choice (
choices: ['10.0.0.1','10.0.0.2','10.0.0.3'],
description: 'Select a host to be delpoyed',
name: 'host'
)
}
stages {
stage("reading properties from properties file") {
steps {
// Use a script block to do custom scripting
script {
def props = readProperties file: 'extravars.properties'
env.var1 = props.var1
env.var2 = props.var2
}
echo "The variable 1 value is $var1"
echo "The variable 2 value is $var2"
}
In above code,i used pipeline utility steps plugin and able to read variables from extravars.properties file. Is it same way we can do for jenkins parameters also? Or do we have any suitable method to take care of passing this parameters via a file from git repo?
Also is it possible to pass variable for node label also?
=====================================================================
Below are the improvements which i have made in this project
Used node label plugin to pass the node name as variable
Below is my vars/sayHello.groovy file content
def call(body) {
// evaluate the body block, and collect configuration into the object
def pipelineParams= [:]
body.resolveStrategy = Closure.DELEGATE_FIRST
body.delegate = pipelineParams
body()
pipeline {
agent
{
node
{
label "${pipelineParams.slaveName}"
}
}
stages {
stage("reading properties from properties file") {
steps {
// Use a script block to do custom scripting
script {
// def props = readProperties file: 'extravars.properties'
// script {
readProperties(file: 'extravars.properties').each {key, value -> env[key] = value }
//}
// env.var1 = props.var1
// env.var2 = props.var2
}
echo "The variable 1 value is $var1"
echo "The variable 2 value is $var2"
}
}
stage ('stage2') {
steps {
sh "echo ${var1}"
sh "echo ${var2}"
sh "echo ${pipelineParams.appVersion}"
sh "echo ${pipelineParams.hostIp}"
}
}
}
}
}
Below is my vars/params.groovy file
properties( [
parameters([
choice(choices: ['10.80.66.171','10.80.67.6','10.80.67.200'], description: 'Select a host to be delpoyed', name: 'host')
,string(defaultValue: 'fxxxxx.x86_64', description: 'Enter app version', name: 'app_version')
])
] )
Below is my jenkinsfile
def _hostIp = params.host
def _appVersion = params.app_version
sayHello {
slaveName = 'master'
hostIp = _hostIp
appVersion = _appVersion
}
Now Is it till we can improve this?Any suggestions let me know.
I have the following (simplified) Jenkins pipeline code.
jobParams.groovy
List get(Object paramVars = {}) {
def params = []
params += [
choice(
choices: ['branch', 'tag'],
name: 'RELEASE_TYPE'
),
string(
defaultValue: '',
name: 'VERSION'
),
]
return params
}
pipeline.groovy
def call() {
properties([
parameters(
jobParams.get()
)
])
pipeline {
agent { label 'deploy-slave' }
stages {
stage('Prepare') {
steps {
script {
// Do some logic here and set a job parameter?
}
}
}
}
}
}
This works fine. When the pipeline starts the job parameters are set and available for the next time the job runs.
However, is it also possible to set job parameters dynamically after some logic in a pipeline step?
It turned out to be pretty easy!
I created a jobProperties.groovy file in my shared pipeline library, which composes the parameter list and calls the properties() function.
def call() {
params = [
string(
defaultValue: '',
description: 'Version to deploy',
name: 'VERSION'
),
]
if (env.HANDLER == 'ansible') {
params += [
string(
defaultValue: '',
description: 'DEPLOY_ARGS | Ad hoc "ansible-playbook" args. Example to limit hosts to' +
' deploy to "-l somehost"',
name: 'DEPLOY_ARGS'
),
]
} else if (env.HANDLER == 'capistrano') {
params += [
string(
defaultValue: '',
description: 'DEPLOY_ARGS | Ad hoc "cap" args. Example to limit hosts to' +
' deploy to "-z somehost"',
name: 'DEPLOY_ARGS'
),
]
}
properties([
parameters(
params
)
])
}
pipeline.groovy
def call() {
pipeline {
agent { label 'deploy-slave' }
stages {
stage('Prepare') {
steps {
script {
jobProperties()
}
}
}
}
}
}
I think that if you don't have a shared pipeline library, the code of jobParams.groovy can be also put directly in the script {} wrapper of the pipeline.
It is, but there are some complications as params is an immutable map.
We use a shared library function we wrote when we want to change our params during job execution.
This will probably require admin for script approvals.
The first function is for setting a new string param, or updating an existing one with a new value.
The second and third jobs are just interfaces for adding a new option to a choice param for either your current job, or a different job.
The fourth is the main grunt for this choice adding logic. (not called directly)
The organizationFolder is based on us using the Github branch source plugin.
/**
* Change param value during build
*
* #param paramName new or existing param name
* #param paramValue param value
* #return nothing
*/
def setParam(String paramName, String paramValue) {
List<ParameterValue> newParams = new ArrayList<>();
newParams.add(new StringParameterValue(paramName, paramValue))
try {
$build().addOrReplaceAction($build().getAction(ParametersAction.class).createUpdated(newParams))
} catch (err) {
$build().addOrReplaceAction(new ParametersAction(newParams))
}
}
/**
* Add a new option to choice parameter for the current job
*
* #param paramName parameter name
* #param optionValue option value
* #return nothing
*/
def addChoice(String paramName, String optionValue) {
addChoice($build().getParent(), paramName, optionValue)
}
/**
* Add a new option to choice parameter to the given job
*
* #param paramName parameter name
* #param optionValue option value
* #return nothing
*/
def addChoice(String jobName, String paramName, String optionValue) {
List jobNames = jobName.tokenize("/")
Job job = ((OrganizationFolder)Jenkins.getInstance().getItem(jobNames[0])).getItem(jobNames[1]).getItem(jobNames[2])
addChoice(job, paramName, optionValue)
}
/**
* Add a new option to choice parameter to the given job
* Will be added as the first (default) choice
* #param job job object
* #param paramName parameter name
* #param optionValue option value
* #return
*/
def addChoice(Job job, String paramName, String optionValue) {
ParametersDefinitionProperty paramsJobProperty = job.getProperty(ParametersDefinitionProperty.class);
ChoiceParameterDefinition oldChoiceParam = (ChoiceParameterDefinition)paramsJobProperty.getParameterDefinition(paramName);
List<ParameterDefinition> oldJobParams = paramsJobProperty.getParameterDefinitions();
List<ParameterDefinition> newJobParams = new ArrayList<>();
for (ParameterDefinition p: oldJobParams) {
if (!p.getName().equals(paramName)) {
newJobParams.add(0,p);
}
}
List<String> choices = new ArrayList(oldChoiceParam.getChoices());
choices.add(0,optionValue);
ChoiceParameterDefinition newChoiceParam = new ChoiceParameterDefinition(paramName, choices, oldChoiceParam.getDefaultParameterValue().getValue(), oldChoiceParam.getDescription());
newJobParams.add(newChoiceParam);
ParametersDefinitionProperty newParamsJobProperty = new ParametersDefinitionProperty(newJobParams);
job.removeProperty(paramsJobProperty);
job.addProperty(newParamsJobProperty);
}
As mentioned the params map is immutable, however, as described here Jenkins also creates an environment variable for each build param
So the option I've used to override a build param is to use the environment var rather than the value in the params map eg:
if (environment == "PRD") {
env.vpc_id = 'vpc-0fc6d952bbbf0000'
}
// Now run your script or command that refers to the environment var
sh './script.sh'
I Guess you are talking about deciding downstream parameters dynamically for a job. It surely can be done the way its depicted in the code below.
#Library("shared-library") _
Map buildDetails = [:]
downStreamParams = [["\$class: 'StringParameterValue', name: 'TARGET_WORKSPACE', value: 'prod'"]]
pipeline {
agent {
label 'builds'
}
stages {
stage('Get Details'){
steps {
script {
buildDetails = [
"releaseType":"minor",
"workspace":"prod",
"featureType":"ENHANCEMENT",
"PARAMS=Jenkins_Controller_Image":true,
"PARAMS=APPLY":true,
"PARAMS=PLUGIN_CLEANUP":true,
"PARAMS=RESTART":true,
]
buildDetails.each{ key, value ->
println("$key:$value")
if(key.contains("PARAMS=")){
"[\$class: 'BooleanParameterValue', name: \"${key.split('=')[1]}\", value: true]"
}
}
}
}
}
stage("Build"){
steps{
script{
job = "build( job: jobFullName,parameters: ${downStreamParams},propagate: true,wait: true)"
evaluate(job)
}
}
}
}
}
Using Jenkins pipeline we have our own build script. Also all of our projects have a rakefile which is what we use to do a lot of the building steps. Our typical jenkins build executes 3 rake tasks but we do have some exceptions and that has to do when we have a angular website we try to build with it.
I've configured my pipeline like this:
buildGitProject {
repository='https://anonymous.visualstudio.com/Project/_git/my-csharp-project-with-angular'
branchName= 'master'
solutionName='MyCSharpSolution.sln'
emailTo='someone#aol.com'
preRakeCommands=['install_npm_dependencies', 'ng_build']
}
that relies on our build script which is this:
def call(body) {
def args= [:]
body.resolveStrategy = Closure.DELEGATE_FIRST
body.delegate = args
body()
def agentName = "windows && ${args.branchName}"
def remoteConfig = org.pg.RemoteConfigFactory.create(args.repository);
pipeline {
agent none
options {
buildDiscarder(logRotator(numToKeepStr: org.pg.Settings.BUILDS_TO_KEEP))
skipStagesAfterUnstable()
timestamps()
}
stages {
stage("checkout") {
agent any
steps {
checkoutFromGit(remoteConfig, args.branchName)
}
}
stage('build') {
agent{node{ label agentName as String}}
steps {
buildSolution(args.solutionName, args.get('preRakeCommands', []), args.get('postRakeCommands', []))
}
}
stage('test') {
agent{node{ label agentName as String}}
steps {
testSolution(args.solutionName)
}
}
}
}
}
which fails in the build stage.
buildSolution.groovy
def call(String solutionName, ArrayList preRakeCommands, ArrayList postRakeCommands) {
unstash 'ws'
String[] rakeCommands = [
"build_solution[${solutionName}, Release, Any CPU]",
"copy_to_deployment_folder",
"execute_dev_dropkick"
]
String[] combinedRakeCommand = (preRakeCommands.plus(rakeCommands).plus(postRakeCommands)) as String[]
executeRake( combinedRakeCommand )
stash name: 'deployment', includes: 'deployment/**/*'
}
executeRake.groovy
def call(String... rakeTasks) {
def safeRakeTasks = rakeTasks.collect{ "\"$it\"" }.join(' ');
bat script: "rake ${safeRakeTasks}"
}
in the jenkins build log it says:
08:43:09 C:\jenkins_repos\Project\my-csharp-project-with-angular>rake "install_npm_dependencies" "ng_build" "[Ljava.lang.String;#11bd466"
I have no idea how or why it is using a string pointer because I thought that plus concated arrays and ArrayList... Plus it is in Jenkins so it is a pain to test.
List a = ['a1','a2','a3']
String [] s = ['s1','s2','s3']
List b = ['b1','b2','b3']
println a.plus(s as List).plus(b)
output:
[a1, a2, a3, s1, s2, s3, b1, b2, b3]
Another approach:
List a = ['a1','a2','a3']
String[] s = ['s1','s2','s3']
List b = ['b1','b2','b3']
println ([*a,*s,*b])
alternatively
println a + [*s] + b
which should perform better
Scenario: I want to trigger few down stream jobs(Job A and Job B ....) dynamically based on the input parameter received by the current job.
import hudson.model.*
def values = ${configname}.split(',')
def currentBuild = Thread.currentThread().executable
println ${configname}
println ${sourceBranch}
values.eachWithIndex { item, index ->
println item
println index
def job = hudson.model.Hudson.instance.getJob(item)
def params = new StringParameterValue('upstream_job', ${sourceBranch})
def paramsAction = new ParametersAction(params)
def cause = new hudson.model.Cause.UpstreamCause(currentBuild)
def causeAction = new hudson.model.CauseAction(cause)
hudson.model.Hudson.instance.queue.schedule(job, 0, causeAction, paramsAction)
}
How about something like this? I was getting a comma separated list from the upstream system and I splitted them as individaul string which is internally jobs. Making a call by passing each individual strings.
this Jenkinsfile would do that:
#!/usr/bin/env groovy
pipeline {
agent { label 'docker' }
parameters {
string(name: 'myHotParam', defaultValue: '', description: 'What is your param, sir?')
}
stages {
stage('build') {
steps {
script {
if (params.myHotParam == 'buildEverything') {
build 'mydir/jobA'
build 'mydir/jobB'
}
}
}
}
}
}
How do I pass variables between stages in a declarative pipeline?
In a scripted pipeline, I gather the procedure is to write to a temporary file, then read the file into a variable.
How do I do this in a declarative pipeline?
E.g. I want to trigger a build of a different job, based on a variable created by a shell action.
stage("stage 1") {
steps {
sh "do_something > var.txt"
// I want to get var.txt into VAR
}
}
stage("stage 2") {
steps {
build job: "job2", parameters[string(name: "var", value: "${VAR})]
}
}
If you want to use a file (since a script is the thing generating the value you need), you could use readFile as seen below. If not, use sh with the script option as seen below:
// Define a groovy local variable, myVar.
// A global variable without the def, like myVar = 'initial_value',
// was required for me in older versions of jenkins. Your mileage
// may vary. Defining the variable here maybe adds a bit of clarity,
// showing that it is intended to be used across multiple stages.
def myVar = 'initial_value'
pipeline {
agent { label 'docker' }
stages {
stage('one') {
steps {
echo "1.1. ${myVar}" // prints '1.1. initial_value'
sh 'echo hotness > myfile.txt'
script {
// OPTION 1: set variable by reading from file.
// FYI, trim removes leading and trailing whitespace from the string
myVar = readFile('myfile.txt').trim()
}
echo "1.2. ${myVar}" // prints '1.2. hotness'
}
}
stage('two') {
steps {
echo "2.1 ${myVar}" // prints '2.1. hotness'
sh "echo 2.2. sh ${myVar}, Sergio" // prints '2.2. sh hotness, Sergio'
}
}
// this stage is skipped due to the when expression, so nothing is printed
stage('three') {
when {
expression { myVar != 'hotness' }
}
steps {
echo "three: ${myVar}"
}
}
}
}
Simply:
pipeline {
parameters {
string(name: 'custom_var', defaultValue: '')
}
stage("make param global") {
steps {
tmp_param = sh (script: 'most amazing shell command', returnStdout: true).trim()
env.custom_var = tmp_param
}
}
stage("test if param was saved") {
steps {
echo "${env.custom_var}"
}
}
}
I had a similar problem as I wanted one specific pipeline to provide variables and many other ones using it to get this variables.
I created a my-set-env-variables pipeline
script
{
env.my_dev_version = "0.0.4-SNAPSHOT"
env.my_qa_version = "0.0.4-SNAPSHOT"
env.my_pp_version = "0.0.2"
env.my_prd_version = "0.0.2"
echo " My versions [DEV:${env.my_dev_version}] [QA:${env.my_qa_version}] [PP:${env.my_pp_version}] [PRD:${env.my_prd_version}]"
}
I can reuse these variables in a another pipeline my-set-env-variables-test
script
{
env.dev_version = "NOT DEFINED DEV"
env.qa_version = "NOT DEFINED QA"
env.pp_version = "NOT DEFINED PP"
env.prd_version = "NOT DEFINED PRD"
}
stage('inject variables') {
echo "PRE DEV version = ${env.dev_version}"
script
{
def variables = build job: 'my-set-env-variables'
def vars = variables.getBuildVariables()
//println "found variables" + vars
env.dev_version = vars.my_dev_version
env.qa_version = vars.my_qa_version
env.pp_version = vars.my_pp_version
env.prd_version = vars.my_prd_version
}
}
stage('next job') {
echo "NEXT JOB DEV version = ${env.dev_version}"
echo "NEXT JOB QA version = ${env.qa_version}"
echo "NEXT JOB PP version = ${env.pp_version}"
echo "NEXT JOB PRD version = ${env.prd_version}"
}
there is no need for (hidden plugin) parameter definitions or temp-file access. Sharing varibles across stages can be acomplished by using global Groovy variables in a Jenkinsfile like so:
#!/usr/bin/env groovy
def MYVAR
def outputOf(cmd) { return sh(returnStdout:true,script:cmd).trim(); }
pipeline {
agent any
stage("stage 1") {
steps {
MYVAR = outputOf('echo do_something')
sh "echo MYVAR has been set to: '${MYVAR}'"
}
}
stage("stage 2") {
steps {
sh '''echo "...in multiline quotes: "''' + MYVAR + '''" ... '''
build job: "job2", parameters[string(name: "var", value: MYVAR)]
}
}
}
I have enhanced the existing solution by correcting syntax .Also used hidden parameter plugin so that it does not show up as an extra parameter in Jenkins UI. Works well :)
properties([parameters([[$class: 'WHideParameterDefinition', defaultValue: 'yoyo', name: 'hidden_var']])])
pipeline {
agent any
stages{
stage("make param global") {
steps {
script{
env.hidden_var = "Hello"
}
}
}
stage("test if param was saved") {
steps {
echo"About to check result"
echo "${env.hidden_var}"
}
}
}
}