I have a Jenkinsfile that takes a bunch of params ( 50 aprox.), and other 50 for input processing:
pipeline {
agent { label 'ansible24' }
parameters {
string(name: 'NAME', defaultValue: 'Nightly Valid', description: ' instance name')
// ... x50
}
script {
def filename = "configuration.yml"
def yaml = readYaml file: filename
yaml.global.name = "${params.NAME}".toString()
// ... x50
}
Tomorrow, I will also have a validation for each field.
How could I extract this logic in separated files?
I already saw this: How do you load a groovy file and execute it
but it doesn't help a lot for the case of params and my case is not scripted.
Any idea ?
Related
I am creating jobs for use with Terraform. There are several environments and the number is growing all the time. Rather than update both the pipeline file and the jobdsl file as the parameters change, I started working from the standpoint of scanning the repo for environment files and updating the pipeline and jobdsl file as needed.
My jobdsl script:
#Library('mylib') _
params = [
"serviceName": "infrastructure-${repo}",
"repoUrl": "${repoUrl}",
"sshCredentials": 'git-readonly',
"environment": "${env.Environment}",
"configParams": getTFConfigs(
repoUrl,
"env/${env.AccountName}/${env.AWSRegion}/${env.Environment}")
]
template = libraryResource('dslTemplates/infra.groovy')
jobDsl scriptText: helpers.renderTemplate(template, params)
shared library method: getTFConfigs
#!/usr/bin/env groovy
#NonCPS
import java.util.zip.ZipEntry
import java.util.zip.ZipInputStream
def call(String repoUrl, String filter=""){
def gitProc = new ProcessBuilder(
"git",
"archive",
"--format=zip",
"--remote=${repoUrl}",
"main").start()
def zipIn = new ZipInputStream(gitProc.inputStream)
def zipMembers = []
while (true) {
def ZipEntry entry = zipIn.getNextEntry()
if (entry == null) break
if ( (entry.getName()).contains(filter) ) {
entryName = entry.getName()
zipMembers.push("${entryName}")
}
}
println zipMembers
return zipMembers
}
dslTemplates/infra.groovy template
pipelineJob("${serviceName}") {
description("Apply TF for ${serviceName} to all environment configurations")
definition {
parameters {
<% configParams.each { %>
booleanParam(name: "<%= "${it}" %>", defaultValue: true, description: "<%= "${it}" %>" )
<% } %>
}
logRotator {
numToKeep(20)
}
cpsScm {
scm {
git{
remote{
url("${repoUrl}")
credentials("${sshCredentials}")
branch('*/main')
}
}
}
scriptPath('infra.groovy')
}
}
}
Template result
...
definition {
parameters {
booleanParam(name: env1.tfvars, defaultValue: true, description: env1.tfvars )
booleanParam(name: env2.tfvars, defaultValue: true, description: env2.tfvars )
}
...
When the seed job runs and executes the code, the parameters should be updated with a checkbox for each environment. However, the jobdsl fails with this:
ERROR: (script, line 6) No signature of method: javaposse.jobdsl.dsl.helpers.BuildParametersContext.booleanParam() is applicable for argument types: (java.util.LinkedHashMap) values: [[name:env1.tfvars, defaultValue:true, ...]]
Possible solutions: booleanParam(java.lang.String), booleanParam(java.lang.String, boolean), booleanParam(java.lang.String, boolean, java.lang.String)
Finished: FAILURE
I have tried to applying "toString()" at various steps and cannot seem to find any solution to this.
I have tried to write the entire jobdsl script to a file and read it back in using "jobDsl targets: filename" and got the same result!
Banging my head! as it were!
Thanks
It looks like you used Pipeline Syntax for the parameters in DSL script. If you want to define a parameter in a DSL script do not use name, defaultValue and description. (See Job DSL Plugin)
booleanParam('BOOL_PARAM', true, 'This is a boolean param')
I'm trying to convert my jenkins pipeline to a shared library since it can be reusable on most of the application. As part of that i have created groovy file in vars folder and kept pipeline in jenkins file in github and able to call that in jenkins successfully
As part of improving this i want to pass params, variables, node labels through a file so that we should not touch jenkins pipeline and if we want to modify any vars, params, we have to do that in git repo itself
pipeline {
agent
{
node
{
label 'jks_deployment'
}
}
environment{
ENV_CONFIG_ID = 'jenkins-prod'
ENV_CONFIG_FILE = 'test.groovy'
ENV_PLAYBOOK_NAME = 'test.tar.gz'
}
parameters {
string (
defaultValue: 'test.x86_64',
description: 'Enter app version',
name: 'app_version'
)
choice (
choices: ['10.0.0.1','10.0.0.2','10.0.0.3'],
description: 'Select a host to be delpoyed',
name: 'host'
)
}
stages {
stage("reading properties from properties file") {
steps {
// Use a script block to do custom scripting
script {
def props = readProperties file: 'extravars.properties'
env.var1 = props.var1
env.var2 = props.var2
}
echo "The variable 1 value is $var1"
echo "The variable 2 value is $var2"
}
In above code,i used pipeline utility steps plugin and able to read variables from extravars.properties file. Is it same way we can do for jenkins parameters also? Or do we have any suitable method to take care of passing this parameters via a file from git repo?
Also is it possible to pass variable for node label also?
=====================================================================
Below are the improvements which i have made in this project
Used node label plugin to pass the node name as variable
Below is my vars/sayHello.groovy file content
def call(body) {
// evaluate the body block, and collect configuration into the object
def pipelineParams= [:]
body.resolveStrategy = Closure.DELEGATE_FIRST
body.delegate = pipelineParams
body()
pipeline {
agent
{
node
{
label "${pipelineParams.slaveName}"
}
}
stages {
stage("reading properties from properties file") {
steps {
// Use a script block to do custom scripting
script {
// def props = readProperties file: 'extravars.properties'
// script {
readProperties(file: 'extravars.properties').each {key, value -> env[key] = value }
//}
// env.var1 = props.var1
// env.var2 = props.var2
}
echo "The variable 1 value is $var1"
echo "The variable 2 value is $var2"
}
}
stage ('stage2') {
steps {
sh "echo ${var1}"
sh "echo ${var2}"
sh "echo ${pipelineParams.appVersion}"
sh "echo ${pipelineParams.hostIp}"
}
}
}
}
}
Below is my vars/params.groovy file
properties( [
parameters([
choice(choices: ['10.80.66.171','10.80.67.6','10.80.67.200'], description: 'Select a host to be delpoyed', name: 'host')
,string(defaultValue: 'fxxxxx.x86_64', description: 'Enter app version', name: 'app_version')
])
] )
Below is my jenkinsfile
def _hostIp = params.host
def _appVersion = params.app_version
sayHello {
slaveName = 'master'
hostIp = _hostIp
appVersion = _appVersion
}
Now Is it till we can improve this?Any suggestions let me know.
I want to have a check in jenkins job, where if the jenkins parameters have been modified then I want to display an Input modal to proceed or abort the job .But for that I need to verify if the parameters have been modified.
So, I can get modified parameters value in jenkins job, but how to retrieve default values of those parameters so that I can verify if any parameters have been modified?
def DEFAULT_VALUE = "42"
pipeline {
agent any
parameters { string(name: 'MY_PARAM', defaultValue: DEFAULT_VALUE, description: '') }
stages {
stage('Example') {
steps {
script {
if (params.MY_PARAM == DEFAULT_VALUE) {
echo 'Default value used'
} else {
echo 'Non-default value used'
}
}
}
}
}
We've got a set of groovy scripts that our users invoke in their jenkinsfile that sets some common job properties. However, we haven't been able to figure out how to preserve their existing parameters when we do this update.
snippet of our groovy code:
def newParamsList = []
def newbool = booleanParam(defaultValue: false, description: "deploy", name: "deploy_flag")
newParamsList.add(newbool)
def newParams = parameters(newParamsList)
properties([ //job property declaration
jobProperties,
disableConcurrentBuilds(),
newParams,
addSchedule,
])
However, this overwrites the parameter definitions, so if the user had specified a different parameter definition in their jenkins file before invoking our groovy, it's been wiped out.
I can get access to the existing parameters using currentBuild.rawBuild.getAction(ParametersAction), but if I understand correctly, I need the ParameterDefinition not the ParameterValue in order to set the property. I tried currentBuild.rawBuild.getAction(ParametersDefinitionProperty.class) thinking I could use that like ParametersAction, but it returns null.
Is it possible to get the parameter definitions inside the groovy being called from a Jenkinsfile? Or is there a different way that would let us add an additional parameter to the job without wiping out the existing ones currently defined in the jenkinsfile?
So the way we do this, is treat it all like a simple list, then join them together. So jenkinsfile's first get a list from the shared library, before adding their own to the list and then they set the params (not the shared library)
Repos jenkinsfiles do this:
#!groovy
#Library('shared') _
// Call shared libaray for common params
def paramList = jobParams.listParams ([
"var1": "value",
"var2": "value2"
])
// Define repo specific params
def addtionalParams = [
booleanParam(defaultValue: false, name: 'SOMETHING', description: 'description?'),
booleanParam(defaultValue: false, name: 'SOMETHING_ELSE', description: 'description?'),
]
// Set Jenkins job properties, combining both
properties([
buildDiscarder(logRotator(numToKeepStr: '20')),
parameters(paramList + addtionalParams)
])
// Do repo stuff
Our shared library looks like this:
List listParams(def body = [:]) {
//return list of parameters
config = BuildConfig.resolve(body)
// Always common params
def paramsList = [
choice(name: 'ENV', choices: ['dev', 'tst'].join('\n'), description: 'Environment'),
string(name: 'ENV_NO', defaultValue: "1", description: 'Environment number'),
]
// Sometimes common params, switch based on jenkinsfile input
def addtionalParams = []
switch (config.var1) {
case 'something':
case 'something2':
addtionalParams = [
choice(name: 'AWS_REGION', choices: ['us-west-2'].join('\n'), description: 'AWS Region to build/deploy'),
]
break
case 'something3':
addtionalParams = [
string(name: 'DEBUG', defaultValue: '*', description: 'Namespaces for debug logging'),
]
break
}
return paramsList + addtionalParams
}
We did the following groovy code to retrieve the parameters definitions and add new parameters to existing ones (we don't have any knowledge about what the user will put as parameters). If you have something more simple, I take it:
boolean isSupported = true
// nParamsis the List of new parameters to add //
Map initParamsMap = this.initializeParamsMap(nParams)
currentBuild.rawBuild.getParent().getProperties().each { k, v ->
if (v instanceof hudson.model.ParametersDefinitionProperty) {
// get each parameter definition
v.parameterDefinitions.each { ParameterDefinition paramDef ->
String param_symbol_name = null
// get the symbol name from the nested DescriptorImpl class
paramDef.class.getDeclaredClasses().each {
if(it.name.contains('DescriptorImpl')){
param_symbol_name = it.getAnnotation(Symbol).value().first()
}
}
// ... processing... //
if( !initParamsMap.containsKey(paramDef.name) ) {
//Valid parameter types are booleanParam, choice, file, text, password, run, or string.
if (param_symbol_name == 'choice') {
String defaultParamVal = paramDef.defaultParameterValue == null ? null : paramDef.defaultParameterValue.value
tempParams.add(
"$param_symbol_name"(name: paramDef.name,
defaultValue: defaultParamVal,
description: paramDef.description,
choices: paramDef.choices)
)
} else if (param_symbol_name == 'run') {
logError {"buildParametersArray does not support yet already existing RunParameterDefinition " +
"in current job parameters list, so the job parameters will not be modified"}
isSupported = false
} else {
tempParams.add(
"$param_symbol_name"(name: paramDef.name,
defaultValue: paramDef.defaultParameterValue.value,
description: paramDef.description)
)
}
}
}
}
}
if( isSupported) {
properties([parameters(tempParams)])
}
I think you can also do something like this:
// Get existing ParameterDefinitions
existing = currentBuild.rawBuild.parent.properties
.findAll { it.value instanceof hudson.model.ParametersDefinitionProperty }
.collectMany { it.value.parameterDefinitions }
// Create new params and merge them with existing ones
jobParams = [
booleanParam(name: 'boolean_param', defaultValue: false)
/* other params */
] + existing
// Create properties
properties([
parameters(jobParams)
])
Note: But you should either run it in a non-sandboxed environment or use with #NonCPS
There is an example how to add additional string parameter NEW_PARAM into job with name test:
job = Jenkins.instance.getJob("test")
ParametersDefinitionProperty params = job.getProperty(ParametersDefinitionProperty.class);
List<ParameterDefinition> newParams = new ArrayList<>();
newParams.addAll(params.getParameterDefinitions());
newParams.add(new StringParameterDefinition("NEW_PARAM", "default_value"));
job.removeProperty(params);
job.addProperty(new ParametersDefinitionProperty(newParams));
Scenario: I want to trigger few down stream jobs(Job A and Job B ....) dynamically based on the input parameter received by the current job.
import hudson.model.*
def values = ${configname}.split(',')
def currentBuild = Thread.currentThread().executable
println ${configname}
println ${sourceBranch}
values.eachWithIndex { item, index ->
println item
println index
def job = hudson.model.Hudson.instance.getJob(item)
def params = new StringParameterValue('upstream_job', ${sourceBranch})
def paramsAction = new ParametersAction(params)
def cause = new hudson.model.Cause.UpstreamCause(currentBuild)
def causeAction = new hudson.model.CauseAction(cause)
hudson.model.Hudson.instance.queue.schedule(job, 0, causeAction, paramsAction)
}
How about something like this? I was getting a comma separated list from the upstream system and I splitted them as individaul string which is internally jobs. Making a call by passing each individual strings.
this Jenkinsfile would do that:
#!/usr/bin/env groovy
pipeline {
agent { label 'docker' }
parameters {
string(name: 'myHotParam', defaultValue: '', description: 'What is your param, sir?')
}
stages {
stage('build') {
steps {
script {
if (params.myHotParam == 'buildEverything') {
build 'mydir/jobA'
build 'mydir/jobB'
}
}
}
}
}
}