I am trying to do the below things as part of Jenkins pipeline dsl.
I have a yaml file where i store all my static values.
I created a pipeline job which should show 2 parameters.
a)region : northamerica/europe
b)environment : this should come based on the region selected.
I am defining the 2 functions outside of the pipeline so that i can use them during the parameters section.
Syntax:
#!/usr/bin/env groovy
def yaml_file = "JenkinsFiles/environments.yaml"
def getRegions() {
def var_regions = []
yaml_settings.environments.each { key, value -> var_regions.add(key) }
return var_regions
}
def getEnvironments(String region) {
def var_envs = []
yaml_settings.environments."${region}".non_prod.each { key, value -> var_envs.add("\"" + key + "\"") }
return var_envs
}
environment {
yaml_settings = {}
}
pipeline {
agent
{
node
{
label 'docker'
}
}
stages {
stage('Prepare') {
steps{
script{
yaml_settings = readYaml file: "${yaml_file}"
list_regions = getRegions()
properties([
parameters([
choice(choices: list_regions , description: 'Please select region to deploy', name: 'REGION'),
[$class: 'CascadeChoiceParameter', choiceType: 'PT_SINGLE_SELECT', description: 'Please select environment to deploy', filterLength: 1, filterable: false, name: 'ACP_ENVIRONMENTS', randomName: 'choice-parameter-deploy-env', referencedParameters: 'REGION', script: [$class: 'GroovyScript', fallbackScript: [classpath: [], sandbox: false, script: ''], script: [classpath: [], sandbox: true, script: """
envs = getEnvironments($REGION)
return $envs
"""]]]])])}}}}}
Issue:
The getEnvironments method is not returning the value into the variable and its not getting effected in the parameter. But $region value is coming into though.I can do if else based on the reference parameter and get the value but i dont want to use if else coz i will get many values down the line.
HELP APPRECIATED!!
As with many other questions, the issue here is that Jenkins needs to know the parameters before it executes your pipeline. Once the pipeline is running, the parameters have already been defined, and any change to the parameters won't impact this build.
You may want to take a look at the ActiveChoice plugin to address this.
Related
I am having a somewhat difficult time figuring out how to make a simple Jenkins pipeline to print values from a simple map.
I use extendedChoice plugin.
The requirement is the following:
The user has a dropdown selection of names, once selected a name, job will simply print (in log) its value (.key).
this is the code I am trying to work with, made numerous changes and still get various errors and nothing works.
if anyone has any idea, will be glad to hear about it :D
def data = ["john": "33", "alex": "45", "michael": "22"]
properties([
parameters ([
extendedChoice(
name: 'CHOICE',
description: 'name and age selection',
type: 'PT_SINGLE_SELECT',
value: data.key // i think i am writing this wrong.. i need to see names in selection dropdown box
)
])
])
pipeline {
agent any
stages {
stage('print choice') {
steps {
println params.CHOICE.value // how to print .value for user i selected?
}
}
}
}
Here is a working Pipeline for your example.
def data = ["john": "33", "alex": "45", "michael": "22"]
properties([
parameters ([
extendedChoice(
name: 'CHOICE',
description: 'name and age selection',
type: 'PT_SINGLE_SELECT',
value: "${data.keySet().join(',').toString()}"
)
])
])
pipeline {
agent any
stages {
stage('print choice') {
steps {
println params.CHOICE
println data.get(params.CHOICE)
}
}
}
}
Using the declarative pipeline syntax, I want to be able to define parameters based on an array of repos, so that when starting the build, the user can check/uncheck the repos that should not be included when the job runs.
final String[] repos = [
'one',
'two',
'three',
]
pipeline {
parameters {
booleanParam(name: ...) // static param
// now include a booleanParam for each item in the `repos` array
// like this but it's not allowed
script {
repos.each {
booleanParam(name: it, defaultValue: true, description: "Include the ${it} repo in the release?")
}
}
}
// later on, I'll loop through each repo and do stuff only if its value in `params` is `true`
}
Of course, you can't have a script within the parameters block, so this won't work. How can I achieve this?
Using the Active Choices Parameter plugin is probably the best choice, but if for some reason you can't (or don't want to) use a plugin, you can still achieve dynamic parameters in a Declarative Pipeline.
Here is a sample Jenkinsfile:
def list_wrap() {
sh(script: 'echo choice1 choice2 choice3 choice4 | sed -e "s/ /\\n/g"', , returnStdout: true)
}
pipeline {
agent any
stages {
stage ('Gather Parameters') {
steps {
timeout(time: 30, unit: 'SECONDS') {
script {
properties([
parameters([
choice(
description: 'List of arguments',
name: 'service_name',
choices: 'DEFAULT\n' + list_wrap()
),
booleanParam(
defaultValue: false,
description: 'Whether we should apply changes',
name: 'apply'
)
])
])
}
}
}
}
stage ('Run command') {
when { expression { params.apply == true } }
steps {
sh """
echo choice: ${params.service_name} ;
"""
}
}
}
}
This embeds a script {} in a stage, which calls a function, which runs a shell script on the agent/node of the Declarative Pipeline, and uses the script's output to set the choices for the parameters. The parameters are then available in the next stages.
The gotcha is that you must first run the job with no build parameters in order for Jenkins to populate the parameters, so they're always going to be one run out of date. That's why the Active Choices Parameter plugin is probably a better idea.
You could also combine this with an input command to cause the pipeline to prompt the user for a parameter:
script {
def INPUT_PARAMS = input message: 'Please Provide Parameters', ok: 'Next',
parameters: [
choice(name: 'ENVIRONMENT', choices: ['dev','qa'].join('\n'), description: 'Please select the Environment'),
choice(name: 'IMAGE_TAG', choices: getDockerImages(), description: 'Available Docker Images')]
env.ENVIRONMENT = INPUT_PARAMS.ENVIRONMENT
env.IMAGE_TAG = INPUT_PARAMS.IMAGE_TAG
}
Credit goes to Alex Lashford (https://medium.com/disney-streaming/jenkins-pipeline-with-dynamic-user-input-9f340fb8d9e2) for this method.
You can use CHOICE parameter of Jenkins in which user can select a repository.
pipeline {
agent any
parameters
{
choice(name: "REPOS", choices: ['REPO1', 'REPO2', 'REPO3'])
}
stages {
stage ('stage 1') {
steps {
// the repository selected by user will be printed
println("$params.REPOS")
}
}
}
}
You can also use the plugin Active Choices Parameter if you want to do multiple select : https://plugins.jenkins.io/uno-choice/#documentation
You can visit pipeline syntax and configure in below way to generate code snippet and you can put it in the jenkins file:
Copy the snippet code and paste it in jenkinsfile at the start.
In my Jenkins declarative pipeline I need to grab values from a text file in order to populate my choice parameters.
I came with the following pipeline but parameter fails to display its choice values and show fallback "ERROR" instead.
I can't figure out why this fallback to fallbackScript as buiding the pipeline is a success.
Source text file
client1 dev
client1 prod
client1 test
client2 prod
client2 test
client3 test
client4 test
client4 dev
Pipeline script
#!/usr/bin/env groovy
// Source text file
datasource_filepath = '/path/to/my_text_file.txt'
File datasource_file = new File(datasource_filepath)
// Keep only first field and enclose all list elements with double quotes
List client_names = datasource_file
.readLines()
.collect {
it.split().head()
}
.collect {
it.replaceAll(it, "\"$it\"")
}
.sort()
// Function to generate groovy script populating choices into the parameter
String ChoicesScript(List choices){
return "return $choices"
}
// Generates Groovy script for parameter
String client_name_script = ChoicesScript(client_names)
// Debug output
println 'Client names (' + client_names.getClass() + '):\n\n' + client_names + '\n\n'
println 'Client names generated script(' + client_name_script.getClass() + '):\n\n' + client_name_script + '\n\n'
// Choice parameters settings
properties([
parameters([
[
$class: 'ChoiceParameter',
choiceType: 'PT_SINGLE_SELECT',
name: 'CLIENT_NAME',
script:
[
$class: 'GroovyScript',
fallbackScript: [
classpath: [],
sandbox: false,
script: 'return ["ERROR"]'
],
script: [
classpath: [],
sandbox: false,
script: client_name_script
]
]
]
])
])
pipeline {
agent any
stages {
stage('Build'){
steps {
echo 'Building...'
}
}
}
}
I have a Jenkins Pipeline contained in a script that dynamically builds dropdown menus for input parameters that get used later in subsequent steps. The first time I run the build it creates the dropdowns but doesn't display them to the user, it just continues on to the next step using the default inputs.
If I then run the build a second time it displays the dropdowns and waits for choice selection by the user before continuing on to the next step. How do I get the dropdowns to display and wait on the first run as well?
Here is a pic of the dropdowns the second time I run the build:
Here is my Pipeline script:
pipeline {
agent any
stage ('Build Parameters') {
steps {
script {
properties([
parameters([
choice(
choices: [ 'MyEsxiServer.esxi' ],
description: 'On which ESXi Server should this script run?',
name: 'esxiServer'
),
choice(
choices: [ 'dev','stage','prod' ],
description: 'In which environment should this script run?',
name: 'environment'
),
[$class: 'ChoiceParameter',
choiceType: 'PT_SINGLE_SELECT',
description: 'On which runner should this script run?',
filterLength: 1,
filterable: false,
name: 'runner',
randomName: 'choice-parameter-596645940283131',
script: [$class: 'GroovyScript',
script: [classpath: [], sandbox: false,
script:
'''// Build choices for runner drop down
import groovy.json.JsonSlurper
log = new File('/tmp/groovy.log')
// execute the OS cmd and return the results
def sout = new StringBuilder()
def serr = new StringBuilder() //standard out and error strings
//Assemble command
def cmd = "/home/jenkins/bpb/testautomation/ansible/playbooks/listVMs.sh"
log.append("\\ncmd: " + cmd + "\\n") //debug
//Execute OS command
def proc = cmd.execute()
proc.waitForProcessOutput(sout, serr) //Wait for command to complete
proc.waitForOrKill(10000) //Set timeout
log.append("sout: " + sout + "\\n") //debug
log.append("serr: " + serr + "\\n") //debug
// translate JSON to List
def soutList = new JsonSlurper().parseText(sout.toString())
log.append("soutList: " + soutList + "\\n") //debug
def List vmList = soutList.vms.sort()
log.append("vmList: " + vmList + "\\n") //debug
return vmList
'''
]
]
],
choice(
choices: [ 'CentOS51','Comodore64','BeOS' ],
description: 'What is the target Operating System?',
name: 'targetOS'
)
]) //parameters
]) //properties
} //script
} //steps
} //stage
stage ('Execute Ansible') {
steps {
ansiblePlaybook credentialsId: 'b7a24821-8dc3-40d0-8cee-ef284e07393a',
disableHostKeyChecking: true,
colorized: true,
installation: 'Ansible',
inventory: "testautomation/ansible/${params.environment}/${params.environment}.inv",
playbook: 'testautomation/ansible/playbooks/run_cmd_inside_vm.yml',
extras: "-v -e runner=${params.runner} -e shell_cmd=/home/kcason/Desktop/${params.targetOS}/3-run.sh"
} //steps
} //stage
} // stages
} // pipeline
Not at all sadly. You can start a job with or without parameters (depending on how it is configured). The properties call changes this configuration but only after the job is started (without parameters/ with the old parameters). Jenkins cannot do that before since the properties call could be anywhere in the script and could depend on steps before it.
This article https://dev.to/pencillr/jenkins-pipelines-and-their-dirty-secrets-1 discusses this topic in more detail.
Also see the open issue in the jenkins issue tracker https://issues.jenkins-ci.org/plugins/servlet/mobile#issue/JENKINS-41929
pipeline {
agent any
stages {
stage("foo") {
steps {
script {
env.RELEASE_SCOPE = input message: 'User input required', ok: 'Release!',
parameters: [choice(name: 'RELEASE_SCOPE', choices: 'patch\nminor\nmajor',
description: 'What is the release scope?')]
}
echo "${env.RELEASE_SCOPE}"
}
}
}
}
In this above code, The choice are hardcoded (patch\nminor\nmajor) -- My requirement is to dynamically give choice values in the dropdown.
I get the values from calling api - Artifacts list (.zip) file names from artifactory
In the above example, It request input when we do the build, But i want to do a "Build with parameters"
Please suggest/help on this.
Depends how you get data from API there will be different options for it, for example let's imagine that you get data as a List of Strings (let's call it releaseScope), in that case your code be following:
...
script {
def releaseScopeChoices = ''
releaseScope.each {
releaseScopeChoices += it + '\n'
}
parameters: [choice(name: 'RELEASE_SCOPE', choices: ${releaseScopeChoices}, description: 'What is the release scope?')]
}
...
hope it will help.
This is a cutdown version of what we use. We separate stuff into shared libraries but I have consolidated a bit to make it easier.
Jenkinsfile looks something like this:
#!groovy
#Library('shared') _
def imageList = pipelineChoices.artifactoryArtifactSearchList(repoName, env.BRANCH_NAME)
imageList.add(0, 'build')
properties([
buildDiscarder(logRotator(numToKeepStr: '20')),
parameters([
choice(name: 'ARTIFACT_NAME', choices: imageList.join('\n'), description: '')
])
])
Shared library that looks at artifactory, its pretty simple.
Essentially make GET Request (And provide auth creds on it) then filter/split result to whittle down to desired values and return list to Jenkinsfile.
import com.cloudbees.groovy.cps.NonCPS
import groovy.json.JsonSlurper
import java.util.regex.Pattern
import java.util.regex.Matcher
List artifactoryArtifactSearchList(String repoKey, String artifact_name, String artifact_archive, String branchName) {
// URL components
String baseUrl = "https://org.jfrog.io/org/api/search/artifact"
String url = baseUrl + "?name=${artifact_name}&repos=${repoKey}"
Object responseJson = getRequest(url)
String regexPattern = "(.+)${artifact_name}-(\\d+).(\\d+).(\\d+).${artifact_archive}\$"
Pattern regex = ~ regexPattern
List<String> outlist = responseJson.results.findAll({ it['uri'].matches(regex) })
List<String> artifactlist=[]
for (i in outlist) {
artifactlist.add(i['uri'].tokenize('/')[-1])
}
return artifactlist.reverse()
}
// Artifactory Get Request - Consume in other methods
Object getRequest(url_string){
URL url = url_string.toURL()
// Open connection
URLConnection connection = url.openConnection()
connection.setRequestProperty ("Authorization", basicAuthString())
// Open input stream
InputStream inputStream = connection.getInputStream()
#NonCPS
json_data = new groovy.json.JsonSlurper().parseText(inputStream.text)
// Close the stream
inputStream.close()
return json_data
}
// Artifactory Get Request - Consume in other methods
Object basicAuthString() {
// Retrieve password
String username = "artifactoryMachineUsername"
String credid = "artifactoryApiKey"
#NonCPS
credentials_store = jenkins.model.Jenkins.instance.getExtensionList(
'com.cloudbees.plugins.credentials.SystemCredentialsProvider'
)
credentials_store[0].credentials.each { it ->
if (it instanceof org.jenkinsci.plugins.plaincredentials.StringCredentials) {
if (it.getId() == credid) {
apiKey = it.getSecret()
}
}
}
// Create authorization header format using Base64 encoding
String userpass = username + ":" + apiKey;
String basicAuth = "Basic " + javax.xml.bind.DatatypeConverter.printBase64Binary(userpass.getBytes());
return basicAuth
}
I could achieve it without any plugin:
With Jenkins 2.249.2 using a declarative pipeline,
the following pattern prompt the user with a dynamic dropdown menu
(for him to choose a branch):
(the surrounding withCredentials bloc is optional, required only if your script and jenkins configuration do use credentials)
node {
withCredentials([[$class: 'UsernamePasswordMultiBinding',
credentialsId: 'user-credential-in-gitlab',
usernameVariable: 'GIT_USERNAME',
passwordVariable: 'GITLAB_ACCESS_TOKEN']]) {
BRANCH_NAMES = sh (script: 'git ls-remote -h https://${GIT_USERNAME}:${GITLAB_ACCESS_TOKEN}#dns.name/gitlab/PROJS/PROJ.git | sed \'s/\\(.*\\)\\/\\(.*\\)/\\2/\' ', returnStdout:true).trim()
}
}
pipeline {
agent any
parameters {
choice(
name: 'BranchName',
choices: "${BRANCH_NAMES}",
description: 'to refresh the list, go to configure, disable "this build has parameters", launch build (without parameters)to reload the list and stop it, then launch it again (with parameters)'
)
}
stages {
stage("Run Tests") {
steps {
sh "echo SUCCESS on ${BranchName}"
}
}
}
}
The drawback is that one should refresh the jenkins configration and use a blank run for the list be refreshed using the script ...
Solution (not from me): This limitation can be made less anoying using an aditional parameters used to specifically refresh the values:
parameters {
booleanParam(name: 'REFRESH_BRANCHES', defaultValue: false, description: 'refresh BRANCH_NAMES branch list and launch no step')
}
then wihtin stage:
stage('a stage') {
when {
expression {
return ! params.REFRESH_BRANCHES.toBoolean()
}
}
...
}
this is my solution.
def envList
def dockerId
node {
envList = "defaultValue\n" + sh (script: 'kubectl get namespaces --no-headers -o custom-columns=":metadata.name"', returnStdout: true).trim()
}
pipeline {
agent any
parameters {
choice(choices: "${envList}", name: 'DEPLOYMENT_ENVIRONMENT', description: 'please choose the environment you want to deploy?')
booleanParam(name: 'SECURITY_SCAN',defaultValue: false, description: 'container vulnerability scan')
}
The example of Jenkinsfile below contains AWS CLI command to get the list of Docker images from AWS ECR dynamically, but it can be replaced with your own command. Active Choices Plug-in is required.
Note! You need to approve the script specified in parameters after first run in "Manage Jenkins" -> "In-process Script Approval", or open job configuration and save it to approve
automatically (might require administrator permissions).
properties([
parameters([[
$class: 'ChoiceParameter',
choiceType: 'PT_SINGLE_SELECT',
name: 'image',
description: 'Docker image',
filterLength: 1,
filterable: false,
script: [
$class: 'GroovyScript',
fallbackScript: [classpath: [], sandbox: false, script: 'return ["none"]'],
script: [
classpath: [],
sandbox: false,
script: '''\
def repository = "frontend"
def aws_ecr_cmd = "aws ecr list-images" +
" --repository-name ${repository}" +
" --filter tagStatus=TAGGED" +
" --query imageIds[*].[imageTag]" +
" --region us-east-1 --output text"
def aws_ecr_out = aws_ecr_cmd.execute() | "sort -V".execute()
def images = aws_ecr_out.text.tokenize().reverse()
return images
'''.stripIndent()
]
]
]])
])
pipeline {
agent any
stages {
stage('First stage') {
steps {
sh 'echo "${image}"'
}
}
}
}
choiceArray = [ "patch" , "minor" , "major" ]
properties([
parameters([
choice(choices: choiceArray.collect { "$it\n" }.join(' ') ,
description: '',
name: 'SOME_CHOICE')
])
])