I have a big Jenkinsfile that I would like to re-use for other projects, but I have different parameters per project, so i've tried having one file per project containing only those parameters like this :
Jenkinsfile
node {
checkout scm
def options = []
def optionsBuilder = load pwd() + '/global-scripts/optionsBuilder.groovy'
options.addAll(optionsBuilder.buildOptions(env.JOB_BASE_NAME))
properties { options }
}
global-scripts/optionsBuilder.groovy
def buildOptions(jobName) {
echo "load " + pwd() + "/project-scripts/" + jobName + ".groovy"
def jobOptionsBuilder = load pwd() + "/project-scripts/" + jobName + ".groovy"
return jobOptionsBuilder.buildOptions()
}
return this
project-scripts/job.groovy
def buildOptions() {
def options = [buildDiscarder(logRotator(numToKeepStr: '5')),
parameters([string(name: 'releaseVersion', defaultValue: env.releaseVersion, description: 'Version that needs to be released'),
string(name: 'nextVersion', defaultValue: env.nextVersion, description: 'Next snapshot version' ),
string(name: 'branch', defaultValue: env.branch, description: 'Branch that needs to be released'),
booleanParam(name: 'sendRocketChatNotification', defaultValue: true, description: 'Send notification to Rocket_Chat'),
booleanParam(name: 'sendEmail', defaultValue: true, description: 'Send an email with changelog'),
booleanParam(name: 'dryRun', defaultValue: false, description: 'No git push and no mvn deploy')])]
return options
}
return this
But it seems i can't find the right syntax .. Jenkins throws me this error :
java.lang.ClassCastException: org.jenkinsci.plugins.workflow.multibranch.JobPropertyStep.properties expects java.util.List<hudson.model.JobProperty> but received class org.jenkinsci.plugins.workflow.cps.CpsClosure2
at org.jenkinsci.plugins.structs.describable.DescribableModel.coerce(DescribableModel.java:394)
at org.jenkinsci.plugins.structs.describable.DescribableModel.buildArguments(DescribableModel.java:318)
I have the impression that in your Jenkinsfile you should just write
properties(options)
Related
I have the following Jenkins pipeline that needs to grep a file - I know that it takes some time till the file is created. Is it possible to retry a single command, in my example is :
env.LOG_STATUS = sh(script: "ssh ${params.zeek_server_username}#${params.zeek_server_ip} cat /data/zeek-data/logs/current/smtp.log | grep 1.1.1.2",returnStdout: true)
BTW ,
how can I retry in step and stage level?
pipeline {
agent any
parameters {
string(name: 'zeek_nic_id_1', defaultValue: 'ens192', description: '')
string(name: 'zeek_nic_id_2', defaultValue: 'ens224', description: '')
string(name: 'zeek_server_ip', defaultValue: '1.1.1.1', description: '')
string(name: 'zeek_server_username', defaultValue: 'user', description: '')
}
stages {
stage ('Verify if IP 1.1.1.2 can be found in SMTP logs ') {
steps {
script {
env.LOG_STATUS = sh(script: "ssh ${params.zeek_server_username}#${params.zeek_server_ip} cat /data/zeek-data/logs/current/smtp.log | grep 1.1.1.2",returnStdout: true)
sh 'echo $LOG_STATUS'
}
}
}
}
}
I have a Jenkins Pipeline contained in a script that dynamically builds dropdown menus for input parameters that get used later in subsequent steps. The first time I run the build it creates the dropdowns but doesn't display them to the user, it just continues on to the next step using the default inputs.
If I then run the build a second time it displays the dropdowns and waits for choice selection by the user before continuing on to the next step. How do I get the dropdowns to display and wait on the first run as well?
Here is a pic of the dropdowns the second time I run the build:
Here is my Pipeline script:
pipeline {
agent any
stage ('Build Parameters') {
steps {
script {
properties([
parameters([
choice(
choices: [ 'MyEsxiServer.esxi' ],
description: 'On which ESXi Server should this script run?',
name: 'esxiServer'
),
choice(
choices: [ 'dev','stage','prod' ],
description: 'In which environment should this script run?',
name: 'environment'
),
[$class: 'ChoiceParameter',
choiceType: 'PT_SINGLE_SELECT',
description: 'On which runner should this script run?',
filterLength: 1,
filterable: false,
name: 'runner',
randomName: 'choice-parameter-596645940283131',
script: [$class: 'GroovyScript',
script: [classpath: [], sandbox: false,
script:
'''// Build choices for runner drop down
import groovy.json.JsonSlurper
log = new File('/tmp/groovy.log')
// execute the OS cmd and return the results
def sout = new StringBuilder()
def serr = new StringBuilder() //standard out and error strings
//Assemble command
def cmd = "/home/jenkins/bpb/testautomation/ansible/playbooks/listVMs.sh"
log.append("\\ncmd: " + cmd + "\\n") //debug
//Execute OS command
def proc = cmd.execute()
proc.waitForProcessOutput(sout, serr) //Wait for command to complete
proc.waitForOrKill(10000) //Set timeout
log.append("sout: " + sout + "\\n") //debug
log.append("serr: " + serr + "\\n") //debug
// translate JSON to List
def soutList = new JsonSlurper().parseText(sout.toString())
log.append("soutList: " + soutList + "\\n") //debug
def List vmList = soutList.vms.sort()
log.append("vmList: " + vmList + "\\n") //debug
return vmList
'''
]
]
],
choice(
choices: [ 'CentOS51','Comodore64','BeOS' ],
description: 'What is the target Operating System?',
name: 'targetOS'
)
]) //parameters
]) //properties
} //script
} //steps
} //stage
stage ('Execute Ansible') {
steps {
ansiblePlaybook credentialsId: 'b7a24821-8dc3-40d0-8cee-ef284e07393a',
disableHostKeyChecking: true,
colorized: true,
installation: 'Ansible',
inventory: "testautomation/ansible/${params.environment}/${params.environment}.inv",
playbook: 'testautomation/ansible/playbooks/run_cmd_inside_vm.yml',
extras: "-v -e runner=${params.runner} -e shell_cmd=/home/kcason/Desktop/${params.targetOS}/3-run.sh"
} //steps
} //stage
} // stages
} // pipeline
Not at all sadly. You can start a job with or without parameters (depending on how it is configured). The properties call changes this configuration but only after the job is started (without parameters/ with the old parameters). Jenkins cannot do that before since the properties call could be anywhere in the script and could depend on steps before it.
This article https://dev.to/pencillr/jenkins-pipelines-and-their-dirty-secrets-1 discusses this topic in more detail.
Also see the open issue in the jenkins issue tracker https://issues.jenkins-ci.org/plugins/servlet/mobile#issue/JENKINS-41929
In my Jenkins Pipeline, the first stage executes a freestyle job. Therefore, I need to get the output/logs from that job because it will return a string of IPs which will be used on my second stage.
def getEc2ListResult = []
pipeline {
agent {
label 'master'
}
stages{
stage('Get EC2 List'){
steps {
script {
def getEc2List = build job: 'get-ec2-by-tag', parameters: [
string(name: 'envTagValue', value: "${envTagValue}"),
string(name: 'OS', value: "${OS}")
]
getEc2ListResult = getEc2List.getPreviousBuild().getLog(20)
}
}
}
}
}
This is the error that I'm getting:
hudson.remoting.ProxyException: groovy.lang.MissingMethodException: No signature of method: org.jenkinsci.plugins.workflow.support.steps.build.RunWrapper.getLog() is applicable for argument types: (java.lang.Integer) values: [20]
Possible solutions: getId(), getAt(java.lang.String), getClass()
getEc2List is of type RunWrapper, also getEc2List.getPreviousBuild().
RunWrapper doesn't supply a getLog() api, it is supplied by rawBuild.
you can get getEc2List's rowBuild by call getEc2List.rawBuild or getEc2List.getRawBuild().
But getRawBuild() is not in #Whitelisted of RunWrapper, so you will get following message in jenkins log:
Scripts not permitted to use method
org.jenkinsci.plugins.workflow.support.steps.build.RunWrapper
getRawBuild. Administrators can decide whether to approve or reject
this signature.
One option, it's to ask Jenkins admin to change Script Approve
Another option, it's do as following:
stage('') {
environment {
JENKINS_AUTH = credentials('<credentail id of jenkins auth>')
// add an credentail with a jenkins user:password before use it at here
}
steps {
script {
def getEc2List = build job: 'get-ec2-by-tag', parameters: [
string(name: 'envTagValue', value: "${envTagValue}"),
string(name: 'OS', value: "${OS}")
]
logUrl = getEc2List.absoluteUrl + 'consoleText'
log = sh(script: 'curl -u {JENKINS_AUTH} -k' + logUrl,
returnStdout: true).trim()
// parse the log to extract the ip
...
}
}
}
I have a multibranch pipeline in Jenkins.
I defined multiple check boxes (over 20) for each parameter to be passed to a script, which then starts my application and runs corresponding test case (this might not be an optimal solution but this framework was created before I started at current company and I am not going to refactor it):
booleanParam(name: 'cluster_number', defaultValue: false, description: '')
booleanParam(name: 'post_cluster_wu', defaultValue: false, description: '')
etc.
I need to collect user selection for each checkbox (true-false). I would prefer to do it in a loop, like this:
sh """
for (element in params) {
// testing:
echo "${element.key} ${element.value}"
}
"""
but keep getting an error:
[Pipeline] End of Pipeline
groovy.lang.MissingPropertyException: No such property: element for class: groovy.lang.Binding
at groovy.lang.Binding.getVariable(Binding.java:63)
at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onGetProperty(SandboxInterceptor.java:264)
at org.kohsuke.groovy.sandbox.impl.Checker$6.call(Checker.java:288)
Also tried to put loop outside of shell script. No luck so far.
steps {
echo "username: ${params.OWNER_USERNAME}"
for (element in params) {
echo "${element.key} ${element.value}"
}
...
Wonder if anyone was able to loop through params?
Thanks in advance!
This works:
pipeline {
agent any
parameters {
booleanParam(name: 'alpha', defaultValue: true)
booleanParam(name: 'beta', defaultValue: true)
booleanParam(name: 'gamma', defaultValue: false)
}
stages {
stage('only') {
steps {
script {
params.keySet().each {
echo "The value of the ${it} parameter is: ${params[it]}"
}
}
}
}
}
}
pipeline {
agent any
stages {
stage("foo") {
steps {
script {
env.RELEASE_SCOPE = input message: 'User input required', ok: 'Release!',
parameters: [choice(name: 'RELEASE_SCOPE', choices: 'patch\nminor\nmajor',
description: 'What is the release scope?')]
}
echo "${env.RELEASE_SCOPE}"
}
}
}
}
In this above code, The choice are hardcoded (patch\nminor\nmajor) -- My requirement is to dynamically give choice values in the dropdown.
I get the values from calling api - Artifacts list (.zip) file names from artifactory
In the above example, It request input when we do the build, But i want to do a "Build with parameters"
Please suggest/help on this.
Depends how you get data from API there will be different options for it, for example let's imagine that you get data as a List of Strings (let's call it releaseScope), in that case your code be following:
...
script {
def releaseScopeChoices = ''
releaseScope.each {
releaseScopeChoices += it + '\n'
}
parameters: [choice(name: 'RELEASE_SCOPE', choices: ${releaseScopeChoices}, description: 'What is the release scope?')]
}
...
hope it will help.
This is a cutdown version of what we use. We separate stuff into shared libraries but I have consolidated a bit to make it easier.
Jenkinsfile looks something like this:
#!groovy
#Library('shared') _
def imageList = pipelineChoices.artifactoryArtifactSearchList(repoName, env.BRANCH_NAME)
imageList.add(0, 'build')
properties([
buildDiscarder(logRotator(numToKeepStr: '20')),
parameters([
choice(name: 'ARTIFACT_NAME', choices: imageList.join('\n'), description: '')
])
])
Shared library that looks at artifactory, its pretty simple.
Essentially make GET Request (And provide auth creds on it) then filter/split result to whittle down to desired values and return list to Jenkinsfile.
import com.cloudbees.groovy.cps.NonCPS
import groovy.json.JsonSlurper
import java.util.regex.Pattern
import java.util.regex.Matcher
List artifactoryArtifactSearchList(String repoKey, String artifact_name, String artifact_archive, String branchName) {
// URL components
String baseUrl = "https://org.jfrog.io/org/api/search/artifact"
String url = baseUrl + "?name=${artifact_name}&repos=${repoKey}"
Object responseJson = getRequest(url)
String regexPattern = "(.+)${artifact_name}-(\\d+).(\\d+).(\\d+).${artifact_archive}\$"
Pattern regex = ~ regexPattern
List<String> outlist = responseJson.results.findAll({ it['uri'].matches(regex) })
List<String> artifactlist=[]
for (i in outlist) {
artifactlist.add(i['uri'].tokenize('/')[-1])
}
return artifactlist.reverse()
}
// Artifactory Get Request - Consume in other methods
Object getRequest(url_string){
URL url = url_string.toURL()
// Open connection
URLConnection connection = url.openConnection()
connection.setRequestProperty ("Authorization", basicAuthString())
// Open input stream
InputStream inputStream = connection.getInputStream()
#NonCPS
json_data = new groovy.json.JsonSlurper().parseText(inputStream.text)
// Close the stream
inputStream.close()
return json_data
}
// Artifactory Get Request - Consume in other methods
Object basicAuthString() {
// Retrieve password
String username = "artifactoryMachineUsername"
String credid = "artifactoryApiKey"
#NonCPS
credentials_store = jenkins.model.Jenkins.instance.getExtensionList(
'com.cloudbees.plugins.credentials.SystemCredentialsProvider'
)
credentials_store[0].credentials.each { it ->
if (it instanceof org.jenkinsci.plugins.plaincredentials.StringCredentials) {
if (it.getId() == credid) {
apiKey = it.getSecret()
}
}
}
// Create authorization header format using Base64 encoding
String userpass = username + ":" + apiKey;
String basicAuth = "Basic " + javax.xml.bind.DatatypeConverter.printBase64Binary(userpass.getBytes());
return basicAuth
}
I could achieve it without any plugin:
With Jenkins 2.249.2 using a declarative pipeline,
the following pattern prompt the user with a dynamic dropdown menu
(for him to choose a branch):
(the surrounding withCredentials bloc is optional, required only if your script and jenkins configuration do use credentials)
node {
withCredentials([[$class: 'UsernamePasswordMultiBinding',
credentialsId: 'user-credential-in-gitlab',
usernameVariable: 'GIT_USERNAME',
passwordVariable: 'GITLAB_ACCESS_TOKEN']]) {
BRANCH_NAMES = sh (script: 'git ls-remote -h https://${GIT_USERNAME}:${GITLAB_ACCESS_TOKEN}#dns.name/gitlab/PROJS/PROJ.git | sed \'s/\\(.*\\)\\/\\(.*\\)/\\2/\' ', returnStdout:true).trim()
}
}
pipeline {
agent any
parameters {
choice(
name: 'BranchName',
choices: "${BRANCH_NAMES}",
description: 'to refresh the list, go to configure, disable "this build has parameters", launch build (without parameters)to reload the list and stop it, then launch it again (with parameters)'
)
}
stages {
stage("Run Tests") {
steps {
sh "echo SUCCESS on ${BranchName}"
}
}
}
}
The drawback is that one should refresh the jenkins configration and use a blank run for the list be refreshed using the script ...
Solution (not from me): This limitation can be made less anoying using an aditional parameters used to specifically refresh the values:
parameters {
booleanParam(name: 'REFRESH_BRANCHES', defaultValue: false, description: 'refresh BRANCH_NAMES branch list and launch no step')
}
then wihtin stage:
stage('a stage') {
when {
expression {
return ! params.REFRESH_BRANCHES.toBoolean()
}
}
...
}
this is my solution.
def envList
def dockerId
node {
envList = "defaultValue\n" + sh (script: 'kubectl get namespaces --no-headers -o custom-columns=":metadata.name"', returnStdout: true).trim()
}
pipeline {
agent any
parameters {
choice(choices: "${envList}", name: 'DEPLOYMENT_ENVIRONMENT', description: 'please choose the environment you want to deploy?')
booleanParam(name: 'SECURITY_SCAN',defaultValue: false, description: 'container vulnerability scan')
}
The example of Jenkinsfile below contains AWS CLI command to get the list of Docker images from AWS ECR dynamically, but it can be replaced with your own command. Active Choices Plug-in is required.
Note! You need to approve the script specified in parameters after first run in "Manage Jenkins" -> "In-process Script Approval", or open job configuration and save it to approve
automatically (might require administrator permissions).
properties([
parameters([[
$class: 'ChoiceParameter',
choiceType: 'PT_SINGLE_SELECT',
name: 'image',
description: 'Docker image',
filterLength: 1,
filterable: false,
script: [
$class: 'GroovyScript',
fallbackScript: [classpath: [], sandbox: false, script: 'return ["none"]'],
script: [
classpath: [],
sandbox: false,
script: '''\
def repository = "frontend"
def aws_ecr_cmd = "aws ecr list-images" +
" --repository-name ${repository}" +
" --filter tagStatus=TAGGED" +
" --query imageIds[*].[imageTag]" +
" --region us-east-1 --output text"
def aws_ecr_out = aws_ecr_cmd.execute() | "sort -V".execute()
def images = aws_ecr_out.text.tokenize().reverse()
return images
'''.stripIndent()
]
]
]])
])
pipeline {
agent any
stages {
stage('First stage') {
steps {
sh 'echo "${image}"'
}
}
}
}
choiceArray = [ "patch" , "minor" , "major" ]
properties([
parameters([
choice(choices: choiceArray.collect { "$it\n" }.join(' ') ,
description: '',
name: 'SOME_CHOICE')
])
])