I am creating a jenkins pipeline job to seed jobs using the jenkins job DSL plugin. How do I get the workspace path inside the DSL file? The jenkins pipeline code is as such:
#!groovy
node{
stage("build jobs"){
ws{
git poll: true, credentialsId: 'xxx', url: 'ssh://git#aaaaa.cc.xxx.com:/xxx/xxx.git'
checkout scm
jobDsl(removedJobAction: 'DISABLE', removedViewAction: 'DELETE', targets: 'jobs/*.groovy', unstableOnDeprecation: true)
}
}
}
The DSL code that is failing is:
hudson.FilePath workspace = hudson.model.Executor.currentExecutor().getCurrentWorkspace()
With the error:
Processing DSL script pipeline.groovy
java.lang.NullPointerException: Cannot invoke method getCurrentWorkspace() on null object
at org.codehaus.groovy.runtime.NullObject.invokeMethod(NullObject.java:91)
at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:48)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
at org.codehaus.groovy.runtime.callsite.NullCallSite.call(NullCallSite.java:35)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:117)
at pipeline.run(pipeline.groovy:1)
at pipeline$run.call(Unknown Source)
Variables created in the pipeline area are not accessible inside the job DSL step
I stumbled upon this because there seems to be no good way. Here is how I do it:
node {
stage('test') {
sh 'pwd > workspace.txt'
jobDsl scriptText: '''
String workspace = readFileFromWorkspace('workspace.txt').trim()
def file = new File(workspace, 'test.txt')
file.append('It worked!')'''
}
}
So first grab the workspace in the pipeline script and then pass it to the job dsl. If you have more than just the workspace variable, that you need in your scripts I suggest transferring via a properties file:
node {
stage('test') {
sh 'echo "workspace="$(pwd) > build.properties'
jobDsl scriptText: '''
Properties props = new Properties();
props.load(streamFileFromWorkspace('build.properties'))
def file = new File(props.getProperty('workspace'), 'test.txt')
file.append('It worked!')'''
}
}
This can be achieved by using SEED_JOB variable:
String workspacePath = SEED_JOB.lastBuild.checkouts[0].workspace
It is described in project's wiki:
Access to the seed job is available through the SEED_JOB variable. The variable contains a reference to the internal Jenkins object that
represents the seed job. The actual type of the object depends on the
type of job that runs the DSL. For a freestyle project, the object is
an instance of hudson.model.FreeStyleProject. See the Jenkins API
Documentation for details.
The SEED_JOB variable is only available in scripts, not in any classes
used by a script. And it is only available when running in Jenkins,
e.g. in the "Process Job DSLs" build step.
The following example show how to apply the same quiet period for a
generated job as for the seed job.
job('example') {
quietPeriod(SEED_JOB.quietPeriod) }
You can use the __FILE__ variable in a Job DSL script to get the path of the current script. Maybe you can use that to derive the workspace directory. See Script Location for details.
def scriptDir = new File(__FILE__).parent.absolutePath
You can pass the workspace argument to job dsl. for example:
The pipeline code as follow:
node {
step([
$class: 'ExecuteDslScripts',
scriptText: 'job("example-2")'
])
step([
$class: 'ExecuteDslScripts',
targets: ['jobs/projectA/*.groovy', 'jobs/common.groovy'].join('\n'),
removedJobAction: 'DELETE',
removedViewAction: 'DELETE',
lookupStrategy: 'SEED_JOB',
additionalClasspath: ['libA.jar', 'libB.jar'].join('\n'),
additionalParameters: [
message: 'Hello from pipeline',
credentials: 'SECRET'
WORKSPACE: env.WORKSPACE
]
])
}
https://github.com/jenkinsci/job-dsl-plugin/wiki/User-Power-Moves#use-job-dsl-in-pipeline-scripts
Related
I'm a using the jenkins pipeline. My usecase is that the developer are using a simple *.ini file that is parsed by a python script to add or remove stage within the jenkinsfile whenever they want. I don't want them to manually edit the jenkinsfile because they won't know how it works.
Expected behaviour is:
When a build is triggered I would like to first execute a python script which might write into the jenkinsfile to add/remove stage according to the *.ini file.
As far as I understand, when an event trigger a jenkins build, the first thing it does is opening the jenkinsfile. However I would like to know if it's possible to run some prebuild script before that ?
Thanks
Edit: here's a simple view of run of the pipeline (blue ocean UI)
The ini file might for example remove in the stage Compilation the step Building Plan C by removing the groovy code doing that in the jenkins file
Give an example for reference.
node {
git url: '', branch: '', credentialsId: ''
def parseStr = sh(script: 'python parser.py xxx.ini', returnStdout: true).trim()
// the python parser expect to return a JSON string like:
// {'run_stage1': false, 'run_stage2': true}
def parseObj = readJSON text: parseStr
stage('stage 1') {
if(parseObj.run_stage1) {
echo 'stage1'
...
}
}
stage('stage 1') {
if(parseObj.run_stage2) {
echo 'stage1'
....
}
}
}
Jenkins pipeline had supply apis: readJSON, readYaml, readProperties to read JSON, YAML and Properties files.
If you choose any of them to replace ini file, you can drop the python parser to make your pipeline more simple
In one stage of my declarative jenkins pipeline codes, it executes a bash script(sh '''./a.sh''', script "a.sh" is maintained outsides) - in that script, the value of "jarVersion" is injected in ${WORKSPACE}/.jarVersion (echo "jarVersion=${jarVersion}" > ${WORKSPACE}/.jarVersion). At later stage, we need get the value of jarVersion. We use load "${WORKSPACE}/.jarVersion" and ${jarVersion} to get the value. It works when we do so in pipeline script.
However, when we set this pipeline as a shared library (put it in /vars/testSuite.groovy) and call it in another pipeline script. It can not recognize var ${jarVersion}.
Please advise how to solve the issue. A common question is: how to transfer a value in a script from stage A to stage B?
stage('getJarVersion'){
steps{
script{
load "${WORKSPACE}/.jarVersion"
currentBuild.description = "jarVersion:${jarVersion}"
}
}
}
I expected it could work as it is in pipeline scripts.
But it shows:
groovy.lang.MissingPropertyException: No such property: jarVersion for class: testSuite
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.unwrap(ScriptBytecodeAdapter.java:53)
at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.getProperty(ScriptBytecodeAdapter.java:458)
at com.cloudbees.groovy.cps.sandbox.DefaultInvoker.getProperty(DefaultInvoker.java:34)
at com.cloudbees.groovy.cps.impl.PropertyAccessBlock.rawGet(PropertyAccessBlock.java:20)
at testSuite.call(/jenkins/jobs/TestSuite1/builds/11/libs/pipelineUtilities/vars/testSuite.groovy:84)
With the stages under the same groovy file, you have to declare the object out of the stage blocks and before the node block. So for each stage, you can define the value inside the variable:
Pipeline {
def my_var
stage('stage1'){
---------
}
stage('stage2'){
---------
}
}
If you are defining a stage per file, you have to create the closures with the input object and to pass it in the call from the parent groovy file:
test.groovy:
def call(def my_obj, String my_string) {
stage('my_stage') {
println(my_obj)
}
}
parent_test.groovy
test(obj_value,string_value)
How do I invoke Global environment variables in Jenkinsfile?
For example, if I have a variable -
name:credentialsId
value:xxxx-xxxx-xxxxx-xxxxxxxxx
How do I use it in the groovy script?
I tried ${credentialsId}, but it didn't work. It will just give error:
java.lang.NoSuchMethodError: No such DSL method '$' found among steps [ArtifactoryGradleBuild, ........
In a Jenkinsfile, you have the "Working with the Environment" which mentions:
The full list of environment variables accessible from within Jenkins Pipeline is documented at localhost:8080/pipeline-syntax/globals#env,
The syntax is ${env.xxx} as in:
node {
echo "Running ${env.BUILD_ID} on ${env.JENKINS_URL}"
}
See also "Managing the Environment".
How can I pass the Global variables to the Jenkinsfile?
When I say Global variables - I mean in
Jenkins -> Manage Jenkins -> Configure System -> Global properties -> Environment variables
See "Setting environment variables"
Setting an environment variable within a Jenkins Pipeline can be done with the withEnv step, which allows overriding specified environment variables for a given block of Pipeline Script, for example:
Jenkinsfile (Pipeline Script)
node {
/* .. snip .. */
withEnv(["NAME=value"]) {
... your job
}
}
When referring to env in Groovy scope, simply use env.VARIABLE_NAME, for example to pass on BUILD_NUMBER of upstream job to a triggered job:
stage ('Starting job') {
build job: 'TriggerTest', parameters: [
[$class: 'StringParameterValue', name: 'upstream_build_number', value: env.BUILD_NUMBER]
]
}
Scripted pipeline
To read an environment variable whose name you know, use env.NAME
To read an environment variable whose name is not known until runtime use env.getProperty(name).
For example, a value from a YAML config file represents an environment variable name:
config.yaml (in workspace)
myconfig:
key: JOB_DISPLAY_URL
Jenkinsfile
node {
println("Running job ${env.JOB_NAME}")
def config = readYaml(file:'config.yaml')
def value = env.getProperty(config.myconfig.key)
println("Value of property ${config.myconfig.key} is ${value}")
}
For getting values all env.VAR, env['VAR'], env.getProperty('VAR') are fine.
For setting values the only safe way at the moment is withEnv. If you try to assign values to env.VAR it may not work in some cases like for parallel pipelines (like in JENKINS-59871).
Another syntax is $ENV:xxxx
node {
echo "Running $ENV.BUILD_ID on $ENV.JENKINS_URL" }
This worked for me
How can I trigger build of another job from inside the Jenkinsfile?
I assume that this job is another repository under the same github organization, one that already has its own Jenkins file.
I also want to do this only if the branch name is master, as it doesn't make sense to trigger downstream builds of any local branches.
Update:
stage 'test-downstream'
node {
def job = build job: 'some-downtream-job-name'
}
Still, when executed I get an error
No parameterized job named some-downtream-job-name found
I am sure that this job exists in jenkins and is under the same organization folder as the current one. It is another job that has its own Jenkinsfile.
Please note that this question is specific to the GitHub Organization Plugin which auto-creates and maintains jobs for each repository and branch from your GitHub Organization.
In addition to the above mentioned answers: I wanted to start a job with a simple parameter passed to a second pipeline and found the answer on http://web.archive.org/web/20160209062101/https://dzone.com/refcardz/continuous-delivery-with-jenkins-workflow
So i used:
stage ('Starting ART job') {
build job: 'RunArtInTest', parameters: [[$class: 'StringParameterValue', name: 'systemname', value: systemname]]
}
First of all, it is a waste of an executor slot to wrap the build step in node. Your upstream executor will just be sitting idle for no reason.
Second, from a multibranch project, you can use the environment variable BRANCH_NAME to make logic conditional on the current branch.
Third, the job parameter takes an absolute or relative job name. If you give a name without any path qualification, that would refer to another job in the same folder, which in the case of a multibranch project would mean another branch of the same repository.
Thus what you meant to write is probably
if (env.BRANCH_NAME == 'master') {
build '../other-repo/master'
}
You can use the build job step from Jenkins Pipeline (Minimum Jenkins requirement: 2.130).
Here's the full API for the build step: https://jenkins.io/doc/pipeline/steps/pipeline-build-step/
How to use build:
job: Name of a downstream job to build. May be another Pipeline job, but more commonly a freestyle or other project.
Use a simple name if the job is in the same folder as this upstream Pipeline job;
You can instead use relative paths like ../sister-folder/downstream
Or you can use absolute paths like /top-level-folder/nested-folder/downstream
Trigger another job using a branch as a param
At my company many of our branches include "/". You must replace any instances of "/" with "%2F" (as it appears in the URL of the job).
In this example we're using relative paths
stage('Trigger Branch Build') {
steps {
script {
echo "Triggering job for branch ${env.BRANCH_NAME}"
BRANCH_TO_TAG=env.BRANCH_NAME.replace("/","%2F")
build job: "../my-relative-job/${BRANCH_TO_TAG}", wait: false
}
}
}
Trigger another job using build number as a param
build job: 'your-job-name',
parameters: [
string(name: 'passed_build_number_param', value: String.valueOf(BUILD_NUMBER)),
string(name: 'complex_param', value: 'prefix-' + String.valueOf(BUILD_NUMBER))
]
Trigger many jobs in parallel
Source: https://jenkins.io/blog/2017/01/19/converting-conditional-to-pipeline/
More info on Parallel here: https://jenkins.io/doc/book/pipeline/syntax/#parallel
stage ('Trigger Builds In Parallel') {
steps {
// Freestyle build trigger calls a list of jobs
// Pipeline build() step only calls one job
// To run all three jobs in parallel, we use "parallel" step
// https://jenkins.io/doc/pipeline/examples/#jobs-in-parallel
parallel (
linux: {
build job: 'full-build-linux', parameters: [string(name: 'GIT_BRANCH_NAME', value: env.BRANCH_NAME)]
},
mac: {
build job: 'full-build-mac', parameters: [string(name: 'GIT_BRANCH_NAME', value: env.BRANCH_NAME)]
},
windows: {
build job: 'full-build-windows', parameters: [string(name: 'GIT_BRANCH_NAME', value: env.BRANCH_NAME)]
},
failFast: false)
}
}
Or alternatively:
stage('Build A and B') {
failFast true
parallel {
stage('Build A') {
steps {
build job: "/project/A/${env.BRANCH}", wait: true
}
}
stage('Build B') {
steps {
build job: "/project/B/${env.BRANCH}", wait: true
}
}
}
}
The command build in pipeline is there to trigger other jobs in jenkins.
Example on github
The job must exist in Jenkins and can be parametrized.
As for the branch, I guess you can read it from git
Use build job plugin for that task in order to trigger other jobs from jenkins file.
You can add variety of logic to your execution such as parallel ,node and agents options and steps for triggering external jobs. I gave some easy-to-read cookbook example for that.
1.example for triggering external job from jenkins file with conditional example:
if (env.BRANCH_NAME == 'master') {
build job:'exactJobName' , parameters:[
string(name: 'keyNameOfParam1',value: 'valueOfParam1')
booleanParam(name: 'keyNameOfParam2',value:'valueOfParam2')
]
}
2.example triggering multiple jobs from jenkins file with conditionals example:
def jobs =[
'job1Title'{
if (env.BRANCH_NAME == 'master') {
build job:'exactJobName' , parameters:[
string(name: 'keyNameOfParam1',value: 'valueNameOfParam1')
booleanParam(name: 'keyNameOfParam2',value:'valueNameOfParam2')
]
}
},
'job2Title'{
if (env.GIT_COMMIT == 'someCommitHashToPerformAdditionalTest') {
build job:'exactJobName' , parameters:[
string(name: 'keyNameOfParam3',value: 'valueOfParam3')
booleanParam(name: 'keyNameOfParam4',value:'valueNameOfParam4')
booleanParam(name: 'keyNameOfParam5',value:'valueNameOfParam5')
]
}
}
I'm exploring Jenkins 2.0 pipelines. So far my file is pretty simple.
node {
stage "checkout"
git([url:"https://github.com/luxengine/math.git"])
stage "build"
echo "Building from pipeline"
}
I can't seem to find any way to set the directory that git will checkout to. I also can't find any kind of documentation related to that. I found https://jenkinsci.github.io/job-dsl-plugin/ but it doesn't seem to match what I see on other tutorials.
Clarification
Looks like you are trying to configure Pipeline job (formerly known as Workflow). This type of job is very distinct from Job DSL.
The purpose of Pipeline job is to:
Orchestrates long-running activities that can span multiple build slaves. Suitable for building pipelines (formerly known as workflows) and/or organizing complex activities that do not easily fit in free-style job type.
Where as Job DSL:
...allows the programmatic creation of projects using a DSL. Pushing job creation into a script allows you to automate and standardize your Jenkins installation, unlike anything possible before.
Solution
If you want to checkout your code to specific directory then replace git step with more general SCM checkout step.
Final Pipeline configuration should look like that:
node {
stage "checkout"
//git([url:"https://github.com/luxengine/math.git"])
checkout([$class: 'GitSCM',
branches: [[name: '*/master']],
doGenerateSubmoduleConfigurations: false,
extensions: [[$class: 'RelativeTargetDirectory',
relativeTargetDir: 'checkout-directory']],
submoduleCfg: [],
userRemoteConfigs: [[url: 'https://github.com/luxengine/math.git']]])
stage "build"
echo "Building from pipeline"
}
As a future reference for Jenkins 2.0 and Pipeline DSL please use built-in Snippet Generator or documentation.
This can be done by using the directive of dir:
def exists = fileExists '<your target dir>'
if (!exists){
new File('<your target dir>').mkdir()
}
dir ('<your target dir>') {
git url: '<your git repo address>'
}
First make clear that you are using Jenkins Job DSL.
You can do this like this:
scm {
git {
wipeOutWorkspace(true)
shallowClone(true);
remote {
url("xxxx....")
relativeTargetDir('checkout-folder')
}
}
}
https://jenkinsci.github.io/job-dsl-plugin/
This above address gives you the chance simply to type in upper left aread for example 'scm' and than it will show in which relationships 'scm' can be used. Than you can select 'scm-freestylejob' and afterwards click on the '***' than you can see the details.
The general start point for Jenkins Job DSL is here:
https://github.com/jenkinsci/job-dsl-plugin/wiki
You can of course ask here on SO or on Google Forum:
https://groups.google.com/forum/#!forum/job-dsl-plugin
pipeline {
agent any
stages{
stage("Checkout") {
steps {
dir('def exists = fileNotExists \'git\'') {
bat label: '', script: 'sh "mkdir.sh'
}
dir ('cm') {
git branch: 'dev',
credentialsId: '<your credential id>',
url: '<yours git url>'
}
}
} //End of Checkout stage
stage("TestShellScript") {
steps {
bat label: '', script: 'sh "PrintNumber.sh"'
}
}
}//End of stages
} // End of pipeline
Note: cat mkdir.sh
#!/bin/bash
#Create a directory
mkdir git
You are using the Pipeline Plugin, not the Job DSL Plugin. In the Pipeline Plugin, if you want to define something, where there is not yet a function available in the Pipeline syntax, you can define it yourself.