We use Jenkins Job DSL for our CI setup. Since we are using a special command only available in the traditional Jenkinsfile syntax, we need to use a pipeline job.
Inside of the pipeline job we check out our project from Git. We are using the pipeline job for multiple projects, so we want to inject the git url into the pipeline script.
This is a short version of our script generating the pipeline job:
def createPipelineJob(def jobName, def gitUrl) {
pipelineJob(jobName) {
environmentVariables(GIT_URL: gitUrl)
definition {
cps {
script('''
node {
sh 'env | sort'
}
''')
sandbox(true)
}
}
}
}
This creates the following XML config:
<flow-definition>
<actions/>
<description/>
<keepDependencies>false</keepDependencies>
<properties>
<EnvInjectJobProperty>
<info>
<propertiesContent>GIT_URL=my-git.url</propertiesContent>
<loadFilesFromMaster>false</loadFilesFromMaster>
</info>
<on>true</on>
<keepJenkinsSystemVariables>true</keepJenkinsSystemVariables>
<keepBuildVariables>true</keepBuildVariables>
<overrideBuildParameters>false</overrideBuildParameters>
<contributors/>
</EnvInjectJobProperty>
</properties>
<triggers/>
<definition class="org.jenkinsci.plugins.workflow.cps.CpsFlowDefinition">
<script>
node { sh 'env | sort' }
</script>
<sandbox>true</sandbox>
</definition>
</flow-definition>
But if i run this, the GIT_URL environment variable is not listed (other environment variables are). But if i instead create the pipeline job manually with this setup, the GIT_URL environment variable is printed just fine. Creating the job manually pretty much creates the same xml configuration:
<flow-definition plugin="workflow-job#2.15">
<actions>
<io.jenkins.blueocean.service.embedded.BlueOceanUrlAction plugin="blueocean-rest-impl#1.3.1">
<blueOceanUrlObject class="io.jenkins.blueocean.service.embedded.BlueOceanUrlObjectImpl">
<mappedUrl>blue/organizations/jenkins/test-jobname</mappedUrl>
<modelObject class="flow-definition" reference="../../../.."/>
</blueOceanUrlObject>
</io.jenkins.blueocean.service.embedded.BlueOceanUrlAction>
</actions>
<description/>
<keepDependencies>false</keepDependencies>
<properties>
<com.sonyericsson.rebuild.RebuildSettings plugin="rebuild#1.27">
<autoRebuild>false</autoRebuild>
<rebuildDisabled>false</rebuildDisabled>
</com.sonyericsson.rebuild.RebuildSettings>
<EnvInjectJobProperty plugin="envinject#2.1.5">
<info>
<propertiesContent>GIT_URL=my-git.url</propertiesContent>
<secureGroovyScript plugin="script-security#1.35">
<script/>
<sandbox>false</sandbox>
</secureGroovyScript>
<loadFilesFromMaster>false</loadFilesFromMaster>
</info>
<on>true</on>
<keepJenkinsSystemVariables>true</keepJenkinsSystemVariables>
<keepBuildVariables>true</keepBuildVariables>
<overrideBuildParameters>false</overrideBuildParameters>
</EnvInjectJobProperty>
<org.jenkinsci.plugins.workflow.job.properties.PipelineTriggersJobProperty>
<triggers/>
</org.jenkinsci.plugins.workflow.job.properties.PipelineTriggersJobProperty>
</properties>
<definition class="org.jenkinsci.plugins.workflow.cps.CpsFlowDefinition" plugin="workflow-cps#2.41">
<script>
node { sh 'env | sort' }
</script>
<sandbox>true</sandbox>
</definition>
<triggers/>
<disabled>false</disabled>
</flow-definition>
We are pretty lost because we are new to jenkins and this problem is holding us for days now.
Edit:
The job is generated on the jenkins master node but executed on a slave node
Jenkins 2.37.3
Environment Injector Plugin 2.1.5
Pipeline 2.5
This is more of a comment than an answer, but I modified and tested your DSL code and it works fine.
I created a DSL job using the script:
def createPipelineJob(def jobName, def gitUrl) {
pipelineJob(jobName) {
environmentVariables(GIT_URL: gitUrl)
definition {
cps {
script('''
node {
sh "echo $GIT_URL"
}
''')
sandbox(true)
}
}
}
}
createPipelineJob('new-job-2','my-git.url')
The resulting pipeline job has the same XML as the one you posted (minus the shell script), and building the pipeline job prints the value of GIT_URL.
[new-job-1] Running shell script
+ echo my-git.url
my-git.url
My recommendation:
If the short version you posted (or maybe try mine) doesn't work, I would try to see if upgrading Jenkins or the plugins makes any difference.
If the short version you posted or mine does work, maybe post the full version, perhaps there's an error there.
As it turns out the Environment Injector Plugin was not installed successfully, therefore the script did not run properly. So all i had to do was to restart Jenkins and everything worked just fine. Special thanks to Javier Garcés ensuring me that my script was indeed correct.
Related
I created a Junit jenkins test case where a in-memory jenkins instance is launched (as we use #Rule jenkinsrule). The code of the test case is available here.
The test case will create a FreeStyleProject (= seed job) which will use as Groovy script DSL a maven.groovy file
But when the test case is executed, the following message is reported during the the job build execution. The message reports ghe consequence of the import/parsing of the mavenJob.groovy file as the job expects that a new job will be created.
Legacy code started this job. No cause information is available
Running as SYSTEM
Building in workspace /var/folders/t2/jwchtqkn5y76hrfrws7dqtqm0000gn/T/j h5344303144116520886/workspace/test0
Processing provided DSL script
ERROR: java.io.IOException: Unable to read /var/folders/t2/jwchtqkn5y76hrfrws7dqtqm0000gn/T/j h5344303144116520886/jobs/mvn-spring-boot-rest-http/config.xml
Finished: FAILURE
And of course no stack trace of the error is stdout or stderr.
How can I investigate the problem and fix it ?
Remark:
If I use the config.xml file and import it in a separate jenkins instance, the job succeeded
config.xml file generated, it looks good (vs same config.xml file created using the UI)
<?xml version='1.1' encoding='UTF-8'?>
<project>
<keepDependencies>false</keepDependencies>
<properties/>
<scm class="hudson.scm.NullSCM"/>
<canRoam>false</canRoam>
<disabled>false</disabled>
<blockBuildWhenDownstreamBuilding>false</blockBuildWhenDownstreamBuilding>
<blockBuildWhenUpstreamBuilding>false</blockBuildWhenUpstreamBuilding>
<triggers/>
<concurrentBuild>false</concurrentBuild>
<builders>
<javaposse.jobdsl.plugin.ExecuteDslScripts>
<scriptText>mavenJob('mvn-spring-boot-rest-http') {
description 'A Maven Job compiling the project Spring Boot Rest HTTP Example'
parameters {
gitParameter {
name 'SELECTED_TAG'
description 'The Git tag to checkout'
type 'PT_TAG'
defaultValue '2.3.4-2'
branch ''
branchFilter 'origin/(.*)'
quickFilterEnabled false
selectedValue 'DEFAULT'
sortMode 'DESCENDING_SMART'
tagFilter '*'
useRepository '.*rest-http-example.git'
listSize '10'
}
}
scm {
git {
remote {
url 'https://github.com/snowdrop/rest-http-example.git'
// branch('$SELECTED_TAG')
branch('2.3.4-2')
}
}
}
rootPOM 'pom.xml'
goals 'clean install'
}</scriptText>
<usingScriptText>true</usingScriptText>
<sandbox>false</sandbox>
<ignoreExisting>false</ignoreExisting>
<ignoreMissingFiles>false</ignoreMissingFiles>
<failOnMissingPlugin>false</failOnMissingPlugin>
<failOnSeedCollision>false</failOnSeedCollision>
<unstableOnDeprecation>false</unstableOnDeprecation>
<removedJobAction>IGNORE</removedJobAction>
<removedViewAction>IGNORE</removedViewAction>
<removedConfigFilesAction>IGNORE</removedConfigFilesAction>
<lookupStrategy>JENKINS_ROOT</lookupStrategy>
</javaposse.jobdsl.plugin.ExecuteDslScripts>
</builders>
<publishers/>
<buildWrappers/>
</project>
Many thanks in advance for your help.
I created a thread discussion here too: https://groups.google.com/g/jenkinsci-users/c/mRSwARFapyA
Charles
The problem was related to many missing dependencies needed to run the test case.
I upgraded the build.gradle file and now that works.
https://github.com/ch007m/jenkins-job-dsl/blob/jenkins-2.271/build.gradle#L53-L72
BTW, the error message reported was not correlated at all to the root cause and How to fix the problem. that should be improved within the code ;-)
I want to get the pom version in one of my stage.For this i have this pipeline script. I have Pipeline Utility Steps plugin installed.
stage ('Publish Stage') {
steps {
pom = readMavenPom file: 'pom.xml'
echo pom.version
}
}
I am getting error with this, can anybody tell what mistake am i doing?
You cannot assign to a groovy variable in a declarative pipeline like that. You can only run steps in the steps{} block. If you wrap this in a script{} block, it will work. Take care not to declare the variable locally if you need it available elsewhere.
Writing steps{} block and wrapping it inside script{} would work but in my case, I just used the variable through the environment.
stage('Read Pom Version'){
pom = readMavenPom file: 'pom.xml'
env.POM_VERSION = pom.version
sh '''#!/bin/bash -xe
echo $POM_VERSION
'''.stripIndent()
}
Let say the version inside my pom is 1.0.0. Above jenkins stage prints following:
1.0.0
echo 1.0.0
I have created a Jenkins Pipeline job. In this job I want to do the build using Ant. I have configured the Ant variable in Manage **Jenkins > Global Tool Configuration** as Ant1.9.1= D:\path_to_hybris\hybris\bin\platform\apache-ant-1.9.1.
In a freestyle jenkins Job, I know that the build.xml location can be specified as in the below screenshot:
but I am unable to understand how to specify the ant groovy script beyond this point, especially where can we mention the path to build.xml file:
def antHome = tool 'Ant1.9.1'
????
????
you can use ant wrapper in Jenkins`s pipeline groovy script.
withAnt(installation: 'LocalAnt') {
// some block
sh "ant build"
//for windows
bat "ant build"
}
Remember to configure the ant tool in the Jenkins "Global Tool Configuration" with the same name "LocalAnt".
You can try this:
def antVersion = 'Ant1.9.1'
withEnv( ["ANT_HOME=${tool antVersion}"] ) {
sh '$ANT_HOME/bin/ant target1 target2'
}
Under Windows this would look like this (I didn't test it though):
def antVersion = 'Ant1.9.1'
withEnv( ["ANT_HOME=${tool antVersion}"] ) {
bat '%ANT_HOME%/bin/ant.bat target1 target2'
}
This assumes that you have Ant configured in Jenkins with name 'Ant1.9.1'.
I needed this multiple times within the same Jenkinsfile that needs to be executed on both linux and windows agents so I created a method for it.
You can call ant like this
callAnt("-v -p")
if you add this method definition to your jenkinsfile
def callAnt(String Parameters) {
if (isUnix()) {
env.PATH = "${tool 'ant'}/bin;${env.PATH}"
sh "ant ${Parameters}"
}
else {
env.PATH = "${tool 'ant'}\\bin;${env.PATH}"
bat "ant ${Parameters}"
}
}
I'm converting some Jenkins jobs to DSL scripts.
Some of these use github for SCM and as this is supported by the DSL this is easy enough to configure. However, after over 100 job conversions, for the first time I need to specify a Git executable (all jobs so far have used the default) and there doesn't seem to be a way to do this. The job.xml shows this:
<scm class="hudson.plugins.git.GitSCM" plugin="git#2.4.4">
<configVersion>2</configVersion>
<userRemoteConfigs>...</userRemoteConfigs>
<branches>...</branches>
<doGenerateSubmoduleConfigurations>false</doGenerateSubmoduleConfigurations>
<gitTool>Ubuntu Git</gitTool>
<submoduleCfg class="list"/>
<extensions>
<hudson.plugins.git.extensions.impl.SparseCheckoutPaths>
<sparseCheckoutPaths>
<hudson.plugins.git.extensions.impl.SparseCheckoutPath>
<path>
octane.pricing/octane.trader/server/work/mif_interface/cfg
</path>
</hudson.plugins.git.extensions.impl.SparseCheckoutPath>
</sparseCheckoutPaths>
</hudson.plugins.git.extensions.impl.SparseCheckoutPaths>
</extensions>
</scm>
I can do all of this using the DSL apart from <gitTool>Ubuntu Git</gitTool>.
This isn't mentioned in the DSL so I presume this isn't supported so I tried using the configure block (bearing in mind I'm still learning exactly how to use that). Tried a few things but the one I most expected to work:
configure { project ->
project << 'hudson.plugins.git.GitSCM' {
paramDefs << 'gitTool' {
string('Ubuntu Git')
}
}
}
But no dice - the XML still shows the "default" option.
I'm surprised this can't be specified directly in the DSL but can anyone see what I am doing wrong with that configure block?
The best option is to use the nested configure block of the Git SCM context:
job('example') {
scm {
git {
remote {
github('owner/repo')
}
configure { scmNode ->
scmNode / gitTool('changeme')
}
}
}
}
See configure in the Job DSL API Viewer and more info about the Configure Block in the Job DSL wiki.
In deploy scenario i need to create and run jenkins task on list of hosts, i.e. create something like parametrized task (where ip address is a parameter) or a task on Multijob Plugin with HOST axis, but run by only 2 ones in parallel over multiple hosts.
One of the option could be to run ansible with the list of hosts, but i'd like to see a status per each host separately, and relaunch a jenkins job if needed.
The main option is to use Job DSL Plugin or Pipeline Plugin, but here i need help to understand what classes/methods of dsl groovy code should be used to achieve this.
Can anyone help with it?
Assume that the hosts have been configured as Jenkins slaves already.
Assume that hosts are provided in pipeline job parameter
HOSTS as whitespace separated list. Following example should get you started:
def hosts_pairs = HOSTS.split().collate(2)
for (pair in host_pairs) {
def branches = [:]
for (h in pair) {
def host = h // fresh variable per iteration; it will be mutated
branches[host] = {
stage(host) {
node(host) {
// do the actual job here, e.g.
// execute a shell script
sh "echo hello world"
}
}
}
}
parallel branches
}
A combination of Matrix project and Throttle Concurrent Builds Plugin is possible.
All you need is to setup a single user-defined axis (e.g. "targetHost") with all IP addresses as values and set the desired throttling under "Throttle Concurrent Builds" (please note that you have to enable the "Execute concurrent builds if necessary" option to tell jenkins to allow concurrent execution).
The axis values are available during every child build in the corresponding environment variable (e.g. targetHost).
Below is an example config.xml with simple ping&wait build step:
<?xml version='1.0' encoding='UTF-8'?>
<matrix-project plugin="matrix-project#1.7.1">
<actions/>
<description></description>
<keepDependencies>false</keepDependencies>
<properties>
<hudson.plugins.throttleconcurrents.ThrottleJobProperty plugin="throttle-concurrents#1.9.0">
<maxConcurrentPerNode>2</maxConcurrentPerNode>
<maxConcurrentTotal>2</maxConcurrentTotal>
<categories class="java.util.concurrent.CopyOnWriteArrayList"/>
<throttleEnabled>true</throttleEnabled>
<throttleOption>project</throttleOption>
<limitOneJobWithMatchingParams>false</limitOneJobWithMatchingParams>
<matrixOptions>
<throttleMatrixBuilds>true</throttleMatrixBuilds>
<throttleMatrixConfigurations>true</throttleMatrixConfigurations>
</matrixOptions>
<paramsToUseForLimit></paramsToUseForLimit>
</hudson.plugins.throttleconcurrents.ThrottleJobProperty>
</properties>
<scm class="hudson.scm.NullSCM"/>
<canRoam>true</canRoam>
<disabled>false</disabled>
<blockBuildWhenDownstreamBuilding>false</blockBuildWhenDownstreamBuilding>
<blockBuildWhenUpstreamBuilding>false</blockBuildWhenUpstreamBuilding>
<triggers/>
<concurrentBuild>true</concurrentBuild>
<axes>
<hudson.matrix.TextAxis>
<name>targetHost</name>
<values>
<string>127.0.0.1</string>
<string>127.0.0.2</string>
<string>127.0.0.3</string>
<string>127.0.0.4</string>
<string>127.0.0.5</string>
</values>
</hudson.matrix.TextAxis>
</axes>
<builders>
<hudson.tasks.Shell>
<command>sleep 7
ping -c 7 $targetHost
sleep 7</command>
</hudson.tasks.Shell>
</builders>
<publishers/>
<buildWrappers/>
<executionStrategy class="hudson.matrix.DefaultMatrixExecutionStrategyImpl">
<runSequentially>false</runSequentially>
</executionStrategy>
</matrix-project>
Good luck!