Update:
From the bottom of the Automatically Generated DSL wiki entry ... The generated DSL is only supported when running in Jenkins,....
Since slackNotifier is generated DSL, it doesn't appear that there is a way to test this in our particular infrastructure. We're going to write a function which generates the config using the configure block.
I have a seed job definition which is failing gradle test even though it seems to work fine when we use it in Jenkins.
Job Definition Excerpt
//package master
// GitURL
def gitUrl = 'https://github.com/team/myapp'
def slackRoom = null
job('seed-dsl') {
description('This seed is updated from the seed-dsl-updater job')
properties {
//Set github project URL
githubProjectUrl(gitUrl)
}
...
// publishers is another name for post build steps
publishers {
mailer('', false, true)
slackNotifier {
room(slackRoom)
notifyAborted(true)
notifyFailure(true)
notifyNotBuilt(true)
notifyUnstable(true)
notifyBackToNormal(true)
notifySuccess(false)
notifyRepeatedFailure(false)
startNotification(false)
includeTestSummary(false)
includeCustomMessage(false)
customMessage(null)
buildServerUrl(null)
sendAs(null)
commitInfoChoice('NONE')
teamDomain(null)
authToken(null)
}
}
}
The gradle test command works fine when I comment out the with the slackNotifier declaration, but fail with the following error when it's enabled:
Test output excerpt
Caused by:
javaposse.jobdsl.dsl.DslScriptException: (script, line 79) No signature of method: javaposse.jobdsl.dsl.helpers.publisher.PublisherContext.slackNotifier() is applicable for argument types: (script$_run_closure1$_closure9$_closure14) values: [script$_run_closure1$_closure9$_closure14#d2392a1]
Possible solutions: stashNotifier(), stashNotifier(groovy.lang.Closure)
at javaposse.jobdsl.dsl.DslScriptLoader.runScriptEngine(DslScriptLoader.groovy:135)
at javaposse.jobdsl.dsl.DslScriptLoader.runScriptsWithClassLoader_closure1(DslScriptLoader.groovy:78)
According to the migration doc, slackNotifer has been supported since 1.47. In my gradle.build, I'm using 1.48. I see the same errors with plugin version 1.50
gradle.build excerpt
ext {
jobDslVersion = '1.48'
...
}
...
// Job DSL plugin including plugin dependencies
testCompile "org.jenkins-ci.plugins:job-dsl:${jobDslVersion}"
testCompile "org.jenkins-ci.plugins:job-dsl:${jobDslVersion}#jar"
...
The gradle.build also includes the following, as suggested by the [testing docs] *(https://github.com/jenkinsci/job-dsl-plugin/wiki/Testing-DSL-Scripts).
testPlugins 'org.jenkins-ci.plugins:slack:2.0.1'
What do I need to do to be able to successfully test my job definitions. Is this a bug, or have I missed something else?
removed incorrect reply
EDIT
I see I missed the point.
The new approach is to reuse the #DataBoundConstructor exposed by plugins, so nothing needs to be written to support a new plugin assuming it has a DataBoundConstructor
Your SlackNotifier has this - note the DSL converts the lowercase first letter for you
#DataBoundConstructor
public SlackNotifier(
final String teamDomain,
final String authToken,
final String room,
final String buildServerUrl,
final String sendAs,
final boolean startNotification,
final boolean notifyAborted,
final boolean notifyFailure,
final boolean notifyNotBuilt,
final boolean notifySuccess,
final boolean notifyUnstable,
final boolean notifyBackToNormal,
final boolean notifyRepeatedFailure,
final boolean includeTestSummary,
CommitInfoChoice commitInfoChoice,
boolean includeCustomMessage,
String customMessage) {
...
}
Unfortunately there is an embedded type in the parameter list CommitInfoChoice and this does not have a DataBoundConstructor and its an enum too.
public enum CommitInfoChoice {
NONE("nothing about commits", false, false),
AUTHORS("commit list with authors only", true, false),
AUTHORS_AND_TITLES("commit list with authors and titles", true, true);
...
}
I'll go out on a limb and say that it won't work out the box until the nested enum implements a databound constructor and also has a descriptor, sorry.
I don't have the plugin but you can look at the XML for a real created job with the plugin and see what goes into this section. I suspect it is a nested structure
You can try the job dsl google group - link to a post about the generic approach
We ran into this as well. The solution for us was to add the slack plugin version we were using on jenkins to our list of plugins in gradle.
To be more specific, in our build.gradle file under dependencies, we added the following code to get our plugins included and hence allow the auto-generated DSL to work.
You can see this described here and an example of a different plugin next to testPlugins:
https://github.com/jenkinsci/job-dsl-plugin/wiki/Testing-DSL-Scripts
Like the following:
dependencies {
...
// plugins to install in test instance
testPlugins 'org.jenkins-ci.plugins:ghprb:1.31.4'
testPlugins 'com.coravy.hudson.plugins.github:github:1.19.0'
}
Related
]I am a junior dev trying to lear about Jenkins, I have been learning on my own for a couple of months. Currently I have a pipeline (just for learning purposes) which runs static analysis on a folder, and then publish it, I have been able to send a report through email using jelly templates, from there I realized it is posbile to instantiate the classes of a plugin to use its methods so I went to the cppcheck javadoc and did some trial and error so I can get some values of my report and then do something else with them, so I had something like this in my pipeline:
pipeline {
agent any
stages {
stage('analysis') {
steps {
script{
bat'cppcheck "E:/My_project/Source/" --xml --xml-version=2 . 2> cppcheck.xml'
}
}
}
stage('Test'){
steps {
script {
publishCppcheck pattern:'cppcheck.xml'
for (action in currentBuild.rawBuild.getActions()) {
def name = action.getClass().getName()
if (name == 'org.jenkinsci.plugins.cppcheck.CppcheckBuildAction') {
def cppcheckaction = action
def totalErrors = cppcheckaction.getResult().report.getNumberTotal()
println totalErrors
def warnings = cppcheckaction.getResult().statistics.getNumberWarningSeverity()
println warnings
}
}
}
}
}
}
}
which output is:
[Pipeline] echo
102
[Pipeline] echo
4
My logic (wrongly) tells me that if I can access to the report and statistics classes like that and uses their methods getNumberTotal() and getNumberWarningSeverity() respectively, therefore I should be able to also access the DiffState class in the same way and use the valueOf() method to get an enum of the new errors. But adding this to my pipeline:
def nueva = cppcheckaction.getResult().diffState.valueOf(NEW)
println nueva
Gives me an error:
org.jenkinsci.plugins.scriptsecurity.sandbox.RejectedAccessException: No such field found: field org.jenkinsci.plugins.cppcheck.CppcheckBuildAction diffState
at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.unclassifiedField(SandboxInterceptor.java:425)
at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onGetProperty(SandboxInterceptor.java:409)
...
I can see in the javadoc there is a diffState class with a valueOf() method, but I cannot access to it is therre any other way to get the new errors between the last build and the current one?
I see 2 issues that could be causing this:
CppcheckResult doesn't have a member variable diffState so you can't access it obviously
If you check the javadoc of CppcheckResult the class does have:
private CppcheckReport report;
public CppcheckStatistics getReport()
and
private CppcheckStatistics statistics;
public CppcheckStatistics getStatistics()
there is no member (and getter method) for diffState so maybe try to call:
/**
* Get differences between current and previous statistics.
*
* #return the differences
*/
public CppcheckStatistics getDiff(){
my suggestion: cppcheckaction.getResult().getDiff().valueOf(NEW). Furthermore CppcheckWorkspaceFile does have a method getDiffState().
Please have a look at the script approval of your Jenkins (see here).
The syntax error might appear because Jenkins (Groovy Sandbox) blocks the execution of an (for the Jenkins) "unknown" and potential dangerous method.
Jenkins settings - Script Approval - Approve your blocked method
I am trying to use a global class that I've defined in a shared library to help organise job parameters. It's not working, and I'm not even sure if it is possible.
My job looks something like this:
pipelineJob('My-Job') {
definition {
// Job definition goes here
}
parameters {
choiceParam('awsAccount', awsAccount.ALL)
}
}
In a file in /vars/awsAccount.groovy I have the following code:
class awsAccount implements Serializable {
final String SANDPIT = "sandpit",
final String DEV = "dev",
final String PROD = "prod"
static String[] ALL = [SANDPIT, DEV, PROD]
}
Global pipeline libraries are configured to load implicitly from the my repository's master branch.
When attempting to update the DSL scripts I receive the error:
ERROR: (myJob.groovy, line 67) No such property: awsAccount for class: javaposse.jobdsl.dsl.helpers.BuildParametersContext
Why does it not find the class, and is it even possible to use shared library classes like this in pipeline job?
Disclaimer: I know it works using Jenkinsfile. Unfortunatelly, not tested usng Declarative Pipelines - but no answers yet, so it may be worth a try
Regarding your first question: there are some reasons why a class from your shared-lib could not be found. Starting from the library import, the library syntax, etc. But they definitvely work for DSL. To be more precise about it, additional information would be great. But be sure that:
You have your groovy class definition using exactly the directory structure as described in the documentation (https://www.jenkins.io/doc/book/pipeline/shared-libraries/)
Give a name to the shared-lib in jenkins as you configure it and be sure is exactly the name you use in the import
Use the import as described in the documentation (under Using Libraries)
Regarding your second question (the one that names this SO question): yes, you can include parameter jobs from information in your shared-lib. At least, using Jenkinsfiles. You can even define properties to be included in the pipelie. I got it working with a tricky syntax due to different problems.
Again, I am using Jenkinsfile and this is what worked for me:
In my shared-lib class, I added a static function that introduces the build parameters. Notice the input parameters that function needs and its usage:
class awsAccount implements Serializable {
//
static giveMeParameters (script) {
return [
// Some parms
script.string(defaultValue: '', description: 'A default parameter', name: 'textParm'),
script.booleanParam(defaultValue: false, description: 'If set to True, do whatever you need - otherwise, do not do it', name: 'boolOption'),
]
}
}
To introduce those parameters in the pipeline, you need to place the returned value of the function into the parameters array
properties (
parameters (
awsAccount.giveMeParameters (this)
)
Again, notice the syntax when calling the function. Similar to this, you can also define functions in the shared-lib that return properties and use them in multiple jobs (disableConcurrentBuilds, buildDiscarder, etc)
I am setting up a Jenkins Pipeline, which calls an external library with a compare XML function written in Groovy that utilises xmlunit.
The function looks as follows:
import java.util.List
import org.custommonkey.xmlunit.*
// Gives you a list of all the differences.
#NonCPS
void call(String xmlControl, String xmlTest) throws Exception {
String myControlXML = xmlControl
String myTestXML = xmlTest
DetailedDiff myDiff = new DetailedDiff(compareXML(myControlXML,
myTestXML));
List allDifferences = myDiff.getAllDifferences();
assertEquals(myDiff.toString(), 0, allDifferences.size());
}
However when running the pipeline in Jenkins it returns a java.io.NotSerializableException.
Checking StackOverflow it seemed like adding a the #NonCPS annotation might help.
But sadly it did not make a difference.
What more could I try to resolve the java.io.NotSerializableException?
Ideally I'd like to be able to invoke the script with some kind of unit test before I have it execute on a Jenkins.
Is there any way to test a Job DSL script other than having jenkins run it?
Besides the examples in job-dsl-gradle-example, you can also go a step further and write tests for individual files or jobs. For example let's assume you have a job configuration file located in jobs/deployJob.groovy
import javaposse.jobdsl.dsl.DslScriptLoader
import javaposse.jobdsl.dsl.MemoryJobManagement
import javaposse.jobdsl.dsl.ScriptRequest
import spock.lang.Specification
class TestDeployJobs extends Specification {
def 'test basic job configuration'() {
given:
URL scriptURL = new File('jobs').toURI().toURL()
ScriptRequest scriptRequest = new ScriptRequest('deployJob.groovy', null, scriptURL)
MemoryJobManagement jobManagement = new MemoryJobManagement()
when:
DslScriptLoader.runDslEngine(scriptRequest, jobManagement)
then:
jobManagement.savedConfigs.each { String name, String xml ->
with(new XmlParser().parse(new StringReader(xml))) {
// Make sure jobs only run manually
triggers.'hudson.triggers.TimerTrigger'.spec.text().isEmpty()
// only deploy every environment once at a time
concurrentBuild.text().equals('false')
// do a workspace cleanup
buildWrappers.'hudson.plugins.ws__cleanup.PreBuildCleanup'
// make sure masked passwords are active
!buildWrappers.'com.michelin.cio.hudson.plugins.maskpasswords.MaskPasswordsBuildWrapper'.isEmpty()
}
}
}
}
This way you are able to go through every XML node you want to make sure to have all the right values set.
Have a look at the job-dsl-gradle-example. The repo contains a test for DSL scripts.
Doing it in the same way as crasp but using Jenkins test harness as explained in Jenkins Unit Test page, which is slower but would work with auto-generated DSL giving syntax errors as explained here.
After setting the code as explained here, you can just do a test like this one:
#Unroll
void 'check descriptions #file.name'(File file) {
given:
JobManagement jobManagement = new JenkinsJobManagement(System.out, [:], new File('.'))
Jenkins jenkins = jenkinsRule.jenkins
when:
GeneratedItems items = new DslScriptLoader(jobManagement).runScript(file.text)
then:
if (!items.jobs.isEmpty()) {
items.jobs.each { GeneratedJob generatedJob ->
String text = getItemXml(generatedJob, jenkins)
with(new XmlParser().parse(new StringReader(text))) {
// Has some description
!description.text().isEmpty()
}
}
}
where:
file << TestUtil.getJobFiles()
}
My requirement is to invoke some processing from a Jenkins build server, to determine whether the domain model has changed since the last build. I've come to the conclusion that the way forward is to write a script that will invoke a sequence of existing scripts from the db-migration plugin. Then I can invoke it in the step that calls test-app and war.
I've looked in the Grails doc, and at some of the db-migration scripts, and I find I'm stuck - have no idea where to start trying things. I'd be really grateful if someone could point me at any suitable sources. BTW, I'm a bit rusty in Grails. Started to teach myself two years ago via proof of concept project, which lasted 6 months. Then it was back to Eclipse rich client work. That might be part of my problem, though I never go involved in scripts.
One thing I need in the Jenkins evt is to get hold of the current SVN revision number being used for the build. Suggestions welcome.
Regards, John
Create a new script by running grails create-script scriptname. The database-migration plugins scripts are configured to be easily reused. There are is a lot of shared code in _DatabaseMigrationCommon.groovy and each script defines one target with a unique name. So you can import either the shared script or any standalone script (or multiple scripts) and call the targets like they're methods.
By default the script generated by create-script "imports" the _GrailsInit script via includeTargets << grailsScript("_GrailsInit") and you can do the same, taking advantage of the magic variables that point at installed plugins' directories:
includeTargets << new File("$databaseMigrationPluginDir/scripts/DbmGenerateChangelog.groovy")
If you do this you can remove the include of _GrailsInit since it's already included, but if you don't that's fine since Grails only includes files once.
Then you can define your target and call any of the plugin's targets. The targets cannot accept parameters, but you can add data to the argsMap (this is a map Grails creates from the parsed commandline arguments) to simulate user-specified args. Note that any args passed to your script will be seen by the database-migration plugin's scripts since they use the same argsMap.
Here's an example script that just does the same thing as dbm-generate-changelog but adds a before and after message:
includeTargets << new File("$databaseMigrationPluginDir/scripts/DbmGenerateChangelog.groovy")
target(foo: "Just calls dbmGenerateChangelog") {
println 'before'
dbmGenerateChangelog()
println 'after'
}
setDefaultTarget foo
Note that I renamed the target from main to foo so it's unique, in case you want to call this from another script.
As an example of working with args, here's a modified version that specifies a default changelog name if none is provided:
println 'before'
if (!argsMap.params) {
argsMap.params = ['foo2.groovy']
}
dbmGenerateChangelog()
println 'after'
Edit: Here's a fuller example that captures the output of dbm-gorm-diff to a string:
includeTargets << new File("$databaseMigrationPluginDir/scripts/_DatabaseMigrationCommon.groovy")
target(foo: "foo") {
depends dbmInit
def configuredSchema = config.grails.plugin.databasemigration.schema
String argSchema = argsMap.schema
String effectiveSchema = argSchema ?: configuredSchema ?: defaultSchema
def realDatabase
boolean add = false // booleanArg('add')
String filename = null // argsList[0]
try {
printMessage "Starting $hyphenatedScriptName"
ByteArrayOutputStream baos = new ByteArrayOutputStream()
def baosOut = new PrintStream(baos)
ScriptUtils.executeAndWrite filename, add, dsName, { PrintStream out ->
MigrationUtils.executeInSession(dsName) {
realDatabase = MigrationUtils.getDatabase(effectiveSchema, dsName)
def gormDatabase = ScriptUtils.createGormDatabase(dataSourceSuffix, config, appCtx, realDatabase, effectiveSchema)
ScriptUtils.createAndPrintFixedDiff(gormDatabase, realDatabase, realDatabase, appCtx, diffTypes, baosOut)
}
}
String xml = new String(baos.toString('UTF-8'))
def ChangelogXml2Groovy = classLoader.loadClass('grails.plugin.databasemigration.ChangelogXml2Groovy')
String groovy = ChangelogXml2Groovy.convert(xml)
// do something with the groovy or xml here
printMessage "Finished $hyphenatedScriptName"
}
catch (e) {
ScriptUtils.printStackTrace e
exit 1
}
finally {
ScriptUtils.closeConnection realDatabase
}
}
setDefaultTarget foo