I have a setup where several different kinds of tests are run by different Jenkins jobs, each using a separate Jenkinsfile. Since the pipeline definitions in each Jenkinsfile were fairly similar, I figured, I could reduce code duplication by moving the pipeline definition into a function in a groovy script included by the Jenkinsfiles. The function would have several parameters to specify the details where the pipelines differ. Alas, I haven't managed to get it to work. I reduced the code to a "hello world" pipeline which runs fine when specified in the Jenkinsfile directly, but fails in the script function.
Jenkinsfile:
def integrationTestsModule = evaluate readTrusted("jenkins/integration_tests.groovy")
integrationTestsModule.defineTestPipeline()
jenkins/integration_tests.groovy:
def defineTestPipeline() {
pipeline {
agent {
node {
label "clang11-it"
}
}
options {
disableConcurrentBuilds()
}
stages {
stage('build') {
steps {
sh 'echo "Hello world!"'
}
}
}
}
}
Error:
21:28:13 java.lang.NoSuchMethodError: No such DSL method 'options' found among steps [_OcAction, _OcContextInit, _OcWatch, addBadge, addEmbeddableBadgeConfiguration, addErrorBadge, addHtmlBadge, addInfoBadge, addShortText, addWarningBadge, ansiColor, archive, bat, build, catchError, checkout, compareVersions, container, containerLog, createSummary, deleteDir, dependencyCheckPublisher, dir, dockerFingerprintFrom, dockerFingerprintRun, echo, envVarsForTool, error, fileExists, findFiles, gatlingArchive, getContext, git, httpRequest, input, isUnix, junit, library, libraryResource, load, lock, mail, milestone, node, nodesByLabel, pagerduty, pagerdutyChangeEvent, parallel, podTemplate, powershell, prependToFile, properties, publishChecks, publishHTML, publishIssues, pwd, pwsh, readCSV, readFile, readJSON, readManifest, readMavenPom, readProperties, readTrusted, readYaml, recordIssues, removeBadges, removeHtmlBadges, removeSummaries, resolveScm, retry, scanForIssues, script, sh, sha1, sha256, slackSend, slackUploadFile, slackUserIdFromEmail, slackUserIdsFromCommitters, sleep, sshagent, stage, stash, step, tar, tee, timeout, timestamps, tm, tool, touch, unarchive, unstable, unstash, untar, unzip, validateDeclarativePipeline, verifySha1, verifySha256, waitForQualityGate, waitUntil, warnError, withChecks, withContext, withCredentials, withDockerContainer, withDockerRegistry, withDockerServer, withEnv, withMaven, withVault, wrap, writeCSV, writeFile, writeJSON, writeMavenPom, writeYaml, ws, xunit, zip] or symbols [AUnit, BoostTest, CTest, CUnit, Check, CppTest, CppUnit, Custom, FPCUnit, GitUsernamePassword, GoogleTest, JUnit, MSTest, MbUnit, NUnit2, NUnit3, PHPUnit, PVSStudio, QtTest, UnitTest, Valgrind, absolute, acuCobol, adoptOpenJdkInstaller, agent, ajc, all, allBranchesSame, allOf, allure, always, analysisParser, androidLintParser, ansibleLint, ant, antFromApache, antOutcome, antTarget, any, anyOf, apiToken, apiTokenProperty, aquaScanner, architecture, archiveArtifacts, armCc, artifactManager, artifactsPublisher, authorizationMatrix, axivion, axivionSuite, batchFile, bitbucket, bitbucketBranchDiscovery, bitbucketBuildStatusNotifications, bitbucketForkDiscovery, bitbucketPublicRepoPullRequestFilter, bitbucketPullRequestDiscovery, bitbucketServer, bitbucketSshCheckout, bitbucketTagDiscovery, bitbucketTrustEveryone, bitbucketTrustNobody, bitbucketTrustProject, bitbucketTrustTeam, bitbucketWebhookConfiguration, bitbucketWebhookRegistration, bluepearl, booleanParam, brakeman, branch, buckminster, buildButton, buildDiscarder, buildDiscarders, buildParameter, buildRetention, buildSelector, buildingTag, builtInNode, bzt, cadence, cargo, caseInsensitive, caseSensitive, ccm, certificate, changeRequest, changelog, changeset, checkStyle, checkoutToSubdirectory, choice, choiceParam, clair, clang, clangAnalyzer, clangTidy, cleanWs, clock, cmake, codeAnalysis, codeChecker, codeNarc, command, concordionPublisher, configFile, configFileProvider, configMapVolume, containerEnvVar, containerLivenessProbe, containerTemplate, coolflux, copyArtifactPermission, copyArtifacts, copyartifact, cpd, cppCheck, cppLint, credentials, cron, crumb, cssLint, cssText, cssUrl, dark, darkSystem, dart, default, defaultFolderConfiguration, defaultView, demand, dependenciesFingerprintPublisher, dependency-check, dependencyCheck, dependencyTrackPublisher, dependencycheck, detekt, diabC, disableConcurrentBuilds, disableResume, discoverReferenceBuild, docFx, docker, dockerCert, dockerLint, dockerServer, dockerTool, dockerfile, downstream, doxygen, drMemory, dscanner, dumb, dupFinder, durabilityHint, dynamicPVC, eclipse, embUnit, emptyDirVolume, emptyDirWorkspaceVolume, envInject, envVar, envVars, envVarsFilter, environment, equals, erlc, errorProne, esLint, excludeCategory, excludeFile, excludeMessage, excludeModule, excludeNamespace, excludePackage, excludeType, executor, expression, extendedChoice, failed, faviconUrl, file, fileParam, filePath, findBugs, findbugsPublisher, fingerprint, fingerprints, flake8, flawfinder, flexSdk, frameOptions, freeStyle, freeStyleJob, fromDocker, fromScm, fromSource, fxcop, gcc, gcc3, gcc4, gendarme, ghsMulti, git, gitBranchDiscovery, gitHooks, gitHubBranchDiscovery, gitHubBranchHeadAuthority, gitHubExcludeArchivedRepositories, gitHubExcludeForkedRepositories, gitHubExcludePrivateRepositories, gitHubExcludePublicRepositories, gitHubForkDiscovery, gitHubIgnoreDraftPullRequestFilter, gitHubPullRequestDiscovery, gitHubSshCheckout, gitHubTagDiscovery, gitHubTopicsFilter, gitHubTrustContributors, gitHubTrustEveryone, gitHubTrustNobody, gitHubTrustPermissions, gitTagDiscovery, gitUsernamePassword, github, githubAccessToken, githubProjectProperty, githubPush, globalConfigFiles, gnat, gnuFortran, goLint, goVet, groovyScript, gtester, hadoLint, hashicorpVault, headRegexFilter, headWildcardFilter, hostPathVolume, hostPathWorkspaceVolume, hyperlink, hyperlinkToModels, iar, iarCstat, ibLinter, ideaInspection, includeCategory, includeFile, includeMessage, includeModule, includeNamespace, includePackage, includeType, infer, inheriting, inheritingGlobal, installSource, intel, invalids, invokerPublisher, isRestartedRun, issueCommentTrigger, issues, jacocoPublisher, java, javaDoc, javadoc, jcReport, jdk, jdkInstaller, jgit, jgitapache, jgivenPublisher, jnlp, jobBuildDiscarder, jobConfigHistory, jobName, jsHint, jsLint, jsUrl, junitParser, junitPublisher, junitTestResultStorage, klocWork, kotlin, ktLint, kubeconfig, kubernetes, kubernetesAgent, label, lastCompleted, lastDuration, lastFailure, lastGrantedAuthorities, lastStable, lastSuccess, lastSuccessful, lastWithArtifacts, latestSavedBuild, legacy, legacySCM, list, local, location, logRotator, loggedInUsersCanDoAnything, mailer, masterBuild, maven, maven3Mojos, mavenConsole, mavenErrors, mavenGlobalConfig, mavenLinkerPublisher, mavenMojos, mavenWarnings, merge, metrowerksCodeWarrior, mineRepository, modelsim, modernSCM, msBuild, myPy, myView, nagFortran, namedBranchesDifferent, never, newContainerPerStage, nfsVolume, nfsWorkspaceVolume, node, nodeProperties, nodejs, nodejsci, nonInheriting, none, nonresumable, not, oc, oelintAdv, onFailure, openTasksPublisher, organizationFolder, otDockerLint, override, overrideIndexTriggers, owaspDependencyCheck, paneStatus, parallelsAlwaysFailFast, parameters, passed, password, pattern, pcLint, pep8, perfReport, perforce, performanceReport, performanceTest, perlCritic, permalink, permanent, persistentVolumeClaim, persistentVolumeClaimWorkspaceVolume, php, phpCodeSniffer, phpStan, pipeline, pipeline-model, pipeline-model-docker, pipelineGraphPublisher, pipelineMaven, pipelineTriggers, pit, plainText, plugin, pmdParser, podAnnotation, podEnvVar, podLabel, pollSCM, portMapping, prefast, preserveStashes, previous, prismConfiguration, projectNamingStrategy, protoLint, proxy, pruneTags, pullRequestReview, puppetLint, pyDocStyle, pyLint, qacSourceCodeAnalyser, qtTranslation, queueItemAuthenticator, quietPeriod, rateLimit, rateLimitBuilds, recordIssues, relative, resharperInspectCode, resourceRoot, retainOnlyVariables, revApi, rfLint, robocopy, ruboCop, run, runParam, sSHLauncher, sarif, scala, schedule, scmRetryCount, scriptApproval, scriptApprovalLink, search, secretEnvVar, secretVolume, security, shell, simian, simple-theme-plugin, simpleBuildDiscarder, skipDefaultCheckout, skipStagesAfterUnstable, skipped, slackNotifier, slave, snapshotDependencies, sonarQube, sourceRegexFilter, sourceWildcardFilter, specific, sphinxBuild, spotBugs, spotbugsPublisher, ssh, sshPublicKey, sshUserPrivateKey, standard, status, string, stringParam, styleCop, styleLint, sunC, suppressAutomaticTriggering, suppressFolderAutomaticTriggering, swapSpace, swiftLint, tag, tagList, taskScanner, taskingVx, teamSlugFilter, testCase, text, textParam, themeManager, tiCss, timestamper, timestamperConfig, timezone, tmpSpace, tnsdl, toolLocation, triggeredBy, trivy, tsLint, unsecured, untrusted, upstream, userSeed, usernameColonPassword, usernamePassword, validatingString, vaultFile, vaultString, veracodePipelineScanner, viewsTabBar, warningsParsers, warningsPlugin, weather, withAnt, withSonarQubeEnv, workspace, x509ClientCert, xUnitDotNet, xlc, xmlLint, yamlLint, yuiCompressor, zip, zptLint] or globals [agentGenerator, archive, assemble, buildAndPushContainer, buildVersionBump, buildWasTriggeredByIssueComment, changelogBE, checkDockerBuild, checkPRLabeling, commonCheckout, contains, copyImageEPAToERp, cordovaBuildAndroid, createSimulatoriOS, currentBuild, customDeploymentBE, dashboardDeploymentBE, deleteSimulatoriOS, dependencyReport, dependencyTrack, dependencyTrackCpp, dependencyTrackNodeJS, deploymentBE, determineDeployedVersionBE, docker, env, fastlane, fastlaneFinishRelease, fileDownloader, fileTemplater, finishRelease, getBuildCause, getCertificates, getLabels, gitOpsRegisterDeployment, gitTagVersionBump, gradle, gradleAssemble, gradleAssembleBarmerECare, gradleCache, gradleCreateRelease, gradleCreateReleaseEpa, gradleCreateVersionRelease, gradleStdout, helmLintGitOpsCharts, instrumentationTests, integrationTests, jacocoReport, jacocoReportBE, javadocAndroid, kubectlBE, kubectlDryRunBE, kubectlRemoteCommand, labelProcessing, loadGitHubCLIConfiguration, loadGithubConfiguration, loadGithubSSHConfiguration, loadNexusConfiguration, loadProjectConfiguration, loadSonarqubeConfiguration, manageUsers, manager, mavenCreateRelease, needTools, objectStorageS3cmd, openshift, owaspScan, params, performDependencyCheckBE, pipeline, pipelineChangeLogMobile, pipelineChangelogGitHubRepos, pipelineCopyUbuntuImage, pipelineCreateProductRelease, pipelineCreateRelease, pipelineGitOpsDeployment, pipelineGitOpsDeploymentBox, pipelineGitOpsDeploymentRelease, pipelineMonitorCertificates, pipelineMonitorKubernetesWorkers, pipelinePromoteProductRelease, pipelinePromoteProductReleaseRUTU, pipelineReTagGitHubRepos, pipelineSharedLibraryJarUpdate, pipelineSyncArgoCD, pipelineUpdateInitialTSLSecretEpa, pipelineUpdateKubernetesWorkers, podSpecLint, podUpdate, prepareCordovaRelease, prepareRelease, promoteDockerImage, promoteHelmChart, publishArtifacts, publishGitHubStatus, publishNodeArtifact, publishPodspec, scanImageVulnerabilities, scm, shadowAssemble, slackNotifyBuild, slackNotifyBuildFailed, slackNotifyBuildRestored, slackNotifyBuildUnstable, slackNotifyDeploymentComplete, slackNotifyDeploymentFailed, slackNotifySTFailedBE, slackNotifySTRestoredBE, slackNotifySTUnstableBE, slackSendClient, staticAnalysis, staticAnalysisBE, staticAnalysisCpp, staticAnalysisPRAndroid, staticAnalysisPRiOS, staticAnalysisTypeScript, staticAnalysisiOS, stopOldBuilds, swiftFormatLint, systemTestsBE, systemTestsOpsBE, templateApplicationYmlBE, triggerDeployment, triggerKvConnectDL, triggerSmoketestsBE, triggerSmoketestsOpsBE, unitTest, unitTests, uploadToGooglePlay, uploadToTestFlight, validateChangeLogMobile, vaultSecrets, versionBump, wickedCheckout]
21:28:13 at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:219)
21:28:13 at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:124)
...
What is going wrong exactly? Is there any way to get it to work?
There are a few steps to get this to work.
This is the documentation
You can add individual functions to call or even have a whole pipeline definition with parameters
Could use Jenkins description setter plugin in Jenkins UI to set the build description to the matched regex from build logs, but I failed to replicate the same functionality as Jenkins script, used the below lines but ended up with errors.
publishers {
buildDescription(/^build-descrip: (.*)/)
}
O/p
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
java.lang.NoSuchMethodError: No such DSL method 'publishers' found among steps [ArtifactoryGradleBuild, MavenDescriptorStep, VersionNumber, addInteractivePromotion, ansiColor, archive, artifactoryDistributeBuild, artifactoryDownload, artifactoryEditProps, artifactoryMavenBuild, artifactoryNpmInstall, artifactoryNpmPublish, artifactoryPromoteBuild, artifactoryUpload, bat, build, catchError, checkout, collectEnv, conanAddRemote, conanAddUser, container, containerLog, deleteDir, deployArtifacts, dir, dockerFingerprintFrom, dockerFingerprintRun, dockerNode, dockerPullStep, dockerPushStep, echo, emailext, emailextrecipients, envVarsForTool, error, fileExists, findBuildScans, findFiles, getArtifactoryServer, getContext, git, initConanClient, input, isUnix, jiraComment, jiraIssueSelector, jiraSearch, junit, library, libraryResource, load, lock, logstash, logstashSend, mail, milestone, newArtifactoryServer, newBuildInfo, newGradleBuild, newMavenBuild, newNpmBuild, node, nodesByLabel, parallel, podTemplate, powershell, properties, publishBuildInfo, publishHTML, pwd, readCSV, readFile, readJSON, readManifest, readMavenPom, readProperties, readTrusted, readYaml, resolveScm, retry, rtAddInteractivePromotion, rtBuildInfo, rtDeleteProps, rtDockerPush, rtDownload, rtGradleDeployer, rtGradleResolver, rtGradleRun, rtMavenDeployer, rtMavenResolver, rtMavenRun, rtNpmDeployer, rtNpmInstall, rtNpmPublish, rtNpmResolver, rtPromote, rtPublishBuildInfo, rtServer, rtSetProps, rtUpload, runConanCommand, script, sh, sha1, slackSend, sleep, sloccountPublish, sshagent, stage, stash, step, svn, tee, timeout, tm, tool, touch, unarchive, unstable, unstash, unzip, vSphere, validateDeclarativePipeline, waitUntil, warnError, withContext, withCredentials, withDockerContainer, withDockerRegistry, withDockerServer, withEnv, wrap, writeCSV, writeFile, writeJSON, writeMavenPom, writeYaml, ws, xrayScan, xrayScanBuild, zip] or symbols [all, allOf, always, ant, antFromApache, antOutcome, antTarget, any, anyOf, apiToken, architecture, archiveArtifacts, artifactManager, attach, authorizationMatrix, batchFile, bitbucket, booleanParam, branch, brokenBuildSuspects, brokenTestsSuspects, buildAllBranches, buildAnyBranches, buildButton, buildChangeRequests, buildDescription, buildDiscarder, buildName, buildNamedBranches, buildNoneBranches, buildParameter, buildRegularBranches, buildTags, buildingTag, caseInsensitive, caseSensitive, certificate, changeRequest, changelog, changeset, checkoutToSubdirectory, choice, choiceParam, cleanWs, clock, cloud, command, configFile, configFileProvider, configMapVolume, containerEnvVar, containerLivenessProbe, containerTemplate, copyArtifactPermission, copyArtifacts, credentials, cron, crumb, culprits, default, defaultView, demand, developers, disableConcurrentBuilds, disableResume, docker, dockerCert, dockerfile, downloadSettings, downstream, dumb, durabilityHint, elasticSearch, emptyDirVolume, emptyDirWorkspaceVolume, envVar, envVars, environment, equals, exact, expression, extendedChoice, file, fileParam, filePath, findText, fingerprint, frameOptions, freeStyle, freeStyleJob, fromScm, fromSource, git, gitHubBranchDiscovery, gitHubBranchHeadAuthority, gitHubForkDiscovery, gitHubTagDiscovery, gitHubTrustContributors, gitHubTrustEveryone, gitHubTrustNobody, gitHubTrustPermissions, gitParameter, github, githubPush, globalConfigFiles, gradle, headRegexFilter, headWildcardFilter, hostPathVolume, hostPathWorkspaceVolume, hyperlink, hyperlinkToModels, inheriting, inheritingGlobal, installSource, isRestartedRun, jacoco, jdk, jdkInstaller, jgit, jgitapache, jnlp, jobName, kubernetes, label, lastCompleted, lastDuration, lastFailure, lastGrantedAuthorities, lastStable, lastSuccess, lastSuccessful, lastWithArtifacts, latestSavedBuild, legacy, legacySCM, list, local, location, logParser, logRotator, loggedInUsersCanDoAnything, logstash, masterBuild, maven, maven3Mojos, mavenErrors, mavenMojos, mavenWarnings, modernSCM, msbuild, msbuildError, msbuildWarning, myView, never, newContainerPerStage, nfsVolume, nfsWorkspaceVolume, node, nodeProperties, nonInheriting, nonStoredPasswordParam, none, not, onFailure, overrideIndexTriggers, paneStatus, parallelsAlwaysFailFast, parameters, password, pattern, permalink, permanent, persistentVolumeClaim, persistentVolumeClaimWorkspaceVolume, pipeline-model, pipelineTriggers, plainText, plugin, podAnnotation, podEnvVar, pollSCM, portMapping, preserveStashes, projectNamingStrategy, proxy, queueItemAuthenticator, quietPeriod, rabbitMq, rateLimitBuilds, recipients, redis, regex, remotingCLI, requestor, rule, run, runParam, s3CopyArtifact, s3Upload, schedule, scmRetryCount, scriptApprovalLink, search, secretEnvVar, secretVolume, security, shell, skipDefaultCheckout, skipStagesAfterUnstable, slackNotifier, slave, sourceRegexFilter, sourceWildcardFilter, specific, ssh, sshPublisher, sshPublisherDesc, sshTransfer, sshUserPrivateKey, stackTrace, standard, status, string, stringParam, swapSpace, syslog, tag, text, textParam, tmpSpace, toolLocation, triggeredBy, unsecured, upstream, [![upstreamDevelopers, userSeed, usernameColonPassword, usernamePassword, viewsTabBar, weather, wil][1]][1]dcards, withAnt, workspace, zfs, zip] or globals
In the scripted pipeline, you can do it like -
timestamps {
node() {
currentBuild.description = "text"
currentBuild.displayName = "text"
}
}
You are trying to execute JobDSL in pipelineDSL. That won't work
The pipelineDSL syntax to edit the build description is
currentBuild.description = "my new description"
wrap it in a script block if you are using declarative pipeline
To parse build logs, you should use sh step return output
def myVar = sh (
script: 'echo "toto"',
returnStdout: true
).trim()
now on the myVar variable you can use regex
def myRegexResult = (myVar =~ /myRegex/)
Using the plugin: https://plugins.jenkins.io/groovy-postbuild/
def matcher = manager.getLogMatcher(".*Total time: (.*)\$")
if(matcher?.matches()) {
manager.addShortText(matcher.group(1), "grey", "white", "0px", "white")
}
I'm trying to remove the warning, Using the ‘stage’ step without a block argument is deprecated. So, I'm doing is adding block for stage
stage 'Install Tool' {
// code
}
Now, jenkins gives the error
java.lang.NoSuchMethodError: No such DSL method 'Install Tool' found among steps [archive, bat, build, catchError, checkout, deleteDir, dir, dockerFingerprintFrom, dockerFingerprintRun, echo, emailext, emailextrecipients, envVarsForTool, error, fileExists, findFiles, getContext, git, httpRequest, input, isUnix, library, libraryResource, load, mail, milestone, node, parallel, properties, pwd, readFile, readJSON, readManifest, readMavenPom, readProperties, readTrusted, readYaml, resolveScm, retry, script, sh, slackSend, sleep, stage, stash, step, svn, timeout, timestamps, tool, touch, unarchive, unstash, unzip, validateDeclarativePipeline, waitUntil, withContext, withCredentials, withDockerContainer, withDockerRegistry, withDockerServer, withEnv, wrap, writeFile, writeJSON, writeMavenPom, ws, zip] or symbols [all, allOf, always, ant, antFromApache, antOutcome, antTarget, any, anyOf, apiToken, architecture, archiveArtifacts, artifactManager, batchFile, booleanParam, branch, buildButton, buildDiscarder, caseInsensitive, caseSensitive, choice, choiceParam, clock, cloud, command, cron, crumb, defaultView, demand, disableConcurrentBuilds, docker, dockerfile, downloadSettings, downstream, dumb, envVars, environment, expression, file, fileParam, filePath, fingerprint, frameOptions, freeStyle, freeStyleJob, git, github, githubPush, gradle, hyperlink, hyperlinkToModels, installSource, jdk, jdkInstaller, jgit, jgitapache, jnlp, jobName, junit, label, lastDuration, lastFailure, lastGrantedAuthorities, lastStable, lastSuccess, legacy, legacySCM, list, local, location, logRotator, loggedInUsersCanDoAnything, masterBuild, maven, maven3Mojos, mavenErrors, mavenMojos, mavenWarnings, modernSCM, myView, node, nodeProperties, nonStoredPasswordParam, none, not, overrideIndexTriggers, paneStatus, parameters, password, pattern, pipeline-model, pipelineTriggers, plainText, plugin, pollSCM, projectNamingStrategy, proxy, queueItemAuthenticator, quietPeriod, run, runParam, schedule, scmRetryCount, search, security, shell, skipDefaultCheckout, skipStagesAfterUnstable, slave, stackTrace, standard, status, string, stringParam, swapSpace, text, textParam, tmpSpace, toolLocation, unsecured, upstream, usernameColonPassword, usernamePassword, viewsTabBar, weather, zfs, zip] or globals [currentBuild, docker, env, params, pipeline, scm]
at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:149)
at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:108)
at groovy.lang.GroovyObject$invokeMethod$30.call(Unknown Source)
Is adding block the right thing to do?
The correct Groovy syntax would be:
stage('Install Tool') {
// code
}