Apply a quality gate to only one of the tools - jenkins

I have a series of jenkins pipeline jobs which build bitbucket pull requests.
Those pipelines have a Quality stage, which records issues using warningsng for GCC warnings and Coverity defects (via the generic issues tool).
The relevant part of my pipeline is:
post {
always {
recordIssues(
blameDisabled: true,
forensicsDisabled: true,
tools: [gcc(id: "${env.PRODUCT}-static-gcc",
name: "${PRODUCT} GCC warnings",
pattern: "rw/build-analysis.log"),
issues(id: "${env.PRODUCT}-coverity-defects",
name: "${PRODUCT} Coverity Defects",
pattern: "rw/warning-ng-defects-*.xml")]
)
}
}
Since we have fixed normally all of our GCC warnings, but we still have lots of Coverity defects, I wanted to apply a quality gate to the gcc tool, to reject pull requests with warnings, but not to the issues one, since we accept for now, pull requests still having Coverity defects.
But my understanding, from the reading of the pipeline syntax steps reference, is that a quality gate applies to the whole list of tools in recordIssues, and doesn't seem to be applicable to only one of the tools.
Here's what I had tried:
post {
always {
recordIssues(
blameDisabled: true,
forensicsDisabled: true,
tools: [gcc(id: "${env.PRODUCT}-static-gcc",
name: "${PRODUCT} GCC warnings",
pattern: "rw/build-analysis.log",
qualityGates: [[threshold: 1, type: 'TOTAL', unstable: true]]),
issues(id: "${env.PRODUCT}-coverity-defects",
name: "${PRODUCT} Coverity Defects",
pattern: "rw/warning-ng-defects-*.xml")]
)
}
}
But according to the documentation, it seems logical that it was just plainly ignored.
So, is there a way to apply a quality gate to only one of a tool in a tools list, in recordIssues?

So in the end, the solution was simple and "only" consisted in using recordIssues twice. This worked for me:
post {
always {
recordIssues(
blameDisabled: true,
forensicsDisabled: true,
tool: issues(id: "${env.PRODUCT}-coverity-defects",
name: "${PRODUCT} Coverity Defects",
pattern: "rw/warning-ng-defects-*.xml")
)
recordIssues(
blameDisabled: true,
forensicsDisabled: true,
qualityGates: [[threshold: 1, type: 'TOTAL', unstable: false]],
tool: gcc(id: "${env.PRODUCT}-static-gcc",
name: "${PRODUCT} GCC warnings",
pattern: "rw/build-analysis.log")
)
}
}
Thank you very much #MattSchuchard!

Related

Jenkinsfile / recordIssues create "filter" Array from file

We are using recordIssues from the "Warnings Next Generation Plugin" to Visualise Results that have been generated by the Trivy-Dockerimage-Scanner "aquasec/trivy". This tool can scan docker images against several CVE Databases.
The results are generated with
recordIssues(enabledForFailure: true,
aggregatingResults: true,
qualityGates: [[threshold: 1, type: 'TOTAL_ERROR', unstable: true]],
filters: [
excludeType('CVE-2017-15095'),
excludeType('CVE-2017-17485'),
],
tools: [issues(pattern: '*-issues.json', reportEncoding: 'UTF-8')])
I have approx. 180 CVEs that can be ignored as they are not affecting our Software. I would like to generate the array dynamically from a external file instead of adding 180 times excludeType('CVE-2017-17485'),and so on ...
Update
Based on Matts kind answer the script looks like this now :
stages {
stage("Scan") {
steps {
sh """#!/bin/bash
echo "do some stuff"
"""
script {
List<String> cve = readYaml file: 'ignored_cves.yaml'
}
recordIssues(enabledForFailure: true,
aggregatingResults: true,
qualityGates: [[threshold: 1, type: 'TOTAL_ERROR', unstable: true]],
filters: cve.collect {excludeType(it)},
tools: [issues(pattern: '*-issues.json', reportEncoding: 'UTF-8')])
}
}
}
But when running in Jenkins I receive :
groovy.lang.MissingPropertyException: No such property: cve for class: groovy.lang.Binding
at groovy.lang.Binding.getVariable(Binding.java:63)
at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onGetProperty(SandboxInterceptor.java:271)
I am wondering how to access cve Groovy Object inside recordIssue. Possibly I have to do this inside a Jenkins Library.

How do I dynamically load a Jenkins pipeline library from Perforce? [duplicate]

In continuation to jenkins-pipeline-syntax-for-p4sync - I am not able to get the "Poll SCM" option work for my pipeline job.
Here is my configuration:
"Poll SCM" is checked and set to poll every 10 minutes
Pipeline script contains the following:
node ('some-node') // not actual value
{
stage ('checkout')
{
checkout([
$class: 'PerforceScm',
credential: '11111111-1111-1111-1111-11111111111', // not actual value
populate: [
$class: 'AutoCleanImpl',
delete: true,
modtime: false,
parallel: [
enable: false,
minbytes: '1024',
minfiles: '1',
path: '/usr/local/bin/p4',
threads: '4'
],
pin: '',
quiet: true,
replace: true
],
workspace: [
$class: 'ManualWorkspaceImpl',
charset: 'none',
name: 'jenkins-${NODE_NAME}-${JOB_NAME}',
pinHost: false,
spec: [
allwrite: false,
clobber: false,
compress: false,
line: 'LOCAL',
locked: false,
modtime: false,
rmdir: false,
streamName: '',
view: '//Depot/subfolder... //jenkins-${NODE_NAME}-${JOB_NAME}/...' // not actual value
]
]
]
)
}
stage ('now do something')
{
sh 'ls -la'
}
}
Ran the job manually once
Still, polling does not work and job does not have a "Perforce Software Polling Log" link like a non-pipelined job has when configuring the perforce source and Poll SCM in the GUI.
It's like the PerforceSCM is missing a poll: true setting - or i'm doing something wrong.
Currently I have a workaround in which I poll perforce in a non-pipelined job which triggers a pipelined job, but then I have to pass the changelists manually and I would rather the pipeline job to do everything.
edit: versions
jenkins - 2.7.4
P4 plugin - 1.4.8
Pipeline plugin - 2.4
Pipeline SCM Step plugin - 2.2
If you go to the Groovy snippet generator and check the "include in polling" checkbox, you'll see that the generated code includes a line item for it:
checkout([
poll: true,
As an aside, you may run into problems at the moment using ${NODE_NAME} in your workspace name. The polling runs on the master, so it might not properly find the change number of your previous build. If that's the case, I know a fix for it should be coming shortly.
After updating all the plugins to latest (as of this post date) and restarting the jenkins server - the polling appears to be working with the exact same configuration (job now has the poll log link).
I'm not sure what exactly resolved the issue - but I consider it resolved.

Getting Percentage coverage from Karma.js with Jenkins

I am running karma.js for unit testing and integrating with jenkins pipeline. My goal is to read the type of error thrown from Karma, if it is error related to percentage i want to terminate the job, otherwise continue even if there are other errors like unit test failures etc. (this is a requirement and there are reasons for it.)
I didn't find a way to do this. Any thoughts are appreciated!
karma start ibx-test/olb/karma.conf.js --browsers PhantomJS --log-level warn --single-run
coverageReporter: {
type: 'lcov',
dir: 'unit-tests/coverage/',
check: {
global: {
lines: 100 //This is just for testing
}
}
}
16:17:43 [Unit Test] 09 03 2017 21:17:43.024:ERROR [coverage]:
PhantomJS 2.1.1 (Linux 0.0.0): Coverage for lines (90.33%) does not
meet global threshold (100%)
EDIT: I found "Process xUnit test result report" in pipeline syntax under "Build step", can i use this somehow? Is there correlation between karma reports and xUnit?
i found a way to do this. The "Process xUnit test result report" helps doing exactly this. I checkedout the Pipeline syntax and it gave me the below script and it worked.
step([$class: 'XUnitBuilder', testTimeMargin: '3000', thresholdMode: 1,
thresholds: [
[$class: 'FailedThreshold', failureNewThreshold: '',
failureThreshold: '2', unstableNewThreshold: '',
unstableThreshold: '1'],
[$class: 'SkippedThreshold', failureNewThreshold: '',
failureThreshold: '', unstableNewThreshold: '',
unstableThreshold: '']],
tools: [[$class: 'JUnitType', deleteOutputFiles: false,
failIfNotNew: false, pattern: 'ibx-test/reports/unit-tests/PhantomJS_2.1.1_(Linux_0.0.0)/ibx-test/reports/unit-tests/*.xml',
skipNoTestFiles: false, stopProcessingIfError: false]]])
thresholdMode: means number of (failed or skipped) tests will be used for threshold. 1 for number and 2 for percent. I used 1, so i could just make one test fail and i get the desired result.
FailedThreshold: Is the class to be used for threshold for failures.
SkippedThreshold: can be used for skipped tests. I am not using it yet.
I am not paying attention on other parameters as of now for this testing.
As you can see my value is 2 (failureThreshold: '2'). As soon as i have 2 tests failing, the build fails and terminates.

jenkins pipeline warnings graph

I started to migrate some jobs in jenkins to pipeline execution.
Is there any chance to see the warnings graph in multi branch pipeline jobs? Within my older projects I can configure the graphs with "Configure the trend graph" option. These option will crash in the pipeline syntax tool.
Is there any option to make the graph visible?
I have in my Jenkinsfile:
stage ('Warnings gcc') {
step([$class: 'WarningsPublisher', canComputeNew: false, canResolveRelativePaths: false, defaultEncoding: '', excludePattern: '', healthy: '', includePattern: '', messagesPattern: '', parserConfigurations: [[parserName: 'GNU Make + GNU C Compiler (gcc)', pattern: 'error_and_warnings.txt']], unHealthy: ''])
}
stage ('Warnings clang') {
step([$class: 'WarningsPublisher', canComputeNew: false, canResolveRelativePaths: false, defaultEncoding: '', excludePattern: '', healthy: '', includePattern: '', messagesPattern: '', parserConfigurations: [[parserName: 'Clang (LLVM based)', pattern: 'error_and_warnings_clang.txt']], unHealthy: ''])
}
The reports will be generated but no graph is displayed.
UPDATE: Now it still did not work but it is also impossible to use the snipped generator for the warnings plugin.
Entering a file name in the snipped generator for the warning plugin results in a java null pointer exception:
javax.servlet.ServletException: java.lang.NullPointerException
at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:796)
at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876)
at org.kohsuke.stapler.MetaClass$5.doDispatch(MetaClass.java:236)
at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:58)
at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746)
at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876)
at org.kohsuke.stapler.MetaClass$10.dispatch(MetaClass.java:362)
some more lines follows ...
Mentioned in the revision log for the warnings plugin V 4.56:
Fixed deactivation of trend graphs (using the analysis collector plug-in)
But I use 4.57 and it still did not display any graph!
I posted the problem also to jenkins user list (no feedback for weeks) and also added bug report and bug report
Can anyone reproduce the problem or is the multi-branch pipeline simply still broken at all? Seems to be that there are not so much users for this plugin...
These issues have been resolved with the latest versions of both Jenkins, the pipelines plugin[s], and the plugins you have mentioned above.
Additionally, the bugs you specifically reported have been resolved:
[FIXED JENKINS-39553] Make GitHub plugin BuildableItem aware (#153)
[FIXED JENKINS-39532] Do not access the workspace for pipelines
Update your Jenkins instance and all of your plugins (some of them have interdependencies on others) and after the suggested restart you should be able to display the graph successfully.

Cobertura code coverage report for jenkins pipeline jobs

I'm using the pipeline plugin for jenkins and I'd like to generate code coverage report for each run and display it along with the pipeline ui. Is there a plugin I can use to do that(e.g. Cobertura but it doesn't seem to be supported by pipeline)?
There is a way to add a pipeline step to publish your coverage report but it doesn't show under the BlueOcean interface. It will show fine in the normal UI.
pipeline {
agent any
stages {
...
}
post {
always {
junit '**/nosetests.xml'
step([$class: 'CoberturaPublisher', autoUpdateHealth: false, autoUpdateStability: false, coberturaReportFile: '**/coverage.xml', failUnhealthy: false, failUnstable: false, maxNumberOfBuilds: 0, onlyStable: false, sourceEncoding: 'ASCII', zoomCoverageChart: false])
}
}
}
Note that one of the parameters to the Cobertura plugin is the XML that it will use ('**/coverage.xml' in the example).
If you are using python, you will want to use something like:
nosetests --with-coverage --cover-xml --cover-package=pkg1,pkg2 --with-xunit test
Nowadays you can also use the cobertura command directly in a Jenkinsfile
stage ("Extract test results") {
cobertura coberturaReportFile: 'path-to/coverage.xml'
}
source: https://issues.jenkins-ci.org/browse/JENKINS-30700
The answer from hwjp is correct, however there are extra parameters that you can add to the command that are not easy to find.
Once you have installed the Cobertura plugin, you can find the cobertura step options in
Job Dashboard Page -> Pipeline Syntax -> Steps Reference
There's also a snippet generator which is really useful to get started at
Job Dashboard Page -> Pipeline Syntax
example command:
cobertura coberturaReportFile: 'coverage.xml', enableNewApi: true, lineCoverageTargets: '80, 60, 70'
enableNewApi is a good one to set to true, as the new API is much prettier :D
setting coverage targets will automatically fail the job if the code coverage is too low
Generate report using command line cobertura-report in specified directory and attach results as artifacts.
cobertura-report [--datafile file] --destination dir [--format
html|xml] [--encoding encoding] directory [--basedir dir]

Resources