I'm working on implementing Checkmarx scans in our code repository. I'm using Jenkins and the CheckMarx plugin to accomplish this task. There are some folders I want to exclude from the scan; referencing the Jenkins plugin documentation it seems like all I have to do is add the folder names in the 'excludeFolders' field. However that doesn't appear to work, or maybe I don't have the value entered correctly.
I've tried '/test', 'test/', '!/test//*' but none work and the folder is still registered and zipped before it is uploaded to our CheckMarx server.
Below is what I have in my pipeline:
stage("Running CheckMarks for Layer"){
steps{
script{
def layer_dir = readFile file: 'layer-list'
def layer_list = layer_dir.split('\\r?\\n')
println (layer_list)
layer_list.each { layer ->
print (layer)
dir("${env.WORKSPACE}/layers/layer-name/$layer"){
step([
$class: 'CxScanBuilder',
comment: 'Layer scanning',
credentialsId: 'XXXX',
excludeFolders: 'test',
exclusionsSetting: 'global',
failBuildOnNewResults: false,
failBuildOnNewSeverity: 'MEDIUM',
filterPattern: '''!**/_cvs/**/*, !Checkmarx/Reports/*.*''',
fullScanCycle: 10,
incremental: true,
fullScansScheduled: true,
generatePdfReport: true,
preset: '36',
teamPath: "\\path\\to\\codebase",
projectName: "$layer",
sastEnabled: true,
sourceEncoding: '1',
vulnerabilityThresholdResult: 'FAILURE',
waitForResultsEnabled: true
])
}
}
}
}
}
Any suggestions on how to exclude the 'test' folder?
You should change your exclusionsSetting to 'job' instead of 'global', we can't override the global configurations.
Then you can add more filters in the filterPattern.
the filtering is really flakey - did you have any luck????
try add it to the filterPattern as !Test/*.* as well and play around with that...
Modify like below to exclude both target and test folders.
excludeFolders: 'target, test'
Jenkins console log:
[Cx-Debug]: Excluded Dir: src/test
[Cx-Debug]: Excluded Dir: target
If you are running on Windows you need to use the following pattern: !**\\test\\**\\*
On Linux: !**/test/**/*
Related
I want to build "zip" file and exclude multiple files: Jenkinsfile and test1.txt
I am able to exclude one specific file. This works:
stage ('First stage') {
steps {
zip zipFile: 'test.zip', archive: false, exclude: 'Jenkinsfile'
}
But how I can add to exclude test1.txt file as well?
I tried:
exclude: 'Jenkinsfile', 'test1.txt'
exclude: 'Jenkinsfile' 'test1.txt'
But it doesn't work!!!
Suggested solution on this link How to exclude Jenkinsfile and automation scripts in zip file pipeline utility plugin in Jenkins pipeline is NOT APPLICABLE!!!
Because exclude option is supported as explained on this link:
https://www.jenkins.io/doc/pipeline/steps/pipeline-utility-steps/#zip-create-zip-file
Thank you!!!
I am trying to use the artifacts created in the workspace post jenkins build in a postbuild shell script.
I am not able to use them as workspace artifacts are automatically getting deleted before it comes postbuild script.
Could anyone help me to address this?
When the post-build stage is running, your workspace is already removed. When you think of it, your regular stage and post-build stage may even be running on different nodes, so there can't be any expectation that the files are in your workspace.
To access your artifacts in the post-build stage, you need to fetch them manually, e.g. by using Copy Artifact plugin:
post {
always {
// fetch artifacts of this job and this number to $WORKSPACE
step([
$class: 'CopyArtifact',
filter: '*',
fingerprintArtifacts: true,
optional: true,
projectName: "${JOB_NAME}",
selector: [$class: 'SpecificBuildSelector',
buildNumber: "${BUILD_NUMBER}"]
])
script {
try {
for(file in findFiles(glob: "*")) {
println "Found file ${file}"
}
} catch(error) {
println "Failed to find files"
}
}
}
}
In continuation to jenkins-pipeline-syntax-for-p4sync - I am not able to get the "Poll SCM" option work for my pipeline job.
Here is my configuration:
"Poll SCM" is checked and set to poll every 10 minutes
Pipeline script contains the following:
node ('some-node') // not actual value
{
stage ('checkout')
{
checkout([
$class: 'PerforceScm',
credential: '11111111-1111-1111-1111-11111111111', // not actual value
populate: [
$class: 'AutoCleanImpl',
delete: true,
modtime: false,
parallel: [
enable: false,
minbytes: '1024',
minfiles: '1',
path: '/usr/local/bin/p4',
threads: '4'
],
pin: '',
quiet: true,
replace: true
],
workspace: [
$class: 'ManualWorkspaceImpl',
charset: 'none',
name: 'jenkins-${NODE_NAME}-${JOB_NAME}',
pinHost: false,
spec: [
allwrite: false,
clobber: false,
compress: false,
line: 'LOCAL',
locked: false,
modtime: false,
rmdir: false,
streamName: '',
view: '//Depot/subfolder... //jenkins-${NODE_NAME}-${JOB_NAME}/...' // not actual value
]
]
]
)
}
stage ('now do something')
{
sh 'ls -la'
}
}
Ran the job manually once
Still, polling does not work and job does not have a "Perforce Software Polling Log" link like a non-pipelined job has when configuring the perforce source and Poll SCM in the GUI.
It's like the PerforceSCM is missing a poll: true setting - or i'm doing something wrong.
Currently I have a workaround in which I poll perforce in a non-pipelined job which triggers a pipelined job, but then I have to pass the changelists manually and I would rather the pipeline job to do everything.
edit: versions
jenkins - 2.7.4
P4 plugin - 1.4.8
Pipeline plugin - 2.4
Pipeline SCM Step plugin - 2.2
If you go to the Groovy snippet generator and check the "include in polling" checkbox, you'll see that the generated code includes a line item for it:
checkout([
poll: true,
As an aside, you may run into problems at the moment using ${NODE_NAME} in your workspace name. The polling runs on the master, so it might not properly find the change number of your previous build. If that's the case, I know a fix for it should be coming shortly.
After updating all the plugins to latest (as of this post date) and restarting the jenkins server - the polling appears to be working with the exact same configuration (job now has the poll log link).
I'm not sure what exactly resolved the issue - but I consider it resolved.
I'm exploring Jenkins 2.0 pipelines. So far my file is pretty simple.
node {
stage "checkout"
git([url:"https://github.com/luxengine/math.git"])
stage "build"
echo "Building from pipeline"
}
I can't seem to find any way to set the directory that git will checkout to. I also can't find any kind of documentation related to that. I found https://jenkinsci.github.io/job-dsl-plugin/ but it doesn't seem to match what I see on other tutorials.
Clarification
Looks like you are trying to configure Pipeline job (formerly known as Workflow). This type of job is very distinct from Job DSL.
The purpose of Pipeline job is to:
Orchestrates long-running activities that can span multiple build slaves. Suitable for building pipelines (formerly known as workflows) and/or organizing complex activities that do not easily fit in free-style job type.
Where as Job DSL:
...allows the programmatic creation of projects using a DSL. Pushing job creation into a script allows you to automate and standardize your Jenkins installation, unlike anything possible before.
Solution
If you want to checkout your code to specific directory then replace git step with more general SCM checkout step.
Final Pipeline configuration should look like that:
node {
stage "checkout"
//git([url:"https://github.com/luxengine/math.git"])
checkout([$class: 'GitSCM',
branches: [[name: '*/master']],
doGenerateSubmoduleConfigurations: false,
extensions: [[$class: 'RelativeTargetDirectory',
relativeTargetDir: 'checkout-directory']],
submoduleCfg: [],
userRemoteConfigs: [[url: 'https://github.com/luxengine/math.git']]])
stage "build"
echo "Building from pipeline"
}
As a future reference for Jenkins 2.0 and Pipeline DSL please use built-in Snippet Generator or documentation.
This can be done by using the directive of dir:
def exists = fileExists '<your target dir>'
if (!exists){
new File('<your target dir>').mkdir()
}
dir ('<your target dir>') {
git url: '<your git repo address>'
}
First make clear that you are using Jenkins Job DSL.
You can do this like this:
scm {
git {
wipeOutWorkspace(true)
shallowClone(true);
remote {
url("xxxx....")
relativeTargetDir('checkout-folder')
}
}
}
https://jenkinsci.github.io/job-dsl-plugin/
This above address gives you the chance simply to type in upper left aread for example 'scm' and than it will show in which relationships 'scm' can be used. Than you can select 'scm-freestylejob' and afterwards click on the '***' than you can see the details.
The general start point for Jenkins Job DSL is here:
https://github.com/jenkinsci/job-dsl-plugin/wiki
You can of course ask here on SO or on Google Forum:
https://groups.google.com/forum/#!forum/job-dsl-plugin
pipeline {
agent any
stages{
stage("Checkout") {
steps {
dir('def exists = fileNotExists \'git\'') {
bat label: '', script: 'sh "mkdir.sh'
}
dir ('cm') {
git branch: 'dev',
credentialsId: '<your credential id>',
url: '<yours git url>'
}
}
} //End of Checkout stage
stage("TestShellScript") {
steps {
bat label: '', script: 'sh "PrintNumber.sh"'
}
}
}//End of stages
} // End of pipeline
Note: cat mkdir.sh
#!/bin/bash
#Create a directory
mkdir git
You are using the Pipeline Plugin, not the Job DSL Plugin. In the Pipeline Plugin, if you want to define something, where there is not yet a function available in the Pipeline syntax, you can define it yourself.
How can I update a jira issue from within a Jenkinsfile (jenkins-worflow/pipeline)?
Is there a way I could use the Jira Issue Updater plugin as a step in the Jenkinsfile?
I know I could use the Jira RestAPI, but I'm trying to figure out if I can re-use the functionality provided by the jira-updater-issue.
What I'm looking for is a something similar to the example below calling Junit archiver, and atifact archiver, but calling jira updater.
node {
git url: 'https://github.com/jglick/simple-maven-project-with-tests.git'
def mvnHome = tool 'M3'
sh "${mvnHome}/bin/mvn -B -Dmaven.test.failure.ignore verify"
step([$class: 'ArtifactArchiver', artifacts: '**/target/*.jar', fingerprint: true])
step([$class: 'JUnitResultArchiver', testResults: '**/target/surefire-reports/TEST-*.xml'])
}
The Jira Plugin is compatible with Pipeline.
This should work:
step([$class: 'hudson.plugins.jira.JiraIssueUpdater',
issueSelector: [$class: 'hudson.plugins.jira.selector.DefaultIssueSelector'],
scm: [$class: 'GitSCM', branches: [[name: '*/master']],
userRemoteConfigs: [[url: 'https://github.com/jglick/simple-maven-project-with-tests.git']]]])
You can get a full reference in the built-in Pipeline Snippet Generator.
The JIRA Steps Plugin provides a more declarative way to update an existing Jira Ticket:
node {
stage('JIRA') {
# Look at IssueInput class for more information.
def testIssue = [fields: [ // id or key must present for project.
project: [id: '10000'],
summary: 'New JIRA Created from Jenkins.',
description: 'New JIRA Created from Jenkins.',
customfield_1000: 'customValue',
// id or name must present for issuetype.
issuetype: [id: '3']]]
response = jiraEditIssue idOrKey: 'TEST-01', issue: testIssue
echo response.successful.toString()
echo response.data.toString()
}
}
Since you would like to use the Jenkinsfile to define your pipeline, that should be the preferred way for you to go...
As this was way harder for me than it should be, here is a working example. This will update a custom field of a ticket with a specific value:
step([$class: 'IssueFieldUpdateStep',
issueSelector: [$class: 'hudson.plugins.jira.selector.ExplicitIssueSelector', issueKeys: ticket],
fieldId: field,
fieldValue: value
])
The snippet generator did not work for me. The variables ticket, field and value are all strings. Starting from this you can look for options here: https://www.jenkins.io/doc/pipeline/steps/jira/
Yes, seems like this page answers your question:
https://wiki.jenkins-ci.org/display/JENKINS/Jira+Issue+Updater+Plugin
After you install the plugin, add a build step, or pre/post build step to call this plugin
There you can give it the REST URL to your Jira server, the creds and the JQL to find the issues