I have one Jenkinsfile with two testing stages.
stage("test 1") {
dir("test 1") {
git url: "repo"
sh "${mvnHome}/bin/mvn -f pom.xml (...)"
git url: "repo"
sh "${mvnHome}/bin/mvn -f pom.xml (...)"
step([$class: 'Publisher', reportFilenamePattern: "${workspace}/(path)/testng-results-mod.xml"])
}
}
stage("test 2") {
dir("test_2") {
git url: "repo"
sh "${mvnHome}/bin/mvn -f pom.xml (...)"
git url: "repo"
sh "${mvnHome}/bin/mvn -f pom.xml (...)"
step([$class: 'Publisher', reportFilenamePattern: "${workspace}/(path)/testng-results-mod.xml"])
}
}
This is working fine. Both stages generates testng results file. But unfortunately i end up with just one xml result file, because first file is being overwriten by second test. Both test use same name.
There are two link in jenkins (build) to TestNG results, but they all lead to this same xml file. In build directory there is also just one file.
Is there any option to generate each test under different name? So they both will be avaliable in the end?
Ok. So after testing... it looks like you can have multiple xml files in output directory. TestNG pluging is displaying them as one report.
So Ive just added option to rename output file to _1.xml and _2.xml and it works.
Related
I am using jenkins scripted file.
I have .key file stored in jenkins files ( Where all env files are present ).
And I need to copy that file to code folder.
Like i want to store device.key in src/auth/keys.
Then will run test on code in pipeline.
I am using scripted Jenkinsfile. And i am unable to find any way to this.
node{
def GIT_COMMIT_HASH
stage('Checkout Source Code and Logging Into Registry') {
echo 'Logging Into the Private ECR Registry'
checkout scm
sh "git rev-parse --short HEAD > .git/commit-id"
GIT_COMMIT_HASH = readFile('.git/commit-id').trim()
# NEED TO COPY device.key to /src/auth/key
}
stage('TEST'){
nodejs(nodeJSInstallationName:'node'){
sh 'npm install'
sh 'npm test'
}
}
}
How I solved this:
I installed Config File Provider Plugin
I added the files as custom files for each environment
In the JenkinsFile I replace the configuration file from the project with the one comming from jenkins:
stage('Add Config files') {
steps {
configFileProvider([configFile(fileId: 'ID-of-Jenkins-stored-file', targetLocation: 'relative-path-to-destination-file-in-the-project')]) {
// some block , maybe a friendly echo for debugging
} } }
Please see the plugin doc as it is capable of replacing tokens in XML and json files and many others.
I have a git url maven project which I want to only deal one of its submodule.
I write in pipeline script :
...
stage("mvn build") {
steps {
script {
sh "mvn package -DskipTests=true"
}
}
}
error arise: The goal you specified requires a project to execute but there is no POM in this directory (/xx/jenkins/workspace/biz-commons_deploy). so I add command :
sh "cd cmiot-services/comm" # subdir of biz-commons_deploy
def PWD = pwd();
echo "##=${PWD} "
sh "mvn package -DskipTests=true"
not work, print ##=/root/.jenkins/workspace/biz-commons_deploy, the error is the same as before .
how can I solve this problem and why the echo and error use different user space?
I make it using sh "mvn -f cmiot-services/comm/pom.xml package -DskipTests=true",still not know where this two user path come from and why sh cd not work.
steps {
sh '''
# list items in current directory to see where is your pom.xml
ls -l
# run job by comment out following two lines, if you don't know the
# relative path of folder where pom.xml insides exactly
cd <folder where pom.xml insides>
mvn package -DskipTests=true
'''
}
As Yong answered, every sh steps are independent, imagine Jenkins is opening a new ssh connection on your slave each time.
For your script, instead of a workaround with sh, why not using build in dir step ?
Something like this should do it :
stage("mvn build") {
steps {
script {
dir('cmiot-services/comm') {
sh "mvn package -DskipTests=true"
}
}
}
}
when you are executing Jenkins Pipline, the current directory is the Jenkins workspace directory.
You can add a step to clone the repo that your code is in (granted that the environment you are running the Jenkins instance is able to connect to your repo and clone).
You can then navigate into the directory that has the pom.xml. And finally execute the maven command.
...
stage("Clone Repo") {
steps {
script {
sh "git clone ssh://git#bitbucket.org:repo/app.git"
}
}
}
stage("mvn build") {
steps {
script {
sh "cd app/"
sh "pwd"
sh "mvn package -DskipTests=true"
}
}
}
I have asked this question on the Jenkins mailing list.
I have an upstream component, libraryA, I build, archive, and deploy via a Maven job, jobA. This works great. I have a downstream Maven job, jobB, that has a dependency on libraryA. This also works great, except…
I have a completely separate pipeline job, pipelineA specified by Jenkinsfile. Within that Jenkinsfile, I build a specific branch of libraryA I don’t want archived or deployed. In my Jenkinsfile I have “withMaven(mavenLocalRepo: ‘libraryA/.repository’, publisherStrategy: ‘EXPLICIT’)”, and inside that, “sh “””[…]mvn clean package sonar:sonar[…]””” (Any typos here are probably the fault of my typing here as I did not copy-paste. There are no errors from Jenkins when executing these steps.) I have also tried “options: [artifactsPublisher(disabled: true)]” in place of “publisherStrategy: ‘EXPLICIT’” and had the same results. I have verified when pipelineA builds libraryA, it does NOT get deployed to my remote Maven repository, and I expect it not to get deployed there. Good.
So, what happens?
Well, if I build pipelineA followed by jobB, jobB gets its copy of libraryA from pipelineA, causing the build to fail. If I then run jobA, jobB succeeds as expected.
I could change the version of libraryA in the branch pipelineA builds, but I’d rather not do that as it’s not correct for my particular use case. What else could I do? What did I miss? (I do not admin this Jenkins instance, so my access is limited in that respect.)
For clarity here is an approximation of the pipeline's Jenkinsfile:
#!groovy
pipeline {
environment {
GIT_CREDENTIAL_ID = 'git_credential_name'
}
agent any
tools {
maven 'internal_maven'
jdk 'openjdk'
git 'internal_git'
}
stages {
stage('Build Parent') {
steps {
checkoutRepos([[checkoutDir: 'parent', branch: 'gold', url: 'ssh://git#fake-git-url/parent.git']], env.GIT_CREDENTIAL_ID)
withMaven(publisherStrategy: 'EXPLICIT') {
sh """
cd parent
mvn -U clean verify
"""
}
stash name: 'parent', includes: 'pom.xml'
}
}
stage('Build libraryParent') {
steps {
unstash 'parent'
checkoutRepos([[checkoutDir: 'libraryParent', branch: 'gold', url: 'ssh://git#fake-git-url/libraryParent.git']], env.GIT_CREDENTIAL_ID)
withMaven(mavenLocalRepo: 'libraryParent/.repository', publisherStrategy: 'EXPLICIT') {
sh """
mvn install:install-file -Dpackaging=pom -Dfile=pom.xml -DpomFile=pom.xml
cd libraryParent
mvn clean verify -P jacoco sonar:sonar -U -Dsonar.host.url=https://fake-sonar-url -Dsonar.scm.provider=git
"""
}
dir('libraryParent/libraryA/target/') {
stash name: 'libraryA', includes: 'libraryA-1000-SNAPSHOT.jar'
}
}
}
stage('Build appA') {
steps {
sonarAnApp 'appA'
}
}
stage('Build appB') {
steps {
sonarAnApp 'appB'
}
}
stage('Build appC') {
steps {
sonarAnApp 'appC'
}
}
}
}
def sonarAnApp(final String appName) {
unstash 'parent'
unstash 'libraryA'
checkoutRepos([[checkoutDir: appName, branch: 'gold', url: "ssh://git#fake-git-url/${appName}.git"]], env.GIT_CREDENTIAL_ID)
withMaven(mavenLocalRepo: "$appName/.repository", publisherStrategy: 'EXPLICIT') {
sh """
mvn install:install-file -Dpackaging=pom -Dfile=pom.xml -DpomFile=pom.xml
mvn install:install-file -Dfile=libraryA-1000-SNAPSHOT.jar
cd $appName
mvn clean verify -P jacoco sonar:sonar -U -Dsonar.host.url=https://fake-sonar-url -Dsonar.scm.provider=git
"""
}
}
I am trying to zip the folders which are created as output of my jenkins pipeline job using pipeline script. By googling i came to know the Jenkins
Pipeline Utility Steps - zip zipFile
https://jenkins.io/doc/pipeline/steps/pipeline-utility-steps/#code-zip-code-create-zip-file to zip folders/files but could not get exact pipeline syntax to zip.
In my job workspace, I have a folder by name 'Test' which has 2 sub folders as 'Test1', 'Test2'. Each sub folder will have .dll files. So, I would like to zip entire 'Test' folder with all subfolder.
node(Jenkinks_1)
{
echo "ZIP"
zip zipFile: 'Test.zip', dir:'C:\\workspace\\Build_Sample\\Test'
echo "END - ZIP"
}
Below are the Console Output from Jenkins:
Started by user XXXXX
[Pipeline] node
Running on Jenkinks_1 in C:\workspace\Build_Sample
[Pipeline] {
[Pipeline] echo
ZIP
[Pipeline] echo
END - ZIP
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS
Looking for some guidance to zip the folders using pipeline syntax. Appreciate your inputs.
I wanted to zip some files as output of my jenkins pipeline job
First, try the same operation in stages and step, as in here:
pipeline {
agent any
stages {
stage ('push artifact') {
steps {
sh 'mkdir archive'
sh 'echo test > archive/test.txt'
zip zipFile: 'test.zip', archive: false, dir: 'archive'
archiveArtifacts artifacts: 'test.zip', fingerprint: true
}
}
...
}
It uses archiveArtifacts to record the result.
If using an absolute path does now work, try a relative one ('..')
As seen by the OP Sri, zip zipFile is part of, and requires the JENKINS Pipeline Utility Steps Plugin.
See "Implemented Steps".
Regarding the syntax to be used for multi-criteria file selection, NicolasW notes in the comments that the documentation is vague: "use glob ant-style syntax"...
He got it to work though, with a basic coma separated syntax.
E.g.
zip zipFile: 'test.zip', archive: false, glob: 'config-/**/,scripts/**/*.*
But, as noted by Tanvir in the comments, issue 44078 means you need to replace zip by:
script{ zip zipFile: 'test.zip', archive: false, dir: 'archive' }
Meaning you need to use a script block.
Was able to Zip after installing the Pipeline Utility Steps plugin.
I came across this because zip was ... not installed on the host.
Reminder to self : If you need zip, install it first.
sudo yum install zip
you can just use sh (jenkins server need install zip);
sh '''
zip -r algo.zip algo
'''
pipeline script like this
node {
stage('Clean'){
cleanWs()
}
stage('Checkout') {
git branch: 'develop', url: 'ssh://user#ip:29418/prj.git'
}
stage('Zip') {
dir('algo-python') {
sh '''
zip -r algo.zip algo
'''
}
}
stage('Upload zip'){
dir('algo-python') {
sh '''
source /etc/profile
export HADOOP_USER_NAME=dev
hdfs dfs -put -f algo.zip /user/dev/zipfile/
'''
}
}
}
I'm using Jenkins 2.x with a Jenkinsfile to run a pipeline.
I have built a job using Jenkinsfile and I want to invoke the Analysis Collector Plugin so I can view the report.
Here is my current Jenkinsfile:
#!groovy
node {
stage 'Build '
echo "My branch is: ${env.BRANCH_NAME}"
sh 'cd gitlist-PHP && ./gradlew clean build dist'
stage 'Report'
step([$class: 'JUnitResultArchiver', testResults: 'gitlist-PHP/build/logs/junit.xml'])
step([$class: 'hudson.plugins.checkstyle.CheckStylePublisher', checkstyle: 'gitlist-PHP/build/logs/phpcs.xml'])
step([$class: 'hudson.plugins.dry.DryPublisher', CopyPasteDetector: 'gitlist-PHP/build/logs/phpcpd.xml'])
stage 'mail'
mail body: 'project build successful',
from: 'siregarpandu#gmail.com',
replyTo: 'xxxx#yyyy.com',
subject: 'project build successful',
to: 'siregarpandu#gmail.com'
}
I want to invoke invoke Checkstyle, Junit and DRY plugin from Jenkins. How do I configure these plugins in the Jenkinsfile? Do these plugins support pipelines?
The following configuration works for me:
step([$class: 'CheckStylePublisher', pattern: 'target/scalastyle-result.xml, target/scala-2.11/scapegoat-report/scapegoat-scalastyle.xml'])
For junit configuration is even easier:
junit 'target/test-reports/*.xml'
step([$class: 'hudson.plugins.checkstyle.CheckStylePublisher', checkstyle: 'gitlist-PHP/build/logs/phpcs.xml'])
Also according to source code repo, the argument 'checkstyle' should be named 'pattern'.
Repo:
https://github.com/jenkinsci/checkstyle-plugin/blob/master/src/main/java/hudson/plugins/checkstyle/CheckStylePublisher.java#L42
This is how I handle this:
PMD
stage('PMD') {
steps {
sh 'vendor/bin/phpmd . xml build/phpmd.xml --reportfile build/logs/pmd.xml --exclude vendor/ || exit 0'
pmd canRunOnFailed: true, pattern: 'build/logs/pmd.xml'
}
}
PHPCPD
stage('Copy paste detection') {
steps {
sh 'vendor/bin/phpcpd --log-pmd build/logs/pmd-cpd.xml --exclude vendor . || exit 0'
dry canRunOnFailed: true, pattern: 'build/logs/pmd-cpd.xml'
}
}
Checkstyle
stage('Checkstyle') {
steps {
sh 'vendor/bin/phpcs --report=checkstyle --report-file=`pwd`/build/logs/checkstyle.xml --standard=PSR2 --extensions=php --ignore=autoload.php --ignore=vendor/ . || exit 0'
checkstyle pattern: 'build/logs/checkstyle.xml'
}
}
JDepend
stage('Software metrics') {
steps {
sh 'vendor/bin/pdepend --jdepend-xml=build/logs/jdepend.xml --jdepend-chart=build/pdepend/dependencies.svg --overview-pyramid=build/pdepend/overview-pyramid.svg --ignore=vendor .'
}
}
The full example you can find here: https://gist.github.com/Yuav/435f29cad03bf0006a85d31f2350f7b4
Reference links
https://jenkins.io/doc/pipeline/steps/
It appears that the plugins need to be modified to support working as Pipeline Steps, so if they have not been updated, they don't work.
Here is a list of compatible plugins that have been updated:
https://github.com/jenkinsci/pipeline-plugin/blob/master/COMPATIBILITY.md
And here is the documentation about how the plugins need to be updated to support Pipelines:
https://github.com/jenkinsci/pipeline-plugin/blob/master/DEVGUIDE.md