I am running two sets of pytest tests, one for py27 and one for py37 (inside tox framework if it can help)
The test results are saved in two files: test_27.xml and test_37.xml
I'd like to differentiate between the Python versions in the Jenkins interface.
Is there any way to do that with Junit Jenkins plugin ?
Could you think of any workaround ?
Thanks !
I don't use Jenkins myself and have no idea what the Junit Jenkins plugin does, but as you mention tox: You can run the two environments in different pipelines running in parallel. They will each have their own reports then. You would define one pipeline calling tox -e py27 and the other calling tox -e py37 and you would have it all neatly separated.
The testenv would look something like this (see substitutions):
[testenv]
commands = pytest --junitxml=junit-{envname}.xml
Each pipeline would have their own reporting step, details about this can be glimpsed from this article by Dave Hunt.
This is the best solution I found so far: Change pytest testsuite name in xml report
Related
I am trying to use Jenkins' Build Number in the naming of a Log that I would want to be saved as a post build action
Will the below format work
C:\Jenkins\workspace\Jmeter_Jenkins_Test_Job\Jenkins_Results\"${env.BUILD_NUMBER}"results.jtl
As per Building a software project wiki article the environment variable you're looking for is BUILD_NUMBER and in case of Windows operating system you can access it as:
%BUILD_NUMBER%
so if you want to amend JMeter result file name to include build number you can do something like:
jmeter -n -t /path/to/test.jmx -l /path/to/result-%BUILD_NUMBER%-.jtl
and in the runtime the variable will be evaluated to the current Jenkins build number:
More information just in case: Continuous Integration 101: How to Run JMeter With Jenkins
Without having laid my hands on a Jenkins installation for quite some time:
Yes, you can do that and it has been done before!
You could do something like:
pipeline {
agent any
stages {
stage('test') {
sh 'path/to/jmeter.bat -n -t ${env.WORKSPACE}my_test.jmx -l my_test${env.BUILD_ID}_${env.BUILD_NUMBER}.jtl'
}
}
}
I would propose to create the HTML dashboard report first though and then publish that in Jenkins - you could use https://jenkins.io/doc/pipeline/steps/htmlpublisher/ to do that. Further you should avoid absolute paths in favor of using the WORKSPACE environment variable (see https://jenkins.io/doc/book/pipeline/jenkinsfile/#using-environment-variables for reference).
If you need some general idea of how to run the test via Jenkins you could have a look at https://code-maven.com/jenkins-pipeline-running-external-programs and https://jmeter.apache.org/usermanual/get-started.html#non_gui
If you already tried to achieve something and need more specific help, please come forward with some more detail.
I'm pretty new to Jenkins and trying to run or be able to run multiple jenkins jobs in parallel with different goals (inside my goals will have a mvn command, also cucumber #tags). Basically I want to run multiple cucumber tags on multiple jenkins jobs at the same time. From the research I've done so far looks like I have a few options - multijob or pipeline plugins..please advice. Thanks!
If you don't worry about any resource consumption, you can use separated jobs as below and also you can hook them each other or trigger them with one job via Jenkins features:
Example job configurations :
1. job :
mvn clean test -Dcucumber.options="--tags #Smoke"
2. job :
mvn clean test -Dcucumber.options="--tags #Regression"
I am using a Jenkins pipeline to process files for a UAT.
Stages are like:
Get the files
Put them in the right directory
Run bash script
Run another bash script
Run another bash script
Send results
I would like to be able to run 1 or more times this pipeline based on the number of EODs I want to run.
So if EODs are:
20180720, 20180721, 20180722
I want the pipeline to run 3 times.
So;
for eod in '20180720, 20180721, 20180722'
run pipeline $eod
done
I know you can launch it with the url and parameters like John mentions below, and I am looking for a more simple solution, if there is any.
How can I do this?
I am trying to write the test cases to validate the Jenkinsfile, But the load script function not working expecting the extension to be provided and throwing ResourceException exception loadScript("Jenkinsfile")
Is their better way to test the Jenkinsfile
The problem is that there are not enough tools for the development of pipelines. Pipelines is DSL and it imposes a restrictions.
There is an interesting approach to using flags. For example, test which defines outside pipeline(in job). If test=true, a pipeline change some "production" logic to "test" - select another agent, load artifacts into another repository, run another command and so on.
But recently appeared Pipeline Unit Testing Framework. It allows you to unit test Pipelines and Shared Libraries before running them in full. It provides a mock execution environment where real Pipeline steps are replaced with mock objects that you can use to check for expected behavior.
Useful links:
Jenkins World 2017: JenkinsPipelineUnit: Test your Continuous Delivery Pipeline
Pipeline Development Tools
You can validate your Declarative Pipeline locally thanks to Jenkins built-in features.This can be done using a Jenkins CLI command or by making an HTTP POST request with appropriate parameters.
The command is the following:
curl -s -X POST -F "jenkinsfile=<YourJenkinsfile" \
https://user:password#jenkins.example.com/pipeline-model-converter/validate
For a practical example follow this guide:
https://pillsfromtheweb.blogspot.com/2020/10/validate-jenkinsfile.html
Trying to use the nifty Jenkins Pipeline, I had problems finding out how to publish NUnit test results.
I am able to run the tests by specifying the following command in the pipeline script:
stage 'Test'
bat '"C:\\Program Files (x86)\\NUnit 2.6.4\\bin\\nunit-console-x86.exe" "ProjectName\\bin\\Release\\UnitTests.net.dll"'
But how to make Jenkins "publish" the test results is not obvious. The Snippet Generator only suggests junit, and that does not seem to work.
I used nunit plugin version 0.21 and was able to publish results using
stage("PublishTestReport"){
nunit testResultsPattern: 'TestResult.xml'
}
(TestResult.xml is at the root of jenkins workspace in this above example)
Investigating the NUnit plugin for Jenkins led me on to this issue, where I found the solution:
step([$class: 'NUnitPublisher', testResultsPattern: 'TestResult.xml', debug: false,
keepJUnitReports: true, skipJUnitArchiver:false, failIfNoResults: true])
Adding this to the pipeline script worked for me!
However, it seemed the following should also work (but at the present, apparently, it does not): Using the Snippet Generator, select this:
step: General Build Step
Publish NUnit test result report
This generates the following in the Pipeline script:
step <object of type hudson.plugins.nunit.NUnitPublisher>
This fails!