It looks like support for test results analyzer was added to the pipeline plugin from the jira issue below. I'm having trouble figuring out how to acutally implement the plugin using a pipeline script.
https://issues.jenkins-ci.org/browse/JENKINS-30522
Regardless of the jira issue, how can I run the test results analyzer through my pipeline?
When you add test reports to your pipeline script this works automatically. The "Test Results Analyzer" button shows up right away for jobs that have tests, including those that use the pipeline plugin.
For example when using the standard "junit" report plugin like this, it should work out of the box:
stage('Unit tests') {
steps {
sh 'unit-tests.sh'
}
post {
always {
junit 'path/to/report.xml'
}
}
}
Related
I recently tried to add a "Publish test results" step within my Jenkins pipeline, but was wondering where to see/retrieve the TestResults.xml file that should be generated while deploying.
For reference, here's the code I'm using, straight from the documentation:
stage("Publish NUnit Test Report") {
steps {
nunit testResultsPattern: 'TestResult.xml'
}
}
Thanks a lot for your help!
Background
After a lot of hard work we finally got a Jenkins CI pulling code from out GitHub repositories and are now doing Continuous Integration as well as Deployment.
We get the code and only deploy it if all the tests pass, as usual.
Now I have checked that there are a number of plugins for Java that besides running the tests, also do test coverage, like Cobertura.
But we don't use Java. We use Elixir.
In the Elixir world, we have excoveralls, which is a facade for the coveralls API. The coveralls API supports jenkins so it stands to reason I would find a Coveralls Plugin for Jenkins.
I was wrong. There is nothing.
Questions
So now I have a test coverage metric that is basically useless because I can't integrate it with Jenkins.
Are there any Erlang/Elixir plugins one can use with Jenkins for code coverage?
I also created a Issue in the projects ( which seems to be abandoned ... ) https://github.com/parroty/excoveralls/issues/167
I have a stage to publish the coverage on my Jenkinsfile. I'm not sure if that is the metric that you want but...
stage('Publish Coverage') {
when{
branch 'master'
}
steps {
publishHTML target: [
allowMissing: true,
alwaysLinkToLastBuild: true,
keepAll: true,
reportDir: 'cover',
reportFiles: 'excoveralls.html',
reportName: 'Coverage Report'
]
}
}
I have found 2 ways of doing this:
Using Hex package JUnit formatter together with junit post pipeline step
Using covertool together with Cobertura Jenkins pluing
Option 1
This solution works and is quite nice. It forces me to change the test_helper.exs but that is a minor inconvenience overall. It is nice but it only offers the most basic of reports and for me this is where it fails.
Option 2
The option I decided to go with. Yes, making the Jenkinsfile work for Cobertura was a nightmare, specially because in previous versions it was not even possible and because there is contradictory information scattered all over the place.
However, once you get that Jenkinsfile going, you get to rip those sweet reports from Cobertura. Cobertura was made with Java in mind, there is no two ways about it. In the reports you see things like Class coverage and such, but you can easily translate that do modules. The interface offers a lot more information and tracks coverage over time, which is something I actually want.
For future notice, here is my Jenkinsfile:
pipeline {
agent any
environment {
SOME_VAR = "/home/deployer"
}
stages {
stage("Build") {
steps {
sh "MIX_ENV=test mix do deps.get, deps.compile"
}
}
stage("Test") {
steps {
sh "mix test --cover"
}
}
stage("Credo"){
steps{
sh "mix credo --strict"
}
}
stage("Deploy"){
when{
expression{
env.BRANCH_NAME == "master"
}
}
steps{
sh '''
echo "Deploy with AWS or GCP or whatever"
'''
}
}
}
post{
always{
cobertura coberturaReportFile: "coverage.xml"
}
}
}
Of notice:
1. I am extremely Nazi with my code, so I also use Credo. You can further configure it as to not blow the entire pipeline because you missed a new line at the end of file but as I said, I am quite Nazi with my code.
2. The Deploy stage only runs if the pushed branch is Master. There are other ways of doing this, but I found it that having this way for a small project was good enough.
Overall I like covertools for now but I don't know if the first solution has the same potential. At least I didn't see it.
Hope this post helps!
Original thread:
https://elixirforum.com/t/excoveralls-plugin-for-jenkins-ci/18842
Another way to post coverage from Jenkins for Elixir project is using ExCoveralls option mix coveralls.post. This allows you to post the coverage from any host, including your Jenkins server. Based on the example on this Jenkins tutorial page, you can write in Jenkinsfile like this:
pipeline {
agent any
stages {
// Assuming all environment variables are set beforehand
stage('run unit test') {
steps {
sh 'echo "Run Unit Test and Post coverage"'
sh '''
MIX_ENV=test mix coveralls.post --token $COVERALLS_REPO_TOKEN --sha $GIT_COMMIT --branch $GIT_BRANCH --name "jenkins" --message $GIT_COMMIT_MSG
'''
}
}
}
}
According to the documentation in https://jenkinsci.github.io/job-dsl-plugin/#method/javaposse.jobdsl.dsl.helpers.wrapper.MavenWrapperContext.buildName
Following code should update build name in Build History in Jenkins jobs:
// define the build name based on the build number and an environment variable
job('example') {
wrappers {
buildName('#${BUILD_NUMBER} on ${ENV,var="BRANCH"}')
}
}
Unfortunately, it is not doing it.
Is there any way to change build name from Jenkins Job DSL script?
I know I can change it from Jenkins Pipeline Script but it is not needed for me in this particular job. All I use in the job is steps.
steps {
shell("docker cp ...")
shell("git clone ...")
...
}
I would like to emphasise I am looking for a native Jenkins Job DSL solution and not a Jenkins Pipeline Script one or any other hacky way like manipulation of environment variables.
I have managed to solve my issue today.
The script did not work because it requires build-name-setter plugin installed in Jenkins. After I have installed it works perfectly.
Unfortunately, by default jobdsl processor does not inform about missing plugins. The parameter enabling that is described here https://issues.jenkins-ci.org/browse/JENKINS-37417
Here's a minimal pipeline changing the build's display name and description. IMHO this is pretty straight forward.
pipeline {
agent any
environment {
VERSION = "1.2.3-SNAPSHOT"
}
stages {
stage("set build name") {
steps {
script {
currentBuild.displayName = "v${env.VERSION}"
currentBuild.description = "#${BUILD_NUMBER} (v${env.VERSION})"
}
}
}
}
}
It results in the following representation in Jenkins' UI:
setBuildName("your_build_name") in a groovyPostBuild step may do the trick as well.
Needs Groovy Postbuild Plugin.
I defined a Jenkins Declarative pipeline to CI/CD my project. I am using gradle as my build tool. However I don't want to use the Gradle Wrapper and check it int the VCS. So I planed on using the jenkins tools functionality as below so that I can update the version number if I need to in future. But it doesn't seem to work.
pipeline {
agent any
tools {
gradle "gradle-4.0"
}
stage("Compile") {
steps {
sh 'gradle project/build.gradle classes'
}
}
I get the error "script.sh: gradle: not found".
I tried to echo PATH and that doesn't contain the path of this autoinstalled gradle tool. Please help.
Looks like there is an issue on the gradle plugin for Jenkins on plugin version 1.26. Please see the link to the bug reported below.
https://issues.jenkins-ci.org/browse/JENKINS-42381
On my Jenkins pipleline I run unit tests on both: debug and release configurations. Each test configuration generates separate JUnit XML results file. Test names on both: debug and release configuration are same. Currently I use the following junit command in order to show test results:
junit allowEmptyResults: true, healthScaleFactor: 0.0, keepLongStdio: true, testResults: 'Test-Dir/Artifacts/test_xml_reports_*/*.xml'
The problem is that on Jenkins UI both: debug and release tests results are shown together and it is not possible to know which test (from debug or release configuration) is failed.
Is it possible to show debug and release tests results separately? If yes, how can I do that?
We run the same integration tests against two different configurations with different DB types. We use maven and the failsafe plugin, so I take advantage of the -Dsurefire.reportNameSuffix so I can see the difference between the two runs.
The following is an example block of our Jenkinsfile:
stage('Integration test MySql') {
steps {
timeout(75) {
sh("mvn -e verify -DskipUnitTests=true -DtestConfigResource=conf/mysql-local.yaml " +
"-DintegrationForkCount=1 -DdbInitMode=migrations -Dmaven.test.failure.ignore=false " +
"-Dsurefire.reportNameSuffix=MYSQL")
}
}
post {
always {
junit '**/failsafe-reports/*MYSQL.xml'
}
}
}
In the report, the integration tests run against mysql then show up with MYSQL appended to their name.
It looks no solution for my question.
As a workaround I changed JUnit XML report format and included build variant name (debug/release) as a package name.