I have a serenity bdd framework that I use to functionally test an app. I want to put this into a jenkins pipeline but I have come up with an issue.
When I run the pipeline stage with the mvn clean verify command I use to run the tests I get BUILD SUCCESS and it causes the pipeline to always show as green. The BUILD SUCCESS is separate from the results of the tests within it so I could have a successful build with multiple errors and failures from the tests within the BDD framework. How do I get the Jenkins pipeline to report based on the serenity test results rather than the overall build?
A
Related
I needed to setup a Jenkins pipeline and I wanted to know if unit test should run before the actual compilation of the project
The Jenkins pipeline currently runs unit and Integration test before the bundling and wanted know if that's ok
I am working with:
Jenkins 2.64
Gradle 3.5
Spring Framework 4.3.8
I am able to execute a Main java class through a Gradle task through Jenkins.
Therefore I can execute a Job N times to work around JMX and for each Job execution I have the Build History where I am able to see the logging outputs from my business classes for each interaction.
Note the history saves each logging outputs for each Job's execution
The problem is with testing.
Through Jenkins I am able to execute a Job related with a Gradle Test command, such as: gradle test --tests. Here two behaviors:
Gradle saves the logging outputs from my business classes through its own output directory build\reports\tests\test (it is the expected, I am fine with this)
Jenkins worked fine about the execute the Gradle test command but all the logging outputs are only posted through the Gradle Test reports but not in Jenkins, it in Build History.
Thus the Jenkins does not contain the logging outputs from my business classes. Therefore if I execute N times through my developing cycle the Test Job. I only am able to see the latest logging outputs only through the Gradle Test Reports, sadly Jenkins does not keep these logging outputs through Build History.
How resolve this? Some basic configuration or a plugin is need it?
In order to save junit reports in Jenkins, you can use JUnit Plugin
Individual reports are located at build/test-results/test/.xml*
If you are using Declarative Pipelines you can use the following script
junit 'build/test-results/test/*.xml'
Since you are pointing to build\reports\tests\test\index.html which is the gradle test report, you can also save that report using HTML Publisher Plugin
My suggestiong is to use JUnit Plugin for better integration.
To save artifacts from your build you can use the archiveArtifacts pipeline command
It would look something like:
archiveArtifacts artifacts: 'build/reports/tests/test/*'
I am using Jenkins for integration testing.
Just to give the context. At the moment I have a separate build server which produces the build daily and Jenkins is not used as the build server. The build server executes the unit testing in my case.
When build process is complete it invokes the Jenkins job. In that job Jenkins start to deploy the build into the Virtual machine. I have a script for doing this.
Followed to that my plan is to run several scripts for doing the end-to-end testing.
Now I have several question in this regard:
How to parallelize the execution of the end-to-end tests?
As I am adding scripts after script I am getting worried how manageable it will be?
I am always using the web interface for adding and changing the scripts. How to do this from the command line?
Any ideas for a good tutorial? Any pointers from all of you? Thanks!
Looks like Build Flow Plugin is what I need.
https://github.com/jenkinsci/build-flow-plugin
You might want to try and see if you can use the Build Pipeline plugin before build flow. Much better visualization of what is going on, less scripting.
I link Build and deploy jobs in one sequence and then have unit and integration test jobs linked separately off the build job. You can then use Fail The Build plugin to have downstream jobs fail upstream ones.
I've a simple Jenkins job where i runt python script to run a bunch of tests. I see the build as UNSTABLE even i see no obvious errors in the Jenkins job as well as no tests failing.
Whats i'm running is python
I'm not using any external test framework like nose. Its just a python script.
"Unstable" is the result Jenkins assigned when tests are failing. The only other way to get "Unstable" result is through the use of some post-build plugins. If you don't have any other plugins that are setting the build as unstable, then you have to take a closer look at all the tests.
If you paste the console output here, that would help as well
I have 2 jobs in Jenkins:
Build and run unit tests
(Build and) run integration tests
Job-2 is a Downstream project of Job-1.
Job-1 initiates build and run unit tests on it.
Job-2 initiates build as well and run integration tests.
I would like to change that, and make Job-2 to run the tests on the result build that was initiated by Job-1.
You could use Copy Artifact Plugin and use Job-1 artifacts from Job-2 to run some tests on them. Refer to this post for more information. Hope it will help.