I've created one Jenkins job to execution 5 test cases and it works, but not in terms of build creation.
Build status is coming as SUCCESS even if there is any failure test
cases during execution.
For instance, 2 test cases out of 5 got failed during my recent run, but the build status has become SUCCESS!!!
Please help me in correcting this.....
Note: I'm using Jenkins 1.617 & ANT 1.9.4 for the integration.
If you are executing junits in jenkins, default status is unstable for jenkins job and success for maven/ant build.
If you are using an ant build add the below property to junit task to make the build failed failureproperty="test.failed". If you want to fail the build even if one junit fails, add property haltonfailure="yes"
If you are using maven add the argument -Dmaven.test.failure.ignore=false
Related
I have separated my build tasks in several jobs. The following is a simplified structure.
The basic jobs are:
compile
quick-tests
full-tests-part-1
full-tests-part-2
Additional jobs are:
full-tests
full-build
Each job is configured to accept a workspace parameter to allow all jobs to be executed on the same code (workspace). This is passed from any calling job. Besides, all jobs are running on a slave node.
Job full-tests is configured to bundle the "full tests". Therefore it uses the build step Trigger/call builds on other projects with projects full-tests-part-1, full-tests-part-2.
Job full-build is configured to bundle the build pipeline. Therefore it uses the build step Trigger/call builds on other projects with projects compile, full-tests.
Job compile checks out the Git repository and compiles the code. Afterwards it starts the quick tests. Therfore it uses the post build action Trigger parametrized build on other projects with project quick-tests.
All compilation and test runs are triggered by Ant scripts.
When I trigger the full-build job, the following job sequence results:
full-build
compile (triggers quick-tests)
full-tests
full-tests-part-1
full-tests-part-2
quick-tests
The problem is, that the "full tests" are failing, whereas compile and quick-tests succeed. Both, the full and the quick tests, use the same build.xml in the same workspace, but the "full tests" complains.
Quick tests:
Buildfile: J:\jenkins-slave\workspace\full-build\272\project-path\build.xml
Full tests:
ERROR: Unable to find build script at J:\jenkins-slave\workspace\full-build\272\project-path\build.xml
Does anyone know whether there is a difference in "seeing" the workspace files depending on whether the job is part of the build (triggered as build step) or not (triggered as post build action).
If anything is unclear, please let me know.
Within my freestyle Jenkins-job I´m executing unit-tests via the "execute Windows batch-command"-step:
call "C:\Program Files (x86)\NUnit.org\nunit-console\nunit3-console.exe" MyAssembly.dll
call SomeOtherProcess
As there are tests that fail, I´d expected the build to fail as well. However the test-publishing-step for NUnit markes the build as unstable:
Build step 'Publish NUnit test result report' changed build result to UNSTABLE
If I´d remove the SomeOtherProcess-line from my batch-script, everything works fine and the errors produced by nunit are reported as error in the build-process.
I read a similar issue for the JUnit-test-reporter (Jenkins JUnit Plugin reports a build as unstable even if test fails). Obviously that reporter does not even support failing the build. I´m not sure if the same applies to the NUnit-reporter as well.
The plugin set the result to UNSTABLE because the option, by default, failedTestsFailBuild is set to false.
You can control the behavior applies of NUnit, setting failedTestsFailBuild to true. When you call from a scripted or declarative pipeline.
The issue is the GUI doesn't reflect all the options available for this plugin. There is a PR opened to include this option inside the freestyle pipeline, you can vote up or ask the status of this PR.
To change to an error you need to catch the unstable result and set it to failure using a plugin or a scripted or declarative pipeline.
I want to execute a shell scripts if the Jenkins job build successful or another scripts if the Jenkins job build failed.
I added the post build task plugin, but it seems only can execute a shell in all status or just successful status, cannot specify another shell script to be run once build failed, the two scripts should be run exclusively.
are there anyone can help me on this?
use Post Build Task plugin for conditional steps
I use Conditional BuildStep Plugin and Text Finder Run Condition Plugin to execute steps when the build fails.
But there's a limitation with this method: You have to find a text that is displayed ONLY when your build fails.
See the pictures below:
I add exit 0 at the end of the script to force the build to be successful so that the program can execute the next step, which is Conditional steps(multiple).
You will see Execute script after a failed build. being displayed on console output when the job fails.
This is the execution result.
You will also need to find a text that is displayed ONLY when the first step succeeds so that you can trigger another script.
I'm going through the tutorial on YSlow and Phantom js in Jenkins here: http://yslow.org/phantomjs/
Everything appears to be working great except the Jenkins builds are failing. I think this is due to the violations that YSlow is finding (6 for the particular site I am measuring). I'd rather have the build be successful (or unstable) vs. failed though
Is that possible with this or will I have to resort to something like the postgroovy or text finder plugin?
This is the console output:
phantomjs.exe yslow.js -i grade -t 50 --format junit http://www.somesite.com 1>yslow.xml
D:\Apps\Jenkins\workspace\YSlow_Test>exit 6
Build step 'Execute Windows batch command' marked build as failure
Thanks
Any non-zero exit code at the end of your Execute Windows batch command build step will result in build step being marked as failure.
To have the build step marked as success, you need an exit code of 0. I don't know anything about "yslow" or "phantomjs" and why they are giving you exit code of non-zero, but from "batch" side of things, you need only write exit 0 at the end of your build step if you want to overwrite the exit code of your phantomjs command.
You can then use Text Finder plugin to parse the console log and mark build as unstable when certain conditions are met.
Reading over this answer, Configuring yslow on Jenkins looks like you need TAP plugin to have the functionality of unit testing marking the build as unstable automatically
I have a job in Jenkins which triggers another one.
How do I make the job abort if the triggered one fails?
Thanks
Gil
You can open http://$HOST/jenkins/job/$PROJECT/$BUILD/stop
Use parametrized trigger plugin from here.
Use it as a build step ("Trigger/call builds on other projects"), check "Block until the triggered projects finish their builds" box, and choose "Fail this build step if the triggered build is worse or equal to FAILURE".
You can add the post build task to the triggered job that will:
run if Finished: FAILURE found in the build log,
execute shell:
curl "$UPSTREAM_BUILD/stop" >/dev/null
$UPSTREAM_BUILD neds to be specified by parameter or you can determine it somehow using API...
I am currently setting up my matrix builds to abort all other configurations if one failed and it works as above.