Cannot detect file type because of error, and : Failed to copy Build step 'Publish Performance test result report' changed build result to FAILURE Finished: FAILURE. whenever i run the script always i face this..
Take a look into Jenkins Console Output - it should give you the reason for the failure.
Most probably the Performance Plugin fails to find JMeter's .jtl results file in Jenkins Workspace, either the .jtl results file is missing or you're pointing the Performance Plugin to the incorrect location.
If you have a Script step to run a JMeter test like:
jmeter -n -t test.jmx -l result.jtl
You should be able to use simply result.jtl in the Performance Plugin.
Check out:
Performance Trend Reporting
How to Use the Jenkins Performance Plugin
and Running Performance Tests articles for more details on various aspects of Jenkins Performance Plugin use cases.
Related
I am using Jmeter for API's functional testing. For this, have added Response Assertion.
Even though it's failing but Jenkin's build appeared as Succeed.
Is there any way to mark Jenkin build as Failed when our Assertions are failed?
Please help out on this, let me know if any more info is required.
It depends on how do you launch JMeter in Jenkins, if it's just a command-line non-GUI execution like jmeter -n -t test.jmx -l result.jtl then it doesn't produce any error exit status code and this is something Jenkins checks.
The options are in:
Migrating to JMeter Maven Plugin which provides jmeter-check-results goal
Migrating to Taurus which provides Pass/Fail Criteria subsystem
And finally you can add a JSR223 Listener to your Test Plan and force JMeter to exit by adding the next code in the "Script" area:
if (!prev.isSuccessful()) {
System.exit(1)
}
I have multiple xml result files collected from different builds and I have to display a consolidated report on jenkins.
Is there any plugin available to perform the above action?
We also tried JUnit plugin, we got the following error:
ERROR: Step 'Publish JUnit test result report' failed: Test reports were found but none of them are new. Did leafNodes run? For example, D:\jenkins\workspace\test-1\testng-results.xml is 14 min old
try to check the timestamp when the junit report was generated and last execution time of jenkins job. Both timestamp should match.
If you see a difference, delete the report and re-generate the report.
How can I publish the results of the code coverage to the trend graph even if the job fails after the jacoco analysis?
For Findbugs, PMD, etc. this can be achieved with the canRunOnFailed setting. Is there something similar for JaCoCo?
On project level I see the trend graph which only considers the successful builds. I would also like to see the coverage graph updated if the job fails.
This is tracked by JENKINS-28479:
ProjectActions (and trend graphs) do not appear for failed projects
This problem is not only related to the JUnit plug-in. In general, every plug-in that has been migrated its actions to a SimpleBuildStep has this problem. The affected source code is in SimpleBuildStep.
A test case that exposes this bug is WarningsPluginTest#should_not_skip_failed_builds_with_option_run_always
This is still open.
Original answer:
This is not specific to the maven jacoco build step: all you need to do is make sure that build step always return "success".
If you are using pipelines, you can wrap your step in a try/catch.
If you are using a regular job definition, you could consider a shell/windows command step where you return 0 (or exit /b 0) after the mvn command.
The next build step would be the mvn sonar:sonar publication one, which will then always be executed.
I have similar issue, that blocks me. I need to run my JMeter tests with Jenkins. But Jenkins validates the JMeter build as successful when it’s actually failed. I wonder what I am doing wrong so that the jmeter won't return fail when an assertion fails.
I run a Windows Batch script calling jMeter. Here's how:
COMMAND LINE
cd C:\apache-jmeter-3.1\bin
jmeter -n -c -t C:\Users\maria\Desktop\Automation\WIP\Test-page.jmx -l C:\Users\maria\Desktop\Automation\WIP\Test-page.xml
And saves the results in .xml to validate the report with the Performance plugin (v 2.0) and Jenkins ver. 2.32.1
CONSOLE
Created the tree successfully using C:\Users\maria\Desktop\Automation\WIP\Test-page.jmx
Starting the test # Fri Dec 30 11:05:58 GMT 2016 (1483095958952)
Waiting for possible Shutdown/StopTestNow/Heapdump message on port 4445
summary = 1 in 00:00:00 = 3.1/s Avg: 206 Min: 206 Max: 206 Err: 1 (100.00%)
Tidying up ... # Fri Dec 30 11:05:59 GMT 2016 (1483095959326)
... end of run
Performance: Recording JMeter reports 'C:\Users\maria\Desktop\Automation\WIP\Test-page.xml'
Performance: Parsing JMeter report file 'C:\Jenkins\jobs\Test demo\builds\3\performance-reports\JMeter\Test-page.xml'.
Performance: Percentage of errors greater or equal than 0% sets the build as unstable
Performance: Percentage of errors greater or equal than 0% sets the build as failure
Finished: SUCCESS
As you can see I have 1 error. But the Performance plugin sets the build to success.
PLUGIN SETTINGS
performance plugin settings
Jmeter properties
jmeter.save.saveservice.assertion_results=all
jmeter.save.saveservice.output_format=xml
jmeter.save.saveservice.response_data=true
TEST CASE
Test-page.jmx
Attempts I tried and didn’t work for me:
older version of the Performance plugin (v 1.11 and v.16)
.jtl format
.xml
.csv
adding "echo" to the command line in the end
setting jmeter.save.saveservice.response_data=true
Ran out of ideas how to make the Performance plugin count the errors and raise the "Failed" flag :(
Set "Use error thresholds on single build" from 0 to 1.
This should mean that the jenkins build fails if at least 1 error is in your JMeter test case, where a 0 ignores the error count (If I remember this correctly).
Sounds like a Performance Plugin issue, I believe you should report it via Jenkins Bug Tracker
In the meantime to work this around I would suggest returning non-zero exit code for the JMeter build step, the fastest and the easiest way is running your JMeter test using Taurus tool as a wrapper. Taurus has powerful Pass/Fail Criteria subsystem where you can define the conditions of your build failure. If the condition will be met Taurus execution step will be finished with code 3. Jenkins is smart enough to automatically fail command-line tasks with non-zero exit code.
Minimal working Taurus config file:
---
execution:
scenario: my-test
scenarios:
my-test:
script: C:\Users\maria\Desktop\Automation\WIP\Test-page.jmx
services:
- module: passfail
criteria:
- succ<100%, stop as failed
More information, if needed: Meetup Recap on Using Taurus to Automate JMeter and Jenkins Tests
You can also change JMeter exit code directly form the JMeter test via i.e. JSR223 Scripting Elements like:
if (some condition) {
System.exit(1);
}
but in this case you won't have much flexibility in terms of setting failure criteria
I set my Select evaluation mode to standard mode, Set the radio button on Error Threshold and set Unstable and Failed to 1
I'm going through the tutorial on YSlow and Phantom js in Jenkins here: http://yslow.org/phantomjs/
Everything appears to be working great except the Jenkins builds are failing. I think this is due to the violations that YSlow is finding (6 for the particular site I am measuring). I'd rather have the build be successful (or unstable) vs. failed though
Is that possible with this or will I have to resort to something like the postgroovy or text finder plugin?
This is the console output:
phantomjs.exe yslow.js -i grade -t 50 --format junit http://www.somesite.com 1>yslow.xml
D:\Apps\Jenkins\workspace\YSlow_Test>exit 6
Build step 'Execute Windows batch command' marked build as failure
Thanks
Any non-zero exit code at the end of your Execute Windows batch command build step will result in build step being marked as failure.
To have the build step marked as success, you need an exit code of 0. I don't know anything about "yslow" or "phantomjs" and why they are giving you exit code of non-zero, but from "batch" side of things, you need only write exit 0 at the end of your build step if you want to overwrite the exit code of your phantomjs command.
You can then use Text Finder plugin to parse the console log and mark build as unstable when certain conditions are met.
Reading over this answer, Configuring yslow on Jenkins looks like you need TAP plugin to have the functionality of unit testing marking the build as unstable automatically