We have WCAT based load tests. Now we want to use Jenkins continuous integration tool. What we want to see is a trend graph of "requests/sec" metric.
I didn't find any suitable plugin in plugins page to achieve this.
Could anyone suggest me solution except to write my own plugin?
If you can get the WCAT data into CSV format, the Plot plugin should work for you.
Related
Is there any way to compare two files in jenkins scripted pipeline and mail the difference? We can write a batch script and call it in jenkins but want to understand if there is any way to do it in jenkins pipeline.
Any suggestions on this is highly appreciated.
If you only want to compare size, the option suggested by Sharp would be a solution. Apart from that, if you want to get a diff, there aren't a lot of built in tools within Jenkins that can help you. You could try the Artifact Diff plugin, but it hasn't been updated in years and may not have the necessary capabilities you are looking for. Using external diff tools you can call would be the best way to go here. As you mentioned, you would have to write a batch script that performs the diff and writes it into a patch file. You could then use Jenkins to email the patch files if they exist.
I have a build-system where one of the tools in the toolchain is analyzing the code and performing static code analysis. The metrics are outputted in an xml-file similair to the one below:
<metrics>
<metric name="metric_a">65</metric>
<metric name="metric_a">32</metric>
<metric name="metric_a">42</metric>
</metrics>
What I want to in Jenkins is to be able to parse this file and then be able to visualize the metrics over time as well as set thresholds so build should fail if for example metric_a is below a certain value.
I've been looking for a suitable plugin but the closest I've found is the Warnings Plugin. However the Warnings Plugin parses logs and aggregates the results by utself rather than parsing an actual file with the final metrics.
Are there any other plugins suitable for handling "custom-metrics" or what is by best option?
For visualization, you could go for the Measurment Plots Plugin.
For failing the build when falling below thresholding on a custom metric, I can only think of making the metric's name/format identical to some existing plugins, e.g. Findbugs or PMD and configure those plugins accordingly.
My project has about 20 build steps and i want to monitor how much time each step takes over builds. I found Jenkins doesn't display such info.
Can use any of Jenkins tools or plugin to do that?
Was just looking for the same and found your post. There is this plugin though it doesn't show you a nice graph or anything it will add the raw timestamps to your log lines on a per project basis. https://wiki.jenkins-ci.org/display/JENKINS/Timestamper If you install it you have to configure each project separately to check the add timestamps to console box (there is a script for doing all projects at once but I didn't want to chance that and only need it to diagnose a particularly slow build).
Did you try this. Makes sense to your case.
https://wiki.jenkins-ci.org/display/JENKINS/build-metrics-plugin
I'm calculating code coverage on every build done in Jenkins and producing a coverage XML report. That's recorded really nicely inside of Jenkins with Cobertura, but what I'd really like is to be able to somehow get at the total branch coverage number so I can automatically publish to a medium the rest of my team can easily consume (i.e. Slack).
As a bonus, getting the difference in coverage from the last run would be even better. I don't see any environment variables that hold this, and haven't found anything detailing a simple way to do this in Cobertura docs. I know I can hack some code together to do this myself, but if there's a simple way I'm missing or something someone else has already built, I'd much rather do that.
I believe you'll have to do this yourself.
The existing Slack plugin just sends build start/success/failure notifications etc.
The Cobertura plugin unfortunately doesn't seem to be built on top of the static code analysis plugin, so there probably isn't much in the way of graphs, difference reports and all that.
You could try adding /api/json to the end of a Cobertura report URL for a Jenkins build — most endpoints reveal some information in this way. If there's some useful information, that could be a basis for whatever you want to hack together.
We've written a framework to test the performance of our Java application (none of the existing frameworks, eg JMeter, were appropriate). The framework produces various metrics, e.g. mean/min/max transactions per second.
We'd like each Jenkins build to display these metrics so that we can keep track of whether a commit has improved performance or not.
I can't figure out how to do this.
One idea is to modify our performance test framework to output a HTML file, then somehow make Jenkins display/link to it on the build results page.
Any advice gratefully received.
The Peformance Plugin can show the results of JMeter and JUnit test in the nice, graphical fashion. And on the plugin page there is a description on how to use it.
This is an open-source plugin hosted on GitHub. The JUnit and JMeter parser are already there, but You can implement your own just by subclassing PerformanceReportParser. It's pretty easy and you can just fork the repo and start your implementation.
I agree that it is hard (if not impossible) to squeeze all the information into standard formats, like JUnit. They are good for quick identification of problems. Once you know there is a problem - you need more information that is usually free-form or custom-formatted to fit your particular needs. So we use both: JUnit that can be immediately processed by Jenkins to decide if the build is stable or not, draw the nice trend graph, etc. We also produce an HTML report that is much more detailed.
Now to your immediate question: you can simply archive your HTML file as an artifact (there is a standard post-build step to do that). Then a link to it will be displayed among the artifacts for the build. There are permalinks to the latest artifacts and latest successful build artifacts:
http://[server]/job/[job_name]/lastCompletedBuild/artifact/foo.html
http://[server]/job/[job_name]/lastSuccessfulBuild/artifact/foo.html
You may bookmark those links and have quick and easy one-click access to your results.
You could use the HTML Publisher Plugin to publish the resulting HTML page. That would be pretty straightforward.
If you want better integration you could try to create output that follows the same format JMeter produces, and use the Performance Plugin.
For best result you could take Łukasz advice and modify the Performance Plugin to your needs. That requires the most effort on your part, of course.