Jenkins test results screen shows only pass/failed results.
I would like to show quantitive (number/percentage/time duration etc') results, parsed from logs.
e.g. memory usage, run time of specific methods etc..
What is the best way to do so?
Thanks
good question.
i imagine you're probably looking to extend your Jenkins instance with some plugins that describe more info about the tests you've run. this plugin seems relevant, but requires some experience with JMeter (java-based performance measurment tool) to generate the output that this plugin can then read and display output from a JMeter task that you can run every time your build runs:
https://wiki.jenkins-ci.org/display/JENKINS/Performance+Plugin
the 'readme' in the above plugin details page specifies how to set up a project to run JMeter( see the 'Configuring a project to run jmeter performance test:' near the bottom.
another way to do similar not so immediately tied to a specific jenkins build is to run resource monitors (like Cacti or collectd) on the machines running the tests and analyze those results post-build, but again, outside of the Jenkins context.
HTH.
Related
I ask you to consult on the following question about allure: I use jenkins + pytest to run the tests. The same tests run on several virtual machines, these machines differ in operating systems (different linux distributions) and test environment. After running the tests, I want to combine the results from all the machines into one report. - here the question arises - if I put all the reports in one directory and generate a report, then the results from different machines will be considered as rerun of the same test and combined into one. How can I get around this? so as not to be combined and so that it was possible to somehow sort out which result from which machine. Thanks.
i have solve this by override the names of tests/suites.
Meaning you have to make some code implementation, work with the before listeners, there you can get the current test name and override it. Set the test name by OS + Browser or something unique.
When you combine reports, they will be unique and properly displayed.
I ran into a similar issue with behave where Allure was treating each parallel build as a retry of the first build. I realize this isn't the same as pytest, but perhaps it'll help.
I was inspired by the previous answer and started experimenting. By changing the scenario name(s) within the feature, I was able to make Allure recognize each parallel build as separate tests. I accomplished this by adding a before_feature method to my environment.py file that simply added the hostname to each scenario name within that feature:
def before_feature(context, feature):
for scenario in feature.scenarios:
scenario.name = f'[{socket.gethostname()}] {scenario.name}'
Originally, I tried to directly change scenario.name in before_scenario but that seemed to have no effect in Allure.
This is a broad question, so any answers are deeply appreciated. I need to continually log the size of several build files (in this case some CSS and JS files), preserve this log and ideally show it as a dashboard in Jenkins.
I know that I can setup a cron job and execute a bash script to grab the files and log their size, but I'm not sure where this file would live and how to display it. Ideally the result would be a dashboard plot or bar graph over time.
Thanks.
P.S. I'm open to other logging suggestions, but Jenkins seems like the appropriate system to do this in.
Update: this isn't perfect but it works. Google Spreadsheets has a simple API for posting data, so this can work as an endpoint for any script you want to write that logs your data.
It's not a Jenkins solution, but gets the job done.
In my search leading up to this, I did come across JMeter, and the Performance Plugin for Jenkins, which were contenders for a possible solution.
I'm struggling with something for quite a while and can't find a solution,
I got a test project (cucumber, maven) I configure jenkins to pull the project from github, build and execute the code (selenium test script) on jenkins slave and that works perfect, I added few more slave, tagged them and I'm able to execute the same job parallel(the same test cases on different machines)
my next step is to use grid extra (https://github.com/groupon/Selenium-Grid-Extras) in order to use some cool features like video recording, browser updating, selenium updates etc...
now, I know that in order to use the grid I need to address it via my code and also define the desired capabilities (browser, os etc...).
currently when I'm running the same job twice, my second request will be queued till the first one ends, if I will run the same code from two developers machine it will run at 2 different nodes and the grid can handle both request.
not sure what is wrong with my jenkins configuration or my grid hub configuration, I checked it again and again and all looks good :-)
so guess I'm missing something
any advice/direction/idea will be highly appreciated.
Thanks
Ronen
I need to generate a weekly report on our Jenkins build cluster. One of the reports is to display a list of jobs that have the longest build time.
The solution I can come up with is to parse the "Build history" page on each slave (also master) and for each build of a job, parse the build page and look for "Took x min x sec on slave-xx".
This feels quite cumbersome, does anyone know a better solution using Jenkins API or Groovy script console?
Thanks
You can get the build data for your report through the Jenkins API. For a given job, you can
retrieve the list of builds with duration information using something like:
http://jenkins:8080/job/my-job/api/json?tree=builds[id,number,duration,timestamp,builtOn]
To see a list of all the API-available build data for a given job:
http://jenkins:8080/job/my-job/api/json?tree=builds[*]
Once you have a query that retrieves the job information that you need for
your report, it should be straightforward to loop over the jobs.
Most Jenkins pages have a link at the bottom to the REST API that describes a bit about accessing the API for that page, e.g. http://jenkins:8080/job/my-job/api.
How about using plugins?
Check this out:
https://wiki.jenkins-ci.org/display/JENKINS/build-metrics-plugin
There are few others too which you can try depending on how much customization/features you want to do/display:
https://wiki.jenkins-ci.org/display/JENKINS/Global+Build+Stats+Plugin - This is quite extensive
https://wiki.jenkins-ci.org/display/JENKINS/Project+Statistics+Plugin
https://wiki.jenkins-ci.org/display/JENKINS/eXtreme+Feedback+Panel+Plugin
We have a situation where something causes our Jenkins master to start leaking memory at a certain time each night. No jobs are running at this time, so I'm not interested in logs of build jobs.
We would like to figure out what is happening around this time, but we cannot see much in the jenkins.out.log, since it doesn't log a lot and lacks timestamps.
Does anyone know if there is a way to turn on more detailed operational (not just errors) logging in Jenkins?
You could use the logs accessible from the UI (Manage Jenkins ยป System Log) and set the log level to ALL for the default log recorder in Log Levels. This log is only accessible from the UI though.
Alternatively, you could increase the log level of the regular jenkins.log by passing --debug=9 to the Jenkins command. If you're on CentOS, changing the value of JENKINS_DEBUG_LEVEL in /etc/sysconfig/jenkins has the same effect.
You could also use the Jenkins/Hudson monitoring plugin to see snapshots of memory use etc inside the JVM.
Also you could install sar (sysstat) to see how the server itself is doing over time (if Linux).