I have JMeter setup on Jenkins. I am running some performance test with it. On Jenkins, I have setup the TestLink plugin.
On JMeter, I have 'View Results Tree' tree setup, and export the results to a xml file. With the results with the following format:
<testResults version="1.2">
<httpSample t="81" lt="81" ts="1463588977147" s="true" lb="Unit_EdgeSpanTest_transmission-submit_Case1" rc="200" rm="OK" tn="CinemaCMSAPITest 1-1" dt="text" by="382">
<assertionResult>
<name>Response Assertion-ResponseCode</name>
<failure>false</failure>
<error>false</error>
</assertionResult>
<assertionResult>
<name>Response Assertion-TextResponse</name>
<failure>false</failure>
<error>false</error>
</assertionResult>
</httpSample>
Now I have problem get JMeter results report back to testLink. Googled, not much resource out there tell me what to do. I could only find one resource from Jenkins:https://wiki.jenkins-ci.org/display/JENKINS/Integrating+TestLink++Jenkins++JMeter
But it did not help much. I am not using Ant...
Does anyone has successfully made this work? And could give me some hint of what to do?
Thank you very much in advance!
I recall I had to submit JMeter results to TestLink some time ago. In my case each JMeter sampler represented an individual test case so I used the Beanshell Assertion and TestLink Java API Client combination. If you're not familiar with Beanshell - check out How to Use BeanShell: JMeter's Favorite Built-in Component
If you're not comfortable with coding I guess you can access the API endpoints using HTTP Request or SOAP/XML-RPC Request samplers.
There is also a number of discussions regarding the API on TestLink forum
Hope this helps.
I followed the instruction described in the question to configure the testLink, JMeter, and Jenkins. Just with one problem: I first tried to use the script from the link above, but it seems always returned the wrong result. I wrote a simple java script to transform JMeter XML test result file to TAP format. Except that the link from the question is a good instruction to follow to setup the integration between JMeter/Jenkins/TestLink:
To summarize:
From Jenkins:
install TestLink plugin
Configure TestLink from Jenkins > Configuration
Create a project > Configure:
Invoke TestLink: Put the TestLink Version/Test Project Name/Test Plan Name/Build Name/Custom Fields/Test Plan Custom Fields according to the settings from the TestLink.
Add build action to run the java function/shell script
Result Seeking Strategy: Add the tap directory in 'Include Pattern' Add JMeterTestCase to Key Custom Field.
From TestLink:
Create custom Fields,
Add the JMeterTestCase custom field to test specs. And set the test specs to 'Automated'
Assign the test specs/cases to the according test plan.
From JMeter:
Make sure the test result listener generates xml format result.
Have a script transform JMeter Test result from xml format to Tap format.
In the link from the question, it has a script for doing that, and I also attached a simple script wrote in Java for transforming JMeter Result xml file to TAP file in java:
https://github.com/yueran/jmeterResult_xmlToTap
Related
I couldn't find any solutions to this particular need.
Basically I have a GUI Job and I need the status of the Sonarqube Analysis so I can later send a POST Request with it.
(I'm aware that pipeline exists and works great but because a specific reason I need it to be GUI)
On the pipeline you have the WaitForQualityGate.status(), I've tried using this but no success.
Example of what is desired
Any insights? Thanks in advance
You can use the SonarQube Rest API to get the status.
Whenever you run SonarQube analysis through Jenkins Pipeline, upon the successful analysis you will see report-task.txt created in the workspace folder.
Note: The location of report-task.txt file depends on the tool that was used to generate it. The mvn sonar:sonar task defaults to path target/sonar. In my case, I used sonarscanner to analyse a nodejs project. So the location of report-tast.txt is .scannerwork.
Now, you will get the ceTaskUrl and ceTaskId in report-task.txt. You can use that ceTaskUrl to get the analysisId.
Then, you can use the below api to get the quality gate status using analysisId.
http://<sonarqube_host>/api/qualitygates/project_status?analysisId=$ANALYSIS_ID"
Now, try to get the curl output of the above API into a variable.
If you mean to say that you want a custom variable message to pop up in your Jenkins GUI based on the SonarQube scan status, then that would require you to:
Clone the original Jenkins source code
Add a custom HTML button/div/graphic
Compile the Jenkins code
Build the new code
Execute the generated JAR
Else, you can try some plugins available on Jenkins that would give you the ability to render conditional outputs. No promises on whether they can actually help you change the original GUI.
Any alternative traditional approach wouldn't be able to fulfill your GUI requirement.
I have a set of tests which run within Jenkins and at the end of the tests (in the post section) I generate an allure report. The tests are created with Webdriverio.
The last time I did anything like this, I was using standard selenium running with Junit and due to that combination, the Jenkins Rest API had access to things like failedCount, skipCount, passes and failed counts. This however seems to only be possible with JUnit.
Does allure expose this to Jenkins somehow - or am I going to have to try and scrape the statistics from a pre-generated report?
(as a side note, I have tried to configure a second reporter in webdriverio, but while that seems to create some xml files locally, the folder I configure is not appearing on Jenkins. Also, webdriverio doesn't support JUnit as a framework - currently it's using mocha.)
I eventually managed to figure this out.
There is url which you can go to to get a json file with the summary information.
So if you report is at /allure - then you navigate to /widgets/summary.json
I'm currently editing the Jenkins' job description using a function in my jenkinsfile:
currentBuild.description = "${get_user_name()} bla bla"
I want, after the build is done, to read a pytest html file (which is generated during the job) and extract number of passed / failed tests, and add that data to the description.
The extraction itself is already done in a python script I have, however, I'm not sure how can I add that data to the description.
One way I thought of was an http request to the jenkins web API, but that seems a bit convoluted, and specifically, I had some problems doing that in the past.
Is there a way to make it simple using http, or using another (built-in) way?
As far as I know, JMeter allows you to send multiple POST request with different parameters (e.g. { "value": "value1"}, {"value": "value2"}, ...) However, I'm more comfortable using a terminal-based interface similar to ab or siege. Basically, I need to load test a server simulating the case in which some requests are not previously cached.
Are there alternatives to JMeter for Linux that are able to use different parameters for a POST request?
UPDATE
As far as I can tell, JMeter requires the creation of a test plan (jmx file) in order to run via the command line. Unfortunately, this test plan needs to be built using the GUI, which is precisely what I want to avoid.
UPDATE 2
I will use JMeter because it offers dynamic parameters for POST requests and most alternatives depend on JMeter. However, if anyone knows of a standalone library that works exclusively from the terminal (similar to ab), please let me know.
you can use JMeter in terminal mode, it's called Non GUI mode.
To variabilize just use CsV dataset to load variables (varName for example )per thread, then use ${varName}
See :
http://jmeter.apache.org/usermanual/get-started.html#non_gui
http://jmeter.apache.org/usermanual/component_reference.html#CSV_Data_Set_Config
Nice report at end:
http://jmeter.apache.org/usermanual/generating-dashboard.html
If you don't want to use GUI even for building the test, then look at :
https://github.com/flood-io/ruby-jmeter
It allows you to generate the JMX from a DSL file.
Examples here:
https://github.com/flood-io/ruby-jmeter/tree/master/examples
DSL here:
https://github.com/flood-io/ruby-jmeter/blob/master/lib/ruby-jmeter/DSL.md
$LOAD_PATH.unshift(File.join(File.dirname(__FILE__), '..', 'lib'))
require 'ruby-jmeter'
test do
csv_data_set_config name:'MyCsv', filename: '/path to file', variableNames: 'myParam'
threads count: 10 do
visit name: 'Qwant Search', url: 'https://lite.qwant.com/?q=flood.io&t=web&p=${myParam}'
end
end.jmx(file: "path to your output plan")
Save file to ruby-jmeter-csv.rb
You can then generate the plan with:
ruby ruby-jmeter-csv.rb
And run it in non gui mode.
In fact JMeter GUI should be used for tests development and debugging only, when it comes to running the load test - it is recommended to run JMeter in command line mode, via Ant task or Maven plugin. Also there is a couple of more "geek" alternatives, i.e.:
JMeter .jmx scripts are basically XML files so you can use your favourite text editor to create or amend them
You can use JMeter API to create and kick off JMeter tests using Java language
If you're still looking for an alternative, here are few free and open source load testing tools
Grinder - you can write scripts in Jython
Gatling - you can write scripts in Scala-based DSL
Tsung - this guy exists for Linux and Unix-based platforms only, Erlang-based. Scripts are XML files.
Taurus - automation framework which supports all aforementioned tools (and some more), Python based, configuration files have simple YAML syntax.
See Open Source Load Testing Tools: Which One Should You Use? for more information on the above tools and comparison of them with JMeter
I am running sitespeed.io tests in Jenkins, and have configured it to output junit format test results.
I'd now like to publish those test results to SonarQube (I realise I can publish them in Jenkins, but I have a requirement to keep everything in one place as much as possible).
However, when I add the test results file into the Sonar analysis (using sonar.junit.reportsPath=/path/to/sitespeed-results, having named my results file TESTS-sitespeed.xml) SonarQube doesn't seem to show any results on its dashboard.
I understand that SonarQube also has a setting to configure the location of test files, and this is often cited as a reason for test results not being ingested correctly, and that leads me to wonder whether what I'm trying to do is possible.
Any help would be greatly appreciated.
Thanks,
Jez
the property sonar.junit.reportsPath will read and parse the report but will only save the information if the class name indicated by the report can be mapped back to a java resource of the project.
I have no idea about the output of your test in surefire format but I sense the classname won't match any resource of your project and so your report is parsed but data is ignored.