In my JMeter setup, I have the following config in my Test Aggregate Report (named loadTestAggregate) in order to list all the erroring URLs in my load test :
When I run my JMeter test locally, this Test Aggregate Report correctly displays the results (as a .csv) and the URLs are displayed as required (i.e. the response code, response message and URL are all displayed).
I then set up this test to run via Jenkins (with the Performance plugin installed).
Below is the Jenkinsfile used to run this test via Jenkins;
sh "jmeter -Jjmeter.save.saveservice.url=true -n -t loadTest.jmx -JTest_Url=${env.TEST_URL} -l jmeter_results.jtl"
perfReport errorFailedThreshold: 5.00, sourceDataFiles:'loadTestAggregate.csv'
However, when the test is completed and I look in Jenkins via the Performance plugin, the URLs are not displayed.
All that's displayed is the following;
Is it possible to get these (erroring) URLs to be listed via the Jenkins Performance plugin?
If so, am I missing something here as my config for the Test Aggregate report works locally but not via Jenkins, so I can only presume that it's a Performance plugin issue.
Many thanks
They're "aggregate" charts per Sampler label
As a workaround you can change the Sampler label (HTTP Request) to the Sampler URL, it can be done via JSR223 PostProcessor and the following Groovy code:
prev.setSampleLabel(prev.getUrlAsString())
where prev stands for the previous SampleResult, see Top 8 JMeter Java Classes You Should Be Using with Groovy article for more information.
Demo:
Related
I set up a Jenkins to integrate the JMeter script. Using the Freestyle project in Jenkins, I enabled "This project is parameterized" to set a String parameters for the Threads, Loop_Count and Think Time.
In .jmx file, I used the Function Parameters function to define those
variables, as shown below image:
User Defined Variable
In Jenkins, I configured the parameterization as shown below:
Set Parameterization
Command Line
However, when running the test for 40 users using the Build parameter in Jenkins, it looks like the # of threads/users are not correct that is being executed, the Samples that are being generated only 3 for most of the pages. Only the Homepage (which is the first page on the test) is only getting the correct # of Samples, but the rest of the URLS/pages are not correct. Below the actual output.
Output
Can you please help what might the causing this issue, I already checked the Jmeter script and jenkins config and appears to be correct but still I'm getting the issue. Thanks for the help.
Setting parameters in Jenkins itself is not sufficient to pass them to JMeter, you need to pass this parameterized value to JMeter startup script via -J command-line argument
-Jusers=%users%
End-to-end demo:
More information: Apache JMeter Properties Customization Guide
I have built a Jenkins job to run automated ZAP-Proxy scans.
I used curl -X POST -u YOUR_USER:YOUR_USER_PASSWORD http://YOUR_JENKINS_URL/job/YOUR_JOB to build the job from the terminal. Is there a way to display the console output in the terminal while the job is building?
It is possible, but it's a little more complicated for two reasons:
When you trigger a new build via curl, then the build will not start immediately. The build will enter the build queue, and it will only start executing once Jenkins found a suitable executor. Before that time, there is no build URL at all.
Technically, the (continuous) console output is delivered in multiple fragments that must be retrieved individually via HTTP.
So, once you triggered the build, you need to find its URL after if left the build queue. It can be done nicely in Groovy -- as a simpler heuristic, you could just wait for certain time and then use the lastBuild reference.
For fetching the console log fragments, you'll use the <buildUrl>/logText/progressiveText end point. Create a loop fetching that URL and checking the X_More_Data and X_Text_Size HTTP headers for information on whether (and what) console output is available. You can do that in bash with curl in a loop; I found this Groovy example on the Web.
In the end, the most elegant solution is probably to
trigger a new build by submitting a Groovy script that will trigger the build and then waits for the build to leave the build queue, returning the build URL.
Then use another script that will poll/fetch/display that build's console output.
If you use the CLI interface for submitting the script, then you can do all those steps in a single script. If you use the REST API, then second part ("continuous output") probably won't work due to output buffering on REST API side.
I have builds numbered 1, 2, 3 & 4 for my job named TEST_JOB. With jenkins-cli.jar, am planning to execute a command where I can get build numbers of my TEST_JOB.
Have tried using cli commands such as console (which prints the outputs console of the last build/ current build) & build (which runs the job) but neither of which returns a build number.
Am planning to do this via command prompt. Any idea as to how should I proceed?
Why using Jenkins-cli jar? I mean you can do it using that but I think the better option is to access job/build metadata using Jenkins API.
How to get started with Jenkins API? you can see job metadata in JSON/XML format in the following URL:
JENKINS_URL/job/JOB_NAME/api/json
JENKINS_URL/job/JOB_NAME/api/xml
In the job level, you can see there details as: parameters, builds list and more
In the build level, you can see there details as: who started the build, parameters, duration, result
And then from code perspective, you can access JSON/XML using curl command or wrap it with python code.
I am trying to write the test cases to validate the Jenkinsfile, But the load script function not working expecting the extension to be provided and throwing ResourceException exception loadScript("Jenkinsfile")
Is their better way to test the Jenkinsfile
The problem is that there are not enough tools for the development of pipelines. Pipelines is DSL and it imposes a restrictions.
There is an interesting approach to using flags. For example, test which defines outside pipeline(in job). If test=true, a pipeline change some "production" logic to "test" - select another agent, load artifacts into another repository, run another command and so on.
But recently appeared Pipeline Unit Testing Framework. It allows you to unit test Pipelines and Shared Libraries before running them in full. It provides a mock execution environment where real Pipeline steps are replaced with mock objects that you can use to check for expected behavior.
Useful links:
Jenkins World 2017: JenkinsPipelineUnit: Test your Continuous Delivery Pipeline
Pipeline Development Tools
You can validate your Declarative Pipeline locally thanks to Jenkins built-in features.This can be done using a Jenkins CLI command or by making an HTTP POST request with appropriate parameters.
The command is the following:
curl -s -X POST -F "jenkinsfile=<YourJenkinsfile" \
https://user:password#jenkins.example.com/pipeline-model-converter/validate
For a practical example follow this guide:
https://pillsfromtheweb.blogspot.com/2020/10/validate-jenkinsfile.html
The Setup - A jenkins job using jenkins parameters testApp and testEnv. The Execution Batch looks like this:
C:\jmeter\apache-jmeter-3.2\bin\jmeter.bat -n -t
C:\JMeter\Scripts\API_scripts\%testApp%.jmx -Jtestenv=%testEnv% -JtestApp=%testApp% -JtestBrowser=NA -l
C:\AUTO_Results\jtl\%testApp%_%testEnv%.jtl
Post-build Actions
Console output (build lob) parsing with a global rule so that the Failures that are logged in the Jenkins Console window will consider the JMeter script failing. (discussed Jenkins shows JMeter script failure even though the script actually passed)
Triggered parameterized build - this is a separate jmeter script that updates a wiki page with either PASS/FAIL and uploads the JMeter report.
The Issue - How do I get the downstream Triggered build to use the parameters from the upstream script? I set the Parameter = Current build parameters but it's not applying those. Also, I wont know the value of the testResult parameter until the upstream build finishes. I tried adding %testResult%=PASS to the 'Predefined parameters' box
As per Parameterized Trigger Plugin page:
The parameters section can contain a combination of one or more of the following:
a set of predefined properties
properties from a properties file read from the workspace of the triggering build
the parameters of the current build
Subversion revision: makes sure the triggered projects are built with the same revision(s) of the triggering build. You still have to make sure those projects are actually configured to checkout the right Subversion URLs.
Restrict matrix execution to a subset: allows you to specify the same combination filter expression as you use in the matrix project configuration and further restricts the subset of the downstream matrix builds to be run.
So you basically need to copy over the parameters you would like to have in the "downstream" job from the current one.
As a workaround to current performance plugin limitations you can consider running JMeter using Taurus tool as a wrapper, it has flexible and powerful pass/fail criteria subsystem which will basically return to Jenkins non-zero exit code triggering build failure in case of issue in the test. If everything goes well Taurus exit code will be 0 which is considered successful by Jenkins. Check out How to Run Taurus with the Jenkins Performance Plugin article for more details.