We are using Jenkins pipeline to run jmeter tests for testing one of our application API. EVeryting is working ok but there are cases where the Application returns an error. We would like to log the request payload for such failures and also the timestamp so that we can investigate in the application about corresponding failures.
Is there a way, I can instruct jmeter to log the Request Data for cases which result in failure?
The easiest option is adding a Listener like Simple Data Writer to your test plan.
The configuration to save the timestamp and payload would look like:
Once the test finishes you will be able to observe requests details (if any) using View Results Tree listener.
More information: How to Save Response Data in JMeter
Related
We do something called feature testing like so -> https://blog.twitter.com/engineering/en_us/topics/insights/2017/the-testing-renaissance.html
TLDR of that article, we send request to microservice(REST POST with body), mock GCP Storage, mock downstream api call so the entire microservice can be refactored. Also, we can swap out our platforms/libs with no changes in our testing which makes us extremely agile.
My first questions is can DataFlow (apache beam) receive a REST request to trigger the job? I see much of the api is around 'create job' but I don't see 'execute job' in the docs while I do see get status returns the status of job execution. I just don't see a way to trigger a job to
read from my storage api (which is mockable and sits in front of GCP)
process the file hopefully across many nodes
call the apis downstream (which is also mockable)
Then, I simply want to in my test simulate the http call, then when file is read, return a real customer file and then after done, my test will verify all the correct requests were sent to the apis downstream.
We are using apache beam in our feature tests though not sure if it's the same version as google's dataflow :( as that would be the most ideal!!! -> hmmm, is there a reported apache beam version of google's dataflow we can get?
thanks,
Dean
thanks,
Dean
Apache Beam's DirectRunner should be very close to Dataflow's environment, and it's what we recommend for this type of single-process pipeline test.
My advise would be the same: Use the DirectRunner for your feature tests.
You can also use the Dataflow runner, but that sounds like it would be a full integration test. Depending on the data source / data sink, you may be able to pass it mocking utilities.
BigQueryIO is a good example. It has a withTestServices method that you can use to pass objects that mock the behavior of external services
I have a Jenkins Server which runs a Jenkins job - JobA
and I have a Java Spring boot web service, where I store the Jenkins build data and serve it to various other systems.
I want that whenever JobA finishes each build, it should call an endpoint on my web service and post the build data (bascically the same data that I get from calling the Jenkins API: https://<my-jenkins-server>/job/JobA/<buildNumber>/api/json) to the end point of my web service, which accepts a POST requests and JSON payload to update the build details like job name, build number, build result, build url, build duration, timestamp, test results etc. in my database.
The endpoint of my web service should still be called even if the build fails.
I can already get all this data from the above REST API call, but I do not want to use that because, using that if I want to get the updated data in my database, I will have to poll the Jenkins REST API multiple times, (like maybe every 15 minutes or more frequently) which will increase the load on my Jenkins server.
I know that I can set up a Jenkins job to either call a script or call another downstream job through which I can then call a curl POST request to my end point.
But my problem is that I do not know where/how to get the JSON data for the build be sent in the POST request?
Can someone help me out please?
You can get the lastbuild information and there is a field called building which says if the build is running or not.
https://host/job/project/lastBuild/api/json?depth=1
I Currently have a Pipeline built in Jenkins to run my newman Test Cases. So the definition for it is in three steps
Call the Async Test cases.
Register a Webhook and Wait for it to respond back to continue forward.
Call the remaining Test cases.
I'm using the existing "Webhook Step Plugin" to register and wait for my Webhook in Jenkins.
Definition:
Pipeline Definition
Problem:
This resgisterWebhook() method is returning a random URL everytime because it is using the UUID logic to generate a random token. And since its random everytime, my external System cannot know it.
Question:
So i'm looking for a way, where the hook URL will be constant always, so that it can be hardcoded in my external System and called once the Async Operation is completed.
Don't use the webhook step, either use the webhook trigger plugin, or use the generic webhook trigger inside pipeline. (these may be the same thing - I may have just stuttered. Sorry if so!).
Anyway, that lets you set up a static URL you can use to trigger builds.
Using Jmeter GUI, I recorded a test scenario (placing an order) and the script ran successfully. But when I replay the test scripts it doesn't function as it was recorded to do, it did not make an order.
After query the dev, found that with each item selected, the server generate a CSRF token, and put the token in the URL path (Like: /cart/add/type/product_id/7245985/_csrf_token/b46c0aec2e5891808ec42141b1956943204ae8f8) when the item is added to the shopping cart. This is all recorded in the script. This path with the token is used to add the item to cart.
My question is how to test this dynamic token when it is concatenated in the path of URL?
Any help are appreciated.
If you have not already added Tree View Listener to your Test Plan, then add it now. You can use it to view the details of requests & responses. JMeter considers a request successful if it gets "some" response from Server-side. It does not matter if the response is functionally valid or not. So, in order to make sure that JMeter is sending valid parameters and receiving expected response, you will have to check the details of requests / responses in Tree view listener.
You can also add Response Assertions to requests so JMeter itself verifies that it is getting expected responses.
Important Tips:
Use TreeView Listener for debugging only. In real load test keep it disabled as it consumes lot of memory.
Do not use response assertions excessively as they consume lot of memory as well.
JMeter is not a browser-based tool. It just deals with back-end requests. Hence it is expected to be very fast. So nothing wrong with that. You should remove un-necessary timers as there is nothing wrong with it being fast.
If your requests involve some kind of login authorization then have a look at this question for further details Load testing using jmeter with basic authentication
Recording doesn't guarantee working script, it gives you only a "skeleton" and usually you need to perform some correlation (the process of extracting mandatory dynamic parameter from previous response and adding it to the next request).
Reference material:
Building a Web Test Plan
Building an Advanced Web Test Plan
How to use JMeter for Login Authentication?
How to make JMeter behave more like a real browser
Is there a way in Testdroid to email test-reports automatically?
Ideally I would like to set up a configuration where after every test run, we can send everyone on a team the test results.
Test finished hook:
Test finished hook: If you set hook URL, POST call will be sent to that URL when test run is finished.
But you will need to implement a web service etc for the mailing function.
Testdroid Cloud integration API
http://help.testdroid.com/customer/portal/topics/810087-testdroid-cloud-integration-api/articles