Is there a way in Testdroid to email test-reports automatically?
Ideally I would like to set up a configuration where after every test run, we can send everyone on a team the test results.
Test finished hook:
Test finished hook: If you set hook URL, POST call will be sent to that URL when test run is finished.
But you will need to implement a web service etc for the mailing function.
Testdroid Cloud integration API
http://help.testdroid.com/customer/portal/topics/810087-testdroid-cloud-integration-api/articles
Related
Because we need to have the ability to schedule email delivery, we have migrated to using Sidekiq to send emails with the deliver_later method. Under the old regime that used deliver_now, our testing could use
ActionMailer::Base.deliveries[index]
to inspect the recipient, subject, body, attachments, etc...
For testing purposes, is there an equivalent mechanism to inspect the contents of queued email when using Sidekiq and deliver_later?
For testing contents of the email, you have to render the email template for which the job needs to be executed. I suggest you should separate unit tests:
Job spec to check if email is getting enqueued with correct parameters.
Email spec to check the email contents.
If you want to go the end-to-end style, use Sidekiq::Testing.inline! to execute the job and then use ActionMailer::Base.deliveries[index] like before.
The solution turned out to be to execute perform_enqueued_jobs after all emails were queued. After this, all of the existing testing mechanisms worked.
See https://api.rubyonrails.org/v7.0.4/classes/ActiveJob/TestHelper.html for additional information.
We are using Jenkins pipeline to run jmeter tests for testing one of our application API. EVeryting is working ok but there are cases where the Application returns an error. We would like to log the request payload for such failures and also the timestamp so that we can investigate in the application about corresponding failures.
Is there a way, I can instruct jmeter to log the Request Data for cases which result in failure?
The easiest option is adding a Listener like Simple Data Writer to your test plan.
The configuration to save the timestamp and payload would look like:
Once the test finishes you will be able to observe requests details (if any) using View Results Tree listener.
More information: How to Save Response Data in JMeter
We do something called feature testing like so -> https://blog.twitter.com/engineering/en_us/topics/insights/2017/the-testing-renaissance.html
TLDR of that article, we send request to microservice(REST POST with body), mock GCP Storage, mock downstream api call so the entire microservice can be refactored. Also, we can swap out our platforms/libs with no changes in our testing which makes us extremely agile.
My first questions is can DataFlow (apache beam) receive a REST request to trigger the job? I see much of the api is around 'create job' but I don't see 'execute job' in the docs while I do see get status returns the status of job execution. I just don't see a way to trigger a job to
read from my storage api (which is mockable and sits in front of GCP)
process the file hopefully across many nodes
call the apis downstream (which is also mockable)
Then, I simply want to in my test simulate the http call, then when file is read, return a real customer file and then after done, my test will verify all the correct requests were sent to the apis downstream.
We are using apache beam in our feature tests though not sure if it's the same version as google's dataflow :( as that would be the most ideal!!! -> hmmm, is there a reported apache beam version of google's dataflow we can get?
thanks,
Dean
thanks,
Dean
Apache Beam's DirectRunner should be very close to Dataflow's environment, and it's what we recommend for this type of single-process pipeline test.
My advise would be the same: Use the DirectRunner for your feature tests.
You can also use the Dataflow runner, but that sounds like it would be a full integration test. Depending on the data source / data sink, you may be able to pass it mocking utilities.
BigQueryIO is a good example. It has a withTestServices method that you can use to pass objects that mock the behavior of external services
I Currently have a Pipeline built in Jenkins to run my newman Test Cases. So the definition for it is in three steps
Call the Async Test cases.
Register a Webhook and Wait for it to respond back to continue forward.
Call the remaining Test cases.
I'm using the existing "Webhook Step Plugin" to register and wait for my Webhook in Jenkins.
Definition:
Pipeline Definition
Problem:
This resgisterWebhook() method is returning a random URL everytime because it is using the UUID logic to generate a random token. And since its random everytime, my external System cannot know it.
Question:
So i'm looking for a way, where the hook URL will be constant always, so that it can be hardcoded in my external System and called once the Async Operation is completed.
Don't use the webhook step, either use the webhook trigger plugin, or use the generic webhook trigger inside pipeline. (these may be the same thing - I may have just stuttered. Sorry if so!).
Anyway, that lets you set up a static URL you can use to trigger builds.
I have a rake task in my rails application that published jobs to facebook. And then changes some model values. So one way to test the rake task is to invoke the rake task and check values that have been changed.
test 'z' do
# setup some data
Rake::Task['job:publish_to_facebook'].invoke
Rake::Task['job:publish_to_facebook'].reenable
# assert table values that has been changed.
end
But how I can test whether jobs are successfully published on facebook? Is there any better strategy except using capybara and selenium-webdriver ?
Even if i use stubbing and mocking then how can i verify that my jobs are published on facebook?
Most tests should not contact an external API, mainly because it will slow down tests and you might also run into rate limits.
Even if i use stubbing and mocking then how can i verify that my jobs are published on facebook?
The point of stubbing and mocking is precisely not to publish to Facebook. Instead, you would create a class called Facebook (for example) with a method like def post_message(message). This is the app's front door to Facebook, all calls to Facebook go through this class. Then you can use a library like Mocha to overwrite def post_message during testing. You can use it to verify that the application is attempting to post a message, and verify the message itself is correct. It won't actually post the message.
As I mentioned, you do want to make some tests with real calls to Facebook (though not many). These could be in a test like you've shown above, which is an integration test, or it could also be a smaller unit test of the Facebook class I suggested above, which would be a better starting point. For this purpose, you'd want to establish a test account on Facebook. Then your test should clear all messages in the setup and use Facebook's API to verify that the messages were actually posted.