Suppose I use QTPs recovery scenario manager to set the playback synchronization timeout to 0. The handler would return with "continue with next statement".
I'd do that to make sure that any following playback statements don't waste their time waiting for the next non-existing/non-matching step before failing:
I have a lot of GUI tests that kind of get stuck because let's say if 10 controls are missing, their (consecutive) playback steps produce 10 timeout waits before failing. If the playback timeout is 30 seconds, I loose 10x30 seconds=5 minutes execution time while it really would be sufficient to wait for 30 seconds ONCE (because the app does not change anymore -- we waited a full timeout period already).
Now if I have 100 test cases (=action iterations), this possibly happens 100 times, wasting 500 minutes of my test exec time window.
That's why I come up with the idea of a recovery scenario function setting the timeout to 0 after/upon the first failed playback step. This would accelerate the speed while skipping the rightly-FAILED step, yet would not compromise the precision/reliability of identifying the next matching GUI context (which creates a PASSED step).
Then of course upon the next passed playback step, I would want to restore the original timeout value. How could I do that? This is my question.
One cannot define a recovery scenario function that is called for PASSED steps.
I am currently thinking about setting a method function for Reporter.ReportEvent, and "sniffing" for PASSED log entries there. I'd install that method function in the scenario recovery function which sets timeout to 0. Then, when the "sniffer" function senses a ReportEvent call with PASSED status during one of the following playback steps, I'd reset everything (i.e. restore the original timeout, and uninstall the method function). (I am 99% sure, however, that .Click and .Set methods do not call ReportEvent to write their result status...so this option might probably not work.)
Better ideas? This really bugs me.
It sounds to me like you tests aren't designed correctly, if you fail to find an object why do you continue?
One possible (non recovery scenario) solution would be to use RegisterUserFunc to override the methods you are using in order to do an obj.Exist(0) before running the required method.
Function MyClick(obj)
If obj.Exist(1) Then
obj.Click
Else
Reporter.ReportEvent micFail, "Click failed, no object", "Object does not exist"
End If
End Function
RegisterUserFunc "Link", "Click", "MyClick"
RegisterUserFunc "WebButton", "Click", "MyClick"
''# etc
If you have many controls of which some may be missing and you know that after 10 seconds you mentioned (when the first timeout occurs), nothing more will show up, then you can use the exists method with a timeout parameter.
Something like this:
timeout = 10
For Each control in controls
If control.exists(timeout) Then
do something with the control
Else
timeout = 0
End If
Next
Now only the first timeout will be 10 seconds. Each and every subsequent timeout in your collection of controls will have the timeout set to 0 which will save your time.
Related
I have a scenario where after I disable a button, I check for the data persistence in the database. It takes some time to persist data in the database( roughly 3 mins). My tests are started through sauce labs so after 90 seconds the time out and my session is closed.
I do take screenshots of the tests at the tearDown Method. when data persistence takes more than 90 seconds the screenshots method is failing. I want to take screenshots only when the driver is alive, how can I check for it?
takeAllureScreenShot();
}```
You can increase how long Sauce Labs waits before shutting down a session by configuring the idleTimeout desired capability (docs for which are here).
By default, this is set to 90 seconds; It sounds like you should increase it to something like 200 seconds.
Assuming you're using Java and a Selenium 3 session with vendor name-spacing, you could do that like so:
// This is a new capabilities object to hold the nested vendor-specific options
MutableCapabilities sauceOptions = new MutableCapabilities();
sauceOptions.setCapability("idleTimeout", 200);
// assuming your desired capabilities are called 'capabilities'
capabilities.setCapability("sauce:options", sauceOptions);
(If you just wanted to check that the session was still alive, you could do so by doing something "trivial" like checking the page title inside an try/catch block. If an exception is thrown, the session is over! If you get a response, it's not).
I am using the Serverless Framework to consume messages from SQS. Some of the messages sent to the queue do not get consumed. They go straight to the in-flight SQS status and from there to my dead letter queue. When I look at my log of the consumer, I can see that it consumed and successfully processed 9/10 messages. One is always not consumed and ends up in the dead letter queue. I am setting reservedConcurrency to 1 so that only one consumer can run at a time. The function consumer timeout is set to 30 seconds. This is the consumer code:
module.exports.mySQSConsumer = async (event, context) => {
context.callbackWaitsForEmptyEventLoop = false;
console.log(event.Records);
await new Promise((res, rej) => {
setTimeout(() => {
res();
}, 100);
});
console.log('DONE');
return true;
}
Consumer function configuration follow:
functions:
mySQSConsumer:
handler: handler.mySQSConsumer
timeout: 30 # seconds
reservedConcurrency: 1
events:
- sqs:
arn: arn:aws:sqs:us-east-1:xyz:my-test-queue
batchSize: 1
enabled: true
If I remove the await function, it will process all messages. If I increase the timeout to 200ms, even more messages will go to straight to the in-flight status and from there to the dead letter queue. This code is very simple. Any ideas why it's skipping some messages? The messages that don't get consumed don't even show up in the log using the first console.log() statement. They seem entirely ignored.
I figured out the problem. The SQS queue Lambda function event triggering works differently than I thought. The messages get pushed into the Lambda function, not pulled by it. I think this could be engineered better by AWS, but it's what it is.
The issue was the Default Visibility Timeout set to 30 seconds together with Reserved Concurrency set to 1. When the SQS queue gets filled up quickly with thousands of records, AWS starts pushing the messages to the Lambda function at a rate that is faster than the rate at which the single function instance can process them. AWS "assumes" that it can simply spin up more instances of the Lambda to keep up with the backpressure. However, the concurrency limit doesn't let it spin up more instances - the Lambda function is throttled. As a result, the function starts returning failure to the AWS backend for some messages, which will, consequently, hide the failed messages for 30 seconds (the default setting) and put them back into the queue after this period for reprocessing. Since there are so many records to process by the single instance, 30 seconds later, the Lambda function is still busy and can't process those messages again. So the situation repeats itself and the messages go back to invisibility for 30 seconds. This repeats total 3 times. After the third attempt, the messages go to the dead letter queue (we configured our SQS queue that way).
To resolve this issue, we increased the Default Visibility Timeout to 5 minutes. That's enough time for the Lambda function to process through most of the messages in the queue while the failed ones wait in invisibility. After 5 minutes, they get pushed back into the queue and since the Lambda function is no longer busy, it will process most of them. Some of them have to go to invisibility twice before being successfully processed.
So the remedy to this problem is either increasing the Default Invisibility Timeout like we did or increasing the number of failures necessary before a message goes to the dead letter queue.
I hope this helps someone.
I have a wxLua Gui app that has a "Run" button. Depending on selected options, Run can take a long time, so I would like to implement a "Cancel" button/feature. But it looks like everything in wxLua is working on one Gui thread, and once you hit Run, pressing Cancel does nothing, the Run always goes to completion.
Cancel basically sets a variable to true, and the running process regularly checks that variable. But the Cancel button press event never happens while Running.
I have never used co-routines; if the Run process regularly yields to a "Cancel check" process, will the Cancel event happen then?
Or is there another way?
(the following assumes that by "Run" you mean a long running operation in the same process and not running an external process using wxExecute or wxProcess.)
"Cancel" event is not triggered because by executing your Run logic you have not given a chance to the UI to handle the click event.
To avoid blocking the UI you need to do something like this. When you click Run button create a co-routine around the function you want to run:
coro = coroutine.create(myLongRunningFunction)
Your Run event is completed at this point. Then in EVT_IDLE event you will be resuming this coroutine as long as it's not complete. It will look something like this:
if coro then -- only if there is a coroutine to work on
local ok, res = coroutine.resume(coro, additional, parameters)
-- your function either yielded or returned
-- you may check ok to see if there was an error
-- res can tell you how far you are in the process
-- coro can return multiple values (just give them as parameters to yield)
if coroutine.status(coro) == 'dead' then -- finished or stopped with error
coro = nil
-- do whatever you need to do knowing the process is completed
end
end
You will probably need to request more IDLE event for as long as your process is not finished as some operating systems will not trigger IDLE events unless there is some other event triggered. Assuming your handler has event parameter, you can do event:RequestMore(true) to ask for more IDLE events (RequestMore).
Your long-running process will need to call coroutine.yield() at the right time (not too short as you will be wasting time to switch back and forth and not too long for users to notice delays in the UI); you probably need to experiment with this, but something timer-based with 100ms or so between calls may work.
You can check for Cancel values either in your IDLE event handler or in the long-running function as you do now. The logic I described will give your application UI a chance to process Cancel event as you expect.
I don't use WXWidgets, but the way I implement cancel buttons in my lua scripts which use IUP is to have a cancel flag, which is set when the button is pressed and the progress display is checked for during the run.
Usage is like this
ProgressDisplay.Start('This is my progress box',100)
for i=1,100 do
ProgressDisplay.SetMessage(i.." %")
fhSleep(50,40) -- Emulate performing the task
ProgressDisplay.Step(1)
if ProgressDisplay.Cancel() then
break
end
end
ProgressDisplay.Reset()
ProgressDisplay.Close()
If you want to see the definition for the ProgressDisplay see:
http://www.fhug.org.uk/wiki/doku.php?id=plugins:code_snippets:progress_bar
I've been working with Jetty's Continuations API and just figured out a very frustrating bug (* at least I think this was the reason*).
If I had read more closely, I would have seen this:
The timeout should be set before the suspend, by a call to
continuation.setTimeout(long); if no timeout is set, then the default
period is used.
So in my code, I was simply suspending a continuation without setting a timeout. When I read this in the documentation...
A timeout of <=0 means the continuation will never expire.
... and then tried setting the timeout to something negative, things started working (* at least it seems so *).
Unfortunately, I haven't been able to figure out what the "default period" is for a timeout. Where can I find this out?
I guess I figured this out while stepping through the debugger... the default time period is 30 seconds. I found the source code...
http://www.jarvana.com/jarvana/view/org/eclipse/jetty/aggregate/jetty-all/8.0.4.v20111024/jetty-all-8.0.4.v20111024-sources.jar!/org/eclipse/jetty/continuation/FauxContinuation.java?format=ok
... and it seems like this FauxContinuation implementation of a Continuation was being constructed and the timeout is initialized to 30 seconds.
private long _timeoutMs = 30000;
I'm using Polly to retry web service calls in case the call fails with WebException, because I want to make sure the method executed correctly before proceeding. However sometimes web methods still throw exception even after retrying several times and I don't want to retry forever. Can I use Polly to show some confirmation dialog, e.g. "Max retry count reached! Make sure connection is enabled and press retry." Then retry counter should reset to initial value and start again. Can I achieve this using only Polly or should I write my own logic? Ideas?
Polly has nothing in-built to manage dialog boxes as it is entirely agnostic to the context in which it is used. However, you can customise extra behaviour on retries with an onRetry delegate so you can hook a dialog box in there. Overall:
Use an outer RetryForever policy, and display the dialog box in the onRetry action configured on that policy.
If you want a way for the user to exit the RetryForever, a cancel action in the dialog could throw some other exception (which you trap with a try-catch round all the policies), to cause an exit.
Within the outer policy, use an inner Retry policy for however many tries you want to make without intervention.
Because this is a different policy instance from the retryforever, and has fixed retry count, the retry count will automatically start afresh each time it is executed.
Use PolicyWrap to wrap the two retry policies together.
In pseudo-code:
var retryUntilSucceedsOrUserCancels = Policy
.Handle<WhateverException>()
.RetryForever(onRetry: { /* show my dialog box*/ });
var retryNTimesWithoutUserIntervention = Policy
.Handle<WhateverException>()
.Retry(n); // or whatever more sophisticated retry style you want
var combined = retryUntilSucceedsOrUserCancels
.Wrap(retryNTimesWithoutUserIntervention);
combined.Execute( /* my work */ );
Of course the use of the outer RetryForever() policy is just an option: you could also build the equivalent manually.