I'm writing coded UI test's for an ASP.NET MVC web application. I'm using Visual Studio 2012.
Basically I have a bunch of test methods like below:
I use a ordered test file to run all of them at once in the order I would like.
The problem is that the way the tests work is that if one test fails, it is marked as ref/failed in the output, and often a test will work if I run it again. I want to be able to set my ordered test to run and if a test method fails it will automatically try to run that test method one more time.
How do I do this? I was thinking of putting an if statement at the end of each test method, like:
if(TestFailed) {
RecallTest();
}
I would use some sort of counter, but how do I write the above in a coded UI test?
There are several ways you can handle this. The most obvious is something like this:
[TestMethod]
public void myTestMethod()
{
int iteration = 0;
if (iteration < 2)
{
try
{
// the usual test code goes here
}
catch (Exception)
{
// handles any exception kind
iteration++;
myTestMethod();
}
}
}
This will recursively call the test method only if the method fails. You can set it to catch any exception (Exception) or to catch specific exceptions only - just change the exception kind in the catch block.
This is typically discouraged in CodedUI tests because it can interact badly with the test runner's built in exception reporting (I'd recommend looking at adding logging to report that you got a failure). It will give you the one-time only rerun, however (Caveat: I haven't actually tried this myself - you might need to tweak the code to ensure that your test cleanup occurs depending on your test code).
Try using the playback error handling:
Playback.PlaybackError += Playback_PlaybackError;
This way you can register an event handler that is called every time the playback encounters an error.
In the event handling method you can tell the playback to repeat the action that threw the error:
static void Playback_PlaybackError(object sender, PlaybackErrorEventArgs e)
{
e.Result = PlaybackErrorOptions.Retry;
}
A counter can be added to prevent an endless loop of error and retry.
Related
I have a method written in a Grails service, which processes a lot of data.
I noticed that, sometimes, the method returns success but the data is not persisted to the database.
I debugged it, following all the data till the end of the method and everything is fine, however data is not persisted.
The following image demonstrates the what I just explained. You can see the end of the method, in which a Map object is filled with persistent object metadata. Even you can see the console which contains the printend Hibertate SQL
How can I detect whether a rollback mechanism is thrown after successful method returning?
This is my connection properties for Oracle 12c database. Others configurations are Grails defaults
dataSource.pooled=true
hibernate.jdbc.use_get_generated_keys=true
hibernate.cache.use_second_level_cache=true
hibernate.cache.use_query_cache=false
hibernate.cache.region.factory_class=org.hibernate.cache.ehcache.EhCacheRegionFactory
dataSource.driverClassName=oracle.jdbc.driver.OracleDriver
dataSource.dialect=org.hibernate.dialect.OracleDialect
dataSource.url=jdbc:oracle:thin:#172.16.1.20:1521:db
dataSource.username=<USER>
dataSource.password=<PASS>
hibernate.default_schema=<SCHEMA>
The service is anotated as #Transactional
#Transactional
class SincronizacionService {
}
Any Idea?
When using GORM's save method, also use failOnError:true. By default, save method silently fails. However, if you use failOnError:true, it will throw an exception if the data is not persisted.
If you do not want to stop the program when the data fails to save, you can use the try-catch block to log data that failed to save and let the algorithm continue to do it work.
Hope that helps.
I found the problem. In this method actaDenunciaService.generarActaDenuncia(denuncia), there is a peculiarity. In a part of the method is located the following snippet:
try {
DNomenclador nomenclador = nomencladorService.obtenerNomencladorDNomenclador(meta.valor.toLong())
if (!nomenclador) {
return toReturn(limpiarTexto(meta.valor))
} else {
return toReturn(nomenclador.valor)
}
} catch (Exception e) {
return toReturn(limpiarTexto(meta.valor))
}
A team member changed this line nomencladorService.obtenerNomencladorDNomenclador(meta.valor.toLong()). The change represented a huge improvement of memory saving. However, the team member did not take into account a business process, which does not take into account the method he used.
Yes, a runtime exception is being thrown.
And the treatment, depending on the objective of the method, is correct
For the future, this is how the method will be from now on:
try {
DNomenclador nomenclador = nomencladorService.obtenerNomencladorDNomencladorLibre(meta.valor.toLong())
if (!nomenclador) {
return toReturn(limpiarTexto(meta.valor))
} else {
return toReturn(nomenclador.valor)
}
} catch (Exception e) {
e.printStackTrace()
return toReturn(limpiarTexto(meta.valor))
}
nomencladorService.obtenerNomencladorDNomencladorLibre(meta.valor.toLong()) for the business process
e.printStackTrace() for tracing any other peculiarity
Thanks a lot to everybody who had collaborated on finding this error
I found the error!
An error thrown inside a method for generating a PDF document with data, appearsto be failing. The second line shows this
try {
denuncia.xmlFirmadoServ = dfileManagerService.guardarDFile(signatureResponse.resultado, "xmlfirmadoservidor.xml", usuario)
denuncia = actaDenunciaService.generarActaDenuncia(denuncia).denuncia
} catch (Throwable t) {
denunciaUtilService.incrementarNumeroDenuncia(true)
throw t
}
Now, the new question is: If the method is encapsulated inside a try/catchblock, why the catch block is not excecuting?
When I comment the 2nd line inside try/catch block, data is persisted on database
With no comments, generation PDF method is executed till the end, doing all what it must do
Rollback is done here as expected:
#Transactional(propagation = Propagation.REQUIRES_NEW)
def test1() {
def dummy = new Dummy(name: "test1")
dummy.save()
throw new RuntimeException("test1!")
}
But here not - which is probably wrong - try/catch should not affect the behavior:
#Transactional(propagation = Propagation.REQUIRES_NEW)
def test2() {
def dummy = new Dummy(name: "test2")
dummy.save()
try {
throw new RuntimeException("test2!")
} catch (all) {
println all.message
}
}
By default, #Transactional wraps the method such that any non-checked exception (viz., a RuntimeException) will cause the transaction to be rolled back.
If you catch/handle the exception within the method, of course, the exception doesn't propagate up to the transactional wrapper and the transaction won't be marked as rollback-only. This appears to be what you're doing.
It's worth pointing out that you can indicate that the transactional wrapper should rollback transactions if other Exceptions are thrown (and propagate to the wrapper). You can do this with the rollbackFor annotation parameter.
For example,
#Transactional(rollbackFor=Throwable.class)
void doTransactionalWork() throws MyException { ... }
will cause the transaction to be rolled back if any Throwable propagates up to the wrapper, even those that are checked (viz., MyException)
This should be the behavior of any #Transactional method, regardless of whether you're creating a new transaction or inheriting an existing transactional context.
maybe you have misunderstood the purpose of try catch or maybe you are just having a wobbly moment:
#Transactional(propagation = Propagation.REQUIRES_NEW)
def test2() {
//you may be doing other stuff here
//but now about to do some transaction work
//so lets wrap this method around a try catch
try {
//this is happening
def dummy = new Dummy(name: "test2")
dummy.save()
} catch (Exception all) { // or catch (Throwable all) {
// if something went wrong in above save method
//should be caught and runtime exception means roll back
throw new RuntimeException("test2!" +all?.toString())
}
}
I hope it explains where you went wrong but really you wish to do all of this in a service and do the try catch part in the controller -
so you do you transaction work and if things go wrong you may wish to throw additional exceptions from the service that the try catch in the controller would capture and set it to roll back.
I did a sample project years back here hope it helps
eitherway those are someone's experiments and aren't really the way you would go about doing proper coding, I mean it is a rather odd unusual way of doing things and in short he is just trying to make it throw a runtime exception therefore triggering roll back. I stick with my suggestion in the answer that you want to do a one off try catch in the controller. That attempts to capture both validation errors of the object at hand as well as failures within the failure of any given service transactional work. Something like this but probably a lot more work to capture all specific issues and return back to originating page with the underlying issues - having also now rolled back transaction.
I'm currently learning/testing BDD using SpecFlow, and it works great!
Before I choose to ask my question, I have read this one, and I felt like I had to ask my question despite the fact that the same problem is addressed, because of the Exception scenario which is not mentioned.
I'm actually testing this scenario:
Scenario: I should get an error whenever I try to remove an item from an empty stack
Given I have an empty stack
When I pop from it
Then I should get an error
public class StackBehaviour {
public void GivenIHaveAnEmptyStack() { stack = new CustomStack<string>(); }
// This will throw whenever called!
// So the Then method will never be run!
// I feel like I should just put a comment which says why it's empty,
// allowing a fellow programmer to understand the exact intention.
public void WhenIPopFromIt() { stack.Pop(); }
// It is here that it verifies whether the CustomStack meets the expected behaviour.
public void ThenIShouldGetAnError() {
Assert.Throws<IndexOutOfRangeException>(delegate {
stack.Pop();
});
}
private CustomStack<string> stack;
}
public class CustomStack<T> {
public T Pop() {
if (stack.Count == 0)
throw new IndexOutOfRangeException("Cannot pop from an empty stack!");
T item = stack[stack.Count-1];
stack.RemoveAt(stack.Count-1);
return item;
}
private ArrayList stack = new ArrayList();
}
I think that leaving a comment in the When method is correct, so that the business requirement doesn't lack any information, and on the code behind, I put it clear what my intention is exactly by commenting.
What do you think? Any other ideas why I shouldn't make it?
There is another trick that you can use where the bindings help make the feature's meaning a little clearer. In this case, let's start at the end.
Then I should get an error
Your problem is a basically here, you know want an error, but you don't know how to get it. In fact you've already missed it, the error has already occurred in the When step, and in your code example, you moved the action into the then step just so you could get the error.
But what if keep the when performing the action amd re-express our then to reflect what really happens
Then I should have had an error
Seems a trivial change but now our feature reflects that the error should have been associated with the When step and we are simply evaluating it later, and that is something we can code. We just need something to remember the error in the when and deliver it to the then.
private Exception errorFromWhen = null;
public void WhenIPopFromIt()
{
try
{
stack.Pop();
}
catch(Exception ex)
{
errorFromWhen = ex;
}
}
public void ThenIShouldGetAnError()
{
errorFromWhen.ShouldNotBeNull();
errorFromWhen.ShouldBe<IndexOutOfRangeException>();
}
SpecFlow has absolutely no problems with this, in fact due to its mini Dependency injection system, you can even pass this state between binding classes if necessary.
May a scenario not have a When in BDD?
In Specflow, neither given, when or then are mandatory.
However in your example, I don't believe this is a good use of Specflow and BDD. In this answer here Marcus states:
"BDD is about ensuring that we're building the right thing, TDD is about ensuring that we're building it right."
In your example the scope of what is being tested i.e. the CustomStack, should be tested via TDD. It is the end solution that makes use of the CustomStack should be tested via BDD (and hence SpecFlow) e.g. if this CustomStack was being exercised via a certain action on a website.
This answer is also pertinent to your question.
In my MVC service layer I have code such as the following to validate:
protected bool ValidateAccount(Account account)
{
var accounts = _accountRepository.GetPk(account.PartitionKey);
if (accounts.Any(b => b.Title.Equals(account.Title) &&
!b.RowKey.Equals(account.RowKey)))
_validationDictionary.AddError("", "Duplicate title");
return _validationDictionary.IsValid;
}
However in the "action type" methods I absorb exceptions with code like this:
public bool Create(Account account)
{
if (!ValidateAccount(account))
return false;
try
{
_accountRepository.AddOrUpdate(account);
}
catch
{
return false;
}
return true;
}
My controller is coded like this:
public ActionResult Create(BaseViewModel vm)
{
_accountService = new AccountService(new ModelStateWrapper(this.ModelState), vm.Meta.DataSourceID);
if (ModelState.IsValid)
{
_accountService = new AccountService(new ModelStateWrapper(this.ModelState), vm.Meta.DataSourceID);
if (!_accountService.Create(vm.Account))
return View("CreateEdit", vm);
else
return RedirectToAction("Created");
}
return RedirectToAction("Home");
}
return View("CreateEdit", vm);
}
Is this a reasonable approach to take? My one concern is that I might be losing exception information in the service layer.
You should at the very least log your exception somewhere (event log, file system, using Elmah, etc).
The problem with your code is that you'll never know if something bad happened in production environment or you won't be able to discover what failed exactly (just that the code returned false).
Plus, you should never handle all exceptions like you do (unfiltered catch) but only the one you can revert back to a normal state. It's far better to let the application crash than to keep it online unstable.
Avoid handling errors by catching non-specific exceptions, such as System.Exception, System.SystemException, and so on, in application code. There are cases when handling errors in applications is acceptable, but such cases are rare.
See this MSDN article on best practices for handling exceptions and Design Guidelines for Exceptions
Since you are not doing anything with the exception in your Create method in the service layer, I would suggest removing the try/catch from it.
I would still return true or false to indicate whether the create operation was successful. I will however add a try/catch in the caller to make sure exceptions are handled. In your case that could be in your controller action (or the OnException in your base controller.)
Another approach is to leave the try/catch in your Create method but as #Stephane suggested do something with it (like log it) but you could also log it wherever you catch it.
I'm testing come code in a service with transactional set to true , which talks
to a customer supplied web service the main part of which looks like
class BarcodeService {
..
/// some stuff ...
try{
cancelBarCodeResponse = cancelBarCode(cancelBarcodeRequest)
} catch(myCommsException e) {
throw new RuntimeException(e)
}
...
where myCommsException extends Exception ..
I have a test which looks like
// As no connection from my machine, it should fail ..
shouldFailWithCause(RuntimeException){
barcodeServices.cancelBarcodeDetails()
}
The test fails cause it's catching a myCommsException rather than the
RuntimeException i thought i'd converted it to .. Anyone care to point out what
i'm doing wrong ? Also will the fact that it's not a RuntimeException mean any
transaction related info done before my try/catch actually be written out rather
than thrown away ??
Thanks
From what I see, it looks ok. The problem might be in the ///some stuff and the ... parts of the code. Use a debugger to find out exactly where the exception is being thrown.