I want to execute a block of code that changes based on a pass or a fail. Basically looking to set the results from a test in our test case tracking software. I created my own test rules, shown below, but no matter if the test passes or fails it always just calls succeeded.
public class TestRules extends TestWatcher {
#Override
public void succeeded(Description description){
log.error("This should only be called when passing");
}
#Override
public void failed(Throwable e, Description description) {
log.error("This Should only be called when Failing");
}
}
I set the rule in the test as follows
#Rule
public TestRules testRules = new TestRules()
I am hoping I am just missing something stupid and simple.
Related
BaseTest.java:
private static ReportService reportService; // Calling report service interface
#BeforeSuite:
reportService = new ExtentReportService(getConfig()); // New instance of ExtentReportService.
#BeforeMethod:
reportService.startTest(testname); // Starting the test and passing the name and description of the test.
#AfterMethod:
reportService.endTest(); // Ending the test
#AfterSuite:
reportService.close(); // Closing the test
**ExtentReportService.java:** // Contains different extent API methods. (These are designed to be generic.)
protected static ExtentReports extent; // static instance of ExtentReports
protected static ExtentTest test; //static instance of ExtentTTest
#Override // StartTest method
startTest(Method method) {
testMetaData = getTestMetaData(method);
test=extent.startTest(testMetaData.getId(),testMetaData.getSummary());
}
#Override //End test method
endTest() {
extent.endTest(test);
extent.flush();
}
The above is my selenium code.
When I am executing my suite file with parallel="methods" and thread count="3", I am getting the following error: "com.relevantcodes.extentreports.ExtentTestInterruptedException: Close was called before test could end safely using EndTest.".
While debugging, I found that even before all endTest() in AfterMehtod were executed, AfterSuite was being called.
I tried different variations such that the code works, such as, removing static, calling endTest() in the test itself rather than after method, removing close() call from AfterSuite and many other variations. But still getting the same error.
I tried all the possible solutions given on the internet, but to no use.
Attaching a hierarchy file for the ExtentReport used in my project
I also the following solution given in StackOverflow:
Extent report :com.relevantcodes.extentreports.ExtentTestInterruptedException: Close was called before test could end safely using EndTest
Unsynchronized output
XMF file for parallel test.
ExtentReports Intialized in ExtentManager class using Singleton().
public class ExtentManager {
private static ExtentReports extent;
public static ExtentReports getInstance() {
if(extent == null) {
extent = new ExtentReports(System.getProperty("user.dir")+"\target\surefire-reports\html\extent.html", true, DisplayOrder.OLDEST_FIRST);
extent.loadConfig(new File(System.getProperty("user.dir")+"src\test\resources\extentconfig\ReportsConfig.xml"));
}
return extent;
}
}
Declared in TestBase class as global.
public ExtentReports repo= ExtentManager.getInstance();
public static ExtentTest test
Call startTest in public void onTestStart(ITestResult result)
test = repo.startTest(result.getName().toUpperCase());
Call endTest in CustomListener Class both in a)public void onTestFailure(ITestResult result); b)public void onTestSuccess(ITestResult result).
repo.endTest(test)
Call close() OR flush() in #AfterSuite in TestBase class but NOT both!
//repo.close();
repo.flush();
Note: I have ExtentReports ver-2.41.2, and TestNg ver-7.1.0.
After the above steps, error 'Getting closed before endTest call in Selenium using Extent Reports' got resolved.
Extent report generates each test successfully in the report.
Try it out!
So I have a few tests where i've reused steps from within steps.
But i'm now having a nightmare on the maintenance front, in that I cant easily navigate between the steps.
Here's an example:
[Given(#"I have an order")]
public void GivenIHaveAnOrder()
{
Given("an open store");
Given("I am an existing customer");
Given("I am on homepage");
When("I search for a component");
When("I add the component to my basket");
}
How do I navigate to one of those internal steps?
If I wanted to navigate to the "When("I search for a component");" step I cant.
If I were on the feature file I could simply right click the step and "go to definition" but I cant do that here. Does anyone have a solution?
I assume that you are calling the steps with the Given/When- functions, because they are in a different binding class. Am I right?
There is a better way of doing it, than using this functions.
Did you had a look at the driver concept and context injection?
Have a look here: http://www.specflow.org/documentation/Context-Injection/
Simply extract your logic of your steps to a driver class and get an instance from it in the different step classes:
class Driver
{
public void AnOpenStore()
{
...
}
}
[Binding]
public class StepClass1
{
private Driver _driver;
public StepClass1(Driver driver)
{
_driver = driver;
}
[Given(#"I have an order")]
public void IHaveAnOrder()
{
_driver.AnOpenStore();
}
}
[Binding]
public class StepClass2
{
private Driver _driver;
public StepClass2(Driver driver)
{
_driver = driver;
}
[Given(#"an open store")]
public void AnOpenStore()
{
_driver.AnOpenStore();
}
}
When you arrange your step implementations like that, the reusing of other steps is much more easier.
My question is really a follow up question to
RabbitMQ Integration Test and Threading
There it states to wrap "your listeners" and pass in a CountDownLatch and eventually all the threads will merge. This answer works if we were manually creating and injecting the message listener but for #RabbitListener annotations... i'm not sure how to pass in a CountDownLatch. The framework is auto magically creating the message listener behind the scenes.
Are there any other approaches?
With the help of #Gary Russell I was able to get an answer and used the following solution.
Conclusion: I must admit i'm indifferent about this solution (feels like a hack) but this is the only thing I could get to work and once you get over the initial one time setup and actually understand the 'work flow' it is not so painful. Basically comes down to defining ( 2 ) #Beans and adding them to your Integration Test config.
Example solution posted below with explanations. Please feel free to suggest improvements to this solution.
1. Define a ProxyListenerBPP that during spring initialization will listen for a specified clazz (i.e our test class that contains #RabbitListener) and
inject our custom CountDownLatchListenerInterceptor advice defined in the next step.
import org.aopalliance.aop.Advice;
import org.springframework.aop.framework.ProxyFactoryBean;
import org.springframework.beans.BeansException;
import org.springframework.beans.factory.BeanFactory;
import org.springframework.beans.factory.BeanFactoryAware;
import org.springframework.beans.factory.config.BeanPostProcessor;
import org.springframework.core.Ordered;
import org.springframework.core.PriorityOrdered;
/**
* Implements BeanPostProcessor bean... during spring initialization we will
* listen for a specified clazz
* (i.e our #RabbitListener annotated class) and
* inject our custom CountDownLatchListenerInterceptor advice
* #author sjacobs
*
*/
public class ProxyListenerBPP implements BeanPostProcessor, BeanFactoryAware, Ordered, PriorityOrdered{
private BeanFactory beanFactory;
private Class<?> clazz;
public static final String ADVICE_BEAN_NAME = "wasCalled";
public ProxyListenerBPP(Class<?> clazz) {
this.clazz = clazz;
}
#Override
public void setBeanFactory(BeanFactory beanFactory) throws BeansException {
this.beanFactory = beanFactory;
}
#Override
public Object postProcessBeforeInitialization(Object bean, String beanName) throws BeansException {
return bean;
}
#Override
public Object postProcessAfterInitialization(Object bean, String beanName) throws BeansException {
if (clazz.isAssignableFrom(bean.getClass())) {
ProxyFactoryBean pfb = new ProxyFactoryBean();
pfb.setProxyTargetClass(true); // CGLIB, false for JDK proxy (interface needed)
pfb.setTarget(bean);
pfb.addAdvice(this.beanFactory.getBean(ADVICE_BEAN_NAME, Advice.class));
return pfb.getObject();
}
else {
return bean;
}
}
#Override
public int getOrder() {
return Ordered.LOWEST_PRECEDENCE - 1000; // Just before #RabbitListener post processor
}
2. Create the MethodInterceptor advice impl that will hold the reference to the CountDownLatch. The CountDownLatch needs to be referenced in both in the Integration test thread and inside the async worker thread in the #RabbitListener. So we can later release back to the Integration Test thread as soon as the #RabbitListener async thread has completed execution. No need for polling.
import java.util.concurrent.CountDownLatch;
import org.aopalliance.intercept.MethodInterceptor;
import org.aopalliance.intercept.MethodInvocation;
/**
* AOP MethodInterceptor that maps a <b>Single</b> CountDownLatch to one method and invokes
* CountDownLatch.countDown() after the method has completed execution. The motivation behind this
* is for integration testing purposes of Spring RabbitMq Async Worker threads to be able to merge
* the Integration Test thread after an Async 'worker' thread completed its task.
* #author sjacobs
*
*/
public class CountDownLatchListenerInterceptor implements MethodInterceptor {
private CountDownLatch countDownLatch = new CountDownLatch(1);
private final String methodNameToInvokeCDL ;
public CountDownLatchListenerInterceptor(String methodName) {
this.methodNameToInvokeCDL = methodName;
}
#Override
public Object invoke(MethodInvocation invocation) throws Throwable {
String methodName = invocation.getMethod().getName();
if (this.methodNameToInvokeCDL.equals(methodName) ) {
//invoke async work
Object result = invocation.proceed();
//returns us back to the 'awaiting' thread inside the integration test
this.countDownLatch.countDown();
//"reset" CountDownLatch for next #Test (if testing for more async worker)
this.countDownLatch = new CountDownLatch(1);
return result;
} else
return invocation.proceed();
}
public CountDownLatch getCountDownLatch() {
return countDownLatch;
}
}
3. Next add to your Integration Test Config the following #Bean(s)
public class SomeClassThatHasRabbitListenerAnnotationsITConfig extends BaseIntegrationTestConfig {
// pass into the constructor the test Clazz that contains the #RabbitListener annotation into the constructor
#Bean
public static ProxyListenerBPP listenerProxier() { // note static
return new ProxyListenerBPP(SomeClassThatHasRabbitListenerAnnotations.class);
}
// pass the method name that will be invoked by the async thread in SomeClassThatHasRabbitListenerAnnotations.Class
// I.E the method name annotated with #RabbitListener or #RabbitHandler
// in our example 'listen' is the method name inside SomeClassThatHasRabbitListenerAnnotations.Class
#Bean(name=ProxyListenerBPP.ADVICE_BEAN_NAME)
public static Advice wasCalled() {
String methodName = "listen";
return new CountDownLatchListenerInterceptor( methodName );
}
// this is the #RabbitListener bean we are testing
#Bean
public SomeClassThatHasRabbitListenerAnnotations rabbitListener() {
return new SomeClassThatHasRabbitListenerAnnotations();
}
}
4. Finally, in the integration #Test call... after sending a message via rabbitTemplate to trigger the async thread... now call the CountDownLatch#await(...) method obtained from the interceptor and make sure to pass in a TimeUnit args so it can timeout in case of long running process or something goes wrong. Once the async the Integration Test thread is notified (awakened) and now we can finally begin to actually test/validate/verify the results of the async work.
#ContextConfiguration(classes={ SomeClassThatHasRabbitListenerAnnotationsITConfig.class } )
public class SomeClassThatHasRabbitListenerAnnotationsIT extends BaseIntegrationTest{
#Inject
private CountDownLatchListenerInterceptor interceptor;
#Inject
private RabbitTemplate rabbitTemplate;
#Test
public void shouldReturnBackAfterAsyncThreadIsFinished() throws Exception {
MyObject payload = new MyObject();
rabbitTemplate.convertAndSend("some.defined.work.queue", payload);
CountDownLatch cdl = interceptor.getCountDownLatch();
// wait for async thread to finish
cdl.await(10, TimeUnit.SECONDS); // IMPORTANT: set timeout args.
//Begin the actual testing of the results of the async work
// check the database?
// download a msg from another queue?
// verify email was sent...
// etc...
}
It's a bit more tricky with #RabbitListener but the simplest way is to advise the listener.
With the custom listener container factory just have your test case add the advice to the factory.
The advice would be a MethodInterceptor; the invocation will have 2 arguments; the channel and the (unconverted) Message. The advice has to be injected before the container(s) are created.
Alternatively, get a reference to the container using the registry and add the advice later (but you'll have to call initialize() to force the new advice to be applied).
An alternative would be a simple BeanPostProcessor to proxy your listener class before it is injected into the container. That way, you will see the method argumen(s) after any conversion; you will also be able to verify any result returned by the listener (for request/reply scenarios).
If you are not familiar with these techniques, I can try to find some time to spin up a quick example for you.
EDIT
I issued a pull request to add an example to EnableRabbitIntegrationTests. This adds a listener bean with 2 annotated listener methods, a BeanPostProcessor that proxies the listener bean before it is injected into a listener container. An Advice is added to the proxy which counts latches down when the expected messages are received.
This line in TopLevelTransaction (neo4j-kernel-2.1.2) throws a NullPointerException every time I call next() on an iterator obtained via GraphRepository#findAll():
protected void markAsRollbackOnly()
{
try
{
transactionManager.getTransaction().setRollbackOnly(); // NPE here
}
catch ( Exception e )
{
throw new TransactionFailureException(
"Failed to mark transaction as rollback only.", e );
}
}
I found some threads about similar crashes with slightly different stack traces. The accepted solution on this question is to use "proxy" transaction management, but that seems like a band-aid solution. This question also mentions "proxy" transaction management and suggests that there might be something wrong with the #Transactional annotation when using AspectJ.
Is this legitimately a bug, or have I just set up my project incorrectly? My code is essentially the same as in my standalone hello world, with a slightly more complex main class:
#Component
public class Test2 {
#Autowired
FooRepository repo;
public static void main(String[] args) {
AbstractApplicationContext context = new AnnotationConfigApplicationContext("test2");
Test2 test2 = context.getBean(Test2.class);
test2.doStuff();
}
public void doStuff() {
createFoo();
printFoos();
}
#Transactional
public Foo createFoo() {
Foo foo = new Foo();
foo.setName("Derp" + System.currentTimeMillis());
repo.save(foo);
System.out.println("saved " + foo.toString());
return foo;
}
#Transactional
public void printFoos() {
Iterable<Foo> foos = repo.findAll();
System.out.println("findAll() returned instance of " + foos.getClass().getName());
Iterator<Foo> iter = foos.iterator();
System.out.println("iterator is instance of " + iter.getClass().getName());
if(iter.hasNext()) {
iter.next(); // CRASHES HERE
}
}
}
I can post my POM if needed.
I didn't find a bug. Two or three things are required to make this work, depending on whether you want to use proxy or AspectJ transaction management.
First, transaction management must be enabled. Since I'm using annotation-based configuration, I did this by annotating my #Configuration class with #EnableTransactionManagement. Contrary to the docs, the default mode now seems to be AdviceMode.ASPECTJ, not AdviceMode.PROXY.
Next, you need to ensure that the Iterator is used within a transaction. In my example, if I use AdviceMode.PROXY the entire bean containing the #Autowired repository has to be annotated #Transactional. If I use AdviceMode.ASPECTJ I can annotate just the method. This is because the call to the method using the iterator is a self-call from within the bean, and proxy transaction management cannot intercept and manage internal calls.
Finally, if you're using AdviceMode.ASPECTJ you must set up weaving as discussed here.
We're trying to implement global hooks on our specflow tests and are not entirely sure how [BeforeScenario] and [AfterScenario] attributed methods work.
The way I've seen it done, those attributes are always defined in a class containing specific steps used in a few scenarios.
Can they go somewhere so they apply to all scenarios? Or does attributing the methods with [BeforeScenario] and [AfterScenario] cause them to be run for all scenarios, regardless of where they're actually placed?
Hmm... From what I knew and according to the documentation these hooks are always global, i.e. from http://www.specflow.org/documentation/hooks/
Hooks
The hooks (event bindings) can be used to perform additional automation logic on specific events, like before the execution of a scenario.
The hooks are global but can be restricted to run only for features or scenarios with a specific tag (see below). The execution order of hooks for the same event is undefined.
In fact by producing a small demo project with the following
[Binding]
public class Unrelated
{
[BeforeScenario]
public void WillBeCalledIfGlobal()
{
Console.WriteLine("I'm global");
}
}
[Binding]
public class JustTheTest
{
[Given("nothing")]
public void GivenNothing()
{
// Don't do anything
}
}
Then the test specification of
As a developer
In order to understand how BeforeSpecifcation works
I want to know what the following does
Scenario: See if BeforeSpecifcation hook gets called
Given nothing
The get the output
I'm global
Given nothing
-> done: JustTheTest.GivenNothing() (0.0s)
So it really does look as if the documentation is correct, and you should use tagging to control if the BeforeScenario \ AfterScenario are run before or after your scenario.
There is also a good example of how tagging works here -> Feature-scoped step definitions with SpecFlow?
Yes, you can create global BeforeScenario and AfterScenario methods, but in practice I find that this is not desirable, as usually the same before and after steps do not apply to all steps in a test project.
Instead I create a base class for my step definitions, which would have the BeforeScenario and AfterScenarios methods I'd like applied to all of my scenarios e.g.
public class BaseStepDefinitions
{
[BeforeScenario]
public void BeforeScenario()
{
// BeforeScenario code
}
[AfterScenario]
public void AfterScenario()
{
// AfterScenario code
}
}
Note that I have not used the Binding attribute on this class. If you do include it then the BeforeScenario and AfterScenario steps would be global.
I then derive my step definion classes from this base step definition class, so that they will have the Before and After scenario methods e.g.
[Binding]
public class SpecFlowFeature1Steps : BaseStepDefinitions
{
[Given(#"I have entered (.*) into the calculator")]
public void GivenIHaveEnteredIntoTheCalculator(int inputValue)
{
ScenarioContext.Current.Pending();
}
[When(#"I press add")]
public void WhenIPressAdd()
{
ScenarioContext.Current.Pending();
}
[Then(#"the result should be (.*) on the screen")]
public void ThenTheResultShouldBeOnTheScreen(int expectedResult)
{
ScenarioContext.Current.Pending();
}
}
Whilst this approach is not global, by making all StepDefinitions derive from a BaseStepDefinition class we achieve the same outcome.
It also gives more control i.e. if you don't want the BeforeScenario or AfterScenario binding then don't derive from the base steps.
Sorry this doesn't work. As soon as you start using multiple Binding classes you end up with multiple calls. For example if I extend the example above to split the bindings into three classes,
[Binding]
public class SpecFlowFeature1Steps : BaseStepDefinitions
{
[Given(#"I have entered (.*) into the calculator")]
public void GivenIHaveEnteredIntoTheCalculator(int inputValue)
{
//ScenarioContext.Current.Pending();
}
}
[Binding]
public class SpecFlowFeature2Steps : BaseStepDefinitions
{
[When(#"I press add")]
public void WhenIPressAdd()
{
//ScenarioContext.Current.Pending();
}
}
[Binding]
public class SpecFlowFeature3Steps : BaseStepDefinitions
{
[Then(#"the result should be (.*) on the screen")]
public void ThenTheResultShouldBeOnTheScreen(int expectedResult)
{
//ScenarioContext.Current.Pending();
}
}
public class BaseStepDefinitions
{
[BeforeScenario]
public void BeforeScenario()
{
// BeforeScenario code
Console.WriteLine("Before. [Called from "+ this.GetType().Name+"]");
}
[AfterScenario]
public void AfterScenario()
{
// AfterScenario code
Console.WriteLine("After. [Called from " + this.GetType().Name + "]");
}
}
Then when I run it, the output is
Before. [Called from SpecFlowFeature1Steps]
Before. [Called from SpecFlowFeature2Steps]
Before. [Called from SpecFlowFeature3Steps]
Given I have entered 50 into the calculator
-> done: SpecFlowFeature1Steps.GivenIHaveEnteredIntoTheCalculator(50) (0.0s)
And I have entered 70 into the calculator
-> done: SpecFlowFeature1Steps.GivenIHaveEnteredIntoTheCalculator(70) (0.0s)
When I press add
-> done: SpecFlowFeature2Steps.WhenIPressAdd() (0.0s)
Then the result should be 120 on the screen
-> done: SpecFlowFeature3Steps.ThenTheResultShouldBeOnTheScreen(120) (0.0s)
After. [Called from SpecFlowFeature1Steps]
After. [Called from SpecFlowFeature2Steps]
After. [Called from SpecFlowFeature3Steps]
What you can do, in order to control the 'BeforeScenario' and 'AfterScenario' is use tags. This gives you the control of which Scenario's should run which before and after block. Your scenario would look like this:
#GoogleChrome
Scenario: Clicking on a button
Given the user is on some page
When the user clicks a button
Then something should happen
Here you could let the 'BeforeScenario' start a browser session in Google Chrome for you, and implement similar tags for different browsers. Your 'BeforeScenario' would look like this:
[Binding]
class Browser
{
[BeforeScenario("GoogleChrome")]
public static void BeforeChromeScenario()
{
// Start Browser session and do stuff
}
[AfterScenario("GoogleChrome")]
public static void AfterChromeScenario()
{
// Close the scenario properly
}
I think using tags is a nice way of keeping your scenario's clean and give you the extra functionality to let you control what each scenario should do.