I want to execute specflow scenario based on some condition and skip test if condition is not met programmatically.
I have created one test which gets executed for different countries, but I want to execute test only for one country not all.
I tried scenariocontext.current.pending(); In this case steps are getting skipped but in Jenkins, test result is shown as failed.
Since SpecFlow 3.1 you can do it with the UnitTestRuntimeProvider.
[Binding]
public sealed class StepDefinitions
{
private readonly IUnitTestRuntimeProvider _unitTestRuntimeProvider;
public CalculatorStepDefinitions(IUnitTestRuntimeProvider unitTestRuntimeProvider)
{
_unitTestRuntimeProvider = unitTestRuntimeProvider;
}
[When("your binding")]
public void YourBindingMethod()
{
_unitTestRuntimeProvider.TestIgnore("This scenario is always skipped");
}
}
Ignoring is like skipping the scenario. Be careful, as it behaves a little bit different for the different unit test runners (xUnit, NUnit, MSTest, SpecFlow+ Runner).
Also, this works only in step definitions. It is not possible to use it in hooks.
Related
I've encountered a dependency injection scenario which I cannot find a way through.
We currently have an Azure function.
We are using dependency injection via the FunctionsStartup attribute.
That all works fine, until I get asked to make it work for multiple environments.
The tester found it too onerous to deploy to 7 different environments, so I was asked to re-jig the function so that it runs (in a loop) for those environments.
That means 7 different IConfigurations and somehow having 7 separate compartmentalised IOC registrations of services.
I can't think of a way of doing that, without significantly re-structuring the way abstractions are being resolved. Even if you set up registrations in a loop and inject an IEnumerable of a service, when it goes to resolve a child dependency, it just pulls the last one registered, rather than the one which was meant to correlate with the current item being iterated.
So, something like this (using Autofac):
Registration
foreach (var configuration in configurations)
{
containerBuilder.Register<ICosmosDbService<AccountUsage>>(sp =>
{
var dBConfig = CosmosDBHelper.GetProjectDatabaseConfig(configuration.Value, Project.Jupiter);
return CosmosClientInitializer<AccountUsage>.Initialize(dBConfig);
}).As<ICosmosDbService<AccountUsage>>();
}
Usage
private readonly IEnumerable<IAccountUsageService> _accountUsageService;
public JobScheduler(IEnumerable<IAccountUsageService> accountUsageService)
{
_accountUsageService = accountUsageService;
}
[FunctionName("JobScheduler")]
public async Task Run([TimerTrigger("0 */2 * * * *")] TimerInfo myTimer, ILogger log)
{
log.LogInformation($"Job Scheduler Timer trigger function executed at: {DateTime.Now}");
try
{
foreach (var usageService in _accountUsageService)
{
var logs = await usageService.GetCurrentAccountUsage("gfkjdsasjfa");
// ...
}
}
I realise this kind of DI usage is not ideal (and does not even work).
Is there a way to structure an Azure Function such that it can execute for different configurations in a compartmentalised manner? Or is this really just fighting against the technology?
You've got a couple of ways to do this - either inject the right dependencies into the function constructor, or resolve them dynamically using a service-locater type approach with a named instance.
Let's consider the second approach and what it would mean for your implementation. As you demonstrated, you'd be looping through your instances and resolving the dependency you want to use, then invoking it
foreach (var usageService in _accountUsageService)
{
var logs = await usageService.GetCurrentAccountUsage("named-instance");
logs.DoSomething();
}
This is technically possible, but now you're doing batch processing - you're doing more than once piece of work that's been triggered by a single event (the timer object), which means you have to deal with a couple of extra problems. What should you do if there's a failure with one of the instances, and what to do if one of the instances is running slowly?
Ideally, you want functions to do the smallest bit of work they can, and complete quickly - You don't want failure or slowness with one particular instance impacting the other instances. By breaking it down to the smallest piece of work (think, one event trigger does one piece of work) then you can take advantage of the functions runtime for things like retries on failures, and threading and concurrency is now being done for you by the runtime.
You could then think about a couple of ways you could do this. a) multiple function signatures and a service resolver approach, e.g.
public class JobScheduler
{
public JobScheduler(IEnumerable<IAccountUsageService> accountUsageService)
{
_accountUsageService = accountUsageService;
}
[FunctionName("FirstInstance")]
public Task FirstInstance([TimerTrigger("%MetricPoller:Schedule%")] TimerInfo myTimer)
{
var logs = await _accountUsageService.GetNamedInstance("instance-a");
logs.DoSomething();
}
[FunctionName("SecondInstance")]
public Task SecondInstance([TimerTrigger("%MetricPoller:Schedule%")] TimerInfo myTimer)
{
var logs = _accountUsageService.GetNamedInstance("instance-b");
logs.DoSomething();
}
}
or b), multiple classes with the necessary dependencies injected
public class JobSchedulerFirstInstance
{
public JobSchedulerFirstInstance(ILogs logs)
{
_logs = logs;
}
[FunctionName("FirstInstance")]
public Task FirstInstance([TimerTrigger("%MetricPoller:Schedule%")] TimerInfo myTimer)
{
_logs.DoSomething();
}
}
I'd personally lean towards multiple classes approach, and register named instances with my container. A bit of extra wire up work needed, but you'll end up with lots of small classes that all look very similar that are basically jus t plumbing that the functions runtime executes.
I'm working on a project using Grails 2.5.4 and I'm currently trying to run some integration tests that are not running.
I've debugged the issue and found that apparently some dynamic methods on the services to be tested are not there when running in an integration test (if you run that in the context of the application the methods are there and everything works). This happens in many of the test I'm trying to run, I've choose one as an example but the others that are failing have the same problems.
I have this domain class
class Event {
...
static hasMany = [
bundles : Bundle
]
...
}
and a service method to be tested:
#Transactional
class BundleService {
...
void assignEvent(Event event, List bundleIds) {
..
for (id in bundleIds) {
event.addToBundles(Bundle.get(id))
}
}
...
}
So then I run this spock test
class BundleServiceIntegrationSpec extends Specification {
BundleService bundleService
EventService eventService
private BundleTestHelper bundleHelper = new BundleTestHelper()
...
void '04. Test deleteBundleAndAssets method'() {
when: 'a new Bundle is created'
Bundle bundle = bundleHelper.createBundle(project, 'Test Bundle')
and: 'a new Event is created'
Event event = eventService.create(project, 'Test Event')
and: 'the above Bundle is assigned to the Event'
bundleService.assignEvent(event, [bundle.id])
...
}
it fails in the line moveEvent.addToBundles(Bundle.get(id)) of BundleService with the following exception
groovy.lang.MissingMethodException: No signature of method:
net.domain.Event.addToBundles() is applicable for argument
types: (net.domain.Bundle) values: [Test Bundle]
Possible solutions: getBundles()
at net.service.BundleService.$tt__assignEvent(BundleService.groovy:101)
The problem is that the method addToBundles() that should be added dynamically by Grails to the Event class because of the hasMany collection "bundles" is not added. As I mentioned, if you run the application and use this service the method is there and everything works.
I tried changing the base class of the test (from Specification to IntegrationSpec) since I belive here is where the dynamic capabilities as well as transaction management and other things for integration tests are managed, but it didn't worked.
Is there any reason why this method that should be there in the service is not present in the context of integration tests? Thanks
You are missing grails.test.mixin.Mock annotation in your test class. Grails unit test uses this mixin to generate all domain related methods for a class so you can use this domain correctly in the unit test. Something like this should do the trick:
#Mock([Event])
class BundleServiceIntegrationSpec extends Specification {
BundleService bundleService
EventService eventService
private BundleTestHelper bundleHelper = new BundleTestHelper()
...
void '04. Test deleteBundleAndAssets method'() {
when: 'a new Bundle is created'
Bundle bundle = bundleHelper.createBundle(project, 'Test Bundle')
and: 'a new Event is created'
Event event = eventService.create(project, 'Test Event')
and: 'the above Bundle is assigned to the Event'
bundleService.assignEvent(event, [bundle.id])
...
}
More about testing domain classes can be found here: https://grails.github.io/grails2-doc/2.4.5/guide/testing.html#unitTestingDomains
#Szymon Stepniak Thanks for your answer, and sorry for the late reply. I've tested what you proposed but it didn't work. Later I've read that the
grails.test.mixin.Mock annotation is only for unit tests, and it should not be used in integration tests. This is also true for #TestFor and #TestMixin annotations as well (I've read about this in this post).
So after this a collegue at work proposed me to search for this kind of annotations in other tests thinking that maybe that could lead to some kind of test pollution between tests, and after removing a #TestFor annotation in one of the tests that ran previously as part of the whole integration test suite, the failing test that I posted started working. The strangest thing (appart from the compiler not complaining about this) is that the offending test (the one from which I removed the #TestFor annotation) was passing all green, it wasn't even failing!
So if someone has a similar problem I suggest to search for this kind of unit test annotations anywhere in the whole integration tests suite and remove it because the compiler won't complain, but in my experience it could have influence on other tests and it can lead to very strange behaviour.
I have seen facets of this question asked elsewhere but there was no solid answer, so I ideally want to have a couple of [AfterScenario] bindings run and do some logic based upon the completion state, i.e did it pass/fail, and how did it fail etc.
I know there is a ScenarioContext.Current.TestError and some other related classes, but I cannot find docs that indicate if this can be used when tests are run in parallel, so is the above ScenarioContext ok to be used in this context or is there some other way to do it?
You can't use ScenarioContext.Current in parallel execution, but you can get the current ScenarioContext via Context Injection (see https://specflow.org/documentation/Context-Injection/)
So this works in parallel execution:
[Binding]
public class ScenarioStatusBinding
{
private readonly ScenarioContext _scenarioContext;
public ScenarioStatusBinding(ScenarioContext scenarioContext)
{
_scenarioContext = scenarioContext;
}
[AfterScenario]
public void CheckStatusOfScenario()
{
Console.WriteLine(_scenarioContext.TestError);
Console.WriteLine(_scenarioContext.ScenarioExecutionStatus);
}
}
How do I programmatically skip a test in the Spock framework? I know I can annotate a test with #Ignore to skip it, or use #IgnoreIf to skip tests based on environmental variables and the like. But is there a way to run arbitrary code that decides whether or not a test should run?
For example, let's say I have an integration test that has to connect to a third-party service's sandbox environment. Outages in the service's sandbox environment cause the test to fail. Assuming I've written a method canConnectToService that checks if the test will be able to connect to this service, how do I write a test that will be skipped if canConnectToService() returns false?
Use JUnit's Assume class. Specifically, you could write Assume.assumeTrue(canConnectToService()) at the beginning of your test to skip the test if the third party service is unavailable. This method will throw an AssumptionViolatedException if canConnectToService() returns false, and Spock ignores tests that are interrupted by an AssumptionViolatedException for JUnit compatibility (see this bug report).
There is another alternative (maybe it didn't exists before):
Using instance inside #Requires or #IgnoreIf:
Examples using inheritance, but not required:
abstract class BaseTest extends Specification {
abstract boolean serviceIsOnline()
#Requires({ instance.serviceIsOnline() })
def "some test" () { .. }
}
SubSpecification:
class OnlineTest extends BaseTest {
boolean serviceIsOnline() {
// Test connection, etc.
return true
}
}
class SkipTest extends BaseTest {
boolean serviceIsOnline() {
return false
}
}
Documentation
instance
The specification instance, if instance fields, shared
fields, or instance methods are needed. If this property is used, the
whole annotated element cannot be skipped up-front without executing
fixtures, data providers and similar. Instead, the whole workflow is
followed up to the feature method invocation, where then the closure
is checked, and it is decided whether to abort the specific iteration
or not.
As an extra, another way you can programmatically skip a test is using the where label:
class MyTest extends Specification {
List getAvailableServices() {
// You can test connections here or your conditions
// to enable testing or not.
return available
}
#Unroll
def "Testing something"() {
setup:
URL url = serviceUrl.toURL()
expect:
assert url.text.contains("Hello")
where:
serviceUrl << availableServices
}
}
Hi i m trying to change the auto-generated testCases in grails
#TestMixin(GrailsUnitTestMixin)
class KLAKSpec {
void setUp() {
// Setup logic here
}
void tearDown() {
// Tear down logic here
}
void testSomething() {
fail "Implement me"
}
}
to Spock Type test format which is something like this
#TestFor(GrailsUnitTestCase)
class #artifact.name# extends #artifact.superclass# {
def "feature method"() {
setup:
when:
then:
where:
}
}
Althought i have added a _Events.groovy script under the scipt folder and added a Spec.groovy file in artifacts folder which changes the name when i auto generate the list.
Can any one please let me knw how i can change to spec format.
I wrote a blog post on this some time ago: Auto-generate Spock specs for Grails artifacts. The post was written pre-Grails 2 so it's still using the old superclasses rather than #TestFor but it should be easy enough to adapt. We still use this technique on our project where we're still on Grails 1.3.7. It's a bit of a blunt instrument as Grails doesn't expose a unique event for test generation but it works fine.