I would like to add the same line to the start of each one of my SpecFlow tests.
This line specifies a list of several scenarios which will change over time, and therefore it is not feasible to maintain this list for every test.
For example:
Given I have set my site theme to <MyTheme>
|Theme Names|
|Theme 1 |
|Theme 2 |
|Theme 3 |
|Theme 4 |
|Theme 5 |
I'd like to have this test repeat for each of the themes. The list of themes is not set in stone, and should be maintained in a single place.
So far, I've successfully managed to create a Generator Plugin, and I was planning on using this plugin to alter the SpecFlow feature immediately before the Test Class is generated. However, I can't see a way to edit the scenario from within this context.
Is it possible to get and set the scenario text from within an implementation of IUnitTestGeneratorProvider?
I'm not set on using this method, so if anyone can suggest a better way to do this then I'd accept that too.
Apologies if I've gotten some terminology wrong- I've only just started using SpecFlow.
Edit:
I'm adding this section to provide clarification as to what I'm actually after.
Suppose I had a test suite containing 800 tests. We have a business requirement to run each of those 800 tests on each of our available themes. The list of available themes can change at any time, and it would be unfeasible to maintain this list in more than a single location.
So, for example, if I had the following two tests:
Example A:
Given I set context to < site >
Given I go to base url
When I type <username> in username field
When I type <password> in password field
When I click login button
Examples:
| site | username | password |
| MySuperSite | chris | mypassword |
| MySuperSite2 | chris2 | mypassword |
Given I am logged in
Given I go to base url
When I click logout button
Then I am logged out
I could simply manually change these tests to something along the lines of:
Example B:
Given I am using the < theme > theme
Given I set context to < site >
Given I go to base url
When I type <username> in username field
When I type <password> in password field
When I click login button
Examples:
| site | username | password | theme |
| MySuperSite | chris | mypassword | theme1 |
| MySuperSite2 | chris2 | mypassword | theme1 |
| MySuperSite | chris | mypassword | theme2 |
| MySuperSite2 | chris2 | mypassword | theme2 |
| MySuperSite | chris | mypassword | theme3 |
| MySuperSite2 | chris2 | mypassword | theme3 |
Given I am using the < theme > theme
Given I am logged in
Given I go to base url
When I click logout button
Then I am logged out
Examples:
| theme |
| theme1 |
| theme2 |
| theme3 |
There are a few issues with this:
The tests become bloated with data that is repeated
If we no longer choose to support theme2 then someone will have to go through each test and remove it from the examples table (not too bad in the example above, but imaging we had >800 tests)
Tests that already have example tables will have their table size multiplied by the number of themes we are supporting (>40)
There is a large risk of manual error caused by someone inadvertently forgetting to add a specific theme to a test
Objective:
I'd like to be able to have our testers write tests in the style of Example A, but have the tests themselves compile down to how they would if they were written in the style of Example B.
I've created a generator plugin for specflow, with a view of intercepting the test creation and then programmatically adding the line Given I am using the < theme > theme, and then update or add any example data as required. However, I can't seem to be able to do this from here.
Can anyone tell me if this is possible, and if so, how should I go about it?
OK, I figured this out. It took a few steps:
After looking through the source code on GitHub, I found UnitTestFeatureGenerator which appears to be the class responsible for turning the specflow files into unit tests.
I then created a new class that inherited from UnitTestFeatureGenerator and hides the GenerateUnitTestFixture method from the base class.
In the body of my GenerateUnitTestFixture class, I then add the extra steps required into the scenario before handing off of base.GenerateUnitTestFixture to generate the unit tests. Here's the gist of it:
public class MultiThemeUnitTestFeatureGenerator : UnitTestFeatureGenerator, IFeatureGenerator
{
public MultiThemeUnitTestFeatureGenerator(IUnitTestGeneratorProvider testGeneratorProvider, CodeDomHelper codeDomHelper, GeneratorConfiguration generatorConfiguration, IDecoratorRegistry decoratorRegistry)
: base(testGeneratorProvider, codeDomHelper, generatorConfiguration, decoratorRegistry)
{}
public new CodeNamespace GenerateUnitTestFixture(Feature feature, string testClassName, string targetNamespace)
{
foreach (var scenario in feature.Scenarios)
{
scenario.Steps.Insert(0, new Given {Text = "Given I have <Theme> set as my current theme"});
//add any other steps you need....
}
return base.GenerateUnitTestFixture(feature, testClassName, targetNamespace);
}
}
Once I had all of this set up, I needed a way to tell specflow to use my new class instead of the currently registered UnitTestFeatureGenerator. This was the complicated bit to get working as the documentation pretty much just said "Coming soon". Thankfully, I found an excellent blog post which outlines all the pitfalls.
My IGeneratorPlugin implementation looks like this:
public class MultiThemeGeneratorPlugin : IGeneratorPlugin
{
public void RegisterDependencies(ObjectContainer container)
{}
public void RegisterCustomizations(ObjectContainer container, SpecFlowProjectConfiguration generatorConfiguration)
{
container.RegisterTypeAs<MultiThemeFeatureGeneratorProvider, IFeatureGeneratorProvider>("default");
}
public void RegisterConfigurationDefaults(SpecFlowProjectConfiguration specFlowConfiguration)
{}
}
Note That I register IFeatureGeneratorProvider as opposed to IFeatureGenerator. I had to create an implementation of IFeatureGeneratorProvider that returns an instance of the implementation of IFeatureGenerator that I was interested in:
public class MultiThemeFeatureGeneratorProvider : IFeatureGeneratorProvider
{
private readonly ObjectContainer _container;
public MultiThemeFeatureGeneratorProvider(ObjectContainer container)
{
_container = container;
}
public int Priority
{
get { return int.MaxValue; }
}
public bool CanGenerate(Feature feature)
{
return true;
}
public IFeatureGenerator CreateGenerator(Feature feature)
{
return _container.Resolve<MultiThemeUnitTestFeatureGenerator>();
}
}
I can't think of a simple way to do this, but my approach would probably be to exclude the theme from the tests but have the build server run the tests in an environment where the theme is already set up (by an environment variable or via some config file or similar) and then run the entire suite n times, once for each theme.
So I would have the scenarios look like this
Given I am using the currently configured theme
Given I am logged in
Given I go to base url
When I click logout button
Then I am logged out
where the first step reads the theme from the config file or environment variable and sets up the tests to use that theme.
Once the build server has run these tests, it would change the config/environment variable to the next theme and the run the suite again.
This gives you the ability to specify the themes in a single place (on the build server, or in a file that the BS uses to discover the themes) and have the tests run using each of those themes. It also might give you some performance advantage as you could parallelize the tests on the build server as each one would be a different test step and each test step could be run independently and on different machines.
For the devs you would only be able to test one theme at once and would have to manually alter the config/env variable to change to a different theme, which might not be ideal.
Someone else might have a better idea of how to do this in the way you are envisioning, fingers crossed
Related
I have a situation where I want to create some data using a post request and then use the result of that post in few other tests, is it possible to do this using spock. From the below I would like to reuse the response (or something thats extracted from it. How can I do that?
def "AddAssessment valid output case"() {
given:
def request = """
{
"assessmentTopic" : "$assessmentTopic",
"assessmentSubTopic" : "$assessmentSubTopic",
"duration" : "$duration",
"assessmentLevel" : "$assessmentLevel"
}
"""
when:
def result = mvc.perform(
MockMvcRequestBuilders
.post("/assessments/createAssessment")
.content(request)
.contentType(MediaType.APPLICATION_JSON)
)
then:
MvcResult response = result
.andExpect(MockMvcResultMatchers.status().isOk())
.andExpect(MockMvcResultMatchers.jsonPath('$.*').doesNotExist())
.andReturn()
where:
assessmentTopic | assessmentSubTopic | duration | assessmentLevel
"Maths" | "Calculus" | "120" | "1"
"Civils" | "History" | "500" | "4"
"UPSC" | "GK" | "120" | "5"
"Chemistry" | "Elements" | "120" | "3"
}
Have you considered making the response you wish to re-use a #Shared variable and initialise it in a setupSpec() method? That is what you usually do if within one specification you wish to share data between feature methods.
See also the Spock manual, section "Fields":
#Shared res = new VeryExpensiveResource()
Sometimes you need to share an object between feature methods. For example, the object might be very expensive to create, or you might want your feature methods to interact with each other. To achieve this, declare a #Shared field. Again it’s best to initialize the field right at the point of declaration. (Semantically, this is equivalent to initializing the field at the very beginning of the setupSpec() method.)
I have a Page class and having some method which extract the Web page. Now I want to call these method into the Where block of Spock to pass as data provider. But when I call the method then it throws an error as it did not find it. But the same is accessible from out of Where block. Why is it so?
Example
def "Validate all article has its id"(){
when:"I am at Overview page"
at OverviewPage
then:"I should the article id of all article"
getAllCountOf(articleInfo).size() == actualCount
where:
articleInfo |actualCount
TopArticleListInfo.ARTICLE_THUMBNAIL |filter.getCount()
}
In the above code 'filter.getCount()' can not be accessed from Where block, But the same method is accessible in when or then block.
i want to understand the logic behind the scene It looks like , Where block is not able to find this method statically, need to create an object to call this.
When I tried the solution provided by erdi, But that also not solving the issue
when:"I am at Overview page"
at OverviewPage
then:"I should the article id of all article"
getAllCountOf(articleInfo).size() == page."$actualCount"
where:
articleInfo |actualCount
TopArticleListInfo.ARTICLE_THUMBNAIL |'getRowCount().size()'
Here getRowCount().size() substitute in "$actualCount". But still it throwing the error
Error Message
getAllCountOf(articleInfo).size() == page."$actualCount"
| | | | |
| | 10 | groovy.lang.MissingPropertyException: Unable to resolve getRowCount().size() as content for inca.sundashboard.pageobject.OverviewPage, or as a property on its Navigator context. Is getRowCount().size() a class you forgot to import?
|
I'm using Dynamic Method Names in my tests, here is a small example:
def "Validate all article has its id"(){
when: "I am at Overview page"
at OverviewPage
then: "I should the article id of all article"
getAllCountOf(articleInfo).size() == "$actualCountMethod"().getCount()
where:
articleInfo | actualCountMethod
TopArticleListInfo.ARTICLE_THUMBNAIL | filter
}
I'd like to configure a choice parameter in Jenkins.
The parameter I'd like to configure is called CIDR.
I tried using "Extended choice parameter" plugin but to no avail.
What I'm trying to do, is to let the user manually insert the chosen CIDR, considering the CIDRs which are already in use -> I want to run a groovy script to populate the string description with CIDRs which are already in use.
In order to list the already in use CIDRs, I wrote the following Groovy code:
#!/usr/local/bin/groovy
def p = ['/usr/local/bin/aws', 'ec2', 'describe-vpcs'].execute() | 'grep CidrBlock'.execute() | ['awk', '{print $2}'].execute() | ['tr', '-d', '"\\"\\|,"'].execute()
p.waitFor()
println p.text
The script runs properly in terminal:
itai#Itais-MacBook-Pro ~ - $ groovy cidrs.groovy
172.31.0.0/16
172.51.0.0/16
172.51.0.0/16
I even accepted a suspicious signature in Jenkins in-script approvals to allow the script to run.
But when I insert it to the Groovy script section of the string description and run the "build the job with parameters", the string dropdown stays empty.
What am I doing wrong?
Looks trivial issue. Try below.
Change From :
println p.text
To:
return p.text
The reason why the parameter kept being empty is that as it seems, the "Extended Choice Parameter" plugin expects the output to be an array.
Changing the script to the following code solved the issue:
#!/opt/groovy-2.4.12/bin/groovy
def p = ['/usr/bin/aws', 'ec2', 'describe-vpcs'].execute() | 'grep CidrBlock'.execute() | ['awk', '{print $2}'].execute() | ['tr', '-d', '"\\"\\|,"'].execute()
p.waitFor()
def output = []
p.text.eachLine { line ->
output << line
}
output.each {
println it
}
Now the parameter is populated with the available CIDRs.
I have a Spock integration test that looks something like this:
class PriceTierControllerIntegrationSpec extends IntegrationSpec {
PriceTierController controller
def setup() {
controller = new PriceTierController()
}
def "applyDiscount() method will redirect user to success view"() {
when:
controller.applyDiscount()
then:
controller.response.redirectedUrl == '/priceTier/success'
}
Then in the controller, the logic is simply:
class PriceTierController {
def applyDiscount() {
redirect action: 'success'
}
def success() {
}
}
When I run this Spock test on my local machine, the test passes. However, on the build server, I get the following error:
controller.response.redirectedUrl == '/priceTier/success'
| | | |
| | /test/success false
| | 8 differences (46% similarity)
| | /(t---)e(st--)/success
| | /(pric)e(Tier)/success
| org.codehaus.groovy.grails.plugins.testing.GrailsMockHttpServletResponse#dc42543
com.test.PriceTierController#193d6547
For some reason, on the build server, the Spock test thinks that the controller name is test instead of priceTier, and the test will fail. This only seems to happen for Spock integration tests, as the Spock unit tests and a few legacy Grails mixin tests all pass fine.
Does anybody know what could be causing this problem?
I've also just experienced this same issue, and it seems that it comes down to the test framework extracting the controller name from the the name of the testing class.
The convention is that the test class is named <controller name>ControllerSpec
In the above case, the test class should be named PriceTierControllerSpec so that the test framework will successfully resolve the controller to PriceTeir.
Naming the class according to these guidelines seems to fix this problem.
Further reference can be found here: https://jira.grails.org/browse/GRAILS-10962
Using grails 2.2.0 given this simple Domain
class Order {
static mapping = {table "ticket_order"}
String foo
}
And associated spock test
#TestFor(Order)
class OrderSpec extends Specification {
def "sensible constraints for order class"() {
setup:
mockForConstraintsTests(Order)
when:
def order = new Order(
foo : foo
)
order.validate()
then:
!order.errors.hasFieldErrors("foo")
where:
foo = "bar"
}
}
I get this output
grails> test-app unit: Order -echoOut
| Running 1 spock test... 1 of 1
--Output from sensible constraints for order class--
| Failure: sensible constraints for order class(uk.co.seoss.presscm.OrderSpec)
| Condition not satisfied:
!order.errors.hasFieldErrors("foo")
|| | |
|| | true
|| org.codehaus.groovy.grails.plugins.testing.GrailsMockErrors: 1 errors
|| Field error in object 'uk.co.seoss.presscm.Order' on field 'foo': rejected value [null];
Could someone please explain why I'm getting that null, have I not set the property correctly? I've tried a few simpler formulations of this with no joy. It works fine in a standard unit test.
It looks to me like your mixing the data-driven and interaction based testing styles.
The where block is mentioned only in the context of data-driven, and the when/then combo in the context of interaction testing.
Try put def foo = "bar" at the top of the test.