I am currently working for a large organisation with about 2k developers working in our IT department. We maintain many things including our e-commerce platform and there are currently about 30 projects currently impacting that.
Recently all of our teams have been instructed to deliver a series of automated tests using Concordion and Selenium Webdriver. For a while this has been going fairly well and many tests have been created but lately maintaining the existing tests while our e-commerce platform constantly changes has been somewhat of a nightmare. We have thousands of test scripts covering many parts of our website but there does not seem to be any facility in Concordion to split scripts into reusable compartments which could then be maintained once, rather than having to make changes to hundreds of HTML files for one change.
How are other people approaching this?
The goal of Concordion is not to implement test scripts as HTML, but rather for the HTML to describe the behaviour that you are testing (what you are trying to achieve). The implementation details (how it is being tested) are implemented as Java code. This code can then be structured with an appropriate level of abstraction so that each change to the system under test only requires a change to one part of the code.
Your HTML specifications should only need to change on the rare occasions that the business rules change.
These concepts are described further on the Hints and Tips tab of the Concordion home page.
Thank you for sharing your experience with us. It’s great to hear / read about large scale application of behavior driven development / specification by example.
One approach that could help you is to focus on key examples (http://gojko.net/2014/05/05/focus-on-key-examples). During specification workshops the entire team is working to get a common understanding of the new user needs and requirements. Then you go on and write specification documents containing key examples. There you should not try to cover everything, but to write only as many examples as necessary to express the common understanding.
Additionally, you should try to identify concepts, on which the examples are based. Are there some examples related to a similar topic – this is probably an underlying concept. It is often easier to understand the examples, if they focus just on one concept (e.g. the validation of a card number). Each concept can be usually described with only a few examples.
Do you have any other types of automated tests (e.g. unit tests)? Are you experiencing the same maintainability challenges with these other tests? Could you use good practices from these other test types to improve your Concordion approach?
Could you tell us more about your setup? How many active specifications have you already created within your company?
Related
In the expert system D3WEB, it is possible to insert\develop\use Ontology. However, I cannot get the point what's the purpose to introduce ontology in D3WEB?
The nice example on this page, https://www.d3web.de/Wiki.jsp?page=Demo%20-%20Ontology , shows how to develop an ontology in D3WEB. In my opinion, it can be more efficiently developed using Protégé. If the contents shall be changed with a real application, for instance, an ontology about 'dog', in the real application there could be instance dog A, B, C, D. It might be not feasible to 'insert' the instances into the D3WEB knowledge base. However, if the ontology changes over time, how to use the ontology in D3WEB then?
In my opinion, the best way is to develop an ontology outside of D3WEB using Java code. However, I believe the designer of D3WEB would have a nice reason to introduce ontology in D3WEB. I will appreciate it if someone let me know.
This is a somewhat common question we get regarding d3web-KnowWE, one reason might be, that our naming is somewhat misleading. So let me explain.
First there is d3web the java framework to run knowledge bases with strong problem solving knowledge, including rules, decision trees, flow-charts, covering lists, cost-benefit dialog strategies, time based reasoning, and so on. This framework in its core does not provide any GUIs, but is meant to integrate problem solving capabilities in other applications/expert systems. It also does not provide a way to properly create/author the knowledge bases it runs, aside maybe from doing it in the Java code on an API level.
To also provide proper means to author and develop a knowledge base, including some basic dialogs to run, demo, test, and debug the authored knowledge bases, we began working on the wiki system KnowWE, which today is basically a heavily extended JSPWiki. The page d3web.de itself for example is also just a build of KnowWE with specific content.
While we were working on and with KnowWE, we began to really like the approach to edit and author large knowledge bases in this 'wiki way', were you automatically support multiple distributed users to work on the same knowledge base, have automatic versioning, can add nice documentation directly beside the actual formal knowledge, can generate knowledge using script (because it's all just simple text markup), and so forth. Also, the underlying architecture of KnowWE became quite good and mature over the years.
So after some time of this, we found ourselves in the need to also author large ontologies. And yes, Protégé is a nice tool to develop ontologies, but for our use cases, it was just not well suited and we also found it to not scale very well. So we began to implement some simple markups to also allow to also develop ontologies in KnowWE. After then recognizing, that authoring ontologies the 'wiki way' indeed works pretty nicely, we decided to again also share these tools with everybody else on d3web.de. And that is why today you can author/develop both d3web knowledge bases and ontologies in KnowWE, although there is no actual connection/interoperability between both as of now. That would be nice of course and maybe we add this in the future, but for KnowWE is just a development environment for these two knowledge representation.
Maybe you can see KnowWE similar to an IDE like eclipse or IntelliJ, where the same application can be used to develop many different programming languages. KnowWE does the same for different knowledge representations.
A problem is maybe, that historically, we didn't differentiate very well between KnowWE and d3web, because KnowWE was narrowly used to build d3web knowledge bases. We also like to call KnowWE and its distribution package d3web-KnowWE for example. But maybe this should change...
Thanks for pointing this out, I will try to correct/clarify this on d3web.de
We have been using waterfall to develop and enhance the system in our company and recently we are moving into Agile and the management is interested in BDD.
My understanding is that BDD is a tool for business and development team to refine the stories and at the same time these scenarios will become test cases for the feature. This sounds perfect to me but since we already have the features available, how can BDD work in this type of situation?
Should we just write up the stories and scenarios per our knowledge of the feature?
My only concern of the above is the coverage of the scenarios. Or we shouldn't worry and keep adding new scenarios and test it whenever the team came up with new ones?
Prompted by yet another person who mailed me with the same question today, I've written up an answer for this.
Short version, you can use BDD to help you understand what the system actually does, and why, but you'll be clarifying the requirements rather than exploring them.
Additionally, you asked, "Should we just write up the stories and scenarios per our knowledge of the feature?"
I'd speak with any stakeholders you can find, ask them what the system should do, then look to see if it actually does it. Systems designed before adopting a practice of conversations with examples often don't do what the originators intended. You can then differentiate between the behaviour you've actually got, while creating a new backlog from the behaviour you want.
I advise grabbing someone who's good at asking questions and spotting missing scenarios to have these discussions with (usually a tester). Because you already have some knowledge of the system, it's likely you'll be very good at describing what you think it does, while missing gaps.
If you don't have any automated testing yet and you want to start using BDD, I would suggest you to start by writing some Scenarios for some of your manual tests scripts, I find it to be a good way to train the writing of the BDD style, then as Lunivore said, you should work together with the business people and QA to find out better about the behaviour of your system and preferably write the scenarios with them.
We are looking to initiate a bdd-style approach, inspired by Gojko Adzic's specification by example. Implementation is in java and devs are already writing junit tests.
Key requirement is that specifications (acceptance tests) can be written, read and maintained by non-developers. The project will run as an agile team - so it's fine if devs have to instrument the specs. However, I don't want developers, testers or domain experts having to read or write something that looks like code.
So far I've looked at FitNesse, Concordion and various others (e.g. Spock). I've rejected spock and similar tools because they target developers as the primary audience. FitNesse seems to meet most of the requirements.
Concordion is probably current favourite however: specs looks cleaner and simpler.
So my question (actually three):
Any suggestions for other tools I should look at?
Has anyone been successful with using concordion (or another tool) in this way?
Is concordion still actively developed/supported? Difficult to tell from website and most related SO questions are several years old.
Thanks.
I've worked with a number of teams implementing Specification by Example with Concordion. We train our whole team up to write Concordion specifications in HTML. Only a small subset of HTML is required, so we can train a newbie in about 30 minutes. Typically we have the testers writing the HTML specification, with the BA or Scrum Master sometimes writing them.
We've used Eclipse (Web Page Editor) for editing the HTML. This works well, except that Concordion requires valid XHTML, and Eclipse does not allow HTML to be validated as XHTML. This mostly shows with <br> tags being used rather than <br/>. We cover this off in training. We also train the whole team in the use of source control. By using Eclipse, we have a single user interface for editing and source control. We also find that having the team using the same IDE is a step on the journey to a cross-functional team.
I know of another team where the BA is writing the specifications using a Mac-based HTML editor.
Concordion is actively maintained, with rapid responses on the mailing list (Yahoo) and the issues list. The Concordion codebase is stable. The active development over the last year or so has focussed on the extension mechanism, allowing users to add commands and listeners (eg. for capturing screenshots on test failure).
Also take a look at Cucumber and JBehave, both of which allow specifications to be written in plain text.
If you choose to use FitNesse, it may also be worth looking at Slim, which sits behind FitNesse in place of Fit. It provides slightly different table formats to Fit, and I've found it suits BDD much better.
Just to update the topic, you can also consider jnario.
In the light of BDD, would it be a good idea to use Cucumber features and scenarios to write up the functional requirements document at the start of a new Rails project?
Probably not. This would be a case of BDUF (http://en.wikipedia.org/wiki/Big_Design_Up_Front).
It's unlikely you'll be able to think of all the scenarios up front. You should split the high-level requirements into features to help estimate and prioritise them, but leave the detailed scenario writing to just before you're ready to begin implementing each feature.
If you are not the one making all the decisions and someone thinks you need these,
then it might appear to be a better choice than MS word.
I actually joined a project with a million un-implemented features,
so we had loads of integrations test in theory, just none
of them were actually implemented.
Its months later and we still have some,
its really difficult working in an environent where
everything is failing at once.
Its better to have 1 failing step at a time.
I also think feautures should be written by application developers who
understand user flow realities.
I like the business or client to explain stuff to me in a conversational syle,
a bit at a time, not their entire vision for world domination in 10 000 words,
I will keep that for bed time.
We've got an Excel spreadsheet floating around right now (globally) at my company to capture various pieces of information about each countries technology usage. The problem is that it goes out, gets changes, but they're never obvious, and often conflicting - and then we have to smash them together. To me, the workbook is no more than a garbage in/garbage out type application waiting to be written.
In a company that has enough staff and knowledge to dedicate to Enterprise projects, for some reason, agile and language/frameworks such as Rails, Grails, etc. are frowned upon. That said, I can't help but think that this is almost a perfect fit for the need, given the scaffolding features for extremely simple implementations of capturing raw fields with only a couple lookups (i.e. a pre-defined category). I'm thinking this would be considered a very appropriate use of these frameworks.
Has anyone worked on these types of quick and dirty apps before in normally large-scale, heavy-handed enterprise environments with success? Any tips for communicating this need/appropriateness to non-technical management?
The only way to get this implemented in a rigid organization is to get this working and demo it -- without approval. It's very hard for management to say no to a finished project.
I work for a really big company & have written many utility apps based on Rails (as well as contributed to some larger Rails projects). That said, the biggest concern is not the quality of the app, but who's going to support/maintain it when you leave or get hit by the bus.
IMHO, The major fear that an enterprise organization has - especially if the application becomes more critical to it's core business - is how to support it. If it doesn't fit into it's neat little box of supported technologies, it's less likely to happen.
Corporations have been bitten by this many times in the past & are cautious when bringing in new technology.
So, if you can drum up more folks to learn Ruby/Rails in your group (or elsewhere in your company), you may be able to make a good case for it. Otherwise, sad to say, your probably better off implementing something on Sharepoint :-(.
If you already have a Java infrastructure, then creating a Grails app will require little to no additional IT ramp up to support and maintain. The support and maintenance cost and effort should be the same as for a Java application (i.e. Grails apps run on Tomcat, use the same JVM, use the same diagnostic/profiling tools, etc.).
In my experience, larger IT organizations have a harder time supporting Ruby when its not already in the toolchain because its a new language, new deployment environment, and requires a considerable amount of support and maintenance ramp up.
I would develop a minimal viable product, then make friends with someone in IT who can help you deploy it into a staging or production environment. Then get a few of the users to hop on board and test it like its a Beta product. After that, open it up to a larger audience.
So as others have said, forgiveness over permission, but be smart about the impact on the IT organization.