The Spock testing framework has the Damage Control plugin that generates nice reports for your tests, and Spock has the spock.lang.Issue annotation, which:
Indicates that a feature method or specification relates to one or more issues in an external issue tracking system
It seems a no-brainer that the damage control report should be able to render the link in its report, but I can not figure out to do it.
As far as I know the #Issue annotation is currently unused. Either you ask the writers of Damage control to implement what you ask or you implement it yourself and send them a pull request :-)
Related
I plan to use https://github.com/risoflora/brookframework as an embedded static server in a Delphi application. It requires https://github.com/risoflora/libsagui and seems very fast. But I can't find an option to suppress messages coming from the libsagui DLL.
A common scenario is when the desired port can't be bound, I prefer to get an exception or some error handler callback, rather than get a MessageBox which can't be controlled by my application.
Any info or suggestion?
I'm going to take a flying leap here and just toss out what occurs to me at the moment. First, DLLs are not supposed to have ANY sort of UI activity. If someone does that deliberately and not because they just don't know, then it usually signals there's a serious problem afoot in the run-time activity.
Second, assuming that this signals a serious problem, then there's likely to be some mechanism in place to avoid it. This is for an embedded situation, so one might ask "why can't a desired port be bound" if the person doing the embedding is also responsible for configuring the overall embedded environment? One does not leave things undefined or unconfigured in embedded situations or the application just blows up at some point. So the obvious question is, why are YOU (as the person who configured the embedded environment) writing code that's trying to connect to unavailable ports? And why are YOU also complaining that the underlying library isn't throwing an exception rather than throwing up its virtual hands saying it doesn't know how to handle a request YOUR CODE should have known better than to make?
In other words, if you feel the need to add a layer to your code between your app and the platform you configured to support it, in order to validate your own requests, then you're free to do so. Otherwise, I'm guessing the embedded library designer expects you to know the limits you configured into it. (Or there may be some he did not anticipate.)
I've done a lot of embedded programming in my past, and if I was using a configurable kernal that I configured to only support 10 tasks (because I knew my application didn't need more than that), then in my client code I'd do a check to ensure that said limit wasn't reached BEFORE creating a new task and expecting to get an exception back if I was too stupid to follow my own design guidelines. In embedded situations, "exceptions" can lead to auto-shutdown or even self-destruct sequences. There is no effective way to handle "exceptions" in real-time situations, and often even just embedded situations.
Imagine pushing on a brake pedal in a car and an exception is thrown -- the user is saying "slow down" and the library says, "Sorry, I can't do that right now". It's shades of "2001 A Space Odyssy" with HAL telling Dave "No". That's what "exceptions" in embedded systems can result in.
What's different between that and your app saying, "Connect to this URL via this port" and the underlying library saying, "Sorry I can't do that right now". What is your exception handling code going to do? You need to detect that stuff FIRST rather than wait for an exception to be thrown by the embedded library.
Finally, this is more of a rant than anything else ... but one thing I've found sorely lacking in the vast majority of open-source projects on github I've looked at is any clear explanation of one or more USE-CASE MODELS. In this situation, WHY did the author choose to design this embedded DLL so it has situations where it simply throws up its hands by displaying an error message in a low-level MessageBox rather than providing some mechanism for handling the issue more elegantly? Obviously, he had SOMETHING in mind when he did that. Unfortunately for us, he did not explain it anywhere. Here we are trying to figure out how to deal with something that, on its face, makes no sense -- a DLL throwing up its hands when faced with an exceptional situation.
So my answer is, in the "spirit" of open-source code repos, you're free to dig into the code and try to figure out the USE-CASE MODEL that the author had in mind when he created this thing, and adapt your code to fit that model. Maybe start by looking at examples of code he posted that use this library, although if they're your typical trival examples people often post, then they won't offer a lot of insight into the overall USE-CASE MODEL that is leading to your situation.
Said another way, assume the library is behaving "as designed". Then WHAT IS THIS LIBRARY EXPECTING YOU TO DO IN YOUR CODE IN ORDER TO AVOID THAT SITUATION FROM ARISING?
Hopefully someone will see this who has a much shorter definitive answer to that.
My team is using scrum and TFS to manage a new software project. Some members of the team want to solve a problem in a rather unusual way. I need to know if anyone has dealt with something similar. (This is partly a scrum/project mgt question, and partly TFS because if TFS makes one approach much easier than another, that will influence the decision.)
There are parts of this software system that have not yet been specified in any manner—not via User Stories, Acceptance Criteria, tests or anything. They are to some extent "corner cases" or error-handling cases. Yet the software already behaves in a particular way when these cases are encountered. (This may be by displaying a generic error message or going down a common path to some resolution.) This situation exists because the software is designed to be error-tolerant.
Team members want to define and thereby lock down the unspecified behavior, so it won't change. If it does change—and in particular if it changes for the worse (e.g., crashing instead of displaying a generic error message), they want to catch that.
But they are proposing to do this by writing tests or Acceptance Criteria that match what the system already does in the corner cases. My position is that any currently-unspecified behavior that we want to keep stable should be defined via new User Stories first, not via Acceptance Criteria or tests. What is the proper scrum/TFS way to document and lock down as-yet-unspecified behavior in an existing body of software (preferably with the least effort)?
It is something that the users of the system will benefit from, so do it as a user story:
As a user I want the product to be stable and not crash when a specific behaviour occurs so that I have a good user experience
To satisfy this user story you may well write a technical task that leads to tests that simply match the existing edge case.
Why bother making it a user story? Well this work item will need to be prioritised against other backlog items, including new functionality. By creating a user story we allow it to be prioritised using the standard Scrum backlog approach.
I have recently been tasked to look into Workflow Foundation. The actual goal would be to implement a system in which the end users can define custom workflows in the deployed application (and of course, use them). Personally I have never used WF before (and reading around here on SO people are very doubtful about it - so am I reading those questions/answers), and I am having a hard time finding my way around it given the sparse learning resources available.
Anyway, there are some questions, for example, this, which mention something they call dynamic or user-defined workflows. They point out that WF makes it possible to "rehost" the designer, so that end-users can define their own new workflows after the application is deployed (without developer intervention (?), this is the part I am not really sure about).
I have been told by fellow employees that this way we could implement an application in which once this feature is implemented we would no longer have to keep modifying the application every time a new workflow is to be implemented. However, they also pointed out that they just "heard it", they don't have firsthand experience themselves either.
I have been looking around for samples online but the best thing I could find was a number guess app - barely more than a simple hello world. So not much that would point me to the right direction of how this user-defined workflow feature actually works and how it can be used, what its limitations are etc.
My primary concern is this: it is alright that one can define custom workflows but no workflow is worth a penny without the possibility of actually inputting data throughout the process. For example, even if the only thing I need to do is to register a customer in a complaint management system, I would need the customer's name, contact, etc. If the end user should be able to define any workflow the given toolset makes possible then of course there needs to be a way to provide the workflow consumers with a way of inputting data through forms. If the workflow can be of pretty much any nature then so needs to be the data - otherwise if we need to implement the UIs ourselves then this "end-user throws together a workflow" feature is kind of useless because they would still end up at us requiring to implement a form or some sort of data input for the individual steps.
So I guess that there should be a way of defining the "shape" of the data that needs to be filled at any given user interaction phase of the workflow which I can investigate and dynamically generate forms based on the data. So for example, if I found that the required data was made up of a name and a date of birth, then I would need to render a textbox and a datepicker on the page.
What I couldn't really figure out from the Q&As here and elsewhere is whether this is even possible. Can I define and then later "query" the structure of the data to be passed to the workflow at any point? If so, how? If not, how should this user-defined workflow feature even be used, what is it good for?
To clarify it a little, I could imagine something as specifying a complex type, which would be the view model (input model) in a regular MVC app, and then I could reflect over it, get the properties and render input fields based on that.
Windows Workflow Foundation is about machine workflows, not business workflows. True, it is the foundational tool set Microsoft created for building their business workflow products. But out of the box WWF does not have the components you need to quickly and easily build business workflows. If you want to send an email in a workflow, you have to write that from scratch. Just about anything you can think of doing from a business point of view you have to write from scratch.
If you want to easily create business workflows using Microsoft products check out the workflow stuff in SharePoint. It is the easiest of the Microsoft products to work with (in my experience.) If that does not meet your needs there are other products like BizTalk.
K2 is another company with a business workflow product that uses WWF as their base to more easily build business workflows, the older K2 products actually create web pages automatically to collect the data from the user.
WWF is very low level, arguably it lost traction after they re-wrote the whole thing in 4.0. While not publically stated by Microsoft, my personal opinion is Service Fabric (from Microsoft) achieves the goals WWF originally tried to solve which was a "more robust programming environment."
We have a workflow engine that presents a list of available workflows (I mean workflow definitions, not instances) and user can click on the "Execute" link next to any workflow to ,well, execute a new instance of that workflow. I want to do this "execute a workflow" story(feature?) in the BDD way.
Story: execute a workflow
Scenario: execute a workflow by clicking on execute link in workflow list and nothing goes wrong
Given I am a user with sufficient rights
And I have added a workflow called "wf"
When I click on the execute link next to "wf" in the workflows list
When I view the list of workflow executions
Then the output is:
"""
1 | wf1 | not started
"""
(1st column: item#, 2nd: workflow name, 3rd: state)
I kinda feel this is more like a mess than a nice cut DBB scenario, I'm specially concerned with the acceptance criteria. My mind is not clear about how exactly I should approach something coarse-grained and user-coupled like "executing a workflow". I mean when it is API you are doing, everything is clear but what if you are describing some behavior which is initiated through (human)user interaction and the result of which is evident from initiating another use-case with complex output (like a list of items). The criteria for knowing that the workflow is indeed executed is to see a new item in the list of workflow executions, which is another story for itself. I kinda feel confused here.
Should I talk to database layer and check for the row that stores the newly created workflow instance -or- should I check for the presence of the item that points to the new instance in the list of workflow executions? If the second, then how exactly? should I check for all columns with correct values in one scenario or each column in it's own scenario?
May I refer you to a post I did quite recently on Acceptance Criteria vs. Scenarios? I think the example might be more illuminating if you use something resembling a specific use of the workflow engine, rather than a generic one. For instance, here's a fake pet shop which I'm using to try out an automation tool. I've then written scenarios around the pet shop, rather than trying to specify generic automation concerns.
If your customers are sometimes in healthcare, for instance, knock up a fake diagnosis tool which uses your engine and write a scenario around that. It might seem like a bit of work to start with, but I've found it pays back for itself very quickly.
Story: A doctor diagnoses black death
Scenario: The doctor starts the diagnosis
Given I am doctor with rights to use the system
And I've added a workflow called "diagnosis"
When I choose the "diagnosis" workflow
Then it should tell me that it's not started.
This is the benefit you're looking for - that a user gets some information, not that something is stored in a database. As far as possible, a scenario should push for the end value! So maybe it should even say something like:
Story: A doctor diagnoses Black Death
Scenario: The doctor starts the diagnosis
Given I am doctor with rights to use the system
And I want to diagnose a patient
When I choose "Diagnosis"
Then the system should prompt me to start diagnosing.
Given that all the symptoms match Black Death
When I perform the diagnosis
Then I should be able to diagnose the patient with Black Death.
Any smaller steps which are needed to make this easy and aesthetic are really usability issues. Don't use BDD frameworks to describe usability concerns (though they can step through them, thus giving you your regression tests). Instead, try it manually. BDD isn't a substitute for manual testing, it just helps out a bit.
If you can create a vaguely realistic use of the workflow engine, it'll help you to think of scenarios which you might miss. For instance, I have no idea right now how this workflow can be associated with a particular patient. I find specific, imaginative examples tend to help people visualise other scenarios more than something vague, generic and all-encompassing.
Also, try phrasing it in the same language the business might use, thinking about the business outcomes that you really want. Try not to think about how to implement the scenario - instead, just write it. This will be much, much easier if you actually go and talk to your business people or customers about the scenarios they can think of!
Any complexity needed to make the scenario run then goes into the code, where it's easier to maintain and refactor.
As an additional benefit, by identifying particular customers with particular needs, you can help your customer avoid the trap of putting every possible feature into the workflow engine "in case someone needs it". By talking through real scenarios with real people, they'll be able to help identify who needs which features the most, reducing scope and helping you to deliver as much value as possible.
We have utilized Specflow and WatIn for acceptance tests at my current project. The customer wants us to use Microsoft coded-ui instead. I have never tested coded ui, but from what I've seen so far it looks cumbersome. I want to specify my acceptance tests up front, before I have a ui, not as a result of some record/playback stuff. Anyway, can someone please tell me why we should throw away the Specflow/watin combo and replace it with coded ui?
I've also read that you can combine specflow with coded ui, but it looks like a lot of overhead for something which I am already doing fine in specflow.
I wrote a blog post on how to do this you might find useful
http://rburnham.wordpress.com/2011/03/15/bdd-ui-automation-with-specflow-and-coded-ui-tests/
The pro's and con's of Coded UI Test that i can think of is your testing the application exactly how the user will be using it. This is good for acceptance test but it also has its limitations. Its also really good for end to end testing. In the past UI Tests have been know to be fragile. For example when MS created the VS2010 UI almost all of the UI tests broke. The main reason being is the technology change. Coded UI tests do help to limit this from happening by the way it matches a control. It uses more of a probability based match. This mean it will try to find the best match based on the information it has such as control name. For us Coded UI tests was our choice because of technology limitations. Our Legacy app is VB and although CUIT does not work great, i'm in the progress of writing an extension to get better control information, it was still our only choice. Also keep in mind CUIT is new and has its own limitations. You should be prepared to be very structured in the way you lay out your project as maintaining your UIMaps can be a bit of manual work due to the current end to end behaviour in VS2010, for example creating a CUIT from an existing action recording always places the test in a UIMap called UIMap.uitest and there is no way to change that or transfer to another UIMap. If you use multiple ui maps this means you will need to record your steps first and then use them in your test. However being in .net it its still very flexible.
By far the best thing about specflow is its gerkin syntax for readability and living documentation. Normally your testing features or behaviours of your app which is where the value comes from It generally aims the test just below the UI. There is a little less chance of the test breaking when the UI changes here but there. Specflow to me is great when your application is under constant change and you want to ensure existing features remain working. It fits well in a Scrum environment as well where you can write your scenario's as a description about how it should work. One limitation to specflow i can see is its open for interpretation. Because of this it can be easy to write a test that is not very reusable and hard to maintain. I like to use more generic terms to describe my steps like "Log in as User1" instead of "Go to Login Page, Enter Username and Password, Click login". Describing it more granular makes it harder to reuse tightly couples it to the UI. How the login actually work should be up to the code behind not the specflow feature.
Combining the 2 however to us seems more beneficial than just using Coded UI Tests. If we decide to completely change the UI we would at least have the behaviours that are expected stored in our specflow features in a way anyone can understand. In the end you need to consider how the application will evolve and the type of application.