So our Company just started using Jira, and i've created a new "Issue" type for Test Plans.
However I'm sorta running into the problem of that when creating a "session" in Jira (Capture for instance) that test plans and actual bugs are all kinda under the same thing.
It seems to me that Jira was never meant to really be used to store test plans, am I missing something? I've looked at the Jira guides to make it work for keeping test plans, but it just seems so convoluted and overly complex that it seems like another TCM tool makes more sense.
Has anyone else ran into this? Am I missing a better way to do TCM with Jira?
Thank
JIRA is a ticketing system.
As it's marketing buzzword bingo says: "Issue tracking and code integration to plan, collaborate, and ship great products." It does more if you find or create the necessary plugins.
It does issue tracking by itself, and code integration with other Atlassian platforms.
It does not do test management.
There are plugins for that (like Zephyr) by third parties, but in general, I don't recommend mixing test case management with issue tracking. Make your separate TCM tool to create a ticket in your separate ticketing system.
Related
We have been using waterfall to develop and enhance the system in our company and recently we are moving into Agile and the management is interested in BDD.
My understanding is that BDD is a tool for business and development team to refine the stories and at the same time these scenarios will become test cases for the feature. This sounds perfect to me but since we already have the features available, how can BDD work in this type of situation?
Should we just write up the stories and scenarios per our knowledge of the feature?
My only concern of the above is the coverage of the scenarios. Or we shouldn't worry and keep adding new scenarios and test it whenever the team came up with new ones?
Prompted by yet another person who mailed me with the same question today, I've written up an answer for this.
Short version, you can use BDD to help you understand what the system actually does, and why, but you'll be clarifying the requirements rather than exploring them.
Additionally, you asked, "Should we just write up the stories and scenarios per our knowledge of the feature?"
I'd speak with any stakeholders you can find, ask them what the system should do, then look to see if it actually does it. Systems designed before adopting a practice of conversations with examples often don't do what the originators intended. You can then differentiate between the behaviour you've actually got, while creating a new backlog from the behaviour you want.
I advise grabbing someone who's good at asking questions and spotting missing scenarios to have these discussions with (usually a tester). Because you already have some knowledge of the system, it's likely you'll be very good at describing what you think it does, while missing gaps.
If you don't have any automated testing yet and you want to start using BDD, I would suggest you to start by writing some Scenarios for some of your manual tests scripts, I find it to be a good way to train the writing of the BDD style, then as Lunivore said, you should work together with the business people and QA to find out better about the behaviour of your system and preferably write the scenarios with them.
I have a project with 4 teams handling different parts of a fairly large web based system. What would be the best way of handling this in Jira and Jira Agile?
One project and board per team or is there another way of organizing this in Jira Agile?
Read about using components but since I'm fairly new to Jira Agile I don't know how to do that.
If many team are working on different part of the same system, I suggest you create only one project. From my experience there is a lot of chance you will want to move 'items' from one team to another or link two items that are related but not handle by the same team.
Like previous posts mentioned, GreenHopper is just an (agile) plugin that help you manage items in JIRA, allowing to display and manage them in different ways. The plugins is especially useful if your are following a development process like Scrum.
With the last version of the tools I was able to easily share a single 'backlog' of work to do between multiple teams and then 'assign' items to a specific team when it's time to work on it. When planning for the next Sprint, for example.
Of course having a clear view of your process is key.
I think you are confused about a few things, which is understandable for somebody new to the tool.
Jira is the software you are using and Greenhopper is an external company that produces plugins for it, mainly the Agile plugin. Components are within Jira and they are simply a way you can categorize your issues. I suggest you find a tutorial video on jira to introduce you better to the tool.
Now, again, these are just tools. They don't dictate how you should work. Jira is very flexible that way so you can do Agile, Scrum, Kanban or whatever methodology you want within it.
You need to first decide what methodology/process you will adopt. Basically, how will you guys work?
Are all the teams working in sprints? Do they have the same deadlines? Are the stories-issues parts of a whole or can each team deliver a full feature on it's own?
For example, if one team is infrastructure, one is UI and one is DB, their parts will likely come together to make a "whole" feature, that is complete and tested.
Another example is if team A is doing a Reporting Module and team B is doing a Login module, their features aren't usually related and they can work separately.
So basically, you can't ask anybody here to give you the straight answer. Stop focusing on the tool and understand how you will work and what makes sense for you.
And remember: agile is trial and error. Try something and if it doesn't work, adapt.
I am currently creating a business case for rolling out TFS 2010 as our source control and bug/release management tool.
We currently use OnTime for our bug tracking software and subversion for our SCM.
I was wondering what advantages TFS 2010 has over OnTime?
I have done some thinking so far and would love to hear responses:
TFS 2010 allows linking changesets->work items->builds
TFS 2010 provides greater customisation of workflow than OnTime
TFS 2010 is integrated into the Visual Studio IDE - This requires less apps to be open and less window flicking
Thanks in advance.
TFS is one of the least intuitive Version Control systems I have ever had the misfortune to have to use. It may have numerous "bullet point" advantages over OnTime (and other comparable systems), in terms of raw feature-lists and capabilities, but the key factor is whether it can fit in with your working processes.
My experience with TFS is that you will be required to adapt to the TFS way of working, because adapting TFS to your way of working will be impossible or too difficult to justify.
We recently reviewed a number of possible alternatives to replace a system comprising SVN and a manual bug-tracking system (Excel spreadsheets). On-Time was evaluated but deemed too expensive and complex.
In the end we opted to continue using SVN, but drastically revised (simplified) our repository structures and chose to combine SVN with the FogBugz issue tracking system. The integration between these two systems was fairly rudimentary "out-of-the-box", but required only a little effort on our part to achieve the much closer level of integration we desired. Certainly far LESS effort than my previous experience of a TFS roll-out involved.
Our SVN/FogBugz system is also now integrated with a FinalBuilder build automation suite.
The result is a system that not only fits our working practices perfectly (since we devised the means by which the systems would integrate to achieve that) but which is also infinitely adaptable as our working practices evolve.
I think that it really depends on the size of your team(s), and what you want out of source control.
I used bugzilla in combination with Perforce for a couple of years and found that both were really very good at their own individual things while working in a very small team (2-3 people), but the suffered from a lack of integration between them and from some little idiosyncrasies that took time to get used to.
I recently moved to a new job where TFS is used extensively. There are 4 main teams in this company with 10-12 developers in each, split into further project teams below that level, and it is in this kind of environment that TFS really shines imo. It's biggest advantages in my view are:
1) The integration with Visual Studio - it's not just a case of having less windows open, but it really does speed things up and make your life easier. Things like VS automatically checking out files for you as you work (no issues with accidental checkouts due to lockless editing), being able to synronise local + TFs builds, being able to quickly compare the local version against previous ones..yes you can get 3rd party plugins to integrate but none to this level and with the same stability.
2) The communication features - simple things like integraton with Live Messenger (provided you configure TFS correctly) are great for large teams. We use WLM to communicate accross the office and for collaboration as its just quicker than walking over to someone else every time you need to ask a quick question.
3) Linking builds/changelists to tasks - Yes other SCMs do this but again it's just done in a very nice, integrated fashion..I guess it's nothing special to TFS but personally I like how it tracks this.
4) Ease of merging/lockless editing. I've had experience with some other merge tools and the TFS one works nicely enough, making merging after concurrent editing pretty simple. It's very similar to perforce in this respect, but also with a usually pretty effective auto-merge tool which I use for tiny edits that I know cannot cause any potential issues with edits other developers are working on.
5) Auto building/build management. Working with a couple of large solutions containing 20-30 projects that depend on each other, this is a godsend. We have it set to queue up a build every 20 minutes IF something has changed, and when one has happened its listed in the history log..so easy to see when you need to update your local libraries.
I don't have any experience with configuring it other than build management, but I have heard that this is the worst part of TFS..that its a bit of a pain to get everything running correctly.
So, translating that to a business case..I'd say that if you are a Microsoft software house with large/multiple teams, then the time savings and productivity improvements that you will see as a result of the above features are worth the investment in setting it up. Its free to use in most cases as you will probably have a MSDN subscription (maybe some CAL issues but i'm not sure) so your biggest cost will be in user training and configuration.
Firstly, I would suggest to consider what is your primary concern, what is the problem that you are tying to solve by rolling out TFS.
In terms of version control I would recommend the blog post from Martin Fowler on Version Control Tools and a follow up results of a version control systems survey. Admittedly this might be and is a subjective view of the subject but one that seems to be pretty popular. TFS clearly looses in comparison to other Version Control Systems.
I currently work with TFS2008 and we have migrated from SourceSafe and IBM ClearCase/ClearQuest and there is no doubt that TFS is far better then any of the previous tool, still it has its serious shortcomings and the new version will only partially address those.
Addressing the individual point you have raised:
TFS allows to link builds with changesets and work items, but so many other systems
I have not used OnTime but the workflow customisation can be both an advantage and a hindrance. Potentially, there might be a lot of work involved in creating a custom process template and you would still need a sensible UI on top of it (Team Explorer or Web Access might not be sufficient)
Integration with Visual Studio is an advantage but there are add-ons to Visual Studio that allow integration with other source control providers
On the advantages of TFS I would probably mention
Distributed builds and separate build agents - if you do many builds
Full integration with Visual Studio via the Team Explorer
Extensive reporting infrastructure (though you can only take full advantage of it when using MSTest for all the testing)
SharePoint collaboration site for each project
Given the substantial cost of rolling out full TFS installation I would really consider what real business benefit would this solution give you that others don't.
Not shure about TFS, but the UI of OnTime is kind of non intuitive.
Also I dont like that you have different fields for Bugs and Tasks. Of course you can always add your own fields, but the default layout should be ready to use.
We endet up using only "Bugs" even if it is a task.
I dont say its a bad product, but if TFS has a better UI for bugtracking now (which it hadnt 4years ago when I had to use it and hated it ), then this would be an argument for TFS.
Sorry to hear that you want to get rid of SVN. Thats a hard decision.
I'm not sure about the licensing for the Axios OnTime but if you have an MSDN subscription then it's no additional cost. See the blog post here
I've been using TFS 2008 only for version control and while it's a nice upgrade from VSS some things that we're tyring to do aren't exactly in line with what is expected. That said, I've written a quick little web app that fills in those gaps. It was pretty easy to develop against using the API and there's lots of addons to help with specific tasks.
Probably not the answer you want to hear, but I'd be doing my damnedest to make a business case against TFS.
In any event, my general advice would be to try it out yourself (or in a small team) on some very small, but real project - maybe some tool you need on a once-off basis, code that can be thrown away or easily migrated to another system because it's small. There's nothing like actually using the system!
I have used OnTime and Subversion. I have not used TFS as bug tracker, but I've used it for source control. The source control part of it is basically still the bad old Visual SourceSafe. If you are currently using Subversion you will be swearing your head off any time you need to rename a file or, heaven forbid, delete a file and then create one with the same name - never mind any branching or merging. It's hard to convey in a post just how primitive and fragile it is as a source control system - that's why you really have to use it. You'll see what I mean when you find yourself stuck with a file you can neither check in nor delete and some meaningless error. Not that Subversion is perfect - but it's a decade ahead of VSS!
The workflow part of TFS, which I've only briefly played with, seems very "heavy" to me. That is, it really restricts the user to that workflow and requires a lot of steps that are often unnecessary. This stuff can help, but it can also just as easily get in the way. A good system provides the workflow when it's needed, but allows you to bypass it when it would just get in the way. When we used OnTime, we found that even its relatively unobtrusive workflow was often just more trouble than it was worth. Of course, this all depends on the specifics of your situation. How are you using OnTime workflows now and what do you want out of TFS that OnTime doesn't provide?
Linking changesets to bugs can be done with Subversion as well. It supports some extensibility mechanism - I don't remember the details, but FogBugz uses it (we switched to it after OnTime). Linking the to builds can be done by adding a simple svn tag command to your build script. Visual Studio integration can be done with VisualSVN.
The cost is also a huge downside of TFS. It is very expensive for what it does, especially when you take into account how well it does it. Yes, it's "free" if you have to have an MSDN subscription for every developer anyway - but do you have to, without TFS? Subversion is free, full stop. OnTime and FogBugz are far more reasonably priced.
I would strongly recommend against TFS. I once tried to restore the source code from a crashed instance, but I gave up after a few days, so source code was lost (= it failed to do the one thing a VCS should do). Of course, I might have done something wrong, but it's not easy to get everything right when the restore guide is two miles long, and it really is something that should happen so rarely that nobody is experienced with it.
Now I use Subversion/Trac, which gets the job done (and customizing the workflow in Trac is so easy it's not fun, compared to TFS).
For the time being, avoid TFS!
I would stick with SVN + FinalBuilder and then choose between FogBugz or CounterSoft Gemini.
I finally was able to complete the installation of TFS and started the creation of my first team project which introduced me to the process template.
After following to the link to Microsoft's site for process template information I was inundated with new information to consider. What templates have all of you had experience with that either worked out very well for you or were more of a stumbling block to the project? What were the biggest advantages and disadvantages you've encountered?
Some information about my project, I'm the lead developer for a small company and will be using TFS/VSTS to create an intranet portal to consolidate the end users day to day and increase automation to enhance productivity etc. It's entirely new development taking advantage of C#, ASP.NET and SQL Server 2008.
Ideally I'd like to take advantage of features to enhance collaboration with the stake holders to help add desired features and to track the status of development and offer feedback etc. I was also looking to take advantage of JetBrain's TeamCity for my TFS so if any specific template / software really adds cohesion between TFS, TeamCity, Developers, and Stakeholders that would be ideally what I'm interested in.
Are you already using a software development process like scrum? If yes you can try this Team Process Template over here.
How large is your project team and the project? Microsoft has published one of it's internal Process Templates (MPT) over here. You can get some guidiance and inspiration from this template.
As tangurena mentioned. People use the standard templates, change the bug a bit and store some documents there. I would recommend to keep the process 'light' as well.
However the process template isn't all.
Here are some ideas what I would do (in your case):
Create some high order workitems (features/stories) which stakeholdes can create (constraints and TFS user groups are your friend). They can then access their requested features via the TFS Work Item Web Access. That way you don't need a CAL for them
Create some reports which show planned work accodring to releases.
Setup the build automation and create Reports (a.k.a. Release Notes) from your workitems according to the builds.
What were the biggest advantages and disadvantages you've encountered?
Imho the biggest disadvantage is that you start believing that the template is your silver bullet. It's not, it's your starting point.
The TFS ecosystem offers you alot opportunities to create own bits of software that fit your needs. Just check out the TFS API.
Here is another nice agile-based template (original is on SSW, but you have to get around a login wall).
This template helps enhance cohesion between developers, managers, and other stakeholders by including more robust support for project process (documentation, reviews, &c., &c.). For example, there are types built in for process elements like release plans.
In general terms, I'd favour as small a process as you can manage. The more states, the more fields you have, the more likely the information in them is just plain wrong.
We're running with our own version on the Agile template. Most of what we did to it was delete stuff.
You can use the TFS API to log builds into the database, which should enable you to bridge TeamCity and TFS. Other than that, I'd probably just go with the web interface that comes with TFS, I don't think you need third party software for this.
K.I.S.S.! I created a custom work item based off the Agile one. And thats it, just one work item. There is a "System Severity" that IT uses and a "Business Priority" that the client/customer uses. There is also a "Request Type". With those three along with the built in Area and Iteration the entire team, including the clients can query the work items to get only the items they care about for the release they are concerned with (or all of them regardless of the release).
I did not modify the state machine much at all. This left us with something that is very flexible for everyone. Everything from blue sky requests to the mundane content/visual bugs can be logged there.
The client uses TFS Web Access (unlimited CAL) and the devs (me and 1 other) use VS. At my last job I created the same setup, the dev team was a team of 5 and it worked even better there! I was dev lead there as well and technical PM.
The biggest advantage was having a very flexible system for everyone, when using 1 work item type for everything. The disadvantage would be a learning curve for the client, but once they knew how to use it most like it. A suggestion would be to look into cheaper tools out there for a similar implementation, but, our .edu discount with MS cant be beat.
I would have to say that you must identify the system you will use for your company's SDLC first. The process template is merely a tool and without a good understanding of the underlying process it will not help and can make things more difficult. User adoption is crucial to the success of the SDLC and process template.
We use Scrum for Team System. We chose this due to our experience with Scrum as an SDLC methodology. There are several excellent books and articles on the web to help you get up to speed. Scrum will tie together the business stakeholders into the process.
In our system the Product Manager is in total charge of Product Backlog Items and works with myself and the CTO to prioritize them into Sprint Backlog Items.
The only change we have made to the process template was to add a "Failed Test" state and corresponding workflow.
It might not be the best template for you but I still wanted to mention it here: XP for Team System. It is basically a simplified version of MSF for Agile Software Development:
[...] it removes some of the setup tasks that an XP project will probably not want to undertake and changes the Work Item Type name Scenario to Story.
My company are imposing Jira and Zephyr on us for defect tracking and test management. We're quite happily using TFS 2008 for both these jobs at the moment, but management have never let the fact that something isn't broken stop them from trying to fix it.
Are there any tools/plug-ins that will allow us to synchronise between the remotely hosted repositories and our in-house TFS server?
Probably too late, but the company might want to look at the new features for bug tracking and manual tests coming in the 2010 release. Nice as Jira is, I doubt it will integrate well with the historical debugger and the ability to include a video of the test, as well as information on the test environment, and have it all be part of the work item.