I am using Specflow with Xunit and what I want is to have the (under the covers) generated unit tests have an Xunit.TraitAttribute added to them. Is that possible with Gherkin and SpecFlow?
Could you clarify what will guide how you would decide which ones should have the attributes? Unless it's a blanket rule, you'll want something that gives you more control than having the same attribute uniformly added to every generated test. Assuming there are T4 templates involved, you could obviously do something in there - bit the bottom line is that angle of attack is going to be messy.
You could so it as a post compile step with Mono.Cecil handily enough. Obviously PostSharp could do stuff like that too, but it might be overkill.
Related
I am in the process of learning ASP.NET Core MVC, while experimenting, I noticed that when I try to refactor the code using the built in refactoring tools provided by Visual Studio they don't work very well. For example:
Renaming a class via the solution explorer pane usually fails. When it succeeds it will fail to rename the class on the view declaration (eg: #model ClassNameIsNotRenamed);
If I rename a controller or action, the helper tags such as asp-controller="NotRenamed" and asp-action="NotRenamed" do not get updated.
etc.
I don't want to get too detailed on what works and what does not work, the point that I am trying to make is that VS 2017 does not appear to do a good job when it comes to refactoring.
So my question is... Am I asking to much from VS 2017? Is there something that I can do to make refactoring work better?
Thanks.
Doing something like a rename requires that your project can build successfully. If you attempt rename something like a class, and the project either hasn't been built or failed the build, the rename will fail as well. This is fairly logical, as doing a rename refactor requires tracking references, which it can't do without the IL.
As far as your Razor views and controller/action references go, you're dealing with strings and/or code that is not compiled. As a result, it doesn't participate in code-based refactoring generally. Certain things like renames give you option to search strings as well, which you can do to catch more places where things should be renamed, such as your controller/action references. However, that can also cause unwanted side effects if you happen to be using the same string somewhere else, in a different context (which is why string replacement is not done by default).
I have a messy Delphi 7 legacy system to maintain and develop. I am already reading "Working effectively with legacy code" and I like this book very much.
In order to start following the advices in the book, I created a test project and tried to write a single test. To do this I need to add some unit to the test project, but here lies the problem: the system under test has horrific uses dependencies. One unit uses some other unit, that uses some other unit and so on, and so on. It seems that most units directly or indirectly use one particular unit, and this unit in turn has 170 dependencies in its uses clause. There are indirect circular dependencies also.
Currently I am trying to add all of the legacy system's units into the test project, but I am running into all kind of problems, like "unit xxx was compiled with a different version of xxx", and others.
So I wonder if I am doing something wrong. I have used unit testing before, but in my own projects, that were smaller and with better structure and modularization. What are the options I have in this situation? Am I missing something?
You will always have dependencies in your code. Well, as long as you have code re-use, you will have dependencies. Since you are testing a legacy system, wholesale re-structuring is out of the question.
So you simply need to accept the dependencies. The most convenient and practical approach is to have a single unit tests project. That project contains all your unit tests. Use the facilities of your runner program to run only specific tests at any one time.
This leads to your project have the same list of units in its .dpr file as the main project. That's what you have currently tried and it's the right approach.
Your problem sounds like you are sharing the DCU directory (unit output directory) between the main project and the unit tests project. And you have different compiler options for the two projects. That's the most likely explanation for the error you report.
There are a couple of obvious solutions:
Align the compiler options for both projects. Then they can share DCUs.
Have separate DCU directories for the two projects.
Option 2 is much more robust and is best practise. However, you should try to understand why the compiler options differ. It's quite possible that your compiler options in the new unit tests project will need to be changed so that the units under test compile and function as desired. In modern Delphi I would use option sets to ensure consistency of compiler options.
Now, there may be other technical problems that you are facing, and my explanation of the error may not be quite right since I'm having to guess a little. But the bottom line is that having the same list of units in your .dpr files is the way to go.
After poking around in the files installed by Delphi, I have found a file
"C:\Program Files\Embarcadero\RAD Studio\7.0\ObjRepos\en\UnitTestFrameworks.xml"
This file has a number of sections for setting options that affect the test case code generated by the test case wizard that gets invoked when you create a new test case unit. Most of these options are self-explanatory but it would be great to see some documentation.
In particular, I'd like to:
add a second TestClassComment.
have multiple options in the wizard's test case base class combo box. I am able to change the value of the single item in this list by changing the value of the node in the UnitTestFrameworks.xml file but cannot get multiple items to appear in the list.
Markup the test class and methods with some attributes.
For the moment I'm only looking at the Delphi Win 32 personality.
Am I on the right track here? Is documentation available or failing that, does anyone have any tips?
I don't think there is any documentation on this.
If there is, David Intersimone at Embarcadero is who I'd ask for it.
Failing that, there are two options you might try.
Create some "bare" unit tests with what you want and save them as templates
Write your own wizard in Delphi to create unit tests
The unit tests are only text files, so the second option wouldn't be any harder than trying to hack the wizard. You could create your own unit test wizard in Delphi and hook it into the tools menu quite easily.
If you wanted to, you could have it read your class files to create test skeletons for all of your methods, or create a command line tool to generate them.
I haven't yet installed my license of NCover 3, and am still running 1.5.8 on my build server. I am trying to exclude full assemblies and specific classes that I don't want included in the report, because they are artificially lowering the coverage results.
In NCoverExplorer, I was playing around with the options because there is a coverage exclusions section in the Options tab where you can specify full namespaces. I've entered the fully qualified classes, and for some reason, only a handful of them get excluded, and I cannot figure out why. For example, when I add System.ComponentModel.Composition to the list, it never gets excluded!
Is this just a bug in 1.5.8 that I have to live with for now, since it is a beta and also no longer supported? Although I do have a new license for the server, I'd like to be able to do some coverage at home on my personal computer.
I found a really great article on using a CoverageExcludeAttribute to make NCover automatically skip those classes / methods that are marked with this attribute. Is this the best option?
Did you try using regular expressions to include/exclude the assemblies -
//ias AuctionSniper([.\w]*?)(?<!Tests)
includes all assemblies that begin with AuctionSniper but don't end with Tests e.g. AuctionSniper.Main.exe
You can specify multiple patterns separated by semicolons.
or //ias .*vendorsupplied.*;.*tests
This works with NCover 3 - you can give it a try if it works for the free/community edition.
One way to get part way there is to also use the "assemblies to include in coverage" option. This allowed me to ignore System.ComponentModel.Composition. However, exlcuding most of the other classes and namespaces still doesn't work. It's a little odd how when I exclude an entire class, it only excludes the methods, but not any of the contained classes.
Is there any way to measure code coverage with DUnit? Or are there any free tools accomplishing that? What do you use for that? What code coverage do you usually go for?
Jim McKeeth: Thanks for the detailed answer. I am talking about unit testing in the sense of a TDD approach, not only about unit tests after a failure occured. I'm interested in the code coverage I can achieve with some basic prewritten unit tests.
I have just created a new open source project on Google Code with a basic code coverage tool for Delphi 2010. https://sourceforge.net/projects/delphicodecoverage/
Right now it can measure line coverage but I'm planning to add class and method coverage too.
It generates html reports with a summary as well as marked up source showing you what lines are covered (green), which were not (red) and the rest of the lines that didn't have any code generated for them.
Update:
As of version 0.3 of Delphi Code Coverage you can generate XML reports compatible with the Hudson EMMA plugin to display code coverage trends within Hudson.
Update:
Version 0.5 brings bug fixes, increased configurability and cleaned up reports
Update:
Version 1.0 brings support for emma output, coverage of classes and methods and coverage of DLLs and BPLs
I don't know of any free tools. AQTime is almost the defacto standard for profiling Delphi. I haven't used it, but a quick search found Discover for Delphi, which is now open source, but just does code coverage.
Either of these tools should give you an idea of how much code coverage your unit tests are getting.
Are you referring to code coverage from unit tests or stale code? Generally I think only testable code that has a failure should be covered with a unit test (yes I realize that may be starting a holy war, but that is where I stand). So that would be a pretty low percentage.
Now stale code on the other hand is a different story. Stale code is code that doesn't get used. You most likely don't need a tool to tell you this for a lot of your code, just look for the little Blue Dots after you compile in Delphi. Anything without a blue dot is stale. Generally if code is not being used then it should be removed. So that would be 100% code coverage.
There are other scenarios for stale code, like if you have special code to handle if the date ever lands on the 31st of February. The compiler doesn't know it can't happen, so it compiles it in and gives it a blue dot. Now you can write a unit test for that, and test it and it might work, but then you just wasted your time a second time (first for writing the code, second for testing it).
There are tools to track what code paths get used when the program runs, but that is only simi-reliable since not all code paths will get used every time. Like that special code you have to handle leap year, it will only run every four years. So if you take it out then your program will be broken every four years.
I guess I didn't really answer your question about DUnit and Code Coverage, but I think I may have left you with more questions then you started with. What kind of code coverage are you looking for?
UPDATE: If you are taking a TDD approach then no code is written until you write a test for it, so by nature you have 100 test coverage. Of course just because each method is exercised by a test does not mean that its entire range of behaviors is exercised. SmartInspect provides a really easy method to measure which methods are called along with timing, etc. It is a little less then AQTime, but not free. With some more work on your part you can add instrumentation to measure every code path (branches of "if" statements, etc.) Of course you can also just add your own logging to your methods to achieve a coverage report, and that is free (well, expect for your time, which is probably worth more then the tools). If you use JEDI Debug then you can get a call stack too.
TDD really cannot easily be applied retroactively to existing code without a lot of refactoring. Although the newer Delphi IDEs have the ability to generate unit test stubs for each public method, which then gives you 100% coverage of your public methods. What you put in those stubs determines how effective that coverage is.
I use Discover for Delphi and it does the work, for unit testing with DUnit and Functional testing with TestComplete.
Discover can be configured to run from the command line for automation.
As in:
Discover.exe Project.dpr -s -c -m
Discover works great for me. It hardly slows down your application, unlike AQTime. This may not be a problem for you anyway, of course. I think the recent versions of AQTime perform better in this respect.
I've been using Discover" for years, worked excellently up to and including BDS2006 (which was the last pre-XE* version of Delphi i used and still use), but its current opensourced state, it's unclear how to make it work with XE* versions of Delphi. A shame really, because I loved this tool, fast and convenient in almost every way.
So now I'm moving to delphi-code-coverage...