When we build our MVC app, we have a build process that pushes the site on to our UAT box.
Once published, We would like to run an automated tool that crawls all the links in the app and checks for broken links and any other issues (such as usability/accessibility, etc.)
What tool(s) exists that will crawl a site and generate a report a report on broken links, etc.?
Can it be integrated in to our CI (TFS) Build?
In the absence of any alternative advice, we went with this solution:
http://wummel.github.com/linkchecker/
It's working well for us. One of the key advantages of this system is that it has a command-line mode with lots of options AND it can produce multiple format reports (HTML, Sitemap, CSV, XML) in a single crawl. Other tools I've used in the past had to run multiple times, consuming more bandwidth and time.
The beauty of this is that we can add it to our build process and then automate the output. When the build is pushed to UAT, LinkChecker runs. When it has finished, the HTML output is emailed to us and the XML output is parsed. All 404 errors are raised as bugs in TFS.
Quite a nice solution.
Related
I've been trying to make my Jenkins builds 1 stop shopping for everyone to make it easy for QA and for on-boarding never team members. Be they business, QA, DEV, etc...
I currently use HTML reports to publish links to various documents about how to deploy a build, how to test a build, design docs, etc...
Unfortunately, many of our builds are 1-2 gigs in size. For each HTML report you publish, that plugin copies the build repo into a report folder under html reports.
So for a product that has a build usage of 1.7 gigs... When I publish 21 reports, it uses and additional 22 gigs of disk space... That is per build, if you keep 10 builds and you set save old reports, I've turned that off, but still the disk space usage is a problem...
Then multiply that by 20 different jobs.. Well, suddenly disk space becomes a serious problem.
So... How to publish external link, HTML reports, etc... That doesn't use so much extra space?
We have tried to use the description before, had problems with making that readable, but it worked for the static links with out disk space problems. For the html report we do generate in the build, that did not work in initial efforts. If that really is the best option, we can give that a go again.
A couple of solutions come to mind.
Create and deploy a single page that has links to where the other documentation can be found.
Check out the Sidebar-Link Plugin. It allows you to put links in the left menu bar (sidebar).
We've written a framework to test the performance of our Java application (none of the existing frameworks, eg JMeter, were appropriate). The framework produces various metrics, e.g. mean/min/max transactions per second.
We'd like each Jenkins build to display these metrics so that we can keep track of whether a commit has improved performance or not.
I can't figure out how to do this.
One idea is to modify our performance test framework to output a HTML file, then somehow make Jenkins display/link to it on the build results page.
Any advice gratefully received.
The Peformance Plugin can show the results of JMeter and JUnit test in the nice, graphical fashion. And on the plugin page there is a description on how to use it.
This is an open-source plugin hosted on GitHub. The JUnit and JMeter parser are already there, but You can implement your own just by subclassing PerformanceReportParser. It's pretty easy and you can just fork the repo and start your implementation.
I agree that it is hard (if not impossible) to squeeze all the information into standard formats, like JUnit. They are good for quick identification of problems. Once you know there is a problem - you need more information that is usually free-form or custom-formatted to fit your particular needs. So we use both: JUnit that can be immediately processed by Jenkins to decide if the build is stable or not, draw the nice trend graph, etc. We also produce an HTML report that is much more detailed.
Now to your immediate question: you can simply archive your HTML file as an artifact (there is a standard post-build step to do that). Then a link to it will be displayed among the artifacts for the build. There are permalinks to the latest artifacts and latest successful build artifacts:
http://[server]/job/[job_name]/lastCompletedBuild/artifact/foo.html
http://[server]/job/[job_name]/lastSuccessfulBuild/artifact/foo.html
You may bookmark those links and have quick and easy one-click access to your results.
You could use the HTML Publisher Plugin to publish the resulting HTML page. That would be pretty straightforward.
If you want better integration you could try to create output that follows the same format JMeter produces, and use the Performance Plugin.
For best result you could take Łukasz advice and modify the Performance Plugin to your needs. That requires the most effort on your part, of course.
I apologize for the length of this post but I needed to include a lot of information for proper answers. I hope this does not discourage responses...
Our shop historically has coded web sites using Classic ASP with some newer ASP.NET sites configured as web sites. As everyone knows this means that the source files (*.asp, *.aspx, and *.aspx.vb (or *.aspx.cs)) files are deployed to development and production servers as is.
The configuration management process was (and still is) entirely manual and includes the following steps (requirements):
Taking copies of the modified files and storing them in a "release" folder for archiving.
Taking copies of the production files that will be replaced and storing them in a "archive" folder for easier rollback.
Generating a diff report of before and after source files for code review or general reference when diagnosing a post-release issue.
The developer who coded the changes is not the person who performs the production release. The original developer is required to hand off the source files to another developer for some additional testing and production deployment.
To make the situation more difficult (not with the above..but with what I talk about below) we do not follow a formal release schedule. As individual bugs or enhancements are completed they are released. This means we could easily be making several releases to a site a week. It is even possible that a given site gets two different releases to individual pages on the same day!
Since I came on board I have been trying to transition the team to newer technologies like ASP.NET web applications and ASP.NET MVC. (We have also taken on responsibility for stand-alone applications and console utilities used for non-web processes...so my dilemma still applies.)
The difference between these technologies and the legacy technologies is the pre-compiling. Instead of deploying the code-behind files (*.aspx.vb (or *.aspx.cs)) a dll or exe gets deployed. This type of deployment package has raised several questions (issues ??).
Generating difference reports when the source has been compiled. While the newly modified source files are sitting on the developers system the production copy is a compiled copy.
Making sure that changes related to other bugs or enhancements are not included in the particular release. This would apply to both the original developer and the person performing the release.
Allowing the original developer to pass along the changed files to another developer for build, testing, and deployment.
Up to now I was the only developer on the team working on these types of sites and applications so the conflicts and issues mentioned above where non-existent. (I skip the difference report step and the I do my own deployments.) However, I am trying to push the rest of the team to embrace this plus allow for better distribution of bugs and enhancement tasks.
We are currently using VSS but I am pushing (and will most likely succeed) in getting us moved over to TFS. Some ideas I have are
Setting up a separate build system for use by the developer to do the deployment. This will solve two problems -- (1) Different versions/patches of Visual Studio and other libraries between developers and (2) instances where the person performing the release has checked out files locally for another change. (Of course this does not guarantee differences between the build system and the original developer but at least that means the release is from a consistent config.
Using labels to tag just the modified files. My problem is that while I can identify (and pull down for a build) the modified files, how do I identify the files that need to be included in the build but have not changed. Again, the idea is to not included checked in files that are related to un-released changes.
Using labels to tag all the files for the release (the modified files and the unchanged files). My problem with this is similar to the last one...how do I make sure that a file checked in by another developer (say they went on vacation) for an un-related change is not labelled and included in this build.
Using the labels I could probably write a script to generate difference reports for the labeled version and the previously labeled version. If the process works properly that should result in exactly what changes are included in the the particular release..?
Any other ideas, concerns, points of interest? While I do have some flexibility of the process some of the requirements (like difference report or some way to easily view differences and having separate developer/deployer) are most likely untouchable.
Thank you so much for any help you can provide on this.
To keep track of different versions of the code and to help you manage very fast release cycles (daily) vs long term enhancements you can use branches in TFS.
There is a ton of information out there on branching, but in general I like to try to keep things simple. For example, have one branch called "release" and another "development". Everybody works on the development branch but the code to be deployed to production is merged into the release branch right before release.
This blog post describes the process:
http://team-foundation-server.blogspot.com/2008/01/how-we-branch-our-code-in-tfs.html
Well, based on my experience with VS2003 vs VS2010 for example is that the project structures are different and allowing VS to do a conversion often times results in a solution that either requires a lot of refactoring or is unusable. Having said that; if you can transition everything over to TFS2010 then one way to handle it is to setup different projects for each solution and use the TFS built in version handling for the different releases. You can also set up a build server and schedule nightly builds. If the build is ok then you can push this version into testing and ultimately production. You should really read up on TFS because it's totally different from VSS and is definitely a huge upgrade in allowing you to do team-focused development.
P.S. TFS has a really good Sharepoint integration which will help you and your team keep track of all the bugs and tasks.
I would like to hear the best practices or know how people perform the following task in TFS 2008.
I am intending on using TFS for building and storing web applications projects. Sometimes these projects can contain 100's of files (*.cs, *.acsx etc)
During the lifetime of the website, a small bug will get raised resulting in say a stylesheet change, and a change to default.aspx.cs for example.
On checking in these changes to TFS, and automated build would be triggered (great!), however for deploying the changes to the target production machine, I only need to deploy for example:
style.css
default.asx
MyWebApplications.dll
So my question is, can MSBuild be customized to generate a "code pack" of only the files which require deploying to the production server based on the changeset which cause the re-build?
You are probably going to have a hard time getting MSBuild itself to do this, but the ideal tool to use in your situation is the Web Deployment Tool, aka MSDeploy. With this tool you can tell it to deploy the changes to the target website. It will determine only the changed files and then just deploy those. Also you can perform customization to the deployment and a whole bunch of other stuff. It's a really great tool.
This seems like it should be obvious so maybe I am not understanding something. I have TFS setup. My web apps code is in source control. I can check-in/out and create builds but I don't quite understand what the recommended method for publishing the code to the production environment is? I was looking for something like "publish release to web server" but can't find anything. I think my larger problem is that I can't find a good work-flow diagram for how a team is supposed to move from editing code to testing and then to deploying to the production servers all within the TFS system. If anyone has any thoughts or links to good walkthroughs on publishing a website from within TFS that would be fantastic.
For the workflow I would suggest the code promotion model.
Within TFS you have the idea of build quality. SO when you initially check code in the build will kick off with a quality of say "ready for test" if the build passes the testers can decide when they want to take a build for testing, when they have tested a build they can mark it as "ready for deployment", you can then use that realeas to deploy.
Unless things have changed in TFS 2010 you have to roll your own implementation for deployment. We used MS build to build MSIs using Wix, then had the testers test the deployment process for us.
Have a look at the following msbuild extensions to help with deploying to test/live environments:
http://msbuildtasks.tigris.org/
http://msbuildextensionpack.codeplex.com/
I would really recommend you spend a little time on your build process and get it water tight, there is nothing more embarrassing that deploying a release to live that doesn't work and the client loosing faith over it.
I don't believe there is a built-in mechanism in Team Build to publish websites, however this can be achieved by overriding AfterDropBuild or AfterEndToEndIteration in your build definition to include necessary tasks for copying the build output to your web servers.