I am reading lots of articles here about the subject and try to find a nice set of tools to integrate and create a Complete CI environment.
Among those articles I particularly liked this one: Continuum as a Jenkins replacement?
I have installed here, but not yet configured because we are doing some testes with GIT and Jenkins. We have Nexus working fine.
Now I need to have some recommendations to cover the following topics:
Code Coverage - JaCoCo, Cobertura, Sonar?
Code Quality Analysis
PMD, Sonar? ANT, MAVEN, GRADLE?
Maybe I am mixing some concepts here regarding the use of some tools, like SonarQube, if that is the case I am sorry.
So, I am open to "hear" about this. Thanks
Ant, Maven and Gradle are build tools. You pick one that suites your team. Gradle/Maven may directly work with artifact repos. Ant will require help from Ivy.
Coverage: JaCoCo supports newer Java(7, 8).. Not so much luck with Cobertura
Sonar: If you can integrate your code with SonarQube that takes care of bit Code Quality Metrics (like Duplicate detection, Findbug , Security ..etc)
On Jenkins
You can integrate Jenkins with all of the above tools. You can create Jenkins jobs for practically everything and have tons of plugins to make even simpler. There are more plugins to make Jenkins' based CD easier: Build Flow and workflow plugin.
Related
In my project we are using QC to execute our test cases(QTP), moving forward we would be eliminating QC (for cost reasons).
As far as I explored MSBuild & Jenkins, they would be suitable.
But MSBuild will trigger the execution when a new build pushed to the repository. Also it will automatically test on the latest build.
Is there any other CI tool available to execute test cases through QTP?
I will be executing automation once in a release. Also we install our application by manual since it requires lots of configuration.
Take a look at HP Application Automation Tools.
This plugin basically replaces the need for QC, and is developed by HP.
Create a Jenkins job using this plugin on the same Jenkins installation used to build your code, then you can configure your job to run your tests as soon the code is available (e.g. on a nightly basis).
See here for a helpful guide on how to implement a simple Jenkins job using this plugin.
They also host the code on Github, which is very useful if you need to change the behavior of the plugin to suit your needs.
I'm trying to convince my Configuration Manager to use Jenkins with RTC (witch we already use for Source Control, manual builds and many other things), as my boss asked.
I'd like to know why to use the Jenkins with RTC. I know RTC can do Continuous Integration, but what Jenkins can add to it?
RTC doesn't have a build engine in itself. It has a build engine toolkit which abstract the actual build engine (BuildForge, Hudson, Jenkins, ...)
If you want to do continuous integration from RTC, you need a build engine.
The official one is IBM BuildForge (not free).
Since RTC4, Jenkins (free) is also officially supported, with an RTC plugin for Jenkins to install on the Jenkins side (as well as the Build System Toolkit in the RTC downloads)
What Jenkins can add is:
a greater familiarity with the tool
a great list of plugins
the possibility to experiment locally (even without RTC)
a good connectivity with other source control tools (SVN, Git) in addition of Jazz RTC.
I want to integrate some mutation testing to ensure the quality of my junit tests. I want to have the results in the sonar dashboard of my project.
The sonar pitest plugin seems to do what I want, but there are some issues with maven 3 and it is still under development.
Has anyone ever tried this plugin ?
Are there others alternatives ?
I am fairly sure there are no other mutation testing plugins for sonar.
I've not actually tried the pit sonar plugin myself, but I understand the issue with maven 3 can be worked around
see
https://groups.google.com/forum/?fromgroups=#!topic/pitusers/bkDD13iBtHU
The PIT maven plugin itself is quite mature and is being used successfully in production. If you encounter issues with the sonar plugin you could just publish the maven reports from your CI server.
Please do feedback any issues you hit - I'm sure Alexandre will be kean to fix them.
Does anyone have specific experiences with using TFS 2008/2010 AND Jenkins for Continuous Integration (CI)? We are trying to decide which CI server to use. Our team works exclusively in Microsoft .NET/Visual Studio 2010/C#. We have the following requirements:
Automatically build our web project on every checkin.
Run unit tests with each build.
Automatically deploy green builds to development and/or test environments.
Provide pretty reports.
Provide build/deployment notifications via email.
I realize that installing a tool won't necessarily give us this functionality out-of-the-box and that we will have to integrate with other tools like MSBuild to achieve this.
I'm looking for specific features that Jenkins has that TFS 2008/2010 does not or vice versa. Also which is easier to maintain, use, etc.
I would highly recommend using Jenkins - it will do all of your requirements out of the box apart from possibly #3, but if you can script your deployments then it can do that as well.
Here are some links to help you get your builds up and running:
Blog about doing .NET builds in Jenkins
Jenkins Windows installers
Installing the Jenkins master and slaves as Windows services
Disclaimer: I have no experience with TFS, but I think open solutions are nearly always more flexible and extensible (and cheaper !) than proprietary products.
Late to this game, but I have used both TFS 2010 and Jenkins for CI. TFS 2010 has minimum set of CI tools in it. However, when you want to create a CI pipeline, it's a completely different story while Jenkins can easily create the pipeline.
If you are looking at only CI for one build either one should work. However, when it comes to the entire pipeline, Jenkins is way to go. With TFS it can be done, but Jenkins is better choice.
Here's quick bullet points:
TFS:
With a build definition you can compile, execute tests, return changeset/workitems, send an email when a build is broken
natural integration with visual studio
extremely hard to create CI pipeline. Requires custom handler and extensive workflow work. Not as intuitive as creating a build definition.
Because of the 3rd bullet, it's not easy to maintain/customize/scale CI pipeline
Jenkins:
Need to create a msbuild config file for CI, which is not much pain comparing to creating CI pipeline using TFS. However, TFS gives better/easier tool to create a build definition. however, it is not bad creating config file for msbuild for a project.
Creating a CI pipeline is very easy. Just chain them using upstream/downstream jenkins job trigger and passing an artifact from previous job.
Since Jenkins is very flexible, it is easy to create a jenkins plugin to meet your own needs and provide it to opensource community :)
In summary, if you need complete automated build, test, and deployment system go with Jenkins. If you just need only build and test, TFS might give you an edge over Jenkins.
If you're using Team 2010-2012, there is no reason whatsoever to bring in Jenkins. Team has all the features you listed, and the build process is ridiculously flexible.
Note that if you are stuck on Team 2008 or earlier, you should seriously look at Jenkins -- 2008 and earlier are quite primitive and inflexible compared to 2010 or later.
Are there any CI-Systems for Delphi like Hudson for Java?
Does Hudson has any Delphi integrations?
We use Hudson :), which works just fine with Delphi.
Here's a complete setup for one of my projects:
<?xml version='1.0' encoding='UTF-8'?>
<project>
<actions/>
<description></description>
<keepDependencies>false</keepDependencies>
<properties/>
<scm class="hudson.scm.CVSSCM">
<cvsroot>:sspi:cvsonly:foo#bar:/baz</cvsroot>
<module>SIP</module>
<canUseUpdate>false</canUseUpdate>
<flatten>true</flatten>
<isTag>false</isTag>
<excludedRegions></excludedRegions>
</scm>
<canRoam>true</canRoam>
<disabled>false</disabled>
<triggers class="vector"/>
<builders>
<hudson.tasks.BatchFile>
<command>"C:\Program Files\Nant\bin\nant" -buildfile:etc\sip.build build-d7 test</command>
</hudson.tasks.BatchFile>
</builders>
<publishers>
<hudson.tasks.BuildTrigger>
<childProjects>quux, foozle, wibble</childProjects>
<threshold>
<name>SUCCESS</name>
<ordinal>0</ordinal>
<color>BLUE</color>
</threshold>
</hudson.tasks.BuildTrigger>
</publishers>
<buildWrappers/>
</project>
sip.build's delphi-7 target builds the project's test suite (DUnit's TextTestRunner) and so the output of the job is simply the count of failed/error tests thanks to:
var
R: TTestResult;
begin
R := TextTestRunner.RunRegisteredTests;
ExitCode := R.ErrorCount + R.FailureCount;
end;
FinalBuilder Server
now replaced with :
Continua CI
We use CruiseControl.NET, which works perfectly with Delphi.
It seems Embarcadero itself uses Hudson. See the comment from Nick Hodges, Development Manager at Embarcadero. http://blogs.embarcadero.com/nickhodges/2010/03/10/39369#comment-29952
Hudson is great, it is also available as MSI setup. Users of older Delphi versions (pre MSBuild), can create build scripts with the popular Apache Ant and NAnt build script environments. Simple command line (batch script) builds are possible too of course.
One of the highlights of this
installer is that it comes with a JRE,
to be fully self-contained. This is
because Windows users don't normally
know what to do with the *.war file,
and they generally don't like going to
the command prompt and running Java
command manually.
So it is easy to try without going through too many installers and command lines first.
And after installation make sure to visit the plug in page in the administration app, there are many extensions available which are also useful for Delphi developers. A plugin overview is online here.
Hudson tools for Delphi: there is also a Code Coverage tool for Delphi with Hudson support here.
TeamCity is another very nice CI system. It is a commercial product, but offers a free license for up to 20 users and 20 build configurations on 3 build agents.
Bamboo is a CI system for any language. It is available from Atlassian. Its price starts at $10 - this includes all features and unlimited users, but limited to 10 plans, running builds only on the server where it is installed. It can use Amazon Elastic Compute Cloud instances as remote agents.
Automated Build Studio from SmartBear. With Automated Build Studio Server, it introduced CI some time before FinalBuilder.
We use jenkins.
Jenkins has a RAD Studio plugin to integrate the compilation of Delphi projects into jobs. It's not mandatory to use it (plugin), we use command-line tools and MSBuild to compile the projects.
The project DUnItX (to create unit testing) integrates seamlessly with Jenkins.
Here you can find some articles about the integration of Jenkins and Delphi. Introduction, installation,... Are in Spanish but you can translate without problems and with included images and code are quite simple.
1- Integración continua con Delphi (GIT) (INTRODUCTION)
2- Integración continua con Delphi (Jenkins) – Instalación (INSTALLATION)
3- Integración continua con Delphi (Jenkins) – Primer proyecto (FIRST PROJECT)
4- Integración continua con Delphi (Jenkins) – Pipelines (USE OF PIPELINES)
Add our Parabuild to the list of tools to check out - it builds practically anything that can be built from the command line - that includes Delphi, and it provides integration with Perforce and other version control systems such as Git and Bazaar.
Hudson is simple to use, with some web-based assistants, as Delphi projects are built with MSBuild.
If you need more customization and 3rd party tools integration (integrate metrics tools for example) then CruiseControl.NET is the ideal solution.
Have a look to my CC.NET Dashboard page for DelphiCodeToDoc (and you can have a look to the configuration file also). I'm doing checkout + compilation of gui and command line version and unit test + unit test execution and report + ftp upload build delivery.
Just to throw my support for CruiseControl.NET, as Delphi 2007 and above are based around MSBuild, then there should be no problem with builds via CC.NET. There is a patch somewhere for DUNit, so that it outputs the results to an XML file, and this can then be integrated into the builds results charts.
errrm, can you define what you are actually looking for?
You use the phrase continuous integration, but do you just mean continuous build to make sure it compiles and likes, or do you actually mean continuous integration in the sense that you also run regression tests?
I would be curios as to how you do that (Auto-it?).
Not that it really matters too much, because the wonderful Hudson (which is decidedly not just for java) has numerous plug-ins, but will also allow you to execute a DOS batch script - any script that you care to write, to build, to test, to report results, etc.
Another option (although not free, $10 for limited # of users) is Atlassian Bamboo. And JetBrains TeamCity is another that is free (with limited # of users). I've tried both with Delphi 2009/2010, and they work quite well. Both allowed integration with Perforce among other SCM's.
We use Jenkins CI system with Owly CI tool as the build system and dependency handler.
Take a look on this tutorial
Initial setup takes less than 5 minutes