I am setting up the build system for a team that produces APIs used on several platforms and architectures. There has been a lot of work already spent on setting up Ant to build all of the Java code, so I would prefer to stick with Ant if possible.
Where I am stumped is how to build the C++ software. Here are the platforms and languages I need to support:
Java - Linux - 32bit & 64bit: Ant
Java - Windows - 32bit & 64bit: Ant
C++ - Linux - 32bit & 64bit: Ant w/CppTasks (question #1)
C++ - Windows - 32bit: (question #2)
Note: C++ on Windows is MS Visual Studio C++ projects.
I think the answer to question #1 is CppTasks because that seems to be the standard way to build C++ from Ant.
For question #2, I could also use CppTasks, but the developers will be compiling in Visual Studio, so it seems beneficial to use their Visual Studio project for building, which means calling MSBuild from Ant.
Has anyone tried this before and has a good solution for building Java & C++ on both Linux and Windows?
Do you use a Continuous Build System like Jenkins?
With Jenkins, your builds can be automatically triggered by check in/commit, time of day, and/or on command. The great thing about Jenkins is that you can have it automatically build all of the various versions of your software.
For example, you'll probably need to run make on Linux C++ but use msbuild on Windows systems, and you'll need to trigger a build on a Linux machine and one for a Windows machine. Jenkins can be setup to do this automatically. When a commit happens, all your various builds on all of your systems can be triggered at once. Then, you can store the builds you need on Jenkins and have your users actually pull the type they need off the project they need.
There are so many ways this could be setup, but the easiest is to simply create four separate jobs (One for Java 32bit, Java 64bit, C++ Linux, and C++ Microsoft). You don't necessarily need a separate Microsoft Java build (at least in theory), but there's nothing stopping you.
You can have a single Jenkins server run "slave" jobs on other build systems, so you could have Jenkins live on the 64Bit Linux system, but use a 32bit Linux system as a slave to do the 32bit build, and call a Windows slave to do the Visual Basic build. That way, all of your jobs are located in a central place, but you can use the environments you want.
If you've never used a Continuous Build system, download Jenkins and play around with it. It's free and open source, and very, very easy to use. You can run it on any machine that has a JDK or JRE 1.6. If you download the Windows version, it even comes with the JRE already built in.
Your best bet is to use a continuous build system and allow it to handle the mess. By the way, there's also Bamboo, CruiseControl, and Hudson (which was split from Jenkins a few months ago)
TeamCity should fit the bill very well. It supports Ant and MSBuild natively and has a pretty good cross plartform story (written in Java but excellent integration with e.g. Win).
Dont see any benefit in wrapping you Win MSBuild-based builds in yet another build system.
The list for this looks a little bit different (in my opinion)
Java -Maven for all platforms
C++ - Maybe Maven as well (Check http://duns.github.com/maven-nar-plugin/).
Related
Since one can have a nice Docker container to run an entire build in, it would be fantastic if the tools used by the container to build and run the code would be accessible to the host.
Imagine the following use-case:
Imagine that you're developing a Java application using OpenJDK 12 and Maven 3.6.1 in order to build, run all tests and package the entire application into an executable .jar file.
You create a Docker container that serves as a "build container". This container has OpenJDK 12 and Maven 3.6.1 installed and can be used to build and package your entire application (you could use it locally, during development and you could also use it on a build-server, triggering the build whenever code changes are pushed).
Now, you actually want to start writing some code... naturally, you'll go ahead and open your project in your favorite IDE (IntelliJ IDEA?), configure the project SDK and whatever else needs to be configured and start rocking!
Would it not be fantastic to be able to tell IntelliJ (Eclipse, NetBeans, VSCode, whatever...) to simply use the same tools with the same versions as the build container is using? Sure, you could tell your IDE to delegate building to the "build container" (Docker), but without setting the appropriate "Project SDK" (and other configs), then you'd be "coding in the dark"... you'd be losing out on almost all the benefits of using a powerful IDE for development. No code hinting, no static code analysis, etc. etc. etc. Your cool and fancy IDE is in essence reduced to a simple text editor that can at least trigger a full-build by calling your build container.
In order to keep benefiting from the many IDE features, you'll need to install OpenJDK 12, Maven 3.6.1 and whatever else you need (in essence, the same tools you have already spent time configuring your Docker image with) and then tell the IDE that "these" are the tool it should use for "this" project.
It's unfortunately too easy to accidentally install the wrong version of the tool on your host (locally), that could potentially lead to the "it works on my machine" syndrome. Sure, you'd still spot problems later down the road once the project is built using the appropriate tools and versions by the build container/server, but... not to mention how annoying things can become when having to maintain an entire zoo of tools an their versions on your machine (+ potentially having to deal with all kind of funky incompatibilities or interactions between all the tools) when you happen to work on multiple projects (one project needs JDK 8, the other JDK 11, the other uses Gradle, not Maven, then you also need Node 10, Angular 5, but also 6, etc. etc. etc.).
So far, I only came across all kind of funky workarounds, but no "nice" solution. The most tolerable I found so far is to manually expose (copy) the tools from the container on the host machine (e.g.: define a volume shared by both and then execute a manual script that would not copy the tools from the container into the shared volume directory so that the host can access them as well)... while this would work, it unfortunately involves a manual step, which means that whenever the container is updated (e.g.: new versions of certain tools are used or even additional, completely new ones) then the developer needs to remember to perform the manual copying step (execute whatever script explicitly) in order have all the latest and greatest stuff available to the host once again (of course, this could potentially mean updating IDE configs as - but this - version upgrades at least - can be mitigated to a large degree by having the tools reside at non-version specific paths).
Does anyone have some idea how to achieve this? VM's are out of the question and would seem like an overkill... I don't see why accessing Docker container resources in a read-only fashion should not be possible and reuse and reference appropriate tooling during both development and build.
We have a Jenkins server running on a Linux box and we are using it for Java projects (Maven builds).
We shall have a couple of .NET applications coming up. Shall I be able to build them on the Linux box? Do I need to add a windows box? Any help will be appreciated.
If these are .NET Framework (and not .NET Core) applications, the easiest way to go about it is to add a Windows box with everything you need for building your .NET apps (e.g. install .NET Build tools), etc. and then setup a Jenkins slave on that box.
Then on your builds for .NET, you specify a requirement to use the slave on the Windows box to build it.
N.B. Keep the Windows build node as clean as possible - i.e. Resist the temptation and do not install the full Visual Studio on it (!). Only install the SDKs you need.
I am working with a team that develops a Java application using the following well-established toolchain for automated building, testing and continuous integration:
OS: Ubuntu
IDE: Eclipse
Build tools: Ant
Testing framework: JUnit
Source control: Subversion
CI server: Jenkins
A typical Jenkins job will grab the Java source from Subversion, and run Ant targets to build the code, run automated tests and create deployment artifacts.
We are now considering writing a .Net plugin for Windows clients to access our application's API from MS Excel. We will probably write it using either C# or F# - it's early days, and we haven't settled on a language yet, but F# seems as if it may offer some benefits in terms of being able to express API actions using a combinator-based DSL.
We would like to do as much of this work as possible on Linux, using Mono, and to use our existing CI infrastructure to build and test our software.
My first impression is that the toolchain will look something like this:
OS: Linux
IDE: Monodevelop / VIM (Eclipse support for Mono, and especially for F#, seems lacking)
Build Tool: NAnt
Testing framework: NUnit
Source control: Subversion
CI server: Jenkins, with NAnt plugin
Does anyone have any experience of developing with this kind of toolchain? The two questions I would like answered are:
What are the main pitfalls in this approach for developers used to the Java ecosystem?
Are there better alternatives to NAnt and NUnit for building and running automated tests, especially for F#?
Using F# with MonoDevelop on Linux seems like the way to go if you're developing cross-platform or server-side applications that can be developed on Linux (and occasionaly tested on Mac/Win).
However, I don't think that you'll be able to develop Excel plugin on Linux if you're targeting Windows users. You'll definitely need to run Excel on Windows and to test things, you'll probably also need to do (a part of) the development on Windows (I suppose the integration is the tricky bit - though you could develop & test some core functionality on Linux).
On Windows you can use free Visual Studio Shell with F#. The MonoDevelop integration for F# is (hopefuly) quite good, but Visual Studio gives you probably better experience and you'll need to use Windows for quite a few tasks anyway...
What are the main pitfalls in this approach for developers used to the Java ecosystem?
You are using a Linux operating system to develop an extension for Excel, which is primarily Windows. The Mono Platform is pretty excellent - but you could run into bugs - either with C# or F#. It's not as big of an issue as it was a few years ago, but worth considering. If you do decide to stick on the Mono / Linux platform - MonoDevelop is the way to go.
Are there better alternatives to NAnt and NUnit for building and running automated tests, especially for F#?
Take a look at FsUnit if you plan on going with F#. It has some nice syntax assertions, etc. It's complimentary to NUnit, so you won't be in uncharted waters.
Are there any CI-Systems for Delphi like Hudson for Java?
Does Hudson has any Delphi integrations?
We use Hudson :), which works just fine with Delphi.
Here's a complete setup for one of my projects:
<?xml version='1.0' encoding='UTF-8'?>
<project>
<actions/>
<description></description>
<keepDependencies>false</keepDependencies>
<properties/>
<scm class="hudson.scm.CVSSCM">
<cvsroot>:sspi:cvsonly:foo#bar:/baz</cvsroot>
<module>SIP</module>
<canUseUpdate>false</canUseUpdate>
<flatten>true</flatten>
<isTag>false</isTag>
<excludedRegions></excludedRegions>
</scm>
<canRoam>true</canRoam>
<disabled>false</disabled>
<triggers class="vector"/>
<builders>
<hudson.tasks.BatchFile>
<command>"C:\Program Files\Nant\bin\nant" -buildfile:etc\sip.build build-d7 test</command>
</hudson.tasks.BatchFile>
</builders>
<publishers>
<hudson.tasks.BuildTrigger>
<childProjects>quux, foozle, wibble</childProjects>
<threshold>
<name>SUCCESS</name>
<ordinal>0</ordinal>
<color>BLUE</color>
</threshold>
</hudson.tasks.BuildTrigger>
</publishers>
<buildWrappers/>
</project>
sip.build's delphi-7 target builds the project's test suite (DUnit's TextTestRunner) and so the output of the job is simply the count of failed/error tests thanks to:
var
R: TTestResult;
begin
R := TextTestRunner.RunRegisteredTests;
ExitCode := R.ErrorCount + R.FailureCount;
end;
FinalBuilder Server
now replaced with :
Continua CI
We use CruiseControl.NET, which works perfectly with Delphi.
It seems Embarcadero itself uses Hudson. See the comment from Nick Hodges, Development Manager at Embarcadero. http://blogs.embarcadero.com/nickhodges/2010/03/10/39369#comment-29952
Hudson is great, it is also available as MSI setup. Users of older Delphi versions (pre MSBuild), can create build scripts with the popular Apache Ant and NAnt build script environments. Simple command line (batch script) builds are possible too of course.
One of the highlights of this
installer is that it comes with a JRE,
to be fully self-contained. This is
because Windows users don't normally
know what to do with the *.war file,
and they generally don't like going to
the command prompt and running Java
command manually.
So it is easy to try without going through too many installers and command lines first.
And after installation make sure to visit the plug in page in the administration app, there are many extensions available which are also useful for Delphi developers. A plugin overview is online here.
Hudson tools for Delphi: there is also a Code Coverage tool for Delphi with Hudson support here.
TeamCity is another very nice CI system. It is a commercial product, but offers a free license for up to 20 users and 20 build configurations on 3 build agents.
Bamboo is a CI system for any language. It is available from Atlassian. Its price starts at $10 - this includes all features and unlimited users, but limited to 10 plans, running builds only on the server where it is installed. It can use Amazon Elastic Compute Cloud instances as remote agents.
Automated Build Studio from SmartBear. With Automated Build Studio Server, it introduced CI some time before FinalBuilder.
We use jenkins.
Jenkins has a RAD Studio plugin to integrate the compilation of Delphi projects into jobs. It's not mandatory to use it (plugin), we use command-line tools and MSBuild to compile the projects.
The project DUnItX (to create unit testing) integrates seamlessly with Jenkins.
Here you can find some articles about the integration of Jenkins and Delphi. Introduction, installation,... Are in Spanish but you can translate without problems and with included images and code are quite simple.
1- Integración continua con Delphi (GIT) (INTRODUCTION)
2- Integración continua con Delphi (Jenkins) – Instalación (INSTALLATION)
3- Integración continua con Delphi (Jenkins) – Primer proyecto (FIRST PROJECT)
4- Integración continua con Delphi (Jenkins) – Pipelines (USE OF PIPELINES)
Add our Parabuild to the list of tools to check out - it builds practically anything that can be built from the command line - that includes Delphi, and it provides integration with Perforce and other version control systems such as Git and Bazaar.
Hudson is simple to use, with some web-based assistants, as Delphi projects are built with MSBuild.
If you need more customization and 3rd party tools integration (integrate metrics tools for example) then CruiseControl.NET is the ideal solution.
Have a look to my CC.NET Dashboard page for DelphiCodeToDoc (and you can have a look to the configuration file also). I'm doing checkout + compilation of gui and command line version and unit test + unit test execution and report + ftp upload build delivery.
Just to throw my support for CruiseControl.NET, as Delphi 2007 and above are based around MSBuild, then there should be no problem with builds via CC.NET. There is a patch somewhere for DUNit, so that it outputs the results to an XML file, and this can then be integrated into the builds results charts.
errrm, can you define what you are actually looking for?
You use the phrase continuous integration, but do you just mean continuous build to make sure it compiles and likes, or do you actually mean continuous integration in the sense that you also run regression tests?
I would be curios as to how you do that (Auto-it?).
Not that it really matters too much, because the wonderful Hudson (which is decidedly not just for java) has numerous plug-ins, but will also allow you to execute a DOS batch script - any script that you care to write, to build, to test, to report results, etc.
Another option (although not free, $10 for limited # of users) is Atlassian Bamboo. And JetBrains TeamCity is another that is free (with limited # of users). I've tried both with Delphi 2009/2010, and they work quite well. Both allowed integration with Perforce among other SCM's.
We use Jenkins CI system with Owly CI tool as the build system and dependency handler.
Take a look on this tutorial
Initial setup takes less than 5 minutes
Are there any tools/ ant tasks that could be integrated into the ant build file to build Symbian applications? Similar to what we have for Android (AndroidAnt), BlackBerry (BB Ant Tools), .NET (Nant)
There isn't any officially supported stuff as far as I know, but (having done it myself) I know it is perfectly possible to build Symbian apps using Ant by simply calling the command line tools from ant commands.
If this is a precursor to automatically building symbian apps using a build server of some description, remember that Symbian builds have an annoying habit of polluting the build environment so you will need to invest some time into automated environment cleaning and environment setup.
This article is a good starting point: Continuous integration on Symbian OS
However, it doesn't cover the 'polluting the build environment' problem, which is an important one. In-house, we have fully automated Symbian builds using ANT and Cruisecontrol. We get around this problem by deleting/expanding the entire epoc32 tree with each build. This approach is fool-proof, but makes the builds very time consuming, which kills some of the benefits of continuous integration....
Actually Helium is the name of a Ant based build framework available through the Symbian foundation (http://developer.symbian.org/wiki/index.php/Helium).
It contains Ant build scripts and Symbian dedicated tasks (but not only, there are also general features...).
Developer.symbian.com and symbian.org links doesn't work anymore.
But mentioned continuous integration tutorial is available inside Symbianosunit package by Penrillian