I want to be able to have one set of build/props files that will build as many EAR projects as I tell it to.
Each EAR project will have its own set of modules, WARs, JARS, etc.
I'm want to set in a props file what each EAR project consists of, and I want to tell the build each time which EAR(s) to build.
In general, what is a good way to structure the build/props files?
So far, I have one build.xml file and one props file.
In the props file, I have an entry for each EAR project, for example:
Foo=fooWar, foojar
Bar=barWar, barJar1, barJar2
Not sure if that's a good idea or not, and I'm thinking how to tell the build which projects to build.
Since it could be many projects, I'm thinking I would want another props entry like:
projectsToBuild=Foo, Bar
And then before each build, the build person would edit "projectsToBuild".
Is this a good road to go down?
If you're building multiple projects in one hit, then it sounds like you don't have any proper dependency management in place. Have a look at using Apache Ivy to publish and retrieve from a shared repository (e.g. Artifactory), so that you can decouple your project and build them independently.
Related
So, at work, I frequently have to create virtually identical ant scripts. Basically the application we provide to our clients is designed to be easily extensible, and we offer a service of designing and creating custom modules for it. Because of the complexity of our application, with lots of cross dependencies, I tend to develop the module within our core dev environment, compile it using IntelliJ, and then run a basic ant script that does the following tasks:
1) Clean build directory
2) Create build directory and directory hierarchy based on package paths.
3) Copy class files (and source files to a separate sources directory).
4) Jar it up.
The thing is, to do this I need to go through the script line by line and change a bunch of property names, so it works for the new use case. I also save all the scripts in case I need to go back to them.
This isn't the worst thing in the world, but I'm always looking for a better way to do things. Hence my idea:
For each specific implementation I would provide an ant script (or other file) of just properties. Key-value pairs, which would have specific prefixes for each key based on what it's used for. I would then want my ant script to run the various tasks, executing each one for the key-value pairs that are appropriate.
For example, copying the class files. I would have a property with a name like "classFile.filePath". I would want the script to call the task for every property it detects that starts with "classFile...".
Honestly, from my current research so far, I'm not confident that this is possible. But... I'm super stubborn, and always looking for new creative options. So, what options do I have? Or are there none?
It's possible to dynamically generate ANT scripts, for example the following does this using an XML input file:
Use pure Ant to search if list of files exists and take action based on condition
Personally I would always try and avoid this level of complexity. Ant is not a programming language.
Looking at what you're trying to achieve it does appear you could benefit from packaging your dependencies as jars and using a Maven repository manager like Nexus or Artifactory for storage. This would simplify each sub-project build. When building projects that depend on these published libraries you can use a dependency management tool like Apache ivy to download them.
Hope that helps your question is fairly broad.
I was wondering if anyone could help.We have the following project structure in our company :
Code/Common
Code/Project1
Code/Project2
etc...
When the Common Project builds, it has a PostBuild Event that copies all the relevant files into the Code/Common/Binaries folder. Then all the other Projects reference the Common components in this folder.
However, what we are struggling with is that when TFS Online checks-out the solution it does so to c:\a\src and the Common binaries are placed in c:\a\src\Binaries. Now, when the other projects (Project1 etc) do their build it cannot find the Common Assemblies, as not only are they removed, but the paths are different from what it expects them to be in c:\a\src\Common\Binaries instead of c:\a\src\Binaries.
Is there anyway to tell the build server to not delete those files in the "Binaries" directory and to specify the folder location to checkout to? Or how one one go about solving such a problem?
Thanks very much
A build server is a transient thing, you cannot rely on files to be there.
You need to either Create Nuget Packages for you common output and then consume these in your other projects (the 'proper' way), or you will need to check your dependencies into source control after each build so you can then reference them in subsequent builds (the 'really frowned apon' way).
Using TFS 2010 I need to build a solution that depends on a couple of other solutions held in different team projects. I'm editing my build definition and I create a list of "Projects to Build'. When it comes to build the parent solution I get an error because it's unable to reference assemblies created by the other solutions. So I go back to edit build definition and add /p:ReferencePath="c:\builds\3\referencedproject\binaries\" to the "MSBuild Arguments" Build process parameters.
Problem is, I don't want to hard code the c:\build\3. I guess there may be a $() property I can use in its place - can anyone please advise?
The other problem I have is that the 'Main' project and the two other projects that it references live at the same level in the source code. As far as I can tell, I have to set the source control folder to be the level above this - which happens to be the root. This means that TFS Build does a get of all the projects on the root - which includes dozens of projects that are not required for my build. It's not a critical issue since it makes no attempt to compile these non-related projects but it does increase the time for the build cycle to complete. Is this only way to avoid this to "group" the projects that are required for my build into a different TFS source folder?
For the references issue there are a few options. The most common one taking an explicit dependency on a specific version by checking in the binary to TFS.
For example, if you have Team Project A that has a dependency on Team Project B, I would assume they are setup as separate projects because they evolve differently, probably have different teams working on them, and have different release cycles. The common approach to managing this dependency is to checkin B.dll into Team Project A (usually in a lib folder specifically for this purpose), then use a file reference from within Project A's solution/projects to the dll in the lib folder.
This approach lets the Project A team explicitly choose which version of B.dll they wish to depend on, and make an explicit decision to adopt newer versions of B.dll on their own timetable.
For the other question of how to have a build definition download only select source code paths, you can specify multiple lines in the Workspace mapping screen when setting up a build definition. For example you could have the following:
$\ProjectA -> $(SourceDir)\ProjectA
$\ProjectB -> $(SourceDir)\ProjectB
This would download Project A + B but not C.
I'll start with problem 2 as this should be easiest to solve.
You have 2 options both involve changing the workspace mapping of your Build Definition.
You don't have to map at the folder "above", you can map individual folders so if your source looks like this.
$/TP/SolutionA
$/TP/Folder1/SolutionB
$/TP/Folder1/SolutionC
$/TP/Folder2/SolutionD
and you only want to include SolutionA and SolutionC in your build, you could set the workspace up as follows.
This will get just the code you need and preserve the relative paths between them.
Another option is to use cloaking, you map the "Root" folder and then cloak any folders you want the build to ignore.
Both of these methods will restrict the amount of source being downloaded when the build runs, and also prevent "continuous" builds from starting when checkins occur in the folders that haven't been mapped / Cloaked.
Problem number 1.
As Dylan suggests, probably the best thing to do is to use Binary References between solutions. Especially solutions in seperate team projects.
Check out my answer to this question for a full description.
I am considering switching a Maven project that I manage to Apache-Ant/Ivy. I need more control over the build process and am getting very frustrated with Maven. Please no comments about how great Maven is. My question is about Ivy.
I would like to set up a "standard" Ant build template that can later be used for other projects with minimal changes.
I will set up a central "enterprise" repository where we can place third-party libraries that are not available in the public Maven repositories (e.g. commercial libraries, Sun libraries, proprietary libraries, etc.). This enterprise repository will be available on our local LAN, but may not be available from outside the office.
Each developer will have a private repository in ~/.ivy/repository. I would like the Ant build to automatically update this private repository with changed versions of libraries from the enterprise repository.
In ~/.ivy/ant, I plan on placing "standard" modules for including in the individual project build.xml files, using the include task in Ant 1.8. These modules will provide things like Scala and Clojure compilation targets with different versions for different Scala and Clojure versions (e.g.: scala-compile-2.9.1.xml, clojure-compile-1.3.xml, etc.) The build modules will be available in the enterprise repository and should be updated automatically in the private repositories if they change.
Each project will follow a standard Maven directory structure: ${project}/src/main/java, ${project}/target/classes, etc.
In the past, I tried using Ivy but the Ant build files got to be very large (> 500 lines for the template build file) and hard to manage/edit. I am hoping that by putting standard targets in their own build modules in the ~/.ivy/ant directory, I can avoid that code bloat.
Can this be done? Am I way off base? The only documentation I can find on Ivy is at the Apache web site (http://ant.apache.org/ivy). Is there any other documentation available, including books?
Rather sensible idea about dividing template build file into includable helper files. Personally, now i'm switchin' a really large project from ant (no dependency managment at all - only copying files from ftp) to ant/ivy solution. So i've done this way - i have a file with milestones targets - i.e ready-to-compile, compiled, ready-to-archiving, archived - so on. I think u got the idea. I've configured dependencies of all this targets ( dependencies in terms of ant, do't get me wrong). In that way - compiled depends on ready-to-compile, ready-to-compilede depends on initialized - smth like this. This targets don't have body - they are for including in every build-file of every module of your multi-module project. The sole purpose of this targets for maintaining the STATE of build, because of this import stuff things become rather tricky and it's hard time to know what target was overriden, and when this target would be run. But with this file i can easily change state of vy build on every sensible milestone. I want in one module to compile help files with exteran exe. No problem - in this project i just do this - ready-to-archiving depends on the target for compiling help. And as this milestones targets are included - i can override only some of them - all others would presere the desired way of building project.
Another part of my strategy - mixins build files - for every specific area. So i have a file for ivy. There i put initializing, resolving, publishing and so on. When i want to use ivy - i just include this file and manage depdendencies through my milestones targets. If the build is typical - i only include this file and have a convention-over-configuration functionality. All out of the box. How?? Just combining with other mixins. Mixins may include other mixins to depend on them. So each mixin is a reusable part of my build strategy. The stuff from OOP - single-concerned unit. In your case it's scala mixin with targets specific for scala stuff.
Then i have delegate.xml that delegates child projects common build activities. I have dist, all, test and whatever u want for multimodule project. The build order is evaluated with ant-ivy task buildlist.
There also some other files - but this are the strategically basic files that helped me to have a reusable and maintanable build with this BIG and VERY Conservative project. So, if u are interested about details, don't be shy and contact me. I will be very pleased to help you, because ivy docs are really comlicated and incomplete.
EDIT: About books - Ant in Action may help you, i took several ideas from this book, and i really highly recommend it everyone to read. There u can find ivy stuff, also. And about ivy docs - sorry, it's all that is available. But when i was in trouble with this cumbersome ivy+ant - i found several interesting articles on private blogs. So ... that may fill the gap some way.
For those of you that use Ant with multiple projects, where do you put the build.xml files? Do you put one in each project, or do you put them in a separate project that contains all your Ant-related files?
The usual recommendation is to put a build.xml in each project. But this has a few drawbacks:
It makes it hard to reuse common targets in multiple projects.
Sometimes you want to use Ant to export a project from source control and deploy it. Obviously you can't do this if the build file is in the project itself.
But if you put them all in a common location:
People need to be aware of their location to use them; they can't just use "ant -find" to find the current project's file.
You can't have different build instructions for different branches of the project.
What do you guys do?
EDIT: Thanks for the good suggestions so far. As far Maven, these aren't Java projects, and I get the impression that Maven is only meant for Java.
Place the Ant files with the project. That is the de facto standard and recommended by the creator of Ant. I will try to address some of the issues you have brought up:
Reuse of common targets should be done using techniques as described by Eric Hatcher in his book Java Development with Ant. Basically, you extract all commonality into a some top level files that all other Ant files "inherit" from.
Using Ant to export a project from source control seems odd to me, but if you want to do this, use a different Ant file :-) You can make a target like ant export -Dproject=foo/bar.
For Ant, I recommend you grab that book - it has a ton of helpful techniques.
The real recommendation I would make though is to drop Ant and convert over to Maven - like the Apache Software Foundation did (they maintain both Ant and Maven).
If you're working with independent projects, you can:
put your build.xml at the top level
place common Ant definitions (Antlib) into another project (e.g. config)
use svn:externals to import the common Antlib definition (from 'config') into your project
EDIT The trick with svn:externals is that if you link to the HEAD of some common files, it may happen that they will change after a couple of months/years. So each time you tag, you should change the svn:externals to point to a fix version of the included project. This may come handy when a project has to be rebuild years after it was last built.
My rule of thumb is to put the build.xml file in the directory under which all files are referenced. In other words, no relative paths should start with "../". Where I live, that usually means putting it in the "trunk" directory, which has src, lib, build, docs, etc underneath it.
Doing this makes the paths much cleaner in the file, and it makes it obvious how to build the project.
Where I have multiple projects that need to build, I will create a separate build.xml for each project, and a central build.xml in the directory all the project are in that calls those other build.xml files. That gives you a lot of flexibility with very little work.
I'd expect an Ant build file to be located at the top of a project (it's already a pain to have to look at a the build file to "discover" how to build the project, so if I have to locate it first, it'll drive me totally crazy). Now, regarding all the drawbacks you mentioned, I'm tempted to say: why don't you use Maven?
The way I have done this is in the past (Now I just use Maven):
Have a build.xml in the root of each project
Create an overarching build.xml
for all projects and place it in
the trunk of my repository
The overarching buid.xml has
checkout tasks for each project.
I am guessing when you mentioned
export from repository, you
actually meant import.
The overarching build file also
defines the dependencies, if any
You may update individual projects using each project's individual build file
If you do have common tasks defined, you may inherit from a common build file as well as someone else suggested.
Looks like your set of projects might be a good candidate for migration to Maven, I realize it is not always possible but if you have time, you might want to look into it.