How can I best share Ant targets between projects? - ant

Is there a well-established way to share Ant targets between projects? I have a solution currently, but it's a bit inelegant. Here's what I'm doing so far.
I've got a file called ivy-tasks.xml hosted on a server on our network. This file contains, among other targets, boilerplate tasks for managing project dependencies with Ivy. For example:
<project name="ant-ivy-tasks" default="init-ivy"
xmlns:ivy="antlib:org.apache.ivy.ant">
...
<target name="ivy-download" unless="skip.ivy.download">
<mkdir dir="${ivy.jar.dir}"/>
<echo message="Installing ivy..."/>
<get src="http://repo1.maven.org/maven2/org/apache/ivy/ivy/${ivy.install.version}/ivy-${ivy.install.version}.jar"
dest="${ivy.jar.file}" usetimestamp="true"/>
</target>
<target name="ivy-init" depends="ivy-download"
description="-> Defines ivy tasks and loads global settings">
<path id="ivy.lib.path">
<fileset dir="${ivy.jar.dir}" includes="*.jar"/>
</path>
<taskdef resource="org/apache/ivy/ant/antlib.xml"
uri="antlib:org.apache.ivy.ant"
classpathref="ivy.lib.path"/>
<ivy:settings url="http://myserver/ivy/settings/ivysettings-user.xml"/>
</target>
...
</project>
The reason this file is hosted is because I don't want to:
Check the file into every project that needs it - this will result in duplication, making maintaining the targets harder.
Have my build.xml depend on checking out a project from source control - this will make the build have more XML at the top-level just to access the file.
What I do with this file in my projects' build.xmls is along the lines of:
<property name="download.dir" location="download"/>
<mkdir dir="${download.dir}"/>
<echo message="Downloading import files to ${download.dir}"/>
<get src="http://myserver/ivy/ivy-tasks.xml" dest="${download.dir}/ivy-tasks.xml" usetimestamp="true"/>
<import file="${download.dir}/ivy-tasks.xml"/>
The "dirty" part about this is that I have to do the above steps outside of a target, because the import task must be at the top-level. Plus, I still have to include this XML in all of the build.xml files that need it (i.e. there's still some amount of duplication).
On top of that, there might be additional situations where I might have common (non-Ivy) tasks that I'd like imported. If I were to provide these tasks using Ivy's dependency management I'd still have problems, since by the time I'd have resolved the dependencies I would have to be inside of a target in my build.xml, and unable to import (due to the constraint mentioned above).
Is there a better solution for what I'm trying to accomplish?

If you are using ANT 1.8+, then you could just import the build.xml directly from the hosted location.
http://ant.apache.org/manual/Tasks/import.html
Since Ant 1.8.0 the task can also
import resources from URLs or
classpath resources (which are URLs,
really). If you need to know whether
the current build file's source has
been a file or an URL you can consult
the property ant.file.type.projectname
(using the same example as above
ant.file.type.builddocs) which either
have the value "file" or "url".
<!-- importing.xml -->
<project name="importing" basedir="." default="...">
<import file="http://myserver/ivy/ivy-tasks.xml"/>
</project>

If you use Antlibs you can package them all inside a JAR file. Then simply copy this file into the ${ANT_HOME}/lib directory to use them.

After some additional searching, a possible solution would be to use SVN externals to check out specific required files that may be needed by the build.xml.
However, this would only work for users who are using Subversion as source control. It would still be nice to have a SCM-agnostic solution for users who aren't using Subversion, or another SCM that supports similar functionality.

What we've done is to create a project called 'bootstrap' which contains the various xml-files needed for the other projects at our office.
So to set up your development environment you run build.xml in bootstrap which copies the xml-files (like your ivy-stuff, and other targets) to a known location, and then your build files include these like this:
<import file="${ant.bootstrap.dir}/ant-commons.xml" />
<import file="${ant.bootstrap.dir}/ant-commons-ear.xml" />
Our bootstrap build.xml contains this:
<target name="install">
<fail unless="ant.bootstrap.dir" message="ant.bootstrap.dir ${missing.property.message}"/>
<copy todir = "${ant.bootstrap.dir}">
<fileset dir = "src/xml"/>
</copy>
</target>

Related

ivy jar located in my dep lib

how can I tell ant to find Ivy's jar in my own lib? ant just kept looking at it's home folder even when I've explicitly told it to find the jar somewhere else.
I would recommend removing the ivy jar from the ANT home directory. (For some very odd reason it's not normally packaged with ANT).
Instead I recommend including a special task to ensure ivy is installed.
<available classname="org.apache.ivy.Main" property="ivy.installed"/>
<target name="install-ivy" description="Install ivy" unless="ivy.installed">
<mkdir dir="${user.home}/.ant/lib"/>
<get dest="${user.home}/.ant/lib/ivy.jar" src="http://search.maven.org/remotecontent?filepath=org/apache/ivy/ivy/2.4.0/ivy-2.4.0.jar"/>
<fail message="Ivy has been installed. Run the build again"/>
</target>
Analysis
The ANT manual outlines the order in which jars a loaded by ANT at startup.
-lib jars in the order specified by the -lib elements on the command line
jars from ${user.home}/.ant/lib (unless -nouserlib is set)
jars from ANT_HOME/lib
This will always happen and unfortunately it won't matter what you do inside your build file.....
Jars in the ANT_HOME/lib
In my opinion, putting jars in the ANT_HOME effectively creates a bespoke installation of ANT. It makes your projects less portable across machines, and the customizations are frequently forgotten and undocumented.
So if you have control over the build server I would recommend removing any ANT tasks your find here.
Jars in the ${user.home}/.ant/lib
Placing jars here is less objectionable for the following reasons
Directory owned by the user running the build
Can be ignored at run-time by by using the commandline option -nouserlib
The only jar I put here is ivy... All other jars exist in the ivy cache (including ANT tasks)
You can place Ivy binaries in some folder inside you project folder. For example, in my case, it's etc/build/. I put where ivy.jar and jsch.jar.
After that provide the correct namespace in project defenfition and load Ivy.
<project name="somename" basedir="." xmlns:ivy="antlib:org.apache.ivy.ant">
<target name="ivy-load">
<path id="ivy.lib.path">
<pathelement location="${basedir}/etc/build/ivy.jar"/>
<pathelement location="${basedir}/etc/build/jsch.jar"/>
</path>
<taskdef resource="org/apache/ivy/ant/antlib.xml" uri="antlib:org.apache.ivy.ant" classpathref="ivy.lib.path"/>
</target>
<target name="ivy-init" depends="ivy-load">
<ivy:settings file="${basedir}/etc/ivysettings/ivysettings.xml"/>
<ivy:resolve conf="${ivy.conf}"/>
</target>
...
</project>

Using "antcall" in included ant files

I have a shared ant script b.ant which internally use antcall. It calculates a property that the client scripts use. I use include instead of import client scripts to avoid unintentional overwriting of targets, but this gives me a problem with the antcall.
When using include all targets in b are prefixes, and depends attributes in b are updated accordingly. This is however not true for antcall. Is there a there are way to handle this, i.e. make antcall always call the "local" ant target?
I can workaround this by using import, but then I'll get all the overwrite problems. It is not possible to use depends instead of antcall.
Example files
I have two files:
a.ant
<project>
<include file="b.ant" as="b" />
<target name="test-depends" depends="b.depend">
<echo>${calculated-property}</echo>
</target>
<target name="test-call" depends="b.call">
<echo>${calculated-property}</echo>
</target>
</project>
b.ant
<project>
<target name="depend" depends="some-target">
<property name="calculated-property" value="Hello World"/>
</target>
<target name="call">
<antcall target="some-target" inheritrefs="true"/>
<property name="calculated-property" value="Hello World"/>
</target>
<target name="some-target"/>
</project>
Example output
Calling test-depend works as expected but test-call fails with this output:
b.call:
BUILD FAILED
D:\ws\rambo2\ws-dobshl\ant-test\b.ant:6: The following error occurred while executing this line:
Target "some-target" does not exist in the project "null".
Total time: 258 milliseconds
Ant is a dependency matrix specification language. Usually a bunch of <antcall/>, <ant/>, <include/> and <import/> is a sign of a poorly written build script. It's a developer trying to force Ant to act like a programming language.
For developer, it makes sense to break up a program into smaller files. Even Python and Perl scripts can benefit from this. However, breaking up an Ant build script usually causes problems. We had a developer who went through every project and broke up all the build.xml files into six or seven separate build files in order to improve the process. It basically broke the whole Ant dependency mechanism. To fix it, he then tossed in a bunch of <ant/> calls and <include> tasks. In the end, it meant that each target was called between 12 to 20 times.
Not using <import/> and <antcall/> isn't a hard and fast rule. But, I've been using Ant for years and rarely ever used these mechanisms. When I do, it's usually for a shared build file that multiple projects will use (which sounds like what you have) but instead of defining targets in my shared build file, I define macros. This eliminates the target namespace issues that you are having, and the macros work better because they act more like Ant tasks. This is especially true with the introduction of <local/> in Ant 1.8.
See if you can restructure the shared build file into using <macrodef/> instead of targets. It will make it much easier to include your shared build file.
Give a name to the <project> in b.ant and then change the target of the <antcall>:
<project name="b"> <!-- Give the project a name -->
<target name="depend" depends="some-target">
<property name="calculated-property" value="In b.depend"/>
</target>
<target name="call">
<!-- Specify the name of the project containing the target -->
<antcall target="b.some-target" inheritrefs="true"/>
<property name="calculated-property" value="In b.call"/>
</target>
<target name="some-target"/>
</project>
The result of ant -f a.ant test-call:
b.call:
b.some-target:
test-call:
[echo] In b.call
BUILD SUCCESSFUL
With the changes to b.ant, the <include> in a.ant can be simplified by removing the as attribute:
<include file="b.ant" />

Setting up Ant / Ivy

So, let's assume that I have an already installed SVN and installed ANT / Ivy locally.
I want to have the "shared" part of the ivy config point to some kind of share on a server. How would I need to set this up?
I know I have to dig through the ivy jar and pull out the ivysettings file and modify shared repositories.
So let's assume that I have a server on my intranet at MyServer.intranet.net and my team's folder was under /path/to/NetAdmin (thus the full path would be MyServer.intranet.net/path/to/NetAdmin ) How would I get this set up as a team repository for shared libraries? Would I just specify it and when I package the projects it writes the dependencies there?
Thanks
Here what I did:
I created a Subversion project called ivy.dir.
In this ivy.dir project, I have the latest ivy.jar.
In the ivy.dir, I have the ivysettings.xml setup for our environment. For example, we use a local Artifactory Maven repository for our own jars. The ivysettings.xml in the ivy.dir project points to that.
I created a file called ivy.tasks.xml. This is an Ant build file.
The ivy.tasks.xml looks like this:
<project name="Ivy.Tasks"
xmlns:ivy="http://ant.apache.org/ivy"
xmlns:jacoco="antlib:org.jacoco.ant">
<property environment="env"/>
<!-- Add Ivy Tasks -->
<taskdef uri="http://ant.apache.org/ivy"
resource="org/apache/ivy/ant/antlib.xml">
<classpath>
<fileset dir="${ivy.dir}">
<include name="ivy*.jar"/>
</fileset>
</classpath>
</taskdef>
<ivy:settings file="${ivy.dir}/ivysettings.xml"/>
</project>
Notice that I have my own Ivy settings, thank you. I didn't have to munge up the one in the ivy.jar (although I could have since everyone will use my ivy.jar file!). My ivysettings.xml looks like this:
<ivysettings>
<!-- I'll explain this part below -->
<property name="env.EXECUTOR_NUMBER" value="0" override="false"/>
<caches
defaultCacheDir="${ivy.default.ivy.user.dir}/cache-${env.EXECUTOR_NUMBER}"
resolutionCacheDir="${ivy.dir}/../target/ivy.cache"/>
<!-- Just the standard stuff you find in the `ivysettings.xml in the ivy.jar -->
<settings defaultResolver="default"/>
<include file="${ivy.dir}/ivysettings-public.xml"/> <!-- This one is different -->
<include url="${ivy.default.settings.dir}/ivysettings-shared.xml"/>
<include url="${ivy.default.settings.dir}/ivysettings-local.xml"/>
<include url="${ivy.default.settings.dir}/ivysettings-main-chain.xml"/>
<include url="${ivy.default.settings.dir}/ivysettings-default-chain.xml"/>
</ivysettings>
The big change is the ivysetting-public.xml file:
<ivysettings>
<resolvers>
<ibiblio name="public"
m2compatible="true"
checkmodified="true"
root="http://repos.vegicorp.com/artifactory/libs-release" />
</resolvers>
</ivysettings>
It's pointing to my local Maven repository -- my Artifactory server.
Now, for a developer to use Ivy, all they have to do is:
In the root of their project in Subversion, add a svn:external. This svn:external will be used to bring my ivy.dir project into their Subversion project.
In the build.xml
Add an Ivy namespace definition to their build.xml in the <project> definition.
Set the property ivy.dir to `${basedir}/ivy.dir.
Use the <import> task to import ${ivy.dir}/ivy.tasks.xml into their build.xml file.
Something like this:
<project name="post-a-matic" default="package" basedir="."
xmlns:ivy="http://ant.apache.org/ivy">
<property name="ivy.dir" value="${basedir}/ivy.dir"/>
<import file="${ivy.dir}/ivy.tasks.xml"/>
<!-- A whole bundle of properties are set -->
<target name="clean">
<delete dir="${target.dir}"/>
<ivy:cleancache/> <!-- Look: They have access to Ivy! -->
</target>
<target name="-resolve">
<ivy:resolve/>
</target>
<target name="compile"
depends="-resolve">
<ivy:cachpath
pathid="main.classpath"
conf="compile,provided"/>
<!-- Boy that's easy! -->
<javac srcdir="${main.srcdir}"
destdir="${main.destdir}"
classpathref="main.classpath"/>
</target>
<!-- On and on -->
This solves a lot of problems:
You can update the ivy.settings and everyone will have the updated settings. This ended up being very important to us because we use Jenkins and I wanted Jenkins to clean the ivy cache on each build. Whoops! That cleans out the ivy cache on builds that are being executed at the same time! I solved the problem by changing the ivysettings.xml file to define a different Ivy cache for each Jenkins build executor. One the Jenkins server, you have Ivy caches called $HOME/.ivy2/cache-0, $HOME/.ivy2/cache-1, etc. Each executor can delete it's own Ivy cache without affecting the others. Users, meanwhile will just have $HOME/.ivy2/cache-0.
You also can update Ivy when a new jar comes out. You update your Ivy jar file, and everyone gets the lated.
Big one of course is that Ivy installs itself when a project is checked out.
And an extra special bonus: You could use your ivy.dir and ivy.tasks.xml file to install other tasks. For example, each of our projects must run itself through Findbugs, PMD, CPD (part of the PMD project, Checkstyle, and use JaCoCo. for test coverage.
Each one of these projects consist of a jar file, and a <taskdef> to pull the task definitions into Ant. And, how do you use these tasks too? They're not defined in the standard Ant model. Developers don't know how to use them.
I've added these jars into my ivy.dir project, and installed all of those task definitions into my ivy.tasks.xml file. I also defined easy to use <macrodef> for most of these tasks, so it's easy for the developers to use them. In fact, I've even included the old Ant-Contrib tasks just for fun.
Now, once you add ivy.dir into your project, you have all of these extra tasks, and you have nothing to install on your machine.
You don't need to change the ivy jar. Just create a filesystem resolver in an ivysettings file and publish to this. Here's an example:
good ivy tutorial for local repository?
You'll find that ivy is very flexible and can support pretty much any mechanism for hosting files.
Personally, I'd consider installing a Maven repository manager like Nexus or Artifactory and use this to host both your builds dependencies and build outputs. In the long run it's a lot easier, especially if you're doing Java development.

Ant: "Duplicated project name in import" with imported build file

I have several build files which all import the same base build file, like this:
base.xml:
<project name="base">
<!-- does not define a 'build' target -->
</project>
buildA.xml:
<project name="buildA">
<import file="base.xml" />
<target name="build">
<ant antfile="buildB.xml" target="build"
inheritall="false" inheritrefs="false" />
</target>
</project>
buildB.xml:
<project name="buildB">
<import file="base.xml" />
<target name="build">
...snip...
</target>
</project>
(Module A depends on module B.)
Now, the above calling of B's build target from buildA.xml gives the following error:
Duplicated project name in import. Project base defined first in buildA.xml and again in buildB.xml
Since both buildA.xml and buildB.xml inherit the same base.xml, this seems unavoidable.
How could I get rid of this error?
Based on sudocode's answer, I solved the problem. Because the absolute path to base.xml is different in both cases, Ant does not recognize it as the same file. Even though inheritAll is set to false, the context of the calling task is preserved and this causes the name clash.
To solve this, one can omit the name attribute from base.xml. Since Ant 1.8, the import task has an attribute as, which can be used to reference base targets when the base project is nameless. If you don't override any targets, you can use include instead of import. I'm on 1.7, so that does not help me.
For previous versions of Ant, you can go through an exec call to prevent proliferation of the Ant context entirely (then you get two running Ant instances). Better yet, find a way to import the exact same base.xml (with the same absolute path) in both files.
Are you using Ant 1.6? This resolved Ant bug looks like the same issue.
EDIT
I tried to reproduce the dir structure you refer to in your recent comment.
./base.xml
./buildA
./buildA/buildA.xml
./buildB
./buildB/buildB.xml
And amended the build files accordingly, e.g.
<project name="buildA">
<import file="../base.xml"/>
<target name="build">
<ant antfile="../buildB/buildB.xml" target="build" inheritall="false" inheritrefs="false"/>
</target>
</project>
I still get no build error for the following with ant 1.8.2 or 1.7.1:
ant -f buildA/buildA.xml build

BND Ant task - wrap non-OSGi jars

I'm trying to use Ant bndwrap task to wrap non-OSGi jars in a directory. My current Ant configuration for this is:
<target name="wrap-jars" description="Wrap non-OSGi jars">
<taskdef resource="aQute/bnd/ant/taskdef.properties" classpath="${biz.aQute:bnd:jar}"/>
<bndwrap output="${dist.dir}/app-modules">
<fileset dir="${dist.dir}/app-modules" includes="*.jar" />
</bndwrap>
<move overwrite="true" todir="${dist.dir}/app-modules" >
<fileset dir="${dist.dir}/app-modules" includes="*.bar" />
<mapper type="glob" from="*.bar" to="*.jar" />
</move>
</target>
This works fine, but the problem is that it also wraps existing OSGi jar, which causes problems. For instance, I noticed it changes Bundle-SymbolicName header to some default value. It might be changing something else, which I don't want. I only want it to operate on jars that have no OSGi info at all.
Is there some way to tell BND to ignore existing OSGi headers in manifest, or complete jars that are already OSGi-fied?
I would store non-OSGi jars in a separate folder and modify the fileset to process only that folder.
I've noticed that recent bnd versions (for example, 2.1.0) now honour the Bundle-SymbolicName when rewrapping OSGi jars.
just change your fileset to exclude that jar

Resources