Ant (with ant-contrib) build - path / dependency issue - Reference Not Found - ant

I'm trying to create a system where I can use a common build.xml as a template for a vast majority of our builds. I'm very close. Basically, the application-level build.xml file just needs to define a few things (target jar, dependencies (as a list), and a few other items of state) and then import the template build.xml.
The problem I have is that I'm trying to take the dependencies and build up a classpath programmatically. I'm very close, I think. However, once the classpath is referenced in the compile step, it tries to resolve a reference which seemingly was properly prepared before the compile step occurs. And, it results in a BUILD FAILED : Reference cpDependency4 not found.
Note: COTS stuff is stored from the same directory, so making that part work was trivial. The Project dependencies may all come from different subdirectories and require pulling in the child dependencies/ directory in also.
Here's an example of the items prepared in the application build.xml:
<property name="PROJECT_DEPENDENCIES" value="/path/to/other/dependency1,/path/to/other/dependency2,/path/to/other/dependency3,/path/to/other/dependency4" />
<property name="COTS_DEPENDENCIES" value="commons-io-*.jar,log4j-*.jar,spring-aop-*.RELEASE.jar,spring-beans-*.RELEASE.jar,spring-context-*.RELEASE.jar,spring-core-*.RELEASE.jar,spring-jms-*.RELEASE.jar,spring-oxm-*.RELEASE.jar,spring-web-*.RELEASE.jar"/>
<!-- ... -->
<import file="${DEVELOPMENT_BASE_DIR}/common/ant/antTemplate.xml" />
and then the antTemplate file contains:
<path id="cpDependency0" />
<var name="trackDependencyPath" value="cpDependency0" />
<var name="index" value="0"/>
<for list="${PROJECT_DEPENDENCIES}" param="dependency">
<sequential>
<var name="prior" value="${index}"/>
<math result="index" operand1="${index}" operation="+" operand2="1" datatype="int" />
<property name="var${index}" value="#{dependency}" />
<path id="cpDependency${index}">
<path refid="cpDependency${prior}"/>
<fileset dir="#{dependency}/jar/${env}" includes="*.jar" />
<fileset dir="#{dependency}/jar/${env}/dependencies" includes="*.jar" />
</path>
<var name="trackDependencyPath" value="cpDependency${index}" />
</sequential>
</for>
<path id="classpath.compile">
<fileset dir="${COTS_DIR}/main" includes="${COTS_DEPENDENCIES}" />
<path refid="${trackDependencyPath}" />
</path>
Which seems to set things up properly, but... later, when it hits the compile target:
<target name="compile" depends="init">
<echo message="Compiling ${ant.project.name}..." />
<javac source="1.7" target="1.7" debug="${debug}" srcdir="${SOURCE_JAVA_MAIN_DIR}" destdir="${BUILD_JAVA_MAIN_DIR}" classpathref="classpath.compile" />
<jar destfile="${PACKAGE_DIR}/${TARGET_JAR}" basedir="${BUILD_JAVA_MAIN_DIR}" excludes="*.jar" />
<copy todir="${PACKAGE_DIR}">
<path refid="classpath.compile" />
</copy>
</target>
I get the following error (on the javac line):
BUILD FAILED
/path/to/common/ant/antTemplate.xml:188: Reference cpDependency4 not found.

Your problem is that :
<path id="classpath.compile">
<fileset dir="${COTS_DIR}/main" includes="${COTS_DEPENDENCIES}" />
<path refid="${trackDependencyPath}" />
</path>
refers not to a refid but to a property (or antcontrib variable) which in this case
has the same value => 'cpDependency4'
-- EDIT after comment --
Even using the string 'cpDependency4' won't help as the path id is never set correctly within for loop, as some test reveals :
<for list="${PROJECT_DEPENDENCIES}" param="dependency">
<sequential>
...
<var name="trackDependencyPath" value="cpDependency${index}" />
<!-- put pathconvert here -->
</sequential>
</for>
Try :
<pathconvert property="foo"><path refid="cpDependency4"/></pathconvert>
or
<pathconvert property="foo"><path refid="cpDependency${index}"/></pathconvert>
and you'll get :
Reference cpDependency4 not found.
in both cases. Increase noiselevel to debug and you'll see :
Adding reference: cpDependency0
Setting ro project property: trackDependencyPath -> cpDependency0
Setting ro project property: index -> 0
Setting ro project property: prior -> 0
Setting ro project property: index -> 1
Setting project property: var1 -> /path/to/other/dependency1
Adding reference: cpDependency${index}
Property "env" has not been set
Property "env" has not been set
Setting ro project property: trackDependencyPath -> cpDependency1
Setting ro project property: prior -> 1
Setting ro project property: index -> 2
Setting project property: var2 -> /path/to/other/dependency2
Overriding previous definition of reference to cpDependency${index}
Adding reference: cpDependency${index}
Property "env" has not been set
Property "env" has not been set
Setting ro project property: trackDependencyPath -> cpDependency2
Setting ro project property: prior -> 2
Setting ro project property: index -> 3
Setting project property: var3 -> /path/to/other/dependency3
Overriding previous definition of reference to cpDependency${index}
Adding reference: cpDependency${index}
Property "env" has not been set
Property "env" has not been set
Setting ro project property: trackDependencyPath -> cpDependency3
Setting ro project property: prior -> 3
Setting ro project property: index -> 4
Setting project property: var4 -> /path/to/other/dependency4
Overriding previous definition of reference to cpDependency${index}
Adding reference: cpDependency${index}

Related

How to specify oracle odbjc.jar in Liquibase ant configuration

I have following ant configuration:
<project name="pcebuild" basedir="." default="updateDatabase" xmlns:liquibase="antlib:liquibase.integration.ant" >
<taskdef resource="liquibase/integration/ant/antlib.xml" uri="antlib:liquibase.integration.ant">
<classpath path="c:\Users\artur.skrzydlo\Documents\liquibase-3.3.2-bin\liquibase.jar"/>
</taskdef>
<property name="liquiChangeLogFile" value="${basedir}/liquibase/db.changelog-master.xml"/>
<property name="db.driver" value="oracle.jdbc.OracleDriver"/>
<property name="db.url" value="jdbc:oracle:thin:#websph:1521:XE"/>
<target name="updateDatabase" description="Updates database with new changes using Liquibase">
<liquibase:updateDatabase changeLogFile="${liquiChangeLogFile}" >
<liquibase:database driver="${db.driver}" url="${db.url}" user="${db.user}" password="${db.pasword}"/>
</liquibase:updateDatabase>
</target>
</project>
After running this task I get an error :
Class not found: oracle.jdbc.OracleDriver
According to documentation :
driver The fully qualified class name of the JDBC driver.
I suppose that this error may rise because there is no place where I place classpath to my ojdbc.jar file. I am able to run this update command from command line, however there I can specify "classpath" argument which point to my ojdbc.jar file. And I don's see any place in this ant task definition where could i place it such a path. How can I do this ? What am I doing wrong ?
In your <liquibase:updateDatabase> tag you can have a classpathref attribute. So I have something like this:
<path id="driver.classpath">
<filelist files="${classpath}" />
</path>
...
<liquibase:updateDatabase
databaseref="main-schema"
changelogfile="${changeLogFile}"
classpathref="driver.classpath"
logLevel="debug"
>
...
And ${classpath} is an Ant property, set in a properties file:
classpath: /Users/me/place/lib/classes12.jar

Get immediate subdirectory of a root directory containing a subdirectory

I have a following directory structure
root_dir
fixed_dir
random_dir
subdir1
subdir2
subdir2.1
subdir3
subdir3.1
subdir3.2
In the ANT build file I know the root_dir, fixed_dir, and one directory that is either random_dir or a subdirectory below random_dir (subdirX). I need to determine the path of random_dir given some subdirX. Is it possible to find this directory in ANT and if so, how?
Here is a tested solution for finding the immediate subdirectory of a root directory that contains some subdirectory subdirX at any level of nesting given the file structure provided in the question.
<property name="root.dir" location="${basedir}/root_dir" />
<property name="subdirX" value="subdir2.1" />
<target name="find-immediate-subdir-of-root-containing-subdirX">
<dirset dir="${root.dir}" includes="**/${subdirX}" id="mydirset" />
<pathconvert property="random_dir" pathsep="${line.separator}" refid="mydirset">
<mapper type="regexp"
from="^(${root.dir}${file.separator}[^${file.separator}]+).*"
to="\1"/>
</pathconvert>
<echo message="${random_dir}" />
</target>
Output
find-immediate-subdir-of-root-containing-subdirX:
[echo] /ant/project/basedir/root_dir/random_dir
BUILD SUCCESSFUL
Total time: 1 second
With Ant addon Flaka you'll get the parent as property of a file object (see Flaka Manual, section 3.7.2, i.e.
<project xmlns:fl="antlib:it.haefelinger.flaka">
<!-- let standard ant tasks, i.e. echo
understand EL expresssions -->
<fl:install-property-handler />
<echo>#{file('${basedir}').parent}</echo>
<!-- or without fl:install-property-handler use fl:echo-->
<fl:echo>#{file('${basedir}').parent}</fl:echo>
</project>
so you would use :
#{file('${yoursubdir}').parent

Workaround for nested subant property override behaviour (since 1.8.0)

I am using apache ant in the following way:
I have projects P1, P2, P3. Some of those projects have modules, let say P2 has M1, M2, etc.
All projects have their own ant build script, and all of them have to implement set of predefined targets (build, dist, etc), the scripts expect some properties to be defined when called. let say (build.dir.base)
The modules follow similar structure, so every modules, has its own build file that implements the predefined set of targets, and expects some properties to be set. (let say build.dir.base - same as for the projects)
I also have a global ant script that builds all the projects (or subset)
In code that would look like:
build-all.xml:
<project name="x">
<property name="build.dir.base" location="system.build.base" />
<target name="build">
<echo message="build.dir.base as defined for build-all.xml=${build.dir.base}" />
<subant antfile="build.xml" target="build" inheritAll="false" inheritRefs="false">
<dirset dir="${system.base}" includes="${project-list}" />
<property name="build.dir.base" location="${build.dir.base}" />
</subant>
</target>
</project>
build.xml (one for each project with modules, no subant if project does not have modules):
<project name="y">
<property name="build.dir" location="${basedir}/build" />
<target name="build">
<echo message="build.dir.base as defined for project=${build.dir.base}" />
<subant antfile="build.xml" target="build" inheritAll="false" inheritRefs="false">
<dirset dir="project.base" includes="${module-list}" />
<property name="build.dir.base" location="${build.dir.base}/${name}" />
</subant>
</target>
</project>
And for the projects that have modules:
build.xml (for a module):
<project name="z">
<property name="build.dir.base" location="build.dir.base" />
<target name="build">
<echo message="build.dir.base as defined for module=${build.dir.base}" />
</target>
</project>
This structure allows projects to be built independently, also modules can be build independently or the whole system can be built using the build-all.xml.
Also, the final product has the folowing structure:
${system.build.base}/P1,
${system.build.base}/P2/M1
${system.build.base}/P2/M2
etc
However, since ant >= 1.8.0 this is not possible any more. The reason is that
<property name="build.dir.base" location="${basedir}/build" /> in build-all.xml
takes precedence over <property name="build.dir.base" location="${build.dir.base}/${name}" /> in build.xml (build for projects). So, the destination of the project "submodules" is in ${system.build.base}/M1 instead of ${system.build.base}/P2/M1
This is explained Properties defined on the command line cannot be overridden by nested elements. Since Ant 1.8.0. the same is true for nested structures of tasks: if a build file A invokes B via an task setting a property with a nested element and B contains an tasks invoking C, C will see the value set in A, even if B used a nested element as well.
There is no way to override a property for subant if some of the parents also defined that property. This is severe, since the build should be aware for properties that the parent used for some unrelated reason.
Is there workaround for this incompatible change in behavior? Since, my build system heavily relies on the fact that <subant inheritAll="false" inheritRefs="false"> will do the exec without polluting the sub-builds.
https://issues.apache.org/bugzilla/show_bug.cgi?id=49891 also seems related to this question.
After trying out many different things (including the technique of filtering out the unwanted property then resetting it), I have concluded that it is impossible to override command-line properties in a subant task.
Not a complete answer, but I've found you can prevent specific properties from being inherited by the subtask by doing something like:
<resources id='aoeu'>
<intersect>
<propertyset>
<propertyref builtin="commandline"/>
</propertyset>
<propertyset negate="true">
<propertyref name="property.not.to.be.inherited"/>
</propertyset>
</intersect>
</resources>
<groovy>
ant.propertyset(id:"snth") {
project.references["aoeu"].each {
propertyref(name:"${it.name}")
}
}
</groovy>
<subant buildpathref="buildpath">
<propertyset>
<propertyset refid="snth"/>
</propertyset>
<target name="zvwm"/>
</subant>
It is not impossible!! you can overwrite the properties using a scriptdef
self.getProject().setInheritedProperty(key,value). check out API
You can use the antcontrib var task.
http://ant-contrib.sourceforge.net/tasks/tasks/variable_task.html

What does the Ant warning "Reference * has not been set at runtime..." mean?

Since the upgrade from Ant 1.6.5 to 1.7.1, my build output starts off with:
Warning: Reference project.classpath has not been set at runtime, but was found during
build file parsing, attempting to resolve. Future versions of Ant may support
referencing ids defined in non-executed targets.
Warning: Reference project.test.classpath has not been set at runtime, but was found during
build file parsing, attempting to resolve. Future versions of Ant may support
referencing ids defined in non-executed targets.
I have problems understanding this and why it is outputted, let alone trying to get rid of it. Any help would be appreciated!
EDIT:
Classpath is defined:
<path id="project.classpath">
<pathelement path="./${build.dir}" />
<path refid="libs.ant" />
<fileset dir="${lib.dir}/dependencies/bar" includes="compile/*.jar" />
<fileset dir="${lib.dir}/dependencies/foo" includes="provided/*.jar" />
</path>
<!-- The classpath for compiling this projects' tests -->
<path id="project.test.classpath">
<path refid="project.classpath" />
<fileset dir="${lib.dir}/dependencies/bar" includes="test/*.jar" />
<fileset dir="${lib.dir}/dependencies/foo" includes="test/*.jar" />
</path>
<property name="project.classpath" refid="project.classpath" />
It is referenced (e.g. in ) in this way:
<classpath refid="project.classpath"/>
I had the same issue before. Just make sure the "project.classpath" is defined in the beginning of the build file before other targets reference it. Also your "libs.ant" path should be defined before "project.classpath".
In Ant 1.8 this will actually be error instead of warning.
You can set classpath in the build file as following, to get rid of this warning. See reference. http://ant.apache.org/manual/using.html
<project ... >
<path id="project.class.path">
<pathelement location="lib/"/>
<pathelement path="${java.class.path}/"/>
<pathelement path="${additional.path}"/>
</path>
<target ... >
<rmic ...>
<classpath refid="project.class.path"/>
</rmic>
</target>
<target ... >
<javac ...>
<classpath refid="project.class.path"/>
</javac>
</target>
</project>
Unless this is a typo, it looks like trouble. You really should rename one for clarity even if it isn't the cause of the warning. It still sounds like project.classpath is defined in different targets and they are called in the wrong order. You may need to show more code for more help.
<property name="project.classpath" refid="project.classpath" />

Ant - using variables in .properties files

I am trying to replace placeholders in source files with values defined in a .properties file using the copy task with **
My build.xml contains
<target name="configure">
<echo message="Creating DB configuration" />
<copy todir="${dir.out}" overwrite="true">
<fileset dir="${dir.in}" />
<filterchain>
<expandproperties/>
<replacetokens begintoken="<" endtoken=">" propertiesResource="conf.properties" />
</filterchain>
</copy>
</target>
A sample from the conf.properties:
tbs.base_directory = d:/oracle/oradata/my_app
tbs.data_file = ${tbs.base_directory}/data01.dbf
I want to refer from within the .properties file to variables, in this case I would like to substitute tbs.base_directory in tbs.data_file.
Unfortunately it is not substituted. Any ideas?
Thanks
Problem is that expandproperties applies to copied file not to the property resource you are using to define your tokens. A possible solution is to first load conf.properties to force properties expansion and dump it into a temporary file that is used for token substitution. Something like the following should work:
<target name="configure">
<echo message="Creating DB configuration" />
<!-- force expanding properties in tokens property file -->
<loadproperties srcfile="conf.properties" />
<!-- dump expanded properties in a temp file -->
<echoproperties prefix="tbs" destfile="conf.expanded.properties"/>
<copy todir="${dst.out}" overwrite="true">
<fileset dir="${dir.in}" />
<filterchain>
<expandproperties/>
<!-- use temporary file for token substitution -->
<replacetokens begintoken="<" endtoken=">" propertiesResource="conf.expanded.properties" />
</filterchain>
</copy>
<!-- delete temp file (optinal)-->
<delete file="conf.expanded.properties"/>
</target>
Drawback of this solution is that it only works as long as you can select the properties to write in the temporary file (i.e all properties in the conf.properties file starts with the same prefix).

Resources