Check for packages latest version in Ivy - ant

Question

We use Ivy for dependency management. In order to guarantee stability and traceability, we fix version numbers for all dependencies in our ivy files, plus we use transitive=false to avoid dependency trees to grow uncontrolled. The second has only the disadvantage that it may require a few tests to complete the ivy file.
Since we fix version numbers, we don't get updated about the existence of a later version of a package. What we don't want is to get the freshest version of a dependency at build time. What we want is to periodically check for available updates and later decide whether and which packages to update.
As an example, here are our Spring dependencies as of 01/14/2016
<dependency org="org.springframework" name="spring-core" rev="4.2.4.RELEASE" transitive="false" conf="runtime->*"/>
<dependency org="org.springframework" name="spring-aop" rev="4.2.4.RELEASE" transitive="false" conf="runtime->*"/>
<dependency org="org.springframework" name="spring-beans" rev="4.2.4.RELEASE" transitive="false" conf="runtime->*"/>
<dependency org="org.springframework" name="spring-context" rev="4.2.4.RELEASE" transitive="false" conf="runtime->*"/>
<dependency org="org.springframework" name="spring-context-support" rev="4.2.4.RELEASE" transitive="false" conf="runtime->*"/>
<dependency org="org.springframework" name="spring-expression" rev="4.2.4.RELEASE" transitive="false" conf="runtime->*"/>
<dependency org="org.springframework" name="spring-jdbc" rev="4.2.4.RELEASE" transitive="false" conf="runtime->*"/>
<dependency org="org.springframework" name="spring-orm" rev="4.2.4.RELEASE" transitive="false" conf="runtime->*"/>
<dependency org="org.springframework" name="spring-tx" rev="4.2.4.RELEASE" transitive="false" conf="runtime->*"/>
<dependency org="org.springframework" name="spring-web" rev="4.2.4.RELEASE" transitive="false" conf="runtime->*"/>
<dependency org="org.springframework" name="spring-webmvc" rev="4.2.4.RELEASE" transitive="false" conf="runtime->*"/>
<dependency org="org.springframework" name="spring-test" rev="4.2.4.RELEASE" transitive="false" conf="test->*"/>
<dependency org="org.springframework.plugin" name="spring-plugin-core" rev="1.2.0.RELEASE" transitive="false" conf="runtime->*"/>
<dependency org="org.springframework.plugin" name="spring-plugin-metadata" rev="1.2.0.RELEASE" transitive="false" conf="runtime->*"/>
<dependency org="org.springframework.batch" name="spring-batch-core" rev="3.0.6.RELEASE" transitive="false" conf="runtime->*"/>
<dependency org="org.springframework.batch" name="spring-batch-infrastructure" rev="3.0.6.RELEASE" transitive="false" conf="runtime->*"/>
But we have a lot more. So I am asking if there is a smarter way to check for possible updates for all packages (we now have 101 packages).
Ant's ivy:report won't show the availability of a later version. Manually checking 101 packages on Maven is boring.
We also have a local Artifactory installation, I'm saying that if it could prove useful for purpose.
Any idea? What I would like to see is a report with current and latest version numbers of packages in an Ivy file

Answer 1

I just found an ivy task checkdepsupdate designed to solve your problem:
<target name="resolve" description="Use ivy to resolve classpaths">
<ivy:resolve/>
<ivy:checkdepsupdate showTransitive="false" revisionToCheck="latest.release"/>
</target>
Taking the ivy file in my example below it prints the following report detailing, the latest releases for my 3rd party dependencies.
[ivy:checkdepsupdate] Dependencies updates available :
[ivy:checkdepsupdate] org.slf4j#slf4j-api 1.7.5 -> 1.7.13
[ivy:checkdepsupdate] org.slf4j#slf4j-log4j12 1.7.5 -> 1.7.13
[ivy:checkdepsupdate] junit#junit 4.11 -> 4.12
I think this might be what you are looking for.
Miscellaneous
At the risk of stating the obvious, by setting transitive=false you take upon yourself the job of managing the entire tree of dependencies. For simple projects that's fine but you're now discovering the downsides of this approach. Projects like Spring have deliberately split their deliverables into multiple jars to increase flexibility. It allows you to only download what you need and avoid the inclusion of one incredibly large monolithic spring jar.
I would recommend a couple of things to improve your ivy experience
Embrace ivy's management of transitive dependencies
Use dynamic revisions
Publish to a repository, in order to create a release record
Transitive dependencies and classpath management
In my ivy file I will generally only include the module that contains the class I'm using, letting ivy take care of the other dependencies. I also use ivy configurations to group dependencies by function. My end goal is to use configurations to populate a java classpath, so some of my dependencies are required at compile time, others at run-time, finally testing frequently requires jars that would never be shipped with the release.
Example ivy file:
<ivy-module version="2.0">
<info organisation="com.myspotontheweb" module="demo"/>
<configurations>
<conf name="compile" description="Required to compile application"/>
<conf name="runtime" description="Additional run-time dependencies" extends="compile"/>
<conf name="test" description="Required for test only" extends="runtime"/>
</configurations>
<dependencies>
<!-- compile dependencies -->
<dependency org="org.slf4j" name="slf4j-api" rev="1.7.5" conf="compile->default"/>
<!-- runtime dependencies -->
<dependency org="org.slf4j" name="slf4j-log4j12" rev="1.7.5" conf="runtime->default"/>
<!-- test dependencies -->
<dependency org="junit" name="junit" rev="4.11" conf="test->default"/>
</dependencies>
</ivy-module>
The SLFJ project is an excellent example of how one would use a standard programming API, but at runtime decide on a particular implementation based on the jars included on the classpath. In the above example I tell my build to use the log4j implementation jar at runtime, which will in turn pull down a compatible version of log4j and everything it depends on.
Finally, note how each configuration extends the other? This means that the test configuration will include the jars in both the compile and runtime configurations. Exactly what I'd need when running a unit test using junit.
This is my standard resolve task in ANT:
<target name="resolve" depends="install-ivy" description="Use ivy to resolve classpaths">
<ivy:resolve/>
<ivy:report todir='${build.dir}/ivy-reports' graph='false' xml='false'/>
<ivy:cachepath pathid="compile.path" conf="compile"/>
<ivy:cachepath pathid="test.path" conf="test"/>
</target>
The compile and test classpaths are now auto-populated and ready for use as references:
<target name="compile" depends="resolve" description="Compile code">
<mkdir dir="${build.dir}/classes"/>
<javac srcdir="${src.dir}" destdir="${build.dir}/classes" includeantruntime="false" debug="true" classpathref="compile.path"/>
</target>
<target name="test" depends="compile" description="Run unit tests">
<mkdir dir="${build.dir}/test-reports"/>
<junit printsummary="yes" haltonfailure="yes">
<classpath>
<path refid="test.path"/>
<pathelement path="${build.dir}/classes"/>
</classpath>
..
..
</junit>
</target>
And the resolve task has created a record of each classpath maintained by the build.
Dynamic revisions
When you publish to an ivy repository you can specify the release type. This allows ivy to automatically determine the latest published version of a particular release type. By default two types of release are supported:
integration
release
The former corresponds to the Maven concept of Snapshot releases. Built binaries under the control of another team within your organisation, but not ready for release yet. The latter is of course for binaries that are fully approved and released, ideal for 3rd party dependencies.
The following is an example showing the theorical use of the two dynamic revisions:
<dependencies>
<!-- compile dependencies -->
<dependency org="myorg" name="teamA" rev="latest.integration" conf="compile->default"/>
<dependency org="myorg" name="teamB" rev="latest.integration" conf="compile->default"/>
<dependency org="myorg" name="teamC" rev="latest.integration" conf="compile->default"/>
<dependency org="org.slf4j" name="slf4j-api" rev="latest.release" conf="compile->default"/>
<!-- runtime dependencies -->
<dependency org="org.slf4j" name="slf4j-log4j12" rev="latest.release" conf="runtime->default"/>
<!-- test dependencies -->
<dependency org="junit" name="junit" rev="latest.release" conf="test->default"/>
</dependencies>
So this would achieve your desire. Your build would automatically register new dependencies from 3rd party projects.
Publish to a repository, in order to create a release record
Time does not stand still nor does a project's dependency tree. Due to the high number of direct dependencies a modern Java program may have it can become very confusing to resolve the dependencies.
But.... How does one reproduce an older build? We might tag our source code, but how does one keep track of the dependencies at that point in time.
I decide to publish each release into a Maven repository:
how to publish 3rdparty artifacts with ivy and nexus
Here's a snippet
<target name="prepare" description="Generate POM">
<!-- Optional: Intermediate file containing resolved version numbers -->
<ivy:deliver deliverpattern="${build.dir}/ivy.xml" pubrevision="${publish.revision}" status="release"/>
<!-- Generate the Maven POM -->
<ivy:makepom ivyfile="${build.dir}/ivy.xml" pomfile="${build.dir}/donaldduck.pom"/>
</target>
<target name="publish" depends="init,prepare" description="Upload to Nexus">
<ivy:publish resolver="nexus-deploy" pubrevision="${publish.revision}" overwrite="true" publishivy="false" >
<artifacts pattern="${build.dir}/[artifact](-[classifier]).[ext]"/>
</ivy:publish>
</target>
Since I'm using Nexus I need to generate a Maven POM file for my module. Notice the use of the tasks deliver and makepom? The first will create a temp ivy file containing the resolved version numbers of each of my dependencies. This means the resultant POM file in Maven contains the real versions I used to build my code.
You could expand upon this idea and additionally publish the following alongside your released binary:
Javadocs jar
Source code jar
Ivy reports jar
Junit reports jar
etc
In my opinion the release repository should be the unchanging record for your release and important compliment to the source code repository. Indeed in a large corporate organisation, this kind of file based release record could outlive your source code repository technology (Clearcase -> Subversion -> Git -> ??).