I have a simple ant build with the following ivy dependencies:
<ivy-module version="2.0">
<info organisation="de.jflex" module="examples.simple"/>
<dependencies>
<dependency org="junit" name="junit" rev="4.12"/>
<dependency org="com.google.guava" name="guava" rev="26.0-jre"/>
<dependency org="com.google.truth" name="truth" rev="0.36"/>
</dependencies>
</ivy-module>
ant build fails with:
circular dependency found: com.google.truth#truth;0.36->com.google.testing.compile#compile-testing;0.12->com.google.truth#truth;0.35
But it's not a hard circular dependency: they depend on different versions. How do I work around this?
You are only getting a warning, which is also a default value of the circularDependencyStrategysetting. If you really want to hide this, you can set that attribute to ignore. But it looks like that even if you change this, dependecy resolution will still fail, due to unresolved com.google.gwt#gwt-user;2.8.0: com.google.gwt#gwt;2.8.0->org.sonatype.oss#oss-parent;4
Related
We use Ivy for dependency management. In order to guarantee stability and traceability, we fix version numbers for all dependencies in our ivy files, plus we use transitive=false to avoid dependency trees to grow uncontrolled. The second has only the disadvantage that it may require a few tests to complete the ivy file.
Since we fix version numbers, we don't get updated about the existence of a later version of a package. What we don't want is to get the freshest version of a dependency at build time. What we want is to periodically check for available updates and later decide whether and which packages to update.
As an example, here are our Spring dependencies as of 01/14/2016
<dependency org="org.springframework" name="spring-core" rev="4.2.4.RELEASE" transitive="false" conf="runtime->*"/>
<dependency org="org.springframework" name="spring-aop" rev="4.2.4.RELEASE" transitive="false" conf="runtime->*"/>
<dependency org="org.springframework" name="spring-beans" rev="4.2.4.RELEASE" transitive="false" conf="runtime->*"/>
<dependency org="org.springframework" name="spring-context" rev="4.2.4.RELEASE" transitive="false" conf="runtime->*"/>
<dependency org="org.springframework" name="spring-context-support" rev="4.2.4.RELEASE" transitive="false" conf="runtime->*"/>
<dependency org="org.springframework" name="spring-expression" rev="4.2.4.RELEASE" transitive="false" conf="runtime->*"/>
<dependency org="org.springframework" name="spring-jdbc" rev="4.2.4.RELEASE" transitive="false" conf="runtime->*"/>
<dependency org="org.springframework" name="spring-orm" rev="4.2.4.RELEASE" transitive="false" conf="runtime->*"/>
<dependency org="org.springframework" name="spring-tx" rev="4.2.4.RELEASE" transitive="false" conf="runtime->*"/>
<dependency org="org.springframework" name="spring-web" rev="4.2.4.RELEASE" transitive="false" conf="runtime->*"/>
<dependency org="org.springframework" name="spring-webmvc" rev="4.2.4.RELEASE" transitive="false" conf="runtime->*"/>
<dependency org="org.springframework" name="spring-test" rev="4.2.4.RELEASE" transitive="false" conf="test->*"/>
<dependency org="org.springframework.plugin" name="spring-plugin-core" rev="1.2.0.RELEASE" transitive="false" conf="runtime->*"/>
<dependency org="org.springframework.plugin" name="spring-plugin-metadata" rev="1.2.0.RELEASE" transitive="false" conf="runtime->*"/>
<dependency org="org.springframework.batch" name="spring-batch-core" rev="3.0.6.RELEASE" transitive="false" conf="runtime->*"/>
<dependency org="org.springframework.batch" name="spring-batch-infrastructure" rev="3.0.6.RELEASE" transitive="false" conf="runtime->*"/>
But we have a lot more. So I am asking if there is a smarter way to check for possible updates for all packages (we now have 101 packages).
Ant's ivy:report won't show the availability of a later version. Manually checking 101 packages on Maven is boring.
We also have a local Artifactory installation, I'm saying that if it could prove useful for purpose.
Any idea? What I would like to see is a report with current and latest version numbers of packages in an Ivy file
I just found an ivy task checkdepsupdate designed to solve your problem:
<target name="resolve" description="Use ivy to resolve classpaths">
<ivy:resolve/>
<ivy:checkdepsupdate showTransitive="false" revisionToCheck="latest.release"/>
</target>
Taking the ivy file in my example below it prints the following report detailing, the latest releases for my 3rd party dependencies.
[ivy:checkdepsupdate] Dependencies updates available :
[ivy:checkdepsupdate] org.slf4j#slf4j-api 1.7.5 -> 1.7.13
[ivy:checkdepsupdate] org.slf4j#slf4j-log4j12 1.7.5 -> 1.7.13
[ivy:checkdepsupdate] junit#junit 4.11 -> 4.12
I think this might be what you are looking for.
Miscellaneous
At the risk of stating the obvious, by setting transitive=false you take upon yourself the job of managing the entire tree of dependencies. For simple projects that's fine but you're now discovering the downsides of this approach. Projects like Spring have deliberately split their deliverables into multiple jars to increase flexibility. It allows you to only download what you need and avoid the inclusion of one incredibly large monolithic spring jar.
I would recommend a couple of things to improve your ivy experience
Embrace ivy's management of transitive dependencies
Use dynamic revisions
Publish to a repository, in order to create a release record
Transitive dependencies and classpath management
In my ivy file I will generally only include the module that contains the class I'm using, letting ivy take care of the other dependencies. I also use ivy configurations to group dependencies by function. My end goal is to use configurations to populate a java classpath, so some of my dependencies are required at compile time, others at run-time, finally testing frequently requires jars that would never be shipped with the release.
Example ivy file:
<ivy-module version="2.0">
<info organisation="com.myspotontheweb" module="demo"/>
<configurations>
<conf name="compile" description="Required to compile application"/>
<conf name="runtime" description="Additional run-time dependencies" extends="compile"/>
<conf name="test" description="Required for test only" extends="runtime"/>
</configurations>
<dependencies>
<!-- compile dependencies -->
<dependency org="org.slf4j" name="slf4j-api" rev="1.7.5" conf="compile->default"/>
<!-- runtime dependencies -->
<dependency org="org.slf4j" name="slf4j-log4j12" rev="1.7.5" conf="runtime->default"/>
<!-- test dependencies -->
<dependency org="junit" name="junit" rev="4.11" conf="test->default"/>
</dependencies>
</ivy-module>
The SLFJ project is an excellent example of how one would use a standard programming API, but at runtime decide on a particular implementation based on the jars included on the classpath. In the above example I tell my build to use the log4j implementation jar at runtime, which will in turn pull down a compatible version of log4j and everything it depends on.
Finally, note how each configuration extends the other? This means that the test configuration will include the jars in both the compile and runtime configurations. Exactly what I'd need when running a unit test using junit.
This is my standard resolve task in ANT:
<target name="resolve" depends="install-ivy" description="Use ivy to resolve classpaths">
<ivy:resolve/>
<ivy:report todir='${build.dir}/ivy-reports' graph='false' xml='false'/>
<ivy:cachepath pathid="compile.path" conf="compile"/>
<ivy:cachepath pathid="test.path" conf="test"/>
</target>
The compile and test classpaths are now auto-populated and ready for use as references:
<target name="compile" depends="resolve" description="Compile code">
<mkdir dir="${build.dir}/classes"/>
<javac srcdir="${src.dir}" destdir="${build.dir}/classes" includeantruntime="false" debug="true" classpathref="compile.path"/>
</target>
<target name="test" depends="compile" description="Run unit tests">
<mkdir dir="${build.dir}/test-reports"/>
<junit printsummary="yes" haltonfailure="yes">
<classpath>
<path refid="test.path"/>
<pathelement path="${build.dir}/classes"/>
</classpath>
..
..
</junit>
</target>
And the resolve task has created a record of each classpath maintained by the build.
Dynamic revisions
When you publish to an ivy repository you can specify the release type. This allows ivy to automatically determine the latest published version of a particular release type. By default two types of release are supported:
integration
release
The former corresponds to the Maven concept of Snapshot releases. Built binaries under the control of another team within your organisation, but not ready for release yet. The latter is of course for binaries that are fully approved and released, ideal for 3rd party dependencies.
The following is an example showing the theorical use of the two dynamic revisions:
<dependencies>
<!-- compile dependencies -->
<dependency org="myorg" name="teamA" rev="latest.integration" conf="compile->default"/>
<dependency org="myorg" name="teamB" rev="latest.integration" conf="compile->default"/>
<dependency org="myorg" name="teamC" rev="latest.integration" conf="compile->default"/>
<dependency org="org.slf4j" name="slf4j-api" rev="latest.release" conf="compile->default"/>
<!-- runtime dependencies -->
<dependency org="org.slf4j" name="slf4j-log4j12" rev="latest.release" conf="runtime->default"/>
<!-- test dependencies -->
<dependency org="junit" name="junit" rev="latest.release" conf="test->default"/>
</dependencies>
So this would achieve your desire. Your build would automatically register new dependencies from 3rd party projects.
Publish to a repository, in order to create a release record
Time does not stand still nor does a project's dependency tree. Due to the high number of direct dependencies a modern Java program may have it can become very confusing to resolve the dependencies.
But.... How does one reproduce an older build? We might tag our source code, but how does one keep track of the dependencies at that point in time.
I decide to publish each release into a Maven repository:
how to publish 3rdparty artifacts with ivy and nexus
Here's a snippet
<target name="prepare" description="Generate POM">
<!-- Optional: Intermediate file containing resolved version numbers -->
<ivy:deliver deliverpattern="${build.dir}/ivy.xml" pubrevision="${publish.revision}" status="release"/>
<!-- Generate the Maven POM -->
<ivy:makepom ivyfile="${build.dir}/ivy.xml" pomfile="${build.dir}/donaldduck.pom"/>
</target>
<target name="publish" depends="init,prepare" description="Upload to Nexus">
<ivy:publish resolver="nexus-deploy" pubrevision="${publish.revision}" overwrite="true" publishivy="false" >
<artifacts pattern="${build.dir}/[artifact](-[classifier]).[ext]"/>
</ivy:publish>
</target>
Since I'm using Nexus I need to generate a Maven POM file for my module. Notice the use of the tasks deliver and makepom? The first will create a temp ivy file containing the resolved version numbers of each of my dependencies. This means the resultant POM file in Maven contains the real versions I used to build my code.
You could expand upon this idea and additionally publish the following alongside your released binary:
Javadocs jar
Source code jar
Ivy reports jar
Junit reports jar
etc
In my opinion the release repository should be the unchanging record for your release and important compliment to the source code repository. Indeed in a large corporate organisation, this kind of file based release record could outlive your source code repository technology (Clearcase -> Subversion -> Git -> ??).
I am dealing with a project using using ANT to build the source code into a EAR. The project over couple of years has grown to a mammoth size, more than fifty modules, and not surprisingly it takes 2 hours to build the source code.
The obvious decision that I made was to migrate this to a modular build using IVY for dependency management between sub-modules of the EAR, so I build only the modules that have changed and then package a new EAR every time. I am stuck trying to find out a good way of figuring out dependencies between these sub-modules. Note that 3rd party dependencies were easy to crack. I simply move all that is in the lib folder of these sub-modules to IVY. But, the former is where I am stuck.
This is what they do:
Copy source from all modules to a directory
Compile "everything" into a global.jar
Add this global.jar to the classpath
Build every individual module thereafter
Now how do I figure out that for example Module C depends on module A and B and so A and B should go as ivy dependency in the former? One way could be to add one module at a time in eclipse and try building, and then eleminate failures because of missing classes; but there has to be a better way than this, I cannot imagine spending the next one month of life figuring that out :)
First of all you need to create ivy repository. I don't know how to do it. Try to find it on ivy site. After that you need to create build.xml and ivy.xml file.
Example build.xml which builds some product which depends on module1
<project name="ivy.test" xmlns:ivy="antlib:org.apache.ivy.ant">
<target name="resolve">
<!-- this line tells ivy to use ${ivy.settings.file} where are ivy repositories; more info http://ant.apache.org/ivy/history/latest-milestone/settings.html -->
<ivy:configure file="${ivy.settings.file}"/>
<!-- resolve ivy dependencies from ivy.xml and put them in ivy cache dir -->
<ivy:resolve file="${build.dir}/ivy.xml" transitive="false" />
<!-- finally copy resolved dependencies to ${ivy.lib.dir} -->
<ivy:retrieve conf="default" pattern="${ivy.lib.dir}/[artifact].[ext]" />
</target>
</project>
Example ivy.xml used to resolve dependencies (will try to find module1 in one of ivy repository defined in ivysettings file)
<ivy-module version="2.0">
<info organisation="your.organisation" module="modulename">
<ivyauthor name="yourname" />
</info>
<configurations>
<conf name="default" description="default conf" />
</configurations>
<dependencies>
<dependency name="module1" org="your.organisation" rev="latest.release" conf="default->default" />
</dependencies>
</ivy-module>
Example ivy.xml used by module1 project (this project depends on module2 and module3; module1 is also published in repository)
<ivy-module version="2.0">
<info organisation="your.organisation" module="module1"> <!-- module name is used in dependency -->
<ivyauthor name="yourname"/>
</info>
<configurations>
<conf name="default" description="default configuration"/>
</configurations>
<publications>
<artifact name="module" ext="dll" type="dll" conf="default"/>
</publications>
<dependencies>
<dependency name="module2" org="your.organisation" rev="latest.release" conf="default->default" />
<dependency name="module3" org="your.organisation" rev="latest.release" conf="default->default" />
</dependencies>
</ivy-module>
I have a huge dependency which exports a number of dependencies. I wish to restrict my retriev to couple of them. The pattern is [artifact]-[revision].[ext].
How do i specify this in ivy:retrieve task call
Configurations in ivy is the mechanism for controlling groups of dependencies within ivy.
Once these configurations have been setup in your ivy file it becomes simple to retrieve them within your ANT build as follows:
<ivy:retrieve pattern="lib/[artifact].[ext]" conf="my_custom_conf"/>
Perhaps you could supply some more details of what you want to achieve and someone can demonstrate how to setup a configuration for this purpose. (I'd also recommend searching the Stackoverflow ivy tag, for other examples)
Update
If an ivy module publishes more than one artifact it's possible to restrict the dependency in your ivy file as follows:
ivy.xml
<configurations>
..
<conf name="archives" description="Configuration containing only archive files"/>
</configurations>
<dependencies>
..
<dependency org="acme" name="foo" rev="2.0" conf="archives->default">
<artifact name="a1" type="tar"/>
<artifact name="an" type="zip"/>
</dependency>
</dependencies>
Alternatively..
Look into the remote modules's ivy.xml. There may already be a configuration setup for these files, in which case it becomes a lot simpler (because it's been pre-setup)
<dependency org="acme" name="foo" rev="2.0" conf="archives->remotearchives"/>
The "conf" part of the dependency is mapping the remote configuration onto your local one.
I set up a project following the getting started guide. I have to use ivy. These are my dependencies:
<dependencies>
<dependency org="org.jboss.spec" name="jboss-javaee-all-6.0" rev="3.0.1.Final" conf="build->default" />
<dependency org="org.jboss.arquillian.junit" name="arquillian-junit-container" rev="1.0.3.Final" conf="test->default(*)" transitive="true"/>
<dependency org="org.jboss.arquillian.junit" name="arquillian-junit-core" rev="1.0.3.Final" conf="test->default(*)" transitive="true"/>
<dependency org="org.jboss.weld.arquillian.container" name="arquillian-weld-ee-embedded-1.1" rev="1.1.2.Final" conf="test->default(*)" />
<dependency org="org.jboss.weld" name="weld-core" rev="1.1.10.Final" conf="test->default(*)" />
<dependency org="org.slf4j" name="slf4j-log4j12" rev="1.6.4" conf="test->default(*)" />
</dependencies>
Problem: Although I map to "*", the deps are not resolved transitive. Do I have to add every single jar by hand, just because I am stuck with ivy? or am I missing something?
Clarification:
I use the mapping "myconf->default()" transitive="true".
I read this as follows: "take the default conf of the dependency and map it to "myconf". (): if the dependeny does not provide "default", use every conf it provides. and all this should be done transitive, meaning every sub-dependency will also be mapped.
But what I get is: just the jars specified, and a lot of CNFE when I run the test.
I read about arquillian-container poms that are referenced in maven projects and I am beginning to fear that there is no working "out of the box" dependency mapping mechanism for ivy and arquillian. I am happy Iif anyone can confirm this or provide a working (best: tested) dependency configuration that I can use. Thank you very much!
I'd recommend that your ivy file always declare a set of configurations. Configurations are the logical groupings of jars within your build.
The following example creates a configuration for the 3 classpaths used in a typical java build:
compile
runtime
test
(Note also the "extends" keyword)
<ivy-module version="2.0">
<info organisation="com.myspotontheweb" module="demo"/>
<configurations>
<conf name="compile" description="Required to compile application"/>
<conf name="runtime" description="Additional run-time dependencies" extends="compile"/>
<conf name="test" description="Required for test only" extends="runtime"/>
</configurations>
<dependencies>
<!-- compile dependencies -->
<dependency org="org.jboss.spec" name="jboss-javaee-all-6.0" rev="3.0.1.Final" conf="compile->default" />
<!-- test dependencies -->
<dependency org="org.jboss.arquillian.junit" name="arquillian-junit-container" rev="1.0.3.Final" conf="test->default"/>
<dependency org="org.jboss.arquillian.junit" name="arquillian-junit-core" rev="1.0.3.Final" conf="test->default"/>
<dependency org="org.jboss.weld.arquillian.container" name="arquillian-weld-ee-embedded-1.1" rev="1.1.2.Final" conf="test->default" />
<dependency org="org.jboss.weld" name="weld-core" rev="1.1.10.Final" conf="test->default" />
<dependency org="org.slf4j" name="slf4j-log4j12" rev="1.6.4" conf="test->default" />
</dependencies>
</ivy-module>
The configuration mappings will then map the local configuration to the remote one, as follows:
conf="compile->default"
The remote "default" configuration is normally all you need and will include the remote module's compilation dependencies. For a more detailed explanation of how Maven modules are translated I suggest reading the following answer:
How are maven scopes mapped to ivy configurations by ivy
Finally, your build file can use these configurations to create populated ANT classpaths:
<target name="init" description="Use ivy to resolve classpaths">
<ivy:resolve/>
<ivy:report todir='build/ivy' graph='false' xml='false'/>
<ivy:cachepath pathid="compile.path" conf="compile"/>
<ivy:cachepath pathid="runtime.path" conf="runtime"/>
<ivy:cachepath pathid="test.path" conf="test"/>
</target>
The "report" task is especially useful to document the versions of each jar on the classpath.
In my ivy.xml file I have dependency statements like:
<dependency conf="*->*" org="gnu" name="gcc" rev="4.2.1" changing="false">
<artifact name="gcc" ext="tbz2" e:classifier="src"/>
</dependency>
'rev=' can be a fixed value or 'latest' or whatever.
In my build file I would like to get hold of the value of 'rev' to be uses in further work flow after resolve is run.
Is there any way to do that or read a property for this?
Use the artifactproperty task
<ivy:artifactproperty name="version.[artifact]" value="[revision]"/>
<echo message="gcc version: ${version.gcc}"/>