I'm trying to set up an Ant + Ivy build for a personal project. Everything was making sense, and working nicely, until I got to LWJGL. Everything from LWJGL is resolved, except the natives.
The Readme.md on their website makes it seem that it is possible to get these through Ivy:
LWJGL 3 can be used with Maven/Gradle/Ivy, with the following
dependencies:
org.lwjgl:lwjgl:${version}
org.lwjgl:lwjgl-platform:${version}:natives-windows
org.lwjgl:lwjgl-platform:${version}:natives-linux
org.lwjgl:lwjgl-platform:${version}:natives-osx
The files I want are definitely on the maven central repository, so there must be a way of getting them through Ivy. I have set up my ivy.xml file like so:
<ivy-module version="1.0" xmlns:extra="http://ant.apache.org/ivy/extra">
<info organisation="foo" module="bar"/>
<publications>
<artifact name="baz" type="jar"/>
</publications>
<dependencies>
<dependency org="org.lwjgl" name="lwjgl" rev="3.0.0a"/>
<dependency org="org.lwjgl" name="lwjgl-platform" rev="3.0.0a" extra:classifier="natives-linux"/>
<dependency org="org.lwjgl" name="lwjgl-platform" rev="3.0.0a" extra:classifier="natives-osx"/>
<dependency org="org.lwjgl" name="lwjgl-platform" rev="3.0.0a" extra:classifier="natives-windows"/>
</dependencies>
</ivy-module>
And my resolve task in ant:
<target name="resolve" description="Retrive dependencies with Ivy">
<ivy:retrieve/>
</target>
For some reason, this downloads all the artifacts from "org.lwjgl:lwjgl:3.0.0a" (jar, javadoc, and sources), but does not download any of the natives from "org.lwjgl:lwjgl-platform:3.0.0a". I spent a long time on Google, and finally managed to find the "extra:classifier" syntax in someone else's ivy.xml file on Github, but to no avail (I got my hopes up too early). There must be something I'm missing, so I hope someone on SO can help.
Extra artifacts in Maven modules must be explicitly retrieved in the ivy dependency declaration.
You'll also need to specify the pattern used in the retrieve task, because "classifier" is a Maven specific tag and optional.
Example
├── build.xml
├── ivy.xml
└── target
└── lib
├── lwjgl-3.0.0a.jar
├── lwjgl-platform-3.0.0a-natives-linux.jar
├── lwjgl-platform-3.0.0a-natives-osx.jar
└── lwjgl-platform-3.0.0a-natives-windows.jar
build.xml
<project name="demo" default="resolve" xmlns:ivy="antlib:org.apache.ivy.ant">
<property name="build.dir" location="target"/>
<target name="resolve">
<ivy:retrieve pattern="${build.dir}/lib/[artifact]-[revision](-[classifier]).[ext]"/>
</target>
</project>
ivy.xml
<ivy-module version="1.0" xmlns:extra="http://ant.apache.org/ivy/extra">
<info organisation="foo" module="bar"/>
<dependencies>
<dependency org="org.lwjgl" name="lwjgl" rev="3.0.0a" conf="default"/>
<dependency org="org.lwjgl" name="lwjgl-platform" rev="3.0.0a">
<artifact name="lwjgl-platform" type="jar" extra:classifier="natives-linux"/>
<artifact name="lwjgl-platform" type="jar" extra:classifier="natives-osx"/>
<artifact name="lwjgl-platform" type="jar" extra:classifier="natives-windows"/>
</dependency>
</dependencies>
</ivy-module>
Related
I want to download artifacts using ivy:resolve, but using a pattern that uses the [conf] attribute. So I have defined the following in
IvySettings.xml
<caches defaultCacheDir="${env.IvyLib}\cache" artifactPattern="[conf]\[artifact].[ext]" checkUpToDate="false" />
Notice the pattern artifactPattern="[conf]\[artifact].[ext]", so I want to resolve the dependencies and place them in a folder as per their configuration. I have defined the following configuration in
Ivy.xml
<configurations>
<conf name="ConfGroup1" description="First group of dependencies"/>
<conf name="ConfGroup2" description="Second group of dependencies"/>
</configurations>
<dependencies>
<dependency org="derby-db" name="derby-db" rev="10.2.2.0" conf="ConfGroup1->default">
<artifact name="derby-db" type="zip" ext="zip" />
</dependency>
<dependency org="derby-db" name="derby-db" rev="10.4.1.3" conf="ConfGroup2->default">
<artifact name="derby-db" type="zip" ext="zip" />
</dependency>
</dependencies>
Now the dependencies get resolved fine, but only 1 folder gets created for configuration: default. No folder is being created for ConfGroup1 & ConfGroup2 configurations. How can I be able to create multiple folders as per my defined configurations during resolve?
P.S., I know this can be achieved using ivy:retrieve, but I do not want to use it, because it will involve copying artifacts from ivy cache to another place after ivy:resolve, and I have multi-gigabyte worth of artifacts. Copying them separately will create additional overhead during the build, which I cannot afford due to the project requirements.
The ivy retrieve task has a symlinks option, that can be used to conserve space.
Example
├── build.xml
├── ivysettings.xml
├── ivy.xml
└── target
├── ivy-reports
│ ├── ivy-report.css
│ ├── myorg-mymod-ConfGroup1.html
│ └── myorg-mymod-ConfGroup2.html
└── lib
├── ConfGroup1
│ └── db-derby-10.12.1.1.zip -> /home/mark/.ivy2/cache/db-derby/db-derby/zips/db-derby-10.12.1.1.zip
└── ConfGroup2
└── db-derby-10.11.1.1.zip -> /home/mark/.ivy2/cache/db-derby/db-derby/zips/db-derby-10.11.1.1.zip
build.xml
<project name="demo" default="resolve" xmlns:ivy="antlib:org.apache.ivy.ant">
<property name="build.dir" location="target"/>
<target name="resolve">
<ivy:resolve/>
<ivy:report todir='${build.dir}/ivy-reports' graph='false' xml='false'/>
<ivy:retrieve pattern='${build.dir}/lib/[conf]/[module]-[revision].[ext]' symlink="true"/>
</target>
</project>
ivy.xml
<ivy-module version="2.0">
<info organisation="myorg" module="mymod"/>
<configurations>
<conf name="ConfGroup1" description="First group of dependencies"/>
<conf name="ConfGroup2" description="Second group of dependencies"/>
</configurations>
<dependencies>
<dependency org="db-derby" name="db-derby" rev="10.12.1.1" conf="ConfGroup1->default">
<artifact name="db-derby" type="zip" ext="zip" />
</dependency>
<dependency org="db-derby" name="db-derby" rev="10.11.1.1" conf="ConfGroup2->default">
<artifact name="db-derby" type="zip" ext="zip" />
</dependency>
</dependencies>
</ivy-module>
ivysettings.xml
<ivysettings>
<settings defaultResolver="central"/>
<resolvers>
<ibiblio name="central" m2compatible="true" />
<url name="db-derby">
<artifact pattern="http://ftp.heanet.ie/mirrors/www.apache.org/dist//db/derby/[module]-[revision]/[module]-[revision]-bin.[ext]"/>
</url>
</resolvers>
<modules>
<module organisation="db-derby" resolver="db-derby" />
</modules>
</ivysettings>
Building with Ant Ivy, I'm trying to separate my jars into one configuration for 3rd party jars and another configuration for the jars I build and publish. ProjectA uses 3rd party jars and builds a jar that ProjectB depends on, but when I use Ant Ivy confs I can't get ProjectB to retrieve the ProjectA jar.
When I execute the ant script for ProjectB, it builds ProjectA fine. The ProjectA build publishes a jar to the local repository. ProjectB retrieves the necessary jars from the public repository with no problem, but when it tries to retrieve the ProjectA jar, it says UNRESOLVED DEPENDENCY: testproject#ProjectA;2.0.0: configuration not found in testproject#ProjectA;2.0.0: 'localjars'. It was required from testproject#ProjectB;2.0.0 localjars
If I remove all references to the 2nd configuration, localjars, and just use default for everything, it works fine. I really need to sort my jars into the different confs though.
I've successfully used a revision value passed from the ant script in place of "2.0.0" below and referenced with ${revision}, but the conf error is the same.
ProjectA ivy.xml (with a subset of dependencies for brevity):
<ivy-module version="2.0">
<info organisation="testproject" module="ProjectA" revision="2.0.0" status="release" publication="20160524124555"/>
<configurations>
<conf name="default" transitive="false" visibility="public"/>
<conf name="localjars" extends="default" visibility="public"/>
</configurations>
<publications>
<artifact name="projectA-jar-2.0.0" type="jar" conf="localjars" ext="jar"/>
</publications>
<dependencies>
<dependency org="commons-beanutils" name="commons-beanutils" rev="1.7.0" conf="default->master"/>
<dependency org="commons-collections" name="commons-collections" rev="3.1" conf="default->master"/>
</dependencies>
</ivy-module>
ProjectA build.xml publish target:
<target name="publish" depends="package"
description="--> compile test and publish this project in the local ivy repository">
<ivy:publish artifactspattern="${DEPLOY_DIR_LIB}/[artifact].[ext]"
resolver="local" pubrevision="2.0.0" status="release"
srcivypattern="${ivy.dep.file}" forcedeliver="true" overwrite="true" conf="localjars,default"/>
<echo message="project ${ant.project.name} released with version 2.0.0" />
</target>
ProjectB ivy.xml:
<ivy-module version="2.0">
<info organisation="testproject" module="ProjectB" revision="2.0.0" status="release" publication="20160524103113"/>
<configurations>
<conf name="default"/>
<conf name="localjars" extends="default"/>
</configurations>
<publications>
<artifact name="projectB-2.0.0" conf="localjars" type="jar" ext="jar"/>
</publications>
<dependencies>
<dependency org="testproject" name="ProjectA" rev="${revision}" transitive="true" conf="localjars->localjars; default->default"/>
</dependencies>
ProjectB Ant resolve target:
<target name="resolve" description="--> retrieve dependencies with ivy">
<ivy:retrieve pattern="${DEPLOY_DIR_LIB}/[artifact]-2.0.0.[ext]" revision="2.0.0" conf="localjars" />
</target>
Any idea what's wrong? Thanks!
Patrick
Not entirely certain why your configuration is not working. One thing I'd advise is not to disable transitive dependencies. You'll note I have a different approach in creating the "default" configuration in the working example below.
Example
Each project has its own local build and ivy file. Collaboration is via jars published to the "local" repository.
├── build.xml
├── ProjectA
│ ├── build.xml
│ ├── ivy.xml
│ └── src
│ └── Hello.txt
└── ProjectB
├── build.xml
├── ivy.xml
└── src
└── Hello.txt
build.xml
Master build file that builds all modules in the correct order, using the buildlist task.
Additionally I normally include an extra target for installing ivy.
<project name="main" default="publish" xmlns:ivy="antlib:org.apache.ivy.ant">
<available classname="org.apache.ivy.Main" property="ivy.installed"/>
<target name="install-ivy" unless="ivy.installed">
<mkdir dir="${user.home}/.ant/lib"/>
<get dest="${user.home}/.ant/lib/ivy.jar" src="http://search.maven.org/remotecontent?filepath=org/apache/ivy/ivy/2.4.0/ivy-2.4.0.jar"/>
<fail message="Ivy has been installed. Run the build again"/>
</target>
<target name="publish" depends="install-ivy">
<ivy:buildlist reference="build-path">
<fileset dir="." includes="*/build.xml"/>
</ivy:buildlist>
<subant target="publish" buildpathref="build-path"/>
</target>
<target name="clean">
<subant target="clean">
<fileset dir="." includes="*/build.xml"/>
</subant>
</target>
<target name="clean-all" depends="clean">
<ivy:cleancache/>
</target>
</project>
ProjectA/ivy.xml
The "master" configuration contains only artifacts. This naming convention mirrors the scopes used by Maven.
Note also how the "default" configuration extends both "master" and "runtime". This enables clients to pull down everything they will need.
<ivy-module version="2.0">
<info organisation="com.myspotontheweb" module="ProjectA"/>
<configurations>
<conf name="default" description="Master artifact and runtime dependencies" extends="master,runtime"/>
<conf name="master" description="Artifact published by this module"/>
<conf name="compile" description="Required to compile application"/>
<conf name="runtime" description="Additional run-time dependencies" extends="compile"/>
<conf name="test" description="Required for test only" extends="runtime"/>
</configurations>
<publications>
<artifact name="ProjectA" type="jar" ext="jar" conf="master"/>
</publications>
<dependencies>
<!-- compile dependencies -->
<dependency org="org.slf4j" name="slf4j-api" rev="1.7.5" conf="compile->default"/>
<!-- runtime dependencies -->
<dependency org="org.slf4j" name="slf4j-log4j12" rev="1.7.5" conf="runtime->default"/>
<!-- test dependencies -->
<dependency org="junit" name="junit" rev="4.11" conf="test->default"/>
</dependencies>
</ivy-module>
ProjectB/ivy.xml
Note how ProjectA is the only dependency and it maps the remote "default" configuration to the local "compile" configuration.
Another subtle issue is the use of the "latest.integration" dynamic revision. This means we don't need to hardcode the revision of ProjectA.
<ivy-module version="2.0">
<info organisation="com.myspotontheweb" module="ProjectB"/>
<configurations>
<conf name="default" description="Master artifact and runtime dependencies" extends="master,runtime"/>
<conf name="master" description="Artifact published by this module"/>
<conf name="compile" description="Required to compile application"/>
<conf name="runtime" description="Additional run-time dependencies" extends="compile"/>
<conf name="test" description="Required for test only" extends="runtime"/>
</configurations>
<publications>
<artifact name="ProjectB" type="jar" ext="jar" conf="master"/>
</publications>
<dependencies>
<dependency org="com.myspotontheweb" name="ProjectA" rev="latest.integration" conf="compile->default"/>
</dependencies>
</ivy-module>
ProjectA/build.xml
The revision to be published is set as a property, which can be overridden if necessary.
Note also how ivy configurations can be used to control classpaths within the build, using the cachepath task
<project name="ProjectA" default="build" xmlns:ivy="antlib:org.apache.ivy.ant">
<property name="build.dir" location="build"/>
<property name="pub.version" value="1.0"/>
<property name="pub.resolver" value="local"/>
<target name="resolve">
<ivy:resolve/>
<ivy:cachepath pathid="compile.path" conf="compile"/>
<ivy:cachepath pathid="test.path" conf="test"/>
</target>
<target name="build" depends="resolve">
<mkdir dir="${build.dir}"/>
<jar destfile="${build.dir}/${ant.project.name}.jar" basedir="src"/>
</target>
<target name="publish" depends="build">
<ivy:publish pubrevision="${pub.version}" resolver="${pub.resolver}" overwrite="true">
<artifacts pattern="${build.dir}/[artifact].[ext]"/>
</ivy:publish>
</target>
<target name="clean">
<delete dir="${build.dir}"/>
</target>
</project>
ProjectB/build.xml
Identical to the other project just having a different name attribute.
<project name="ProjectB" default="build" ....
I set up a project following the getting started guide. I have to use ivy. These are my dependencies:
<dependencies>
<dependency org="org.jboss.spec" name="jboss-javaee-all-6.0" rev="3.0.1.Final" conf="build->default" />
<dependency org="org.jboss.arquillian.junit" name="arquillian-junit-container" rev="1.0.3.Final" conf="test->default(*)" transitive="true"/>
<dependency org="org.jboss.arquillian.junit" name="arquillian-junit-core" rev="1.0.3.Final" conf="test->default(*)" transitive="true"/>
<dependency org="org.jboss.weld.arquillian.container" name="arquillian-weld-ee-embedded-1.1" rev="1.1.2.Final" conf="test->default(*)" />
<dependency org="org.jboss.weld" name="weld-core" rev="1.1.10.Final" conf="test->default(*)" />
<dependency org="org.slf4j" name="slf4j-log4j12" rev="1.6.4" conf="test->default(*)" />
</dependencies>
Problem: Although I map to "*", the deps are not resolved transitive. Do I have to add every single jar by hand, just because I am stuck with ivy? or am I missing something?
Clarification:
I use the mapping "myconf->default()" transitive="true".
I read this as follows: "take the default conf of the dependency and map it to "myconf". (): if the dependeny does not provide "default", use every conf it provides. and all this should be done transitive, meaning every sub-dependency will also be mapped.
But what I get is: just the jars specified, and a lot of CNFE when I run the test.
I read about arquillian-container poms that are referenced in maven projects and I am beginning to fear that there is no working "out of the box" dependency mapping mechanism for ivy and arquillian. I am happy Iif anyone can confirm this or provide a working (best: tested) dependency configuration that I can use. Thank you very much!
I'd recommend that your ivy file always declare a set of configurations. Configurations are the logical groupings of jars within your build.
The following example creates a configuration for the 3 classpaths used in a typical java build:
compile
runtime
test
(Note also the "extends" keyword)
<ivy-module version="2.0">
<info organisation="com.myspotontheweb" module="demo"/>
<configurations>
<conf name="compile" description="Required to compile application"/>
<conf name="runtime" description="Additional run-time dependencies" extends="compile"/>
<conf name="test" description="Required for test only" extends="runtime"/>
</configurations>
<dependencies>
<!-- compile dependencies -->
<dependency org="org.jboss.spec" name="jboss-javaee-all-6.0" rev="3.0.1.Final" conf="compile->default" />
<!-- test dependencies -->
<dependency org="org.jboss.arquillian.junit" name="arquillian-junit-container" rev="1.0.3.Final" conf="test->default"/>
<dependency org="org.jboss.arquillian.junit" name="arquillian-junit-core" rev="1.0.3.Final" conf="test->default"/>
<dependency org="org.jboss.weld.arquillian.container" name="arquillian-weld-ee-embedded-1.1" rev="1.1.2.Final" conf="test->default" />
<dependency org="org.jboss.weld" name="weld-core" rev="1.1.10.Final" conf="test->default" />
<dependency org="org.slf4j" name="slf4j-log4j12" rev="1.6.4" conf="test->default" />
</dependencies>
</ivy-module>
The configuration mappings will then map the local configuration to the remote one, as follows:
conf="compile->default"
The remote "default" configuration is normally all you need and will include the remote module's compilation dependencies. For a more detailed explanation of how Maven modules are translated I suggest reading the following answer:
How are maven scopes mapped to ivy configurations by ivy
Finally, your build file can use these configurations to create populated ANT classpaths:
<target name="init" description="Use ivy to resolve classpaths">
<ivy:resolve/>
<ivy:report todir='build/ivy' graph='false' xml='false'/>
<ivy:cachepath pathid="compile.path" conf="compile"/>
<ivy:cachepath pathid="runtime.path" conf="runtime"/>
<ivy:cachepath pathid="test.path" conf="test"/>
</target>
The "report" task is especially useful to document the versions of each jar on the classpath.
I have a special situation where I need to package up some jars, and I need BOTH versions of a jar. My ivy definitions looks like this:
<dependency org="blah" name="blahname" rev="1.0.0" conf="baseline->default" />
I would like the same dependency resolved twice, once with version 1.0.0 and another with say version 2.0.0. Is that possible? What's the easiest way to achieve this.
Use ivy configurations to create and manage custom groups of dependencies.
Example
ivy.xml
<ivy-module version="2.0">
<info organisation="com.myspotontheweb" module="demo"/>
<configurations>
<conf name="group1" description="First group of dependencies"/>
<conf name="group2" description="Second group of dependencies"/>
</configurations>
<dependencies>
<dependency org="commons-lang" name="commons-lang" rev="2.6" conf="group1->default"/>
<dependency org="commons-lang" name="commons-lang" rev="2.0" conf="group2->default"/>
</dependencies>
</ivy-module>
build.xml
<project name="demo" default="resolve" xmlns:ivy="antlib:org.apache.ivy.ant">
<target name="resolve">
<ivy:resolve/>
<ivy:retrieve pattern="lib/[conf]/[artifact]-[revision].[ext]"/>
</target>
</project>
Notes:
This example uses the retrieve task to populate the "lib" directory. See also the "cachepath" and "cachefileset" tasks.
Results
Lib directory is populated with the desired jars.
$ tree
.
|-- build.xml
|-- ivy.xml
`-- lib
|-- group1
| `-- commons-lang-2.6.jar
`-- group2
`-- commons-lang-2.0.jar
I'm writing a Java library for my project: I use Ivy for the dependencies management and the publishing of the JAR to my local repository.
When I update ivy.xml (i.e. to add a new external library), everything is OK: all artifacts are retrieved and used.
However, when I publish my library, the ivy-module xml generated by Ivy contains missing/wrong references (often to previous versions of the external libraries).
This is the target I have in my build.xml:
<target name="publish" depends="jar" description="Publish this project in the ivy repository">
<property name="revision" value="${version}" />
<ivy:publish artifactspattern="${jar.dir}/[artifact].[ext]" resolver="projects" pubrevision="${revision}" status="release" update="true" overwrite="true" />
<echo message="project ${ant.project.name} released with version ${revision}" />
</target>
And this is my ivy.xml:
<ivy-module version="2.0">
<info organisation="xyz" module="zyx"/>
<configurations defaultconfmapping="*->*,!javadoc,!sources" />
<dependencies>
<dependency org="ch.qos.logback" name="logback-classic" rev="0.9.28" />
<dependency org="commons-lang" name="commons-lang" rev="2.5"/>
<dependency org="commons-io" name="commons-io" rev="2.0"/>
<dependency org="org.simpleframework" name="simple-xml" rev="2.4.1">
<exclude module="stax"/>
<exclude module="stax-api"/>
</dependency>
<dependency name="AlmaUtils" rev="1.3.10"/>
<!-- Reflections -->
<dependency org="org.reflections" name="reflections" rev="0.9.5-RC2">
<exclude module="logback-classic"/>
</dependency>
<!-- Bouncycastle cryptography -->
<dependency org="org.bouncycastle" name="bcprov-ext-jdk16" rev="1.45"/>
<dependency org="jdom" name="jdom" rev="1.1">
<exclude module="xerces"/>
<exclude module="xalan"/>
</dependency>
<!-- Scripting -->
<dependency name="js-engine" rev="1.0"/>
<dependency org="rhino" name="js" rev="1.7R2"/>
<!-- JGA -->
<dependency name="jga" rev="0.8.1"/>
</dependencies>
Perhaps you should add the following target into your build.
<target name="clean-all" depends="clean" description="Purge ivy cache">
<ivy:cleancache/>
</target>
This will wipe the slate clean and ensure that your build is completely clean.
Ivy is basically an optimized downloader, however, sometimes it can make incorrect caching decisions when you upgrade the version of a complex tree of dependencies. Maven builds are also affected by this problem when the local repository is very large.