I have a huge dependency which exports a number of dependencies. I wish to restrict my retriev to couple of them. The pattern is [artifact]-[revision].[ext].
How do i specify this in ivy:retrieve task call
Configurations in ivy is the mechanism for controlling groups of dependencies within ivy.
Once these configurations have been setup in your ivy file it becomes simple to retrieve them within your ANT build as follows:
<ivy:retrieve pattern="lib/[artifact].[ext]" conf="my_custom_conf"/>
Perhaps you could supply some more details of what you want to achieve and someone can demonstrate how to setup a configuration for this purpose. (I'd also recommend searching the Stackoverflow ivy tag, for other examples)
Update
If an ivy module publishes more than one artifact it's possible to restrict the dependency in your ivy file as follows:
ivy.xml
<configurations>
..
<conf name="archives" description="Configuration containing only archive files"/>
</configurations>
<dependencies>
..
<dependency org="acme" name="foo" rev="2.0" conf="archives->default">
<artifact name="a1" type="tar"/>
<artifact name="an" type="zip"/>
</dependency>
</dependencies>
Alternatively..
Look into the remote modules's ivy.xml. There may already be a configuration setup for these files, in which case it becomes a lot simpler (because it's been pre-setup)
<dependency org="acme" name="foo" rev="2.0" conf="archives->remotearchives"/>
The "conf" part of the dependency is mapping the remote configuration onto your local one.
Related
I want to use the url resolver to download depends. The depends are JS, or XML files. So, I used:
<url name="urlresolver">
<artifact pattern="http://[organisation]/[module]-[revision].[ext]" />
</url>
And
<ivy:retrieve pattern="${build}/[module]-[revision].[ext]"/>
the file is saved in .jar extension.
It's worth digging around the following ivy docs:
Terminology
Main Concepts
Best Practices
The first problem is that your url resolver is not configured to read ivy files for the remote modules (ignoring the first recommendation, in the ivy best practices, to use an ivy file with each module). Without module meta-data ivy will assume you're attempting to download JAR files.
A second problem is that you don't appear to be using an ivy repository to store your files. The following dependency declaration:
<dependency org="yourorg" name="module1" rev="9.1"/>
would be translated into the following URL, using your current settings:
http://yourorg/module1-9.1.jar
The "org" field is designed to specify the organisational unit publishing the module, not the server hostname.
I suspect that you're not really interested in building a repository of files and just want to persuade ivy to download and cache the files? In that case I'd recommend reading the following answer which is using extra attributes on the dependency artifacts to do something similar:
Resolving XSD's using Ivy
Example
ivy.xml
<ivy-module version="2.0" xmlns:e="http://ant.apache.org/ivy/extra">
..
..
<dependency org="yourorg" name="yourmodule1" rev="9.1">
<artifact name="file1" e:hostname="www.server1.com" type="xml"/>
<artifact name="file2" e:hostname="www.server1.com" type="xml"/>
</dependency>
<dependency org="yourorg" name="yourmodule2" rev="9.1">
<artifact name="file3" e:hostname="www.server2.com" type="xml"/>
<artifact name="file4" e:hostname="www.server2.com" type="xml"/>
</dependency>
..
Note:
Each dependency declares the type of each artifact, plus the additional "hostname" attribute.
If the remote modules had ivy.xml, the publications section would alternatively store this artifact information.
Extra artifacts demonstrate ivy's power in enabling any sort of custom attribute meta-data.
ivysettings.xml
..
<url name="urlresolver">
<artifact pattern="http://[hostname]/files/[organisation]/[module]-[revision].[ext]" />
</url>
..
Demonstrates how the resolver makes use of both the standard attributes and the custom "hostname".
Can anybody give me a hint on how to use nested artifact (and artifacts) elements for publishing modules use the ivy ant task. Unfortunately the official documentation does not specify how to use the attributes.
http://ant.apache.org/ivy/history/latest-milestone/use/publish.html
Maybe there is some documentation or some examples that I could not find?
I am aware of the artifact element in ivy.xml files. this is NOT what this question is about.
Does this answer to this question help?
Issues using ivy:publish task
The nested "artifact" elements in the publish task are used to identify the location(s) of the artifacts you've specified as to be published by your module.
So, for example, let's say your module publishes two files:
<ivy-module version="2.0">
<info organisation="someorganisation" module="myapp"/>
<publications>
<artifact name="myapp" type="jar"/>
<artifact name="license" type="txt"/>
</publications>
..
Your publish task might need to source these files from two different locations within the build workspace:
<ivy:publish resolver="${publish.resolver}" pubrevision="${publish.revision}" status="${publish.status}">
<artifacts pattern="${build.dir}/[artifact].[ext]"/>
<artifacts pattern="${src.dir}/licenses/[organisation]/[artifact].[ext]"/>
</ivy:publish>
I'm busily getting my feet wet with ivy. I have an existing nexus repository running on my local PC, and an existing ant build script.
Both work fine.
Part of the build scripts have some files to retrieve our 3rdparty jar files (log4j, xmlbeans, junit, pdf, etc..) from a network share - which is klunky at best.
I want to use ivy's dependency mechanisms to retrieve these files from a nexus repository and use that in the build. Each 3rdparty lib has a name and an arbitrary set of files (jar, dll, license.dat, xml, etc).
Since we have a large number of these 3rdparty libs and each lib has multiple files - manual uploading to nexus is not an option -- i need something i can use to take a set of files, give them a lib name, a version number and upload the result to nexus. then I need to be able to retrieve this from ivy.
I managed to get the upload part to work, but the retreival process does not work. Using our xmlbeans lib as a starting point I created the following ivy.xml file
<ivy-module version="1.0">
<info organisation="thirdparty_tools" module="xmlbeans" status="integration">
<publications>
<artifact name="jsr173_api" type="jar" ext="jar"/>
<artifact name="saxon-dom" type="jar" ext="jar"/>
<artifact name="saxon-xpath" type="jar" ext="jar"/>
<artifact name="saxon" type="jar" ext="jar"/>
<artifact name="xbean" type="jar" ext="jar"/>
<artifact name="xbean_xpath" type="jar" ext="jar"/>
<artifact name="xmlpublic" type="jar" ext="jar"/>
</publications>
</ivy-module>
and then some ant script to publish it to nexus:
<ivy:resolve/>
<ivy:publish <ivy:publish resolver="thirdparty" forcedeliver="true" update="true" revision="${version}" overwrite="true">
<artifacts pattern="[artifact].[ext]"/>
<ivy:publish/>
And this all works fine. It publishes all the jar files to nexus in the expected directory.
The trouble comes when I try to use it in my build.
I created the following ivy.xml file for my build:
<ivy-module version="1.0">
<info organisation="myCompany" module="GLB_Data"/>
<dependencies>
<dependency org="thirdparty_tools" name="xmlbeans" rev="2.2.0"/>
</dependencies>
</ivy-module>
Then when I run my build - it fails to find anything:
::::::::::::::::::::::::::::::::::::::::::::::
:: UNRESOLVED DEPENDENCIES ::
::::::::::::::::::::::::::::::::::::::::::::::
:: thirdparty_tools#jsr173_api;2.2.0: not found
:: thirdparty_tools#saxon-dom;2.2.0: not found
:: thirdparty_tools#saxon-xpath;2.2.0: not found
:: thirdparty_tools#saxon;2.2.0: not found
:: thirdparty_tools#xbean;2.2.0: not found
:: thirdparty_tools#xbean_xpath;2.2.0: not found
:: thirdparty_tools#xmlpublic;2.2.0: not found
::::::::::::::::::::::::::::::::::::::::::::::
The problem seems to be with this pattern:
WARN: ==== public: tried
WARN: http //localhost:8081/nexus/content/groups/public/thirdparty_tools/jsr173_api/2.2.0/jsr173_api-2.2.0.pom
WARN: -- artifact thirdparty_tools#jsr173_api;2.2.0!jsr173_api.jar:
WARN: http //localhost:8081/nexus/content/groups/public/thirdparty_tools/jsr173_api/2.2.0/jsr173_api-2.2.0.jar
ivy seems to be looking for the jsr173_api artifact under its own name, rather than under the xmlbeans folder where it was published to:
[ivy:publish] published jsr173_api to http //localhost:8081/nexus/content/repositories/thirdparty/thirdparty_tools/xmlbeans/2.2.0/jsr173_api-2.2.0.jar
(urls obfuscated to prevent accidents).
so somehow I need to publish differently, or retrieve differently. Ideas and suggestions are much appreciated.
Nexus is primarily a Maven repository, this means one must adapt to the way Maven structures artifacts.
Since you're focused on bulk loading Nexus I suggest looking at the answer to the following question:
Upload artifacts to Nexus, without Maven
If you wish to stick with ivy read on.....
Background
Need a Maven POM
Your first issue is that your Maven module(s) will need a POM file. This file describes the maven module and can be easily generated from the contents of your ivy.xml file (See solution below).
Secondly, Maven assumes that there is one primary artifact being built. For example:
<project>
<modelVersion>4.0.0</modelVersion>
<groupId>com.myspotontheweb</groupId>
<artifactId>donaldduck</artifactId>
<version>1.0.1</version>
<packaging>txt</packaging>
</project>
A Maven client would translate this information into the following URL:
http://<host>/<repo>/com/myspotontheweb/donaldduck/1.0.1/donaldduck-1.0.1.txt
This demonstrates how Nexus can store any type of binary dependency. The packaging parameter defaults to "jar".
How maven handles additional module artifacts
While Maven focuses on a single build artifact, it is possible to add additional supplementary artifacts by posting them into the same directory (as you've done).
These are not listed in the Maven POM. Instead a Maven uses a special "classifier" attribute. The following is a possible dependency declaration.
<dependency>
<groupId>com.myspotontheweb</groupId>
<artifactId>donaldduck</artifactId>
<version>1.0.1</version>
<classifier>metadata</classifier>
<type>n3</type>
</dependency>
A Maven client would translate this into the following URL:
http://<host>/<repo>/com/myspotontheweb/donaldduck/1.0.1/donaldduck-1.0.1-metadata.n3
Open source projects typically release their source code in this manner.
Ivy Solution
So finally how does one publish files into Nexus using ivy?
First of all decide which artifact is the "main" build artifact and add an additional entry for your POM file:
<ivy-module version='2.0' xmlns:e="http://ant.apache.org/ivy/extra">
<info organisation="com.myspotonontheweb" module="donaldduck" revision="1.0.1"/>
<publications>
<artifact name="donaldduck" type="txt"/>
<artifact name="donaldduck" type="pom"/>
<artifact name="donaldduck" type="n3" e:classifier="metadata"/>
<artifact name="donaldduck" type="zip" e:classifier="disto"/>
</publications>
</ivy-module>
The other files can also be listed but each must have a unique classifier attribute..... Here you will be faced with one of the classic problems translating an ANT project into Maven.... Each jar file you publish, will probably need to have a separate POM. They not really "supplementary" artifacts.....
Pretending that you don't need to publish multiple modules.... Use the following build targets to publish your module:
<target name="prepare" description="Generate POM">
<!-- Optional: Intermediate file containing resolved version numbers -->
<ivy:deliver deliverpattern="${build.dir}/ivy.xml" pubrevision="${publish.revision}" status="release"/>
<!-- Generate the Maven POM -->
<ivy:makepom ivyfile="${build.dir}/ivy.xml" pomfile="${build.dir}/donaldduck.pom"/>
</target>
<target name="publish" depends="init,prepare" description="Upload to Nexus">
<ivy:publish resolver="nexus-deploy" pubrevision="${publish.revision}" overwrite="true" publishivy="false" >
<artifacts pattern="${build.dir}/[artifact](-[classifier]).[ext]"/>
</ivy:publish>
</target>
Nexus credentials
And for completeness here's the ivysettings.xml file containing the Nexus repository location and credentials:
<ivysettings>
<settings defaultResolver="nexus-central"/>
<credentials host="somehost" realm="Sonatype Nexus Repository Manager" username="????" passwd="????"/>
<resolvers>
<ibiblio name="nexus-central" root="http://somehost/nexus/content/repositories/central/" m2compatible="true"/>
<ibiblio name="nexus-deploy" root="http://somehost/nexus/content/repositories/repo" m2compatible="true"/>
</resolvers>
</ivysettings>
Update
Downloading artifacts
To retrieve all the published artifacts (not just the main one), you need to list them as follows:
<dependency org="com.myspotontheweb" name="donaldduck" rev="1.0.1">
<artifact name="donaldduck" type="txt"/>
<artifact name="donaldduck" type="n3" e:classifier="metadata"/>
<artifact name="donaldduck" type="zip" e:classifier="distro"/>
</dependency>
Functionally the same as the following Maven fragment:
<dependency>
<groupId>com.myspotontheweb</groupId>
<artifactId>donaldduck</artifactId>
<version>1.0.1</version>
<type>txt</type>
</dependency>
<dependency>
<groupId>com.myspotontheweb</groupId>
<artifactId>donaldduck</artifactId>
<version>1.0.1</version>
<classifier>metadata</classifier>
<type>n3</type>
</dependency>
<dependency>
<groupId>com.myspotontheweb</groupId>
<artifactId>donaldduck</artifactId>
<version>1.0.1</version>
<classifier>distro</classifier>
<type>zip</type>
</dependency>
I have a problem with a build where I have to resolve non-standard artifacts through Apache Ivy.
Problem:
I have dependencies on two artifacts (a.jar and a-lib.jar).
The two dependencies come only as part of a single installer (a_installer.jar).
The installer can be downloaded, the embedded artifacts themselves not.
It's possible to manipulate the installer to unpack the needed dependencies.
Requirements:
I have to resolve/download the artifacts during the build (I cannot keep the installer or the extracted artifacts with my code).
I cannot use a repository to store the extracted artifacts.
Subclassing/Extending Ivy/whatever is perfectly fine.
Has anyone solved a similar problem, or some helpful information to share?
Or maybe I'm approaching the problem in the wrong way? From what I found so far on the web, people seem to use Ivy just to download files and post-process them manually (with Ant/whatever) after the fact, and not actually resolving more complicated dependencies within Ivy.
Thanks
PS: I don't care whether the installer is also put into the ivy download cache, but I would like to download the installer only one time (and not for both dependencies).
The problem with a call to "ivy:retrieve" is that you need to also add an "artifact" tag in your ivy.xml (complete with URL) in order to retrieve a dependency not found in a Maven respository...
I don't like this for two reasons
The ivy.xml should just declare your dependencies, not their locations.
Need additonal custom logic in the build.xml to handle the 3rd party package
Ideally it should be your repository settings that decide how to download the various jars, that is why I like the packager resolver. Even if the library I want is not in Maven, I can configure ivy to handle it.
The following is an example of turning the jreleaseinfo project into an ivy dependency (hosted in sourceforge, I couldn't find it in Maven)
ivy.xml:
<?xml version="1.0" encoding="ISO-8859-1"?>
<ivy-module version="2.0">
<info organisation="com.myspotontheweb" module="ivy_packager"/>
<dependencies>
<dependency org="ch.oscg" name="jreleaseinfo" rev="1.3.0"/>
</dependencies>
</ivy-module>
Declare two resolvers. Default is Maven2, the other is a packager configured to look locally for instructions. (See also the Ivy Roundup project)
ivysettings.xml
<ivysettings>
<settings defaultResolver="maven2"/>
<resolvers>
<ibiblio name="maven2" m2compatible="true"/>
<packager name="repackage" buildRoot="${user.home}/.ivy2/packager/build" resourceCache="${user.home}/.ivy2/packager/cache">
<ivy pattern="file:///${basedir}/repository/[organisation]/[module]/[revision]/ivy.xml"/>
<artifact pattern="file:///${basedir}/repository/[organisation]/[module]/[revision]/packager.xml"/>
</packager>
</resolvers>
<modules>
<module organisation="ch.oscg" name="jreleaseinfo" resolver="repackage"/>
</modules>
</ivysettings>
The magic is containing in the "packager" file. At resolve time this will be used to generate an ANT script that both downloads and extracts the required jars.
(No need to put this logic into your build.xml)
repository/ch.oscg/jreleaseinfo/1.3.0/packager.xml
<packager-module version="1.0">
<property name="name" value="${ivy.packager.module}"/>
<property name="version" value="${ivy.packager.revision}"/>
<property name="zipname" value="${name}-${version}"/>
<resource dest="archive" url="http://sourceforge.net/projects/jreleaseinfo/files/jreleaseinfo/jreleaseinfo%201.3.0/jreleaseinfo-1.3.0.zip/download" sha1="9386d92758e627d04c2480b820731fd538b13a3f" type="zip"/>
<build>
<move file="archive/${zipname}/${zipname}.jar" tofile="artifacts/jars/${name}.jar"/>
</build>
</packager-module>
To reduce the number of files I omitted the module's ivy.xml. This appears to be optional unless you want to declare it's licence and other attributes that should be present in a public repository.
I think this is very straightforward: 'ivy:retrieve' a_installer and then unzip a.j and a-lib into your lib directory (or wherever you want it). This should be easily to do with ant?
I have to wonder if there is some complication you haven't mentioned that prevents you from doing this.
I'm using Ivy to manage the dependencies on my project.
So far, I've specified a dependency on Hibernate and servlet-api. However, the hibernate jar itself has a lot of dependencies that aren't really needed, such as jaas and jacc.
This becomes a show-stopper because jaas and jaac are Sun libraries and therefore their licenses forbid to place them in the Maven repos, so Ivy can't find them there.
How do I make Ivy download Hibernate but not these two ?
As a bonus, if I actually needed those and downloaded their Jars from Sun, in which folder in my machine would Ivy look for them ?
Another option for not downloading any dependencies is to disable them with the transitive attribute. So if you wanted hibernate-core, but none of its dependencies, you could do this:
<dependencies>
<dependency org="org.hibernate" name="hibernate-core"
rev="3.3.1.GA" conf='..'
transitive="false" />
</dependencies>
How do I make Ivy download Hibernate but not these two?
Ivy does this using what it calls "configurations." Your ivy.xml that represents Hibernate will need to provide different configurations to represent different use-cases for hibernate. (There is obviously some use of hibernate that does require jaas and jacc, but apparently you don't make use of that case.)
Here is the documentation on configurations. If you want to provide the ivy.xml you are using for hibernate, I can provide pointers on building configurations that will remove the specific libraries you want removed.
If I actually needed those and downloaded their Jars from Sun, in which folder in my machine would Ivy look for them?
The "directories" that ivy looks in for ivy files and artifacts are specified by the list of resolvers you are using. The list of resolvers is specified in the ivy settings file (usually named ivysettings.xml.) Typically, these aren't local directories, but remote URLs. There is; however, a local-file resolver type that will work for this.
If you do this, you will need to provide both ivy files and the artifacts (jars), each with file-names that match the resolvers patterns. Details on that are in the documentation.
Here is an example local-file resolver from an ivy settings file:
<filesystem name="myfiles" checkconsistency="false" checksums="" transactional="false">
<ivy pattern="/data/repo/[organisation]/[module]-[revision].ivy.xml"/>
<artifact pattern="/data/repo/[organisation]/[module]-[revision].[ext]"/>
</filesystem>
Also note that you will need to point your ivy tasks to the correct resolver. You can do this with the resolver attribute on the ant tasks, or with the defaultResolver attribute on the settings element in the ivy settings file.
Here is the documentation on resolvers.
EDIT: The OP found a less-time intensive workaround for his specific original problem. The "exclude" child-tag of the dependency tag did the job for him:
<dependencies>
<dependency org="org.hibernate" name="hibernate-core" rev="3.3.1.GA" conf='..'>
<exclude name='jaas' />
<exclude name='jacc' />
</dependency>
</dependencies>
Browsing the web and blogs, I found the following ivy-settings to work at grabbing jaas/jacc and hibernate
<ivysettings>
<settings defaultResolver="chained" checkUpToDate="true" />
<resolvers>
<chain name="chained">
<url name="com.springsource.repository.bundles.release">
<ivy pattern="http://repository.springsource.com/ivy/bundles/release/[organisation]/[module]/[revision]/[artifact]-[revision].[ext]" />
<artifact pattern="http://repository.springsource.com/ivy/bundles/release/[organisation]/[module]/[revision]/[artifact]-[revision].[ext]" />
</url>
<url name="com.springsource.repository.bundles.external">
<ivy pattern="http://repository.springsource.com/ivy/bundles/external/[organisation]/[module]/[revision]/[artifact]-[revision].[ext]" />
<artifact pattern="http://repository.springsource.com/ivy/bundles/external/[organisation]/[module]/[revision]/[artifact]-[revision].[ext]" />
</url>
<ibiblio name="ibiblio" m2compatible="true"/>
<ibiblio name="jboss" root="http://repository.jboss.org/maven2/" m2compatible="true"/>
<ibiblio name="java-net-maven1" root="http://download.java.net/maven/1" pattern="${java.net.maven.pattern}" m2compatible="false"/>
<ibiblio name="java-net-maven2" root="http://download.java.net/maven/2/" m2compatible="true"/>
<ibiblio name="compass" m2compatible="true" root="http://repo.compass-project.org" />
</chain>
</resolvers>
The jboss ibibilio resolver is what did the trick at grabbing JAAS/JAAC
My ivy.xml then can then pull it in with
<ivy-module version="2.0">
<info organisation="foo" module="Bar"/>
<dependencies>
<dependency org="com.h2database" name="h2" rev="1.2+"/>
<dependency org="org.hibernate" name="hibernate-annotations" rev="3.4.0.GA"/>
</dependencies>
To answer your second sub-question literally, which nobody has done so far, "in which folder in my machine would Ivy look for JARs?" That depends. Assuming you haven't changed the location in ivysettings.xml or another configuration file: for JAAS,
this would be: (user home)/.ivy2/cache/javax.security/jaas/jars. If Ivy already unsuccessfully tried to find JAAS in the Maven Central or other repo's, that directory tree should already exist for the most part, and all you need to do is create the "jars" directory and place jaas-1.0.01.jar in it. Ivy will no longer complain about the missing dependency in its next invocation.
EDIT: Then again, see the discussion below to see considerations to not do it like this.
((user home) is C:/Users/(username) on Windows 7).