I am using Ivy to publish a snapshot of a built Jar to a locally hosted Nexus repository using the following Ant target.
<target name="publish">
<ivy:publish resolver="nexus_snapshot" pubrevision="SNAPSHOT" overwrite="true">
<artifacts pattern="${dist.dir}/[artifact].[ext]" />
</ivy:publish>
</target>
This appears to work fine, resulting in the Jar and its associated ivy.xml being present in the repository (with filenames mymodule-SNAPSHOT.jar and ivy-SNAPSHOT.jar).
Later, in another build script, I wish to retrieve the Jar and its associated dependencies (i.e. as specified in its ivy.xml) into a directory.
This is the Ant target I'm using.
<target name="deploy">
<delete dir="deploy" />
<mkdir dir="deploy" />
<ivy:settings file="${ivy.dir}/ivy_deploy_settings.xml" />
<ivy:retrieve organisation="myorg" module="mymodule"
inline="true" revision="SNAPSHOT" pattern="deploy/[artifact].[ext]"/>
</target>
This retrieves the Jar to the directory, but not its dependencies. Also, if I add
conf="impl"
to the retrieve, it fails as the configuration is not found.
As such, it seems that the retrieve is simply not referencing the ivy.xml and hence not resolving the dependencies.
Should this work or am I misunderstanding something?
I have now resolved this problem. I believe the issue is simply that Nexus works using POM files rather than Ivy files (by default at least - I can't see any relevant configuration options).
The solution is therefore to generate a suitable POM and publish this along with the Jar.
<target name="publish">
<property name="generated.ivy.file" value="${dist.dir}/ivy.xml" />
<ivy:deliver deliverpattern="${generated.ivy.file}"
organisation="${ivy.organisation}"
module="${ivy.module}" status="integration"
revision="${ivy.revision}"
pubrevision="SNAPSHOT"
conf="impl" />
<ivy:makepom ivyfile="${generated.ivy.file}"
pomfile="${dist.dir}/${ivy.module}.pom"/>
<ivy:publish resolver="nexus_snapshot" pubrevision="SNAPSHOT"
publishivy="false" status="integration" overwrite="true">
<artifacts pattern="${dist.dir}/[artifact].[ext]" />
<artifact name="${ivy.module}" type="pom" ext="pom"/>
</ivy:publish>
</target>
Note that I first generate an Ivy file for the current module (and my desired configuration) to create the POM from.
Related
I've read all the tutorials and examples, and still cannot publish a set of custom jars in my local Ivy repository.
Edit: Basically I want the same behavior as maven-install-plugin.
Here's my setup. I have an Ant task which produces the jars in a given folder. The folder name is not fixed but rather passed as a property in file. I want to get all the jars in this folder and install them in my local Ivy repo so that I can use them on a next step.
Here is my Ant from where I call the ivy:publish:
<project name="Install Ivy Dependencies" xmlns:ivy="antlib:org.apache.ivy.ant" basedir="." default="publish">
<loadproperties srcFile="path_to_folder.properties"/>
<property name="file_pattern" value="${path_to_folder}/[artifact].[ext]" />
<property name="pub_revision" value="1.0.0" />
<target name="resolve">
<ivy:configure file="ivysettings.xml" />
<ivy:resolve file="ivy.xml" />
</target>
<target name="retrieve-all" depends="resolve">
<ivy:retrieve pattern="${file_pattern}" conf="*" />
</target>
<target name="publish" depends="retrieve-all">
<ivy:publish resolver="local" organisation="myOrg" update="true" overwrite="true" pubrevision="${pub_revision}">
<artifacts pattern="${file_pattern}"/>
</ivy:publish>
</target>
</project>
Here's my ivysettings.xml:
<ivysettings>
<resolvers>
<filesystem name="local" local="true"/>
</resolvers>
</ivysettings>
And the ivy.xml:
<?xml version="1.0" encoding="ISO-8859-1"?>
<ivy-module version="2.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="http://ant.apache.org/ivy/schemas/ivy.xsd">
<info organisation="myOrg" module="myModule" revision="1.0.0"/>
<publications>
<artifact name="my-custom-jar" ext="jar" type="jar"/>
<artifact name="my-custom-jar-source" ext="jar" type="source"/>
</publications>
</ivy-module>
The error that I am getting when I call the ant task is:
impossible to publish artifacts for myOrg#myModule;1.0.0: java.lang.IllegalStateException: impossible to publish myOrg#myModule;1.0.0!my-custom-jar.jar using local: no artifact pattern defined
I've managed to run my scenario and to resolve my issues using this tutorial There were two major issues in my code/integration.
First one is that you cannot tell Ivy to publish the artifacts in its repository without providing a path to it. I did this with the filesystem resolver:
<filesystem name="local" local="true" transactional="local">
<ivy pattern="${ivy.default.ivy.user.dir}/local/[module]/ivy-[revision].xml" />
<artifact pattern="${ivy.default.ivy.user.dir}/local/[module]/[artifact]-[revision].[ext]" />
</filesystem>
The stupid think about it is this should be build in. If you copy it as is, then everything works. If the config is different, or pointing to a different location - nothing works and you are not told why. I read tons of docs about Apache Ivy and it was nowhere mentioned that these patterns should point to the local Ivy repository. I thought these were the paths from where the jars should be taken. I actually complained about this, but the Ivy documentation is very confusing. Also I think the examples there are wrong. Who would like to publish the artifacts in their ivy.settings.dir. In my case this directory was in my repository!
There was a second issue. It is a smaller one and again very hard to see and fix. There's something wrong the revision param and again the documentation is messed up. If you specify one and the same string for the revision and pub revision the artifacts aren't publish without any explanation why. I fixed it by removing the revision from ivy.xml file.
Last, but not least, I didn't manage to run successfully the "thing" as Ant task, but with java -jar $IVY_JAR ... Maybe the issue was because of the versions, but I was too tired to try it with the fix.
P.S.#cantSleepNow thanks for the help.
You need to add artifact pattern to resolver in ivysettings.xml, something like (example from ivy documentation):
<ivysettings>
<resolvers>
<filesystem name="local" local="true">
<ivy pattern="${ivy.settings.dir}/1/[organisation]/[module]/ivys/ivy-[revision].xml"/>
<artifact pattern="${ivy.settings.dir}/1/[organisation]/[module]/[type]s/[artifact]-[revision].[ext]"/>
</filesystem>
</resolvers>
</ivysettings>
I am using Ant to build my project and Ivy to resolve its dependencies. My project has a dependency which publishes snapshots to my internal Artifactory server.
If the dependency has released a new snapshot, and I do an <ivy:retrieve />, Ivy gets the new snapshot but keeps the previous snapshot around. So I have two versions of the dependency in my lib directory.
The dependency snapshots are named like depproject-1.0.0+23.jar where 23 is the build number. It is published at an address like http://artifactory.example.com/example-snapshots-local/com.example/depproject/1.0-SNAPSHOT/depproject-1.0.0+23.jar. This is not a Maven repository, and it is configured to store unique snapshots.
I am new to Ivy. Is this the expected behavior? How can I configure Ivy or Ant so that only the latest dependency snapshot is kept?
ivysettings.xml
<?xml version="1.0" encoding="UTF-8"?>
<ivy-settings>
<settings defaultResolver="main" />
<resolvers>
<chain name="main">
<ibiblio
name="artifactory-example-snapshots"
m2compatible="false"
root="http://artifactory.example.com/example-snapshots-local/"
pattern="[organization]/[module]/1.0-SNAPSHOT/[artifact]-[revision](-[classifier]).[ext]" />
<!-- more repos listed -->
</chain>
</resolvers>
</ivy-settings>
ivy.xml
<ivy-module version="2.0">
<info organisation="com.example" module="myproject" />
<dependencies>
<dependency org="com.example" name="depproject" rev="latest.integration" />
</dependencies>
</ivy-module>
I'm assuming you're using jars in the lib directory to create a classpath, something like:
<path id="compile.path">
<fileset dir="lib" includes="*.jar"/>
</path>
Your issue being multiple jars containing the same classes?
I think you have two options:
Use the ivy cachepath task to manage the build classpaths
Purge the lib directory, using he retrieve task to repopulate with the latest jars
The first option may appear more complicated, but it is actually a very powerful way to use ivy. For an example see:
How to avoid copying dependencies with Ivy
So, let's assume that I have an already installed SVN and installed ANT / Ivy locally.
I want to have the "shared" part of the ivy config point to some kind of share on a server. How would I need to set this up?
I know I have to dig through the ivy jar and pull out the ivysettings file and modify shared repositories.
So let's assume that I have a server on my intranet at MyServer.intranet.net and my team's folder was under /path/to/NetAdmin (thus the full path would be MyServer.intranet.net/path/to/NetAdmin ) How would I get this set up as a team repository for shared libraries? Would I just specify it and when I package the projects it writes the dependencies there?
Thanks
Here what I did:
I created a Subversion project called ivy.dir.
In this ivy.dir project, I have the latest ivy.jar.
In the ivy.dir, I have the ivysettings.xml setup for our environment. For example, we use a local Artifactory Maven repository for our own jars. The ivysettings.xml in the ivy.dir project points to that.
I created a file called ivy.tasks.xml. This is an Ant build file.
The ivy.tasks.xml looks like this:
<project name="Ivy.Tasks"
xmlns:ivy="http://ant.apache.org/ivy"
xmlns:jacoco="antlib:org.jacoco.ant">
<property environment="env"/>
<!-- Add Ivy Tasks -->
<taskdef uri="http://ant.apache.org/ivy"
resource="org/apache/ivy/ant/antlib.xml">
<classpath>
<fileset dir="${ivy.dir}">
<include name="ivy*.jar"/>
</fileset>
</classpath>
</taskdef>
<ivy:settings file="${ivy.dir}/ivysettings.xml"/>
</project>
Notice that I have my own Ivy settings, thank you. I didn't have to munge up the one in the ivy.jar (although I could have since everyone will use my ivy.jar file!). My ivysettings.xml looks like this:
<ivysettings>
<!-- I'll explain this part below -->
<property name="env.EXECUTOR_NUMBER" value="0" override="false"/>
<caches
defaultCacheDir="${ivy.default.ivy.user.dir}/cache-${env.EXECUTOR_NUMBER}"
resolutionCacheDir="${ivy.dir}/../target/ivy.cache"/>
<!-- Just the standard stuff you find in the `ivysettings.xml in the ivy.jar -->
<settings defaultResolver="default"/>
<include file="${ivy.dir}/ivysettings-public.xml"/> <!-- This one is different -->
<include url="${ivy.default.settings.dir}/ivysettings-shared.xml"/>
<include url="${ivy.default.settings.dir}/ivysettings-local.xml"/>
<include url="${ivy.default.settings.dir}/ivysettings-main-chain.xml"/>
<include url="${ivy.default.settings.dir}/ivysettings-default-chain.xml"/>
</ivysettings>
The big change is the ivysetting-public.xml file:
<ivysettings>
<resolvers>
<ibiblio name="public"
m2compatible="true"
checkmodified="true"
root="http://repos.vegicorp.com/artifactory/libs-release" />
</resolvers>
</ivysettings>
It's pointing to my local Maven repository -- my Artifactory server.
Now, for a developer to use Ivy, all they have to do is:
In the root of their project in Subversion, add a svn:external. This svn:external will be used to bring my ivy.dir project into their Subversion project.
In the build.xml
Add an Ivy namespace definition to their build.xml in the <project> definition.
Set the property ivy.dir to `${basedir}/ivy.dir.
Use the <import> task to import ${ivy.dir}/ivy.tasks.xml into their build.xml file.
Something like this:
<project name="post-a-matic" default="package" basedir="."
xmlns:ivy="http://ant.apache.org/ivy">
<property name="ivy.dir" value="${basedir}/ivy.dir"/>
<import file="${ivy.dir}/ivy.tasks.xml"/>
<!-- A whole bundle of properties are set -->
<target name="clean">
<delete dir="${target.dir}"/>
<ivy:cleancache/> <!-- Look: They have access to Ivy! -->
</target>
<target name="-resolve">
<ivy:resolve/>
</target>
<target name="compile"
depends="-resolve">
<ivy:cachpath
pathid="main.classpath"
conf="compile,provided"/>
<!-- Boy that's easy! -->
<javac srcdir="${main.srcdir}"
destdir="${main.destdir}"
classpathref="main.classpath"/>
</target>
<!-- On and on -->
This solves a lot of problems:
You can update the ivy.settings and everyone will have the updated settings. This ended up being very important to us because we use Jenkins and I wanted Jenkins to clean the ivy cache on each build. Whoops! That cleans out the ivy cache on builds that are being executed at the same time! I solved the problem by changing the ivysettings.xml file to define a different Ivy cache for each Jenkins build executor. One the Jenkins server, you have Ivy caches called $HOME/.ivy2/cache-0, $HOME/.ivy2/cache-1, etc. Each executor can delete it's own Ivy cache without affecting the others. Users, meanwhile will just have $HOME/.ivy2/cache-0.
You also can update Ivy when a new jar comes out. You update your Ivy jar file, and everyone gets the lated.
Big one of course is that Ivy installs itself when a project is checked out.
And an extra special bonus: You could use your ivy.dir and ivy.tasks.xml file to install other tasks. For example, each of our projects must run itself through Findbugs, PMD, CPD (part of the PMD project, Checkstyle, and use JaCoCo. for test coverage.
Each one of these projects consist of a jar file, and a <taskdef> to pull the task definitions into Ant. And, how do you use these tasks too? They're not defined in the standard Ant model. Developers don't know how to use them.
I've added these jars into my ivy.dir project, and installed all of those task definitions into my ivy.tasks.xml file. I also defined easy to use <macrodef> for most of these tasks, so it's easy for the developers to use them. In fact, I've even included the old Ant-Contrib tasks just for fun.
Now, once you add ivy.dir into your project, you have all of these extra tasks, and you have nothing to install on your machine.
You don't need to change the ivy jar. Just create a filesystem resolver in an ivysettings file and publish to this. Here's an example:
good ivy tutorial for local repository?
You'll find that ivy is very flexible and can support pretty much any mechanism for hosting files.
Personally, I'd consider installing a Maven repository manager like Nexus or Artifactory and use this to host both your builds dependencies and build outputs. In the long run it's a lot easier, especially if you're doing Java development.
I have a build.xml(ant based) which requires some jar from nexus to get copied in existing lib folder. i.e when it builds it should copy the jar from nexus with some version defined & then copy in lib & do compilation.
like happen in maven we define the artifact & its version . If changed will automatically download it from maven repo.
how can i do this in ant based builds?
experts pls advice.
I have taken the example listed in this thread one step further and created a macrodef to clean things up a bit for re-use. See below for downloading two artifacts from nexus (one snapshot, one release).
<project>
<target name="get-all">
<mkdir dir="lib" />
<nexus-get
groupId="foo.bar"
artifactId="some-artifact"
version="1.0.28"
repo="releases"
extension="jar"
dest="lib"
/>
<nexus-get
groupId="foo.bar"
artifactId="another-artifact"
version="1.0.0-SNAPSHOT"
repo="snapshots"
extension="jar"
dest="lib"
/>
</target>
<macrodef name="nexus-get">
<attribute name="groupId"/>
<attribute name="artifactId"/>
<attribute name="version"/>
<attribute name="repo"/>
<attribute name="extension"/>
<attribute name="dest"/>
<sequential>
<get src="http://my-nexus:9999/nexus/service/local/artifact/maven/redirect?r=#{repo}&g=#{groupId}&a=#{artifactId}&v=#{version}&e=#{extension}" dest="#{dest}/#{artifactId}.#{extension}" usetimestamp="true" />
</sequential>
</macrodef>
You would probably be interested in Ivy. It is a sub-project of Ant for dependency management. It is perfect for your situation because it can read Maven repositories and provides Ant tasks for downloading the published artifacts, constructing class paths from them, etc. It supports your use case of getting the most recent version of a dependency if you configure it to ask for the "latest.release" revision of the module.
Although there are surely specific ways to combine ant and maven the simplest thing (if you know the nexus URL and your artifact parameters to construct the download URL) would be just to use the ant Get task.
<project name="MyProject" default="resolveDependencies" basedir=".">
<target name="resolveDependencies">
<mkdir dir="lib" />
<get src="http://search.maven.org/remotecontent?filepath=log4j/log4j/1.2.9/log4j-1.2.9.jar" dest="lib/log4j-1.2.9.jar" usetimestamp="true" />
</target>
</project>
Perhaps use the Maven Ant Tasks.
As shown on http://maven.apache.org/ant-tasks/examples/dependencies.html
Can list dependencies in ant, and also do things like copy them
I think the section Using FileSets and the Version Mapper covers your need
You can use is filesetId, which will give you a fileset reference that can be used to copy files into a particular location. For example, to populate WEB-INF/lib with your dependencies you could use the following:
<artifact:dependencies filesetId="dependency.fileset" useScope="runtime">
<dependency groupId="junit" artifactId="junit" version="3.8.2" scope="test"/>
</artifact:dependencies>
<copy todir="${webapp.output}/WEB-INF/lib">
<fileset refid="dependency.fileset" />
<!-- This mapper strips off all leading directory information -->
<mapper type="flatten" />
</copy>
My goal is have my ant build script build a war file and include the jars that ivy knows this project depends on. The best code I could come up with at the moment is the following
<mkdir dir="dist/lib"/>
<ivy:retrieve pattern="dist/lib/[artifact].[ext]" sync="true"/>
<war destfile="dist/${ivy.module}.war" basedir="build" includes="**/*.class"
webxml="${war.webxml}">
<fileset dir="${war.web}"/>
<lib dir="dist/lib"/>
</war>
The problem with this code is it copies the jars twice. Once in to my dist/lib directory and again in to the war when it's created. It works but I can't shake the feeling there is a better way.
What I would like to do is something more like the following
<ivy:cachepath pathid="locpathref.classpath"/>
<war destfile="dist/${ivy.module}.war" basedir="build" includes="**/*.class"
webxml="${war.webxml}">
<fileset dir="${war.web}"/>
<lib refid="locpathref.classpath"/>
</war>
The problem is that the lib tag does not take in a refid of any kind. Any ideas or am I stuck with an extra set of file copies?
The problem here is that the lib tag is a custom fileset that targets it's files into the war archive's lib sub directory. It might be possible to write a custom war task but I don't think it's worth the effort.
If want to improve the manner in which ivy manages your war's dependencies might I suggest using configurations?
Create a configuration describing the run-time dependencies:
<ivy-module version="2.0">
<info organisation="apache" module="hello-ivy"/>
<configurations>
<conf name="build" description="Libraries needed to for compilation"/>
<conf name="war" extends="build" description="Libraries that should be included in the war file" />
</configurations>
<dependencies>
<dependency org="commons-lang" name="commons-lang" rev="2.0" conf="build->*,!sources,!javadoc"/>
<dependency org="commons-cli" name="commons-cli" rev="1.0" conf="build->*,!sources,!javadoc"/>
</dependencies>
</ivy-module>
Afterwards you retrieve them into a dedicated directory (using a pattern) which can be simply included using the war task's lib tag:
<ivy:retrieve pattern="${lib.dir}/[conf]/[artifact].[ext]"/>
<war destfile="${war.file}" webxml="${resources.dir}/web.xml">
<fileset dir="${resources.dir}" excludes="web.xml"/>
<lib dir="${lib.dir}/war"/>
</war>
The advantage of this approach is that you use the ivy conf attribute of each project dependency to ultimately decide if the jar gets included within the war file or not. The build file no longer cares.
In conclusion I understand that the point of your post was concern for multiple copies of your jar files... Using my suggested approach will further multiple your copies, but I would submit that this is not an issue provided you have a clean target to remove them afterwards.
If you're using Ant 1.8, you can use the technique described here:
http://www.beilers.com/2010/06/ivy-dependency-management-lessons-learned-and-ant-1-8-mapped-resources/
EXAMPLE:
<war destfile="${war.full.path}" webxml="WebContent/WEB-INF/web.xml" manifest="${manifest.path}">
<fileset dir="WebContent">
</fileset>
<classes dir="${build.dir}"/>
<mappedresources>
<restrict>
<path refid="classpath.CORE"/>
<type type="file"/>
</restrict>
<chainedmapper>
<flattenmapper/>
<globmapper from="*" to="WEB-INF/lib/*"/>
</chainedmapper>
</mappedresources>
<zipfileset dir="src" prefix="WEB-INF/classes">
<include name="**/resources/**/*.properties" />
<include name="**/resources/**/*.xml" />
</zipfileset>
</war>