How to publish custom jars to local Apache Ivy repository - ant

I've read all the tutorials and examples, and still cannot publish a set of custom jars in my local Ivy repository.
Edit: Basically I want the same behavior as maven-install-plugin.
Here's my setup. I have an Ant task which produces the jars in a given folder. The folder name is not fixed but rather passed as a property in file. I want to get all the jars in this folder and install them in my local Ivy repo so that I can use them on a next step.
Here is my Ant from where I call the ivy:publish:
<project name="Install Ivy Dependencies" xmlns:ivy="antlib:org.apache.ivy.ant" basedir="." default="publish">
<loadproperties srcFile="path_to_folder.properties"/>
<property name="file_pattern" value="${path_to_folder}/[artifact].[ext]" />
<property name="pub_revision" value="1.0.0" />
<target name="resolve">
<ivy:configure file="ivysettings.xml" />
<ivy:resolve file="ivy.xml" />
</target>
<target name="retrieve-all" depends="resolve">
<ivy:retrieve pattern="${file_pattern}" conf="*" />
</target>
<target name="publish" depends="retrieve-all">
<ivy:publish resolver="local" organisation="myOrg" update="true" overwrite="true" pubrevision="${pub_revision}">
<artifacts pattern="${file_pattern}"/>
</ivy:publish>
</target>
</project>
Here's my ivysettings.xml:
<ivysettings>
<resolvers>
<filesystem name="local" local="true"/>
</resolvers>
</ivysettings>
And the ivy.xml:
<?xml version="1.0" encoding="ISO-8859-1"?>
<ivy-module version="2.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="http://ant.apache.org/ivy/schemas/ivy.xsd">
<info organisation="myOrg" module="myModule" revision="1.0.0"/>
<publications>
<artifact name="my-custom-jar" ext="jar" type="jar"/>
<artifact name="my-custom-jar-source" ext="jar" type="source"/>
</publications>
</ivy-module>
The error that I am getting when I call the ant task is:
impossible to publish artifacts for myOrg#myModule;1.0.0: java.lang.IllegalStateException: impossible to publish myOrg#myModule;1.0.0!my-custom-jar.jar using local: no artifact pattern defined

I've managed to run my scenario and to resolve my issues using this tutorial There were two major issues in my code/integration.
First one is that you cannot tell Ivy to publish the artifacts in its repository without providing a path to it. I did this with the filesystem resolver:
<filesystem name="local" local="true" transactional="local">
<ivy pattern="${ivy.default.ivy.user.dir}/local/[module]/ivy-[revision].xml" />
<artifact pattern="${ivy.default.ivy.user.dir}/local/[module]/[artifact]-[revision].[ext]" />
</filesystem>
The stupid think about it is this should be build in. If you copy it as is, then everything works. If the config is different, or pointing to a different location - nothing works and you are not told why. I read tons of docs about Apache Ivy and it was nowhere mentioned that these patterns should point to the local Ivy repository. I thought these were the paths from where the jars should be taken. I actually complained about this, but the Ivy documentation is very confusing. Also I think the examples there are wrong. Who would like to publish the artifacts in their ivy.settings.dir. In my case this directory was in my repository!
There was a second issue. It is a smaller one and again very hard to see and fix. There's something wrong the revision param and again the documentation is messed up. If you specify one and the same string for the revision and pub revision the artifacts aren't publish without any explanation why. I fixed it by removing the revision from ivy.xml file.
Last, but not least, I didn't manage to run successfully the "thing" as Ant task, but with java -jar $IVY_JAR ... Maybe the issue was because of the versions, but I was too tired to try it with the fix.
P.S.#cantSleepNow thanks for the help.

You need to add artifact pattern to resolver in ivysettings.xml, something like (example from ivy documentation):
<ivysettings>
<resolvers>
<filesystem name="local" local="true">
<ivy pattern="${ivy.settings.dir}/1/[organisation]/[module]/ivys/ivy-[revision].xml"/>
<artifact pattern="${ivy.settings.dir}/1/[organisation]/[module]/[type]s/[artifact]-[revision].[ext]"/>
</filesystem>
</resolvers>
</ivysettings>

Related

optimal chain for resolve in ivy

I have some project's that integrates ivy, i publish my artifact in two possible repositories, local, that respond to a folder, and beta, that respond to a server where archiva is installed and where i share my jars with my colleague.
Now the problem: i need to find a way to build a resolution chains that do this things:
search between the various repository the latest.integration.
retrieve that jar.
if that jar is already in my cache do not have to download it.
if the artifact in the cache has the same version of the artifact in one of the repository take the latest in chronology's order.
now i have try everything that's the setting
Ivy setting
<chain name="resolvechain">
<ibiblio name="b1" root="archivaURLforbeta" m2compatible="true" checkmodified="true" latest="latest-time"/>
<filesystem name="b2" checkmodified="true" >
<artifact pattern="${ivy.local.default.root}/${ivy.local.default.artifact.pattern}" />
</filesystem>
<ibiblio name="b3" root="archivaURLforrelease" m2compatible="true" latest="latest-time"/>
<ibiblio name="b4" m2compatible="true"/>
</chain>
that's the target of resolve in the Build.xml file:
<ivy:settings file="${archiva.set}" />
<ivy:resolve refresh="true" resolveMode="dynamic" changing="true"/>
<ivy:retrieve sync="true" overwritemode="newer" pattern="./lib/[artifact]-[type]-[revision](.[ext])" />
and that's an example on how the dependency are written in ivy.xml file
<dependency org="organization" name="module-name" rev="latest.integration" transitive="false" conf="default" />
the questions are two:
First of all is there any error according to my needs?
Second, what i'm trying to do, is possible?
i asked these two questions because it seems to me that setting checkmodified true and changing true let my application skip entirely my cache, and my projects continuing to download the artifacts every time.
I have found the solution searching between different questions here in stackoverflow and in some issues solution in the ivy comunity with my colleagues.
The FileSystem need it's own latest-time strategy, every resolve take note of their own modules last date revision.
Here the mod in the code:
<filesystem name="b2" checkmodified="true" latest="latest-time">
<artifact pattern="${ivy.local.default.root}/${ivy.local.default.artifact.pattern}" />
</filesystem>

Keeping only the latest dependency snapshot using Ant and Ivy

I am using Ant to build my project and Ivy to resolve its dependencies. My project has a dependency which publishes snapshots to my internal Artifactory server.
If the dependency has released a new snapshot, and I do an <ivy:retrieve />, Ivy gets the new snapshot but keeps the previous snapshot around. So I have two versions of the dependency in my lib directory.
The dependency snapshots are named like depproject-1.0.0+23.jar where 23 is the build number. It is published at an address like http://artifactory.example.com/example-snapshots-local/com.example/depproject/1.0-SNAPSHOT/depproject-1.0.0+23.jar. This is not a Maven repository, and it is configured to store unique snapshots.
I am new to Ivy. Is this the expected behavior? How can I configure Ivy or Ant so that only the latest dependency snapshot is kept?
ivysettings.xml
<?xml version="1.0" encoding="UTF-8"?>
<ivy-settings>
<settings defaultResolver="main" />
<resolvers>
<chain name="main">
<ibiblio
name="artifactory-example-snapshots"
m2compatible="false"
root="http://artifactory.example.com/example-snapshots-local/"
pattern="[organization]/[module]/1.0-SNAPSHOT/[artifact]-[revision](-[classifier]).[ext]" />
<!-- more repos listed -->
</chain>
</resolvers>
</ivy-settings>
ivy.xml
<ivy-module version="2.0">
<info organisation="com.example" module="myproject" />
<dependencies>
<dependency org="com.example" name="depproject" rev="latest.integration" />
</dependencies>
</ivy-module>
I'm assuming you're using jars in the lib directory to create a classpath, something like:
<path id="compile.path">
<fileset dir="lib" includes="*.jar"/>
</path>
Your issue being multiple jars containing the same classes?
I think you have two options:
Use the ivy cachepath task to manage the build classpaths
Purge the lib directory, using he retrieve task to repopulate with the latest jars
The first option may appear more complicated, but it is actually a very powerful way to use ivy. For an example see:
How to avoid copying dependencies with Ivy

Ivy Resolve of "latest.integration" fails when run from Jenkins

so I have an existing ant build and I'd like to resolve its dependencies using Ivy. I have an ivy.xml and ivysettings.xml file which both work with IvyDE in Eclipse and also work when run from the command line against the ivy-2.3.0.jar. (They work as in Ivy reports it has downloaded the dependencies).
The problem I have is that they do not work when running from the any build script when the 'rev' attribute is listed as 'latest.integration'. If I specify a version then the dependency is resolved. To be clear this specific dependency is an internal library that is hosted by Artifactory.
I'm fairly stumped as to what I can change or try to identify the problem. I get the same failure when running an ivy findrevision.
To note Jenkins is running on a mac, but I can use the Ivy jar directly to resolve the dependencies using the same ivy files on that machine, the problem is only related to when resolving through Jenkins.
I realise this isn't likely to be something someone can look at and point directly to the problem, especially given that in seemingly all other situations, even running the snippet from the ant build through eclipse, the resolve works correctly. But if anyone has any suggestions they'd be more than welcome!
Anyway my ivy.xml
<ivy-module version="2.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="http://ant.apache.org/ivy/schemas/ivy.xsd">
<info
organisation="MyOrg"
module="MyModule"
status="integration">
</info>
<dependencies>
<dependency org="MyOrg" name="DEP" rev="latest.integration">
<artifact name="DEP" type="jar" ext="jar" />
</dependency>
<dependency org="junit" name="junit" rev="4.8.1" />
</dependencies>
</ivy-module>
And my ivysettings.xml (The dependency is named for example DEP-0.1-SNAPSHOT.jar)
<?xml version="1.0" encoding="UTF-8"?>
<ivy-settings>
<settings defaultResolver="snapshot" />
<!--Authentication required for publishing (deployment). 'Artifactory Realm' is the realm used by Artifactory so don't change it.-->
<credentials host="HOST" realm="Artifactory Realm" username="USR" passwd="PASS" />
<resolvers>
<chain name="main">
<!-- <ibiblio name="release" m2compatible="true" root="http://HOST/artifactory/libs-release" /> -->
<chain name="snapshot" changingPattern="0.*" checkmodified="true">
<ibiblio name="public" m2compatible="true" root="http://HOST/artifactory/libs-snapshot" />
</chain>
</chain>
</resolvers>
</ivy-settings>
And finally my build script looks something like
<project name="MyModule Continuous Integration" xmlns:ivy="antlib:org.apache.ivy.ant">
....
<target name = "resolveDependencies" >
<ivy:configure file="/MyModule/ivysettings.xml" />
<ivy:findrevision organisation="MyOrg" module="DEP" revision="latest.integration" property="found.revision" />
<echo message="Found revision: ${found.revision}" />
<ivy:resolve file="/MyModule/ivy.xml" refresh="true" type="jar" />
<ivy:cachepath pathid="ivy.dependencies" />
</target>
....
</project>
And for completion the sort of output I'm getting from the Jenkins build is:
[ivy:findrevision] snapshot: Checking cache for: dependency: MyOrg#DEP;latest.integration {}
[ivy:findrevision] don't use cache for MyOrg#DEP;latest.integration: checkModified=true
[ivy:findrevision] default-cache: no cached resolved revision for MyOrg#DEP;latest.integration
[ivy:findrevision] tried http://HOST/artifactory/libs-snapshot/MyOrg/DEP/[revision]/DEP-[revision].pom
[ivy:findrevision] CLIENT ERROR: Not Found url=http://HOST/artifactory/libs-snapshot/MyOrg/DEP/maven-metadata.xml
[ivy:findrevision] maven-metadata not available: http://HOST/artifactory/libs-snapshot/MyOrg/DEP/maven-metadata.xml
[ivy:findrevision] WARN: problem while listing resources in http://HOST/artifactory/libs-snapshot/MyOrg/DEP/ with public:
[ivy:findrevision] WARN: java.lang.NullPointerException null
[ivy:findrevision] tried http://HOST/artifactory/libs-snapshot/MyOrg/DEP/[revision]/DEP-[revision].jar
[ivy:findrevision] maven-metadata not available: http://HOST/artifactory/libs-snapshot/MyOrg/DEP/maven-metadata.xml
[ivy:findrevision] WARN: problem while listing resources in http://HOST/artifactory/libs-snapshot/MyOrg/DEP/ with public:
[ivy:findrevision] WARN: java.lang.NullPointerException null
[ivy:findrevision] public: no ivy file nor artifact found for MyOrg#DEP;latest.integration
Property "found.revision" has not been set
[echo] Found revision: ${found.revision}
Thanks again for any ideas,
Dan.
For future reference, I have managed to resolve this by swapping to the latest Ivy 2.4 alpha build. Seems it must have been a problem with 2.3.0.

Ivy, what is the master configuration and why is it not pulling jvyaml?

I have the following ivy file:
<configurations defaultconfmapping="buildtime">
<conf name="buildtime" visibility="private" description="Libraries needed only for compilation" />
<conf name="runtime" description="Libraries only needed at runtime" />
<conf name="test" description="Libraries only needed for testing" />
</configurations>
<dependencies>
<dependency org="net.java.dev" name="jvyaml" rev="0.2.1" conf="runtime" />
<dependency org="org.apache.solr" name="solr-core" rev="3.6.0" conf="runtime" />
</dependencies>
and I have an ant retrieve task that looks like this:
<target name="retrieve-all" depends="resolve">
<ivy:retrieve pattern="lib/[conf]/[artifact]-[revision].[ext]" conf="*" />
</target>
The weird thing is, that all the solr dependencies download into lib/runtime as I'd expect, but the jvyaml module does not! It 'resolves', but will not download into the lib/runtime directory unless I change the dependency declaration to:
<dependency org="net.java.dev" name="jvyaml" rev="0.2.1" conf="runtime->master" />
What is this master configuration and why is it needed to pull the jvyaml jar, but not solr?
Thanks
I would suggest restructuring your configurations as follows:
<ivy-module version="2.0">
<info organisation="com.myspotontheweb" module="demo"/>
<configurations>
<conf name="compile" description="Libraries needed only for compilation" />
<conf name="runtime" description="Libraries only needed at runtime" extends="compile" />
<conf name="test" description="Libraries only needed for testing" extends="runtime" />
</configurations>
<dependencies>
<dependency org="net.java.dev" name="jvyaml" rev="0.2.1" conf="runtime->default" />
<dependency org="org.apache.solr" name="solr-core" rev="3.6.0" conf="runtime->default" />
</dependencies>
</ivy-module>
Important changes introduced:
Use the more standard "compile" configuration
Configuration inheritance using the "extends" attribute. Compile dependencies can then be automatically included in both the runtime and test configurations.
Use configuration mappings, for example: conf="runtime->default". This makes it obvious which local configuration is associated with which remote configuration.
Configuration mappings explained
Configurations are a powerful ivy feature. When ivy downloads Maven modules it performs an internal translation and assigns a standard set of configurations, listed in this answer:
How are maven scopes mapped to ivy configurations by ivy
When declaring a dependency it's a good idea to always make use of a configuration mapping, so that there is no doubt where the dependencies artifacts are assigned.
For example:
<dependency org="??" name="??" rev="??" conf="runtime->default" />
Here we're saying we want the remote module's default dependencies associated with our local runtime configuration.
In practice, there are only two remote configuration mappings you'll actually need:
default: The remote module's artifact and all it's runtime transitive dependencies
master: The remote module's artifact only (No transitive dependencies)
In conclusion, I think your problem was caused by the fact that the remote Maven module's "runtime" scope does not include Maven module's artifact, instead you were getting the non-existant transitive dependencies of the module jvyaml :-(
Some additional advice
I'd also suggest generating an ivy dependency management report, as follows:
<target name="init" description="Resolve dependencies and populate lib dir">
<ivy:resolve/>
<ivy:report todir="${build.dir}/ivy-report" graph="false"/>
<ivy:retrieve pattern="lib/[conf]/[artifact]-[revision].[ext]"/>
</target>
The report will help explain how each dependency ends up on different configurations. Also really useful for determining how transitive dependencies are being managed.
And finally, here's where the configuration inheritance pays off, creating ivy managed ANT classpaths:
<target name="init" description="Resolve dependencies and set classpaths">
<ivy:resolve/>
<ivy:report todir="${build.dir}/ivy-report" graph="false"/>
<ivy:cachepath pathid="compile.path" conf="compile"/>
<ivy:cachepath pathid="runtime.path" conf="runtime"/>
<ivy:cachepath pathid="test.path" conf="test"/>
</target>
Notice that the original solr-core is not retrieved either.
After your resolve, go the cache and check the ivy.xml files for both modules.
You will see that they publish their artifacts in conf=master only
<artifact name="jvyaml" type="jar" ext="jar" conf="master"/>
<artifact name="solr-core" type="jar" ext="jar" conf="master"/>
Which means, you have to do explicit configuration mapping to denote that your builtime configuration should evoke the 'master' configuration of your dependencies. (check configuration mapping).
HOWEVER, the dependencies of the solr-core, have the configuration mapping as you could see in the ivy.xml file:
<dependency org="org.apache.solr" name="solr-solrj" rev="3.6.0" force="true" conf="compile->compile(*),master(*);runtime->runtime(*)"/>
I think it's the master(*) thingy.
What I usually do is in my own ivy.xml file when I declare dependencies I do the mapping:
<dependency org="net.java.dev" name="jvyaml" rev="0.2.1" conf="runtime->master" />
This one says the runtime be evoking the master configuration in the designated dependency.
You could do
conf="runtime,test->master"
as well

Ivy: Retrieving Snapshot published in Nexus and its dependencies

I am using Ivy to publish a snapshot of a built Jar to a locally hosted Nexus repository using the following Ant target.
<target name="publish">
<ivy:publish resolver="nexus_snapshot" pubrevision="SNAPSHOT" overwrite="true">
<artifacts pattern="${dist.dir}/[artifact].[ext]" />
</ivy:publish>
</target>
This appears to work fine, resulting in the Jar and its associated ivy.xml being present in the repository (with filenames mymodule-SNAPSHOT.jar and ivy-SNAPSHOT.jar).
Later, in another build script, I wish to retrieve the Jar and its associated dependencies (i.e. as specified in its ivy.xml) into a directory.
This is the Ant target I'm using.
<target name="deploy">
<delete dir="deploy" />
<mkdir dir="deploy" />
<ivy:settings file="${ivy.dir}/ivy_deploy_settings.xml" />
<ivy:retrieve organisation="myorg" module="mymodule"
inline="true" revision="SNAPSHOT" pattern="deploy/[artifact].[ext]"/>
</target>
This retrieves the Jar to the directory, but not its dependencies. Also, if I add
conf="impl"
to the retrieve, it fails as the configuration is not found.
As such, it seems that the retrieve is simply not referencing the ivy.xml and hence not resolving the dependencies.
Should this work or am I misunderstanding something?
I have now resolved this problem. I believe the issue is simply that Nexus works using POM files rather than Ivy files (by default at least - I can't see any relevant configuration options).
The solution is therefore to generate a suitable POM and publish this along with the Jar.
<target name="publish">
<property name="generated.ivy.file" value="${dist.dir}/ivy.xml" />
<ivy:deliver deliverpattern="${generated.ivy.file}"
organisation="${ivy.organisation}"
module="${ivy.module}" status="integration"
revision="${ivy.revision}"
pubrevision="SNAPSHOT"
conf="impl" />
<ivy:makepom ivyfile="${generated.ivy.file}"
pomfile="${dist.dir}/${ivy.module}.pom"/>
<ivy:publish resolver="nexus_snapshot" pubrevision="SNAPSHOT"
publishivy="false" status="integration" overwrite="true">
<artifacts pattern="${dist.dir}/[artifact].[ext]" />
<artifact name="${ivy.module}" type="pom" ext="pom"/>
</ivy:publish>
</target>
Note that I first generate an Ivy file for the current module (and my desired configuration) to create the POM from.

Resources