ivy cannot d/l file but wget can - ant

My ivy dependency:
<dependency org="googlecode.com" name="jslint4java" rev="2.0.1" e:suffix="-src">
<artifact name="jslint4java" type="zip"/>
</dependency>
My ivy settings:
<url name="googlecode">
<artifact pattern="http://[module].[organization]/files/[artifact]-[revision][suffix].[ext]"/>
</url>
...
<module organisation="googlecode.com" resolver="googlecode"/>
When I try to resolve the dependencies, I got the following error:
[ivy:retrieve] == resolving dependencies
...#blog;working#...->googlecode.com#jslint4java;2.0.2
[default->*] [ivy:retrieve] tried
http://jslint4java.googlecode.com/files/jslint4java-2.0.2-dist.zip
[ivy:retrieve] CLIENT ERROR: Not Found
url=http://jslint4java.googlecode.com/files/jslint4java-2.0.2-dist.zip
[ivy:retrieve] googlecode: no ivy file nor artifact found for
googlecode.com#jslint4java;2.0.2 [ivy:retrieve] WARN: module not found: googlecode.com#jslint4java;2.0.2 [ivy:retrieve] WARN: ====
googlecode: tried [ivy:retrieve] WARN: -- artifact
googlecode.com#jslint4java;2.0.2!jslint4java.zip: [ivy:retrieve] WARN:
http://jslint4java.googlecode.com/files/jslint4java-2.0.2-dist.zip
I tried to d/l the file by using the url in the last line with wget and the it went well.
But I cannot figure out why ivy failed to d/l the file.

So, I ran the ant task with -d flag to check more info and that's the outcome:
[ivy:resolve] HTTP response status: 404 url=http://jslint4java.googlecode.com/files/jslint4java-2.0.1-src.zip.sha1
[ivy:resolve] CLIENT ERROR: Not Found url=http://jslint4java.googlecode.com/files/jslint4java-2.0.1-src.zip.sha1
[ivy:resolve] HTTP response status: 404 url=http://jslint4java.googlecode.com/files/jslint4java-2.0.1-src.zip.md5
[ivy:resolve] CLIENT ERROR: Not Found url=http://jslint4java.googlecode.com/files/jslint4java-2.0.1-src.zip.md5
Notice that md5 doesn't exist, you have to override the checksums option in the resolver.
<url name="googlecode" checksums="sha1">
I tried the sha1 ONLY, (the default is sha1,md5) and it worked, even though the sha1 failed to download as well.
try to experiment in that area.

I suggest you setup a configuration in your ivy file.
Example
After running the build the following files are present:
|-- build.xml
|-- ivysettings.xml
|-- ivy.xml
`-- lib
`-- jslint4java-2.0.1-src.zip
ivy.xml
<ivy-module version="2.0" xmlns:e="http://ant.apache.org/ivy/extra">
<info organisation="com.myspotontheweb" module="demo"/>
<configurations>
<conf name="source" description="source code distro"/>
</configurations>
<dependencies>
<dependency org="googlecode.com" name="jslint4java" rev="2.0.1" e:suffix="src" conf="source->default">
<artifact name="jslint4java" type="zip"/>
</dependency>
</dependencies>
</ivy-module>
Notes:
Added a "source" configuration. Logical grouping within my ivy project to differentiate from other kinds of dependency.
Note the extra "conf" attribute on the dependency. Mapped to the remote "default" configuration. Unless the remote module has an ivy or POM file default is the safest option.
Revised the extra attribute "suffix". Removed the "-" character. Not a big deal technically, I just think it's best not included in the dependency declaration.
ivysettings.xml
<ivysettings>
<settings defaultResolver="googlecode" />
<resolvers>
<url name="googlecode">
<artifact pattern="http://[module].[organization]/files/[artifact]-[revision]-[suffix].[ext]"/>
</url>
</resolvers>
</ivysettings>
Notes:
Personally I would explicitly state the hostname of the remote repository, here in the settings file (Instead of taking it from the dependency information). This will work though.
build.xml
<project name="demo" default="retrieve" xmlns:ivy="antlib:org.apache.ivy.ant">
<target name="retrieve" description="Use ivy to retrieve artifacts">
<ivy:retrieve pattern="lib/[artifact]-[revision](-[suffix]).[ext]" conf="source"/>
</target>
<target name="clean" description="Cleanup build files">
<delete dir="lib"/>
</target>
<target name="clean-all" depends="clean" description="Additionally purge ivy cache">
<ivy:cleancache/>
</target>
</project>
Notes:
Note how the "suffix" attribute is enclosed within parenthesis. This is an optional extra parameter and may not be part of a dependency.

Related

Use Grab to Download Ivy Dependency Jar when Artifact name is not the same as Module name

Within a Jenkins groovy script I'm trying to download a dependency using the following:
#Grab(group='myorg', module='SuiteCreator', version='1.16.1', conf='jar', transitive=false)
import myorg.myorgAPI
I have a /home/jenkins/.groovy/grapeConfig.xml file with the following:
<?xml version="1.0" encoding="UTF-8"?>
<ivy-settings>
<settings defaultResolver="downloadGrapes"/>
<resolvers>
<chain name="downloadGrapes">
<sftp user="admin" userPassword="pw" host="ivy.myorg.com" name="myrepository" checkmodified="true">
<ivy pattern="/data/ivy/repo/[organisation]/[module]/[branch]/[revision]/ivy-[revision].xml"/>
<artifact pattern="/data/ivy/repo/[organisation]/[module]/[branch]/[revision]/[artifact]-[revision].[ext]"/>
</sftp>
</chain>
</resolvers>
</ivy-settings>
The ivy-1.16.1.xml of the Module I've trying to grab:
<?xml version="1.0" encoding="UTF-8"?>
<ivy-module version="1.0">
<info organisation="myorg" module="SuiteCreator" branch="master" revision="1.16.1" status="release" publication="20190417105814"/>
<configurations>
<conf name="jar" description="Distribution jar"/>
</configurations>
<publications>
<artifact name="myorg-suitecreator" type="jar" ext="jar" conf="jar"/>
</publications>
</ivy-module>
So I'm just trying to grab the artifact: myorg-suitecreator-1.16.1.jar.
When I run my groovy script in Jenkins I get the following error:
2019.07.09 18:06:15 org.codehaus.groovy.control.MultipleCompilationErrorsException:
startup failed:
2019.07.09 18:06:15 General error during conversion: Error grabbing Grapes -- [download failed:
myorg#SuiteCreator#master;1.16.1!SuiteCreator.jar]
2019.07.09 18:06:15
2019.07.09 18:06:15 java.lang.RuntimeException: Error grabbing Grapes -- [download failed: myorg#SuiteCreator#master;1.16.1!SuiteCreator.jar]
2019.07.09 18:06:15 at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
From the error it seems Grape is assuming the Ivy Artifact Name is the same as the Module name. The ivy-1.16.1.xml Artifact Name at: /ivy-module/publications/artifact/#name is defined as myorg-suitecreator However Grab appears to be attempt to download: SuiteCreator.jar.
The artifact pattern in grapeConfig.xml is:
<artifact pattern="/data/ivy/repo/[organisation]/[module]/[branch]/[revision]/[artifact]-[revision].[ext]"/>
And so the file I'm trying to grab is actually: /data/ivy/repo/myorg/SuiteCreator/1.16.1/myorg-suitecreator-1.16.1.jar
Does anyone have any suggestions on how to get this to work (or if Grab can download artifacts with different artifact name to the module name from Ivy?).
I gave up trying to use Grab to achieve this. I found another limitation of Grab is that it doesn't allow the specification of the branch of the artifact you wish to retrieve. I realise that not having releases on a master branch or a single release branch may not be best practice, but we do have this requirement in our dev environment.
Instead I simply used an Invoke Ant build step within Jenkins to retrieve my Ivy artifact. We use ANT already in our dev process so this was not difficult.
The ANT build.xml script is located in the same git repository as the Groovy script I wish to run. The retrieve-suite-creator target is simply an ivy-retrieve
<target name="retrieve-suite-creator" depends="clean, install-ivy">
<ivy:retrieve conf="suite-creator" type="jar" pattern="${build.dir}/[artifact].[ext]" log="${ivy.resolve.log}" settingsRef="${ivy.build.settings}"/>
</target>
Using my ivy.xml (again in the same repo as the groovy script):
<ivy-module version="1.0">
<info organisation="myorg" module="MyAutomation" status="integration" branch="${ivy.branch}"/>
<configurations>
<conf name="suite-creator" description="Configuration for Suite Creator"/>
</configurations>
<dependencies>
<dependency org="myorg" name="SuiteCreator" branch="mybranch" rev="1.16.1" conf="suite-creator->suite-creator" changing="true"/>
</dependencies>
</ivy-module>
I had to add the suite-creator ivy configuration to the SuiteCreator module's ivy.xml (in a separate SuiteCreator Git repo). I couldn't use the existing jar configuration as this also downloaded all the transitive dependencies which I didn't need.
<ivy-module version="1.0">
<info organisation="myorg" module="SuiteCreator" status="integration" branch="${ivy.branch}"/>
<configurations>
<!-- Build configurations -->
<conf name="build" description="Classes used in jar compilation"/>
<conf name="jar" description="Distribution jar"/>
<conf name="suite-creator" description="Just the myorg-suitecreator.jar"/>
</configurations>
<publications>
<artifact name="myorg-suitecreator" type="jar" ext="jar" conf="jar,suite-creator"/>
</publications>
<dependencies>
...
</dependencies>
</ivy-module>
Finally in my Jenkins job after the Invoke Ant build step, I then had an Execute Groovy Script build step, where I had to add the downloaded jar to my Class path.

Ivy Resolve of "latest.integration" fails when run from Jenkins

so I have an existing ant build and I'd like to resolve its dependencies using Ivy. I have an ivy.xml and ivysettings.xml file which both work with IvyDE in Eclipse and also work when run from the command line against the ivy-2.3.0.jar. (They work as in Ivy reports it has downloaded the dependencies).
The problem I have is that they do not work when running from the any build script when the 'rev' attribute is listed as 'latest.integration'. If I specify a version then the dependency is resolved. To be clear this specific dependency is an internal library that is hosted by Artifactory.
I'm fairly stumped as to what I can change or try to identify the problem. I get the same failure when running an ivy findrevision.
To note Jenkins is running on a mac, but I can use the Ivy jar directly to resolve the dependencies using the same ivy files on that machine, the problem is only related to when resolving through Jenkins.
I realise this isn't likely to be something someone can look at and point directly to the problem, especially given that in seemingly all other situations, even running the snippet from the ant build through eclipse, the resolve works correctly. But if anyone has any suggestions they'd be more than welcome!
Anyway my ivy.xml
<ivy-module version="2.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="http://ant.apache.org/ivy/schemas/ivy.xsd">
<info
organisation="MyOrg"
module="MyModule"
status="integration">
</info>
<dependencies>
<dependency org="MyOrg" name="DEP" rev="latest.integration">
<artifact name="DEP" type="jar" ext="jar" />
</dependency>
<dependency org="junit" name="junit" rev="4.8.1" />
</dependencies>
</ivy-module>
And my ivysettings.xml (The dependency is named for example DEP-0.1-SNAPSHOT.jar)
<?xml version="1.0" encoding="UTF-8"?>
<ivy-settings>
<settings defaultResolver="snapshot" />
<!--Authentication required for publishing (deployment). 'Artifactory Realm' is the realm used by Artifactory so don't change it.-->
<credentials host="HOST" realm="Artifactory Realm" username="USR" passwd="PASS" />
<resolvers>
<chain name="main">
<!-- <ibiblio name="release" m2compatible="true" root="http://HOST/artifactory/libs-release" /> -->
<chain name="snapshot" changingPattern="0.*" checkmodified="true">
<ibiblio name="public" m2compatible="true" root="http://HOST/artifactory/libs-snapshot" />
</chain>
</chain>
</resolvers>
</ivy-settings>
And finally my build script looks something like
<project name="MyModule Continuous Integration" xmlns:ivy="antlib:org.apache.ivy.ant">
....
<target name = "resolveDependencies" >
<ivy:configure file="/MyModule/ivysettings.xml" />
<ivy:findrevision organisation="MyOrg" module="DEP" revision="latest.integration" property="found.revision" />
<echo message="Found revision: ${found.revision}" />
<ivy:resolve file="/MyModule/ivy.xml" refresh="true" type="jar" />
<ivy:cachepath pathid="ivy.dependencies" />
</target>
....
</project>
And for completion the sort of output I'm getting from the Jenkins build is:
[ivy:findrevision] snapshot: Checking cache for: dependency: MyOrg#DEP;latest.integration {}
[ivy:findrevision] don't use cache for MyOrg#DEP;latest.integration: checkModified=true
[ivy:findrevision] default-cache: no cached resolved revision for MyOrg#DEP;latest.integration
[ivy:findrevision] tried http://HOST/artifactory/libs-snapshot/MyOrg/DEP/[revision]/DEP-[revision].pom
[ivy:findrevision] CLIENT ERROR: Not Found url=http://HOST/artifactory/libs-snapshot/MyOrg/DEP/maven-metadata.xml
[ivy:findrevision] maven-metadata not available: http://HOST/artifactory/libs-snapshot/MyOrg/DEP/maven-metadata.xml
[ivy:findrevision] WARN: problem while listing resources in http://HOST/artifactory/libs-snapshot/MyOrg/DEP/ with public:
[ivy:findrevision] WARN: java.lang.NullPointerException null
[ivy:findrevision] tried http://HOST/artifactory/libs-snapshot/MyOrg/DEP/[revision]/DEP-[revision].jar
[ivy:findrevision] maven-metadata not available: http://HOST/artifactory/libs-snapshot/MyOrg/DEP/maven-metadata.xml
[ivy:findrevision] WARN: problem while listing resources in http://HOST/artifactory/libs-snapshot/MyOrg/DEP/ with public:
[ivy:findrevision] WARN: java.lang.NullPointerException null
[ivy:findrevision] public: no ivy file nor artifact found for MyOrg#DEP;latest.integration
Property "found.revision" has not been set
[echo] Found revision: ${found.revision}
Thanks again for any ideas,
Dan.
For future reference, I have managed to resolve this by swapping to the latest Ivy 2.4 alpha build. Seems it must have been a problem with 2.3.0.

Why isn't Ivy downloading an artifact from our intranet repo?

I've had a prototype Ivy build working reasonably well. I just looked at it today and I'm seeing that it's finding an artifact on my intranet repo but not downloading it to the local cache or retrieving it in my local build, which causes the build to fail.
The build specifies several other dependencies, most of which are found on mavencentral, and one in another repo in the same local intranet repo that it's finding (but not downloading) the other artifact.
I've tried a few times to clear out the ivy cache and run this again, but it downloads all of the artifacts except for this particular one.
First, here is the relevant output from the build, with some minor pieces elided:
install-dependencies:
:: Apache Ivy 2.3.0 - 20130110142753 :: http://ant.apache.org/ivy/ ::
:: loading settings :: file = <pathtoivysettingsxmlfile>
:: resolving dependencies :: com.att.ecom.poc#coherence_poc;working#<hostname> [not transitive]
confs: [default]
found com.att.ecom.poc#poc-domain-model;0.0.1-SNAPSHOT in mavenCentralSnapshots
found org.apache.commons#commons-lang3;3.1 in central
found org.springframework#spring-aop;4.0.0.RELEASE in central
found org.springframework#spring-beans;4.0.0.RELEASE in central
found org.springframework#spring-context;4.0.0.RELEASE in central
found org.springframework#spring-core;4.0.0.RELEASE in central
found org.springframework#spring-expression;4.0.0.RELEASE in central
found org.springframework#spring-web;4.0.0.RELEASE in central
found com.oracle.coherence#coherence;12.1.2-0-0 in mavenCentralThirdparty
:: resolution report :: resolve 351ms :: artifacts dl 8ms
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 9 | 0 | 0 | 0 || 8 | 0 |
---------------------------------------------------------------------
:: retrieving :: com.att.ecom.poc#coherence_poc
confs: [default]
0 artifacts copied, 8 already retrieved (0kB/5ms)
The repositories "mavenCentralSnapshots" and "mavenCentralThirdparty" are actually on our local intranet repo (which is also confusedly called "maven central").
Notice that it says there are 9 modules, but only 8 were downloaded. The first artifact in the list, "poc-domain-model" is the one that is not being downloaded, but I don't see why.
Here is my "ivysettings.xml" file:
<ivysettings>
<settings defaultResolver="default"/>
<property name="m2-pattern" value="${user.home}/.m2/repository/[organisation]/[module]/[revision]/[module]-[revision](-[classifier]).[ext]" override="false" />
<resolvers>
<chain name="default">
<filesystem name="local-maven2" m2compatible="true" >
<artifact pattern="${m2-pattern}"/>
<ivy pattern="${m2-pattern}"/>
</filesystem>
<ibiblio name="mavenCentralSnapshots" m2compatible="true"
root="http://<hostandport>/nexus/content/repositories/cditspoc-snapshots"/>
<ibiblio name="mavenCentralThirdparty" m2compatible="true"
root="http://<hostandport>/nexus/content/repositories/cditspoc-3rd-party"/>
<ibiblio name="central" m2compatible="true"/>
</chain>
</resolvers>
</ivysettings>
And here is my "ivy.xml":
<ivy-module version="2.0">
<info organisation="com.att.ecom.poc" module="coherence_poc"/>
<dependencies>
<dependency org="com.att.ecom.poc" name="poc-domain-model" rev="0.0.1-SNAPSHOT"/>
<dependency org="org.apache.commons" name="commons-lang3" rev="3.1"/>
<dependency org="org.springframework" name="spring-aop" rev="4.0.0.RELEASE"/>
<dependency org="org.springframework" name="spring-beans" rev="4.0.0.RELEASE"/>
<dependency org="org.springframework" name="spring-context" rev="4.0.0.RELEASE"/>
<dependency org="org.springframework" name="spring-core" rev="4.0.0.RELEASE"/>
<dependency org="org.springframework" name="spring-expression" rev="4.0.0.RELEASE"/>
<dependency org="org.springframework" name="spring-web" rev="4.0.0.RELEASE"/>
<dependency org="com.oracle.coherence" name="coherence" rev="12.1.2-0-0"/>
</dependencies>
</ivy-module>
And finally, here's the excerpt from my build.xml that calls Ivy:
<target name="install-dependencies">
<ivy:resolve transitive="false" type="jar"/>
<ivy:retrieve conf="*" type="jar" pattern="${basedir}/lib/[artifact]-[type]-[revision].[ext]"/>
</target>
Is it obvious what my problem is, or is there something I can do to get more information?
Update:
Here is an elided version of my "com.att.ecom.poc-coherence_poc-default.xml" file:
<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="ivy-report.xsl"?>
<ivy-report version="1.0">
<info
organisation="com.att.ecom.poc"
module="coherence_poc"
revision="working#<hostname>"
conf="default"
confs="default"
date="20140210084517"/>
<dependencies>
<module organisation="com.oracle.coherence" name="coherence">
<revision name="12.1.2-0-0" status="release" pubdate="20131118143843" resolver="mavenCentralThirdparty" artresolver="mavenCentralThirdparty" homepage="" downloaded="false" searched="false" default="false" conf="system, default, optional, compile, *, provided, runtime, javadoc, sources, master" position="8">
<metadata-artifact status="no" details="" size="2283" time="0" location="<homedir>\.ivy2\cache\com.oracle.coherence\coherence\ivy-12.1.2-0-0.xml" searched="false" origin-is-local="false" origin-location="<intranetmavenrepo>/nexus/content/repositories/cditspoc-3rd-party/com/oracle/coherence/coherence/12.1.2-0-0/coherence-12.1.2-0-0.pom"/>
<caller organisation="com.att.ecom.poc" name="coherence_poc" conf="default" rev="12.1.2-0-0" rev-constraint-default="12.1.2-0-0" rev-constraint-dynamic="12.1.2-0-0" callerrev="working#<hostname>"/>
<artifacts>
<artifact name="coherence" type="jar" ext="jar" status="no" details="" size="7027491" time="0" location="<homedir>\.ivy2\cache\com.oracle.coherence\coherence\jars\coherence-12.1.2-0-0.jar">
<origin-location is-local="false" location="<intranetmavenrepo>/nexus/content/repositories/cditspoc-3rd-party/com/oracle/coherence/coherence/12.1.2-0-0/coherence-12.1.2-0-0.jar"/>
</artifact>
</artifacts>
</revision>
</module>
<module organisation="org.springframework" name="spring-web">
<revision name="4.0.0.RELEASE" status="release" pubdate="20131211234952" resolver="central" artresolver="central" homepage="https://github.com/SpringSource/spring-framework" downloaded="false" searched="false" default="false" conf="system, default, optional, compile, *, provided, runtime, javadoc, sources, master" position="7">
<license name="The Apache Software License, Version 2.0" url="http://www.apache.org/licenses/LICENSE-2.0.txt"/>
<metadata-artifact status="no" details="" size="5998" time="0" location="<homedir>\.ivy2\cache\org.springframework\spring-web\ivy-4.0.0.RELEASE.xml" searched="false" origin-is-local="false" origin-location="http://repo1.maven.org/maven2/org/springframework/spring-web/4.0.0.RELEASE/spring-web-4.0.0.RELEASE.pom"/>
<caller organisation="com.att.ecom.poc" name="coherence_poc" conf="default" rev="4.0.0.RELEASE" rev-constraint-default="4.0.0.RELEASE" rev-constraint-dynamic="4.0.0.RELEASE" callerrev="working#<hostname>"/>
<artifacts>
<artifact name="spring-web" type="jar" ext="jar" status="no" details="" size="661567" time="0" location="<homedir>\.ivy2\cache\org.springframework\spring-web\jars\spring-web-4.0.0.RELEASE.jar">
<origin-location is-local="false" location="http://repo1.maven.org/maven2/org/springframework/spring-web/4.0.0.RELEASE/spring-web-4.0.0.RELEASE.jar"/>
</artifact>
</artifacts>
</revision>
</module>
... several other spring and commons artifacts resolved from central
<module organisation="com.att.ecom.poc" name="poc-domain-model">
<revision name="0.0.1-SNAPSHOT" status="integration" pubdate="20140207093019" resolver="mavenCentralSnapshots" artresolver="mavenCentralSnapshots" homepage="" downloaded="false" searched="false" default="false" conf="system, default, optional, compile, *, provided, runtime, javadoc, sources, master" position="0">
<metadata-artifact status="no" details="" size="3337" time="0" location="<homedir>\.ivy2\cache\com.att.ecom.poc\poc-domain-model\ivy-0.0.1-SNAPSHOT.xml" searched="false" origin-is-local="false" origin-location="<intranetmavenrepo>/nexus/content/repositories/cditspoc-snapshots/com/att/ecom/poc/poc-domain-model/0.0.1-SNAPSHOT/poc-domain-model-0.0.1-20140207.173018-85.pom"/>
<caller organisation="com.att.ecom.poc" name="coherence_poc" conf="default" rev="0.0.1-SNAPSHOT" rev-constraint-default="0.0.1-SNAPSHOT" rev-constraint-dynamic="0.0.1-SNAPSHOT" callerrev="working#<hostname>"/>
<artifacts>
</artifacts>
</revision>
</module>
</dependencies>
</ivy-report>
Update:
I finally noticed that the download fails because the exact fully-qualified path doesn't exist on the intranet repo. It's looking for a file with "SNAPSHOT" in the name, but the files in the directory only have timestamps, not "SNAPSHOT".
I noticed the following discussion, which seems relevant: enter link description here .
As a result of this, I modified by "ivysettings.xml" to the following slightly elided version:
<ivysettings>
<settings defaultResolver="default"/>
<property name="m2-pattern" value="${user.home}/.m2/repository/[organisation]/[module]/[revision]/[module]-[revision](-[classifier]).[ext]" override="false" />
<resolvers>
<chain name="default">
<filesystem name="local-maven2" m2compatible="true" >
<artifact pattern="${m2-pattern}"/>
<ivy pattern="${m2-pattern}"/>
</filesystem>
<ibiblio name="mavenCentralSnapshots" m2compatible="true"
root="http://<hostandport>/nexus/content/repositories/cditspoc-snapshots"
pattern="[organisation]/[module]/[revision]/[artifact]-[revision](-[classifier]).[ext]"/>
<ibiblio name="mavenCentralThirdparty" m2compatible="true"
root="http://<hostandport>/nexus/content/repositories/cditspoc-3rd-party"/>
<ibiblio name="central" m2compatible="true"/>
</chain>
</resolvers>
</ivysettings>
Unfortunately, this made no difference, it still just tries to find the "SNAPSHOT" file, which doesn't exist.
Update:
Actually, I realized another extremely important detail in that error where it fails to find the SNAPSHOT artifact on our MavenCentral. Here's the actual excerpt from the output:
:: problems summary ::
:::: WARNINGS
[FAILED ] com.att.ecom.poc#poc-domain-model;0.0.1-SNAPSHOT!poc-domain-model.jar(bundle): (0ms)
==== shared: tried
C:\Users\dk068x\.ivy2\shared\com.att.ecom.poc\poc-domain-model\0.0.1-SNAPSHOT\bundles\poc-domain-model.jar
==== public: tried
http://repo1.maven.org/maven2/com/att/ecom/poc/poc-domain-model/0.0.1-SNAPSHOT/poc-domain-model-0.0.1-SNAPSHOT.jar
What I should have noticed from this is that it isn't even trying to get the artifact from my snapshot repository at all. It's failing to find it on the public mavencentral, not mine. That tells me there's something wrong with my "ivysettings.xml", but I don't know what it would be.
Please check the comment above as well:
however in addition: change your ivy.xml to the following:
<ivy-module version="2.0">
<info organisation="com.att.ecom.poc" module="coherence_poc"/>
<dependencies>
<dependency org="com.att.ecom.poc" name="poc-domain-model" rev="0.0.1-SNAPSHOT"/>
</dependencies>
</ivy-module>
And then enable check the "ivy console" with debug level. (Ivy console in the console view of Eclipse) if possible.

Ivy, what is the master configuration and why is it not pulling jvyaml?

I have the following ivy file:
<configurations defaultconfmapping="buildtime">
<conf name="buildtime" visibility="private" description="Libraries needed only for compilation" />
<conf name="runtime" description="Libraries only needed at runtime" />
<conf name="test" description="Libraries only needed for testing" />
</configurations>
<dependencies>
<dependency org="net.java.dev" name="jvyaml" rev="0.2.1" conf="runtime" />
<dependency org="org.apache.solr" name="solr-core" rev="3.6.0" conf="runtime" />
</dependencies>
and I have an ant retrieve task that looks like this:
<target name="retrieve-all" depends="resolve">
<ivy:retrieve pattern="lib/[conf]/[artifact]-[revision].[ext]" conf="*" />
</target>
The weird thing is, that all the solr dependencies download into lib/runtime as I'd expect, but the jvyaml module does not! It 'resolves', but will not download into the lib/runtime directory unless I change the dependency declaration to:
<dependency org="net.java.dev" name="jvyaml" rev="0.2.1" conf="runtime->master" />
What is this master configuration and why is it needed to pull the jvyaml jar, but not solr?
Thanks
I would suggest restructuring your configurations as follows:
<ivy-module version="2.0">
<info organisation="com.myspotontheweb" module="demo"/>
<configurations>
<conf name="compile" description="Libraries needed only for compilation" />
<conf name="runtime" description="Libraries only needed at runtime" extends="compile" />
<conf name="test" description="Libraries only needed for testing" extends="runtime" />
</configurations>
<dependencies>
<dependency org="net.java.dev" name="jvyaml" rev="0.2.1" conf="runtime->default" />
<dependency org="org.apache.solr" name="solr-core" rev="3.6.0" conf="runtime->default" />
</dependencies>
</ivy-module>
Important changes introduced:
Use the more standard "compile" configuration
Configuration inheritance using the "extends" attribute. Compile dependencies can then be automatically included in both the runtime and test configurations.
Use configuration mappings, for example: conf="runtime->default". This makes it obvious which local configuration is associated with which remote configuration.
Configuration mappings explained
Configurations are a powerful ivy feature. When ivy downloads Maven modules it performs an internal translation and assigns a standard set of configurations, listed in this answer:
How are maven scopes mapped to ivy configurations by ivy
When declaring a dependency it's a good idea to always make use of a configuration mapping, so that there is no doubt where the dependencies artifacts are assigned.
For example:
<dependency org="??" name="??" rev="??" conf="runtime->default" />
Here we're saying we want the remote module's default dependencies associated with our local runtime configuration.
In practice, there are only two remote configuration mappings you'll actually need:
default: The remote module's artifact and all it's runtime transitive dependencies
master: The remote module's artifact only (No transitive dependencies)
In conclusion, I think your problem was caused by the fact that the remote Maven module's "runtime" scope does not include Maven module's artifact, instead you were getting the non-existant transitive dependencies of the module jvyaml :-(
Some additional advice
I'd also suggest generating an ivy dependency management report, as follows:
<target name="init" description="Resolve dependencies and populate lib dir">
<ivy:resolve/>
<ivy:report todir="${build.dir}/ivy-report" graph="false"/>
<ivy:retrieve pattern="lib/[conf]/[artifact]-[revision].[ext]"/>
</target>
The report will help explain how each dependency ends up on different configurations. Also really useful for determining how transitive dependencies are being managed.
And finally, here's where the configuration inheritance pays off, creating ivy managed ANT classpaths:
<target name="init" description="Resolve dependencies and set classpaths">
<ivy:resolve/>
<ivy:report todir="${build.dir}/ivy-report" graph="false"/>
<ivy:cachepath pathid="compile.path" conf="compile"/>
<ivy:cachepath pathid="runtime.path" conf="runtime"/>
<ivy:cachepath pathid="test.path" conf="test"/>
</target>
Notice that the original solr-core is not retrieved either.
After your resolve, go the cache and check the ivy.xml files for both modules.
You will see that they publish their artifacts in conf=master only
<artifact name="jvyaml" type="jar" ext="jar" conf="master"/>
<artifact name="solr-core" type="jar" ext="jar" conf="master"/>
Which means, you have to do explicit configuration mapping to denote that your builtime configuration should evoke the 'master' configuration of your dependencies. (check configuration mapping).
HOWEVER, the dependencies of the solr-core, have the configuration mapping as you could see in the ivy.xml file:
<dependency org="org.apache.solr" name="solr-solrj" rev="3.6.0" force="true" conf="compile->compile(*),master(*);runtime->runtime(*)"/>
I think it's the master(*) thingy.
What I usually do is in my own ivy.xml file when I declare dependencies I do the mapping:
<dependency org="net.java.dev" name="jvyaml" rev="0.2.1" conf="runtime->master" />
This one says the runtime be evoking the master configuration in the designated dependency.
You could do
conf="runtime,test->master"
as well

how to publish 3rdparty artifacts with ivy and nexus

I'm busily getting my feet wet with ivy. I have an existing nexus repository running on my local PC, and an existing ant build script.
Both work fine.
Part of the build scripts have some files to retrieve our 3rdparty jar files (log4j, xmlbeans, junit, pdf, etc..) from a network share - which is klunky at best.
I want to use ivy's dependency mechanisms to retrieve these files from a nexus repository and use that in the build. Each 3rdparty lib has a name and an arbitrary set of files (jar, dll, license.dat, xml, etc).
Since we have a large number of these 3rdparty libs and each lib has multiple files - manual uploading to nexus is not an option -- i need something i can use to take a set of files, give them a lib name, a version number and upload the result to nexus. then I need to be able to retrieve this from ivy.
I managed to get the upload part to work, but the retreival process does not work. Using our xmlbeans lib as a starting point I created the following ivy.xml file
<ivy-module version="1.0">
<info organisation="thirdparty_tools" module="xmlbeans" status="integration">
<publications>
<artifact name="jsr173_api" type="jar" ext="jar"/>
<artifact name="saxon-dom" type="jar" ext="jar"/>
<artifact name="saxon-xpath" type="jar" ext="jar"/>
<artifact name="saxon" type="jar" ext="jar"/>
<artifact name="xbean" type="jar" ext="jar"/>
<artifact name="xbean_xpath" type="jar" ext="jar"/>
<artifact name="xmlpublic" type="jar" ext="jar"/>
</publications>
</ivy-module>
and then some ant script to publish it to nexus:
<ivy:resolve/>
<ivy:publish <ivy:publish resolver="thirdparty" forcedeliver="true" update="true" revision="${version}" overwrite="true">
<artifacts pattern="[artifact].[ext]"/>
<ivy:publish/>
And this all works fine. It publishes all the jar files to nexus in the expected directory.
The trouble comes when I try to use it in my build.
I created the following ivy.xml file for my build:
<ivy-module version="1.0">
<info organisation="myCompany" module="GLB_Data"/>
<dependencies>
<dependency org="thirdparty_tools" name="xmlbeans" rev="2.2.0"/>
</dependencies>
</ivy-module>
Then when I run my build - it fails to find anything:
::::::::::::::::::::::::::::::::::::::::::::::
:: UNRESOLVED DEPENDENCIES ::
::::::::::::::::::::::::::::::::::::::::::::::
:: thirdparty_tools#jsr173_api;2.2.0: not found
:: thirdparty_tools#saxon-dom;2.2.0: not found
:: thirdparty_tools#saxon-xpath;2.2.0: not found
:: thirdparty_tools#saxon;2.2.0: not found
:: thirdparty_tools#xbean;2.2.0: not found
:: thirdparty_tools#xbean_xpath;2.2.0: not found
:: thirdparty_tools#xmlpublic;2.2.0: not found
::::::::::::::::::::::::::::::::::::::::::::::
The problem seems to be with this pattern:
WARN: ==== public: tried
WARN: http //localhost:8081/nexus/content/groups/public/thirdparty_tools/jsr173_api/2.2.0/jsr173_api-2.2.0.pom
WARN: -- artifact thirdparty_tools#jsr173_api;2.2.0!jsr173_api.jar:
WARN: http //localhost:8081/nexus/content/groups/public/thirdparty_tools/jsr173_api/2.2.0/jsr173_api-2.2.0.jar
ivy seems to be looking for the jsr173_api artifact under its own name, rather than under the xmlbeans folder where it was published to:
[ivy:publish] published jsr173_api to http //localhost:8081/nexus/content/repositories/thirdparty/thirdparty_tools/xmlbeans/2.2.0/jsr173_api-2.2.0.jar
(urls obfuscated to prevent accidents).
so somehow I need to publish differently, or retrieve differently. Ideas and suggestions are much appreciated.
Nexus is primarily a Maven repository, this means one must adapt to the way Maven structures artifacts.
Since you're focused on bulk loading Nexus I suggest looking at the answer to the following question:
Upload artifacts to Nexus, without Maven
If you wish to stick with ivy read on.....
Background
Need a Maven POM
Your first issue is that your Maven module(s) will need a POM file. This file describes the maven module and can be easily generated from the contents of your ivy.xml file (See solution below).
Secondly, Maven assumes that there is one primary artifact being built. For example:
<project>
<modelVersion>4.0.0</modelVersion>
<groupId>com.myspotontheweb</groupId>
<artifactId>donaldduck</artifactId>
<version>1.0.1</version>
<packaging>txt</packaging>
</project>
A Maven client would translate this information into the following URL:
http://<host>/<repo>/com/myspotontheweb/donaldduck/1.0.1/donaldduck-1.0.1.txt
This demonstrates how Nexus can store any type of binary dependency. The packaging parameter defaults to "jar".
How maven handles additional module artifacts
While Maven focuses on a single build artifact, it is possible to add additional supplementary artifacts by posting them into the same directory (as you've done).
These are not listed in the Maven POM. Instead a Maven uses a special "classifier" attribute. The following is a possible dependency declaration.
<dependency>
<groupId>com.myspotontheweb</groupId>
<artifactId>donaldduck</artifactId>
<version>1.0.1</version>
<classifier>metadata</classifier>
<type>n3</type>
</dependency>
A Maven client would translate this into the following URL:
http://<host>/<repo>/com/myspotontheweb/donaldduck/1.0.1/donaldduck-1.0.1-metadata.n3
Open source projects typically release their source code in this manner.
Ivy Solution
So finally how does one publish files into Nexus using ivy?
First of all decide which artifact is the "main" build artifact and add an additional entry for your POM file:
<ivy-module version='2.0' xmlns:e="http://ant.apache.org/ivy/extra">
<info organisation="com.myspotonontheweb" module="donaldduck" revision="1.0.1"/>
<publications>
<artifact name="donaldduck" type="txt"/>
<artifact name="donaldduck" type="pom"/>
<artifact name="donaldduck" type="n3" e:classifier="metadata"/>
<artifact name="donaldduck" type="zip" e:classifier="disto"/>
</publications>
</ivy-module>
The other files can also be listed but each must have a unique classifier attribute..... Here you will be faced with one of the classic problems translating an ANT project into Maven.... Each jar file you publish, will probably need to have a separate POM. They not really "supplementary" artifacts.....
Pretending that you don't need to publish multiple modules.... Use the following build targets to publish your module:
<target name="prepare" description="Generate POM">
<!-- Optional: Intermediate file containing resolved version numbers -->
<ivy:deliver deliverpattern="${build.dir}/ivy.xml" pubrevision="${publish.revision}" status="release"/>
<!-- Generate the Maven POM -->
<ivy:makepom ivyfile="${build.dir}/ivy.xml" pomfile="${build.dir}/donaldduck.pom"/>
</target>
<target name="publish" depends="init,prepare" description="Upload to Nexus">
<ivy:publish resolver="nexus-deploy" pubrevision="${publish.revision}" overwrite="true" publishivy="false" >
<artifacts pattern="${build.dir}/[artifact](-[classifier]).[ext]"/>
</ivy:publish>
</target>
Nexus credentials
And for completeness here's the ivysettings.xml file containing the Nexus repository location and credentials:
<ivysettings>
<settings defaultResolver="nexus-central"/>
<credentials host="somehost" realm="Sonatype Nexus Repository Manager" username="????" passwd="????"/>
<resolvers>
<ibiblio name="nexus-central" root="http://somehost/nexus/content/repositories/central/" m2compatible="true"/>
<ibiblio name="nexus-deploy" root="http://somehost/nexus/content/repositories/repo" m2compatible="true"/>
</resolvers>
</ivysettings>
Update
Downloading artifacts
To retrieve all the published artifacts (not just the main one), you need to list them as follows:
<dependency org="com.myspotontheweb" name="donaldduck" rev="1.0.1">
<artifact name="donaldduck" type="txt"/>
<artifact name="donaldduck" type="n3" e:classifier="metadata"/>
<artifact name="donaldduck" type="zip" e:classifier="distro"/>
</dependency>
Functionally the same as the following Maven fragment:
<dependency>
<groupId>com.myspotontheweb</groupId>
<artifactId>donaldduck</artifactId>
<version>1.0.1</version>
<type>txt</type>
</dependency>
<dependency>
<groupId>com.myspotontheweb</groupId>
<artifactId>donaldduck</artifactId>
<version>1.0.1</version>
<classifier>metadata</classifier>
<type>n3</type>
</dependency>
<dependency>
<groupId>com.myspotontheweb</groupId>
<artifactId>donaldduck</artifactId>
<version>1.0.1</version>
<classifier>distro</classifier>
<type>zip</type>
</dependency>

Resources