I have an issue where in I have defined dependancies in ivy.xml on our internal corporate svn. I am able to access this svn site without any proxy task in ant. While my dependencies resides on ibiblio, that’s something outside our corporate, and needs proxy inorder to download something. I am facing problem using ivy here.
I have following in build.xml
<target name="proxy">
<property name="proxy.host" value="xyz.proxy.net"/>
<property name="proxy.port" value="8443"/>
<setproxy proxyhost="${proxy.host}" proxyport="${proxy.port}"/>
</target>
<!-- resolve the dependencies of stratus -->
<target name="resolveTestDependency" depends="testResolve, proxy" description="retrieve test dependencies with ivy">
<ivy:settings file="stratus-ivysettings.xml" />
<ivy:retrieve conf="test" pattern="${jars}/[artifact]-[revision].[ext]"/><!--pattern here specifies where do you want to download lib to?-->
</target>
<target name=" testResolve ">
<ivy:settings file="stratus-ivysettings.xml" />
<ivy:resolve conf="test" file="stratus-ivy.xml"/>
</target>
Following is the excerpt from stratus-ivysettings.xml
<resolvers>
<!-- here you define your file in private machine not on the repo (e.g. jPricer.jar or edgApi.jar)-->
<!-- This we will use a url nd not local file system.. -->
<url name="privateFS">
<ivy pattern="http://xyz.svn.com/ivyRepository/ [organisation]/ivy/ivy.xml"/>
</url>
.
.
.
<url name="public" m2compatible="true">
<artifact pattern="http://www.ibiblio.org/maven2/[organisation]/[module]/[revision]/[artifact]-[revision].[ext]"/>
</url>
.
.
.
So as can be seen here for getting ivy.xml, I don’t need any proxy as its within our own network which cant be accesses when I set proxy. But on the other hand I am using ibiblio as well which is external to our network and works only with proxy. So above build.xml wont work in that case. Can somebody help here.
I don’t need proxy while getting ivy.xml (as if I have proxy, ivy wont be able to find ivy file behind proxy from within the network), and I just need it when my resolver goes to public url.
When using setproxy, use the nonproxyhosts attribute to specify the hosts that the proxy should not be used for (pipe separated). e.g, modify the setproxy task in your example to
<setproxy proxyhost="${proxy.host}"
proxyport="${proxy.port}"
nonproxyhosts="xyz.svn.com"/>
For more details see http://ant.apache.org/manual/Tasks/setproxy.html
Related
I've read all the tutorials and examples, and still cannot publish a set of custom jars in my local Ivy repository.
Edit: Basically I want the same behavior as maven-install-plugin.
Here's my setup. I have an Ant task which produces the jars in a given folder. The folder name is not fixed but rather passed as a property in file. I want to get all the jars in this folder and install them in my local Ivy repo so that I can use them on a next step.
Here is my Ant from where I call the ivy:publish:
<project name="Install Ivy Dependencies" xmlns:ivy="antlib:org.apache.ivy.ant" basedir="." default="publish">
<loadproperties srcFile="path_to_folder.properties"/>
<property name="file_pattern" value="${path_to_folder}/[artifact].[ext]" />
<property name="pub_revision" value="1.0.0" />
<target name="resolve">
<ivy:configure file="ivysettings.xml" />
<ivy:resolve file="ivy.xml" />
</target>
<target name="retrieve-all" depends="resolve">
<ivy:retrieve pattern="${file_pattern}" conf="*" />
</target>
<target name="publish" depends="retrieve-all">
<ivy:publish resolver="local" organisation="myOrg" update="true" overwrite="true" pubrevision="${pub_revision}">
<artifacts pattern="${file_pattern}"/>
</ivy:publish>
</target>
</project>
Here's my ivysettings.xml:
<ivysettings>
<resolvers>
<filesystem name="local" local="true"/>
</resolvers>
</ivysettings>
And the ivy.xml:
<?xml version="1.0" encoding="ISO-8859-1"?>
<ivy-module version="2.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="http://ant.apache.org/ivy/schemas/ivy.xsd">
<info organisation="myOrg" module="myModule" revision="1.0.0"/>
<publications>
<artifact name="my-custom-jar" ext="jar" type="jar"/>
<artifact name="my-custom-jar-source" ext="jar" type="source"/>
</publications>
</ivy-module>
The error that I am getting when I call the ant task is:
impossible to publish artifacts for myOrg#myModule;1.0.0: java.lang.IllegalStateException: impossible to publish myOrg#myModule;1.0.0!my-custom-jar.jar using local: no artifact pattern defined
I've managed to run my scenario and to resolve my issues using this tutorial There were two major issues in my code/integration.
First one is that you cannot tell Ivy to publish the artifacts in its repository without providing a path to it. I did this with the filesystem resolver:
<filesystem name="local" local="true" transactional="local">
<ivy pattern="${ivy.default.ivy.user.dir}/local/[module]/ivy-[revision].xml" />
<artifact pattern="${ivy.default.ivy.user.dir}/local/[module]/[artifact]-[revision].[ext]" />
</filesystem>
The stupid think about it is this should be build in. If you copy it as is, then everything works. If the config is different, or pointing to a different location - nothing works and you are not told why. I read tons of docs about Apache Ivy and it was nowhere mentioned that these patterns should point to the local Ivy repository. I thought these were the paths from where the jars should be taken. I actually complained about this, but the Ivy documentation is very confusing. Also I think the examples there are wrong. Who would like to publish the artifacts in their ivy.settings.dir. In my case this directory was in my repository!
There was a second issue. It is a smaller one and again very hard to see and fix. There's something wrong the revision param and again the documentation is messed up. If you specify one and the same string for the revision and pub revision the artifacts aren't publish without any explanation why. I fixed it by removing the revision from ivy.xml file.
Last, but not least, I didn't manage to run successfully the "thing" as Ant task, but with java -jar $IVY_JAR ... Maybe the issue was because of the versions, but I was too tired to try it with the fix.
P.S.#cantSleepNow thanks for the help.
You need to add artifact pattern to resolver in ivysettings.xml, something like (example from ivy documentation):
<ivysettings>
<resolvers>
<filesystem name="local" local="true">
<ivy pattern="${ivy.settings.dir}/1/[organisation]/[module]/ivys/ivy-[revision].xml"/>
<artifact pattern="${ivy.settings.dir}/1/[organisation]/[module]/[type]s/[artifact]-[revision].[ext]"/>
</filesystem>
</resolvers>
</ivysettings>
So, let's assume that I have an already installed SVN and installed ANT / Ivy locally.
I want to have the "shared" part of the ivy config point to some kind of share on a server. How would I need to set this up?
I know I have to dig through the ivy jar and pull out the ivysettings file and modify shared repositories.
So let's assume that I have a server on my intranet at MyServer.intranet.net and my team's folder was under /path/to/NetAdmin (thus the full path would be MyServer.intranet.net/path/to/NetAdmin ) How would I get this set up as a team repository for shared libraries? Would I just specify it and when I package the projects it writes the dependencies there?
Thanks
Here what I did:
I created a Subversion project called ivy.dir.
In this ivy.dir project, I have the latest ivy.jar.
In the ivy.dir, I have the ivysettings.xml setup for our environment. For example, we use a local Artifactory Maven repository for our own jars. The ivysettings.xml in the ivy.dir project points to that.
I created a file called ivy.tasks.xml. This is an Ant build file.
The ivy.tasks.xml looks like this:
<project name="Ivy.Tasks"
xmlns:ivy="http://ant.apache.org/ivy"
xmlns:jacoco="antlib:org.jacoco.ant">
<property environment="env"/>
<!-- Add Ivy Tasks -->
<taskdef uri="http://ant.apache.org/ivy"
resource="org/apache/ivy/ant/antlib.xml">
<classpath>
<fileset dir="${ivy.dir}">
<include name="ivy*.jar"/>
</fileset>
</classpath>
</taskdef>
<ivy:settings file="${ivy.dir}/ivysettings.xml"/>
</project>
Notice that I have my own Ivy settings, thank you. I didn't have to munge up the one in the ivy.jar (although I could have since everyone will use my ivy.jar file!). My ivysettings.xml looks like this:
<ivysettings>
<!-- I'll explain this part below -->
<property name="env.EXECUTOR_NUMBER" value="0" override="false"/>
<caches
defaultCacheDir="${ivy.default.ivy.user.dir}/cache-${env.EXECUTOR_NUMBER}"
resolutionCacheDir="${ivy.dir}/../target/ivy.cache"/>
<!-- Just the standard stuff you find in the `ivysettings.xml in the ivy.jar -->
<settings defaultResolver="default"/>
<include file="${ivy.dir}/ivysettings-public.xml"/> <!-- This one is different -->
<include url="${ivy.default.settings.dir}/ivysettings-shared.xml"/>
<include url="${ivy.default.settings.dir}/ivysettings-local.xml"/>
<include url="${ivy.default.settings.dir}/ivysettings-main-chain.xml"/>
<include url="${ivy.default.settings.dir}/ivysettings-default-chain.xml"/>
</ivysettings>
The big change is the ivysetting-public.xml file:
<ivysettings>
<resolvers>
<ibiblio name="public"
m2compatible="true"
checkmodified="true"
root="http://repos.vegicorp.com/artifactory/libs-release" />
</resolvers>
</ivysettings>
It's pointing to my local Maven repository -- my Artifactory server.
Now, for a developer to use Ivy, all they have to do is:
In the root of their project in Subversion, add a svn:external. This svn:external will be used to bring my ivy.dir project into their Subversion project.
In the build.xml
Add an Ivy namespace definition to their build.xml in the <project> definition.
Set the property ivy.dir to `${basedir}/ivy.dir.
Use the <import> task to import ${ivy.dir}/ivy.tasks.xml into their build.xml file.
Something like this:
<project name="post-a-matic" default="package" basedir="."
xmlns:ivy="http://ant.apache.org/ivy">
<property name="ivy.dir" value="${basedir}/ivy.dir"/>
<import file="${ivy.dir}/ivy.tasks.xml"/>
<!-- A whole bundle of properties are set -->
<target name="clean">
<delete dir="${target.dir}"/>
<ivy:cleancache/> <!-- Look: They have access to Ivy! -->
</target>
<target name="-resolve">
<ivy:resolve/>
</target>
<target name="compile"
depends="-resolve">
<ivy:cachpath
pathid="main.classpath"
conf="compile,provided"/>
<!-- Boy that's easy! -->
<javac srcdir="${main.srcdir}"
destdir="${main.destdir}"
classpathref="main.classpath"/>
</target>
<!-- On and on -->
This solves a lot of problems:
You can update the ivy.settings and everyone will have the updated settings. This ended up being very important to us because we use Jenkins and I wanted Jenkins to clean the ivy cache on each build. Whoops! That cleans out the ivy cache on builds that are being executed at the same time! I solved the problem by changing the ivysettings.xml file to define a different Ivy cache for each Jenkins build executor. One the Jenkins server, you have Ivy caches called $HOME/.ivy2/cache-0, $HOME/.ivy2/cache-1, etc. Each executor can delete it's own Ivy cache without affecting the others. Users, meanwhile will just have $HOME/.ivy2/cache-0.
You also can update Ivy when a new jar comes out. You update your Ivy jar file, and everyone gets the lated.
Big one of course is that Ivy installs itself when a project is checked out.
And an extra special bonus: You could use your ivy.dir and ivy.tasks.xml file to install other tasks. For example, each of our projects must run itself through Findbugs, PMD, CPD (part of the PMD project, Checkstyle, and use JaCoCo. for test coverage.
Each one of these projects consist of a jar file, and a <taskdef> to pull the task definitions into Ant. And, how do you use these tasks too? They're not defined in the standard Ant model. Developers don't know how to use them.
I've added these jars into my ivy.dir project, and installed all of those task definitions into my ivy.tasks.xml file. I also defined easy to use <macrodef> for most of these tasks, so it's easy for the developers to use them. In fact, I've even included the old Ant-Contrib tasks just for fun.
Now, once you add ivy.dir into your project, you have all of these extra tasks, and you have nothing to install on your machine.
You don't need to change the ivy jar. Just create a filesystem resolver in an ivysettings file and publish to this. Here's an example:
good ivy tutorial for local repository?
You'll find that ivy is very flexible and can support pretty much any mechanism for hosting files.
Personally, I'd consider installing a Maven repository manager like Nexus or Artifactory and use this to host both your builds dependencies and build outputs. In the long run it's a lot easier, especially if you're doing Java development.
I have a build.xml(ant based) which requires some jar from nexus to get copied in existing lib folder. i.e when it builds it should copy the jar from nexus with some version defined & then copy in lib & do compilation.
like happen in maven we define the artifact & its version . If changed will automatically download it from maven repo.
how can i do this in ant based builds?
experts pls advice.
I have taken the example listed in this thread one step further and created a macrodef to clean things up a bit for re-use. See below for downloading two artifacts from nexus (one snapshot, one release).
<project>
<target name="get-all">
<mkdir dir="lib" />
<nexus-get
groupId="foo.bar"
artifactId="some-artifact"
version="1.0.28"
repo="releases"
extension="jar"
dest="lib"
/>
<nexus-get
groupId="foo.bar"
artifactId="another-artifact"
version="1.0.0-SNAPSHOT"
repo="snapshots"
extension="jar"
dest="lib"
/>
</target>
<macrodef name="nexus-get">
<attribute name="groupId"/>
<attribute name="artifactId"/>
<attribute name="version"/>
<attribute name="repo"/>
<attribute name="extension"/>
<attribute name="dest"/>
<sequential>
<get src="http://my-nexus:9999/nexus/service/local/artifact/maven/redirect?r=#{repo}&g=#{groupId}&a=#{artifactId}&v=#{version}&e=#{extension}" dest="#{dest}/#{artifactId}.#{extension}" usetimestamp="true" />
</sequential>
</macrodef>
You would probably be interested in Ivy. It is a sub-project of Ant for dependency management. It is perfect for your situation because it can read Maven repositories and provides Ant tasks for downloading the published artifacts, constructing class paths from them, etc. It supports your use case of getting the most recent version of a dependency if you configure it to ask for the "latest.release" revision of the module.
Although there are surely specific ways to combine ant and maven the simplest thing (if you know the nexus URL and your artifact parameters to construct the download URL) would be just to use the ant Get task.
<project name="MyProject" default="resolveDependencies" basedir=".">
<target name="resolveDependencies">
<mkdir dir="lib" />
<get src="http://search.maven.org/remotecontent?filepath=log4j/log4j/1.2.9/log4j-1.2.9.jar" dest="lib/log4j-1.2.9.jar" usetimestamp="true" />
</target>
</project>
Perhaps use the Maven Ant Tasks.
As shown on http://maven.apache.org/ant-tasks/examples/dependencies.html
Can list dependencies in ant, and also do things like copy them
I think the section Using FileSets and the Version Mapper covers your need
You can use is filesetId, which will give you a fileset reference that can be used to copy files into a particular location. For example, to populate WEB-INF/lib with your dependencies you could use the following:
<artifact:dependencies filesetId="dependency.fileset" useScope="runtime">
<dependency groupId="junit" artifactId="junit" version="3.8.2" scope="test"/>
</artifact:dependencies>
<copy todir="${webapp.output}/WEB-INF/lib">
<fileset refid="dependency.fileset" />
<!-- This mapper strips off all leading directory information -->
<mapper type="flatten" />
</copy>
I am using Ivy to publish a snapshot of a built Jar to a locally hosted Nexus repository using the following Ant target.
<target name="publish">
<ivy:publish resolver="nexus_snapshot" pubrevision="SNAPSHOT" overwrite="true">
<artifacts pattern="${dist.dir}/[artifact].[ext]" />
</ivy:publish>
</target>
This appears to work fine, resulting in the Jar and its associated ivy.xml being present in the repository (with filenames mymodule-SNAPSHOT.jar and ivy-SNAPSHOT.jar).
Later, in another build script, I wish to retrieve the Jar and its associated dependencies (i.e. as specified in its ivy.xml) into a directory.
This is the Ant target I'm using.
<target name="deploy">
<delete dir="deploy" />
<mkdir dir="deploy" />
<ivy:settings file="${ivy.dir}/ivy_deploy_settings.xml" />
<ivy:retrieve organisation="myorg" module="mymodule"
inline="true" revision="SNAPSHOT" pattern="deploy/[artifact].[ext]"/>
</target>
This retrieves the Jar to the directory, but not its dependencies. Also, if I add
conf="impl"
to the retrieve, it fails as the configuration is not found.
As such, it seems that the retrieve is simply not referencing the ivy.xml and hence not resolving the dependencies.
Should this work or am I misunderstanding something?
I have now resolved this problem. I believe the issue is simply that Nexus works using POM files rather than Ivy files (by default at least - I can't see any relevant configuration options).
The solution is therefore to generate a suitable POM and publish this along with the Jar.
<target name="publish">
<property name="generated.ivy.file" value="${dist.dir}/ivy.xml" />
<ivy:deliver deliverpattern="${generated.ivy.file}"
organisation="${ivy.organisation}"
module="${ivy.module}" status="integration"
revision="${ivy.revision}"
pubrevision="SNAPSHOT"
conf="impl" />
<ivy:makepom ivyfile="${generated.ivy.file}"
pomfile="${dist.dir}/${ivy.module}.pom"/>
<ivy:publish resolver="nexus_snapshot" pubrevision="SNAPSHOT"
publishivy="false" status="integration" overwrite="true">
<artifacts pattern="${dist.dir}/[artifact].[ext]" />
<artifact name="${ivy.module}" type="pom" ext="pom"/>
</ivy:publish>
</target>
Note that I first generate an Ivy file for the current module (and my desired configuration) to create the POM from.
I have an issue where in I have defined dependancies in ivy.xml on our internal corporate svn. I am able to access this svn site without any proxy task in ant. While my dependencies resides on ibiblio, that’s something outside our corporate, and needs proxy inorder to download something. I am facing problem using ivy here.
I have following in build.xml
<target name="proxy">
<property name="proxy.host" value="xyz.proxy.net"/>
<property name="proxy.port" value="8443"/>
<setproxy proxyhost="${proxy.host}" proxyport="${proxy.port}"/>
</target>
<!-- resolve the dependencies of stratus -->
<target name="resolveTestDependency" depends="testResolve, proxy" description="retrieve test dependencies with ivy">
<ivy:settings file="stratus-ivysettings.xml" />
<ivy:retrieve conf="test" pattern="${jars}/[artifact]-[revision].[ext]"/><!--pattern here specifies where do you want to download lib to?-->
</target>
<target name=" testResolve ">
<ivy:settings file="stratus-ivysettings.xml" />
<ivy:resolve conf="test" file="stratus-ivy.xml"/>
</target>
Following is the excerpt from stratus-ivysettings.xml
<resolvers>
<!-- here you define your file in private machine not on the repo (e.g. jPricer.jar or edgApi.jar)-->
<!-- This we will use a url nd not local file system.. -->
<url name="privateFS">
<ivy pattern="http://xyz.svn.com/ivyRepository/ [organisation]/ivy/ivy.xml"/>
</url>
.
.
.
<url name="public" m2compatible="true">
<artifact pattern="http://www.ibiblio.org/maven2/[organisation]/[module]/[revision]/[artifact]-[revision].[ext]"/>
</url>
.
.
.
So as can be seen here for getting ivy.xml, I don’t need any proxy as its within our own network which cant be accesses when I set proxy. But on the other hand I am using ibiblio as well which is external to our network and works only with proxy. So above build.xml wont work in that case. Can somebody help here.
I don’t need proxy while getting ivy.xml (as if I have proxy, ivy wont be able to find ivy file behind proxy from within the network), and I just need it when my resolver goes to public url.
When using setproxy, use the nonproxyhosts attribute to specify the hosts that the proxy should not be used for (pipe separated). e.g, modify the setproxy task in your example to
<setproxy proxyhost="${proxy.host}"
proxyport="${proxy.port}"
nonproxyhosts="xyz.svn.com"/>
For more details see http://ant.apache.org/manual/Tasks/setproxy.html