How to combine Ivy and Artifactory (plus Jenkins)? - jenkins

I'm trying to upgrade librarires and artifacts management in my company. Now we're making artifacts (jar, war, ear files) with ant scripts running from Jenkins. I'd like to add Ivy and Artifactory but I'm a little bit unsure what would be best (or at leats good) configuration. As per documentation,
Ivy has three repositories (local, shared and public) and additionaly cache. Reading Artifactory docs, one has to configure Ivy files for Artifactory resolver (not repository?!).
My questions are:
Would that be public repository in Ivy terminology?
Is Ok tu use only one Artifactory repository for development (we have 4 teams) and deployment pipeline?
What would be (approximate) configuration? Is this ok (public ibiblio + enterprise Artifactory)?
<resolvers>
<ibiblio name="libraries" m2compatible="true" />
<url name="arti">
<artifact pattern="http://localhost:8081/artifactory/libs-snapshot-local/[organization]/[module]/[revision]/[artifact]-[revision].[ext]"/>
<ivy pattern="http://localhost:8081/artifactory/libs-snapshot-local/[organization]/[module]/[revision]/ivy-[revision].xml" />
</url>
</resolvers>
Update on 11th Feb:
Just for the record, here is working configuration:
<credentials host="artifactory.host" realm="Artifactory Realm" username="admin" passwd="password"/>
<resolvers>
<chain name="loc-ext">
<filesystem name="tmp-lib">
<artifact pattern="${ivy.settings.dir}/lib/[artifact].[ext]" />
</filesystem>
<ibiblio name="libraries"
m2compatible="true"
checkmodified="true"
root="http://artifactory.host:8081/artifactory/ext-release-local"
/>
</chain>
<ibiblio name="snapshots"
m2compatible="true"
pattern="[organisation]/[module]/[artifact]-[revision].[ext]"
root="http://artifactory.host:8081/artifactory/libs-snapshot-local"
/>
</resolvers>
Chain resolver is for retrieving (f.e. for compiling), tmp_lib has libraries, that are needed only for current task, libraries is sort of enterprise repository and snapshots is for publishing.

We us Artifactory, Jenkins, and Ivy to build most or our projects, but I probably am doing it all wrong.
First, we don't bother with a shared repository. It never made much sense to me. The shared repo looks similar to the private repo, but is shared among developers. What makes that so different from a Artifactory repository that's only shared among the developers?
We also use <ivy:makepom> to create a pom.xml and then use mvn deploy to deploy our artifacts to our Artifactory repo instead of using the <ivy:publish>. We make two poms per jar, one is pom.xml and the other is pom-snapshot.xml. We use Jenkin's Build Promotion Plugin to run mvn deploy with each successful build to deploy the snapshot jar, and then let developers manually "promote a build" via the Build Promotion Plugin to deploy the release version of the jar.
Ivy has the concept of status which looks similar to the snapshot vs. release idea of Maven, but I believe that all these jars with their status are deployed to the same repo. I simply keep the status set to release and depend upon the snapshot/release concept in Maven.
Developers can switch from snapshot versions of a jar to the release version by simply appending or removing -SNAPSHOT from the revision in ivy.xml. This allows developers to use recently built jars without waiting for them to be promoted to the release repository.
My main concern was keeping Artifactory compatible with Maven since some of our projects are Maven projects. Some of this was due to not being 100% familiar with Ivy, and having a firmer grasp with Maven. However, it does work for us and keeps me from worrying about the differences between Ivy and Maven.
Now, that you know we're doing it all wrong, here are the answers to your question:
Would that be public repository in Ivy terminology?
Ivy has the concept of status, but I'm not sure how a Maven project could use that. This is why we use mvn deploy and why we build a snapshot and release pom with each build.
Is Ok tu use only one Artifactory repository for development (we have 4 teams) and deployment pipeline?
Sure, why not? What would be the advantage of making four separate repositories. Besides, even if you go with the separate repositories, you could create them all with a single Artifactory instance. This way, you could assign read/write permissions to each group for each of their repos, but only have to maintain a single Artifactory instance.
What would be (approximate) configuration? Is this ok (public ibiblio + enterprise Artifactory)?
Ours is:
<ivysettings>
<resolvers>
<ibiblio name="public"
m2compatible="true"
checkmodified="true"
root="http://buildl01.tcprod.local/artifactory/libs-release" />
</resolvers>
</ivysettings>
But we're not deploying as Ivy. Just reading from a Maven repo. The libs-release is a virtual repo with includes our local repo and the public repos from Maven, JBoss, and whatever repos we need.
I have a special ivy.dir project that automatically integrates Ivy into our builds. You add in <import file="${ivy.dir}/ivy.tasks.xml"/> into your build, and you automatically have all the necessary Ivy settings. I also create several parallel macros such as` that are drop in replacements with the standard Ant tasks, but do a few mysterious things behind the scenes.
For example <jar.macro> creates a pom.xml, a pom-snapshot.xml, and embeds the Maven info (plus the Jenkins build info) into the jar. This way, developers could start to integrate Ivy into their build process with a minimum of fuss.

Related

Ant+Ivy Jar Version

When I publish my artifact (some api jar), how do I specify version ?
is it revision attribute ? I want to have client-lib-1.0.jar
<ivy-module>
<info organisation="the.org" module="client-lib" revision ="1.0">
<info>
</ivy-module>
Firstly, you need to include a publications section, telling ivy what arifacts you are publishing as a module. Ivy is very flexible and quite capable of publishing modules with multiple files and/or types.
<publications>
<artifact name="client-lib" type="jar"/>
<artifact name="client-lib" type="jar" e:classifier="source"/>
<artifact name="client-lib" type="jat" e:classifier="javadocs"/>
</publications>
Secondly (and the answer to your question) the revision number of the published ivy file is decided at publish time. It gets set by the special "pubrevision" attribute of the publish task. Ivy will search the artifact pattern for the file(s) listed in the ivy file to be published.
<ivy:publish resolver="my-deploy" pubrevision="1.0">
<artifacts pattern="${build.dir}/[artifact](-[classifier]).[ext]"/>
</ivy:publish>
Under the hood a new ivy file is being generated and uploaded alongside the module's file.
How the file is stored in the repository is a matter decided by ivy resolver.
An issue you're likely to encounter is that few people host an Ivy repository these days. Instead Maven is the most common standard.
The following detailed detailed example(s) describes how this process works including the messy POM generation stuff (cause Maven doesn't understand ivy files):
how to publish 3rdparty artifacts with ivy and nexus
Convert ivy.xml to pom.xml

In Ivy what does impossible to publish artifacts FileNotFoundException Error Mean?

I'm working within a large build system that uses Ant/Ivy. I try to use a predefined ant task that uses ivy:publish ant task and get this error:
impossible to publish artifacts for com.company.project1.proj1#MySupportJar;working#server1: java.io.FileNotFoundException: /path_to/ivy-repository/com.company.project1.proj1/MySupportJar/5.1.3.part/MySupportJar-5.1.3.jar (No such file or directory)
The directory in the error message exists up to the version number part (5.1.3.part).
I am new to Ivy but think I get the basics of how it works. I can not find much on the exact meaning of this error so if someone could help or point me to an explanation I think I could resolve the issue from there.
Ant target
<target name="publish-shared" depends="ivyInit, resolve"
description="Publish to the shared repository">
<ivy:publish pubrevision="5.1.3"
resolver="shared"
pubdate="${timestamp}"
forcedeliver="true"
update="true"
conf="distro, docs">
<artifacts pattern="dist/[artifact].[ext]"/>
</ivy:publish>
</target>
Ivy file snippet
<publications>
<artifact name="MySupportJar" type="jar" conf="distro" />
<artifact name="MySupportJar-source" type="source" ext="jar" conf="docs" />
</publications>
Thanks.
Thanks for all the suggestions. Turns out to be a simple solution that I was not looking for.
The problem was permissions at /path_to/ivy-repository/com.company.project1. I did not have write permission. The .part file is a temporary file written by Ivy. Ivy could not write the temporary file so when it got to reading the file it failed to find it.
I'm answering this so that it might help someone later.
Thanks.
I'm fairly new to Ant with Ivy too. What I've done is combine a local Maven repository (Artifactory), with Jenkins as a continuous integration server. When we build a jar, I also produce the Maven pom.xml with it. Then, I use the mvn deploy:file command to deploy the desired build to our Maven repository.
The developer manually deploys the jar to our Maven repository via the Promoted Build plugin to Jenkins. The developer selects the build to deploy to Maven, and then pretty much presses a button, and that build will be deployed.
I actually produce two pomswith each build. One ispom.xmland the other ispom-snapshot.xml`. We deploy the snapshot with each build, so other developers can use the latest jar instead of the officially deployed one. I've put the whole thing in github if you're interested.
The only decent Ant/Ivy documentation I've seen is Manning's Ant in Action by Steve Loughram. If it wasn't for that, I probably would have never even tried Ant with Ivy. The online Ant/Ivy documentation at Apache is just plain awful.
I've looked over what you have. I suspect it might be an issue with your ivy-settings.xml file. Somewhere, it's getting the part string as the valid location for publishing the file. Otherwise, I have no idea.
As I said, we use a Maven repository for our site repository, and then use Maven to actually deploy the jars to the repository. I simply found that much easier to do it that way than to figure out how to do this in pure Ivy. Besides, it also means that our Maven projects can also use the jars from our Ant/Ivy projects.

Is the [conf] token respected when publishing files using Ivy?

I'm learning Ivy (http://ant.apache.org/ivy/) and integrating it as part of our build process. It's going pretty well, but I'm stuck on something with the Ivy publish task. Specifically, I'd like to publish to the local repository with the following pattern (which includes the name of the current configuration):
${ivy.local.default.root}/[organisation]/[module]/[revision]/[type]s/[conf]/[artifact].[ext]
So I set up my custom resolver:
<filesystem name="modifiedLocalRepo">
<ivy pattern="${ivy.local.default.root}/[organisation]/[module]/[revision]/[type]s/[conf]/[artifact].[ext]" />
<artifact pattern="${ivy.local.default.root}/[organisation]/[module]/[revision]/[type]s/[conf]/[artifact].[ext]" />
</filesystem>
And configured the publish task:
<ivy:publish
settingsRef="ivy.ext.settings.custom" <!--Location of definition of modifiedLocalRepo-->
resolver="modifiedLocalRepo"
artifactspattern="bin/Debug/[module].[ext]"
pubrevision="1.1-Test"
overwrite="true"
conf="primary"
/>
And have the conf defined in the ivy.xml file:
<info organisation="Mine" module="Dependency"/>
<configurations defaultconf="primary"> <!--Don't think defaultconf should be necessary-->
<conf name="primary"/>
</configurations>
<publications>
<artifact name="Dependency" ext="dll" type="bin" conf="primary"/>
<artifact name="Dependency" ext="pdb" type="bin" conf="primary"/>
</publications>
I have the process working, ivy configured, resolved, the project building, and the artifacts published (using my custom resolver above). But the [conf] token is always resolving to "default." So the published directory has the structure:
\.ivy2\local\Mine\Dependency\1.1-Test\bins\default\
instead of the structure
\.ivy2\local\Mine\Dependency\1.1-Test\bins\primary\
I've been tinkering around with it, setting the conf value, defaultconf value, etc. everywhere they are valid but it's always "default."
I'm using ant to do the build, and have run in debug mode (-d), inspected the output and noticed nothing useful. I've looked through the output of <echoproperties\> and noticed nothing useful. I've read through the Ivy documentation and only found use of [conf] as a token in retrieve and report output.
I can attach the full xml files I'm using to do my limited test build if it would be useful.
Is the [conf] token valid/respected in the artifact pattern for an Ivy publish? If so, what am I missing? Thanks!
Update:
Was using Ivy 2.3.0-rc2 and decided to try to roll back and use Ivy 2.0.0. Same issue.
Take a look at my Ivy project in github. I set it up, so you can easily add it into an existing Ant project.
We use a Maven repository, so I have it setup to create a pom.xml, and then use mvn deploy:deploy-file to deploy it back to the Maven repository. We use Jenkins for our build, so we use Jenkins to run the mvn deploy-file to do the deployment. I guess we could use <ivy:publish>, but never really tried it.

Nexus to Cache Artifacts using Ivy

Ok; my first stackoverflow question.
I have been trying to use our local Nexus to download artifacts from and to retrieve those artifacts automatically from an outer repositories in case they don't exist. and CACHE them in the way.
I am using Ant and Ivy to handle the dependency.
More or less the question is similar to this question:
Ivy via Nexus proxy
The ultimate answer to my understanding did enable the use of nexus as a repository but it didn't cache (at least not for me).
I have seen lots of material hovering around the subject but nothing of the exact nature (and sadly I couldn't figure it out).
So let me wrap it up.
I am using the local nexus repository using a url resolver, because we should use a URL if we need to publish (read that somewhere):
<chain name="deploy-chain">
<url
name="nexus"
m2compatible="true"
>
<artifact pattern="${nexus.host}/${nexus.pattern}" />
<ivy pattern="${nexus.host}/${nexus.ivy.pattern}" />
</url>
<ibiblio
name="maven.public"
m2compatible="true"
/>
</chain>
The ivy.xml and build.xml are very standard.
So, is it possible to do this using Ivy or is this a nexus issue?
Is it a good practice to publish the artifacts (designated ant target) to Nexus after I retrieve the artifacts in my cache or local repository? (it seems like a really bad practice, but the only option is always a great option)
Is this a misuse of the tools?
The ibiblio resolver is the best way to configure a Maven repository.
The following ivysettings.xml file configures ivy to pull it's dependencies Nexus.
<ivysettings>
<settings defaultResolver="central-proxy"/>
<resolvers>
<ibiblio name="central-proxy" m2compatible="true" root="http://mynexushost/nexus/content/repositories/central/"/>
<ibiblio name="release-repo" m2compatible="true" root="http://mynexushost/nexus/content/repositories/releases/"/>
</resolvers>
</ivysettings>
The "central-proxy" resolver is configured to pull content from the default Maven Central proxy repository that comes by default in Nexus.
The "release-repo" resolver can be used when publishing content to the Nexus release repository.

Resolve Ivy dependency version order

In my build artifact repository, I have builds from both a branch and a trunk.
I have tested that my resolve works to the branch, and grabs the correct item.
But this doesn't work if the dependency is also in the trunk repo. In this case it grabs the dependency from the trunk and not my branch.
I need resolve to pull a dependency from the trunk repo if it's not in the branch, but if the dependency is in the branch, I need the branch version, not the trunk version.
How can I do this in Ivy? I'd prefer it to just find the branch version, then stop looking for it. That way if it's missing from the branch, it "falls through" to the trunk version.
Is it possible the way I am using build numbers is incorrect for this situation?
Here is an example:
trunkbuild - version = "1.0.4"
branchbuild - version = "1.0-SNAPSHOT"
Should my branches be "1.1-SNAPSHOT" instead?
The code in my branches should always be a version AHEAD, not behind the trunk.
I think it's dynamic revisions is what you're looking for:
<dependency org="mygroup" name="myartifact" revision="latest.release"/>
<dependency org="mygroup" name="myartifact" revision="latest.integration"/>
Maven repositories supports two basic types of repository:
Release
Snapshot
Only artifacts built on my trunk branch are published to the Release repository.
So the following ivy resolver setup should be enough to get it working, with "latest.integration" resolving to artifacts in the snapshots repo.
<ivysettings>
<settings defaultResolver="default"/>
<resolvers>
<chain name="default">
<ibiblio name="nexus-central" root="http://myhost/nexus/content/repositories/central" m2compatible="true"/>
<ibiblio name="nexus-releases" root="http://myhost/nexus/content/repositories/releases" m2compatible="true"/>
<ibiblio name="nexus-snapshots" root="http://myhost/nexus/content/repositories/snapshots" m2compatible="true"/>
</chain>
</resolvers>
</ivysettings>
Note:
You could create a repository group within your repository manager and simplify the ivy setup to a single URL.

Resources