I am looking to separate development and production environments by publishing jars to ivy with live and dev qualifiers.
I am looking for a way to trigger ivy from projects that have these dependencies to automatically grab the latest from these environments based on the ant build file.
I am new to ant and ivy and am not finding documentation on if this is possible or not.
Basically, build-live in ant would trigger resolve-live that would use ref="[1.live.0,)", however we would also need a default one for developers in an Eclipse environment to automatically pick up dependencies through the plugin.
You have not indicated what type of repository you are using. I'm going to assume you are not using a Maven repository manager to manage your release repository. Some of these support remote workflow to manage what I like to call "release candidates" (for an example take a look at the staging feature provided by Sonatype Nexus)
For a pure ivy solution I first recommend reading the best practice documentation, specifically the section titled "Dealing with integration versions".
When publishing a new ivy module version one can set the status field. Out of the box ivy supports "integration", "milestone" or "release" but even these can be extended. The status is a label or metadata attribute that appears in the info field of the published ivy file within the ivy repository.
How does this work? When publishing the module as follows:
<ivy:publish resolver="???" pubrevision="1.0.1" status="integration">
<artifacts pattern="build/artifacts/jars/[artifact].[ext]" />
<artifacts pattern="build/artifacts/zips/[artifact].[ext]" />
</ivy:publish>
This states that the release 1.0.1 is an integration release.
This then enables the functionality you're looking for. Ivy's dynamic revisions capability can be used to automatically download the latest version with a particular status as follows:
<dependency org="acme" name="foo" rev="latest.integration" />
Update
Once a module is published into the repository it cannot be changed. Doing so could potentially break builds that rely on that version. Think about, if you change the status of a build how to communicate that change? Instead you use the "status" to indicate how stable a version is. Open source projects will frequently publish several "general availability" or "milestone" releases before the finally approved major version.
To do what you want to do requires server-side repository file management. I recommend looking at the "staging" suite in Sonatype Nexus. This feature keeps each pending release in a temp repo until it's finally approved and merged into the main release area.
Related
Is there any Repository manager that manages the binary dll files and also integrates well with the Jenkins?
Can Nexus be used to manage the dll files as these files are created as a part of Embedded C/C++ Projects and not sure if Nexus Artifact Manager supports/integrates well with such Projects as it mainly supports the Java projects?
Is there a way to automatically manage the upload and download of such project artifacts from Nexus/other artifact managers without the use of POM file?
Suggest in case there are other Artifact Managers that supports binary artifacts.
Artifactory can be used to store any type of binaries.
Starting with Artifactory 4.0, you can create generic repositories which allows uploading packages of any type. You will not need to upload any POM files and Artifactory will not need to calculate any metadata (for example Maven metadata).
To deploy files you can use the REST API or the UI, for example:
curl -uUSER:PASS -T file.dll http://localhost:8081/artifactory/dll-local/path/to/file.dll
If you have a certain layout you would like to use for this repository you can create a custom layout and associate it with the repository. This can be useful for automatic snapshot/integration versions cleanup and other module management tasks.
Disclaimer: I'm affiliated with Artifactory
The Nexus repository manager is java oriented, but can be used to store any files you want. Binaries of all types or even just text configuration files.
To automate the file upload process, you can use maven from command line:
mvn deploy:deploy-file -DgroupId=com.you -DartifactId=file -Dversion=1.0 -Dpackaging=exe -Dfile=c:\out\file.exe -Durl=http://yourserver/nexus/content/repositories/releases -DrepositoryId=releases
Then, to get the file, you should be able to get it directly with the following URL:
wget http://yourserver/nexus/content/repositories/releases/com/you/file/1.0/file-1.0.exe
This is a simple approach to using Nexus as a general artifact repository.
I hope this helps.
The open source version of Nexus (Nexus OSS) is supports many repository formats out of the box including Maven, NuGet, NPM, RubyGems and others. Nexus just runs on Java (e.g. like Jenkins). It is not Java only...
Depending on how you plan to get the DLL files from the repository, different formats might be more or less suited to your usage. You could even use a custom format, but then you rely custom tools.
The scenarios I have seen at many customers are
using a Maven repo and pulling the files in either in a Maven build together with the Maven NAR Plugin (used for native development with C/C++)
using a Maven repo and pulling via plan HTTP GET calls using your scripting language/build tool of choice
using NuGet format and store the DLLs in NuGet packages in the repo and using nuget to retrieve them for the projects
All of these work well.
We have the build system, which builds a lot of components. Built components publish their artifacts by ivy in artifactory.
I want to attach an extra meta-artifact to each component during its publication, but transparently for developers. This meta-artifact will contain the information about build-agent, build-log, changelog, etc.
I have access to build agents, so I can:
change some ivy configuration files on build agents
install a customized ivy with a patch on build agents
modify the ivysettings.xml file
I didn’t find any information in ivy documentation. However, I know that there are methods like “options.getExtraArtifacts()” in the ivy source code, so I guess there is a way.
Waiting for your suggestions.
Thanks in advance!
UPD:
I figured out “options.getExtraArtifacts()” serve an "artifact" subtag in a "publish" tag:
https://ant.apache.org/ivy/history/2.4.0/use/publish.html
Option1 : Artifactory Metadata
Are you using Artifactory Pro? I'm not a user myself but the documentation describes a capability to attach arbitrary XML based metadata to any artifact stored in Artifactory. Standalone REST API that should be simple to call from your build:
https://www.jfrog.com/confluence/display/RTF2X/Attaching+and+Reading+Metadata
The big advantage of this approach is that this data is searchable
Option 2: Jenkins plugin
Again I'm not an Artifactory user, but have always been intrigued by this plugin that reportedly delivers tight integration between Jenkins and Artifactory. The relevent bit is the so called "Build info" that is published by the plugin. I would be surprised if this doesn't include some of the info your looking for.
https://wiki.jenkins-ci.org/display/JENKINS/Artifactory+Plugin
Option 3: Attach a custom metadata file
A typical Maven repository has no support for metadata. Both Artifactory and Nexus have bespoke extensions and it appears that in both products this capability is a paid feature...
If all you want to do is store metadata and don't care about the lack of a search interface, then why not simply attach a file containing your metadata as an additional Module file? The following example documents how to do this:
how to publish 3rdparty artifacts with ivy and nexus
Option 4: Ivy extra attributes
This is only an option if you're storing data in an ivy repository (You are more likely using a Maven repo format).
Ivy has the ability to store custom attributes in the ivy module file.
http://ant.apache.org/ivy/history/latest-milestone/concept.html#extra
The following answer describes how these can also be used to influence the layout of your ivy repository.
Dynamically updating Ivy extra attributes
We are using ant with ivy through Jenkins to do our daily builds. Initially we want to publish our artifacts to our test Ivy repository and once testing is done, we want to copy the same artifacts to our released Ivy repository without rebuilding them. Does the Ivy or Jenkins has any tasks that help with this? Also, when we publish them to test repository we are planning on using integration status, is there a way we can change the status to release when we copy to released Ivy repository? We have multiple modules that we do builds for but only selected of them need to be moved to released Ivy repository. Any help is greatly appreciated.
You can use the install task to copy artifacts between repositories.
A better solution to consider is using the staging suite a feature of Nexus professional. It works by creating a temporary repository for each release candidate, which can be promoted until deemed worthy of release. Other repository managers have similar features, worth considering rather than building your own.
We recently converted to maven, I'm really new to it. I've imported the project into eclipse, the "build automatically" by default is on, and I have no idea what it does. Is there a way for me to see the logging message, to configure it so I can make it not to, say, use the default goal, but a different goal every time? BTW, currently, building in CLI is fine, but the resource filtering is not working in eclipse with "build automatically" turned on.
Thanks in advance!
David
If a maven plugin will be executed in an incremental eclipse build depends on the "lifecycle mapping" configuration.
You will find (detailed) information about this topic on the "M2E plugin execution not covered" site.
A quote from this site:
We call these instructions "project build lifecycle
mapping" or simply "lifecycle mapping" because they define how m2e
maps information from project pom.xml file to Eclipse workspace
project configuration and behaviour during Eclipse workspace build.
There are several possibilities how the lifecycle mapping can be configured (in the pom, in m2e extensions, since m2e 1.1 (SNAPSHOT) in metadata provided by the plugin).
You should have a look at this site and hopefully you will be able to setup a configuration/installation that fits your needs.
At my new gig, they use Ant and cannot be persuaded to move to Maven.
I've looked everywhere for a decent example of how a multi-project ant build system should be assembled. The apache site falls short. I'm looking specifically for best practices to:
Automatically build local projects that are dependencies of a project
Share artifacts from project to their dependents
Export a project's dependencies and generated artifacts (jars) to be inherited by dependent projects
Share third-party dependencies between projects
I'm sure I can do all this without using Ivy - what did people do before Ivy? I really don't want to have to set up a corporate repository or rely on external repositories - the engineers here are really against that and have all their third-party jars checked into src control.
Can anyone point me at a good open source example of a multi-project ant build?
I don't have too much hands on experience with building large numbers of dependent projects with Ant, but this tutorial looks like it will do what you need without any additional tools.