Is there a way when designating a resolver ivy pattern to be able to produce the following output?
C:/MyRepository/MyCompany/MyModule/1.2.3/4/ivy.xml
Currently, the [revision] token resolves to the full 4-digit version number. I'd like to be able to use the first three digits of the revision for a part of the pattern and use the last digit for a subfolder below that. Is this possible or would I have to write custom ant code to do this?
Something like this:
<resolvers>
<filesystem name="myresolver">
<ivy pattern="${my.dir}/[organisation]/[module]/[shortversion]/[rev]/ivy.xml" />
<artifact pattern="${my.dir}/[organisation]/[module]/[shortversion]/[rev]/([target])[artifact].[ext]" />
</filesystem>
</resolvers>
where:
${my.dir} = C:/MyRepository/
and the ivy tokens have these values:
[organisation] = MyCompany
[module] = MyModule
[shortversion] = 1.2.3
[rev] = 4
I realize I'm making up these fictitious tokens (shortrevision and rev), but what I'd like to be able to do is get at the revision number parts (major, minor, build, revision) so that I can use them in the pattern.
ivy supports extra attributes which can attached to the dependency declaration as follows:
<ivy-module version="2.0" xmlns:e="http://ant.apache.org/ivy/extra">
..
<dependency org="MyCompany" name="MyCompany" rev="1.2.3" e:buildnum="4"/>
The extra attributes are used as normal within your resolver patterns as follows:
<resolvers>
<filesystem name="myresolver">
<ivy pattern="${my.dir}/[organisation]/[module]/[revision]/[buildnum]/ivy.xml" />
<artifact pattern="${my.dir}/[organisation]/[module]/[revision]/[buildnum]/[artifact].[ext]" />
</filesystem>
</resolvers>
Related
I am trying to tackle some technical debt in our Ant/Ivy system and one of my current tasks is to address some post-retrieve behaviors we currently have. By default, our build system retrieves Ivy dependencies and then extracts compressed artifacts (tar, tar.bz2, gzip, zip only) to a dependency folder, so that our projects have a consistent dependency location:
(project.root)/dependency/.archive <- the compressed dependency location
(project.root)/dependency/extracted-foo` <- the uncompressed dependency
The extraction occurs in a post-retrieve-artifact trigger so that we get the benefit of some of the metadata (paths, names, types, etc., all prefixed with 'dep'.
We currently have one property that can be set to turn off this default behavior for all the dependencies specified in an ivy.xml file. Thus, we are left with an all-or-nothing situation. If we want something in-between, we currently have to use our build.xml file and write some custom code. This is painful because the metadata is not readily available.
I would like to retain the use of the all-or-nothing flag but allow projects to selectively extract items - we have several projects whose build.xml files would be greatly simplified if we could knock the extraction process down to an attribute on the artifact itself.
Thus, my thinking is to use an extra attribute on the artifact tag to "inject" this information and override the ivy.retrieve.pattern to search for this attribute.
Ivy.xml
<ivy-module version="2.0" xmlns:e="http://ant.apache.org/ivy/extra">
<dependencies>
<dependency org="my.org" name="foo" rev="${foo.version}" conf="${conf.archive}->*" transitive="false">
<artifact name="megapin" type="war" e:expand="expand"/>
</dependency>
</dependencies>
</ivy-module>
Build.xml
This is where I think I'm having trouble getting the expand extra attribute to show up.
Question 1: This does add the "extract" attribute to the artifact name at retrieve time. I can use the contains clause to check if that is there in the dep.to Is there a way to retrieve the extra attributes (e.g., ${dep.extra.expand} ?
<property name="ivy.retrieve.pattern" value="${dependency.dir}/[conf]/[artifact]-[rev])(-[expand]).[ext]"/>
</property>
<target name="ivy-post-retrieve-trigger">
<local name="doexpand"/>
<condition property="doexpand">
<contains string="${dep.to}" substring="expand" casesensitive="false"/>
</condition>
<!-- this step works if the flag is set properly, so I'm leaving out these non-relevant steps-->
<...extract if:isset="doexpand"... />
ivysettings.xml
This file basically has the trigger and other resolver settings.
<triggers>
<ant-call target="ivy-post-retrieve-trigger" prefix="dep" event="post-retrieve-artifact"/>
</triggers>
Question 2: Any suggestions on a "noexpand" name? My concern with the <contains> clause is that the "expand" is going to get hit all the time.
I think I am close to getting this working - but the only information I get is: Property "doexpand" has not been set and thus it is skipping the extraction step. Q3 Any tips/advice/examples on how to use the extra attribute on a trigger with Ant/Ivy?
I ended up adding some additional debugging statements to Ivy (as compiled from source). In the ant-ivy/src/java/org/apache/ivy/ant/AntCallTrigger.java I added the following line:
Message.verbose("\tp.name=" + p.getName() + " | p.value=" + p.getValue() );
If I modified my dependency in my Ivy.xml file to be:
<dependency org="my.org" name="foo" rev="${foo.version}" conf="${conf.archive}->*" transitive="false">
<artifact name="megapin" type="war" e:expand="true"/>
</dependency>
This showed
[ivy:retrieve] p.name=dep.expand | p.value=true
At which point I could do something like
<isset property="dep.expand"/>
or
<istrue value=${dep.expand}/>
This answers my Q1. Additionally, I don't need to add this to the Ivy retrieve pattern (thus changing the filename after retrieve), can use a "true" or "false" value as desired (Q2), and this general guidance answers Q3.
Can the token itself be parsed from other values from within the property file?
Is it possible to evaluate the token key, without hardcoding the token? Can the token itself be parsed from other values from within the property file?
For example, if the properties file has the following tokens (test.properties):
module_no = 01
module_code = bb
title_01_aa = ABC
title_02_aa = DEF
title_03_aa = GHI
title_01_bb = JKL
title_02_bb = MNO
title_03_bb = PQR
Contents of build.xml
<?xml version="1.0" encoding="utf-8"?>
<project default="repl">
<property file="test.properties" />
<target name="repl">
<replace file="test.txt" token="module_title" value="title_${module_no}_${module_code}" />
</target>
</project>
Sample content with text:
Welcome to module_title.
The replace task will result in:
Welcome to title_01_bb.
How to achieve this instead?
Welcome to JKL.
This might be very basic, but please do guide me in the right direction. Thank you.
Nested property expansion does not work by default in Ant as described in the documentation:
Nesting of Braces
In its default configuration Ant will not try to balance braces in property expansions, it will only consume the text up to the first closing brace when creating a property name. I.e. when expanding something like ${a${b}} it will be translated into two parts:
the expansion of property a${b - likely nothing useful.
the literal text } resulting from the second closing brace
This means you can't use easily expand properties whose names are given by properties, but there are some workarounds for older versions of Ant. With Ant 1.8.0 and the the props Antlib you can configure Ant to use the NestedPropertyExpander defined there if you need such a feature.
If you check the workarounds link, one solution is to use a macrodef to copy the property:
<property file="test.properties" />
<target name="repl">
<gettitleprop name="titleprop" moduleno="${module_no}" modulecode="${module_code}" />
<replace file="test.txt" token="module_title" value="${titleprop}" />
</target>
<macrodef name="gettitleprop">
<attribute name="name"/>
<attribute name="moduleno"/>
<attribute name="modulecode"/>
<sequential>
<property name="#{name}" value="${title_#{moduleno}_#{modulecode}}"/>
</sequential>
</macrodef>
I'm rewriting build.xml file from Ant to Phing and everything goes fine with one exception.
I need to add new line at the end of each appended file but I can't find any alternative for fixlastline="true".
In Ant it was
<concat destfile="${libraryFilePrefix}.js" fixlastline="yes">
<!-- many filesets -->
</concat>
In Phing it's like
<append destfile="${libraryFilePrefix}.js">
<!-- many filesets -->
</append>
Is there any attribute that works like fixlastline or maybe I need to find another way to achieve this?
I believe, one of the approaches (and possibly the only one) is applying replaceregexp filter on each fileset. You only need to apply filterchain at the beginning and it will do the job for each fileset, like this:
<append destfile="${libraryFilePrefix}.js">
<filterchain>
<replaceregexp>
<regexp pattern="([^\n])$" replace="$1${line.separator}" ignoreCase="true"/>
</replaceregexp>
</filterchain>
<!-- many filesets -->
</append>
As of Phing 3.x the AppendTask is aware of the fixlastline attribute. Your Ant script provided is now working as expected
<project name="concat-supports-fixlastline" default="concat-fixed-lastline" basedir=".">
<target name="concat-fixed-lastline">
<concat destfile="${libraryFilePrefix}.js" fixlastline="yes">
<!-- many filesets -->
</concat>
</target>
</project>
I need to retrieve some values from an HTML file. I need to use Ant so I can use these values in other parts of my script.
Can this even be achieved in Ant?
As stated in the other answers you can't do this in "pure" XML. You need to embed a programming language. My personal favourite is Groovy, it's integration with ANT is excellent.
Here's a sample which retrieves the logo URL, from the groovy homepage:
parse:
print:
[echo]
[echo] Logo URL: http://groovy.codehaus.org/images/groovy-logo-medium.png
[echo]
build.xml
Build uses the ivy plug-in to retrieve all 3rd party dependencies.
<project name="demo" default="print" xmlns:ivy="antlib:org.apache.ivy.ant">
<target name="resolve">
<ivy:resolve/>
<ivy:cachepath pathid="build.path" conf="build"/>
</target>
<target name="parse" depends="resolve">
<taskdef name="groovy" classname="org.codehaus.groovy.ant.Groovy" classpathref="build.path"/>
<groovy>
import org.htmlcleaner.*
def address = 'http://groovy.codehaus.org/'
// Clean any messy HTML
def cleaner = new HtmlCleaner()
def node = cleaner.clean(address.toURL())
// Convert from HTML to XML
def props = cleaner.getProperties()
def serializer = new SimpleXmlSerializer(props)
def xml = serializer.getXmlAsString(node)
// Parse the XML into a document we can work with
def page = new XmlSlurper(false,false).parseText(xml)
// Retrieve the logo URL
properties["logo"] = page.body.div[0].div[1].div[0].div[0].div[0].img.#src
</groovy>
</target>
<target name="print" depends="parse">
<echo>
Logo URL: ${logo}
</echo>
</target>
</project>
The parsing logic is pure groovy programming. I love the way you can easily walk the page's DOM tree:
// Retrieve the logo URL
properties["logo"] = page.body.div[0].div[1].div[0].div[0].div[0].img.#src
ivy.xml
Ivy is similar to Maven. It manages your dependencies on 3rd party software. Here it's being used to pull down groovy and the HTMLCleaner library the groovy logic is using:
<ivy-module version="2.0">
<info organisation="org.myspotontheweb" module="demo"/>
<configurations defaultconfmapping="build->default">
<conf name="build" description="ANT tasks"/>
</configurations>
<dependencies>
<dependency org="org.codehaus.groovy" name="groovy-all" rev="1.8.2"/>
<dependency org="net.sourceforge.htmlcleaner" name="htmlcleaner" rev="2.2"/>
</dependencies>
</ivy-module>
How to install ivy
Ivy is a standard ANT plugin. Download it's jar and place it in one of the following directories:
$HOME/.ant/lib
$ANT_HOME/lib
I don't know why the ANT project doesn't ship with ivy.
Yes this is very possible.
Note that in order to use this solution you will need to set your JAVA_HOME variable to JRE 1.6 or later.
<project name="extractElement" default="test">
<!--Extract element from html file-->
<scriptdef name="findelement" language="javascript">
<attribute name="tag" />
<attribute name="file" />
<attribute name="property" />
<![CDATA[
var tag = attributes.get("tag");
var file = attributes.get("file");
var regex = "<" + tag + "[^>]*>(.*?)</" + tag + ">";
var patt = new RegExp(regex,"g");
project.setProperty(attributes.get("property"), patt.exec(file));
]]>
</scriptdef>
<!--Only available target...-->
<target name="test">
<!--Load html file into property-->
<loadfile srcFile="D:\Tools\CruiseControl\Build\artifacts\RECO\20110831100942\RECO_merged_report.html" property="html.file"/>
<!--Find element with specific tag and save it to property element-->
<findelement tag="title" file="${html.file}" property="element"/>
<echo message="File : ${html.file}"/>
<echo message="Title : ${element}"/>
</target>
</project>
Output : [echo] Title : <title>Test Report</title>,Test Report
As I don't know what exactly variables you were looking for this particular solution will find all elements that you specify in the tag attribute. Of course you could modify the regex to suit your own specific needs.
Also this is pure build.xml ant with no external dependencies whatsoever.
Sure, but you have to write your own task for it. Visit http://ant.apache.org/manual/develop.html#writingowntask for more information about writing own tasks for Ant. In your Ant task you may parse your HTML file as needed.
I claim, that it is not directly possible with "pure" XML (build.xml) to achieve what you want.
Take a look at the (http://ant.apache.org/manual/Tasks/xmlproperty.html) task and see if it'll work for you. It's pretty straight forward:
<xmlProperty file="${html.file}"
prefix="html."/>
After all, HTML is just a subset of XML. I've used it before to do this very task. No need to write your own task or script.
I don't want to use propertyregex in the AntContrib task, but I need to modify a property. I am using the cabarc command (I can't get the <cab> task to work), and I need to strip out the drive name.
${basedir} = "D:\some\directory\blah\blah"
${cwd} = some\directory\blah\blah"
I need this in order to strip out the path in cabarc (but still using directories). I've ended up doing the following:
<!-- Create a property set with just basedir -->
<!-- Needed for loadproperties to work -->
<propertyset id="cwd">
<propertyref name="basedir"/>
</propertyset>
<loadproperties>
<propertyset refid="cwd"/>
<filterchain>
<tokenfilter>
<replaceregex pattern=".:\\"
replace="cwd="/>
</tokenfilter>
</filterchain>
</loadproperties>
That works, but it's a little complex and will be hard to maintain.
Is there an easier way to do this?
get into the groove ;-)
<groovy>
properties.'cwd' = properties.'basedir'[3..-1]
</groovy>
or with Ant Plugin Flaka :
<project xmlns:fl="antlib:it.haefelinger.flaka" name="World">
<!-- simple echo -->
<fl:echo>#{replace('${basedir}', '$1' , '.:\\\\(.+)' )}</fl:echo>
<!-- set property -->
<fl:let>cwd := replace('${basedir}', '$1' , '.:\\\\(.+)' )</fl:let>
</project>
Disclosure = i'm participating as committer in the Flaka project