Is there an ant command which lists all targets in a file and there depends?
Right now I just use a little power shell script to match lines that contain <target but its not really a good solution. Is there any sort of built in command?
The closest is ant -p (or ant -p -v to get more information). This won't list the target dependencies, but I don't see it as a problem: dependencies are not important for the end user (they just tell how the target works).
What's important is what the target does, which is what should be in its description:
<target name="foo" depends="bar" description="Does the foo operation">
...
</target>
I what you really want is the target dependencies, then reading the xml file is the best you can do.
No there isn't but you can do it like this :
<target name="list.targets">
<xslt in="${basedir}\{build.file}"
out="tmp"
style="${display.targets.xsl}">
</xslt>
<delete file="tmp"/>
</target>
Where ${display.targets.xsl} points to the following .xsl file :
<xsl:stylesheet version = '1.0' xmlns:xsl='http://www.w3.org/1999/XSL/Transform'>
<xsl:output method='text'/>
<xsl:template match="/">
<xsl:for-each select="//target">
<xsl:sort data-type="text" select="#name"/>
<xsl:message terminate="no">
Target : <xsl:value-of select="#name"/><xsl:if test="#depends"> depends on : <xsl:value-of select="#depends"/>
</xsl:if>
<xsl:if test="#description">
Description : <xsl:value-of select="#description"/>
</xsl:if>
</xsl:message>
</xsl:for-each>
</xsl:template>
</xsl:stylesheet>
And ${basedir}{build.file} points to your current build.xml file. The output will be something like this :
[xslt] Loading stylesheet D:\Tools\StackOverFlow\test.xslt
[xslt]
[xslt] Target : build
[xslt]
[xslt] Target : modify.trs
[xslt] Description : Modifies .trs file targets
[xslt]
[xslt] Target : regex depends on : modify.trs
Depending on your build.xml of course.
If you search for "ant dependency graph", you'll find some suggestions on how to produce a .dot file from your build file which can be rendered into a visual graph by GraphViz.
Related
I have a script that compile all the projects (around 50) of my solution like following
msbuild "myProjName.dproj" /t:build /p:config="Release" /fileLogger /flp:ErrorsOnly /nologo
This works just fine but takes forever to compile. In order to make it faster to build i've been trying to leverage all the potential of our modern multi-core machines using the '/maxcpucount' switch explained here : http://msdn.microsoft.com/library/bb651793.aspx
I get about the same compilation time on my 4-core CPU dev machine. No perf gains.
Apparently this can only work when projects need dependencies to be built. The others "workers" would then build these dependencies projects in parallel as the main proj.
So i tried to build a project group in delphi and adding all my projects to it and than run the msbuild command on this .groupproj but it is still as slow as it has always been.
Did any of you achieved to build multiple projets at the same time with msbuild?
If yes can you provide me an explanation?
Thanks!
The following applies to RAD Studio XE4, but it may also apply to earlier or later versions. Also, the dependencies defined in the .groupproj will not be honored with this method. The .groupproj I was trying to parallelize had no inter-project dependencies, so I didn't figure out how to handle this.
When you build a .groupproj file with MSBuild using the Build, Clean or Make target, the build doesn't run in parallel because these targets use the CallTarget task to execute other targets, but CallTarget doesn't execute its targets in parallel.
In order to build separate projects in parallel, the MSBuild project must use a single MSBuild task to build multiple projects at once. The targets must be defined like this:
<Target Name="Build">
<MSBuild Projects="#(Projects)" BuildInParallel="true"/>
</Target>
<Target Name="Clean">
<MSBuild Targets="Clean" Projects="#(Projects)" BuildInParallel="true"/>
</Target>
<Target Name="Make">
<MSBuild Targets="Make" Projects="#(Projects)" BuildInParallel="true"/>
</Target>
Add these to the .groupproj, then remove the other <Target> directives as well as the <Import> directive. (CodeGear.Group.Targets defines some targets to build the projects in the proper order and to build dependencies when you ask to build only a subset of the projects, but it overrides the Build, Clean and Make targets defined in the .groupproj.) Note that this only allows you to build all projects, not just a subset.
BuildInParallel was added in MSBuild 3.5. However, since .groupproj files don't specify the ToolsVersion attribute, MSBuild will use the MSBuild task as defined in version 2.0, which didn't support BuildInParallel. There are two options to fix this:
Add ToolsVersion="3.5" (or a later version) to the root <Project> element of your .groupproj file.
Run MSBuild with the /toolsversion:3.5 (or /tv:3.5 for short) command-line parameter (/toolsversion overrides the ToolsVersion specified in all project files.)
After doing this, run MSBuild with the /maxcpucount (or /m) argument and your projects should build in parallel. However, RAD Studio doesn't handle this transformed project group correctly, so you may want to give the file a different extension to make it clear that it's not a standard RAD Studio project group (any extension that ends in proj will do).
The following XSLT stylesheet performs the transformation described above:
<?xml version="1.0" encoding="utf-8"?>
<xsl:stylesheet version="1.0"
exclude-result-prefixes="msbuild"
xmlns="http://schemas.microsoft.com/developer/msbuild/2003"
xmlns:msbuild="http://schemas.microsoft.com/developer/msbuild/2003"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:output method="xml" indent="yes"/>
<xsl:template match="//msbuild:Project">
<xsl:copy>
<xsl:attribute name="ToolsVersion">3.5</xsl:attribute>
<xsl:apply-templates select="#* | node()"/>
<Target Name="Build">
<MSBuild Projects="#(Projects)" BuildInParallel="true"/>
</Target>
<Target Name="Clean">
<MSBuild Targets="Clean" Projects="#(Projects)" BuildInParallel="true"/>
</Target>
<Target Name="Make">
<MSBuild Targets="Make" Projects="#(Projects)" BuildInParallel="true"/>
</Target>
</xsl:copy>
</xsl:template>
<xsl:template match="//msbuild:Target">
<!-- Do not copy -->
</xsl:template>
<xsl:template match="//msbuild:Import">
<!-- Do not copy -->
</xsl:template>
<xsl:template match="#* | node()">
<xsl:copy>
<xsl:apply-templates select="#* | node()"/>
</xsl:copy>
</xsl:template>
</xsl:stylesheet>
You can apply this stylesheet with MSBuild (4.0 or later: XslTransformation was added in MSBuild 4.0) using this project file (where groupproj2parallel.xslt is the XSLT file above):
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<Target Name="Build" Inputs="$(InputPaths)" Outputs="$(OutputPaths)">
<XslTransformation
XmlInputPaths="$(InputPaths)"
XslInputPath="groupproj2parallel.xslt"
OutputPaths="$(OutputPaths)" />
</Target>
</Project>
You need to specify InputPaths and OutputPaths explicitly on the command line with /p:InputPaths="..." /p:OutputPaths="...", or by specifying them on the Properties parameter of an MSBuild task. (Alternatively, you can just hardcode the file names in the project file.)
The target definitions provided with MSBuild for C# and Visual Basic projects handle dependencies by using the <ProjectReference> items defined in project files, instead of defining dependencies in the solution file. Delphi .dproj files and C++ Builder .cbproj files don't support this, as the underlying CodeGear.Common.Targets doesn't reuse the machinery defined in Microsoft.Common.Targets for <ProjectReference>.
There are two ways to build a Delphi projects: MSBuild or DCC32.exe. MSBuild is recommended as the project files (dproj and groupproj) encapsulate all configuration settings.
However, there are extra over head using MSBuild compare to plain old DCC32.exe. Furthermore, using MSBuild to build Delphi Project Group (.groupproj) doesn't bring any benefifs for multi-core CPUs. The build performance is same as single core CPU.
Here are my statistics to build a 290 dproj files in one single groupproj:
MSBuild a `groupproj` contains 290 `dproj` on 2C/4T CPU: ~100s
MSBuild a `groupproj` contains 290 `dproj` on 4C/8T CPU: ~100s
MSBuild 290 `dproj` run in multi-threads on 2C/4T CPU: ~121s
MSBuild 290 `dproj` run in multi-threads on 4C/8T CPU: ~50s
DCC 290 `dproj` run in multi-threads on 2C/4T CPU: ~37s
DCC 290 `dproj` run in multi-threads on 4C/8T CPU: ~24s
From the reading, we can conclude that MSBuild introduce extra overhead compare to DCC32. To fully utilize CPU cores and threads available, DCC32 is the way to go by sacrifice the convenient of project configuration encapsulation design for .DPROJ.
A msbuild script to build Delphi groupproj in parallel is available at https://github.com/ccy/msbuild.delphi.parallel
Little bit off-topic: you could try the fastdcc part of the IDE fix pack to get faster builds:
http://andy.jgknet.de/blog/ide-tools/ide-fix-pack/
For example, I got a build time of 1 minute going down to 22s!
Let me first provide the background of the problem I'm facing.
I have a directory structure as below.
c:\myDirectory
c:\myDirectory\Project1
c:\myDirectory\Scripts
Under the c:\myDirectory\Scripts there is a script that download the source code (from svn) and creates the c:\myDirectory\Project1 directory.
I have another ant scripts ( c:\myDirectory\Scripts**compile-source.xml ) that compiles the Project1
from an ant script build.xml that is downloaded to c:\myDirectory\Project1
Snippet for c:\myDirectory\Scripts\compile-source.xml
<project name="compile" default="buildAll" basedir=".">
<property file=".\build.properties">
</property>
.......
<import file="${project.home.path}/${project.name}/build.xml"/>
<target name="buildAll">
<antcall target="jar-pack"/>
</target>
</project>
Snippet for c:\myDirectory\Project1\build.xml.
<project name="CommonFeatures" default="jar-pack" basedir=".">
<description>
A build file for the Common Features project
</description>
....
</project>
Note that the basedir for the project is set as "." for both the above ant scripts.
When I execute the script c:\myDirectory\Scripts\compile-source.xml from the c:\myDirectory\Scripts directory the target "jar-pack" present in the c:\myDirectory\Project1\build.xml gets executed.
However, the problem is that basedir attribude in build.xml ( basedir="." ) is the current working directory and in this case its c:\myDirectory\Scripts. Hence the script build.xml errors out since the basedir for build.xml is expected to be c:\myDirectory\Project1. The build.xml script would have worked, if basedir="." were set to "c:\myDirectory\Project1", but unfortunately build.xml file comes from the source code that is downloaded and I'm unable to edit.
So here's my question, Is it possible to do any of the following.
Override the value of the attribude basedir="." in build.xml when the is done in c:\myDirectory\Scripts\compile-source.xml ?
Is it possible to change the basedir in build.xml by any other mechanism so that the script c:\myDirectory\Project1\build.xml is executed under directory c:\myDirectory\Project1 ?
Any other way to resolve this issue?
Any help from Ant experts to overcome this issue is highly appreciated.
You can update basedir using subant task. Check this answer
Create the following build.xml file (assuming it is in Z:/any/folder):
<?xml version="1.0" encoding="UTF-8"?>
<project name="project">
<target name="mytarget">
<subant target="debug">
<property name="basedir" value="X:/any/dir/with/project"/>
<fileset dir="Y:/any/folder/with" includes="build.xml"/>
</subant>
</target>
</project>
The you can execute ant mytarget from Z:/any/folder
You can specifically reference the location of your build file, which is described in this stack overflow thread. This would allow you to get and use the directory your build file resides in as a reference point.
For your case the usage of the subant or ant tasks may be better suited, but nevertheless...
You can (but you should know/consider the side-effects!) extend ant with the common ant-contrib task definitions and use the var task which is able to override properties. Make sure to use the latest version (> 1.0b3).
<!-- adjust to your path and include it somewhere at the beginning of your project file -->
<taskdef resource="net/sf/antcontrib/antlib.xml" classpath="lib/ant-contrib-1.0b3.jar" />
<!-- works e.g. for basedir = /foo/bar to update it to /foo/bar/.. ~ /foo -->
<var name="basedir" value="${basedir}/.." />
update: but one has to be careful, because this does not change . (current working directory) (so <property name="x" location="tmp" /> would be relative to . and not to basedir anymore ; update: setting basedir outside of ant or via <project basedir= also sets . to basedir!). Here is some test target proving the effect on both:
<target name="tst.dummy.basedir-override">
<!-- example output:
tst.dummy.basedir-override:
[echo] basedir before: basedir=D:\tst, '.'=D:\tst\.
[echo] updating it via 'var' to '..'
[echo] basedir now: basedir=D:\tst/.., '.'=D:\tst\.
-->
<property name="cur" location="." /> <!-- makes the relative path absolute -->
<echo message="basedir before: basedir=${basedir}, '.'=${cur}" />
<echo message="updating it via 'var' to '..'" />
<var name="basedir" value="${basedir}/.." />
<property name="cur2" location="." /> <!-- makes the relative path absolute -->
<echo message="basedir now: basedir=${basedir}, '.'=${cur2}" />
</target>
Using the <xslt> task in ant, how do I get the output to generate to stdout?
My XSLT is generating multiple files through xsl:result-document and the normal output is just status information that I'd like to show up with normal Ant output. Ant seems to force me to supply a destdir= or an out= parameter.
Ant 1.8.2 with Saxon 9
Yes ant does this. However XSLT has the element which you can use to get output on the stdout :)
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">
<xsl:output method="xml" indent="yes"/>
<xsl:key name="types" match="a" use="text()"/>
<xsl:template match="/">
<result>
<xsl:message terminate="no">I am a message from xslt!</xsl:message>
</result>
</xsl:template>
</xsl:stylesheet>
Output :
build:
[xslt] Processing C:\Users\Stefanos\Documents\Visual Studio 2010\Projects\stackOverflow\stackOverflow\test.xml to C:\Users\Stefanos\Documents\Vis
ual Studio 2010\Projects\stackOverflow\stackOverflow\out.xml
[xslt] Loading stylesheet C:\Users\Stefanos\Documents\Visual Studio 2010\Projects\stackOverflow\stackOverflow\test.xslt
[xslt] I am a message from xslt!
BUILD SUCCESSFUL
Total time: 0 seconds
Hope it helps!
I recently had a similar scenario; an Ant script with an XSLT task where the style sheet transform generated multiple files using <xsl:result-document>. Since the Ant XSLT task requires the destdir attribute (unless the out attribute has been specified), I used known temp file(s) for the out destination and then implemented a “cleanup” task which deleted the temp file(s).
<target name="removeTemporaryFiles" description="remove temporary files">
<delete file="${workspace}/temp.xhtml"></delete>
…
</target>
I want create Ivy Ant tasks in xml editor in IDE (Intellij iDEA) with autocomplete based on xsd , but I cannot find xsd for register XML namespace xmlns:ivy="antlib:org.apache.ivy.ant"
Where I can find it?
I just copied the ivy jar to INTELLIJ_HOME/lib/ant and now intellij can resolve the ivy tasks.
Or import this ant file to your ant project, its actually the first ivy example in ivy documentation, make sure to always depend on install-ivy target, add your ant file to idea in the ant build window and you dont even have to install ivy and idea recognizes ivy tasks.
<property name="ivy.jar.dir" value="${user.home}/.ivy2/jars" />
<property name="ivy.jar.file" value="${ivy.jar.dir}/ivy.jar" />
<property name="ivy.install.version" value="2.2.0" />
<target name="check-ivy-installed" unless="INTERN-ivy.jar.exists">
<available property="INTERN-ivy.jar.exists" file="${ivy.jar.file}"/>
</target>
<target name="download-ivy" depends="check-ivy-installed" unless="INTERN-ivy.jar.exists">
<echo message="downloading and installing ivy"/>
<mkdir dir="${ivy.jar.dir}"/>
<!-- download Ivy from web site so that it can be used even without any special installation -->
<get src="http://repo1.maven.org/maven2/org/apache/ivy/ivy/${ivy.install.version}/ivy-${ivy.install.version}.jar"
dest="${ivy.jar.file}" usetimestamp="true"/>
<echo message="ivy installed"/>
</target>
<!-- =================================
target: install-ivy
this target is not necessary if you put ivy.jar in your ant lib directory
if you already have ivy in your ant lib, you can simply remove this
target and the dependency the 'go' target has on it
================================= -->
<target name="install-ivy" depends="download-ivy" description="--> install ivy">
<!-- try to load ivy here from local ivy dir, in case the user has not already dropped
it into ant's lib dir (note that the latter copy will always take precedence).
We will not fail as long as local lib dir exists (it may be empty) and
ivy is in at least one of ant's lib dir or the local lib dir. -->
<echo message="Installing ivy"/>
<path id="ivy.lib.path">
<fileset dir="${ivy.jar.dir}" includes="*.jar"/>
</path>
<taskdef resource="org/apache/ivy/ant/antlib.xml"
uri="antlib:org.apache.ivy.ant" classpathref="ivy.lib.path"/>
</target>
To complete Shalom's answer, the location where to add the ivy.jar for the IntelliJ IDEA Community Edition is INTELLIJ_HOME/lib/ant/lib (one more folder to go).
Maybe it also apply to the full version.
might be, there was no xsd in the past time this discussion started, but at least since may 2011 the ivy scheme is well documented at
http://ant.apache.org/ivy/schemas/ivy.xsd
which is linked right from the documentation in http://ant.apache.org/ivy/
so, to start over using the scheme, you just need:
<?xml version="1.0" encoding="UTF-8"?>
<project name="yourproject"
xmlns:ivy="antlib:org.apache.ivy.ant"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://ant.apache.org/ivy/schemas/ivy.xsd"
>
<!-- … -->
I guess this discussion might help you. It appears that there is no xsd for the ivy ant tasks, but Eclipse does autocomplete.
yes, no xsd for ivy ant tasks avialable. but i found the way to make autocomplete in Intellij IDEA
in ant build file need to define additional task:
<property name="ivy.jar.dir" value="C:/Apache/apache-ivy-2.2.0/"/>
<taskdef resource="org/apache/ivy/ant/antlib.xml"
uri="antlib:org.apache.ivy.ant">
<classpath>
<fileset dir="${ivy.jar.dir}" includes="*.jar"/>
</classpath>
</taskdef>
XML plugin for jEdit defines AntCompleteTask (ant task) that produces an xml file called ant-complete.xml. The resulting file looks as follows:
<element-list>
<!-- ... -->
<element name="classpath"
content="(fileset|dirset|extdirs|existing|filelist|pathelement|path)">
<attribute name="id" type="ID" />
<attribute name="location" type="CDATA" />
<attribute name="cache" type="(true|false|on|off|yes|no)" />
<attribute name="refid" type="CDATA" />
<attribute name="description" type="CDATA" />
<attribute name="path" type="CDATA" />
</element>
<!-- ... -->
The generated file may be downloaded as XML plugin archive. Open XML.jar and browse to xml/completion directory. It's syntax is defined in xml-completion-info.dtd.
The task code does not contain an explicit license, but it's at least GPL. Anyway you probably don't need to distribute that task, only to use it and this doesn't require any license.
I can't assess the usability of the resulting xml file, but jEdit uses it for autocompletion.
I would like to be able to pass an XML file to an ANT build script and have it create a folder structure mimicking the nodal structure of the XML, using the build files parent directory as the root.
For Example using:
<root>
<folder1>
<folder1-1/>
</folder1>
<folder2/>
<folder3>
<folder3-1/>
</folder3>
</root>
ant would create:
folder1
-folder1-1
folder2
folder3
-folder3-1
I know how to create a directory, but i'm not sure how to have ANT parse the XML.
One option would be to use the xslt task to do the heavy lifting. For example, generate a second ant script and invoke it.
build.xml:
<project default="mkdirs">
<target name="mkdirs">
<xslt style="mkdir.xslt" in="dirs.xml" out="mkdir.build.xml"/>
<ant antfile="mkdir.build.xml"/>
</target>
</project>
Place mkdir.xslt in the same directory as build.xml:
<xsl:transform xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">
<xsl:template match="text()"/>
<xsl:template match="root">
<project><xsl:text>
</xsl:text>
<xsl:apply-templates/>
</project>
</xsl:template>
<xsl:template match="*">
<mkdir>
<xsl:attribute name="dir">
<xsl:for-each select="ancestor::*">
<xsl:if test="position() != 1">
<xsl:value-of select="name()"/>
<xsl:text>/</xsl:text>
</xsl:if>
</xsl:for-each>
<xsl:value-of select="name()"/>
</xsl:attribute>
</mkdir><xsl:text>
</xsl:text>
<xsl:apply-templates/>
</xsl:template>
</xsl:transform>
Example mkdir.build.xml output from the xslt task:
<?xml version="1.0" encoding="UTF-8"?><project>
<mkdir dir="folder1"/>
<mkdir dir="folder1/folder1-1"/>
<mkdir dir="folder2"/>
<mkdir dir="folder3"/>
<mkdir dir="folder3/folder3-1"/>
</project>
I'm not fluent in XSLT, so it might be possible to improve on the for-each loop.