Is there any way in an Ant script to use the <copy> task (replaceTokens) to preprocess Java files and replace a "__FILE__" string with the file name of the copied file and "__LINE__" with the line number in that file? This would go a long way toward solving Java's lack of a preprocessor.
The docs on replaceTokens show examples of fixed replacement strings, but I can't find any way to have Ant put in the file name and line number, respectively.
Do I need to write my own task handler?
Here's what I came up with. It does __LINE__, but not __FILE__. Oh well :-(.
<target name="preprocess" description="Preprocess the source code">
<mkdir dir="${target.generated.dir}"/>
<copy todir="${target.generated.dir}" includeemptydirs="true" failonerror="true" verbose="true">
<fileset dir="${source.dir}">
<exclude name="${source.main.resources.dir}"/>
</fileset>
<filterchain>
<tokenfilter>
<filetokenizer/>
<scriptfilter language="beanshell" byline="true"><![CDATA[
import java.io.BufferedReader;
import java.io.StringReader;
int count = 1;
BufferedReader br = new BufferedReader(new StringReader(self.getToken()));
StringBuilder builder = new StringBuilder();
String line;
while ((line = br.readLine()) != null) {
builder.append(line.replace("\"__LINE__\"", Integer.toString(count))).append('\n');
count++;
}
self.setToken(builder.toString());
]]></scriptfilter>
</tokenfilter>
</filterchain>
</copy>
</target>
Related
I have a .properties file with translations in Arabic. I am using it to replace strings in an html file. However, when I start the copy task, it completely corrupts the symbols and I get something like this:
اÙÙزادات
Any idea what's causing this and how I can fix it?
build.xml
<target name="copyAndReplace">
<copy todir="..." overwrite="yes" encoding="UTF-8">
<fileset dir="..." includes="*.html"></fileset>
<filterset>
<filtersfile file="***.properties" />
</filterset>
</copy>
</target>
I see some possible problems:
In Java, Properties files are assumed to have ISO-8859-1 encoding. Even if you're not dealing directly with Java, ant is reading a property file. I've run into this problem when opening a property file in Vim and NetBeans editor. Vim saved it in UTF-8 and NetBeans in ISO-8859-1.
You should use the outputencoding attribute of copy task. In Windows, UTF-8 is not the default encoding.
i encountered the same issue, but with images.
In the ant manual i found the following remark:
Note: If you employ filters in your copy operation, you should limit the copy to text files. Binary files will be corrupted by the copy operation. This applies whether the filters are implicitly defined by the filter task or explicitly provided to the copy operation as filtersets. See encoding note.
Maybe that is the source of the problem. I will need to check on my own whether this solves my problem.
Kind regards,
Marc
As mentioned by #Jean Waghetti above, ANT expects the files to be ISO-8859-1 encoded. I posted a similar stack overflow question for Chinese characters.
The only solution I've found is by ensuring my .properties file was ISO-8859-1 and the characters were escaped.
For example مرحبا بالعالم
Would be:
\u0645\u0631\u062D\u0628\u0627 \u0628\u0627\u0644\u0639\u0627\u0644\u0645
This is not ideal as it's not terribly human-readable. I have noticed that eclipse automatically converts it on hover.
You can add some code to translate the utf-8 properties to iso-8859-1 properties and the use the converted and escaped properties
<project name="xyz" default="copyAndReplace">
<property name="srcdir" value="src" />
<property name="propdir" value="src" />
<property name="tmpdir" value="tmp" />
<target name="encodeProps">
<script language="javascript">
importPackage(java.io);
importPackage(java.lang);
var files = new java.io.File(propdir).listFiles();
for (var i in files) {
var f = files[i];
if (!f.getName().endsWith(".properties")) continue;
var io = new InputStreamReader(new FileInputStream(f), "utf-8");
var out = new FileOutputStream(new File(tmpdir, f.getName()));
do {
var c = io.read();
if (c == -1) break;
if (c > 127) {
var s = Integer.toHexString(c);
s = new StringBuilder().append("\\u").append("0000".substring(s.length())).append(s).toString();
out.write(s.getBytes());
} else {
out.write(c);
}
} while (true);
io.close();
out.close();
}
</script>
</target>
<target name="copyAndReplace" depends="encodeProps">
<copy todir="dst" overwrite="yes" encoding="UTF-8" filtering="true">
<fileset dir="${srcdir}" includes="*.html">
</fileset>
<filterset>
<filtersfile file="${tmpdir}/c.properties" />
</filterset>
</copy>
</target>
</project>
I have the following code:
new AntBuilder().zip( destFile: "${file}.zip" ) {
fileset( dir: srcDir ) {
include( name:pattern )
}
}
In this example I'd like ant to create a zip with the same name as the original file, but with a .zip added to the end. Is there a way to do this without knowing the original file's name ahead of time in ant? I'd like to be able to do the same thing with other ant tasks as well.
To put it another way, I'd like the filename to become whatever "pattern" resolves to for each file.
Something like this?
<target name="zip-files">
<taskdef name="groovy" classname="org.codehaus.groovy.ant.Groovy" classpathref="build.path"/>
<dirset id="dirsToZip" dir="src">
<include name="dir*"/>
</dirset>
<groovy>
project.references.dirsToZip.each {
ant.zip(destfile: "${it}.zip", basedir: it)
}
</groovy>
</target>
If find the groovy task's ability to iterate thru a fileset or dirset a very useful feature.
I am new to ANT.
I have a very specific scenario to handle in this:
STEP-1: I need to look for the pattern of filenames in certain ear files. If the pattern matches then I need to extract those files.
STEP-2: And if any file is extracted from a certain ear (similar to zip-file) file, then I need to search for another set of files, and copy those set of files too.
The case to handle is "How to identify if a file is copied from a particular archive" if found then proceed to step 2, else move to next archive.
I have achieved STEP-1 but no idea how to achieve step-2.
STEP-1
<!-- Set via arguments passed -->
<patternset id="pattern.needtocopy" includes="${needtocopyfile.pattern}" excludes="${ignore.pattern}">
</patternset>
<target name="get-binaries-from-baseline">
<for param="binary">
<path>
<fileset dir="${baseline.dir}/target/aaa/bbb/ccc" includes="*.ear" />
</path>
<sequential>
<basename file="#{binary}" property="#{binary}.basename" />
<unzip src="#{binary}" dest="${baseline.dir}">
<patternset refid="pattern.needtocopy" />
<mapper type="flatten" />
</unzip>
</sequential>
</for>
</target>
STEP-2:
????
Need help in this.
Thanks.
Well I resolved the same, using a groovy script based on the resources I could find.
<target name="findJars">
<zipfileset id="found" src="${ear-name}">
<patternset refid="${patternsetref}" />
</zipfileset>
<groovy>
project.references.found.each {
println it.name
println project.properties.'ear-name'
println project.properties.'dest.dir'
}
</groovy>
</target>
And then I added another task which takes this filename and ear-file-name as input and extracts the related jars based on file to search pattern.
The question is likely VERY trivial for anyone familiar with ant, of which I only use the basics thus far.
I know how to rename files, e.g. I already use:
<copy todir="build/css/">
<fileset dir="css/">
<include name="*.css"/>
</fileset>
<globmapper from="*.css" to="*-min.css"/>
</copy>
I know how to calculate an MD5:
<checksum file="foo.bar" property="foobarMD5"/>
I don't know how to include the second into the first, to rename all those files to include their MD5 - the purpose is to serve as webbrowser cache buster. The other cache-busting option, to append "?[something]" is not as good, as is explained on some Google webmaster pages, having the MD5 as part of the name is better.
I managed to produce a somewhat strange solution using for from ant contrib.
But you have to install ant contrib first.
The copy in the sequential does not seem to accept/evaluate mappers (it wouldn't work, I tried with ant 1.7.0), so I had to create an extra move with a filtermapper to create the results.
It does the following:
for each file create an md5sum and save it in property foobarMD5
the property has to be unset before each iteration
I create a new file in the same dir named example.java_foobarMD5.java (Notice that the filename contains the fileextension)
I move all files with .java_ in its name to a new Folder and remove the .java_
I leave this example with .java.
<for param="file">
<path>
<fileset dir="src/" includes="**/*.java"/>
</path>
<sequential>
<echo>Letter #{file}</echo>
<var name="foobarMD5" unset="true"/>
<checksum file="#{file}" property="foobarMD5"/>
<echo>${foobarMD5}</echo>
<copy file="#{file}" tofile="#{file}_${foobarMD5}.java"/>
</sequential>
</for>
<move todir="teststack" verbose="true">
<fileset dir="src/">
<include name="**/*java_*"/>
</fileset>
<filtermapper>
<replacestring from=".java_" to="-"/>
</filtermapper>
</move>
You could do this without having to include ant contrib. I had to implement this for work and was not allowed to introduce that extension for security reasons. The solution I came to was this:
<target name="appendMD5">
<copy todir="teststack">
<fileset dir="css/" includes="**/*.css"/>
<scriptmapper language="javascript"><![CDATA[
var File = Java.type('java.io.File');
var Files = Java.type('java.nio.file.Files');
var MessageDigest = Java.type('java.security.MessageDigest');
var DatatypeConverter = Java.type('javax.xml.bind.DatatypeConverter');
var buildDir = MyProject.getProperty('builddir');
var md5Digest = MessageDigest.getInstance('MD5');
var file = new File(buildDir, source);
var fileContents = FIles.readAllBytes(file.toPath());
var hash = DatatypeConverter.printHexBinary(md5Digest.digest(fileContents));
var baseName = source.substring(0, source.lastIndexOf('.'));
var extension = source.substring(source.lastIndexOf('.'));
self.addMappedName(baseName + '-' + hash + extension);
]]></scriptmapper>
</copy>
</target>
It is worth noting that I wrote this for Java 8 but with some minor tweaks it could work on Java 7. Sadly this won't work for earlier versions of Java without more effort.
For a java project I'd like to merge all third-party jars it depends on into the main jar created by Apache Ant, which I already managed to do.
The problem is that some of these jar-files have signature-files in their META-INF-directories, so when I try to run my jar-file, I get the error message "Invalid signature file digest for Manifest main attributes". After I delete the signature-files manually the error is gone.
I tried to filter the signature files out in my ant-file with an excludes-attribute or an exclude-tag, but nothing seems to have any effect.
This is the ant-task:
<target name="jar" description="Creates the jar file">
<mkdir dir="${jar}"/>
<jar destfile="${jar}/${ant.project.name}.jar" level="9" filesetmanifest="mergewithoutmain">
<zipgroupfileset dir="${lib}" includes="*.jar"/>
<zipfileset dir="${class}"/>
<manifest>
<attribute name="Main-Class" value="${mainclass}"/>
</manifest>
</jar>
</target>
How can I filter files from the resulting jar in this ant-task? Thanks for your help!
carej is right. I've been trying to do this, merging other jars into my application jar excluding some files, and there is no way to use <zipgroupfileset> for it.
My solution is a variant of the unzip/clean-up/jar method: I first merge all the external library jars into one with <zipgroupfileset>, then merge it into mine with <zipfileset> which does allow filtering. In my case it works noticeably faster and is cleaner than unzipping the files to disk:
<jar jarfile="${dist}/lib/external-libs.jar">
<zipgroupfileset dir="lib/">
<include name="**/*.jar"/>
</zipgroupfileset>
</jar>
<sleep seconds="1"/>
<jar jarfile="${dist}/lib/historadar-${DSTAMP}.jar" manifest="Manifest.txt">
<fileset dir="${build}" includes="**/*.*"/>
<zipfileset src="${dist}/lib/external-libs.jar">
<exclude name="*"/>
</zipfileset>
</jar>
The first <jar> puts all the jars it finds in lib/ into external-libs.jar, then I make it wait for one second to avoid getting warnings about the files having modification dates in the future, then I merge my class files from the build/ directory with the content of external-libs.jar excluding the files in its root, which in this case were README files and examples.
Then I have my own README file that lists all information needed about those libraries I include in my application, such as license, website, etc.
To the best of my knowledge there's no way to filter when using <zipgroupfileset>: the include/excludes used there apply to the zips to be merged, not the content within them.
If you have a well-known set of JARs to merge you could use individual <zipset> entries for each one; this approach allows using include/exclude to filter the contents of the source archive.
An alternative approach is to simply unzip everything into a temporary location, remove/modify the unwanted bits, then zip everything back up.
You can use the exclude parameter in zipfileset tag to remove content from merged external JAR files, as this:
<jar jarfile="${dist}/lib/external-libs.jar">
<zipgroupfileset dir="lib/" excludes="META-INF/**/*">
<include name="**/*.jar"/>
</zipgroupfileset>
</jar>
The resulting JAR file will be unsigned.
Alberto's answer works fine but takes time to unzip&rezip the archive. I implemented a new Ant task to use built-in filtering functions, that results in much faster execution:
public class FilterZipTask extends Task {
private Zip zipTask;
private List<FileSet> groupfilesets = new ArrayList<FileSet>();
private String excludes;
public void setExcludesInZips(String excludes) {
this.excludes = excludes;
}
public void addZipGroupFileset(FileSet set) {
groupfilesets.add(set);
}
public void addZip(Zip zipTask) {
this.zipTask = zipTask;
}
#Override
public void execute() throws BuildException {
for (FileSet fileset : groupfilesets) {
Iterator<FileResource> iterator = fileset.iterator();
while (iterator.hasNext()) {
ZipFileSet zfs = new ZipFileSet();
FileResource resource = iterator.next();
zfs.setSrc(resource.getFile());
zfs.setExcludes(excludes);
zipTask.addZipfileset(zfs);
}
}
zipTask.execute();
}
}
And use it in build file as follows:
<taskdef name="filterzip" classname="FilterZipTask"/>
<filterzip excludesInZips="META-INF/*.*">
<zipgroupfileset dir="${deps.dir}" includes="*.jar" />
<zip destfile="${destjar}" />
</filterzip>
Have been struggling with the same issue for a few hours, so ended up writing a new task by extending the existing one:
import java.util.HashSet;
import java.util.Set;
import java.util.StringTokenizer;
import org.apache.tools.ant.taskdefs.Jar;
import org.apache.tools.ant.types.ResourceCollection;
import org.apache.tools.ant.types.ZipFileSet;
public class CustomizedJarTask extends Jar {
protected Set<String> filters;
#Override
public void add(ResourceCollection resources) {
if (filters != null && resources instanceof ZipFileSet) {
ZipFileSet set = ZipFileSet.class.cast(resources);
for (String filter : filters)
set.createExclude().setName(filter);
}
super.add(resources);
}
public void setFilters(String patterns) {
StringTokenizer tokenizer = new StringTokenizer(patterns, ", ", false);
while (tokenizer.hasMoreTokens()) {
if (filters == null)
filters = new HashSet<>();
filters.add(tokenizer.nextToken());
}
}
}
With the above, all I need in the build file is:
<taskdef name="customized-jar" classname="CustomizedJarTask" classpath="${basedir}/bin/" />
<customized-jar jarfile="${directory.build}/external-libs.jar" duplicate="fail" filters="META-INF/**">
<zipgroupfileset dir="${directory.libs}">
<include name="**/*.jar" />
</zipgroupfileset>
</customized-jar>
I was also facing same problem. Googled a lot and found something that worked for me.
Un-jar you jar file delete .
META-INF/.SF ,
META-INF/.DSA
files. Jar it again and run it should not show the error message.
Cause of error is explained here: http://qe-cafe.blogspot.in/2010/06/invalid-signature-file-digest-for.html