Ant: Exclude files from merged jar file - ant

For a java project I'd like to merge all third-party jars it depends on into the main jar created by Apache Ant, which I already managed to do.
The problem is that some of these jar-files have signature-files in their META-INF-directories, so when I try to run my jar-file, I get the error message "Invalid signature file digest for Manifest main attributes". After I delete the signature-files manually the error is gone.
I tried to filter the signature files out in my ant-file with an excludes-attribute or an exclude-tag, but nothing seems to have any effect.
This is the ant-task:
<target name="jar" description="Creates the jar file">
<mkdir dir="${jar}"/>
<jar destfile="${jar}/${ant.project.name}.jar" level="9" filesetmanifest="mergewithoutmain">
<zipgroupfileset dir="${lib}" includes="*.jar"/>
<zipfileset dir="${class}"/>
<manifest>
<attribute name="Main-Class" value="${mainclass}"/>
</manifest>
</jar>
</target>
How can I filter files from the resulting jar in this ant-task? Thanks for your help!

carej is right. I've been trying to do this, merging other jars into my application jar excluding some files, and there is no way to use <zipgroupfileset> for it.
My solution is a variant of the unzip/clean-up/jar method: I first merge all the external library jars into one with <zipgroupfileset>, then merge it into mine with <zipfileset> which does allow filtering. In my case it works noticeably faster and is cleaner than unzipping the files to disk:
<jar jarfile="${dist}/lib/external-libs.jar">
<zipgroupfileset dir="lib/">
<include name="**/*.jar"/>
</zipgroupfileset>
</jar>
<sleep seconds="1"/>
<jar jarfile="${dist}/lib/historadar-${DSTAMP}.jar" manifest="Manifest.txt">
<fileset dir="${build}" includes="**/*.*"/>
<zipfileset src="${dist}/lib/external-libs.jar">
<exclude name="*"/>
</zipfileset>
</jar>
The first <jar> puts all the jars it finds in lib/ into external-libs.jar, then I make it wait for one second to avoid getting warnings about the files having modification dates in the future, then I merge my class files from the build/ directory with the content of external-libs.jar excluding the files in its root, which in this case were README files and examples.
Then I have my own README file that lists all information needed about those libraries I include in my application, such as license, website, etc.

To the best of my knowledge there's no way to filter when using <zipgroupfileset>: the include/excludes used there apply to the zips to be merged, not the content within them.
If you have a well-known set of JARs to merge you could use individual <zipset> entries for each one; this approach allows using include/exclude to filter the contents of the source archive.
An alternative approach is to simply unzip everything into a temporary location, remove/modify the unwanted bits, then zip everything back up.

You can use the exclude parameter in zipfileset tag to remove content from merged external JAR files, as this:
<jar jarfile="${dist}/lib/external-libs.jar">
<zipgroupfileset dir="lib/" excludes="META-INF/**/*">
<include name="**/*.jar"/>
</zipgroupfileset>
</jar>
The resulting JAR file will be unsigned.

Alberto's answer works fine but takes time to unzip&rezip the archive. I implemented a new Ant task to use built-in filtering functions, that results in much faster execution:
public class FilterZipTask extends Task {
private Zip zipTask;
private List<FileSet> groupfilesets = new ArrayList<FileSet>();
private String excludes;
public void setExcludesInZips(String excludes) {
this.excludes = excludes;
}
public void addZipGroupFileset(FileSet set) {
groupfilesets.add(set);
}
public void addZip(Zip zipTask) {
this.zipTask = zipTask;
}
#Override
public void execute() throws BuildException {
for (FileSet fileset : groupfilesets) {
Iterator<FileResource> iterator = fileset.iterator();
while (iterator.hasNext()) {
ZipFileSet zfs = new ZipFileSet();
FileResource resource = iterator.next();
zfs.setSrc(resource.getFile());
zfs.setExcludes(excludes);
zipTask.addZipfileset(zfs);
}
}
zipTask.execute();
}
}
And use it in build file as follows:
<taskdef name="filterzip" classname="FilterZipTask"/>
<filterzip excludesInZips="META-INF/*.*">
<zipgroupfileset dir="${deps.dir}" includes="*.jar" />
<zip destfile="${destjar}" />
</filterzip>

Have been struggling with the same issue for a few hours, so ended up writing a new task by extending the existing one:
import java.util.HashSet;
import java.util.Set;
import java.util.StringTokenizer;
import org.apache.tools.ant.taskdefs.Jar;
import org.apache.tools.ant.types.ResourceCollection;
import org.apache.tools.ant.types.ZipFileSet;
public class CustomizedJarTask extends Jar {
protected Set<String> filters;
#Override
public void add(ResourceCollection resources) {
if (filters != null && resources instanceof ZipFileSet) {
ZipFileSet set = ZipFileSet.class.cast(resources);
for (String filter : filters)
set.createExclude().setName(filter);
}
super.add(resources);
}
public void setFilters(String patterns) {
StringTokenizer tokenizer = new StringTokenizer(patterns, ", ", false);
while (tokenizer.hasMoreTokens()) {
if (filters == null)
filters = new HashSet<>();
filters.add(tokenizer.nextToken());
}
}
}
With the above, all I need in the build file is:
<taskdef name="customized-jar" classname="CustomizedJarTask" classpath="${basedir}/bin/" />
<customized-jar jarfile="${directory.build}/external-libs.jar" duplicate="fail" filters="META-INF/**">
<zipgroupfileset dir="${directory.libs}">
<include name="**/*.jar" />
</zipgroupfileset>
</customized-jar>

I was also facing same problem. Googled a lot and found something that worked for me.
Un-jar you jar file delete .
META-INF/.SF ,
META-INF/.DSA
files. Jar it again and run it should not show the error message.
Cause of error is explained here: http://qe-cafe.blogspot.in/2010/06/invalid-signature-file-digest-for.html

Related

yguard not updating properties file in the jar

I have jar file having some properties files in it like log4j.properties and config.properties. Following is my ant script for yguard. Everything else is working but the properties file updation.
<target name="yguard">
<taskdef name="yguard" classname="com.yworks.yguard.YGuardTask" classpath="lib/yguard.jar" />
<yguard>
<inoutpairs resources="none">
<fileset dir="${basedir}">
<include name="MyApp.jar" />
</fileset>
<mapper type="glob" from="MyApp.jar" to="MyAppObs.jar" />
</inoutpairs>
<externalclasses>
<pathelement location="lib/log4j-1.2.17.jar" />
</externalclasses>
<rename conservemanifest="true" mainclass="com.amit.Application" >
<adjust replaceContent="true" >
<include name="**/*.properties" />
</adjust>
</rename>
</yguard>
</target>
config.properties file
com.amit.Application.param1 = something
I found some question in stackoverflow but they didn't help. One place it was mentioned that the file (like jsp, xml, properties) should be in the jar file which I already have. But my yguard obfuscated file just get the files copied as it is.
I tried many combinations with rename & adjust tags but nothing worked for me.
Following post I already visited
Is it possible to manage logs through Obfuscation with yGuard?
How to include obfuscated jar file into a war file
Apparently you want yGuard to obfuscate the name of the field param1, because com.amit.Application is obviously your entry point and yGuard excludes the given main class automatically. So basically you want the outcome to be something like
com.amit.Application.AÖÜF = something
This isn't possible, because yGuard can only adjust class names in property files, as state here: yGuard Manual

Ant copy task corrupts UTF-8 symbols

I have a .properties file with translations in Arabic. I am using it to replace strings in an html file. However, when I start the copy task, it completely corrupts the symbols and I get something like this:
اÙÙزادات
Any idea what's causing this and how I can fix it?
build.xml
<target name="copyAndReplace">
<copy todir="..." overwrite="yes" encoding="UTF-8">
<fileset dir="..." includes="*.html"></fileset>
<filterset>
<filtersfile file="***.properties" />
</filterset>
</copy>
</target>
I see some possible problems:
In Java, Properties files are assumed to have ISO-8859-1 encoding. Even if you're not dealing directly with Java, ant is reading a property file. I've run into this problem when opening a property file in Vim and NetBeans editor. Vim saved it in UTF-8 and NetBeans in ISO-8859-1.
You should use the outputencoding attribute of copy task. In Windows, UTF-8 is not the default encoding.
i encountered the same issue, but with images.
In the ant manual i found the following remark:
Note: If you employ filters in your copy operation, you should limit the copy to text files. Binary files will be corrupted by the copy operation. This applies whether the filters are implicitly defined by the filter task or explicitly provided to the copy operation as filtersets. See encoding note.
Maybe that is the source of the problem. I will need to check on my own whether this solves my problem.
Kind regards,
Marc
As mentioned by #Jean Waghetti above, ANT expects the files to be ISO-8859-1 encoded. I posted a similar stack overflow question for Chinese characters.
The only solution I've found is by ensuring my .properties file was ISO-8859-1 and the characters were escaped.
For example مرحبا بالعالم
Would be:
\u0645\u0631\u062D\u0628\u0627 \u0628\u0627\u0644\u0639\u0627\u0644\u0645
This is not ideal as it's not terribly human-readable. I have noticed that eclipse automatically converts it on hover.
You can add some code to translate the utf-8 properties to iso-8859-1 properties and the use the converted and escaped properties
<project name="xyz" default="copyAndReplace">
<property name="srcdir" value="src" />
<property name="propdir" value="src" />
<property name="tmpdir" value="tmp" />
<target name="encodeProps">
<script language="javascript">
importPackage(java.io);
importPackage(java.lang);
var files = new java.io.File(propdir).listFiles();
for (var i in files) {
var f = files[i];
if (!f.getName().endsWith(".properties")) continue;
var io = new InputStreamReader(new FileInputStream(f), "utf-8");
var out = new FileOutputStream(new File(tmpdir, f.getName()));
do {
var c = io.read();
if (c == -1) break;
if (c > 127) {
var s = Integer.toHexString(c);
s = new StringBuilder().append("\\u").append("0000".substring(s.length())).append(s).toString();
out.write(s.getBytes());
} else {
out.write(c);
}
} while (true);
io.close();
out.close();
}
</script>
</target>
<target name="copyAndReplace" depends="encodeProps">
<copy todir="dst" overwrite="yes" encoding="UTF-8" filtering="true">
<fileset dir="${srcdir}" includes="*.html">
</fileset>
<filterset>
<filtersfile file="${tmpdir}/c.properties" />
</filterset>
</copy>
</target>
</project>

Preserve Original Filenames in Groovy Ant Task

I have the following code:
new AntBuilder().zip( destFile: "${file}.zip" ) {
fileset( dir: srcDir ) {
include( name:pattern )
}
}
In this example I'd like ant to create a zip with the same name as the original file, but with a .zip added to the end. Is there a way to do this without knowing the original file's name ahead of time in ant? I'd like to be able to do the same thing with other ant tasks as well.
To put it another way, I'd like the filename to become whatever "pattern" resolves to for each file.
Something like this?
<target name="zip-files">
<taskdef name="groovy" classname="org.codehaus.groovy.ant.Groovy" classpathref="build.path"/>
<dirset id="dirsToZip" dir="src">
<include name="dir*"/>
</dirset>
<groovy>
project.references.dirsToZip.each {
ant.zip(destfile: "${it}.zip", basedir: it)
}
</groovy>
</target>
If find the groovy task's ability to iterate thru a fileset or dirset a very useful feature.

Ant writing property file storing filtered properties

I'm facing a very simple problem with ANT script. I have a script that loads and sets many properties loaded from several property files in file system. This properties are used to preconfigure a new project.
The question is: can I write a new property file persisting all the properties that starts with a given prefix (for example "ref.proj.*")?
The number and the name of the properties is variable and so I cannot use the
<propertyfile file="my.properties">
<entry key="ref.proj.first" value="${ref.first}"/>
...
<entry key="ref.proj.n" value="${ref.n}"/>
</propertyfile>
It's possibile to apply a filter to a propertyfile task?
Thanks in advance!
It's taking too long for me to work out all of the kinks. Sorry...
You should look at the <echoproperties> task. This will let you select the various properties and print them out in property = value format.
You could use that as your properties file itself.
The following example uses the groovy ANT task:
<path id="build.path">
<pathelement location="lib/groovy-all-2.1.0.jar"/>
</path>
<target name="create-properties">
<taskdef name="groovy" classname="org.codehaus.groovy.ant.Groovy" classpathref="build.path"/>
<groovy>
new File("my.properties").withWriter { writer ->
properties.findAll { it.key.startsWith("ref.proj") }.each {
writer.println it
}
}
</groovy>
</target>

Find all directories in which a file exists, such that the file contains a search string

I have a directory tree that I need to process as follows:
I have a certain file that needs to be copied to a select few sub directories
A sub directory of interest is one that contains a file within which I can regex match a known search string
Ideally I would like to:
Perform a regex match across all files within a directory
If the regex matches, copy the file to that directory
The trouble is that I am quite new to ANT and I'm having difficulties finding my way around. I can't find any tasks in the docs about per directory operations based on regex search. The closest thing I've found is a regex replace task (<replaceregexp>) that can search and replace patterns across files.
Is this even possible? I'd really appreciate a sample to get started with. I apologize for requesting code - I simply don't know how to begin composing the tasks together to achieve this.
Alternatively I have the option of hardcoding all the copy operations per directory, but it would mean manually keeping everything in sync as my project grows. Ideally I'd like to automate it based on the regex search/copy approach I described.
Thanks!
Your requirement is a bit non-standard, so I've solved it using a custom Groovy task.
Here's a working example:
<project name="find-files" default="copy-files">
<!--
======================
Groovy task dependency
======================
-->
<path id="build.path">
<pathelement location="jars/groovy-all-1.8.6.jar"/>
</path>
<taskdef name="groovy" classname="org.codehaus.groovy.ant.Groovy" classpathref="build.path"/>
<!--
=========================
Search for matching files
=========================
-->
<target name="search-files">
<fileset id="filesContainingSearchString" dir="src">
<include name="**/*.txt"/>
<containsregexp expression="[4-6]\.[0-9]"/>
</fileset>
</target>
<!--
===================================
Copy file into each directory found
===================================
-->
<target name="copy-files" depends="search-files">
<groovy>
project.references.filesContainingSearchString.each { file ->
def dir = new File(file.toString()).parent
ant.copy(file:"fileToBeCopied.txt", toDir:dir)
}
</groovy>
</target>
</project>
Notes:
Groovy jar can be downloaded from Maven Central
Use the copy task with a fileset and regular expression selector :
<copy todir="your/target/dir">
<fileset dir="rootdir/of/your/directorytree" includes="**/*.txt">
<containsregexp expression="[4-6]\.[0-9]"/>
</fileset>
</copy>
This example is taken from the ant manual and slightly adapted.
Means select all files with .txt extension anywhere beyond rootdir/of/your/directorytree that match the regular expression (have a 4,5 or 6 followed by a period and a number from 0 to 9) and copy them to your/target/dir.
Just adapt it for your needs.

Resources