Ant System Class Loader does not honor $CLASSPATH, honors $LOCALCLASSPATH - ant

I am implementing an ant task as a wrapper for another class, which loads other several classes using the system class loader. Now, the task is in the same jar of these other classes, so I wonder why it is not finding them, since the task is running
Please notice that my classes are in the $CLASSPATH env variable. The problem will not occur if I export LOCALCLASSPATH=$CLASSPATH
Minimal example:
<taskdef name="mytask" classname="my.package.MyTask" />
<target name="compile">
<mytask />
</target>
you can easily see the problem here
public class MyTask extends Task {
public void execute() throws BuildException {
try {
ClassLoader cl = ClassLoader.getSystemClassLoader();
// this will only print the ant jar file path
for(URL url: ((URLClassLoader)cl).getURLs()){
log(url.getFile());
}
cl.loadClass("my.package.OtherClass"); // throws an exception
} catch (Exception ex) {
throw new BuildException(ex);
}
}
}

The ant shell script reworks the classpath internally, so the SystemClassLoader contains only a minimal part of the "real" classpath
tl;dr: use
ClassLoader cl = Thread.currentThread().getContextClassLoader();
instead of
ClassLoader cl = ClassLoader.getSystemClassLoader();
From the Mailing List, Rainer Noack:
if you're launching ant via shell script, it is using
oata.launcher.Launcher.java
This class reorganises the classpath a bit. The env variable CLASSPATH
and the classpath commandline argument are stripped and replaced by
the minimum classpath used to launch ant. A child classloader of the
system classloader is created with the original CLASSPATH entries.
The oata.Project is then loaded with this classloader.
The problem is that the loader in ClassLoader.getSystemClassLoader() is actually untouched, the one that changes (i.e., the one that honors $CLASSPATH, -lib, etc.) can be retrieved using Thread.currentThread().getContextClassLoader().

Related

Missing ant property while configuring worklight database via ant task

I am integrating worklight 6.1 official deployment ant task into gradle build script.
We will utilize below script to run flexible continuous integration process.
def antTaskLibClasspath = "ant-task-lib/worklight-ant-deployer.jar"
def verifyDatabaseArguments(String database){
switch(project.ext[database + 'Type']){
case 'derby':
def dbNameProp = database + 'Name'
def dbDirPathProp = database + 'Dir'
def dbProperties = ['databaseName':project.ext[dbNameProp], 'databaseDir':project.ext[dbDirPathProp] ]
.each { key, value ->
if(!value?.trim()){
def errMsg = "Missing mandatory parameter : ${key}"
println errMsg
throw new StopExecutionException(errMsg)
}
}
project.ext[dbDirPathProp] = Eval.me(""" "${project.ext[dbDirPathProp]}" """).replace("\\", "/")
def dbFile = file(project.ext[dbDirPathProp] + "/${project.ext[dbNameProp]}")
if(!dbFile.exists()){
if(dbFile.mkdirs()){
println "Create folders ${project.ext[dbDirPathProp]} for ${database} derby database"
}else {
def errMsg = "Can not create folders for ${database} derby database"
println errMsg
throw new StopExecutionException(errMsg)
}
}
break
case 'oracle':
break
default:
errMsg = "Unsupported database type"
println errMsg
throw new StopExecutionException(errMsg)
break
}
}
verifyDatabaseArguments("worklightDatabase")
verifyDatabaseArguments("worklightReportsDatabase")
task configure {
description "Configure database for worklight server"
doLast {
ant.typedef(name:'configureDatabase',
classname:'com.ibm.worklight.config.ant.database.ConfigureDatabaseTask',
classpath:antTaskLibClasspath
)
ant.configureDatabase(kind: "Worklight"){
switch(worklightDatabaseType){
case 'derby':
derby(database:worklightDatabaseName, datadir:worklightDatabaseDir)
break
}
}
ant.configureDatabase(kind: "WorklightReports"){
switch(worklightReportsDatabaseType){
case 'derby':
derby(database:worklightReportsDatabaseName, datadir:worklightReportsDatabaseDir)
break
}
}
}
}
And these are properties which were referenced in the script overhead. I've set them in gradle.properties where the build.gradle is located.
worklightDatabaseType=derby
# derby database name
worklightDatabaseName=WRKLGHT
# derby database file directory
worklightDatabaseDir=${System.properties['user.home']}/.derby/ibm
worklightReportsDatabaseType=derby
# derby database name
worklightReportsDatabaseName=WLREPORT
# derby database file directory
worklightReportsDatabaseDir=${System.properties['user.home']}/.derby/ibm
The command line shows ant error message which I can't figure out what's going wrong after I try to configure database with command $ > gradle configure.
Line 75 is the place where I start the first invocation of ant.configureDatabase( ...
Did I miss any mandatory configuration which may not documented in the worklight 6.1?
My jdk version is jdk7_60, gradle version 2.4 without wrapper.
Any suggestion/comment is appreciated, thank you~
By the way, if any consultant from IBM mobilefirst team is watching,
please consider to ask your product team to rename ant task which has dash symbol in it's name.
It seems that Groovy AntBuilder can not load ant element name includes dash symbol,
so we have to unzip worklight ant task jar archives and search for the task class we need in defaults.properties,
then redefine ant task using class name manually in gradle script. That's not convenient.
The build progress of worklight project is complex. We don't like to use ant to setup build automation.
Appended after Peter post his suggestion:
Thank you, Peter.
I have step over the ant library issue after adopting your way.
But it stucks again, and I couldn't interpret the error message.
At first I assume this issue is because worklight 6.1 ant deploy task may not be compatible with gradle,
so I wrote ant build script below to test if my assumption is correct,
which implements the same build logic and variable as previous gradle script does.
build.xml
<?xml version="1.0" encoding="UTF-8"?>
<project basedir="." default="install">
<loadproperties srcfile="build.properties" />
<taskdef resource="com/worklight/ant/defaults.properties">
<classpath>
<pathelement location="worklight-ant-task-libs/worklight-ant-deployer.jar"/>
</classpath>
</taskdef>
<target name="configure">
<configuredatabase kind="Worklight">
<derby database="${worklightDatabaseName}" datadir="${worklightDatabaseDir}"/>
</configuredatabase>
<configuredatabase kind="WorklightReports">
<derby database="${worklightReportsDatabaseName}" datadir="${worklightReportsDatabaseDir}"/>
</configuredatabase>
</target>
</project>
build.properties
# derby database name
worklightDatabaseName=WRKLGHT
# derby database file directory
worklightDatabaseDir=${user.home}/.derby/ibm
# derby database name
worklightReportsDatabaseName=WLREPORT
# derby database file directory
worklightReportsDatabaseDir=${user.home}/.derby/ibm
And my build progress fails again.
I have upload the ant error message log file to google drive.
Can anyone give me some suggestion to help me figure it out?
Thank you~
I'm going to use a lot of guesswork here, but according to this section of the Ant Manual, ant.library.dir is only set by Ant's Launcher class, which Gradle's Ant integration probably doesn't use.
I suspect that the Worklight Ant task you're using assumes that property is set and throws an exception. You can check this by running
gradle --stacktrace configure
assuming that configure is the task you want to run. This will show you where the exception is being thrown from (and any causes too).
You might be best off adding an unpacked Ant distribution to your source tree (or perhaps just its lib directory) and adding the following entry to a gradle.properties file in the root of your project:
systemProp.ant.library.dir=<path to Ant>/lib
Of course you should replace <path to Ant> with the actual path to an Ant installation.

Building a Jar of a Frege project using Gradle

I would like to:
use the Frege programming language to write a simple "Hello World" piece of code,
then using the Frege compiler generating the equivalent Java source code,
then building an executable Jar file to run from the command line,
all the previous steps should be "controlled" by Gradle.
I am able to generate the source code (items 1. and 2. from the previous list), but I am not able to specify a "package" structure of the Java source code in output, i.e. I can not see the package Java statement as the first line of code in the generate Java source code. I can specify to the Frege compiler where to put the generated code though (via the -d argument).
I think this is the reason why when building an executable Jar, then launching it, I am seeing similar errors (according to different attempts on Gradle tasks) e.g.: no main manifest attribute.
The Frege source code is saved in a file named HelloFrege.fr, the generated Java source code is in a file named HelloFrege.java (I verified the file contains the expected main method).
Here there's a version of the Gradle "Jar task":
//create a single Jar with all dependencies
task fatJar(type: Jar) {
manifest {
attributes 'Implementation-Title': 'Hello Frege Jar Example',
'Implementation-Version': version,
'Main-Class': 'HelloFrege'
}
baseName = project.name + '-all'
from { configurations.compile.collect { it.isDirectory() ? it : zipTree(it) } }
with jar
}
Here there's another version of the Gradle "Jar" task:
jar {
manifest {
attributes 'Main-Class': 'HelloFrege'
}
}
How can I solve this problem? I would like to avoid to manually add the package reference to the automatically generated Java source code file.
If your module name in Frege is unqualified such as HelloWorld, you will not see the package statement generated in Java. The module name will become the class name and the package will be empty or default package.
If your module name is qualified such as foo.bar.HelloWorld, then foo.bar will be the package name and HelloWorld will be the class name in the generated Java source.
The rule is that the last part of the module name becomes the class name and the qualifiers form the package name in the generated Java source.
I am not sure what gradle can do for you in this regard, but without gradle, the following should at least be possible:
... build your jar, as before ...
jar -uvfe project-all.jar HelloFrege
java -jar project-all.jar # run it
This, of course, is just another way to create a manifest. If this works, then it would be time to investigate why gradle refuses to do it.
Postscriptum: After thinking another minute about what the problem might be, it occurs to me that you may think that the source file name/path has anything to do with the java package name. This is not so in Frege, though it is good practice to have the file path match the package name, and the file base name match the class name (just like in Java). In addition, to remove some confusion, use the module keyword in frege. As explained by Marimuthu, the Java package and class name is derived from the frege module name.
Example:
$ cat Foo.fr
module my.org.Baz where
...
$ java -jar fregec.jar -d bin Foo.fr
This generates the Baz class in package my.org, and creates the class file in bin/my/org/Baz.class
I am posting here my findings so far. The combination of Gradle commands that works for me is the following one (calling it from the command line typing gradle clean generateJavaSrcFromFregeSrc fatJar):
task generateJavaSrcFromFregeSrc {
ant.java(jar:"lib/frege3.21.586-g026e8d7.jar",fork:true) {
arg(value: "-j") // do not run the java compiler
arg(value: "-d")
arg(value: "src/main/java") // the place where to put the generated source code (paired to the -d argument)
arg(value: "src/main/frege/HelloFrege.fr")
}
}
jar {
manifest {
attributes 'Main-Class': 'org.wathever.HelloFrege'
}
}
task fatJar(type: Jar) {
from files(sourceSets.main.output.classesDir)
from files(sourceSets.main.output.resourcesDir)
//from {configurations.compile.collect {zipTree(it)}} // this does not include the autogenerated source code
baseName = project.name + '-fatJar'
from { configurations.compile.collect { it.isDirectory() ? it : zipTree(it) } }
with jar
}
The manifest details need to be specified in the jar block of code, if I specify them in the task fatJar then when running the jar I get no main manifest attribute, in [...].
If I use just the block of code jar with the property from("$projectDir") { include 'lib/**'} to include the Frege jar, then I get errors like java.lang.ClassNotFoundException (I think because the Jar is included as it is and not as a set of .class files).
The folder src/main/java/org/wathever needs to be there before running Gradle (additional info: the Maven convention prefix src/main/java with as a suffix the "Java package" as specified inside the HelloFrege.fr source code: module org.wathever.HelloFrege where)
Some useful details I found:
How to build a fat Jar
Another how to build a fat Jar
An "Hello Frege" example without the Gradle management
The Gradle documentation on how to use the Jar task

Could not create task or type: getProjectData from Ant

I am trying to run an Ant task from within IBM RSA IDE using Ant build ...
I get the following error message:
BUILD FAILED
build.xml:21: Could
not create task or type of type: getProjectData.
Ant could not find the task or a class this task relies upon.
This is common and has a number of causes; the usual
solutions are to read the manual pages then download and
install needed JAR files, or fix the build file:
- You have misspelt 'getProjectData'.
Fix: check your spelling.
- The task needs an external JAR file to execute
and this is not found at the right place in the classpath.
Fix: check the documentation for dependencies.
Fix: declare the task.
- The task is an Ant optional task and the JAR file and/or libraries
implementing the functionality were not found at the time you
yourself built your installation of Ant from the Ant sources.
Fix: Look in the ANT_HOME/lib for the 'ant-' JAR corresponding to the
task and make sure it contains more than merely a META-INF/MANIFEST.MF.
If all it contains is the manifest, then rebuild Ant with the needed
libraries present in ${ant.home}/lib/optional/ , or alternatively,
download a pre-built release version from apache.org
- The build file was written for a later version of Ant
Fix: upgrade to at least the latest release version of Ant
- The task is not an Ant core or optional task
and needs to be declared using <taskdef>.
- You are attempting to use a task defined using
<presetdef> or <macrodef> but have spelt wrong or not
defined it at the point of use
Remember that for JAR files to be visible to Ant tasks implemented
in ANT_HOME/lib, the files must be in the same directory or on the
classpath
Please neither file bug reports on this problem, nor email the
Ant mailing lists, until all of these causes have been explored,
as this is not an Ant bug.
Here's the Ant buildfile:
<!-- Get property locationName. -->
<target name="config">
<echo message="${ear.project.name}" />
<getProjectData projectName="${ear.project.name}" />
</target>
I am not quite sure what the problem is here because the error message seems not helpful. Any suggestions?
I believe getProjectData is an IBM extension to ant. Like you, I had a similar error, but I was able to get it working after ensuring the Run in the same JRE as the workspace option was enabled (which you can find by right-clicking the build file, run-as, Ant Build..., and selecting the option on the JRE tab).
I discovered the solution on the IBM info center:
The Run in the same JRE as the workspace option enables the classpath
of the workbench to access the additional Ant tasks that perform
operations that are specific to the workbench, such as projectImport,
projectBuild, workspaceBuild, ejbDeploy, or earExport. If your Ant
build script uses any Ant tasks that perform workbench operations,
verify that you selected the Run in the same JRE as the workspace
option; otherwise you might get the following error message in the
Console view:
Problem: failed to create task or type <Ant task> Cause:
The name is undefined.
The build file I used looked like this:
<?xml version="1.0"?>
<project name="Test" default="config" basedir=".">
<target name="config">
<getProjectData Basedir="${basedir}" />
<echo message="getProjectData: projectName=${projectName}
nature=${natureName}
workspace=${workspaceName}
basedir=${basedir}" />
</target>
</project>
And output:
Buildfile: C:\DATA\java\workspace\test-java\build.xml
config:
[getProjectData] Setting projectName=test-java
[getProjectData] Retrieved following Project Data :
[getProjectData] workspaceName=C:\DATA\java\workspace
[getProjectData] natureName=Java
[echo] getProjectData: projectName=test-java
nature=Java
workspace=C:\DATA\java\workspace
basedir=C:\DATA\java\workspace\test-java
BUILD SUCCESSFUL
Total time: 78 milliseconds

Problems with Ant optional tasks SSHExec and SCP. Classpath issue?

I'm in the process of modifying an Ant script (currently in use from within MyEclipse) to work from the command line. I'm doing this so anyone can check out the project and build it without MyEclipse. The problem I'm running into is that MyEclipse includes the dependencies behind the scenes. It does this by looking at the workspace's Ant configuration and compiling the classpath based on the selected libraries in the preferences dialog. Long story short, I need to take those dependencies and make the script smart enough to include them on its own, without the help of MyEclipse.
The tasks that are giving me a headache are the sshexec and scp tasks. They are optional ant tasks that require a version of jsch to run. I removed jsch from MyEclipse's Ant classpath and added it to a lib folder in the project itself (lib/dev). MyEclipse immediately complained that the SSHExec class could not find the dependent class, com.jcraft.jsch.UserInfo which is part of jsch-0.1.44.jar.
I don't see a way to set the classpath for Ant from within the build script. I have the following code, which adds a path element to the script, but I don't think Ant uses this unless explicitly associated to a task or another element.
<path id="web-jars">
<fileset dir="${web-lib}">
<include name="**/*.jar" />
</fileset>
<fileset dir="${app-lib}"> <!-- this is where jsch resides -->
<include name="**/*.jar" />
</fileset>
</path>
It seems that I need to use taskdef to define the sshexec and scp tasks:
<taskdef name="sshexec" classname="org.apache.tools.ant.taskdefs.optional.ssh.SSHExec"
classpathref="web-jars"/>
MyEclipse complains about this, "taskdef A class needed by class org.apache.tools.ant.taskdefs.optional.ssh.SSHExec cannot be found: com/jcraft/jsch/UserInfo"
It's clearly in the classpathref, web-jars. And I can't run anything in the script because of this malformed or misconfigured taskdef.
The problem here is that the SSHExec class is loaded from a classloader which itself has no access to your web-jars class loader. Supplying this classpath for the taskdef does not change this. Each class can only load classes from its own classloader and any parent class loaders, but the web-jars classloader is not a parent class loader of SSHExec's class loader (it is likely the other way around, since SSHExec seems to be found here).
It looks like this:
ClassLoader web-jars -------------> application CL -------------> bootstrap CL
taskdef
=> look for SSHExec here
=> look first in parent class loader
=> look for SSHExec here
=> look first in parent class loader
=> look for SSHExec here
=> not found
=> look in our own classpath
=> found, load the class
=> it somehow uses interface UserInfo
=> look for UserInfo here
=> look first in parent class loader
=> look for UserInfo here
=> not found
=> look in our own classpath
=> not found, throw exception.
The VM has no idea to look for UserInfo (and the other JSch classes) in the web-jars classloader.
I suppose the SSHExec task is somewhere in the usual ant classpath, i.e. loaded by the application class loader. Then removing SSHExec from ant's classpath (or adding jsch.jar to it) seems to be the only solution here.
Create ~/.ant/lib and copy jsch.jar in there as part of the build initialisation. Any tasks which do scp/sshexec work should depend on this init target.
<target name="init">
<property name="user.ant.lib" location="${user.home}/.ant/lib"/>
<mkdir dir="${user.ant.lib}"/>
<copy todir="${user.ant.lib}">
<fileset dir="${basedir}/build/tools" includes="jsch-*.jar"/>
</copy>
</target>
<target name="mytarget" depends="init">
<scp todir="user#host"><fileset dir="..."/></scp>
</target>
The Ant within Eclipse unfortunately won't pick this up immediately as it does not read ~/.ant/lib on every execution; After running mytarget within Eclipse once and watching it fail, then go to:
Window>Preferences>Ant>Runtime and press Restore Defaults - this will add any .jar files from ~/.ant/lib to the Global Entries section and you should be good to go.

What classpath do I need for an Ant taskdef?

I'm new to Ant.
Can someone please tell me what value to put for the 'classpathref' for taskdef?
Will it be the path of the class file?
If yes can an example be given because I tried that and its not working for me.
In the taskdef, the classpathref should be a reference to a previously defined path.
The path should include a jar archive that holds the class implementing the task,
or it should point to the directory in the file system that is the root of the class hierarchy.
This would not be the actual directory that holds your class if your class resides in a package.
Here's an example.
MyTask.java:
package com.x.y.z;
import org.apache.tools.ant.BuildException;
import org.apache.tools.ant.Task;
public class MyTask extends Task
{
// The method executing the task
public void execute() throws BuildException {
System.out.println( "MyTask is running" );
}
}
Note that the package name is com.x.y.z, so when deployed -
lets say the classes are put under a directory called classes - we might see the class here in the file system:
$ ls classes/com/x/y/z
MyTask.class
Here's a simple build.xml that uses the task:
<project name="MyProject" basedir=".">
<path id="my.classes">
<pathelement path="${basedir}/classes" />
</path>
<taskdef name="mytask" classpathref="my.classes" classname="com.x.y.z.MyTask"/>
<mytask />
</project>
Note that the classpathref given points at the classes directory - the root of the class hierarchy.
When run, we get:
$ ant
Buildfile: .../build.xml
[mytask] MyTask is running
You can do similar using an explicit classpath, rather than a 'classpathref', for example:
<property name="my.classes" value="${basedir}/classes" />
<taskdef name="mytask" classpath="${my.classes}" classname="com.x.y.z.MyTask"/>

Resources