OutOfMemoryError creating clover-snapshot in Ant running from QuickBuild - ant

I have a QuickBuild server (5.0.14). This is running a build step using ant.
The Ant target looks like this:
<target name="clover-snapshot" depends="with-clover">
<clover-snapshot file="${clover.snapshot.file}"/>
</target>
We are using Clover Version 3.1.5.
This step is failing with the following stacktrace:-
java.lang.OutOfMemoryError: Java heap space
at com.cenqua.clover.util.CloverBitSet.read(CloverBitSet.java:71)
at com.cenqua.clover.PerTestRecordingTranscript.read(PerTestRecordingTranscript.java:45)
at com.cenqua.clover.RecordingTranscripts.readSliceFromDisk(RecordingTranscripts.java:124)
at com.cenqua.clover.RecordingTranscripts$FileRef.read(RecordingTranscripts.java:354)
at com.cenqua.clover.CoverageDataCollator.collatePerTestRecordings(CoverageDataCollator.java:156)
at com.cenqua.clover.CoverageDataCollator.loadCoverageData(CoverageDataCollator.java:68)
at com.cenqua.clover.CloverDatabase.loadCoverageData(CloverDatabase.java:164)
at com.cenqua.clover.CloverDatabase.loadCoverageData(CloverDatabase.java:159)
at com.cenqua.clover.CloverDatabase.loadWithCoverage(CloverDatabase.java:283)
at com.cenqua.clover.tasks.CloverSnapshotTask.cloverExecute(CloverSnapshotTask.java:51)
at com.cenqua.clover.tasks.AbstractCloverTask.execute(AbstractCloverTask.java:55)
at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
at sun.reflect.GeneratedMethodAccessor6.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
at org.apache.tools.ant.Task.perform(Task.java:348)
at org.apache.tools.ant.Target.execute(Target.java:392)
at org.apache.tools.ant.Target.performTasks(Target.java:413)
at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1399)
at org.apache.tools.ant.Project.executeTarget(Project.java:1368)
at org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41)
at org.apache.tools.ant.Project.executeTargets(Project.java:1251)
at org.apache.tools.ant.Main.runBuild(Main.java:811)
at org.apache.tools.ant.Main.startAnt(Main.java:217)
at org.apache.tools.ant.launch.Launcher.run(Launcher.java:280)
at org.apache.tools.ant.launch.Launcher.main(Launcher.java:109)
I take it from the call to invoke() in the above that Clover is being called reflectively within the JVM used by Ant.
As such, I have attempted to allocate more heap space to the Ant JVM. I did this by editing the build step in QuickBuild and specifying the following environment variable:-
ANT_OPTS is set to -Xmx1024m -Xms512m
This has not solved the issue. (I plucked these numbers out of the air, the max size was not calculated in any sort of way.)
My question is, how can I allocate additional heap space to Clover to allow it to execute without running out of memory. Or alternatively, what can I do to confirm these settings are being used?

It looks like you did everything correctly. ANT_OPTS is the right way to solve this problem. I've used pretty much the same approach to solve similar problem. Did you use 64-bit JVM? Did you see that space has been actually allocated?
Anyway, there is a good instruction:
I would also suggest to use VisualVM in order to verify JVM has been created properly and identify the actual cause.

There are two way to increase memory size for ant:
Setting the value of ANT_OPTS property in environment var. For example:
set ANT_OPTS=-Xms256m -Xmx1024m -XX:MaxPermSize=120m
Specifying maxmeory attribute in in ant Java target.
For example:
<target name="MergingHugeJars">
<echo message="Merging jars ${InputJars} into ${OutputJar}"/>
<java jvm="${JVM}" classname="com.krishna.test.MergeHugeJars" fork="true" failonerror="true" maxmemory="512m" >
<arg line="${OutputJar} ${InputJars}"/>
</java>
</target>
You can fiddle with the maxmemory="512m" section of the third line.

You can use jstat to attach to a running java process and see if your settings are being used:
$ jstat -gccapacity 19726
NGCMN NGCMX NGC S0C S1C EC OGCMN OGCMX OGC OC PGCMN PGCMX PGC PC YGC FGC
42560.0 681984.0 42560.0 5312.0 5312.0 31936.0 85248.0 1363968.0 85248.0 85248.0 21248.0 169984.0 21248.0 21248.0 0 0
Xmx is the combination of NGCMX and OGCMX. In english, the maximum heap size you specify via -Xmx is split between the Old generation and New generation.
A quick little script to format this information nicely:
jcmd | grep -v JCmd | while read pid name; do echo -n "Xmx for $name "; jstat -gccapacity $pid | sed '1d' | awk '{ print $2 + $8 " MB" }'; done
Xmx for org.example.MyClass 2045952 MB

Related

Ant: exec executable:error code 99

We are building our app on Linux using Ant and compilation is fine and during deploy on websphere (deploy-build.xml) we are using below code
<exec executable="${shell.cmd}" failonerror="true">
<arg line="${wsadmin.cmd} -conntype ${wsadmin.conntype} -profileName ${was.profile} ${security.options} -lang jython -f ${scripts.dir}/app-server/jy/install-app.jy ${archive.location}"/>
</exec>
<echo message="${app.name} (${archive.location} deployed." />
from logs we can see install-app.jy execution is successful. But still we are facing below error:
BUILD FAILED
/home/EBbuild/env-build/b4b/env-build.xml:50: The following error occurred while executing this line:
/home/EBbuild/env-build/deploy-build.xml:185: exec returned: 99
at org.apache.tools.ant.ProjectHelper.addLocationToBuildException(ProjectHelper.java:508)
at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:418)
at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:288)
at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
at java.lang.reflect.Method.invoke(Method.java:613)
at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
at org.apache.tools.ant.Task.perform(Task.java:348)
at org.apache.tools.ant.Target.execute(Target.java:357)
at org.apache.tools.ant.Target.performTasks(Target.java:385)
at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1337)
at org.apache.tools.ant.Project.executeTarget(Project.java:1306)
at org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41)
at org.apache.tools.ant.Project.executeTargets(Project.java:1189)
at org.apache.tools.ant.Main.runBuild(Main.java:758)
at org.apache.tools.ant.Main.startAnt(Main.java:217)
at org.apache.tools.ant.launch.Launcher.run(Launcher.java:257)
at org.apache.tools.ant.launch.Launcher.main(Launcher.java:104)
Caused by: /home/EBbuild/env-build/deploy-build.xml:185: exec returned: 99
at org.apache.tools.ant.taskdefs.ExecTask.runExecute(ExecTask.java:636)
at org.apache.tools.ant.taskdefs.ExecTask.runExec(ExecTask.java:662)
at org.apache.tools.ant.taskdefs.ExecTask.execute(ExecTask.java:487)
at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:288)
at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
at java.lang.reflect.Method.invoke(Method.java:613)
at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
at org.apache.tools.ant.Task.perform(Task.java:348)
at org.apache.tools.ant.Target.execute(Target.java:357)
at org.apache.tools.ant.Target.performTasks(Target.java:385)
at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1337)
at org.apache.tools.ant.helper.SingleCheckExecutor.executeTargets(SingleCheckExecutor.java:38)
at org.apache.tools.ant.Project.executeTargets(Project.java:1189)
at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:416)
in above error message line 185 is exec code pasted above
We are using bash shell (linux)
RC=99 originates from your Jython script, look at the source.
Use either ant -d env-build.xml (as David already mentioned) or :
<echoxml>
<exec executable="${shell.cmd}" failonerror="true">
<arg line="${wsadmin.cmd} -conntype ${wsadmin.conntype} -profileName ${was.profile} ${security.options} -lang jython -f ${scripts.dir}/app-server/jy/install-app.jy ${archive.location}"/>
</exec>
</echoxml>
to see how your properties are resolved / how your Jython script is actually executed.
Afterwards start your Jython script standalone in bash (as David already mentioned) for further investigations.Don't know Jython, but maybe it has a commandline parameter to increase it's noiselevel !?
Is this the Windows cmd.exe you're executing as ${shell.cmd}?
If so, you may need to add /c as the first argument to your command line.
You may also want to try using ant -d (capture the output, it will produce quite a bit) to show you more details what is going on.
This will show you the actual command line being executed, and it might help you find errors. At least you can then run the commend directly from the command line.
There's no special meaning assigned to an exit code of 99. It maybe that your command is executing, hitting an issue and returning an exit code of 99.
It's really hard to say right now without the information from ant -d.

Java no class def found error elasticsearch?

I followed these steps on https://github.com/karmi/tire to install elasticsearch for use with ruby on rails
curl -k -L -o elasticsearch-0.17.6.tar.gz http://github.com/downloads/elasticsearch/elasticsearch/elasticsearch-0.17.6.tar.gz
tar -zxvf elasticsearch-0.17.6.tar.gz
./elasticsearch-0.17.6/bin/elasticsearch -f
I am running this on MinGW on windows 7, and most of the process went
smoothly.
However, as I try to start the server with ./elasticsearch-0.17.6/bin/
elasticsearch -f I get a java no class def found error:
$ ./elasticsearch-0.17.6/bin/elasticsearch
Exception in thread "main" java.lang.NoClassDefFoundError: org/
elasticsearch/bootstrap/ElasticSearch
Caused by: java.lang.ClassNotFoundException:
org.elasticsearch.bootstrap.ElasticSearch
at java.net.URLClassLoader$1.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
Could not find the main class:
org.elasticsearch.bootstrap.ElasticSearch. Program will exit.
What gives?
Mingw translates paths and environment variables into UNIX equivalents. It's designed to work with its own tools, and running something foreign to it, like java.exe, doesn't always work out as you'd expect. In particular, that script builds a Java classpath in the variable ES_CLASSPATH and then passes it to Java using the -cp switch, and I bet it's passing Java a path it simply doesn't understand.
What I would do is find the line that executes java.exe in that script, and make a copy of the line prepended with "echo" so that you can see the command line that's being executed. Then adjust the script to provide a proper Java classpath.

How to execute an interactive application from Ant build script?

From http://ant.apache.org/manual/Tasks/exec.html :
Note that you cannot interact with the
forked program, the only way to send
input to it is via the input and
inputstring attributes. Also note that
since Ant 1.6, any attempt to read
input in the forked program will
receive an EOF (-1). This is a change
from Ant 1.5, where such an attempt
would block.
How do I launch and interact with interactive console program from ant?
What I want to do is similar to drush sqlc functionality, that is launch the mysql client interpreter using the proper database credentials, but not limited to this use case.
Here's a sample use case:
<project name="mysql">
<target name="mysql">
<exec executable="mysql">
<arg line="-uroot -p"/>
</exec>
</target>
</project>
When run using ant :
$ ant -f mysql.xml mysql
Buildfile: /home/ceefour/tmp/mysql.xml
mysql:
Enter password:
BUILD SUCCESSFUL
Total time: 2 seconds
After inputting password, it immediately exits.
Compare this with what happens when executing directly on the shell (expected behavior):
$ mysql -uroot -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 1122
Server version: 5.1.58-1ubuntu1 (Ubuntu)
Copyright (c) 2000, 2010, Oracle and/or its affiliates. All rights reserved.
This software comes with ABSOLUTELY NO WARRANTY. This is free software,
and you are welcome to modify and redistribute it under the GPL v2 license
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql>
You can launch your command via a shell, redirecting standard input/output/error from/to/to /dev/tty, which corresponds to the controlling terminal of the process.
<target name="dbshell" description="Open a shell for interactive tasks">
<exec executable="/bin/sh">
<arg value="-c"/>
<arg value="mysql -u root -p < /dev/tty > /dev/tty 2> /dev/tty"/>
</exec>
</target>
I have tried running on cosnole and if you do not fork it works.
As mentioned in the doc too.
Beside with eclipse there are additional ways to configure inputhandler.
As is acknowledged here.
http://www.coderanch.com/t/419646/tools/java-program-accept-user-input
A clean way to get this work
http://www.myeclipseide.com/PNphpBB2-viewtopic-t-25337.html

EXEC args (value) with quotes on linux from Ant script

bash shell:
./mimic_cmd "startDaemon()"
Corresponding Ant code:
<exec failonerror="true" executable="/bin/mimic_cmd">
<arg value='"startDaemon()"' />
</exec>
Does the Ant code exactly represent the above command at the bash shell? Based on the debug info, it looks like it:
[exec] Executing '/bin/mimic_cmd' with arguments:
[exec] '"startDaemon()"'
[exec]
[exec] The ' characters around the executable and arguments are
[exec] not part of the command.
Execute:Java13CommandLauncher: Executing '/bin/mimic_cmd' with arguments:
'"startDaemon()"'
The ' characters around the executable and arguments are not part of the command.
However, the Ant code returns and exit code of 1 while the Bash shell command returns 0.
Toggling vmlauncher doesn't help, and paths are all correct.
The same Ant code works on windows with the resulting debug output:
[exec] Executing 'C:\bin\mimic_cmd' with arguments:
[exec] '"startDaemon()"'
[exec]
[exec] The ' characters around the executable and arguments are
[exec] not part of the command.
Execute:Java13CommandLauncher: Executing 'C:\bin\mimic_cmd' with arguments:
'"startDaemon()"'
The ' characters around the executable and arguments are not part of the command.
Can you tell us what mimic_cmd is? (Is it an ELF executable, is it a script -- and if so, what is its contents?)
You don't need nor want the double-quotes inside your ANT XML attributes (incidentally, for it to be well-formed XML you should have written them as " not ", but that changes nothing with respect to this discussion) unless your executable expects them. The corresponding ANT code for either of the following (100% equivalent) shell command lines:
./mimic_cmd "startDaemon()"
./mimic_cmd 'startDaemon()'
./mimic_cmd startDaemon\(\)
./mimic_cmd startDaemon"()"
./mimic_cmd startDaemon'()'
...actually is:
<exec failonerror="true" executable="/bin/mimic_cmd">
<arg value="startDaemon()" />
</exec>
...or, for illustrative purposes:
<!-- spawn a shell with your original command line -->
<exec failonerror="true" executable="/bin/sh">
<arg value="-c" />
<arg value="/bin/mimic_cmd "startDaemon()"" />
</exec>
Why that is so is longwinded to explain; suffices to say that, in your specific case, the only time when you'd have to use double quotes would be when ultimately issuing the command via a *nix shell (either interactively or as part of another script or programatically via the execing of sh -c), and only in order for that shell not to think that the round parens () have special meaning. By the time the shell would in turn spawn mimic_cmd it would have already stripped the double quotes (and substituted backslash-escaped sequences etc. -- see how a *nix shell parses its command line) ANT does not run your command via the shell but rather executes it directly, so in this case mimic_cmd finds itself with a bunch of double quotes on its hand which it apparently doesn't know how to handle.
You essentially have to think of it as replacing all forms of shell quoting and escaping with XML escaping and breaing down into <arg/> tags.
Windows' CMD.EXE is special in the sense that, unline *nix shells, it does minimal parsing (and generally does not care about double quotes in program arguments), leaving it up to the program to figure out what you meant by quoting. (This is actually a hard limitation of Windows' CreateProcess which does not have the notion of argv[], leaving it up to each program to intepret lpCommandLine in whichever way it sees fit; some will get rid of the quotes for you, but that behaviour is extremely inconsistent, e.g. issue echo "bla" on the CMD.EXE prompt to see what CMD.EXE's builtins think about quoting.) Again, in your case the round parens () have no meaning for CMD.EXE so you don't need them even when typing the command at a command prompt. As for ANT, on Windows as on *nix platforms, it spwans mimic_cmd via CreateProcess not CMD.EXE so you don't really want to quote anything.

How can I ensure all output from Ant's exec task goes to stdout?

The Ant exec task has an output property which can be used to tell Ant where the output goes. I've used it to redirect the output to a file. The thing is, if I don't do something with the output, the stuff that Ant prints isn't that much of a help - it's not complete.
Is there someway of setting the output property to System.out?
When executing a batch file with ant's apply or exec tasks on Windows, I found there are special cases where some of the stdout and stderr is not captured by ant. (For example: if you call a batch file that in turn calls other commands (like node.exe), then the stdout and stderror from the child node.exe process is lost.)
I spent a long time trying to debug this! It seems that the batch file's stdout and stderr is captured, however commands called by the batch file are somehow not seen by ant. (perhaps because they are separate child processes). Using the output and error attributes as suggested above doesn't help because only some of the stdout and/or stderr is captured.
The solution I came up with (a hack) is to add these arguments at the end of the command:
<!--Next arg: forces node's stderror and stdout to a temporary file-->
<arg line=" > _tempfile.out 2<&1"/>
<!--Next arg: If command exits with an error, then output the temporary file to stdout, -->
<!--delete the temporary file and finally exit with error level 1 so that -->
<!--the apply task can catch the error if #failonerror="true" -->
<arg line=" || (type _tempfile.out & del _tempfile.out & exit /b 1)"/>
<!--Next arg: Otherwise, just type the temporary file and delete it-->
<arg line=" & type _tempfile.out & del _tempfile.out &"/>
Because this hack only applies to windows, remember to add #osfamily="windows" to the apply or exec task. And create similar task(s) for `#osfamily="unix", etc but without these extra arguments.
The output of exec does go to standard out unless you specify the output attribute.
If you want to output to System.out, then simply do not specify the "output" attribute. If you would like to redirect to a file AND print it to System.out, you can use the tee command, which will redirect output to a given file and also echo it to standard out... I do not know if Windows supports "tee" or an equivalent.
Maybe you want to look at the error, logError, and errorproperty attributes of the exec task too. These deal with the handling of the standard error stream from the exec'd process. There may be useful information there that is going awol for some reason - which might account for the incompleteness you see.
But, if the exec'd process decides to close stdout or stderr and send them elsewhere - there's little you can do.
I have faced similar problem: the output of command execution was suppressed. Perhaps that is the side effect when running cmd under WinXP (I an using maven-antrun-plugin). Anyway setting output="con" worked out perfectly:
<configuration>
<target>
<exec executable="cmd" output="con">
<arg value="/c" />
<arg value="..." />
</exec>
</target>
</configuration>
Working with Ant and Gruntjs:
For anyone trying to get this to work using Gruntjs. I was able to get it working by doing the following (in combination with darcyparker's answer).
In my Ant Build File:
<target description="run grunt js tasks" name="grunt">
<exec dir="/path/to/grunt" executable="cmd" failonerror="true">
<arg value="/c"/>
<arg value="jshint.bat"/> // I broke each task into it's own exec
<arg line=" > jshint.log 2<&1"/>
<arg line=" || (type jshint.log & del jshint.log & exit /b 1)"/>
<arg line=" & type jshint.log & del jshint.log &"/>
</exec>
<exec dir="/path/to/grunt" executable="cmd" failonerror="true">
// another grunt task (IE: uglify, cssmin, ect..)
</exec>
</target>
jshint.bat
#echo off
pushd "C:\path\to\grunt\"
#ECHO _____________________________________________
#ECHO GRUNT JSHINT
#ECHO _____________________________________________
grunt jshint --stack >>jshint.log
NOTE: Path to grunt would be where your Gruntfile.js is located. Also note, I had to initially create the log file (to get it to work with darcyparker's answer) which would output the stack trace from that particular task. This would then give me the grunt task stack output from wherever I call my ant target.
Finally note that pushd "C:\path\to\grunt\" won't be necissary if your bat files are in the same directory as your Gruntfile.js.
I was experiencing this same kind of issue trying to get the build process to fail in Ant after Karma tests intentionally failed, and executing them with "grunt test".
Just added /c before "grunt test", and it worked like a charm
<target name="unittest">
<echo>*** KARMA UNIT TESTING ***</echo>
<exec dir="api_ui" executable="cmd" osfamily="windows" logError="yes" failonerror="true">
<arg value="/c grunt test"/>
</exec>
</target>

Resources