I'm running Ant with output fed to a log file:
ant -logfile file.txt target-name
I'd also like to print some simple progress information to the console though. The answer seems to be a BuildEvent listener that writes to the console every time a new target is hit, but the documentation explicitly states:
A listener must not access System.out and System.err directly since ouput on these streams is redirected by Ant's core to the build event system.
Did I miss something? Is there a way to do this?
Ant replaces the System.out & System.err streams to remap messages printed there through it's own logging system.
That said, you can still get access to the ACTUAL OS streams by using java.io.FileDescriptor#out
Actually, the answer is Log4jListener.
There is a sample log4j configuration for logging into both console and file shown in the above link. You can then use an <echo> task with an appropriate level parameter to selectively decide what gets printed to console.
Thanks for the answers! I'm slow, but this is still something that I'd like to get right.
I've managed to get something working more or less like I want using carej's suggested approach with the java.io.FileDescriptor#out stream and an Ant scriptdef like this:
<scriptdef name="progress-text" language="javascript" >
output = new java.io.PrintStream(new java.io.FileOutputStream(java.io.FileDescriptor.err))
output.println(self.text)
</scriptdef>
Now I'm just left wondering how wize is this approach? Is there inherit risk in using the underlying OS streams directly?
EDIT:
2 Points which might be useful to anyone else with a similar question:
This article has a very good description of the Ant I/O system: http://codefeed.com/blog/?p=68
java.lang.System does something very similar to set System.out and System.err in the first place.
All of this gave me a little more confidence in this approach.
Related
I have an ant script which has a taskdef and the task creates an https internet connection and somethin with that SSL stuff is wrong. Thus I want to set the system property javax.net.debug=all to get some more information.
In java I would do this using the -D option, but in ant this is used for ant properties which is not the same as a system property.
If this wouldn't be a taskdef but instead a java task, I could use the sysproperty property, but it is no java-task.
Googling for this is frustratingly complicated because ant properties and system properties in ant are so similar that most search results are about the other (or about the java-task).
Obviously I am not the only one with the problem, but other people's questions that I have found (like here) are unanswered or went for hack (like here).
One way to set such a property is the ANT_OPTS system variable. You have to be very carefully to not simply skim over answers on google that state that options can be set that way, because it sounds so much like not what it does:
The documentation says:
ANT_OPTS - command-line arguments that should be passed to the JVM.
For example, you can define system properties or set the maximum Java
heap size here.
Who what have expected that? ANT_OPTS are options for the JVM and not for ant like the name suggests. The var which is used for ant options is called ANT_ARGS.
Now I can launch ant like this: ANT_OPTS="-Djavax.net.debug=all" ant myTarget and can see tons of log output.
(However this leaves the question open whether such a variable can be set using XML).
You can declare system properties in the xml with <sysproperty key="key" value="value"/>.
http://www.java2s.com/Code/Java/Ant/SetsystempropertiesinAntbuildscript.htm
You can use scripting:
<script language="javascript">
java.lang.System.setProperty('myKey', 'myValue');
</script>
A 2.1.1 Grails app has a filter, and in development mode lines like this get printed on every request:
Filter says params are: [controller:book, action:list]
How can I stop this? (I don't think it is anything in my code doing it)
Are you using the functional test plugin? A quick Google search returned the suspect println you're seeing in the functional test plugin's grails-app/conf/TestFilters.groovy filter. If you are using the plugin, it looks like it's been removed/commited in Mar 2012, so you may need an update.
Otherwise if you're not using the functional test plugin, I would inspect your grails-app/conf/ directory for any *Filters.groovy and it's contents.
I have been scouring the internet for days, I have a problem similar to this.
I need to retrieve the console output in raw (plain) text. But if I can get it in HTML that is fine too, I can always parse it. The only thing is that I need to get it during the build step, which is a problem since the location where it should be available is truncated...
I have tried retrieving the console output from the following URL's (relative to the job):
/consoleText
/logText/progressiveText
/logText/progressiveHTML
The two text ones are plain text and would be perfect if not for the truncation, same goes for the HTML one... exactly what I need - only its truncated....
I am sure it is possible to retrieve this information somehow, since when viewing /consoleFull there is a real-time update of the console, without truncating or buffering.
However, upon examining that web page, instead of finding the content I desired, I found this code where it should have been (I did not include the full pages code, since it would be mostly irrelevant, and I believe those answering would be able to find out and know what should be there on their own)
new Ajax.Request(href,{
method: "post",
parameters: {"start":e.fetchedBytes},
requestHeaders: headers,
onComplete: function(rsp,_) {
var stickToBottom = scroller.isSticking();
var text = rsp.responseText;
if(text!="") {
var p = document.createElement("DIV");
e.appendChild(p); // Needs to be first for IE
// Use "outerHTML" for IE; workaround for:
// http://www.quirksmode.org/bugreports/archives/2004/11/innerhtml_and_t.html
if (p.outerHTML) {
p.outerHTML = '<pre>'+text+'</pre>';
p = e.lastChild;
}
else p.innerHTML = text;
Behaviour.applySubtree(p);
if(stickToBottom) scroller.scrollToBottom();
}
e.fetchedBytes = rsp.getResponseHeader("X-Text-Size");
e.consoleAnnotator = rsp.getResponseHeader("X-ConsoleAnnotator");
if(rsp.getResponseHeader("X-More-Data")=="true")
setTimeout(function(){fetchNext(e,href);},1000);
else
$("spinner").style.display = "none";
}
});
Specifically, I am hoping there is a way for me to get the content from text whatever it may be. I am not familiar with this language and so am not sure how I might be able to get the content I want. Plugins won't help since I want to retrieve this content as part of my script during the build step
You did pretty much good investigation already. I can only add the following: all console related plug-ins I know are designed as a post build actions.
The Log Trigger plugin provides a post-build action that allows Hudson
builds to search their console log for a given regular expression and
if found, trigger additional downstream jobs.
So it looks like there is no straightforward solution to your problem. I can see the following options:
1. Use tee or something similar (applicable to shell build steps only)
This solution is far from being universal, but it can provide quick access to the latest console output, produced by a command or set of command.
tee - read from standard input and write to standard output and files
Using synonyms on the system level other Jenkins build steps can modified in order to produce console output. File with console output can be referenced through Jenkins or using any other way.
2. Modify Jenkins code
You can just do a quick fix for internal usage or provide a patch introducing specific system-wide setting.
3. Mimic /console behavior
Code in your example is used to request updates from the Jenkins server. As you may expect the server side can return piece of information starting with some offset. Let me show.
Periodically console page sends requests to the server:
Parameters are straightforward:
Response is a chunk of information to be added:
Another request with updated offset (start) value
You can easily understand there is no data by analyzing Content-Length
So the answer is: use url/job-name/build-number/logText/progressiveHtml, specify start offset, send request and receive console update.
I had a similar issue, the last part of my Jenkinsfile build script needs to parse the ConsoleLog for particular error messages to put in an email build report.
First attempt: http request.
It felt like a hack, it mostly worked, but ran into issues when we locked down access to the Jenkins server & my build nodes could no longer perform annon http gets on the page
Second attempt: use the APIs to enumerate the log lines.
It felt like the right thing to do, but it failed horribly as my nodes would take 30 minutes to get through the 100 meg log files. My presumption is that the Jenkins server was not caching the file, so each request involved a re-reading of the entire file up until the point of the last read.
Third and most successful solution: run grep on the server.
node('master') {
sh 'grep some_criteria $JENKINS_HOME/workspace/path/to/job/console.log'
}
it was fast, reliable, and it didn't matter how big the log files were.
Yes, this required trust of the Jenkins admin and knowledge of the directory paths on the Jenkins server - but since I was the admin, I trusted myself to do the right thing. Your mileage may vary.
To add some insight: when the Jenkins build was in progress, the response for the .../consoleText URL maxed out at 10000 lines, exactly.
I was using 'requests()' package in Python. I have tried the same URL with curl and again received only the first 10K lines.
Only after the build has finished both methods returned the full log (>22K lines in my case).
I will research further and hope to report back.
[2015-08-18] Update: It seems that this is a known issue (see here) and it's fixed in Jenkins 1.618 and later. I am still running 1.615 so I cannot verify.
Amir
I have an extension-point defined in ant :
<extension-point name="foo"/>
A lot of tasks contribute to this point in several imported ant files :
<bindtargets targets="bar" extensionPoint="foo" />
However I'm kinda lost as to exactly which tasks are contributing. Is there a way to have ant report the tasks that would be triggered by a given extension point ? More generaly, is there a way to display the "call-graph" (or simply the list of dependencies) of an ant task ?
I tried using verbose options for ant (-v and such), with no luck.
Thanks
First of all, you can try to debug the ANT process in your IDE using remote debugging by adding some parameters to ANT_OPTS (mine is set in ~/.profile):
http://blog.dahanne.net/2010/06/03/debugging-any-java-application/
And profiling may help. I found project Antro on ANT Wiki...
http://sourceforge.net/projects/antro
Maybe you can try it out. The project is said to be designed for ANT, which looks promising in solving your problem.
Also you can use Yourkit Java Profiler to do a CPU profiling. YJP can show the call graph of a java application, but I'm not sure if one can find out which are ANT targets.
The following document shows how to start a java application with YJP agent.
http://www.yourkit.com/docs/95/help/agent.jsp
I know of 2 ways to get this information:
You can get the effective target/extension-point invocation sequence from Ant's console logger. To do this, place Ant's logging facility into verbose mode by passing -verbose on the command line to Ant. There are two lines, one after the other, that dump to the console immediately before most targets as they are invoked in your build script:
A line that shows a summary of the targets in the call sequence starting with the text, Build sequence for target(s) 'artifact' is [...].
A line showing the detailed call sequence (nested targets and antcalls included). This line starts with the text, Complete build sequence is [...]. This listing considers, as much as reasonably possible, the evaluation of any if and unless attributes of each target listed at the point the line is logged to the console.
Simply invoke your Ant build as you would normally with the -verbose option and your console should have the information you're looking for.
You can get a pictorial representation of the call sequence using a tool called Grand. However, it hasn't been updated for quite some time and thus doesn't support extension-points (which is what you're concerned with here). It will interpret antcall's, ant, and depend'encies. It doesn't evaluate the if and unless attributes but simply identifies potential execution sequence - more of a dependency hierarchy than an actual call graph. The project is on Github so an update to support extension-points may not be too difficult.
The graphic is rendered using Graphviz.
For an actual call sequence, use option 1.
This is pretty sloppy, but it works. Ant is actually pretty easily scripted, and if you are using at least Java 6 (or it might be Java 7), javascript support is built in and thus can be used right out of the box. This defines a task that will echo the dependencies of any target in call order:
<scriptdef name="listdepends" language="javascript">
<attribute name="target"/>
<![CDATA[
var done = [];
var echo = project.createTask("echo")
function listdepend(t) {
done.push(t.getName());
var depends = t.getDependencies();
while (depends.hasMoreElements()) {
var t2 = depends.nextElement();
if (done.indexOf(t2)==-1) listdepend(project.getTargets().get(t2));
}
echo.setMessage(t.getName());
echo.perform();
}
var t = attributes.get("target");
if (t!=null) {
var targ = project.getTargets().get(t);
listdepend(targ);
}
]]>
</scriptdef>
In your case, you can create a new target (or not) and call it like so:
<target name="listfoo">
<listdepends target="foo"/>
</target>
As I said, this is somewhat sloppy. It probably isn't very fast (although unless your target triggers thousands of others, it probably isn't noticeably slow). It won't handle antcall tasks (although it could be modified to do so easily) or respond to if and unless attributes. If dependencies nest too far, it may hit a recursion depth limit (but I doubt any project nest them deep enough).
The array is used to make sure that each dependency is listed once (ant would only run them once).
In most modern IDEs there is a parameter that you can set to ensure javac gets enough heap memory to do its compilation. For reasons that are not worth going into here, we are tied for the time being to JBuilder 2005/2006, and it appears the amount of source code has exceeded what can be handled by javac.
Please keep the answer specific to JBuilder 2005/2006 javac (we cannot migrate away right now, and the Borland Make compiler does not correctly support Java 1.6)
I realize how and what parameters should be passed to javac, the problem is the IDE doesn't seem to allow these to be set anywhere. A lot of configuration is hidden down in the Jbuilder Install\bin*.config files, I feel the answer may be in there somewhere, but have not found it.
did you find a good solution for that problem?
I have the same problem and the only solution I found is the following:
The environment variable JAVA_TOOL_OPTIONS can be used to provide parameters for the JVM.
http://java.sun.com/javase/6/docs/platform/jvmti/jvmti.html#tooloptions
I have created a batch file "JBuilderw.bat" with the following content:
set JAVA_TOOL_OPTIONS=-Xmx256m
JBuilderw.exe
Each time I start JBuilder using this batch file the env.var. JAVA_TOOL_OPTIONS will be set and javac.exe will receive the setting.
The JVM displays at the end the following message: "Picked up JAVA_TOOL_OPTIONS: -Xmx256m"
Drawback: all virtual machines started by JBuilder will get that setting. :(
Thanks,
JB
Have a look at http://javahowto.blogspot.com/2006/06/fix-javac-java-lang-outofmemoryerror.html
The arguments that you need to pass to JBuilder's javac is "-J-Xms256m -J-Xmx256m". Replace the 256m with whatever is appropriate in your case. Also, remove the quotes.
This should work for java 1.4, java 1.5 and forward.
BR,
~A
"I realize how and what parameters should be passed to javac, the problem is the IDE doesn't seem to allow these to be set anywhere."
I realized now that you know how to pass the right arguments ONLY not where/how to pass those arguments :-(
How about this : Can you locate where is the JAVA_HOME/bin directory that borland uses ? If yes, then you can rename the javac.exe(to say javacnew.exe) with a javac.bat which in turn will call the javacnew.exe (as well as pass the required arguments) ?
I don't know if this will help since I don't use Borland but in Eclipse, this is a setting that you attach to the program you're going to run. Each program you run in the IDE has configuration specific to it, including arguments to the VM. Is there something like that?
Do you have a jdk.config file located in JBuilder2005/bin/?
You should be able to modify vm parameters in that file like:
vmparam -Xms256m
vmparam -Xmx256m
Let me know if this works, I found it on a page talking about editing related settings in JBuilder 2005.
Edit the jbuilder.config file.
Put in comment those two lines:
vmmemmax 75%
vmmemmin 32m
has they ought to be <1Gb and with a > 1Gb PC , 75% is too big?