In jenkins, you can run a script with sh(). There is an options to returnStdout, which will send stdout to a return value, rather than printing it to the console.
Is it possible to capture stderr in any way? Whether through an option to the function or through some workaround?
Generally, you can redirect a process's STDERR to STDOUT with 2>&1 and collect both.
There's no other way but this redirection workaround at the moment, see https://issues.jenkins-ci.org/browse/JENKINS-44930
Related
I would like to start two different services in my Docker container and exit the container as soon as one of them exits. I looked at supervisor, but I can't find how to get it to quit as soon as one of the managed applications exits. It tries to restart them up to three times, as is the standard setting and then just sits there doing nothing. Is supervisor able to do this or is there any other tool for this? A bonus would be if there also was a way to let both managed programs write to stdout, tagged with their application name, e.g.:
[Program 1] Some output
[Program 2] Some other output
[Program 1] Output again
Since you asked if there was another tool... we designed and wrote a powerful replacement for supervisord that is designed specifically for Docker. It automatically terminates when all applications quit, as well as has special service settings to control this behavior, plus will redirect stdout with tagged syslog-compatible output lines as well. It's open source, and being used in production.
Here is a quick start for Docker: http://garywiz.github.io/chaperone/guide/chap-docker-simple.html
There is also a complete set of tested base-images which are a good example at: https://github.com/garywiz/chaperone-docker, but these might be overkill and the earlier quickstart may do the trick.
I found solutions to both of my requirements by reading through the docs some more.
Exit supervisord on application exit
This can be achieved by using a custom eventlistener. I had to add the following segment into my supervisord configuration file:
[eventlistener:shutdownevent]
command=/shutdownhandler.sh
events=PROCESS_STATE_EXITED
supervisord will start the referenced script and upon the given event being triggered (PROCESS_STATE_EXITED is triggered after the exit of one of the managed programs and it not restarting automatically) will send a line containing data about the event on the scripts stdin.
The referenced shutdownhandler-script contains:
#!/bin/bash
while :
do
echo -en "READY\n"
read line
kill $(cat /supervisord.pid)
echo -en "RESULT 2\nOK"
done
The script has to indicate being ready by sending "READY\n" on its stdout, after which it may receive an event data line on its stdin. For my use case upon receival of a line (meaning one of the managed programs has exited), a SIGTERM is sent to the supervisord process being found by the pid it leaves in its pid file (situated in the root directory by default). For technical completeness, I also included a positive answer for the eventlistener, though that one should never matter.
Tagged output on stdout
I did this by simply starting a tail process in the background before starting supervisord, tailing the programs output log and piping the lines through ts (from the moreutils package) to prepend a tag to it. This way it shows up via docker logs with an easy way to see which program actually wrote the line.
tail -fn0 /var/log/supervisor/program1.log | ts '[Program 1]' &
I'm using supervisord as the entry point for Docker containers as described in https://docs.docker.com/articles/using_supervisord/,
I want all logs to be written to stdout so I can take advantage of builtin tools like docker logs or systemd's journal, especially if running the containers on CoreOS.
for stderr there's redirect_stderr=true option for subprocesses,
is it possible to redirect the subprocess stdout back to supervisord somehow and not deal with actual log files ?
You can redirect the program's stdout to supervisor's stdout using the following configuration options:
stdout_logfile=/dev/fd/1
stdout_logfile_maxbytes=0
Explanation:
When a process opens /dev/fd/1 (which is the same as /proc/self/fd/1), the system actually clones file descriptor #1 (stdout) of that process. Using this as stdout_logfile therefore causes supervisord to redirect the program's stdout to its own stdout.
stdout_logfile_maxbytes=0 disables log file rotation which is obviously not meaningful for stdout. Not specifying this option will result in an error because the default value is 50MB and supervisor is not smart enough to detect that the specified log file is not a regular file.
For more information:
http://veithen.github.io/2015/01/08/supervisord-redirecting-stdout.html
i have a long error message when i execute a task. In order to save
the output i'm writing
task > foo.txt
But when i open foo.txt the error messages are not there.
Any idea? Any alternative?
symfony 1.4/propel
Regards
Javi
Your only redirecting the stdout stream, and you need to trap the stderr stream as well. The syntax varies depending on the shell you're using. For example, in bash
task &> foo.txt
would redirect both stderr and stdout to foo.txt.
In my app, a user can run a script, and if it is run in the terminal, it shows a nice progress bar, how would I replicate this in rails? I mean I could do like something between the two to communicate, but I would prefer if it would just show a "live" shell output.
Thanks :)
If your script is invoked with 2>&1 at the end, backticks invocation will return STDOUT & STDERR.
result = ‘ls 2>&1‘
I am running array jobs with Sun Grid Engine (SGE).
My carefully scripted array job workers generate no stdout and no stderr when they function properly. Unfortunately, SGE insists on creating an empty stdout and stderr file for each run.
Sun's manual states:
STDOUT and STDERR of array job tasks will be written into dif-
ferent files with the default location
.['e'|'o']'.'
In order to change this default, the -e and -o options (see
above) can be used together with the pseudo-environment-vari-
ables $HOME, $USER, $JOB_ID, $JOB_NAME, $HOSTNAME, and
$SGE_TASK_ID.
Note, that you can use the output redirection to divert the out-
put of all tasks into the same file, but the result of this is
undefined.
I would like to have the output files suppressed if they are empty. Is there any way to do this?
No, there is no way to do this.