I have downloaded and extracted Apache Flume.
How can I check if it's ready to run?
To check if Apache-Flume is installed correctly cd to your flume/bin directory and then enter the command flume-ng version.
$ cd <flume home directory>/bin
$ flume-ng version
Make sure that you are in the correct directory by using the ls command. flume-ng will be in the output if you are in the correct directory.
You can try running the command bin/flume-ng agent -c conf -f conf/flume-conf.properties.template -n agent -Dflume.root.logger=INFO,console in the directory you have extracted the Flume binaries.
It should start with some info messages (classpath and maybe SLF4j warnings), some start-up messages and a lot of lines like:
2016-10-13 16:48:22,277 (SinkRunner-PollingRunner-DefaultSinkProcessor) [INFO - org.apache.flume.sink.LoggerSink.process(LoggerSink.java:95)] Event: { headers:{} body: 38 30 32 802 }
I have latest Jenkins and it's Swarm plugin client jar (2.0).
I downloaded the swarm-client jar and I ran the following command to create the slave: (Note: : I'm using user Anonymous to connect to Jenkins master without requiring username/password and have provided Create Slave permission in Global Roles, and all access in Slave section under Slave roles by creating a new role with pattern swarm.*). Jenkins Swarm client: https://wiki.jenkins-ci.org/display/JENKINS/Swarm+Plugin
A user can provide a valid username/password if that user has Create Slave (Global Roles) and other access like connect, delete, etc in Slave Roles in Jenkins under Manage Roles).
$ which swarm-client.jar
~/tools/bin/swarm-client.jar
$ pwd
/home/confman/slaves
$ ~/tools/jdk1.7.0_60/bin/java -jar `which swarm-client.jar` -name "swarm_`hostname -a`" -fsroot "~/slaves/swarm_`hostname -a`" -master http://jenkins_master.srvr.company.com:8083 -username Anonymous -description "`hostname -a` " -executors 5 -labels "ANSIBLE_CENTOS CENTOS" -mode 'exclusive' -showHostName -t java=~/tools/jdk1.8.0_45 -t gradle=~/tools/gradle-2.3 -t Maven=~/tools/apache-maven-3.3.3 -t Groovy2=~/tools/groovy-2.4.5 --showHostName -retry 0 -disableClientsUniqueId &
But, instead of the above command setting Slave's Remote Root Directory to ~/slaves/swarm_jmeter01, it's setting the remote root path including the folder as prefix/relative to from where I ran the above command (as shown below). In my case, hostname command value is jmeter01.
The documentation doesn't mention that this paramter is relative to the current folder (where you'd run the java -jar slave-client jar command).
Has anyone found a solution for this i.e. it can ignore the current folder and set it to ~/slaves/swarm_01 or something like that?
Obviously In my automation to resolve this issue, I can first "cd" (change directory) to the correct folder ~/slaves and then just pass -fsroot "swarm_`hostname -a`" to get remote directory path value set as: ~/slaves/swarm_jmeter01 (in that case, it'll work fine).
~ is shell expansion ... if this not works, then use $HOME instead
I'm having a weird issue with a start-up script which runs a Sinatra script using the shell's "daemon" function. The problem is that when I run the command at the command line, I get output to STDOUT. If I run the command at the command line exactly as it is in the script -- less the daemon part -- the output is correctly redirected to the output file. However, when the startup script runs it (see below), I get stuff to the STDERR log but not to the STDOUT log.
The relevant lines of the script:
#!/bin/sh
# (which is and has been a symlink to /bin/bash
# Source function library.
. /etc/init.d/functions
# Set Some Variables
RUNAS="joeuser"
PID=/var/run/myapp.pid
LOG="/var/log/myapp/app-out.log"
ERR_LOG="/var/log/myapp/app-err.log"
APPLICATION_COMMAND="RAILS_ENV=production ruby /opt/myapp/lib/daemons/my-sinatra-app.rb -p 8002 2>>${ERR_LOG} >>${LOG} &"
# Snip a bunch. This is the applicable line from the "start" case:
daemon --user $RUNAS --pidfile $PID $APPLICATION_COMMAND &> /dev/null
Now, the funky parts:
The error log is written to correctly via the redirect of STDERR.
If I reverse the order of the >> and the 2>> (I'm grasping at straws, here!), the behavior does not change: I still get STDERR logged correctly and STDOUT is empty.
If the output log doesn't exist, the STDOUT redirect creates the file. But, the file remains 0-length.
This used to work. The log directory is maintained by log-rotate. All of the more-recent 'out' logs are 0-length. The older ones are not. It seems like it stopped working some time in April. The ruby code didn't change at any time near then; neither did the startup script.
We're running three different services in this way. Two of them are ruby daemons (one uses sinatra, one does not) and the other is a background java process. This is occurring for BOTH of the ruby processes but is not happening on the java process. Maybe something changed in Ruby?
FTR, we've got ruby 1.8.5 and RHEL 5.4.
I've done some more probing. The daemon function does a bunch of stuff, but the meat of the matter is that it runs the program using runuser. The command essentially looks like this:
runuser -s /bin/bash - joeuser -c "ulimit -S -c 0 >/dev/null 2>&1 ; RAILS_ENV=production ruby /opt/myapp/lib/daemons/my-sinatra-app.rb -p 8002 '</dev/null' '>>/var/log/myapp/app-out.log' '2>>/var/log/myapp/app-err.log' '&'"
When I run exactly that at the command line (both with and without the single-ticks that got added somewhere along the line), I get the exact same screwy behavior w.r.t. the output log. So, it seems to me that this is an issue of how ruby (?) interacts with runuser?
Too long to put in a comment :-)
change the shebang to add #!/bin/sh -x and verify that everything is expanded according to your expectations. Also, when executing from terminal, your .bashrc file is sourced, when executing from script, it is not; might be something in you're environment that differ. One way to find out is to do env from terminal and from script and diff the output
env > env_terminal
env > env_script
diff env_terminal env_script
Happy hunting...
I am running Jenkins from user jenkins thats has $PATH set to something and when I go into Jenkins web interface, in the System Properties window (http://$host/systemInfo) I see a different $PATH.
I have installed Jenkins on Centos with the native rpm from Jenkins website. I am using the startup script provided with the installation using sudo /etc/init.d/jenkins start
Can anyone please explain to me why that happens?
Michael,
Two things:
When Jenkins connects to a computer, it goes to the sh shell, and not the bash shell (at least this is what I have noticed - I may be wrong). So any changes you make to $PATH in your bashrc file are not considered.
Also, any changes you make to $PATH in your local shell (one that you personally ssh into) will not show up in Jenkins.
To change the path that Jenkins uses, you have two options (AFAIK):
1) Edit your /etc/profile file and add the paths that you want there
2) Go to the configuration page of your slave, and add environment variable PATH, with value: $PATH:/followed-by/paths/you/want/to/add
If you use the second option, your System Information will still not show it, but your builds will see the added paths.
I kept running into this problem, but now I just add:
source /etc/profile
As the first step in my build process. Now all my subsequent rules are loaded for Jenkins to operate smoothly.
You can also edit the /etc/sysconfig/jenkins file to make any changes to the environment variables, etc. I simply added source /etc/profile to the end of the file. /etc/profile has all all of the proper PATH variables setup. When you do this, make sure you restart Jenkins
/etc/init.d/jenkins restart
We are running ZendServer CE which installs pear, phing, etc in a different path so this was helpful. Also, we don't get the LD_LIBRARY_PATH errors we used to get with Oracle client and Jenkins.
I tried /etc/profile, ~/.profile and ~/.bash_profile and none of those worked. I found that editing ~/.bashrc for the jenkins slave account did.
The information on this answer is out of date. You need to go to Configure Jenkins > And you can then click to add an Environment Variable key-value pair from there.
eg: export MYVAR=test would be MYVAR is the key, and test is the value.
I found two plugins for that.
One loads the values from a file and the other lets you configure the values in the job configuration screen.
Envfile Plugin — This plugin enables you to set environment variables via a file. The file's format must be the standard Java property file format.
EnvInject Plugin — This plugin makes it possible to add environment variables and execute a setup script in order to set up an environment for the Job.
On my newer EC2 instance, simply adding the new value to the Jenkins user's .profile's PATH and then restarting tomcat worked for me.
On an older instance where the config is different, using #2 from Sagar's answer was the only thing that worked (i.e. .profile, .bash* didn't work).
Couldn't you just add it as an environment variable in Jenkins settings:
Manage Jenkins -> Global properties > Environment variables:
And then click "Add" to add a property PATH and its value to what you need.
This is how I solved this annoying issue:
I changed the PATH variable as #sagar suggested in his 2nd option, but still I got different PATH value than I expected.
Eventually I found out that it was the EnvInject plugin that replaced my PATH variable!
So I could either uninstall EnvInject or just use it to inject the PATH variable.
As many of our Jenkins jobs use that plugin, I didn't want to uninstall it...
So I created a file: environment_variables.properties under my Jenkins home directory.
This file contained the path environment value that I needed:
PATH=$PATH:/usr/local/git/bin/.
From the Jenkins web interface: Manage Jenkins -> Configure System.
In that screen - I ticked the Prepare jobs environment option, and in the Properties File Path field I entered the path to my file: /var/lib/jenkins/environment_variables.properties.
This way every Jenkins job we have receive whatever variables I put in this environment_variables.properties file.
Jenkins also supports the format PATH+<name> to prepend to any variable, not only PATH:
Global Environment variables or node Environment variables:
This is also supported in the pipeline step withEnv:
node {
withEnv(['PATH+JAVA=/path/to/java/bin']) {
...
}
}
Just take note, it prepends to the variable. If it must be appended you need to do what the other answers show.
See the pipeline steps document here.
You may also use the syntax PATH+WHATEVER=/something to prepend /something to $PATH
Or the java docs on EnvVars here.
I only had progress on this issue after a "/etc/init.d/jenkins force-reload". I recommend trying that before anything else, and using that rather than restart.
On my Ubuntu 13.04, I tried quite a few tweaks before succeeding with this:
Edit /etc/init/jenkins.conf
Locate the spot where "exec start-stop-server..." begins
Insert the environment update just before that, i.e.
export PATH=$PATH:/some/new/path/bin
Add
/usr/bin/bash
at
Jenkins -> Manage Jenkins -> configure System -> Shell->Shell
executable
Jenkins use the sh so that even /etc/profile doesn't work for me
When I add this, I have all the env.
Solution that worked for me
source ~/.bashrc
Explanation
I first verified Jenkins was running BASH, with echo $SHELL and echo $BASH (note I'm explicitly putting #!/bin/bash atop the textarea in Jenkins, I'm not sure if that's a requirement to get BASH). sourceing /etc/profile as others suggested was not working.
Looking at /etc/profile I found
if [ "$PS1" ]; then
...
and inspecting "$PS1" found it null. I tried spoofing $PS1 to no avail like so
export PS1=1
bash -c 'echo $PATH'
however this did not produce the desired result (add the rest of the $PATH I expect to see). But if I tell bash to be interactive
export PS1=1
bash -ci 'echo $PATH'
the $PATH was altered as I expected.
I was trying to figure out how to properly spoof an interactive shell to get /etc/bash.bashrc to load, however it turns out all I needed was down in ~/.bashrc, so simply sourceing it solved the problem.
I tried all the things from above - didn't work for me.
I found two solution (both for SSH-Slave)
Go to the slave settings
Add a new environment variable
PATH
${PATH}:${HOME}/.pub-cache/bin:${HOME}/.local/bin
The "${HOME}" part is important. This makes the additional PATH absolute.
Relative path did not work for me.
Option II (pipeline-script)
pipeline {
agent {
label 'your-slave'
}
environment {
PATH = "/home/jenkins/.pub-cache/bin:$PATH"
}
stages {
stage('Test') {
steps {
ansiColor('xterm') {
echo "PATH is: $PATH"
}
}
}
}
}
On Ubuntu I just edit /etc/default/jenkins and add source /etc/profile at the end and it works to me.
Running the command with environment variable set is also effective. Of course, you have to do it for each command you run, but you probably have a job script, so you probably only have one command per build. My job script is a python script that uses the environment to decide which python to use, so I still needed to put /usr/local/bin/python2.7 in its path:
PATH=/usr/local/bin <my-command>
What worked for me was overriding the PATH environment for the slave.
Set: PATH
To: $PATH:/usr/local/bin
Then disconnecting and reconnecting the slave.
Despite what the system information was showing it worked.
I have Jenkins 1.639 installed on SLES 11 SP3 via zypper (the package manager).
Installation configured jenkins as a service
# service jenkins
Usage: /etc/init.d/jenkins {start|stop|status|try-restart|restart|force-reload|reload|probe}
Although /etc/init.d/jenkins sources /etc/sysconfig/jenkins, any env variables set there are not inherited by the jenkins process because it is started in a separate login shell with a new environment like this:
startproc -n 0 -s -e -l /var/log/jenkins.rc -p /var/run/jenkins.pid -t 1 /bin/su -l -s /bin/bash -c '/usr/java/default/bin/java -Djava.awt.headless=true -DJENKINS_HOME=/var/lib/jenkins -jar /usr/lib/jenkins/jenkins.war --javaHome=/usr/java/default --logfile=/var/log/jenkins/jenkins.log --webroot=/var/cache/jenkins/war --httpPort=8080 --ajp13Port=8009 --debug=9 --handlerCountMax=100 --handlerCountMaxIdle=20 &' jenkins
The way I managed to set env vars for the jenkins process is via .bashrc in its home directory - /var/lib/jenkins. I had to create /var/lib/jenkins/.bashrc as it did not exist before.
1- add to your profil file".bash_profile" file
it is in "/home/your_user/" folder
vi .bash_profile
add:
export JENKINS_HOME=/apps/data/jenkins
export PATH=$PATH:$JENKINS_HOME
==> it's the e jenkins workspace
2- If you use jetty :
go to jenkins.xml file
and add :
<Arg>/apps/data/jenkins</Arg>
Here is what i did on ubuntu 18.04 LTS with Jenkins 2.176.2
I created .bash_aliases file and added there path, proxy variables and so on.
In beginning of .bashrc there was this defined.
# If not running interactively, don't do anything
case $- in
*i*) ;;
*) return;;
esac
So it's checking that if we are start non-interactive shell then we don't do nothing here.
bottom of the .bashrc there was include for .bash_aliases
# Alias definitions.
# You may want to put all your additions into a separate file like
# ~/.bash_aliases, instead of adding them here directly.
# See /usr/share/doc/bash-doc/examples in the bash-doc package.
if [ -f ~/.bash_aliases ]; then
. ~/.bash_aliases
fi
so i moved .bash_aliases loading first at .bashrc just above non-interactive check.
This didn't work first but then i disconnected slave and re-connected it so it's loading variables again. You don't need to restart whole jenkins if you are modifying slave variables. just disconnect and re-connect.
If your pipeline is executed on the remote node that is connected via SSH, then actually Jenkins runs agent application that performs incoming actions.
By default zsh shell is used, not the bash (my Jenkins has version 2.346.3).
Furthermore jenkins-agent runs non-login shell which makes default PATH values even if you put some configuration to .zshrc. It will be skipped.
My choice is to put the following shebang at a script start
#!/bin/bash -l
-l option makes bash to run in the login mode and in this case bash performs configurations specified in /etc/profile and ~/.bash_profile.
If you run script in Jenkins pipeline it will look like:
steps {
sh '''#!/bin/bash -l
env
'''
}
Running Fitnesse in Linux using a command line such as:
java -jar fitnesse.jar -c "TestName?test&format=text"
However, the system.out, which shows up in ErrorLogs in a PC environment does not get created.
Is there a switch to turn this on?
Thanks,
John
Here is documentation for the command line options:
http://fitnesse.org/FitNesse.UserGuide.QuickReferenceGuide#FitNesseCommandLINE
It looks like there's a -l option to specify where the logs should be created.
Try specifying -l /tmp/log.out just to make sure there are no permissions issues.