How to add to slave's PATH using Slave SetupPlugin? - jenkins

I have 2 RHEL machines setup in a Master/Slave configuration using Jenkins ver. 1.609.2
The slave is being launched via SSH Slaves Plugin 1.10.
I'm trying to use the Slave Setup Plugin v 1.9 to install the tools that will be necessary for my slave machine to run builds. In particular I am installing sqlplus.
Here is the script that I am running in order to try installing sqlplus:
if command -v sqlplus >/dev/null; then
echo "sqlplus already setup. Nothing to do."
else
#Create directory for sqlplus and unzip it there.
mkdir /jenkins/tools/sqlplus
tar -xvf sqlplussetup/instantclient-basiclite-linux.x64-12.1.0.2.0.tar.gz -C /jenkins/tools/sqlplus || { echo 'unzip failed' ; exit 1; }
tar -xvf sqlplussetup/instantclient-sqlplus-linux.x64-12.1.0.2.0.tar.gz -C /jenkins/tools/sqlplus || { echo 'unzip failed' ; exit 1; }
cd /jenkins/tools/sqlplus/instantclient_12_1
#Create links for the Oracle libs
ln -s libclntsh.so.12.1 libclntsh.so || { echo 'Could not create link' ; exit 1; }
ln -s libocci.so.12.1 libocci.so || { echo 'Could not create link' ; exit 1; }
#Add two lines to .bashrc only if they don't already exist. Export LD_LIBRARY_PATH and add sqlplus to PATH.
grep -q -F 'export LD_LIBRARY_PATH=/jenkins/tools/sqlplus/instantclient_12_1:$LD_LIBRARY_PATH' /home/jenkins/.bashrc || echo 'export LD_LIBRARY_PATH=/jenkins/tools/sqlplus/instantclient_12_1:$LD_LIBRARY_PATH' >> /home/jenkins/.bashrc
grep -q -F 'export PATH=$PATH:/jenkins/tools/sqlplus/instantclient_12_1' /home/jenkins/.bashrc || echo 'export PATH=$PATH:/jenkins/tools/sqlplus/instantclient_12_1' >> /home/jenkins/.bashrc
#Export variables so they can be used right away
export LD_LIBRARY_PATH=/jenkins/tools/sqlplus/instantclient_12_1:$LD_LIBRARY_PATH
export PATH=$PATH:/jenkins/tools/sqlplus/instantclient_12_1
echo "sqlplus has been setup."
fi
This script runs successfully and everything appears to work until I try to run a build and execute the sqlplus command. The build fails because sqlplus is not a recognized command.
My main question is this:
What is the proper way to automatically add an environment variable when launching a slave?
Please note I am looking for an automated way of doing this. I don't want to go into the configuration screen for my slave, tick a checkbox and specify an environment variable. That is counter-productive to what I am trying to achieve which is a slave that is immediately usable for builds once connected.
I pretty much understand why my script doesn't work. When Jenkins is launching the slave it first makes an SSH connection and then it runs my setup script using the command
/bin/sh -xe /jenkins/tmp/hudson8035138410767957141.sh
Where the contents of hudson8035138410767957141.sh is my script from above. So obviously, the export isn't going to work. I was hoping adding the exports to the .bashrc file would get around this but it does not work. I think this is because this script is executed after the ssh connection is established and therefore the .bashrc has already been read.
Problem is I can't figure out any way to work around this limitation.

Bash does not read any of its startup files (.bashrc, .profile etc) for non-interative shells that don't have the --login option set explicitly -- that's why the exports don't work.
So, solution "A" is to keep the bashrc magic that you suggest above, and to add the --login option by changing the first line in your build step to
#!/bin/bash --login
<your script here>
The explicit shebang at on the first line will also prevent excessive debug output that you get from the default's -x option (see your console snippet above).
Alternative solution "B" uses the fact that bash will source any script whose name is given in $BASH_ENV (if that variable is defined and the file exists). Define that variable globally in your slave properties (e.g., set to /jenkins/tools/setup.sh) and add exports as needed during slave setup. Every bash shell build step will read the settings then.
With solution "B" you don't need to use the --login option and you don't have to mess up the .bashrc. However, the "BASH_ENV" feature is only active when bash runs in "bash mode". As Jenkins starts the shell via sh, bash tries to emulate historic sh, which does not have that feature. So, also for B, you need a shebang:
#!/bin/bash
<your script here>
But that you'd need anyway to get rid of the tracing output that's usually too much in production setups.

Related

Pmrep command not found - jenkins

I am fairly new to Informatica. I am trying to automate deployment of Powercenter code from one environment to another using jenkins.
Script:
node('')
{
def application = 'powercenter'
stage('deploy'){
sshagent(['group']) {
sh """ssh -o StrictHostKeyChecking=no user#123.com 'cd /opt/hub/infapwc/server/bin && pmrep connect -r Repository_Service_L1 -d domain -n username -x password'"""
}
}
}
My job is failing with error: pmrep command not found. Informatica is installed on the linux server i am doing ssh in. This works fine in putty. I am not sure what the issue is. Can anyone please help?
You can use $INFA_HOME/bin to $PATH or you can use absolute path of pmrep file.
pmrep file is available in $INFA_HOME/bin. You can check with infa admin person about path.
That won't work; pmrep uses a couple of libraries which are located in the .../server/bin directory as well.
In order to make this work, please add the .../server/bin directory of the PowerCenter installation path (resp. ...\server\bin on Windows) to the PATH environment variable of the user ID which runs the Jenkins script before trying to invoke pmrep.

Extend $PATH variable in git bash under Windows

I'm trying to extend my $PATH variable in git bash (MinGW shell) by adding the following to the file ~/.bashrc
PATH=$PATH':/c/Program Files/maven/apache-maven-3.2.5/bin'
After I did this and restarted the bash it seems like that the $PATH variable was extended like expected:
$ echo $PATH
MANY_OTHER_PATHS:/c/Program Files/maven/apache-maven-3.2.5/bin
But I still cannot execute the programms in the given directory:
$ mvn
bash: mvn: command not found
What went wrong here? How do I extend the PATH variable correctly?
Here are two ideas.
You can have your path with double quote mark.
export PATH=$PATH:"/C/Program Files (x86)/apache-maven-3.3.3/bin"
Or, You can also make symbolic link for the directory.
ln -s "/C/Program Files (x86)/apache-maven-3.3.3/bin" ./mvnbin
export PATH=$PATH:/your-path/mvnbin
It works for me in mingw32 environment.
I needed to add something to my Git Bash path permanently each time I open it. It was Meld.exe path which can be added with:
export PATH=$PATH:"/C/Program Files (x86)/Meld/lib"
In order to execute this command each bash session, you need a ~/.bashrc file. Check if it already exists or create it using notepad ~/.bashrc or touch ~/.bashrc.
You can check where it is with:
echo ~
Open it and add the command that adds the PATH (first command in this response).
I hope you found this useful.
According to this SO post, you need to escape Program Files with quotes. git-bash $PATH cannot parse windows directory with space
Add PATH in Git Bash Permanently | Windows Only
Just in case you are still wondering how to add a path permanently in git bash here is the step-by-step process for Windows users:
Create .bashrc in user's root folder using the below command. It will open notepad and ask you to create the file, click yes.
notepad ~/.bashrc
Put the directory you want to add as below, for more than 1 items repeat the same format in next line:
export PATH=$PATH:"/c/folder/folder/"
Save the file and relaunch the bash.
Next launch will give you a warning like WARNING: Found ~/.bashrc but no ~/.bash_profile, ~/.bash_login or ~/.profile. but git bash will handle it by creating the required files.
SOME INSIGHTS
Git Bash doesn't fetch Window's environment PATH, it maintains its PATH separately in more like a Linux way.
You can run export PATH=$PATH:"/c/folder/folder/" in cmd to add a directory to path, but it will be only for the current session once you close the bash, it will be gone.
.bashrc is a shell script file that will be executed every time you launch a new git bash window. So you can add any type of bash command here. We simply added the export command to add our desired directory to PATH.

How can I create and use a variable inside docker?

I'm trying to get the version from a pom file so that I can use it later in another command. So inside the docker file I have,
RUN VERSION="$(mvn org.apache.maven.plugins:maven-help-plugin:2.1.1:evaluate -Dexpression=project.version|grep -Ev '(^\[|Download\w+:)')"
RUN echo $VERSION
But the echo prints nothing. What I actually want to run is,
RUN mv /abc/abc-${VERSION}.jar /abc/abc.jar
Use ENV like that, it's the way Docker devs want you to do it:
ENV VERSION "$(mvn org.apache.maven.plugins:maven-help-plugin:2.1.1:evaluate -Dexpression=project.version|grep -Ev '(^\[|Download\w+:)')"
RUN echo $VERSION
There is an example is the docs that's very similar to your case: https://docs.docker.com/engine/reference/builder/#environment-replacement
if you group your 2 lines in one, it will work
RUN VERSION="$(mvn org.apache.maven.plugins:maven-help-plugin:2.1.1:evaluate -Dexpression=project.version|grep -Ev '(^\[|Download\w+:)')" ; echo $VERSION
the recommended way is of course the solution proposed by Ilya Peterov, I suspect you want to use it later in your Dockerfile
I will take another example, if you have a script in abc/myscript.sh
RUN cd abc ; ./myscript
will work, while
RUN cd abc
RUN ./myscript
will fail as it will not be in the abc directory

How to update JDK on Jenkins Server cartridge (OpenShift)?

The Jenkins Server cartridge (OpenShift) uses OpenJDK 7u55.
How to update to OpenJDK 7u60 or 8u05 or Oracle JDK (7u60 or 8u05), please?
You can do this using OpenShift's action hooks. Add a script which will check for the existence of the JDK you want to use, and download it if it doesn't exist.
For example, in .openshift/action_hooks/deploy, add this snippet:
#! /bin/bash
JDK_HOME=$OPENSHIFT_DATA_DIR/jdk1.8.0
if [[ ! -L $JDK_HOME && ! -d $JDK_HOME ]]
then
cd $OPENSHIFT_DATA_DIR
wget http://www.java.net/download/jdk8u20/archive/b17/binaries/jdk-8u20-ea-bin-b17-linux-x64-04_jun_2014.tar.gz
tar xvf *.tar.gz
rm -f *.tar.gz
ln -s jdk1.8.0_20 jdk1.8.0
fi
In Jenkins, you can then configure builds to use this JDK by configuring the PATH variable, in an "Execute Shell" action, like so:
export PATH=$OPENSHIFT_DATA_DIR/jdk1.8.0/bin:$PATH
This example retrieves 8u20. Sorry, I'm not sure of the links to use for the exact versions you mention. Also, word of warning, this download is over HTTP, without performing a check against the published MD5 checksums. If you're doing anything serious, you should edit to code snippet to perform that check.

Jenkins / Hudson environment variables

I am running Jenkins from user jenkins thats has $PATH set to something and when I go into Jenkins web interface, in the System Properties window (http://$host/systemInfo) I see a different $PATH.
I have installed Jenkins on Centos with the native rpm from Jenkins website. I am using the startup script provided with the installation using sudo /etc/init.d/jenkins start
Can anyone please explain to me why that happens?
Michael,
Two things:
When Jenkins connects to a computer, it goes to the sh shell, and not the bash shell (at least this is what I have noticed - I may be wrong). So any changes you make to $PATH in your bashrc file are not considered.
Also, any changes you make to $PATH in your local shell (one that you personally ssh into) will not show up in Jenkins.
To change the path that Jenkins uses, you have two options (AFAIK):
1) Edit your /etc/profile file and add the paths that you want there
2) Go to the configuration page of your slave, and add environment variable PATH, with value: $PATH:/followed-by/paths/you/want/to/add
If you use the second option, your System Information will still not show it, but your builds will see the added paths.
I kept running into this problem, but now I just add:
source /etc/profile
As the first step in my build process. Now all my subsequent rules are loaded for Jenkins to operate smoothly.
You can also edit the /etc/sysconfig/jenkins file to make any changes to the environment variables, etc. I simply added source /etc/profile to the end of the file. /etc/profile has all all of the proper PATH variables setup. When you do this, make sure you restart Jenkins
/etc/init.d/jenkins restart
We are running ZendServer CE which installs pear, phing, etc in a different path so this was helpful. Also, we don't get the LD_LIBRARY_PATH errors we used to get with Oracle client and Jenkins.
I tried /etc/profile, ~/.profile and ~/.bash_profile and none of those worked. I found that editing ~/.bashrc for the jenkins slave account did.
The information on this answer is out of date. You need to go to Configure Jenkins > And you can then click to add an Environment Variable key-value pair from there.
eg: export MYVAR=test would be MYVAR is the key, and test is the value.
I found two plugins for that.
One loads the values from a file and the other lets you configure the values in the job configuration screen.
Envfile Plugin — This plugin enables you to set environment variables via a file. The file's format must be the standard Java property file format.
EnvInject Plugin — This plugin makes it possible to add environment variables and execute a setup script in order to set up an environment for the Job.
On my newer EC2 instance, simply adding the new value to the Jenkins user's .profile's PATH and then restarting tomcat worked for me.
On an older instance where the config is different, using #2 from Sagar's answer was the only thing that worked (i.e. .profile, .bash* didn't work).
Couldn't you just add it as an environment variable in Jenkins settings:
Manage Jenkins -> Global properties > Environment variables:
And then click "Add" to add a property PATH and its value to what you need.
This is how I solved this annoying issue:
I changed the PATH variable as #sagar suggested in his 2nd option, but still I got different PATH value than I expected.
Eventually I found out that it was the EnvInject plugin that replaced my PATH variable!
So I could either uninstall EnvInject or just use it to inject the PATH variable.
As many of our Jenkins jobs use that plugin, I didn't want to uninstall it...
So I created a file: environment_variables.properties under my Jenkins home directory.
This file contained the path environment value that I needed:
PATH=$PATH:/usr/local/git/bin/.
From the Jenkins web interface: Manage Jenkins -> Configure System.
In that screen - I ticked the Prepare jobs environment option, and in the Properties File Path field I entered the path to my file: /var/lib/jenkins/environment_variables.properties.
This way every Jenkins job we have receive whatever variables I put in this environment_variables.properties file.
Jenkins also supports the format PATH+<name> to prepend to any variable, not only PATH:
Global Environment variables or node Environment variables:
This is also supported in the pipeline step withEnv:
node {
withEnv(['PATH+JAVA=/path/to/java/bin']) {
...
}
}
Just take note, it prepends to the variable. If it must be appended you need to do what the other answers show.
See the pipeline steps document here.
You may also use the syntax PATH+WHATEVER=/something to prepend /something to $PATH
Or the java docs on EnvVars here.
I only had progress on this issue after a "/etc/init.d/jenkins force-reload". I recommend trying that before anything else, and using that rather than restart.
On my Ubuntu 13.04, I tried quite a few tweaks before succeeding with this:
Edit /etc/init/jenkins.conf
Locate the spot where "exec start-stop-server..." begins
Insert the environment update just before that, i.e.
export PATH=$PATH:/some/new/path/bin
Add
/usr/bin/bash
at
Jenkins -> Manage Jenkins -> configure System -> Shell->Shell
executable
Jenkins use the sh so that even /etc/profile doesn't work for me
When I add this, I have all the env.
Solution that worked for me
source ~/.bashrc
Explanation
I first verified Jenkins was running BASH, with echo $SHELL and echo $BASH (note I'm explicitly putting #!/bin/bash atop the textarea in Jenkins, I'm not sure if that's a requirement to get BASH). sourceing /etc/profile as others suggested was not working.
Looking at /etc/profile I found
if [ "$PS1" ]; then
...
and inspecting "$PS1" found it null. I tried spoofing $PS1 to no avail like so
export PS1=1
bash -c 'echo $PATH'
however this did not produce the desired result (add the rest of the $PATH I expect to see). But if I tell bash to be interactive
export PS1=1
bash -ci 'echo $PATH'
the $PATH was altered as I expected.
I was trying to figure out how to properly spoof an interactive shell to get /etc/bash.bashrc to load, however it turns out all I needed was down in ~/.bashrc, so simply sourceing it solved the problem.
I tried all the things from above - didn't work for me.
I found two solution (both for SSH-Slave)
Go to the slave settings
Add a new environment variable
PATH
${PATH}:${HOME}/.pub-cache/bin:${HOME}/.local/bin
The "${HOME}" part is important. This makes the additional PATH absolute.
Relative path did not work for me.
Option II (pipeline-script)
pipeline {
agent {
label 'your-slave'
}
environment {
PATH = "/home/jenkins/.pub-cache/bin:$PATH"
}
stages {
stage('Test') {
steps {
ansiColor('xterm') {
echo "PATH is: $PATH"
}
}
}
}
}
On Ubuntu I just edit /etc/default/jenkins and add source /etc/profile at the end and it works to me.
Running the command with environment variable set is also effective. Of course, you have to do it for each command you run, but you probably have a job script, so you probably only have one command per build. My job script is a python script that uses the environment to decide which python to use, so I still needed to put /usr/local/bin/python2.7 in its path:
PATH=/usr/local/bin <my-command>
What worked for me was overriding the PATH environment for the slave.
Set: PATH
To: $PATH:/usr/local/bin
Then disconnecting and reconnecting the slave.
Despite what the system information was showing it worked.
I have Jenkins 1.639 installed on SLES 11 SP3 via zypper (the package manager).
Installation configured jenkins as a service
# service jenkins
Usage: /etc/init.d/jenkins {start|stop|status|try-restart|restart|force-reload|reload|probe}
Although /etc/init.d/jenkins sources /etc/sysconfig/jenkins, any env variables set there are not inherited by the jenkins process because it is started in a separate login shell with a new environment like this:
startproc -n 0 -s -e -l /var/log/jenkins.rc -p /var/run/jenkins.pid -t 1 /bin/su -l -s /bin/bash -c '/usr/java/default/bin/java -Djava.awt.headless=true -DJENKINS_HOME=/var/lib/jenkins -jar /usr/lib/jenkins/jenkins.war --javaHome=/usr/java/default --logfile=/var/log/jenkins/jenkins.log --webroot=/var/cache/jenkins/war --httpPort=8080 --ajp13Port=8009 --debug=9 --handlerCountMax=100 --handlerCountMaxIdle=20 &' jenkins
The way I managed to set env vars for the jenkins process is via .bashrc in its home directory - /var/lib/jenkins. I had to create /var/lib/jenkins/.bashrc as it did not exist before.
1- add to your profil file".bash_profile" file
it is in "/home/your_user/" folder
vi .bash_profile
add:
export JENKINS_HOME=/apps/data/jenkins
export PATH=$PATH:$JENKINS_HOME
==> it's the e jenkins workspace
2- If you use jetty :
go to jenkins.xml file
and add :
<Arg>/apps/data/jenkins</Arg>
Here is what i did on ubuntu 18.04 LTS with Jenkins 2.176.2
I created .bash_aliases file and added there path, proxy variables and so on.
In beginning of .bashrc there was this defined.
# If not running interactively, don't do anything
case $- in
*i*) ;;
*) return;;
esac
So it's checking that if we are start non-interactive shell then we don't do nothing here.
bottom of the .bashrc there was include for .bash_aliases
# Alias definitions.
# You may want to put all your additions into a separate file like
# ~/.bash_aliases, instead of adding them here directly.
# See /usr/share/doc/bash-doc/examples in the bash-doc package.
if [ -f ~/.bash_aliases ]; then
. ~/.bash_aliases
fi
so i moved .bash_aliases loading first at .bashrc just above non-interactive check.
This didn't work first but then i disconnected slave and re-connected it so it's loading variables again. You don't need to restart whole jenkins if you are modifying slave variables. just disconnect and re-connect.
If your pipeline is executed on the remote node that is connected via SSH, then actually Jenkins runs agent application that performs incoming actions.
By default zsh shell is used, not the bash (my Jenkins has version 2.346.3).
Furthermore jenkins-agent runs non-login shell which makes default PATH values even if you put some configuration to .zshrc. It will be skipped.
My choice is to put the following shebang at a script start
#!/bin/bash -l
-l option makes bash to run in the login mode and in this case bash performs configurations specified in /etc/profile and ~/.bash_profile.
If you run script in Jenkins pipeline it will look like:
steps {
sh '''#!/bin/bash -l
env
'''
}

Resources