I have a legacy project in Jenkins that hast to be pipelined (for
later parallelization), hence moving from simple tcsh script to
pipeline
running the script as
#!/bin/tcsh
source ./mysetting.sh
update
works but the same pipeline step fails due to missing alias expansion
stage ('update') {
steps {
//should be working but alias expansion fails
sh 'tcsh -c "source ./mysettings.sh; alias; update"'
//manually expanding the alias works fine
sh 'tcsh -c "source ./mysettings.sh; alias; python update.py;"'
}
}
calling alias in the steps properly lists all the set aliases, so I
can see them, but not use them.
I know in bash alias expansion has to be set
#enable shell option for alias_expansion
shopt -s expand_aliases
but in csh/tcsh that should be taken care of by source.
what am I missing?
found the solution:
sh '#!/bin/tcsh \n' +
'source ./mysettings.sh \n' +
'echo "Calling my alias" \n' +
'my_alias \n'
every line starting with sh launches a new shell, so it has to be in one line including line breaks.
further adding to the confusing was that documentation of jenkins says that it starts "a bash" but it launched /bin/sh which in my case pointed to something else
Related
I have a simple jenkins pipeline script that I have some sh code running. The important bit looks like this
script{
sh "cp folderA/file.txt newFolder/"
sh "cp -r folderB/* newFolder/"
sh "cp folderC/\\* newFolder/"
}
When I look at the console output of the jenkins pipeline I see the following
[Pipeline] sh
+ cp folderA/file.txt newFolder/
[Pipeline] sh
+ cp -r folderB/folder1 folderB/folder2 folderB/folder3 newFolder/
[Pipeline] sh
+ cp 'folderC/*' newFolder/
cp: cannot stat 'folderC/*': No such file or director
I've tried various combinations of quotes and escape characters, but jenkins/groovy will either expand the filenames (as in the second example), or add quotes around the 'folderA/*' (as in the third example) which confuses sh because folderC/ exists but 'folderC/' does not and the copy fails. Any idea how to do this properly?
I tried using fileCopyOperation already but ran into issues because I couldn't manipulate the file paths properly. ie: I couldn't figure out how to move many files/folders like folderA/folderB/file.txt to newFolder/file.txt without it becoming newFolder/folderA/folderB/file.txt
I am stuck in trying to get a Jenkinsfile to work. It keeps failing on sh step and gives the following error
process apparently never started in /home/jenkins/workspace
...
(running Jenkins temporarily with -Dorg.jenkinsci.plugins.durabletask.BourneShellScript.LAUNCH_DIAGNOSTICS=true might make the problem clearer)
I have tried adding
withEnv(['PATH+EXTRA=/usr/sbin:/usr/bin:/sbin:/bin'])
before sh step in groovy file
also tried to add
/bin/sh
in Manage Jenkins -> Configure System in the shell section
I have also tried replacing the sh line in Jenkinsfile with the following:
sh "docker ps;"
sh "echo 'hello';"
sh ./build.sh;"
sh ```
#!/bin/sh
echo hello
```
This is the part of Jenkinsfile which i am stuck on
node {
stage('Build') {
echo 'this works'
sh 'echo "this does not work"'
}
}
expected output is "this does not work" but it just hangs and returns the error above.
what am I missing?
It turns out that the default workingDir value for default jnlp k8s slave nodes is now set to /home/jenkins/agent and I was using the old value /home/jenkins
here is the config that worked for me
containerTemplate(name: 'jnlp', image: 'lachlanevenson/jnlp-slave:3.10-1-alpine', args: '${computer.jnlpmac} ${computer.name}', workingDir: '/home/jenkins/agent')
It is possible to get the same trouble with the malformed PATH environment variable. This prevents the sh() method of the Pipeline plugin to call the shell executable. You can reproduce it on a simple pipeline like this:
node('myNode') {
stage('Test') {
withEnv(['PATH=/something_invalid']) {
/* it hangs and fails later with "process apparently never started" */
sh('echo Hello!')
}
}
}
There is variety of ways to mangle PATH. For example you use withEnv(getEnv()) { sh(...) } where getEnv() is your own method which evaluates the list of environment variables depending on the OS and other conditions. If you make a mistake in the getEnv() method and PATH gets overwritten you get it reproduced.
I have a Jenkinsfile which uses three tick marks surrounding a command to execute as in:
sh ''' command '''
We have no idea why three tick marks are required or what role they perform.
This syntax is seen in the Jenkinsfile doc set.
This has nothing at all to do with bash (in which triple-quotes have no special meaning at all), and everything to do with Groovy (the separate, non-bash interpreter that parses Jenkinsfiles).
In Groovy, but not in bash, strings must use triple-quotes to span multiple lines.
In the context of a sh directive in a Jenkinsfile, the content of your triple-quoted string is passed to a shell as a script to execute; however, the syntax is parsed by Groovy, so it's only Groovy that cares about the quotes themselves (as opposed to the quoted content).
Can you give more idea about what kind of command is it, is it a unix command or some script ?
The single quote and its variation like '''(3 ticks) as mentioned in question skip the variable expansion, and it could used to show what is being executed.
echo '''Updating JAVA_HOME variable :
export $JAVA_HOME="$NEW_JAVA_HOME" '''
However in your question, a command (some string) is enclosed between 3 ticks marks and sh tries to execute this command or script. One such example below
$ echo "echo hello" > /tmp/tesh.sh
$ sh '''/tmp/test.sh'''
hello
I am using Jenkins to take a number of parameters, generate an ansible-playbook command and run it. My Jenkins server is also my Ansible server.
My shell says ::
echo $ESXi_IP
echo $VM_NAME
echo $NIC1_MAC
echo $NIC2_MAC
echo $NIC3_MAC
echo $NIC4_MAC
echo $ESXi_HOSTNAME
echo $PLAYBOOK
ansible-playbook $PLAYBOOK --extra-vars "esxi_ip=$ESXi_IP vm_name=$VM_NAME nic1_mac=$NIC1_MAC nic2_mac=$NIC2_MAC nic3_mac=$NIC3_MAC nic4_mac=$NIC4_MAC esxi_hostname=$ESXi_HOSTNAME"
When I run the Job, the output is ::
+ ansible-playbook /root/ansible/sc-ece.yaml --extra-vars 'esxi_ip=5.232.66.49 vm_name=JenkinsTest nic1_mac=00:50:C0:A8:01:02 nic2_mac=00:50:0A:A9:37:A5 nic3_mac=00:50:0A:FF:FE:4C nic4_mac=00:50:AC:10:01:65 esxi_hostname=tmolab13-14iamesxi4'
ERROR! the playbook: /root/ansible/sc-ece.yaml could not be found
The playbook path is correct. there is no issue in it at all.
What seems to be missing here ?
You are correct Matt & Dave. Permissions to the folder was an issue. Thanks !
I'm following guideline how to sign Android apk with Jenkins. I have parametrized Jenkins job with KSTOREPWD and KEYPWD. A part of Jenkins' job configuration (Build->Execute shell) is to take those parameters and store them as environment variables:
export KSTOREPWD=${KSTOREPWD}
export KEYPWD=${KEYPWD}
...
./gradlew assembleRelease
The problem is when the build is over anybody can access the build "Console Output" and see what passwords were entered; part of that output:
08:06:57 + export KSTOREPWD=secretStorePwd
08:06:57 + KSTOREPWD=secretStorePwd
08:06:57 + export KEYPWD=secretPwd
08:06:57 + KEYPWD=secretPwd
So I'd like to suppress echo before output from export commands and re-enable echo after export commands.
By default, Jenkins launches Execute Shell script with set -x. This causes all commands to be echoed
You can type set +x before any command to temporary override that behavior. Of course you will need set -x to start showing them again.
You can override this behaviour for the whole script by putting the following at the top of the build step:
#!/bin/bash +x
Here is an example of how to write the sh parameter in Jenkinsfile with no output in a more secure way, as suggested in official documentation. The set +x does the main magic as has been written in this answer.
The single-quotes will
cause the secret to be expanded by the shell as an environment
variable. The double-quotes are potentially less secure as the secret
is interpolated by Groovy, and so typical operating system process
listings (as well as Blue Ocean, and the pipeline steps tree in the
classic UI) will accidentally disclose it:
Insecure, wrong usage:
node {
withCredentials([string(credentialsId: 'mytoken', variable: 'TOKEN')]) {
sh /* WRONG! */ """
set +x
curl -H 'Token: $TOKEN' https://some.api/
"""
}
}
Correct usage ✅:
node {
withCredentials([string(credentialsId: 'mytoken', variable: 'TOKEN')]) {
sh '''
set +x
curl -H 'Token: $TOKEN' https://some.api/
'''
}
}
In your specific situation (using gradle and jenkins) you could also use a Password Parameter, using Gradle's pattern for environment variables (ORG_GRADLE_PROJECT_prop). Gradle will then set a propproperty on your project.
In your case this would look something like this
And you can use it in your gradle.properties like this
signingConfigs {
release {
storeFile file(KEYSTORE)
storePassword KSTOREPWD
keyAlias ALIAS
keyPassword KEYPWD
}
}
BTW - I recommend using the credentials binding plugin for KEYSTORE