How can I write out environment variable(s) with writeFile in Jenkins pipelines?
It seems such an easy task but I can't find any documentation on how to get it to work.
I tried $VAR, ${VAR} and ${env.VAR}, nothing works...?
In a declarative pipeline (using a scripted block for writeFile) it will look like this:
pipeline {
agent any
environment {
SENTENCE = 'Hello World\n'
}
stages {
stage('Write') {
steps {
script {
writeFile file: 'script.txt', text: env.SENTENCE
}
}
}
stage('Verify') {
steps {
sh 'cat script.txt'
}
}
}
}
Output:
...
[Pipeline] { (Verify)
[Pipeline] sh
[test] Running shell script
+ cat script.txt
Hello World
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS
If you want to avoid groovy, this will work too:
writeFile file: 'script.txt', text: "${SENTENCE}"
To combine your env var with text you can do:
...
environment {
SENTENCE = 'Hello World'
}
...
writeFile file: 'script.txt', text: env.SENTENCE + ' is my newest sentence!\n'
Related
I'm new to Groovy. I'm not able to figure out what's wrong here.
Depends on the choice of input, I expect the script to execute either Step 'Hello' or 'Bye' but it skips both. I mostly orientated to this Jenkins pipeline conditional stage using "When" for choice parameters, but still can't figure it out.
How can I use those choice parameters correctly?
pipeline {
agent any
stages {
stage('Init') {
steps('Log-in'){
echo 'Log-in'
}
}
stage('Manual Step') {
input {
message "Hello or Goodbye?"
ok "Say!"
parameters{choice(choices:['Hello','Bye'], description: 'Users Choice', name: 'CHOICE')}
}
steps('Input'){
echo "choice: ${CHOICE}"
echo "choice params.: " + params.CHOICE //null
echo "choice env: " + env.CHOICE //Hello
}
}
stage('Hello') {
when{ expression {env.CHOICE == 'Hello'}}
steps('Execute'){
echo 'Say Hello'
}
}
stage('Bye') {
when{ expression {env.CHOICE == 'Bye'}}
steps('Execute'){
echo 'Say Bye'
}
}
}
}
Output:
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Init)
[Pipeline] echo
Log-in
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Manual Step)
[Pipeline] input
Input requested
Approved by Admin
[Pipeline] withEnv
[Pipeline] {
[Pipeline] echo
choice: Hello
[Pipeline] echo
choice params.: null
[Pipeline] echo
choice env: Hello
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Hello)
Stage "Hello" skipped due to when conditional
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Bye)
Stage "Bye" skipped due to when conditional
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS
From the docs:
Any parameters provided as part of the input submission will be available in the environment for the rest of the stage.
This means that your parameter CHOICE does not exist in the other stages. If you want to have a parameter that's available on all the stages, you can define a parameter outside of the stage, i.e.:
pipeline {
agent any
parameters {
choice(choices:['Hello','Bye'], description: 'Users Choice', name: 'CHOICE')
}
stages {
stage('Init') {
steps('Log-in') {
echo 'Log-in'
}
}
stage('Manual Step') {
steps('Input') {
echo "choice: ${CHOICE}"
echo "choice params.: " + params.CHOICE
echo "choice env: " + env.CHOICE
}
}
stage('Hello') {
when {
expression { env.CHOICE == 'Hello' }
}
steps('Execute') {
echo 'Say Hello'
}
}
stage('Bye') {
when {
expression {env.CHOICE == 'Bye'}
}
steps('Execute'){
echo 'Say Bye'
}
}
}
}
This will behave as expected. The difference is that the job won't ask you for input, instead, you will provide the wanted parameters before pressing build.
I've inherited some Jenkins pipeline and try to improve it. Jenkins and groovy is quite fresh topic for me, so most probably I'm doing something wrong.
I'm using Jenkins ver. 2.121.3
Main aim was to add build parameter to do some extra cleaning during build. So I've added parameter CLEAN_FIRST with Boolean type and default value false to a job configuration and did something like this in pipeline:
// CLEAN_FIRST = false
// def prefix = CLEAN_FIRST ? "" : "REM"
pipeline {
agent none
stages {
stage('Some step') {
steps {
script {
node('master') {
cleanWs()
try {
def prefix = CLEAN_FIRST ? "" : "REM"
echo "CLEAN_FIRST=$CLEAN_FIRST prefix=$prefix"
bat (label: 'build third party',
script: """
$prefix call cleanSomthing.bat
call doOtherStuff.bat
"""
} finally {
echo "some stuff"
}
} // node
} // script
} // steps
} // stage
} // stages
} // pipeline
Now this doesn't work as expected. "REM" prefix is not added.
Echo prints:
CLEAN_FIRST=false prefix=
And bat invokes cleanSomthing.bat which I wish to avoid (to save on build times).
I've tried to make prefix global, but with same result.
Most probably this is caused by some evaluation order or scoping issue, but I can't put finger on it.
Can someone give me a clue why it doesn't work? How to fix it?
Answered own question. Is this problem fixed on some version of Jenkins?
replace
def prefix = CLEAN_FIRST ? "" : "REM"
with
def prefix = params.CLEAN_FIRST ? "" : "REM"
Ok I've found source of problems. It is a bit funny.
When running this pipeline (tested on Mac machine since it had empty job queue):
pipeline {
agent none
stages {
stage('Some step') {
steps {
script {
node('Mac') {
cleanWs()
try {
def logic = true
def prefix = CLEAN_FIRST ? "Ole" : "REM"
def typeLogic = logic.getClass()
def typeParam = CLEAN_FIRST.getClass()
echo "typeLogic=$typeLogic typeParam=$typeParam"
echo "CLEAN_FIRST=$CLEAN_FIRST prefix=$prefix"
sh (script: """
echo prefix=$prefix
""")
} finally {
echo "some stuff"
}
} // node
} // script
} // steps
} // stage
} // stages
} // pipeline
I've got this outcome:
Running in Durability level: MAX_SURVIVABILITY
[Pipeline] stage
[Pipeline] { (Some step)
[Pipeline] script
[Pipeline] {
[Pipeline] node
Running on master in /Users/builder/jenkins/workspace/EIbuild_MacOS
[Pipeline] {
[Pipeline] cleanWs
[WS-CLEANUP] Deleting project workspace...[WS-CLEANUP] done
[Pipeline] echo
typeLogic=class java.lang.Boolean typeParam=class java.lang.String
[Pipeline] echo
CLEAN_FIRST=false prefix=Ole
[Pipeline] sh
[EIbuild_MacOS] Running shell script
+ echo prefix=Ole
prefix=Ole
[Pipeline] echo
some stuff
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] // script
[Pipeline] }
[Pipeline] // stage
[Pipeline] End of Pipeline
Finished: SUCCESS
So now source the problem is obvious.
Jenkins in configuration promises variable of type Boolean, but in fact provides type String with values are "true" or "false" which are always evaluated as true when used as condition since both values are not empty strings :).
I have a yaml file that contains reference to some variable that define in pipeline.
I need some way to evaluate this $ to real value after I read the yaml.
yaml file
chart_folder: test_chart_${my_suffix}
lint:
enable: false
pipeline look like below
pipeline{
agent{
label "my_node"
}
stages{
stage("test"){
steps{
script {
def my_suffix = "test"
def my_yaml = readYaml file: "my_file.yaml"
echo my_yaml.chart_folder
}
}
}
}
}
The output of execution is
....
[Pipeline] readYaml
[Pipeline] echo
test_chart_${my_suffix}
[Pipeline] }
....
and I want to get the chart_folder as evaluated string
....
[Pipeline] readYaml
[Pipeline] echo
test_chart_test
[Pipeline] }
....
How can I do it?
You can't interpolate, but you can override.
def my_yaml = readYaml file: 'my_file.yaml', text: "chart_folder: 'test_chart_test'"
pipeline {
agent any
stages {
stage('Build') {
steps {
echo 'Building..'
echo "whoami".execute().text
script {
File f = new File('/home/jenkins/test2.txt');
f.createNewFile();
}
}
}
stage('Test') {
steps {
echo 'Testing..'
}
}
stage('Deploy') {
steps {
echo 'Deploying....'
}
}
}
}
Jenkins console log: (got exception: Started by user Edgar Yu Running
in Durability level: MAX_SURVIVABILITY [Pipeline] node Running on
Jenkins in /var/jenkins_home/workspace/test2 [Pipeline] { [Pipeline]
stage [Pipeline] { (Build) [Pipeline] echo Building.. [Pipeline] echo
jenkins
[Pipeline] script [Pipeline] { [Pipeline] } [Pipeline] // script
[Pipeline] } [Pipeline] // stage [Pipeline] stage [Pipeline] { (Test)
Stage 'Test' skipped due to earlier failure(s) [Pipeline] } [Pipeline]
// stage [Pipeline] stage [Pipeline] { (Deploy) Stage 'Deploy' skipped
due to earlier failure(s) [Pipeline] } [Pipeline] // stage [Pipeline]
} [Pipeline] // node [Pipeline] End of Pipeline
java.io.IOException: Permission denied at java.io.UnixFileSystem.createFileExclusively(Native Method) at
java.io.File.createNewFile(File.java:1012)
This is due to Jenkins not implementing Groovy itself but an interpreter (CPS) - https://github.com/cloudbees/groovy-cps
To help deal with the complexities introduced, there are some common Steps implemented to take the trouble out of tasks such as creating a file.
To use Jenkins pipeline steps out of the box, use writeFile:
https://jenkins.io/doc/pipeline/steps/workflow-basic-steps/#code-writefile-code-write-file-to-workspace
writeFile([file: 'file.txt', text: filetxt])
If your deadset on writing your own, I suggest splitting it out into a Shared library, note this will probably cause ScriptSecurity alerts that will require approval:
final class PipelineUtils implements Serializable {
private script=null
private static final PipelineUtils instance = new PipelineUtils()
#NonCPS
String saveFile(String filename, String text) {
String PWD = script.pwd()
String filePath = "${PWD}/${filename}"
File file = new File(filePath)
file.text = text
}
}
See https://github.com/jenkinsci/pipeline-plugin/blob/master/TUTORIAL.md for information regarding #NonCPS and nonserializable objects.
Despite following this answer and others, I am unable to successfully use a local groovy file in my Jenkinsfile (both are in the same repository).
def deployer = null
...
...
...
pipeline {
agent {
label 'cf_slave'
}
options {
skipDefaultCheckout()
disableConcurrentBuilds()
}
stages {
stage ("Checkout SCM") {
steps {
checkout scm
}
}
...
...
...
stage ("Publish CF app") {
steps {
script {
STAGE_NAME = "Publish CF app"
deployer = fileLoader.load ('deployer')
withCredentials(...) {
if (BRANCH_NAME == "develop") {
...
...
...
} else {
deployer.generateManifest()
}
}
}
}
}
...
...
}
deployer.groovy:
#!/usr/bin/env groovy
def generateManifest() {
sh "..."
echo "..."
}
In the console log (stack):
[Pipeline] stage
[Pipeline] { (Publish CF app)
[Pipeline] script
[Pipeline] {
[Pipeline] echo
before loading groovy file
[Pipeline] echo
Loading from deployer.groovy
[Pipeline] load
[Pipeline] // load
[Pipeline] }
[Pipeline] // script
[Pipeline] }
[Pipeline] // stage
Update:
It seems the problem was not with loading the file but rather with the contents of the file, where I execute the following which apparently does not play well:
sh "node $(pwd)/config/mustacher manifest.template.yml config/environments/common.json config/environments/someFile.json"
echo "..."
When only the echo is there, this is the stack.
So not the sh "node ..." nor the echo work. Even changing it just to sh "pwd" fails as well. What could it be? the syntax in the file? the way it is called in the pipeline?
If I will make the same node call in the pipeline (for example in the withCredentials if statement, it works.
Add a return this to the bottom of the deployer.groovy file, and then change you load step to use relative path and extension to groovy file like load('deployer.groovy').
The return this is documented on jenkins.io:
Takes a filename in the workspace and runs it as Groovy source text.
The loaded file can contain statements at top level or just load and run a closure. For example:
def pipeline
node('slave') {
pipeline = load 'pipeline.groovy'
pipeline.functionA()
}
pipeline.functionB()
pipeline.groovy
def pipelineMethod() {
...code
}
return this
Where pipeline.groovy defines functionA and functionB functions (among others) before ending with return this