In my scripted Jenkinsfile I have a line that runs parallel deployments . I omitted the other stages and code for security.
When I run this, it can't find the reportUrl and I get the error: groovy.lang.MissingPropertyException: No such property: teamsUrl for class: groovy.lang.Binding.
However, if I run without the parallel deployments it works and I'm able to reach the value of reportUrl. I tested with echo statements. Any thoughts?
Am I not exiting the parallel statement properly?
stage("Deploy") {
def deployments = [:]
// Code here not pasted
parallel deployments
echo "Deployed to clusters"
}
stage('Reporting') {
def reportUrl = 'https://testurl'
echo "${reportUrl}"
sh """
./my-tool report deploy \
--report-url "${reportUrl}" \
--force
"""
}
EDIT 6/22
reportUrl comes from a groovy file: example.groovy
example = load("deploy/example.groovy")
def reportUrl = example.REPORT_URL['report']
//I can see the url being pulled correctly here
echo "${reportUrl}"
Related
In my Jenkinsfile I want to dynamically find the unity version using a python script like so:
environment {
UNITY_EDITOR = bat(script: "py $WORKSPACE/get_versions.py --unity", returnStdout: true).trim()
UNITY_BASE = "C:/Program Files/Unity/Hub/Editor/$UNITY_EDITOR/Editor/Unity.exe"
UNITY_WRAPPER = "UnityBatchWrapper -silent-crashes -no-dialogs -batchmode -quit -unityPath \"$UNITY_BASE\""
}
post {
always {
script {
echo "Returning license"
licenseReturnStatus = bat (
script: "$UNITY_WRAPPER -returnlicense",
returnStatus: true
) == 0
}
}
From other stackoverflow answers this seems like it should work, but instead my Jenkins job errors out during the post-build step because $UNITY_WRAPPER isn't defined:
groovy.lang.MissingPropertyException: No such property: UNITY_WRAPPER for class: groovy.lang.Binding
I'm thinking the batch step is what's failing, even though Jenkins doesn't complain about it. I've also tried using $env.WORKSPACE and %WORKSPACE% and that doesn't work either.
I'm beginning to think $WORKSPACE doesn't exist til after the environments step...
Turns out I didn't have Python installed since it was an ephemeral GCP builder and I hadn't updated the node label yet.
For anyone reading this that has trouble with bat commands - be sure to put an # sign in front of your command like "#py ..." or else the command will be echoed in the output. Also trim your output so it doesn't have CRLF in it.
I'm newbie to Jenkins pipeline and writing a groovy script to parse a json file. However I'm facing an error which many have faced but none of the solutions worked for me. Below is my Jenkinsfile and error msg.
def envname = readJSON file: '${env.WORKSPACE}/manifest.json'
pipeline {
agent any
stages {
stage('Build') {
steps {
echo WORKSPACE
sh "ls -a ${WORKSPACE}"
}
}
}
}
[Pipeline] Start of Pipeline
[Pipeline] readJSON
[Pipeline] End of Pipeline
org.jenkinsci.plugins.workflow.steps.MissingContextVariableException:
Required context class hudson.FilePath is missing Perhaps you forgot
to surround the code with a step that provides this, such as: node at
org.jenkinsci.plugins.pipeline.utility.steps.AbstractFileOrTextStepExecution.run(AbstractFileOrTextStepExecution.java:30)
I even tried readJSON file: '${WORKSPACE}/manifest.json but that didn't work too. I'm sure the mistake is with the first line since when removing that line, there execution is successful. The docs are pretty helpful but I'm not able to track down where exactly I'm going wrong that is why posted here.
UPDATE:
I tried the following methods def envname = readJSON file: "./manifest.json" and def envname = readJSON file: "${env.WORKSPACE}/manifest.json" and even tried them defining under the steps block. Nothing worked. Below is the error msg I recieved when I defined them under step block
WorkflowScript: 5: Expected a step # line 7, column 13
def envname =
^
Below is the official syntax doc of readJson and I can see that I'm using the correct syntax only. but still doesn't work as expected.
https://www.jenkins.io/doc/pipeline/steps/pipeline-utility-steps/#readjson-read-json-from-files-in-the-workspace
'${env.WORKSPACE}/manifest.json' is interpolating the Groovy env map as a shell variable. You need to interpolate it as a Groovy variable like "${env.WORKSPACE}/manifest.json".
sh "ls -a ${WORKSPACE}" is interpolating the shell environment variable WORKSPACE as a Groovy variable. You need to interpolate it as a shell variable like sh 'ls -a ${WORKSPACE}'.
echo WORKSPACE is attempting to resolve the shell variable WORKSPACE as a first class Groovy variable expression. You need to use the Groovy env map instead like echo env.WORKSPACE.
As for the global variable indefinite type assignment on the first line: if it still throws the error above after making those fixes, then it may be due to invalid use of scripted syntax in a declarative syntax pipeline. You likely need to place it inside a step block within your pipeline in that case.
I've solved this myself with the help of "Matt Schuchard"'s below answer. I'm not sure whether this is the only way to solve but this worked for me.
pipeline {
agent any
stages {
stage('Json-Build') {
steps {
script {
def envname = readJSON file: "${env.WORKSPACE}/manifest.json"
element1 = "${envname.dev}"
echo element1
}
}
}
}
}
I'm new to jenkins and working on a declarative pipeline that runs an AWS cloudformation stack. The parameters to be passed to the stack have been declared in a previous stage.
The issue is that the portion of the command after rendering a variable, and beginning with double hyphen '--' does not get picked up by jenkins.
This is the stage i have:
stage('Run Cloudformation') {
steps {
script {
if(env.BRANCH_NAME != 'master') {
sh """
AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID} \
AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY} \
AWS_DEFAULT_REGION=us-west-2 \
aws cloudformation create-stack --stack-name stack-name \
--template-body ${template} \
--parameters ParameterKey=KeyName,ParameterValue=${keyname} ParameterKey=Region,ParameterValue=${region}
"""
}
}
}
}
}
But the error i get on jenkins console output is:
I've tried changing the order of the template and parameter inputs, but the portion of the command following the rendered variable and beginning with a '--' gets skipped every time.
Appreciate any help with this!
I'm trying to dynamically define a variable I use later in a some shell commands of my Jenkins pipeline and it's throwing an exception. I even tried to predefine the variable from an environment section to no avail. Is this a prohibited operation? My other variable myVar seems to work fine, but it's a constant through the pipeline.
pipeline {
agent any
environment {
py2Ana=""
myVar="ABCDE"
}
stages {
stage('Stage1') {
steps {
sh """
echo myVar=$myVar
echo Find Anaconda2 Python installation...
py2Ana=`which -a python | grep --max-count=1 anaconda2`
if [[ -z "$py2Ana" ]]; then
echo ERROR: must have a valid Anaconda 2 distribution installed and on the PATH for this job.
exit 1 # terminate and indicate error
fi
"""
}
}
}
Exception
groovy.lang.MissingPropertyException: No such property: py2Ana for class: groovy.lang.Binding
at groovy.lang.Binding.getVariable(Binding.java:63)
at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onGetProperty(SandboxInterceptor.java:242)
at org.kohsuke.groovy.sandbox.impl.Checker$6.call(Checker.java:288)
at org.kohsuke.groovy.sandbox.impl.Checker.checkedGetProperty(Checker.java:292)
at org.kohsuke.groovy.sandbox.impl.Checker.checkedGetProperty(Checker.java:268)
at org.kohsuke.groovy.sandbox.impl.Checker.checkedGetProperty(Checker.java:268)
at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.getProperty(SandboxInvoker.java:29)
at com.cloudbees.groovy.cps.impl.PropertyAccessBlock.rawGet(PropertyAccessBlock.java:20)
at WorkflowScript.run(WorkflowScript:21)
As #jxramos stated, Jenkins is trying to resolve the variables in the script. It interprets any $string as a variable that needs substitution.
The solution is to escape the $ of the in-script variables, as follows:
pipeline {
agent any
stages {
stage('test stage'){
steps {
sh """#!/bin/bash
myvar=somevalue
echo "The value is \$myvar"
"""
}
}
}
}
There appears to be a variable substitution precedence that Jenkins enforces in a preprocessing step if you will. In other words there's no delayed expansion as one would find in the Windows batch file behavior with setlocal ENABLEDELAYEDEXPANSION. This explains what's going on, and here's the test pipeline I used to determine this:
pipeline {
agent any
environment {
py2Ana="DEFAULT"
}
stages {
stage('Stage1') {
steps {
sh """
echo py2Ana=$py2Ana
py2Ana=Initialized
echo py2Ana Initialized=$py2Ana
"""
}
}
}
}
This yields the following console output...
Started by user unknown or anonymous
Running in Durability level: MAX_SURVIVABILITY
[Pipeline] node
[Pipeline] {
[Pipeline] withEnv
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Stage1)
[Pipeline] sh
[TestPipeline] Running shell script
+ echo py2Ana=DEFAULT
py2Ana=DEFAULT
+ py2Ana=Initialized
+ echo py2Ana Initialized=DEFAULT
py2Ana Initialized=DEFAULT
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS
Another restriction that this poses is that you truly cannot use dynamic variables in the sh portion of the Jenkins declarative pipeline script since Jenkins will first attempt to resolve all variables before execution. Thus the following will always yield an error
sh """
for filename in /tmp/**; do
echo filename=$filename
done
"""
The error being...
groovy.lang.MissingPropertyException: No such property: filename for class: groovy.lang.Binding
One would need to define a script dynamically (after figuring out a way to escape the $ to write to file), or already have it in the source, to be executed.
The error itself seems really to be caused by the assignment of an empty string.
However: Do you really need that environment variable to be defined in the Jenkinsfile?
To me it looks like you just want to set and read the variable from within the shell script. But the way it's coded the if [[ -z "$py2Ana" ]]; then would never pick up the value set by the shell script - it would always want to use a property from the Jenkinsfile - which didn't work.
You could use if [[ -z "${env.py2Ana}" ]]; then for the if condition which would fix that error but it still would not pick up the value set by the previous line but always read the empty string set in the Jenkinsfile.
To solve this you could either enclose the string in single quotes for the whole string like (maybe you even want to get rid of the myVar then)...:
pipeline {
agent any
stages {
stage('Stage1') {
steps {
sh '''
echo Find Anaconda2 Python installation...
py2Ana=`which -a python | grep --max-count=1 anaconda2`
if [[ -z "$py2Ana" ]]; then
echo ERROR: must have a valid Anaconda 2 distribution installed and on the PATH for this job.
exit 1 # terminate and indicate error
fi
'''
}
}
}
}
... or add a backslash right before $py2Ana like:
pipeline {
agent any
stages {
stage('Stage1') {
steps {
sh """
echo Find Anaconda2 Python installation...
py2Ana=`which -a python | grep --max-count=1 anaconda2`
if [[ -z "\$py2Ana" ]]; then
echo ERROR: must have a valid Anaconda 2 distribution installed and on the PATH for this job.
exit 1 # terminate and indicate error
fi
"""
}
}
}
}
Either way without referencing env.py2Ana in the code I doubt the environment block in the Jenkinsfile still would make sense - that's why I removed it from the examples.
Just add a value to py2Ana
environment {
py2Ana="1234"
myVar="ABCDE"
}
It doesn't create the variable in environment if you pass a empty string :)
I want to pass a variable which I read in stage A towards stage B somehow. I see in some examples that people write it to a file, but I guess that is not really a nice solution. I tried writing it to an environment variable, but I'm not really successful on that. How can I set it up properly?
To get it working I tried a lot of things and read that I should use the """ instead of ''' to start a shell and escape those variables to \${foo} for example.
Below is what I have as a pipeline:
#!/usr/bin/env groovy
pipeline {
agent { node { label 'php71' } }
environment {
packageName='my-package'
packageVersion=''
groupId='vznl'
nexus_endpoint='http://nexus.devtools.io'
nexus_username='jenkins'
nexus_password='J3nkins'
}
stages{
// Package dependencies
stage('Install dependencies') {
steps {
sh '''
echo Skip composer installation
#composer install --prefer-dist --optimize-autoloader --no-interaction
'''
}
}
// Unit tests
stage('Unit Tests') {
steps {
sh '''
echo Running PHP code coverage tests...
#composer test
'''
}
}
// Create artifact
stage('Package') {
steps {
echo 'Create package refs'
sh """
mkdir -p ./build/zpk
VERSIONTAG=\$(grep 'version' composer.json)
REGEX='"version": "([0-9]+.[0-9]+.[0-9]+)"'
if [[ \${VERSIONTAG} =~ \${REGEX} ]]
then
env.packageVersion=\${BASH_REMATCH[1]}
/usr/bin/zs-client packZpk --folder=. --destination=./build/zpk --name=${env.packageName}-${env.packageVersion}.zpk --version=${env.packageVersion}
else
echo "No version found!"
exit 1
fi
"""
}
}
// Publish ZPK package to Nexus
stage('Publish packages') {
steps {
echo "Publish ZPK Package"
sh "curl -u ${env.nexus_username}:${env.nexus_password} --upload-file ./build/zpk/${env.packageName}-${env.packageVersion}.zpk ${env.nexus_endpoint}/repository/zpk-packages/${groupId}/${env.packageName}-${env.packageVersion}.zpk"
archive includes: './build/**/*.{zpk,rpm,deb}'
}
}
}
}
As you can see the packageVersion which I read from stage Package needs to be used in stage Publish as well.
Overall tips against the pipeline are of course always welcome as well.
A problem in your code is that you are assigning version of environment variable within the sh step. This step will execute in its own isolated process, inheriting parent process environment variables.
However, the only way of passing data back to the parent is through STDOUT/STDERR or exit code. As you want a string value, it is best to echo version from the sh step and assign it to a variable within the script context.
If you reuse the node, the script context will persist, and variables will be available in the subsequent stage. A working example is below. Note that any try to put this within a parallel block can be of failure, as the version information variable can be written to by multiple processes.
#!/usr/bin/env groovy
pipeline {
environment {
AGENT_INFO = ''
}
agent {
docker {
image 'alpine'
reuseNode true
}
}
stages {
stage('Collect agent info'){
steps {
echo "Current agent info: ${env.AGENT_INFO}"
script {
def agentInfo = sh script:'uname -a', returnStdout: true
println "Agent info within script: ${agentInfo}"
AGENT_INFO = agentInfo.replace("/n", "")
env.AGENT_INFO = AGENT_INFO
}
}
}
stage("Print agent info"){
steps {
script {
echo "Collected agent info: ${AGENT_INFO}"
echo "Environment agent info: ${env.AGENT_INFO}"
}
}
}
}
}
Another option which doesn't involve using script, but is just declarative, is to stash things in a little temporary environment file.
You can then use this stash (like a temporary cache that only lives for the run) if the workload is sprayed out across parallel or distributed nodes as needed.
Something like:
pipeline {
agent any
stages {
stage('first stage') {
steps {
// Write out any environment variables you like to a temporary file
sh 'echo export FOO=baz > myenv'
// Stash away for later use
stash 'myenv'
}
}
stage ("later stage") {
steps {
// Unstash the temporary file and apply it
unstash 'myenv'
// use the unstashed vars
sh 'source myenv && echo $FOO'
}
}
}
}