I'm bumping up against JENKINS-38706. And since its been open for a while, i'm trying to work around it.
My problem is that i'm running a multinode pipeline, with one of the nodes being a windows slave, with the 255 character path limitations.
So, i'm trying to change the workspace for my windows slave stages, and instead of using C:\jenkins\workspace\job-branch-randomcharacters that the multibranch pipeline uses, i'm trying to move it to c:\w\job\branch.
It immediately fails with:
Branch indexing
Obtained Jenkinsfile from 5bc168fcd5b3707048ad4bca4b5ef7478d759531
Running in Durability level: MAX_SURVIVABILITY
[BFA] Scanning build for known causes...
[BFA] No failure causes found
[BFA] Done. 0s
[Bitbucket] Notifying commit build result
[Bitbucket] Build result notified
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
WorkflowScript: 52: Too many arguments for map key "ws" # line 52, column 15.
ws('C:\\w\\$JOB_NAME\\$BRANCH_NAME') {
My Jenkinsfile snippet:
stage ('Snapshot-WINDOWS') {
agent {
node {
label 'win'
ws('C:\\w\\$JOB_NAME\\$BRANCH_NAME') {
body()
}
}
}
steps {
withMaven(
maven: 'Maven 3.5.3',
mavenSettingsConfig: 'settings'
) {
bat 'mvn clean install'
}
}
}
To answer my own question, instead of using ws() i needed to use customWorkspace, and the $BRANCH_NAME gets automatically added with multibranch pipelines.
stage ('Snapshot-WINDOWS') {
agent {
node {
label 'win'
customWorkspace 'C:\\w\\$JOB_NAME'
}
}
steps {
withMaven(
maven: 'Maven 3.5.3',
mavenSettingsConfig: 'settings'
) {
bat 'mvn clean install'
}
}
}
You forgot to declare the workspace.
Example:
def wsDir = "/some/path/${env.BRANCH_NAME}"
ws (wsDir) {
// some block
}
Related
I created a simple pipline in Jenkins. The remote root directory of my agent is set to my project root path. But when I test, where I am during the build (e.g. by defining a step like sh 'pwd'), I see, that the directory, my steps are executed from is the $WORKSPACE directory (/path_to_remote_root_directory_of_the_agent/workspace/jenkins_project_title). That means, I cannot just start neither my unit tests like sh 'vendor/bin/phpunit ./test/Unit', nor other tasks, that I usually run from the project root folder.
I'm pretty sure, that I simply configured something incorrectly and that in the normal case scripts like this
pipeline {
agent {
label 'devvm-slave-01'
}
stages {
stage('Prepare') {
steps {
sh 'composer install'
...
}
}
...
stage('Checkstyle') {
steps {
sh 'vendor/bin/phpcs --report=checkstyle --report-file=`pwd`/build/logs/checkstyle.xml --standard=PSR2 --extensions=php --ignore=autoload.php --ignore=vendor/ . || exit 0'
checkstyle pattern: 'build/logs/checkstyle.xml'
}
}
}
}
work as expected without any crude workarounds for paths.
What am I doing wrong and how to get it working correctly?
From the section "agent" of the "Jenkins Handbook"'s chapter "Pipeline Syntax":
Parameters
node
agent { node { label 'labelName' } } behaves the same as agent { label 'labelName' }, but node allows for additional options (such as customWorkspace).
So, the solution is the using of the node and its customWorkspace option:
pipeline {
agent {
node {
label 'devvm-slave-01'
customWorkspace '/path/to/my/project'
}
}
...
}
I have a Master (Unix) and a slave Machine (Windows).
I have created a Multibranch pipeline Project on Master and Trigger request all of the Process takes place in Slave. I am trying to send the HTML reports which are being generated at the Slave machine but get Exception:
ERROR: Error: No workspace found!
Sending email to: abhishek.gaur1#pb.com
[Pipeline] }
[Pipeline] // stage
[Pipeline] End of Pipeline
Finished: SUCCESS
I am using the below code in Jenkinsfile:
success {
emailext attachmentsPattern: '**/overview-features.html',
body: '${SCRIPT, template="groovy-html.template"}',
mimeType: 'text/html',
subject: 'Success Pipeline: ${currentBuild.fullDisplayName}',
to: 'abhishek.gaur1#pb.com'
}
The file should be attached to the email and sent. Currently it shows ERROR:
Error: No workspace found!
From my tests it seems the agent none case has a problem in configurations where the workspace is not allocated on the master.
agent none allows to set agents per stage, but the post() block doesn't allow to set an agent, it will run on master without workspace on the case of agent none from what i gathered.
So the only solution for declarative pipeline in that case would be to run the whole build on agent with label Developer30, if your example is complete it should be no problem.
pipeline {
agent {
label 'Developer30'
}
tools {
maven 'MAVEN_HOME'
}
stages {
stage ('Compile Stage') {
steps {
bat 'mvn clean'
}
}
}
post {
success {
// emailext stuff
}
}
}
I am getting started with Jenkins Pipeline. My pipeline has one simple step that is supposed to run on a different agent - like the "Restrict where this project can be run" option.
My problem is that it is running on master.
They are both Windows machines.
Here's my Jenkinsfile:
pipeline {
agent {label 'myLabel'}
stages {
stage('Stage 1') {
steps {
echo pwd()
writeFile(file: 'test.txt', text: 'Hello, World!')
}
}
}
}
pwd() prints C:\Jenkins\workspace\<pipeline-name>_<branch-name>-Q762JIVOIJUFQ7LFSVKZOY5LVEW5D3TLHZX3UDJU5FWYJSNVGV4Q.
This folder is on master. This is confirmed by the presence of the test.txt file.
I expected test.txt to be created on the slave agent.
Note 1
I can confirm that the pipeline finds the agent because the logs contain:
[Pipeline] node
Running on MyAgent in C:\Jenkins\workspace\<pipeline-name>_<branch-name>-Q762JIVOIJUFQ7LFSVKZOY5LVEW5D3TLHZX3UDJU5FWYJSNVGV4Q
But this folder does not exist on MyAgent, which seems related to the problem.
Note 2
This question is similar to Jenkins pipeline not honoring agent specification
, except that I'm not using the build instruction so I don't think the answer applies.
Note 3
pipeline {
agent any
stages {
stage('Stage 1') {
steps {
echo "${env.NODE_NAME}"
}
}
stage('Stage 2') {
agent {label 'MyLabel'}
steps {
echo "${env.NODE_NAME}"
}
}
}
}
This prints the expected output - master and MyAgent. If this is correct, then why is the workspace located in a different folder on master instead of being on MyAgent?
here is an example
pipeline {
agent none
stages {
stage('Example Build') {
agent { label 'build-label' }
steps {
sh 'env'
sh ' sleep 8'
}
}
stage('Example Test') {
agent { label 'deploy-label' }
steps {
sh 'env'
sh ' sleep 5'
}
}
}
}
I faced similar issue and the following pipeline code worked for me (i.e. the file got created on the Windows slave instead of Windows master),
pipeline {
agent none
stages {
stage("Stage 1") {
steps {
node('myLabel'){
script {
writeFile(file: 'test.txt', text: 'Hello World!', encoding: 'UTF-8')
}
// This should print the file content on slave (Hello World!)
bat "type test.txt"
}
}
}
}
}
I'm debugging a completely unrelated issue and this fact was thrown in my face. Apparently the pipeline is processed in the built-in node (previously known as the master node), with the steps being forwarded to the agent.
So even though echo runs on the agent, but pwd() will run on the built-in node. You can do sh 'pwd' to get the path on the agent.
I'm going to use Jenkins pipeline plugin to test several binaries A B C on several nodes 1 2 3.
In the end of my test I would like to have every single result of all possible combinations. So my Pipe may not abort when a single stage fails. It should proceed.
eg: A1 green, A2 green, A3 red, B1 green, B2 red, ..., C3 green
But when the first binary returns with an value unequal zero ("Binary not working on the system") it's stage is marked as FAILURE and any other stages are skipped.
Is there a possibility in Jenkins Pipeline to mark a stage as "UNSTABLE" but proceed with running the other tests?
According to Continue Jenkins job after failed stage while marking stage as failed can't mark this step as failed. The solution of this in running tasks in parallel is not working for my setup. So is it possible to safely mark it as something else? Is it possible to manipulate the result of a stage?
This question How to continue past a failing stage in Jenkins declarative pipeline syntax intents to use a scripted pipeline. I would like to avoid that if it is possible to do it in an other way.
pipeline {
agent {label 'master'}
stages {
stage('A1') {
agent {label 'Node1'}
steps {
sh 'binA'
}
}
stage('A2') {
agent {label 'Node1'}
steps {
sh 'binB' // If this bin fails, all following stages are skipped
}
}
// ...
stage('C3'){
agent {label 'Node3'}
steps {
sh 'binC'
}
}
}
}
Declarative Pipeline: Though using currentBuild.result = 'UNSTABLE' works in declarative pipelines too, Blue Ocean displays all stages as unstable irrespective of which stage fails.
To mark only specific stages as unstable, use the step unstable(message: String) as described here within your stage and install/update the following plugins:
Pipeline: Basic Steps to 2.16 or newer
Pipeline: API Plugin to 2.34 or newer
Pipeline: Groovy to 2.70 or newer
Pipeline Graph Analysis to 1.10 or newer
Sample pipeline stage:
stage('Sign Code') {
steps {
script {
try {
pwd()
sh "<YOUR SCRIPT HERE>"
}
catch (err) {
unstable(message: "${STAGE_NAME} is unstable")
}
}
}
}
Note: This also marks the overall build status as unstable.
There is now a more elegant solution, that not only allows you to set a stage and the job result to unstable. Using catchError, you can set any combination of stage and build result:
pipeline {
agent any
stages {
stage('1') {
steps {
sh 'exit 0'
}
}
stage('2') {
steps {
catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE') {
sh "exit 1"
}
}
}
stage('3') {
steps {
sh 'exit 0'
}
}
}
}
In the example above, all stages will execute, the pipeline will be successful, but stage 2 will show as failed:
As mentioned above, you can freely choose the buildResult and stageResult. You can even fail the build and continue the execution of the pipeline.
Just make sure your Jenkins is up to date, since this is a fairly new feature. (Pipeline: Basic Steps needs to be 2.18 or newer)
For scripted pipeline, you can use try .. catch blocks inside the stages and then set currentBuild.result = 'UNSTABLE'
in the exception handler.
I want to use the mentioned plugin with a declarative pipeline, to be precise I want to convert the following documentation example to a declarative pipeline:
The pipeline code in the upstream job is the following:
stage ('Stage 1. Allocate workspace in the upstream job')
def extWorkspace = exwsAllocate 'diskpool1'
node ('linux') {
exws (extWorkspace) {
stage('Stage 2. Build in the upstream job')
git url: 'https://github.com/alexsomai/dummy-hello-world.git'
def mvnHome = tool 'M3'
sh '${mvnHome}/bin/mvn clean install -DskipTests'
}
}
And the downstream's Pipeline code is:
stage ('Stage 3. Select the upstream run')
def run = selectRun 'upstream'
stage ('Stage 4. Allocate workspace in the downstream job')
def extWorkspace = exwsAllocate selectedRun: run
node ('test') {
exws (extWorkspace) {
stage('Stage 5. Run tests in the downstream job')
def mvnHome = tool 'M3'
sh '${mvnHome}/bin/mvn test'
}
}
Thanks!
I searched everywhere for a clear answer to this, yet never found a definitive answer. So, I pulled the External Workspace Plugin code and read it. The answer is simple as long as the plugins Model doesn't change.
Anytunc's answer is very close, but the issue is getting the path from the External Workspace Plugin and getting it into the customWorkspace configuration.
What I ended up doing was creating a method:
def getExternalWorkspace() {
extWorkspace = exwsAllocate diskPoolId: "jenkins"
return extWorkspace.getCompleteWorkspacePath()
}
and setting my agent to:
agent {
node {
label 'Linux'
customWorkspace getExternalWorkspace()
}
}
If you'd rather not set the entire pipeline to that path, you could create as many external workspaces as you want, then use
...
steps {
dir(getExternalWorkspace()) {
do fancy stuff
...
}
}
...
You can use this agent directive:
agent {
node {
label 'my-defined-label'
customWorkspace '/some/other/path'
}
}