I try to configure Jenkins' seed job, where whole business is in provided DSL script. I want to seperate that script from its configuration, which I want to locate in additional yml file. When I try to read that file:
#Grab('org.yaml:snakeyaml:1.17')
import org.yaml.snakeyaml.Yaml
def workDir = SEED_JOB.getWorkspace()
def config = new Yaml().load(("${workDir}/config.yml" as File).text)
I receive error
java.io.FileNotFoundException: /var/lib/jenkins/workspace/test.dsl/config.yml (No such file or directory)
I suppose that Jenkins is looking for the file on a master host, not an agent node where workspace is located.
Is it possible to read yml file in DSL build step on the agent node? Or maybe I have to execute that seed job always on my master host?
This seems not possible as the jobDsl script is executed on master. You can try force to run the job on master with label master.
From the documentation in section Script location:
Job DSL scripts are executed on the Jenkins master node, but the seed job's workspace which contains the script files may reside on a build node. This mean that direct access to the file specified by FILE may not be possible from a DSL script. See Distributed builds for details.
Related
I have some windows slave at my Jenkins so I need to copy file to them in pipeline. I heard about Copy To Slave and Copy Artifact plugins, but they doesn't have pipeline syntax manual. So I don't know how to use them in pipeline.
Direct copy doesn't work.
def inputFile = input message: 'Upload file', parameters: [file(name: 'parameters.xml')]
new hudson.FilePath(new File("${ENV:WORKSPACE}\\parameters.xml")).copyFrom(inputFile)
This code returns and error:
Caused: java.io.IOException: Failed to copy /var/lib/jenkins/jobs/_dev/jobs/(TEST)job/builds/107/parameters.xml to d:\Jenkins\workspace\_dev\(TEST)job\parameters.xml
Is there any way to copy file from master to slave in Jenkins Pipeline?
As I understand copyFrom is executed on your Windows node, therefore the source path is not accessible.
I think you want to look into the stash/unstash steps (Jenkins Pipeline: Basic Steps), which work across different nodes. Also this example might be helpful.
Pipeline DSL context runs on master node even that your write node('someAgentName') in your pipeline.
Try to use stash/unstash, but it is bad for large files.
Try External Workspace Manager Plugin. It has
pipelines steps and good for large files.
Try to use an intermediate storage. archive() and sh("wget $url") will be helpful.
If the requirement is to copy an executable to the test slave and to publish the test results, this is easy to do without the Copy to Slave plugin.
A shared folder should be created on each test slave (normal Windows shared folder).
After build: Build script copies the executable to the shared directory on each slave. A simple batch script using copy command is sufficient for this.
stage ('Copy to slaves') {
steps {
bat 'call "copy-to-slave.bat"'
}
}
During test: The test script copies the executable to another directory and runs it.
After test: Post-build action "Publish Robot Framework test results" can be used to report the test results. It is not necessary to copy the test result files back to the master first.
I recommend on Pipeline: Phoenix AutoTest plugin
Jenkins plugin website:
https://plugins.jenkins.io/phoenix-autotest/#documentation
GitHub repository of plugin:
https://github.com/jenkinsci/phoenix-autotest-plugin
I have a jenkins pipeline which load a groovy utility script like this :
Utils = load('/var/lib/jenkins/utils/Utils.groovy')
Everything is fine when I execute the pipeline on the master node. In this case I'm able to use the methods inside my class Utils in the pipeline.
node('master'){
stage('stage1'){
def Utils = load('/var/lib/jenkins/utils/Utils.groovy')
Utils.doSomething()
}
}
My problem came when I try to execute my pipeline in a slave. In this case the load above causes the error
java.io.IOException: java.io.FileNotFoundException: /var/lib/jenkins/utils/Utils.groovy (No such file or directory)
To avoid this error, in the pipeline, I load the file in master node like this
node('master'){
stage('stage1'){
Utils = load('/var/lib/jenkins/utils/Utils.groovy')
}
}
node(){
stage('stage2'){
Utils.doSomething()
}
}
This is not very efficient and I don't want to use the master just for loading the file
Have you any advice on how to load a Groovy scipt on a slave node ?
Thank you
First, the above error java.io.IOException: java.io.FileNotFoundException: /var/lib/jenkins/utils/Utils.groovy (No such file or directory) was caused when you tried to load the file when executing on the slave node. But the file is stored in /var/lib/jenkins/utils/Utils.groovy in your master node, which is another computer and another file system, I guess. So, the mistake is logical.
When you execute some pipeline operations on another node (computer, server etc.), and you want to load the file, you need to have it stored in that computer(slave) and load it from there - so the path have to be according to file place in slave computer.
So, I would suggest to:
simply store Utils.groovy file on slave machine and load it from
there
load it to your workspace on master(as you did already)
Also, you can store groovy file code in github and load it from
there not depending on master/slave filesystems (I would advice this option)
I have some groovy code which I am planning to re-use in Jenkins Groovy Post Build plugin of multiple jobs. How can I achieve this? Is there a place I can store the script in a global variable and call that in the jobs where ever I need?
You can load any groovy file living on the Jenkins master within the groovy postbuild and execute it. For example, you could have a special directory on the c drive where all the common scripts live. I'll update my answer later with some code that shows you how to load the script in.
Update
Assuming you have a test.groovy file on your C: drive, it should be as simple as the following in Groovy Postbuild:
evaluate(new File("C:\\test.groovy"))
Please view the comment section of the Groovy Postbuild for more examples and possibly other ways.
Here is the solution that worked for me:
Installed Scriptler plugin for Jenkins and saved the Groovy script in that. Now the script is available in JENKINS_HOME/scriptler/scripts directory. This way we can avoid manual step of copying files to Jenkins master.
Used the groovy file in Post build:
def env = manager.build.getEnvironment(manager.listener) evaluate(new File(env['JENKINS_HOME'] + "\\scriptler\\scripts\\GroovyForPostBuild.groovy"))
This is a copy of my answer to this similar question on StackOverflow:
If you wish to have the Groovy script in your Code Repository, and loaded onto the Build / Test Slave in the workspace, then you need to be aware that Groovy Postbuild runs on the Master.
For us, the master is a Unix Server, while the Build/Test Slaves are Windows PCs on the local network. As a result, prior to using the script, we must open a channel from the master to the Slave, and use a FilePath to the file.
The following worked for us:
// Get an Instance of the Build object, and from there
// the channel from the Master to the Workspace
build = Thread.currentThread().executable
channel = build.workspace.channel;
// Open a FilePath to the script
fp = new FilePath(channel, build.workspace.toString() + "<relative path to the script in Unix notation>")
// Some have suggested that the "Not NULL" check is redundant
// I've kept it for completeness
if(fp != null)
{
// 'Evaluate' requires a string, so read the file contents to a String
script = fp.readToString();
// Execute the script
evaluate(script);
}
I have a simple Jenkinsfile where I want to load some data from the workspace. I am using the pipeline plugin to leverage the Jenkinsfile inside of the repository. The build is farmed off to a matching Jenkins agent. When I try to use "readFile" I get the following message:
java.io.FileNotFoundException: /path/to/jenkins/workspace/XXXXX/project/data.json (No such file or directory)
I also get the same message when trying to load a Groovy file from the workspace.
My Jenkinsfile looks like:
node('master') {
stage "Start"
echo "Starting"
stage "Load File"
def myJson = readFile "data.json"
}
Any ideas why I can't read these files?
Thanks,
Tim
When Jenkins processes a Jenkinsfile it does not automatically pull down the entire source repository. You need to execute "checkout scm" to pull down the contents of the repository. If you fail to do so no other files will be available to the pipeline script.
I want to run a Jenkins job on 4 different slaves (windows, linux, solaris, Mac). Instead of making 4 different jobs I want to have a single job. I can use a Node parameter to execute on different slaves. My job runs a script which uses Jenkins workspace of slave and a few other scripts. My script is in a different folder on each slave, and other required scripts are in a different folder. So now I have created 4 different jobs for each slave and hard-coded Jenkins workspace and other required scripts path.
Is there any way so that I can put all paths in some JSON-like structure and depending on slave will pick those paths? So that I will have 1 job only.
Please suggest, Thanks in advance!
my idea is to use e.g "Execute system Groovy script" to get slave value and then use if statement to assigne proper path and create parameter visible in Environment Variables:
import hudson.model.Computer
import hudson.model.StringParameterValue
import hudson.model.ParametersAction
//get slave name
def slaveName = Computer.currentComputer().getNode().name
def path
//choose path
if(slaveName.equals("slave01")){
path = "C:"
}
if(slaveName.equals("slave02")){
path = "/root"
}
if(slaveName.equals("slave03")){
path = "D:"
}
//pass path as env. variable
build.addAction(new ParametersAction(new StringParameterValue('path', path)))
then you can use variable path in command:
echo %path%
or use Conditional BuildStep Plugin to set separable steps for each operation system and control when each step should be executed
Jenkins is designed to check out files from a version control system (Subversion, Git, whatever) and run tasks. Instead of trying to manage separate files on separate slaves, you should put your scripts in some form of version control and let Jenkins check out the files in the workspace as part of its build process.