I'm currently attempting to run Groovy code using Jenkins Pipeline, but I'm finding that my scripts run the Groovy part of the code on the Master rather than the Slaves despite me noting the agent.
I have a Repo in Git named JenkinsFiles containing the below JenkinsFile, as well as a repo named CommonLibrary, which contains the code run in the Stages.
Here's my JenkinsFile :
#Library(['CommonLibrary']) _
pipeline {
agent { label 'Slave' }
stages {
stage("Preparation") {
agent { label 'Slave && Windows' }
steps {
Preparation()
}
}
}
}
Here's the Preparation.groovy file :
def call() {
println("Running on : " + InetAddress.localHost.canonicalHostName)
}
Unfortunately, I always seem to get the Master returned when I run the Pipeline in Jenkins. I've tried manually installing Groovy on the Slave, and have also removed the Executors on the Master. Any Powershell that gets run triggers correctly on the Slave, and the $NODE_NAME value returns as the Slave, but it's just the Groovy commands that seem to run on the Master.
Any help would be greatly appreciated. Thanks! - Tor
Related
I have a simple Jenkins pipeline script like this:
pipeline {
agent {
label 'agent2'
}
stages {
stage('test') {
steps {
build job: 'doSomething'
}
}
}
}
When running the job, it starts correctly on the node "agent2", but it runs as 'jenkins' (the OS shell user of the master server where Jenkins is installed) instead of the OS SSH Shell user of the node.
The node has own credentials assigned to it, but they are not used.
When I run the job "doSomething" on its own and set "Restrict where this project can be run" to "node1" everything is fine. The job then is run by the correct user.
This behaviour can be easily recreated:
1. create a new OS user (pw: testrpm)
sudo useradd -m testrpm
sudo passwd testrpm
2. create a new node and use this settings:
3. create a freestyle job (called 'doSomething) with a shell step which does 'whoami'
doSomething Job XML
4. create a pipeline job and paste this code into the pipeline script
pipeline {
agent {
label 'agent2'
}
stages {
stage('test') {
steps {
build job: 'doSomething'
}
}
}
}
test-pipeline Job XML
5. run the pipeline and check the console output of the 'doSomething' job. The output of 'whoami' is not 'testrpm' but 'jenkins'
Can somebody explain this behaviour and tell me where the error is ?
If I change the "Usage" of the built-in node to "Only build jobs with label expressions matching this node", than it works as expected an the "doSomething" job is called with the user testrpm:
I coded a generic pipeline which accepts several parameters in order to deploy releases from a pre-defined GitHub repository to specific nodes. I wanted to host this pipeline on a Jenkinsfile on GitHub, so I configured the job to work with a "Pipeline script from SCM". The fact is - when I try and build the job - the Jenkinsfile gets checked out on every node. Is it possible to checkout and execute the Jenkinsfile only on, say, the master node and run the pipeline as intended?
EDIT: As I stated before, the pipeline works just fine and as intended setting the job to work with a pipeline script. The thing is when I try and change it to be a "Pipeline script from SCM", the Jenkinsfile gets checked out on every agent, which is a problem since I don't have git installed on any agent other than master. I want the Jenkinsfile to be checked out only on master agent and be executed as intended. FYI the pipeline below:
def agents = "$AGENTS".toString()
def agentLabel = "${ println 'Agents: ' + agents; return agents; }"
pipeline {
agent none
stages {
stage('Prep') {
steps {
script {
if (agents == null || agents == "") {
println "Skipping build"
skipBuild = true
}
if (!skipBuild) {
println "Agents set for this build: " + agents
}
}
}
}
stage('Powershell deploy script checkout') {
agent { label 'master' }
when {
expression {
!skipBuild
}
}
steps {
git url: 'https://github.com/owner/repo.git', credentialsId: 'git-credentials', branch: 'main'
stash includes: 'deploy-script.ps1', name: 'deploy-script'
}
}
stage('Deploy') {
agent { label agentLabel }
when {
expression {
!skipBuild
}
}
steps {
unstash 'deploy-script'
script {
println "Execute powershell deploy script on agents set for deploy"
}
}
}
}
}
I think that skipDefaultCheckout is what are you looking for:
pipeline {
options {
skipDefaultCheckout true
}
stages {
stage('Prep') {
steps {
script {
........................
}
}
}
}
}
Take a look to the documentation:
skipDefaultCheckout
Skip checking out code from source control by default in the agent directive.
https://www.jenkins.io/doc/book/pipeline/syntax/
I think you are requesting the impossible.
Now:
your Jenkinsfile is inside your jenkins configuration and is sent as such to each of your agents. No need for git on your agents.
Pipeline script for SCM:
Since you use git, SCM = git. So you are saying: my Pipeline needs to be fetched from a git repository. You are declaring the Deploy step to run on agent { label agentLabel }, so that step is supposed to run on another agent than master.
How would you imagine that agent could get the content of the Jenkinsfile to know what to do, but not use git ?
What happens in Jenkins?
Your master agent gets triggered that it needs to build
the master agent checkouts the Jenkinsfile using git (since it is a Pipeline script from SCM)
jenkins reads the Jenkinsfile and sees what has to be done.
for the Prep stage, I'm not quite sure what happens without agent, I guess that runs on master agent.
the Powershell deploy script checkout is marked to run on master agent, so it runs on master agent (note that the Jenkinsfile will get checked out with git two more times:
before starting the stage, because jenkins needs to know what to execute
one more checkout because you specify git url: 'https://github.com/owner/repo.git'...
the Deploy stage is marked to run on agentLabel, so jenkins tries to checkout your Jenkinsfile on that agent (using git)...
You can use Scripted Pipeline to do this, it should basically look like this
node('master') {
checkout scm
stash includes: 'deploy-script.ps1', name: 'deploy-script'
}
def stepsForParallel = [:]
env.AGENTS.split(' ').each { agent ->
stepsForParallel["deploy ${agent}"] = { ->
node(agent) {
unstash 'deploy-script'
}
}
parallel stepsForParallel
you can find all info about jenkins agent section here.
Shortly: you can call any agent by name or label.
pipeline {
agent {
label 'master'
}
}
If it will not work for you, then you will need to set any label on master node and call it by label
pipeline {
agent {
label 'master_label_here'
}
}
I've got a maven, java project and I'm using git.
I want to use jenkins for build + test + deploy (.war file) on tomcat server (on same device)
My current question is about triggering the build with pushing changes into the git repository master. However it did work with jenkins freestyle project. There I could setup my git repository, so it detected any changes and run the build.
But as far as I could make my research using a "pipeline" should be better to run the process with build + test + deploy. So I created a pipeline and also wrote a jenkinsfile.
pipeline {
agent any
stages {
stage('Compile Stage') {
steps {
withMaven(maven: 'maven_3_5_1'){
bat 'mvn clean compile'
}
}
}
stage('Testing Stage') {
steps {
withMaven(maven: 'maven_3_5_1'){
bat 'mvn test'
}
}
}
stage('Deployment Stage (WAR)') {
steps {
withMaven(maven: 'maven_3_5_1'){
bat 'mvn deploy'
}
}
}
}
}
The current problem is, that inside a pipeline project I could not find an option for setting up the git repository. Currently jenkins does not track any changes in git, when I push a change.
What I've to do, so jenkins runs build when changes are detected in git (like in the freestyle project)?
I thank you very much in advance.
Definition Inside the Repository (Jenkinsfile)
You should place the pipeline definition into a file called Jenkinsfile inside your repository.
This has the great advantage that your pipeline is also versioned. Using the Multibranch Project, you can point Jenkins to your Git repo and it will automatically discover all branches containing such Jenkinsfile (and create a job for each of them). You can find more information in the documentation.
In case you don't want jobs for different branches, you can also configure the job to take the pipeline definition from SCM:
With that specified, you can configure the job to poll SCM changes regularly:
Definition in the Job
In case you really don't want to put your pipeline into the repository (I don't recommend this), then you can use the checkout step to get your code:
pipeline {
agent any
stages {
stage('Compile Stage') {
steps {
checkout('https://git.example.com/repo.git')
withMaven(maven: 'maven_3_5_1') {
bat 'mvn clean compile'
}
}
}
// ...
More options for the checkout (e.g. other branches) can be found in the step documentation.
Finally, change the job to be built in regular intervals:
And now comes the point where I'm struggling (while editing the post): This probably builds the project every time (5min in the example). I am not sure, if currentBuild.changeSets contains the changes that are explicitly checked out with checkout. If it does, then you can check, if it contains changes and in such cases abort the build. All not very nice...
I have two Jenkins workflow jobs that start the same job with different parameters, namely, the branch they build. The latter job builds the project on several platforms. The "head" job, that is the worklflow job may start on different machines. Also, there are two linux machines in the setup.
And sometimes it so happens that one of them (say, master) starts on one of the linux machines, and the other one starts on the other. Both of them have to build a target on a linux machine, and since both of them are busy, both jobs stall.
With usual jobs, one can limit where they can run, however, I couldn't find how to limit where a workflow job can run. Obviously, it should be done using the groovy script, but it escapes me how exactly.
Is there a solution to that?
here's a Jenkinsfile to do it globally (this is telling jenkins the entire pipeline must be run on a slave with these three labels):
pipeline {
agent { label 'docker && git && rbenv' }
stages {
stage('commit_stage') {
steps {
echo 'building stuff'
}
}
}
}
you can also select a certain slave or certain capabilities via the node step for any stage or part of a stage:
pipeline {
agent { label 'docker && git && rbenv' }
stages {
stage('commit_stage') {
steps {
// this overrides the top-level agent requirements
node('linux_with_zsh') {
echo 'building stuff'
}
}
}
}
}
Background
Let's say I have two jobs, one 'Pipeline Job' and one 'Build Job'. The 'Pipeline Job' runs on master and is of course a pipeline (using groovy). Then for a build part in the pipeline I use a slave running on Windows, the 'Build Job', which is responsible for building something I can't do on the master. The master is also running on Windows but lack some software needed for the specific build.
The Question
I have a groovy script that looks something like this:
#!groovy
node {
stage('Environment Preparation') {
// fetches stuff and sets up the environment on master
}
stage('Unit Testing') {
// some testing
}
stage('Build on Slave') {
def slaveJob = build job: 'BuildJob'
}
}
It works fine, where 'BuildJob' is "Restrict where this project can be run", i.e., on the slave.
My issue is that I want the output from 'BuildJob' to print in the pipeline logs. Do you have some clever ways of how this could be done? I'm open for everything, so if you know of more clever ways to start the 'BuildJob' etc. I'm eager to here it.
Thanks!
EDITED
You have to approve the things you want to access under script-approval. Not sure if you really neeed getRawBuild but it worked.
Search through console output of a Jenkins job
#!groovy
node {
stage('Environment Preparation') {
// fetches stuff and sets up the environment on master
}
stage('Unit Testing') {
// some testing
}
stage('Build on Slave') {
def slaveJob = build job: 'BuildJob'
println slaveJob.rawBuild.log
}
}
jenkinsurl/scriptApproval/ you approve the following:
method hudson.model.Run getLog
method org.jenkinsci.plugins.workflow.support.steps.build.RunWrapper getRawBuild
Well, sometimes stating a question makes you think from another perspective.
Hopefully someone will benefit from this.
I stumbled upon a Pipeline-Plugin tutorial where they showed how you could use node, to label where script code should run. The resulting groovy file looks something like this:
#!groovy
stage('Environment Preparation') {
node('master') {
// fetches stuff and sets up the environment on master
}
}
stage('Unit Testing') {
node('master') {
// some testing
}
}
stage('Build on Slave') {
node('remote') {
def out = bat script: 'C:\\Build\\build.bat', returnStdout: true
}
}
As you can see the tutorial made me refactor the script a bit. The node('remote') part is what defines that the upcoming stuff should be run on the slave machine.
I had to make some customizations in the batch script, so that everything important was printed to stdout.
You have to let Jenkins know which node is 'remote' by going in to Manage Jenkins > Manage Nodes, choose the slave agent in mind, Configure Node and add 'remote' or whatever suits you to the labels field.