Jenkins declarative pipeline with Docker/Dockerfile agent from SCM - docker

With Jenkins using the Declarative Pipeline Syntax how do i get the Dockerfile (Dockerfile.ci in this example) from the SCM (Git) since the agent block is executed before all the stages?
pipeline {
agent {
dockerfile {
filename 'Dockerfile.ci'
}
}
stage ('Checkout') {
steps {
git(
url: 'https://www.github.com/...',
credentialsId: 'CREDENTIALS',
branch: "develop"
)
}
}
[...]
}
In all the examples i've seen, the Dockerfile seems to be already present in the workspace.

You could try to declare agent for each stage separately, for checkout stage you could use some default agent and docker agent for others.
pipeline {
agent none
stage ('Checkout') {
agent any
steps {
git(
url: 'https://www.github.com/...',
credentialsId: 'CREDENTIALS',
branch: "develop"
)
}
}
stage ('Build') {
agent {
dockerfile {
filename 'Dockerfile.ci'
}
steps {
[...]
}
}
}
[...]
}

If you're using a multi-branch pipeline it automatically checks out your SCM before evaluating the agent. So in that case you can specify the agent from a file in the SCM.

The answer is in the Jenkins documentation on the Dockerfile parameter:
In order to use this option, the Jenkinsfile must be loaded from
either a Multibranch Pipeline or a Pipeline from SCM.
Just scroll down to the Dockerfile section, and it's documented there.
The obvious problem with this approach is that it impairs pipeline development. Now instead of testing code in a pipeline field on the server, it must be committed to the source repository for each testable change. NOTE also that the Jenkinsfile checkout cannot be sparse or lightweight as that will only pick up the script -- and not any accompanying Dockerfile to be built.
I can think of a couple ways to work around this.
Develop against agents in nodes with the reuseNode true directive. Then when code is stable, the separate agent blocks can be combined together at the top of the Jenkinsfile which must then be loaded from the SCM.
Develop using the dir() solution that specs the exact workspace directory, or alternately use one of the other examples in this solution.

Related

jenkinsfile executed on master agent

I am starting to work with jenkinsfiles. The jenkinsfile contains an echo message (i.e. Hello world)
This is my case:
I have jenkins (ver 2.190.1) installed on a pc with s.o. windows (master agent).
My slave agent is a pc with s.o. linux.
I put my jenkinsfile in my scm.
I have successfully configured the pipeline to run the jenkinsfile. Which is done successfully.
Jenkins makes repository checkout on *master agent* and not on *slave agent* (what I want) and *option "Lightweight checkout"* is checked.
I want this behaviour because my pipeline must to work on slave agent. And I don't want my repoository on master agent.
I searched on the net for a possible solution but without results.
Could you give me a suggestion on how to checkout my repository directly on the slave agent?
It can be done this way. From the below example just replace agentLabelName to your agent name.
Scripted Pipeline
node('agentLabelName') {
stage('stageName') {
echo "${env.WORKSPACE}"
//checkout scm // If Jenkinsfile availabe with your SCM
git url: 'https://github.com/samitkumarpatel/test0.git', branch: 'main'
}
}
Declarative Pipeline
pipeline {
agent {
label 'agentLabelName'
}
stages {
stage('stageName') {
steps {
echo "Hello World"
echo "${env.WORKSPACE}"
//checkout scm // If Jenkinsfile availabe with your SCM
git url: 'https://github.com/samitkumarpatel/test0.git', branch: 'main'
}
}
}
}

Checkout and run SCM pipeline only on master node

I coded a generic pipeline which accepts several parameters in order to deploy releases from a pre-defined GitHub repository to specific nodes. I wanted to host this pipeline on a Jenkinsfile on GitHub, so I configured the job to work with a "Pipeline script from SCM". The fact is - when I try and build the job - the Jenkinsfile gets checked out on every node. Is it possible to checkout and execute the Jenkinsfile only on, say, the master node and run the pipeline as intended?
EDIT: As I stated before, the pipeline works just fine and as intended setting the job to work with a pipeline script. The thing is when I try and change it to be a "Pipeline script from SCM", the Jenkinsfile gets checked out on every agent, which is a problem since I don't have git installed on any agent other than master. I want the Jenkinsfile to be checked out only on master agent and be executed as intended. FYI the pipeline below:
def agents = "$AGENTS".toString()
def agentLabel = "${ println 'Agents: ' + agents; return agents; }"
pipeline {
agent none
stages {
stage('Prep') {
steps {
script {
if (agents == null || agents == "") {
println "Skipping build"
skipBuild = true
}
if (!skipBuild) {
println "Agents set for this build: " + agents
}
}
}
}
stage('Powershell deploy script checkout') {
agent { label 'master' }
when {
expression {
!skipBuild
}
}
steps {
git url: 'https://github.com/owner/repo.git', credentialsId: 'git-credentials', branch: 'main'
stash includes: 'deploy-script.ps1', name: 'deploy-script'
}
}
stage('Deploy') {
agent { label agentLabel }
when {
expression {
!skipBuild
}
}
steps {
unstash 'deploy-script'
script {
println "Execute powershell deploy script on agents set for deploy"
}
}
}
}
}
I think that skipDefaultCheckout is what are you looking for:
pipeline {
options {
skipDefaultCheckout true
}
stages {
stage('Prep') {
steps {
script {
........................
}
}
}
}
}
Take a look to the documentation:
skipDefaultCheckout
Skip checking out code from source control by default in the agent directive.
https://www.jenkins.io/doc/book/pipeline/syntax/
I think you are requesting the impossible.
Now:
your Jenkinsfile is inside your jenkins configuration and is sent as such to each of your agents. No need for git on your agents.
Pipeline script for SCM:
Since you use git, SCM = git. So you are saying: my Pipeline needs to be fetched from a git repository. You are declaring the Deploy step to run on agent { label agentLabel }, so that step is supposed to run on another agent than master.
How would you imagine that agent could get the content of the Jenkinsfile to know what to do, but not use git ?
What happens in Jenkins?
Your master agent gets triggered that it needs to build
the master agent checkouts the Jenkinsfile using git (since it is a Pipeline script from SCM)
jenkins reads the Jenkinsfile and sees what has to be done.
for the Prep stage, I'm not quite sure what happens without agent, I guess that runs on master agent.
the Powershell deploy script checkout is marked to run on master agent, so it runs on master agent (note that the Jenkinsfile will get checked out with git two more times:
before starting the stage, because jenkins needs to know what to execute
one more checkout because you specify git url: 'https://github.com/owner/repo.git'...
the Deploy stage is marked to run on agentLabel, so jenkins tries to checkout your Jenkinsfile on that agent (using git)...
You can use Scripted Pipeline to do this, it should basically look like this
node('master') {
checkout scm
stash includes: 'deploy-script.ps1', name: 'deploy-script'
}
def stepsForParallel = [:]
env.AGENTS.split(' ').each { agent ->
stepsForParallel["deploy ${agent}"] = { ->
node(agent) {
unstash 'deploy-script'
}
}
parallel stepsForParallel
you can find all info about jenkins agent section here.
Shortly: you can call any agent by name or label.
pipeline {
agent {
label 'master'
}
}
If it will not work for you, then you will need to set any label on master node and call it by label
pipeline {
agent {
label 'master_label_here'
}
}

Is it Possible to Run Jenkinsfile from Jenkinsfile

Currently we are developing centralized control system for our CI/CD projects. There are many projects with many branches so we are using multibranch pipeline ( This forces us to use Jenkinsfile from project branches so we can't provide custom Jenkinsfile like Pipeline projects ). We want to control everything under 1 git repo where for every project there should be kubernetes YAMLS's, Dockerfile and Jenkinsfile. When developer presses build button, Jenkinsfile from their project repo suppose to run our jenkinsfile. Is it possible to do this?
E.g. :
pipeline {
agent any
stages {
stage('Retrieve Jenkinsfile From Repo') { // RETRIEVE JENKINSFILE FROM REPO
steps {
git branch: "master",
credentialsId: 'gitlab_credentials',
url: "jenkinsfile_repo"
scripts {
// RUN JENKINSFILE FROM THE REPO
}
}
}
}
}
Main reason we are doing this, there are sensetive context in jenkinsfile like production database connections. We don't want to store jenkinsfile under developers' repo. Also you can suggest correct way to achieve that beside using only 1 repo.
EDIT: https://plugins.jenkins.io/remote-file/
This plugin solved all my problems. I could'not try comments below
As an option you can use pipeline build step.
pipeline {
agent any
stages {
stage ('build another job') {
steps {
build 'second_job_name_here'
}
}
}
}
Try load step
scripts {
// rename Jenkinsfile to .groovy
sh 'mv Jenkinsfile Jenkins.groovy'
// RUN JENKINSFILE FROM THE REPO
load 'Jenkinsfile.groovy'
}

How to setup the trigger build when pushing to github repo in Jenkins?

I am using Jenkins 2.89.2 and my project has Jenkinsfile which defines all build pipeline and steps. I have searched online for triggering build when push to github repo and all of them mentioned an options Build when a change was pushed to Github in jenkins configuration page. But in my jenkins configuration page I couldn't find this options. Below is an screenshot. It only has one options Periodically if not otherwise run. I have installed github related plugins but still I couldn't find that option. Is there any other configuration I can change?
Below is my Jenkinsfile:
pipeline {
agent {
label 'master'
}
tools {
maven 'maven-3.5.2'
jdk 'jdk9'
}
stages {
stage ('Checkout SCM') {
steps {
echo 'Checkout from Git...'
checkout scm
}
}
stage ('Build') {
steps {
echo 'Building '
}
}
}
}
After some searching I think I find the solution. The trigger option can be defined in jenkinsfile as below:
pipelineTriggers([
[$class: "GitHubPushTrigger"]
])
You basically have to create a HOOK from jenkins to gitHub, which is located under the Build Triggers option
This option should be available to you since you have the Git Plugin installed.
P.S. the solution you have mentioned will work only for Declarative Pipeline and will not work if you plan to switch to scripted pipeline

Github project setup in Jenkins 2.x

I have a Multibranch Pipeline project which configures Jenkins Jobs based on a Jenkinsfile per branch. The sourcecode is hosted on a Github Enterprise Server.
When I view the configuration of a branch which is created by the Jenkinsfile, I noticed that there is a option GitHub project. This option allows to define the URL of the corresponding GitHub project.
I want to define this property via my Jenkinsfile in Pipeline syntax, but I don't know command to use and how.
Relevant parts of my Jenkinsfile:
pipeline {
agent {
docker {
image 'plinzen/android:latest'
label 'android'
}
}
triggers {
githubPush()
}
stages {
stage('build') {
steps {
checkout scm
sh './gradlew clean assembleDebug'
}
}
}
}
How can I define the GitHub project properties via my Jenkinsfile? I use the Jenkins GitHub Plugin in my project.
You can add a new agent node and add this code snippet to do your things. For more info you can refer to this url also. For Additional Info. Hope this helps.
git(
url: 'git#github.com<repo_name>.git',
credentialsId: 'xpc',
branch: '${branch}'
)

Resources