I'm very new to jenkins. I have a monorepo where I need to check what needs to be build. Created a pipeline element in Jenkins, named it Test and copy pasted a script to do a check what files changed:
def changedFiles = currentBuild.changeSets
.collect { it.getItems() }
.flatten() //Ensures that we look through each commit, not just the first.
.collect { it.getAffectedPaths() }
.flatten()
.toSet() //Ensures uniqueness.
echo("Changed files: ${changedFiles}")
def changedDirectories = changedFiles.collect {
if (it.contains('folder/in/monorepo/project/foo')) { //Code cut for simplicity
return "Foo"
}
return ''
}
.unique()
echo("Changed directories: ${changedDirectories}")
if (changedDirectories.contains("Foo")) {
stage('Foo Build') {
node {
env.NODEJS_HOME = "${tool 'NodeJS'}"
env.PATH="${env.NODEJS_HOME}/bin:${env.PATH}"
dir("folder/in/monorepo/buildscripts/") {
sh 'mybuildscript.sh'
}
}
}
} else {
echo("No Foo Build")
}
The code for loading NodeJS is documentend in the NodeJS plugin.
I installed Jenkins, tried some things and then installed NodeJS plugin and updates for other plugins.
Now, with the updated plugins there is some trouble which basically is related to this: Why does Jenkins creates a subfolder within the workspace#script folder to checkout git code instead of the workspace#script itself?
Instead of cloning into the workspace folder, the git repo gets cloned to folder with a unique id.
This leads to the fact, that my folder like in the command dir("folder/in/monorepo/buildscripts/") is not found or to be more precise: It's created and empty and not the one which is in the git repo because it is executed relative to my empty workspace instead relative to the cloned repo.
My questions are:
Is there a much easier way to achieve what I try? AFAIK I need to implement the search for the folder as in the referenced so question.
Doesn't this security fix from the referenced so question breaks all builds / jenkinsfiles? I mean: Eventually in a pipeline someone wants to call a build script, a build tool or a simple make with the makefile in the repo.
EDIT: I just realised that the only reason why the repo is cloned, is because I use the option "Pipeline script from SCM" with "Lightweight checkout" unchecked. I assume that the intended way would be to use a "Lightweight checkout" to just get the Jenkinsfile and then clone the repo as part of the script. A "Pipeline script" written in the gui, would also never get a clone of the repository.
Related
I am writing a Jenkinsfile to do a number of checks on a repo. The check runs everytime a PR has been created. The goal of my check is to make sure that in my PR, 2 particularly files (a.txt) and (b.txt) have been changed.
The issue is that these 2 files always must be changed when a PR is issued and I dont want my developers to forget about updating these files. So I want to write a check that fails the build if these 2 files have not been updated. I tried something like this
stage('Check if file changed'){
when {
allOf {
changeset "a.txt"
changeset "b.txt"
}
}
steps {
// Do something
}
}
But the issue with this is that it wants the change to happen in the commit. A developer could create a PR with one file updated and then add another commit to update the other file but this check will always want both files updated in the latest commit. Can someone suggest a way for me to accomplish what I need ? Also in the above case, it doest fail the build if a file is not updated. It just skips the stage. I want the build to fail. Any suggestions ?
Locally running gives me the following output
git diff --name-status master
M CHANGELOG.md
M Jenkinsfile
I want to use this in my jenkins build but there I get the error
+ git diff --name-status master
fatal: ambiguous argument 'master': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
script returned exit code 128
The pattern you have used in allOf{} inside when{} DSL is wrong. It would be something like this
pipeline {
agent any;
stages {
stage('build') {
when {
allOf{
changeset "README.md"; changeset "README-one.md"
}
}
steps {
echo "from build"
}
}
}
}
note : both changeset has to be seperate with ; not , or space. I can see In Jenkins documentation they have mention , - might be thats a mistake
I have a jenkins pipeline job which bootstraps itself. The idea here is to self test changes, including changes to the jenkins pipeline groovy scripts, which are stored in git.
I was wondering how to bootstrap in a separate job than the build job. So for example, if I have a job called "build_linux", currently Jenkins will create a "workspace/build_linux" folder, and everything inside there will both bootstrap and build within there.
The issue here is that my build_repository.git is very large, and I don't want to check it out twice at the bootstrap stage and the build stage. However, if I don't check them both out in both stages, git (In actuality I'm repo syncing a manifest with both git repositories from the Google 'repo' scripts) complain that there's code in there which shouldn't be, and either fails or will try to clean it up (which is inefficient).
Ideally I'd just have both "workspace/bootstrap/" and "workspace/linux_build/" job folders.
For example, my pipeline looks like this:
bootstrap.gvy:
--> **entry point from my jenkins config**
stage('Bootstrap') {
node("$BOOTSTRAP_NODE") {
git checkout bootstrap_repository.git
git checkout build_repository.git // I don't want to do this here
pipeline = load "build/build.gvy"
}
pipeline.main(env.JOB_NAME) // which in this case is "build_linux"
}
build/build.gvy:
int main(String jobName) {
switch (jobname) {
case "build_linux":
doBuildLiunx()
break
}
}
def doBuildLinux() {
stage("Build") {
node("$BUILD_NODE") {
git checkout bootstrap_repository.git // I don't want to do this here
git checkout build_repository.git
}
}
}
Any suggestions for how to accomplish this? Perhaps I should use a different workspace for bootstrapping?
Thanks
currently I have an "all inclusive" jenkinsfile which contains various functions.
In order to re-use those functions in other jenkinsfiles I want to put them into separate groovy scripts and load them from the jenkinsfile(s).
scmHandler.groovy:
#!groovy
def handleCheckout() {
if (env.gitlabMergeRequestId) {
echo 'Merge request detected. Merging...'
}
...
}
return this;
in jenkinsfile I do:
...
def scmHandler = load ("test/scmHandler.groovy")
scmHandler.handleCheckout()
I tried to follow the instructions from here but jenkins is constantly complaining that there is no such file scmHandler.groovy an I get:
java.io.FileNotFoundException: d:\jenkins\workspace\myJenkinsJob\test\scmHandler.groovy
Both jenkinsfile and scmHandler.groovy reside in a test/ subdir of the workspace in the git repo of the project to boild and are checked out correctly on master:
/var/lib/jenkins/jobs/myJenkinsJob/workspace#script/test/scmHandler.groovy
However I cannot find them on the slave node where the jenkinsfile executes the build steps inside a node {}. There I only see old versions of the jenkinsfile since the (separated) checkout step is not executed yet.
How do I correctly access the handleCheckout.groovy? What am I miss here?
Actually I find this a neat way to "include" external groovy files without using a separate library.
Use checkout scm before loading scmHandler.groovy
checkout scm
def scmHandler = load ("test/scmHandler.groovy")
scmHandler.handleCheckout()
Jenkins declarative pipelines offer a post directive to execute code after the stages have finished. Is there a similar thing to run code before the stages are running, and most importantly, before the SCM checkout?
For example something along the lines of:
pre {
always {
rm -rf ./*
}
}
This would then clean the workspace of my build before the source code is checked out.
pre is a cool feature idea, but doesn't exist yet. skipDefaultCheckout and checkout scm (which is the same as the default checkout) are the keys:
pipeline {
agent { label 'docker' }
options {
skipDefaultCheckout true
}
stages {
stage('clean_workspace_and_checkout_source') {
steps {
deleteDir()
checkout scm
}
}
stage('build') {
steps {
echo 'i build therefore i am'
}
}
}
}
For the moment there are no pre-build steps but for the purpose you are looking for, it can be done in the pipeline job configurarion and also multibranch pipeline jobs, when you define where is your jenkinsfile, choose Additional Behaviours -> Wipe out repository & force clone.
Delete the contents of the workspace before building, ensuring a fully fresh workspace.
If you do not really want to delete everything and save some network usage, you can just use this other option: Additional Behaviours -> Clean before checkout.
Clean up the workspace before every checkout by deleting all untracked files and directories, including those which are specified in .gitignore. It also resets all tracked files to their versioned state. This ensures that the workspace is in the same state as if you cloned and checked out in a brand-new empty directory, and ensures that your build is not affected by the files generated by the previous build.
This one will not delete the workspace but just reset the repository to the original state and pull new changes if there are some.
I use "Prepare an environment for the run / Script Content"
I have an open-source project, that resides in GitHub and is built using a build farm, controlled by Jenkins.
I want to build it branch-wise using a pipeline, but I don't want to store Jenkinsfile inside the code. Is there a way to accomplish this?
I have encountered the same issue as you. While the idea of having the build process as part of the code is good, there is information that the Jenkinsfile would include that are not intrinsic to the project build itself, but rather are specific to the build environment instance, which may change.
The way I accomplished this is:
Encapsulate the core build process in a single script (build.py or build.sh). This may call specific build tools like Make, CMake, Ant, etc.
Tell Jenkins via the Jenkinsfile to call a function defined in a single global library
Define the global Jenkins build function to call the build script (e.g. build.py) with appropriate environment settings. For example, using custom tools and setting up the PATH.
So for step 2, create a Jenkinsfile in your project containing just the line
build_PROJECTNAME()
where PROJECTNAME is based on the name of your project.
Then use the Pipeline Shared Groovy Libraries Plugin and create a Groovy script in the shared library repository called vars/build_PROJECTNAME.groovy containing the code that sets up the environment and calls the project build script (e.g. build.py):
def call() {
node('linux') {
stage("checkout") {
checkout scm
}
stage("build") {
withEnv([
"PATH+CMAKE=${tool 'CMake'}/bin",
"PATH+PYTHON=${tool 'Python-3'}",
"PATH+NINJA=${tool 'Ninja'}",
]) {
execute 'python build.py'
}
}
}
}
First of all, why do you not want a Jenkinsfile in your code? The pipeline is just as much part of the code as would be your build file.
Other then that, you can load groovy files to be evaluated as a pipeline script. You can do this either from a different location with the from SCM option and then checkout the actual code. But this will force you to manually take care of the branch builds.
Another option would be to have a very basic Jenkinsfile that merely checkouts an external pipeline.
You would get something like this:
node{
deleteDir()
git env.flowScm
def flow = load 'pipeline.groovy'
stash includes: '**', name: 'flowFiles'
stage 'Checkout'
checkout scm // short hand for checking out the "from scm repository"
flow.runFlow()
}
Where the pipeline.groovy file would contain the actual pipeline would look like this:
def runFlow() {
// your pipeline code
}
// Has to exit with 'return this;' in order to be used as library
return this;