Cheers everyone,
I am in a Kubernetes environment and Jenkins plugins are installed at image build time via means of Jenkins CLI.
This causes the Image to have all plugins I require in the JENKINS_HOME inside the image.
When running the image however we mount a single volume to the location of JENKINS_HOME as to persist all files in that location.
This obviously overlays the files of the image with the files on the volume. Classic problem.
Now: In "pure" Docker I was able to simply "re-mount" the plugin folder's location again (I won't go into detail to keep this a little shorter) to "pull it back up to the surface". Thus resulting in all files from the volume BUT the plugins folder from the image.
This functionality does not seem to work in Kubernetes anymore.
Thus I would like to change the directory in which Jenkins stores its plugins (when I install them during build time) so they won't be in JENKINS_HOME anymore and thus won't be "over-mounted" by the volume at runtime.
I am having a hard time finding a config value for the plugin directory. Is there one?
Related
I was trying to create a Jenkins master slave system using Helm Charts. [ https://github.com/jenkinsci/helm-charts/blob/main/charts/jenkins/README.md ].
After deploying I can see the plugins are stored in two locations in the pod/ docker container.
1. /var/jenkins_home/plugins
2. /usr/share/jenkins/ref/plugins
What's the relation between these two? JENKINS_HOME is configured as /var/jenkins_home/. If I need to backup the plugins which directory should I backup ?
For a plugin named ace-editor, I can see these many files in respective folders.
/usr/share/jenkins/ref/plugins contains (1 file) -> ace-editor.jpi
/var/jenkins_home/plugins contains (4 file) -> ace-editor.jpi, ace-editor.jpi, ace-editor.jpi.pinned, ace-editor.jpi.version_from_image
What does this mean?
The directory to back up is always /var/jenkins_home. It is meant to be a persistent volume with all the data. /var/jenkins_home/plugins contains the downloaded .jpi plugin files and directories where each plugin has been unpacked.
The directory /usr/share/jenkins/ref is a "reference" directory on the container image. This means /usr/share/jenkins/ref/plugins is used as a starting point to install a base set of plugins into /var/jenkins_home/plugins the first time you run Jenkins.
The log file containing info about installed reference files is /var/jenkins_home/copy_reference_file.log.
The files with suffix .pinned or .version_from_image are used to control subsequent behaviour for example when plugins have been updated in jenkins_home from the UI and/or to the reference directory in a newer container image. Some users might want to use the bundled plugins in the image only for initial startup, others as the definitive source of all updates. See also environmental variables described in the documentation.
If you want to examine the inner workings you can find shell script jenkins.sh and functions source jenkins-support in the repository root or in /usr/local/bin on the image.
I'm using the stable helm chart in order to deploy it in my OCP environment.
I activate the persistence volume where jenkins save the jenkins_home folder.
I found that the helm chart define the plugin and plugin-dir folder as empty volume folder.
It's cause to download the plugins every time when pod is restarted.
How can I configure it that plugin dir will be also configured as persistence volume?
I think the problem lies in this line.
The default value for the overwritePlugins variable is true.
This means that if you do not explicitly set it to false, the ConfigMap that will be generated will create the following line, in an inline script to be executed
rm -rf /var/jenkins_home/plugins/*
Then, and assuming you have elements in the installPlugins list, it will proceed with (re-downloading) and installing those plugins.
Keep in mind that it seems that AFTER installation, plugins seem to end up to /var/jenkins_home/plugins that (assuming you have set a PVC) is a persistent location.
I want my Cloud Build to push an image to a registry with an incremented tag. So, when the trigger arrives from GitHub, build the image, and if the latest tag was 1.10, tag the new one 1.11. Similarly, the 1.11 value will serve in multiple other steps in the build.
Reading the registry and incrementing the tag is easy (in a bash Cloud Build step), but Cloud Build has no way to pass parameters. (Substitutions come from outside the Cloud Build process, for example from the Git tags, and are not generated inside the process.)
This StackOverflow question and this article say that Cloud Build steps can communicate by writing files to the workspace directory.
That is clumsy. But worse, this requires using shell steps exclusively, not the native docker-building steps, nor the native image command.
How can I do this?
Sadly you can't. The Cloud Builder image have each time their own sandbox and only the /workspace directory is mounted. By the way, all the environment variable, binaries installed and so, doesn't persist from one container to the next one.
You have to use the shell script each time :( The easiest way is to have a file in your /workspace directory (for example env.var file)
# load the environment variable
source /workspace/env.var
# Add variable
echo "NEW=Variable" >> /workspace/env.var
For this, Cloud Build is boring...
I have a series of Groovy scripts in groovy.init.d inside a Docker container which has the JENKINS_HOME mounted to the host file system.
I'm using the standard jenkins-support file to copy the plugins et al.
https://raw.githubusercontent.com/jenkinsci/docker/master/jenkins-support
When I add new files to groovy.init.d, the script adds them no problem, but if I update the scripts then the newer ones are not overwriting the existing ones. I want the groovy files added to the Docker image to overwrite what is on the file system.
Is this possible?
Yes, put .override extension to your file and put to /usr/share/jenkins/ref/init.groovy.d/ on the
image example
/usr/share/jenkins/ref/init.groovy.d/security.groovy.override .
When jenkins starts it will replace existing
/var/jenkins_home/init.groovy.d/security.groovy
file's content with the one above.
I would like Jenkins' data to be written to drive "E:" since this is the large drive on the server. Jenkins itself is installed on "C:".
How do I do that?
The default configuration that I saw is:
Workspace Root Directory: ${ITEM_ROOTDIR}/workspace
Build Record Root Directory: ${ITEM_ROOTDIR}/builds
Will the following changes help me achieve what I need?
Workspace Root Directory: E:/Jenkins/workspace
Build Record Root Directory: E:/Jenkins/builds/${ITEM_FULL_NAME}
In addition, what does "${ITEM_FULL_NAME}" mean?
I figured it out.
In order to save your Jenkins data on other drive you'll need to do the following:
Workspace Root Directory: E:\Jenkins\${ITEM_FULL_NAME}\workspace
Build Record Root Directory: E:\Jenkins\${ITEM_FULL_NAME}\builds
You can modify the path on the config.xml file in the default directory
<projectNamingStrategy class="jenkins.model.ProjectNamingStrategy$DefaultProjectNamingStrategy"/>
<workspaceDir>D:/Workspace/${ITEM_FULL_NAME}</workspaceDir>
<buildsDir>D:/Logs/${ITEM_ROOTDIR}/Build</buildsDir>
EDIT: Per other comments, the "Advanced..." button appears to have been removed in more recent versions of Jenkins. If your version doesn't have it, see knorx's answer.
I had the same problem, and even after finding this old pull request I still had trouble finding where to specify the Workspace Root Directory or Build Record Root Directory at the system level, versus specifying a custom workspace for each job.
To set these:
Navigate to Jenkins -> Manage Jenkins -> Configure System
Right at the top, under Home directory, click the Advanced... button:
Now the fields for Workspace Root Directory and Build Record Root Directory appear:
The information that appears if you click the help bubbles to the left of each option is very instructive. In particular (from the Workspace Root Directory help):
This value may include the following variables:
${JENKINS_HOME} — Absolute path of the Jenkins home directory
${ITEM_ROOTDIR} — Absolute path of the directory where Jenkins stores the configuration and related metadata for a given job
${ITEM_FULL_NAME} — The full name of a given job, which may be slash-separated, e.g. foo/bar for the job bar in folder foo
The value should normally include ${ITEM_ROOTDIR} or ${ITEM_FULL_NAME}, otherwise different jobs will end up sharing the same workspace.
The variables you need are explained here in the jenkins wiki: https://wiki.jenkins.io/display/JENKINS/Features+controlled+by+system+properties
The default variable ITEM_ROOTDIR points to a directory inside the jenkins installation. As you already found out you need:
Workspace Root Directory:
E:/myJenkinsRootFolderOnE/${ITEM_FULL_NAME}/workspace
Build Record Root Directory: E:/myJenkinsRootFolderOnE/${ITEM_FULL_NAME}/builds
You need to achieve this through config.xml nowerdays. Citing from the wiki page linked above:
This used to be a UI setting, but was removed in 2.119 as it did not
support migration of existing build records and could lead to
build-related errors until restart.
I would suggest editing the /etc/default/jenkins
vi /etc/default/jenkins
And changing the $JENKINS_HOME variable (around line 23) to
JENKINS_HOME=/home/jenkins
Then restart the Jenkins with usual
/etc/init.d/jenkins start
Cheers!
You can also edit the config.xml file in your JENKINS_HOME directory.
Use c32hedge's response as a reference and set the workspace location to whatever you want between the tags
By default, Jenkins stores all of its data in this directory on the file system.
There are a few ways to change the Jenkins home directory:
Edit the JENKINS_HOME variable in your Jenkins configuration file (e.g. /etc/sysconfig/jenkins on Red Hat Linux).
Use your web container's admin tool to set the JENKINS_HOME environment variable.
Set the environment variable JENKINS_HOME before launching your web container, or before launching Jenkins directly from the WAR file.
Set the JENKINS_HOME Java system property when launching your web container, or when launching Jenkins directly from the WAR file.
Modify web.xml in jenkins.war (or its expanded image in your web container). This is not recommended.
This value cannot be changed while Jenkins is running.
It is shown here to help you ensure that your configuration is taking effect.
If you go into Configure under Home there is a "Help" note on how to:
Home directory /var/lib/jenkins Help for feature: Home directory
By default, Jenkins stores all of its data in this directory on the file system.
There are a few ways to change the Jenkins home directory:
Edit the JENKINS_HOME variable in your Jenkins configuration file (e.g. /etc/sysconfig/jenkins on Red Hat Linux).
Use your web container's admin tool to set the JENKINS_HOME environment variable.
Set the environment variable JENKINS_HOME before launching your web container, or before launching Jenkins directly from the WAR file.
Set the JENKINS_HOME Java system property when launching your web container, or when launching Jenkins directly from the WAR file.
Modify web.xml in jenkins.war (or its expanded image in your web container). This is not recommended.
This value cannot be changed while Jenkins is running.
It is shown here to help you ensure that your configuration is taking effect.
def finder = new FileNameFinder()
def files = finder.getFileNames '\\pkg\\stage\\', '*.sln'
echo "$files"
jenkins is serching file in /var/lib/jenkins/pkg/stage
it is automatically adding /var/lib/jenkins path before