What is the use of multiple Jenkins plugin directory? - jenkins

I was trying to create a Jenkins master slave system using Helm Charts. [ https://github.com/jenkinsci/helm-charts/blob/main/charts/jenkins/README.md ].
After deploying I can see the plugins are stored in two locations in the pod/ docker container.
1. /var/jenkins_home/plugins
2. /usr/share/jenkins/ref/plugins
What's the relation between these two? JENKINS_HOME is configured as /var/jenkins_home/. If I need to backup the plugins which directory should I backup ?
For a plugin named ace-editor, I can see these many files in respective folders.
/usr/share/jenkins/ref/plugins contains (1 file) -> ace-editor.jpi
/var/jenkins_home/plugins contains (4 file) -> ace-editor.jpi, ace-editor.jpi, ace-editor.jpi.pinned, ace-editor.jpi.version_from_image
What does this mean?

The directory to back up is always /var/jenkins_home. It is meant to be a persistent volume with all the data. /var/jenkins_home/plugins contains the downloaded .jpi plugin files and directories where each plugin has been unpacked.
The directory /usr/share/jenkins/ref is a "reference" directory on the container image. This means /usr/share/jenkins/ref/plugins is used as a starting point to install a base set of plugins into /var/jenkins_home/plugins the first time you run Jenkins.
The log file containing info about installed reference files is /var/jenkins_home/copy_reference_file.log.
The files with suffix .pinned or .version_from_image are used to control subsequent behaviour for example when plugins have been updated in jenkins_home from the UI and/or to the reference directory in a newer container image. Some users might want to use the bundled plugins in the image only for initial startup, others as the definitive source of all updates. See also environmental variables described in the documentation.
If you want to examine the inner workings you can find shell script jenkins.sh and functions source jenkins-support in the repository root or in /usr/local/bin on the image.

Related

Jenkins - How can I change the plugin directory?

Cheers everyone,
I am in a Kubernetes environment and Jenkins plugins are installed at image build time via means of Jenkins CLI.
This causes the Image to have all plugins I require in the JENKINS_HOME inside the image.
When running the image however we mount a single volume to the location of JENKINS_HOME as to persist all files in that location.
This obviously overlays the files of the image with the files on the volume. Classic problem.
Now: In "pure" Docker I was able to simply "re-mount" the plugin folder's location again (I won't go into detail to keep this a little shorter) to "pull it back up to the surface". Thus resulting in all files from the volume BUT the plugins folder from the image.
This functionality does not seem to work in Kubernetes anymore.
Thus I would like to change the directory in which Jenkins stores its plugins (when I install them during build time) so they won't be in JENKINS_HOME anymore and thus won't be "over-mounted" by the volume at runtime.
I am having a hard time finding a config value for the plugin directory. Is there one?

Jenkins with Copy Artifact plugin: copy directory contents with subdirectories

I have a structure of artifacts in another build:
/
/bundle/docs
/bundle/bin
/bundle/bin/scripts
I want to copy all files and sudirectories into the current job's workspace subfolder 'product1' from /bundle/bin. I expect to see in %WORKSAPCE%/product1 contents of /bundle/bin.
I've configured it like this:
Artifacts to copy: bundle/bin/**
But it creates %WORKSAPCE%/product1//bundle/bin instead.
Is it possible?
Seems like that's just how the plugin works. Your options are:
Keep the same configuration and manipulate directories later using sh mv (Linux) or cmd move (Windows) command. This is the workaround used in my environment.
Check the "Flatten directories" option (but this will mix together the /bundle/bin and /bundle/bin/scripts)
Improve the plugin and contribute your code to the community :-)

Jenkins updating init.groovy.d files in a docker container

I have a series of Groovy scripts in groovy.init.d inside a Docker container which has the JENKINS_HOME mounted to the host file system.
I'm using the standard jenkins-support file to copy the plugins et al.
https://raw.githubusercontent.com/jenkinsci/docker/master/jenkins-support
When I add new files to groovy.init.d, the script adds them no problem, but if I update the scripts then the newer ones are not overwriting the existing ones. I want the groovy files added to the Docker image to overwrite what is on the file system.
Is this possible?
Yes, put .override extension to your file and put to /usr/share/jenkins/ref/init.groovy.d/ on the
image example
/usr/share/jenkins/ref/init.groovy.d/security.groovy.override .
When jenkins starts it will replace existing
/var/jenkins_home/init.groovy.d/security.groovy
file's content with the one above.

Obtain the artifact directory path as a relative path for the Release process

We are working on the configuration process for the Continuous Integration for some projects, we are using TFS and now we have a problem with some releases definitions. We want to use the Web Deploy package created in the Build process for the Deployment.
So far the build definition that we have is following:
enter image description here
The path for the creation of the package is the default, so we are able to find it inside the artifact directory. But the problem is when we nee extract the files in the target folder for the website in the server.
The Release definition that we are using is:
enter image description here
In this part in the Download artifact phase the agent doing the release has access to the published files in the build process, so here we know where the .zip package is, and we can have the path using $(System.ArtifactsDirectory), but if we use Deploy IIS App task, as you can see we are connecting to the servers where we are doing the release and $(System.ArtifactsDirectory) give us the local address for the artifacts where the agents are configured, and the variable give the path like C:\agent_work\r1\a, where C is local for the agent, and the .zip file doesn't exist in that address. And we can't build a new path like \Myserver\$(System.ArtifactsDirectory).... , because $(System.ArtifactsDirectory) is an absolute path and as a result the whole path it is : \Myserver\C:\Myfolder....
We need other solution, we have considered in the build process create the package in a different folder, and in this case we always know where is the package, we aren't depending on the agent folders, and in this way we can use as Web Deploy Package path: \Myserver\packagefolder\file.zip, but we would like to use a different solution.
Is there any way to have the artifact folder with a relative path or something like this????
You could use Windows Machine File Copy task to copy the package file from agent to the servers where you are doing the release.
Use this task to copy application files and other artifacts such as
PowerShell scripts and PowerShell-DSC modules that are required to
install the application on Windows Machines. It uses RoboCopy, the
command-line utility built for fast copying of data.
You could use a temporary folder handling the package file on the agent. Such as Build.StagingDirectory. Build variables
Add a packagelocation such as /p:PackageLocation="$(Build.StagingDirectory)\\ in your MSbuild Arguments. Then copy the files from StagingDirectory to your local folder in remote server by using Windows Machine File Copy task.

How to change workspace and build record Root Directory on Jenkins?

I would like Jenkins' data to be written to drive "E:" since this is the large drive on the server. Jenkins itself is installed on "C:".
How do I do that?
The default configuration that I saw is:
Workspace Root Directory: ${ITEM_ROOTDIR}/workspace
Build Record Root Directory: ${ITEM_ROOTDIR}/builds
Will the following changes help me achieve what I need?
Workspace Root Directory: E:/Jenkins/workspace
Build Record Root Directory: E:/Jenkins/builds/${ITEM_FULL_NAME}
In addition, what does "${ITEM_FULL_NAME}" mean?
I figured it out.
In order to save your Jenkins data on other drive you'll need to do the following:
Workspace Root Directory: E:\Jenkins\${ITEM_FULL_NAME}\workspace
Build Record Root Directory: E:\Jenkins\${ITEM_FULL_NAME}\builds
You can modify the path on the config.xml file in the default directory
<projectNamingStrategy class="jenkins.model.ProjectNamingStrategy$DefaultProjectNamingStrategy"/>
<workspaceDir>D:/Workspace/${ITEM_FULL_NAME}</workspaceDir>
<buildsDir>D:/Logs/${ITEM_ROOTDIR}/Build</buildsDir>
EDIT: Per other comments, the "Advanced..." button appears to have been removed in more recent versions of Jenkins. If your version doesn't have it, see knorx's answer.
I had the same problem, and even after finding this old pull request I still had trouble finding where to specify the Workspace Root Directory or Build Record Root Directory at the system level, versus specifying a custom workspace for each job.
To set these:
Navigate to Jenkins -> Manage Jenkins -> Configure System
Right at the top, under Home directory, click the Advanced... button:
Now the fields for Workspace Root Directory and Build Record Root Directory appear:
The information that appears if you click the help bubbles to the left of each option is very instructive. In particular (from the Workspace Root Directory help):
This value may include the following variables:
${JENKINS_HOME} — Absolute path of the Jenkins home directory
${ITEM_ROOTDIR} — Absolute path of the directory where Jenkins stores the configuration and related metadata for a given job
${ITEM_FULL_NAME} — The full name of a given job, which may be slash-separated, e.g. foo/bar for the job bar in folder foo
The value should normally include ${ITEM_ROOTDIR} or ${ITEM_FULL_NAME}, otherwise different jobs will end up sharing the same workspace.
The variables you need are explained here in the jenkins wiki: https://wiki.jenkins.io/display/JENKINS/Features+controlled+by+system+properties
The default variable ITEM_ROOTDIR points to a directory inside the jenkins installation. As you already found out you need:
Workspace Root Directory:
E:/myJenkinsRootFolderOnE/${ITEM_FULL_NAME}/workspace
Build Record Root Directory: E:/myJenkinsRootFolderOnE/${ITEM_FULL_NAME}/builds
You need to achieve this through config.xml nowerdays. Citing from the wiki page linked above:
This used to be a UI setting, but was removed in 2.119 as it did not
support migration of existing build records and could lead to
build-related errors until restart.
I would suggest editing the /etc/default/jenkins
vi /etc/default/jenkins
And changing the $JENKINS_HOME variable (around line 23) to
JENKINS_HOME=/home/jenkins
Then restart the Jenkins with usual
/etc/init.d/jenkins start
Cheers!
You can also edit the config.xml file in your JENKINS_HOME directory.
Use c32hedge's response as a reference and set the workspace location to whatever you want between the tags
By default, Jenkins stores all of its data in this directory on the file system.
There are a few ways to change the Jenkins home directory:
Edit the JENKINS_HOME variable in your Jenkins configuration file (e.g. /etc/sysconfig/jenkins on Red Hat Linux).
Use your web container's admin tool to set the JENKINS_HOME environment variable.
Set the environment variable JENKINS_HOME before launching your web container, or before launching Jenkins directly from the WAR file.
Set the JENKINS_HOME Java system property when launching your web container, or when launching Jenkins directly from the WAR file.
Modify web.xml in jenkins.war (or its expanded image in your web container). This is not recommended.
This value cannot be changed while Jenkins is running.
It is shown here to help you ensure that your configuration is taking effect.
If you go into Configure under Home there is a "Help" note on how to:
Home directory /var/lib/jenkins Help for feature: Home directory
By default, Jenkins stores all of its data in this directory on the file system.
There are a few ways to change the Jenkins home directory:
Edit the JENKINS_HOME variable in your Jenkins configuration file (e.g. /etc/sysconfig/jenkins on Red Hat Linux).
Use your web container's admin tool to set the JENKINS_HOME environment variable.
Set the environment variable JENKINS_HOME before launching your web container, or before launching Jenkins directly from the WAR file.
Set the JENKINS_HOME Java system property when launching your web container, or when launching Jenkins directly from the WAR file.
Modify web.xml in jenkins.war (or its expanded image in your web container). This is not recommended.
This value cannot be changed while Jenkins is running.
It is shown here to help you ensure that your configuration is taking effect.
def finder = new FileNameFinder()
def files = finder.getFileNames '\\pkg\\stage\\', '*.sln'
echo "$files"
jenkins is serching file in /var/lib/jenkins/pkg/stage
it is automatically adding /var/lib/jenkins path before

Resources