How to move Jenkins Build Record Root Directory - jenkins

I am confused on how to move the "Build Record Root Directory". Right now this is my configuration:
So on disk, for a multibranch pipline it looks like this:
Workspace:
Jobs:
So for my job "WindToolsService" the configuration ${ITEM_ROOTDIR}/builds results in the following path:
C:\Jenkins\jobs\WindToolsService\branches\develop\builds
Now I want to move the entire C:\Jenkins\jobs directory to a new disk, what is the correct setting to use?
We have tried the following but it didn't work, the C:\Jenkins\jobs folder was recreated when I tried a build a job.
Copy the contents of “C:\Jenkins\jobs” to “E:\builds”
Rename “C:\Jenkins\jobs” to “C:\Jenkins\jobs_temp” to make sure new setting works
Restart Jenkins
What am I missing here?
Update
Ok - this is not really possible, at least the way I want to do it.
The original path structure is : C:\Jenkins\jobs\JOBNAME\branches\BRANCHNAME\builds but the new path is E:\builds\JOBNAME\BRANCHNAME\builds. This mean I cant just change the setting and cut and paste the contents of the jobs folder, the directory structure is different. Also the jobs folder contains other job information apart from the build records.
I would be cleaner and easier just to move the entire jenkins installation

The jobs folder contains some other information besides the build history, for example it contains the actual job definition (more info here). Additionally the option Build Record Root Directory only affects the builds located in the job folder. So, you will need to keep the jobs folder and all the sub-folders for the jobs (but not the builds folder in all the job folders).
Apparently Jenkins expands the variables ITEM_FULL_NAME and ITEM_ROOTDIR differently. This means that it's not a simple matter of copying the jobs folder to the new location, especially when you use multibranch pipelines, matrix jobs, organisation folders or normal folders. For more info see discussion in comments
Anyway, with that in mind, here is an updated answer (old one is below):
Since it isn't straight forward as copying a folder, the folder structure needs to be transformed, this means that it is best to let Jenkins do the move. The following groovy script will move the builds folders to the correct place by expanding the different macros. The script is executed in the Script Console (Manage Jenkins -> Script Console). Change the newRootBuildsFolder to what ever location you need to move to.
newRootBuildsFolder = new File("E:\builds")
def visitJob(job) {
if (job instanceof Job) {
def oldBuildsFolder = job.getBuildDir()
def newBuildsFolder = new File(newRootBuildsFolder, Jenkins.expandVariablesForDirectory('${ITEM_FULL_NAME}', job.getFullName(), job.getRootDir().getPath()))
new FilePath(oldBuildsFolder).copyRecursiveTo("**", new FilePath(newBuildsFolder))
}
if (job instanceof ItemGroup) {
for (item in job.getItems()) {
visitJob(item)
}
}
}
visitJob(Jenkins.instance)
println "Done"
Please note that this may take several minutes or hours if the instance is big with loads with build history. You can monitor the progress of the script through the file system, i.e. go to the new builds folder and check which jobs has been moved.
After this script has been executed, the builds folder will need to be update in the Jenkins configuration;
Go to Manage Jenkins -> Configure System and change Build Record Root Directory to E:\builds\${ITEM_FULL_NAME}\builds
Restart Jenkins
OLD ANSWER:
Now, with that in mind, the following should work:
Wait until no builds are ongoing, and prevent new builds from starting during the remainder of these procedures.
Copy the contents of C:\jenkins\jobs to E:\builds.
Go to Manage Jenkins -> Configure System and change Build Record Root Directory to E:\builds\${ITEM_FULL_NAME}\builds
Restart Jenkins
Start a new build and ensure that the new build information is written to the folder under E:\builds\. E.g. if we have the job X and previous build was 123, then when we start a new build for X, build 124, then the folder E:\builds\X\builds\124 should be created, and the folder C:\jenkins\jobs\X\builds\124 should not be created.
Later on, when you see that all is working when, then you can delete the old builds folders under C:\jenkins\jobs\**\.

Related

How to delete a build from Jenkins job workspace

I wonder if it is possible to remove only one build (including artifacts) from job workspace.
I tried to "Delete Build" in Build History but all it does is remove build reference from Build History table. I know I can ssh to a server and delete files from the command line but I am looking for a way to do it from Jenkins web interface.
After installing Workspace Cleanup Plugin I am able to wipe out current workspace but I want to keep my other builds in the workspace.
In your Jenkins instance, to be able to have folder/per build - set flag "Use custom workspace" in your job's settings. Here is a brief help info from the setting description:
For each job on Jenkins, Jenkins allocates a unique "workspace directory."
This is the directory where the code is checked out and builds happen.
Normally you should let Jenkins allocate and clean up workspace directories,
but in several situations this is problematic, and in such case, this option
lets you specify the workspace location manually.
One such situation is where paths are hard-coded and the code needs to be
built on a specific location. While there's no doubt that such a build is
not ideal, this option allows you to get going in such a situation.
...
And your custom directory path would look like this:
workspace\$JOB_NAME\$BUILD_NUMBER ~> workspace\my-job-name\123
where $JOB_NAME will be "my-job-name" and $BUILD_NUMBER is the build number, eq. "123".
There is one nasty problem with this approach and this is why I wouldn't recommend to use it - Jenkins will not be able to reclaim disk space for outdated builds. You would have to handle cleanup of outdated builds manually and it is a lot of hassle.
Alternative approach, that gives you more control, tools and is able to keep disk space usage under control (without your supervision) is to use default workspace settings and archive your build output (files, original source code, libraries and etc.) as a post-build action. Very-very handy and gives you access to a whole bunch of great tools like, Copy Artifact Plugin or ArtifactDeployer Plugin in other jobs.
Hope that info helps you make a decision that fits your needs best.
I also use "General/Advanced/Use custom workspace" (as in #pabloduo's answer) on a Windows machine with something like:
C:\${JOB_NAME}\${BUILD_NUMBER}
Just wanted to add a solution for getting rid of the build job's workspaces.
I use Groovy Events Listener Plugin for this.
Using the plug-in's standard configuration I just use the following Groovy script:
if (event == Event.JOB_DELETED){
new File(env.WORKSPACE).deleteDir()
}
And now the custom workspace is deleted when the build job is deleted.
Just be aware that this would also delete non-custom workspaces (because the event is triggered for all jobs on your Jenkins server).

Is the archive directory in Jenkins saved even if I set Max # of builds to keep to a number?

I'm trying to understand the purpose of the archive directory.
From what I've seen, the archive is per build. Meaning if I set the 'Max # of builds to keep' to 3 for example, The archive will also be deleted.
Is that correct?
If so, how can I tell Jenkins to only save my artifact (zip file) created and delete the build directory?
Is there a one archive directory for all builds?
There are two directories for a Jenkins job, by default (there can be just one, builds, in case of a Workflow job, or more, e.g. an additional promotions directory):
JENKINS_HOME/jobs/${JOB_NAME}/builds/ containing sub-dirs according to $BUILD_ID
JENKINS_HOME/jobs/${JOB_NAME}/workspace/
So, there's one workspace per job and one build directory per build.
Don't let the page Administering Jenkins with this change driven by JENKINS-8446 confuse you. The workspace default, as also mentioned in the inline help (of the current v1.635 at the time of this writing), is still:
Manage Jenkins → Configure System → Advanced... → Workspace Root Directory: ${ITEM_ROOTDIR}/workspace
It's NOT ${JENKINS_HOME}/workspace/${JOB_NAME}.
If a discard of old builds takes place the workspace isn't discarded, but, since it exists per job, it is overwritten with every new build. Hence, if you'd like to keep old artifacts you can:
create an "archive" job that uses the Copy Artifact Plugin and connect it as downstream to your build job.
move them to some other location yourself (including creation of uniquely-named sub-dirs to store the artifact(s) in) using a Post-build Action → Groovy Postbuild or → Execute a set of scripts.
There's also the Discard Old Build plugin which enhances Jenkins' built-in discard functionality via a Post-build Action.

Jenkins creating wrong folder for the new jobs that were copied from other existing jobs

Sorry, for the confusing title. I am totally new to Jenkins and have been handed over Jenkins to maintain which was set-up by someone else.
This is Jenkins Master slave config. I have 1 Master and 3 Slaves.
When I create a new job by "copying an existing" job, the new job works fine and no issues.
QUESTION: I see that in Jenkins workspace, this new job is creating a folder with the name of the original job that it was copied from. Why it is not creating a folder with the name of the new job instead?
Now, this is certainly not a show stopper for me, but it seems that Jenkins is creating a folder in workspace for each job that is run. And hence this particular folder is causing some confusion (although notional it is).
Hence, could you help me find out why the new job is creating a workspace folder with the name of original job it was copied from.
BTW, above issue was seen on the Jenkins slave.
It can be solved by configuring the correct building workspace in jenkins job.
General > Advanced > custom workspace > "give your correct workspace"
I had the same problem:
copied some jenkins project and wondered about hard coded workspace paths
Console output of the copied project. Job failed due to missing D: drive.
12:30:44 java.io.IOException: Failed to mkdirs: D:\TEAMS\WORKSPACE\RELEASE_1_1
The problem i had: the 'Advanced project options' were not expanded and the configure GUI had an enormous width, that i didn´t see the button to expand and show the 'advanced' settings.
In fact (thanks to sti): the original project had some hard coded workspace path.
One possibility is that you accidentally triggered the wrong job. You could change the job to print the directory where it executes by adding something like:
echo "XXX $JOB_NAME running in directory $WORKSPACE"
into the build step script. Then look for XXX in the build console log.
Second possibility is that you found an old workspace of the original job. Jenkins leaves workspaces lying around just in case it needs them again so it does not have to make them from scratch.
Third possibility is the original job is configured to use a hard-coded path as workspace. (Custom workspace). If you clone such a job, it would be a good idea to change the hard-coded path. An even better idea would be to let Jenkins manage the workspace and it's naming.
And finally, if all the other possibilities have been checked, you may have found a bug. You could look for it in https://issues.jenkins-ci.org/ and create a bug report if it is a new one.

Hudson/Jenkins PMD Configuration

I am new to Jenkins and just started configuring it. This is what i have done till now:
Installed and configured Jenkins to display the home page. Added PMD plugin.
Set the HUDSON_HOME to a specific directory > C:\Work\Jenkins
Configured a test build to run a simple do-nothing ant script. It runs successfully
Written an independent pmdbuild.xml to run checks on a set of files in C:\myview (I am using clearcase). This xml also copies the output pmd_results.xml to the workspace directory in $HUDSON_HOME/[job-name]/workspace
Now I added the pmdbuild.xml as a step in my primary build. So my build has 2 steps:
a. Run a simple script, do-nothing.
b. Run pmdbuild.xml which generate pmd_results.xml and place it in $HUDSON_HOME/[job-name]/workspace (HARD-CODED as Jenkins PMD plugin expects the file there)
Jenkins picks up the pmd_results.xml automatically with the plugin and displays warnings and everything.
Now the problem:
If I click on a filename in the PMD results, it gives a filenotfound exception as it is looking for the source file in $HUDSON_HOME/[job-name]/workspace.
My java code files are placed in C:\myview (a clearcase snapshot view)
My question is, do I need all my code files to be present inside $HUDSON_HOME/[job-name]/workspace ?? Meaning can't I tell Jenkins to look for the PMD input files in C:\myview or any other directory instead of $HUDSON_HOME/[job-name]/workspace ??
Sorry for the extremely long description.
Jenkins expects that all the code is in the workspace. Usually Jenkins is used to check out a copy of the code into the workspace, and then runs all build steps on the Sources in the Workspace.
Might seem restraining at first, but it saves you a lot of trouble if you need to move Jenkins to another server, or create a slave instance.
So I would suggest you let Jenkins check out your code (there should be a clearcase plugin) into the workspace, and run the analysis on the checked out code.
If there are compelling reasons why your code has to stay where it is (C:\myview in your case) you can still set the workspace of your build to that directory (find this in the job configuration page, you need to click on the 'extended' button to see the option).

Can't see copied Jenkins jobs from one instance to the other in a destination instance folder?

I am copying Jenkins jobs from one instance to the other. I created a folder called "
Old_Jobs" in the destination instance under jobs directory. If I copy all the jobs under this Old_jobs directory and reload configuration from disk, I can't see those jobs in the Jenkins GUI. However, if I copy those jobs under the "jobs" directory, I can see all the jobs in Jenkins GUI.
Is there any way I can see all my copied jobs under /var/lib/jenkins/jobs/Old_Jobs/ directory?
Note- I have tried changing permission to 777 in the destination folder, but it didn't work.
Ownership is also correct in the destination instance.
AFAIK, all the jobs are listed under /jobs/
Since you have created one more directory, "Old_Jobs" under /jobs, the required structure is not present.
Also, I remember facing similar issue (even with keeping same directory structure) and I had to copy "/workspace" folder as well to the new instance.
You can refer required directory structure over here : https://wiki.jenkins.io/display/JENKINS/Administering+Jenkins
It also mentioned below points:
Moving/copying/renaming jobs You can:
Move a job from one installation of Jenkins to another by simply copying the corresponding job directory.
Make a copy of an existing job by making a clone of a job directory by a different name.
Rename an existing job by renaming a directory. Note that the if you change a job name you will need to change any other job that tries
to call the renamed job.
Those operations can be done even when Jenkins is running. For changes
like these to take effect, you have to click "reload config" to force
Jenkins to reload configuration from the disk.

Resources