i have multiple big directories in the workspace directory in Jenkins, these directories are about a year old can I rm -rf these folders or it would break something? and what does it mean when i have the same folder name but with different ending such as #2 , #2#tmp?
As far as I know, you can remove everything without any concern. Of course, you'll lose all the job outputs and archived artifacts.
About #2 folders, if I'm not wrong, they are created when there are concurrent jobs running.
Related
I'm using Jenkins on Windows with the Windchill RV&S (Integrity) configuration management plugin. I've noticed that from time to time, it deletes the #tmp directories for project workspaces, but it won't recreate them. i.e. if I have this directory structure:
C:\PTC\workspace\Project1
C:\PTC\workspace\Project1#tmp
Then sometimes the build will fail when C:\PTC\workspace\Project1#tmp goes missing. I don't know why Jenkins deletes it. Maybe it happens when I enable the cleanCopy option in my pipeline? The build failure message is usually something like this:
C:\PTC\workspace\Project1\build>make
..\..\path\to\compiler ..\..\path\to\source_file.c
#error compiler ..\..\path\to\source_file.c:0 can't create C:\PTC\workspace\Project1#tmp\s634.cx1
..\..\path\to\source_file.c:
make: Error code 1
So I re-create C:\PTC\workspace\Project1#tmp manually and the build works again. How do I tell Jenkins to create these #tmp directories automatically?
Edit:
My builds are happening daily (#midnight). They'll work fine for about a month or so, and then on the same day, all the #midnight builds will fail. Each #midnight build will lose its respective #tmp directory and every file in its workspace will be gone (but all the directories will remain). To recover from this, I have to manually recreate the #tmp directory and specify the cleanCopy option (otherwise Windchill RV&S won't restore the missing files in the empty workspace directories).
I have many jenkins jobs using a shared library
#Library('my_shared_library') _
On the folder of each build of each project using the library, there's a local copy of it, for example:
jenkins/jobs/my_project/jobs/builds/10/libs/my_shared_library
That means, it's copied on each build's history.
Plus, I see it also on the project's workspace, under:
jenkins/workspace/my_project#libs/my_shared_library
On this last directory there's sometimes several copies of this, like :
my_shared_library
my_shared_library#2
my_shared_library#tmp
...
Question is, is there some special way of handling this so that there'll be less copies? Or, why's there a copy on the project's workspace and one on the build history?
This ends up creating lots of files, which caused my jenkins to run out of inodes.
I am confused on how to move the "Build Record Root Directory". Right now this is my configuration:
So on disk, for a multibranch pipline it looks like this:
Workspace:
Jobs:
So for my job "WindToolsService" the configuration ${ITEM_ROOTDIR}/builds results in the following path:
C:\Jenkins\jobs\WindToolsService\branches\develop\builds
Now I want to move the entire C:\Jenkins\jobs directory to a new disk, what is the correct setting to use?
We have tried the following but it didn't work, the C:\Jenkins\jobs folder was recreated when I tried a build a job.
Copy the contents of “C:\Jenkins\jobs” to “E:\builds”
Rename “C:\Jenkins\jobs” to “C:\Jenkins\jobs_temp” to make sure new setting works
Restart Jenkins
What am I missing here?
Update
Ok - this is not really possible, at least the way I want to do it.
The original path structure is : C:\Jenkins\jobs\JOBNAME\branches\BRANCHNAME\builds but the new path is E:\builds\JOBNAME\BRANCHNAME\builds. This mean I cant just change the setting and cut and paste the contents of the jobs folder, the directory structure is different. Also the jobs folder contains other job information apart from the build records.
I would be cleaner and easier just to move the entire jenkins installation
The jobs folder contains some other information besides the build history, for example it contains the actual job definition (more info here). Additionally the option Build Record Root Directory only affects the builds located in the job folder. So, you will need to keep the jobs folder and all the sub-folders for the jobs (but not the builds folder in all the job folders).
Apparently Jenkins expands the variables ITEM_FULL_NAME and ITEM_ROOTDIR differently. This means that it's not a simple matter of copying the jobs folder to the new location, especially when you use multibranch pipelines, matrix jobs, organisation folders or normal folders. For more info see discussion in comments
Anyway, with that in mind, here is an updated answer (old one is below):
Since it isn't straight forward as copying a folder, the folder structure needs to be transformed, this means that it is best to let Jenkins do the move. The following groovy script will move the builds folders to the correct place by expanding the different macros. The script is executed in the Script Console (Manage Jenkins -> Script Console). Change the newRootBuildsFolder to what ever location you need to move to.
newRootBuildsFolder = new File("E:\builds")
def visitJob(job) {
if (job instanceof Job) {
def oldBuildsFolder = job.getBuildDir()
def newBuildsFolder = new File(newRootBuildsFolder, Jenkins.expandVariablesForDirectory('${ITEM_FULL_NAME}', job.getFullName(), job.getRootDir().getPath()))
new FilePath(oldBuildsFolder).copyRecursiveTo("**", new FilePath(newBuildsFolder))
}
if (job instanceof ItemGroup) {
for (item in job.getItems()) {
visitJob(item)
}
}
}
visitJob(Jenkins.instance)
println "Done"
Please note that this may take several minutes or hours if the instance is big with loads with build history. You can monitor the progress of the script through the file system, i.e. go to the new builds folder and check which jobs has been moved.
After this script has been executed, the builds folder will need to be update in the Jenkins configuration;
Go to Manage Jenkins -> Configure System and change Build Record Root Directory to E:\builds\${ITEM_FULL_NAME}\builds
Restart Jenkins
OLD ANSWER:
Now, with that in mind, the following should work:
Wait until no builds are ongoing, and prevent new builds from starting during the remainder of these procedures.
Copy the contents of C:\jenkins\jobs to E:\builds.
Go to Manage Jenkins -> Configure System and change Build Record Root Directory to E:\builds\${ITEM_FULL_NAME}\builds
Restart Jenkins
Start a new build and ensure that the new build information is written to the folder under E:\builds\. E.g. if we have the job X and previous build was 123, then when we start a new build for X, build 124, then the folder E:\builds\X\builds\124 should be created, and the folder C:\jenkins\jobs\X\builds\124 should not be created.
Later on, when you see that all is working when, then you can delete the old builds folders under C:\jenkins\jobs\**\.
We have the Jenkins folder plugin. When a new folder is created, and 'Add an item' is selected, none of the projects from the containing Jenkins database are available to be copied in the 'copy from' selection. What's the easiest way to copy an existing Jenkins project into a new folder?
You need to specify the absolute path.
For example if you create folder
Util
And inside Util you want to copy job "FooJob" from the base level in "copy from" specify
/FooJob
Or of FooJob is inside folder Bar specify
/Bar/FooJob
Select your project
Choose "Move"
Select your folder from the drop-down
The answer given by krtrego (use absolute paths) is correct -- but: you can also copy the job on filesystem level, which is preferable in many cases. That is, on Linux, do
cp -ar $JENKINS_HOME/jobs/jobXYZ $JENKINS_HOME/jobs/folderABC/jobs
Shutdown Jenkins beforehand and start it afterwards (or do not shutdown, and just "reload configuration from disk" afterwards).
Depending on the definition of "easy", this might be the best choice, since
If you're copying many jobs, then this is only way that jobs won't start before you completed moving all of them ("Folder copy" on GUI-level does not work in "quiet-down" mode).
The Folder copy operation sometimes takes ages (literally) for certain jobs.
Tagline: for anything but the most simple tasks, it will be good to avoid Folder Move/Copy.
I am trying to use the Generic Jenkins-Artifactory plugin to deploy the contents of the Jenkins build workspace into Artifactory. This seems to be somewhat fine using the following wildcards
Web\*.msi=>Testing\Web
Web\DeploymentSettings\*.xml=>Testing\Web\DeploymentSettings
Database\Scripts\**=>Testing\Database
However, when it comes to moving the contents of 'Database\Scripts' from the Jenkins workspace, empty folders under 'Database\Scripts' do not get copied into Artifactory. Non-empty folders however are copied successfully. It is important that I maintain the directory integrity/structure so it's a must that I copy these across.
I have considered placing empty text files in the empty directories to have them copy over successfully but I don't want to "pollute" the package.
Please help :-)
Thanks!
Looks like there is no workaround -other than dummy files in the directories.
I see some bugs in Jenkins releated to handling empty directories.
JENKINS-7260 Clone workspace doesn't copy empty directories when cloning entire workspace
JENKINS-20654Empty folders are not copied to the slave
Could you check if answer posted in Hudson: Copy artifact from master to slave fails helps?