We have a fairly straight-forward Jenkins instance with inter-dependent jobs.
In order to move artifacts from one job to another, we make use of the %JENKINS-HOME%/userContent folder and the Copy Artifact plugin.
That is, we copy some files from the result of a build into the userContent folder and then the next job picks these up and continues.
The default Jenkins UI allows you to see the folders and files in the userContent folder along with their filesizes, but is there a way to indicate their creation time/date?
Related
Is there any way to access the previously build artifacts in a post-build-plugin? If yes, how do I access them?
For example:
My build creates a .jar file as an artifact.
In my plugin, I would like to access that .jar file and send it to an external server. That server is going to evaluate the file and depending on the result, I'd like to mark the build as failed/unstable/successful.
All build artifacts are in the workspace, in the same path the previous step saved them, ie build/libs/foo.jar. If you click in workspace, you can find it there, and it would be available for any post-build-plugin if you dont move it or delete it. Post build steps are executed in the workspace so it should be accessible with ie ./build/libs/foo.jar
I am confused on how to move the "Build Record Root Directory". Right now this is my configuration:
So on disk, for a multibranch pipline it looks like this:
Workspace:
Jobs:
So for my job "WindToolsService" the configuration ${ITEM_ROOTDIR}/builds results in the following path:
C:\Jenkins\jobs\WindToolsService\branches\develop\builds
Now I want to move the entire C:\Jenkins\jobs directory to a new disk, what is the correct setting to use?
We have tried the following but it didn't work, the C:\Jenkins\jobs folder was recreated when I tried a build a job.
Copy the contents of “C:\Jenkins\jobs” to “E:\builds”
Rename “C:\Jenkins\jobs” to “C:\Jenkins\jobs_temp” to make sure new setting works
Restart Jenkins
What am I missing here?
Update
Ok - this is not really possible, at least the way I want to do it.
The original path structure is : C:\Jenkins\jobs\JOBNAME\branches\BRANCHNAME\builds but the new path is E:\builds\JOBNAME\BRANCHNAME\builds. This mean I cant just change the setting and cut and paste the contents of the jobs folder, the directory structure is different. Also the jobs folder contains other job information apart from the build records.
I would be cleaner and easier just to move the entire jenkins installation
The jobs folder contains some other information besides the build history, for example it contains the actual job definition (more info here). Additionally the option Build Record Root Directory only affects the builds located in the job folder. So, you will need to keep the jobs folder and all the sub-folders for the jobs (but not the builds folder in all the job folders).
Apparently Jenkins expands the variables ITEM_FULL_NAME and ITEM_ROOTDIR differently. This means that it's not a simple matter of copying the jobs folder to the new location, especially when you use multibranch pipelines, matrix jobs, organisation folders or normal folders. For more info see discussion in comments
Anyway, with that in mind, here is an updated answer (old one is below):
Since it isn't straight forward as copying a folder, the folder structure needs to be transformed, this means that it is best to let Jenkins do the move. The following groovy script will move the builds folders to the correct place by expanding the different macros. The script is executed in the Script Console (Manage Jenkins -> Script Console). Change the newRootBuildsFolder to what ever location you need to move to.
newRootBuildsFolder = new File("E:\builds")
def visitJob(job) {
if (job instanceof Job) {
def oldBuildsFolder = job.getBuildDir()
def newBuildsFolder = new File(newRootBuildsFolder, Jenkins.expandVariablesForDirectory('${ITEM_FULL_NAME}', job.getFullName(), job.getRootDir().getPath()))
new FilePath(oldBuildsFolder).copyRecursiveTo("**", new FilePath(newBuildsFolder))
}
if (job instanceof ItemGroup) {
for (item in job.getItems()) {
visitJob(item)
}
}
}
visitJob(Jenkins.instance)
println "Done"
Please note that this may take several minutes or hours if the instance is big with loads with build history. You can monitor the progress of the script through the file system, i.e. go to the new builds folder and check which jobs has been moved.
After this script has been executed, the builds folder will need to be update in the Jenkins configuration;
Go to Manage Jenkins -> Configure System and change Build Record Root Directory to E:\builds\${ITEM_FULL_NAME}\builds
Restart Jenkins
OLD ANSWER:
Now, with that in mind, the following should work:
Wait until no builds are ongoing, and prevent new builds from starting during the remainder of these procedures.
Copy the contents of C:\jenkins\jobs to E:\builds.
Go to Manage Jenkins -> Configure System and change Build Record Root Directory to E:\builds\${ITEM_FULL_NAME}\builds
Restart Jenkins
Start a new build and ensure that the new build information is written to the folder under E:\builds\. E.g. if we have the job X and previous build was 123, then when we start a new build for X, build 124, then the folder E:\builds\X\builds\124 should be created, and the folder C:\jenkins\jobs\X\builds\124 should not be created.
Later on, when you see that all is working when, then you can delete the old builds folders under C:\jenkins\jobs\**\.
I have a jenkins job that pulls source code from GitHub public repo. I need to pass some files such as instance-specific configuration files containing secrets to the job and merge with source code prior to running build because these files are obviously inappropriate to be put in public SCM. The Jenkins instance is a multi-tenanted shared service.
The config files don't change often so I don't want to implement using file parameter which forces user manually input the file on every run. Another reason file parameter doesn't work is some builds are triggered automatically by SCM.
I don't want to use Config File Provider Plugin either, because the plugin requires jenkins admin access but I want users with job-level privileges manage the files themselves.
Ideally the uploaded files are saved alongside with job config.xml instead of in workspace, because I would like to delete workspace after each build. I can write scripts to copy the files from job config folder to workspace.
Are there any solutions available? Thanks.
If the "special" files are being placed in a folder with say some access privileges to it, couldn't you either run a Pre-SCM-Buildstep to move the files with shell commands, or introduce a regular build step (i.e. after the SCM stuff and before the other build steps) that would also use shell commands to move files?
I have two jenkins jobs which are chained. The first job will run some testNG tests , in which we may get some failures. (A testng-failed.xml file will be generated that has all the tests that failed). In my second job, I need to run using that testng-failed.xml as my suite file. So I am using jenkins copy artifact plugin and saving the testng-failed.xml file. I am able to retrive it in the second job , where it says "Copied 1 artifact from "rerun_exp" build number 7".
But the problem is I dont know where it is saved to be used as my suite file.
You can easily check where the file is by looking in the workspace.
This can even be done directly from the Jenkins web UI, via the "Workspace" link in the sidebar of the job page.
Files matched by the "Artifacts to copy" field are copied to the same directory structure as the source build had.
You can use the "Flatten directories" option if you want the file(s) to be copied to the root of the build workspace. If you click the (?) help icon to the right of that checkbox, you can see documentation for this and for your original question.
I am copying Jenkins jobs from one instance to the other. I created a folder called "
Old_Jobs" in the destination instance under jobs directory. If I copy all the jobs under this Old_jobs directory and reload configuration from disk, I can't see those jobs in the Jenkins GUI. However, if I copy those jobs under the "jobs" directory, I can see all the jobs in Jenkins GUI.
Is there any way I can see all my copied jobs under /var/lib/jenkins/jobs/Old_Jobs/ directory?
Note- I have tried changing permission to 777 in the destination folder, but it didn't work.
Ownership is also correct in the destination instance.
AFAIK, all the jobs are listed under /jobs/
Since you have created one more directory, "Old_Jobs" under /jobs, the required structure is not present.
Also, I remember facing similar issue (even with keeping same directory structure) and I had to copy "/workspace" folder as well to the new instance.
You can refer required directory structure over here : https://wiki.jenkins.io/display/JENKINS/Administering+Jenkins
It also mentioned below points:
Moving/copying/renaming jobs You can:
Move a job from one installation of Jenkins to another by simply copying the corresponding job directory.
Make a copy of an existing job by making a clone of a job directory by a different name.
Rename an existing job by renaming a directory. Note that the if you change a job name you will need to change any other job that tries
to call the renamed job.
Those operations can be done even when Jenkins is running. For changes
like these to take effect, you have to click "reload config" to force
Jenkins to reload configuration from the disk.