JENKINS
I am noticing that the every time I run one of my jobs in Jenkins, there are two files created in the /workspace/build/distributions dir. The two files have the extensions of .tar and .tgz. Every time, I run the job, another set of these files are created. So, if I run the job 3 times, there will be 6 files all together. I have noticed that during the dependency check phase, these artifacts slow things down. Therefore, I wanted to remove them automatically before each time this job runs. I have attempted the configs in the image below. In addition, I have tried the workspace cleanup plugin and that completely deleted the workspace. That is definitely not what I wanted.
Therefore, what would be the best way to go about this.
What scm plugin are you using? Some of the scm plugins allow you to clean the workspace before an update (e.g. SVN's "Emulate clean checkout" and Git's "Clean before checkout" options).
If you're not using a scm plugin, can you remove the files in a batch/shell script during the first build step?
Or perhaps you can go about it from the reverse direction. Can you get rid of the files as the last build step of the job? That way, they are gone when the next build comes along.
Related
I have Jenkins pipeline projects, and everything works fine as long as I run the project at least once per month. If I wait more than a month Jenkins will delete the workspace for that pipeline project, causing the project to do a brand new git checkout and compile. This results in a super slow build, since all of the intermediate object files/etc are regenerated from scratch.
I cannot find what setting in Jenkins is causing it to clean up these older workspaces. If I modify the pipeline to check out to a custom directory instead of the workspace directory then it works fine, so it doesn't appear to be the git plugin itself, or anything like that.
'Discard old builds' is disabled in the General settings for these projects.
Can someone point me to the setting that is causing 'older' workspaces to get cleaned up for some reason?
So I'm running into a specific issue, I have a Jenkins Declarative Pipeline (from an SVN hosted Jenkinsfile) that is configured to not run concurrent builds and abort previous builds when a new build is triggered.
This works perfectly fine, however the problem I am running into is that Jenkins will re-checkout the whole repository to an #2 suffixed workspace directory for the subsequent build (this ONLY happens when a build is automatically aborted after a new one is triggered, if the first build ends successfully, it re-uses the same directory).
I've seen a ton of threads stating that this is by design, but from what I can see that's only when concurrent builds are enabled, but since it's not I'm confused as to what could cause Jenkins to not re-use the same workspace directory?
If the "why" I require this is necessary, I have a few large repositories (for Unreal Engine games specifically), that I need to build and as an optimization measure for the time in compiling, cooking and uploading the game, it makes perfect sense to cancel old builds but instead Jenkins decides to clean checkout 10+GB of game code and assets (20+ in the case of some other games) in another folder becuase it can't reuse a folder that's not having a job/build executed in it already 😅.
Happy to accept all possible solutions/suggestions as I'm getting a lil' tired of pulling my hair out.
I was facing the same issue with my pipeline. I tried deleting the aborted builds and restarted jenkins. I also deleted the #2 directories in my workspace and only kept the main directory out there. Post this,I didn't face the same issue. This could happen because of the jenkins cache. Make sure that your workspace is correctly reflecting the directory name mentioned in your jenkins file.
I am using the cleanup feature in Jenkins, which delete the previous build directory and create a new one every time. This is great, except that I need to maintain certain files in the build directory, so I am trying to delete just the source folder that contain the code.
The problem is that when the build start, the first thing that happen is the git checkout of the code, which means that if I put a delete command in the jenkins script area, it will delete the directory that was checked out, and that obviously won't work.
Is there a way to tell Jenkins to perform commands before the git checkout happen? Or to cleanup selectively the build folder, so Jenkins know what to keep and what to delete?
Use the pre-scm-buildstep plugin. It will let you do all sorts of things prior to touching your SCM.
I have renamed a Jenkins Job from the Jenkins GUI. I changed the Project name in the Configure menu and hitting Save afterwards.
However the workspace name for this Jenkins job has not been changed. What I am finding is upon the job execution a new workspace is getting created with this given new name and none of the contents of the old workspace is getting copied.
So the issue is contents of the old workspace is not copied to the new workspace.
What should I do instead?
I know there are several questions in SO in this area. However those do not answer my question.
Renaming job in jenkins/hudson
Rename a job in Jenkins
So please check this before marking this question as a duplicate.
I was able to workaround this is issue using the Use custom workspace option.
To change this location, I need to choose configure job and click on the Advanced button in the Advanced Project Options section.
After opening the settings, you will find some more configuration options for your job. Look for the Use custom workspace option on the right hand side and check the box.
Reference: Jenkins: Change Workspaces and Build Directory Locations
Workspaces are volatile by nature and may reside on a build node which has gone offline, therefore your build job should not rely on files being present in the workspace. However sometimes you will benefit from a speed-up by reusing unchanged files existing in workspace and decide not to clean them.
When you start a build, a new workspace is (as you noted) created, this is the correct behaviour, you should not need to store files in your workspace between builds but set up your system to load all sources from your vcs. This way you will always be able to make a fresh build from source, there are also a few options available to clear the workspace from old files.
If you do not want to populate the workspace from a source code addon you can always use the custom shell script feature to run a few shell commands to copy the needed files.
I'm already finishing my project build automation :) with Hudson and Nant.
My project structure is something like
$/Project
build.scripts
script1.build
script2.build
build.properties.xml
Code
Project1
Project2
So Hudson downloads from the root $/Project to the workspace folder.
And everything is ok since the build.scripts are in the workspace, I run them very easily, however what is bugging me is the fact that since the build scripts are inside the workspace, then I can't program Hudson to run automatically either based on time or changes because it will always detect changes to the files (note build.properties.xml which I check out and check in at build time to store some stats).
Where do you recommend these files to go in and still get the advantage of having them source-controlled?
What I ended up doing is to NOT check-in changes to those files. I changed my CI workflow to create another file (local to the workspace only) where the changes are written to.
This way, I still get the last build info written somewhere to pick it up, and avoid the issue of Jenkins detecting the change.
PS: I changed from Hudson to Jenkins since I saw that most plugins ran away from the former. The transition was too smooth to be true.