Is there a way to recover builds that were shift+deleted in Jenkins?
I saw the other thread but I think it was only for delete and not for shift+delete
Please help. I lost a few builds from my production job
Even if the build was deleted from disk, you should have the build log and you can get the Changelist at which the build was made and rebuild it.
Assuming you're running Jenkins on Windows and you performed the Shift+Delete on the job folder itself in Windows Explorer:
No.* Jenkins stores all job data in that directory and Windows deletes the files immediately using that method. You will need to restore the jobs from backup if you have them.
*Note: Unless you use a file system level recovery tool, which is out of scope for this exchange. You could ask over on Superuser.
Related
I've recently started to encounter a problem with a new build agent that I have added to my TFS agent pool. The agent runs my build the first time without any problem. However, all subsequent builds fail with this error "##[error]The directory is not empty". This occurs on the initial startup of the build when it is trying to download files from TFS.
Keep in mind that I have set the "clean" option to true for the build and also set the Build.Clean variable to "all"
I've done searches online for this error and most of the info I am finding states that the directory is in-use and that is why it cannot be deleted. The strange thing is that I can manually delete the folder using Windows Explorer and there is no error reported of files in-use. Once I do that, the build will work again, but only on the first run. Why is it that the TFS vNext build cannot delete this folder? Is there a log that I reference that provides more details other than "directory is not empty"?
You could set system.debug=true to enable verbose Debug Mode for TFS vNext Build.
In addition, you could also check the agent log under agent path\_diag path here.
If there are more information for troubleshooting.
Back to your issue, please try to stop you build agent service and restart again. Also update your build agent to latest version.
Besides, you could also choose another driver such as D:\ or E:\ if you are using c:\agent\ which may do the trick.
I am currently running Jenkins ver. 2.121.2. I am in charge to migrate all jenkins builds and workspace to a new location since we have many projects filling our system's SSD. And we decided to have the workspace and build location to be on a HDD. Migrating all the data wasn't hard but the unexpected side effect is, that all jenkins jobs have lost their build history. I thought it would be as simple as copying all files over to the new location.
I changed only 2 parameters in the /var/lib/jenkins/config.xml file. These are from:
<workspaceDir>${ITEM_ROOTDIR}/workspace</workspaceDir>
<buildsDir>${ITEM_ROOTDIR}/builds</buildsDir>
To...
<workspaceDir>/wd-red1/jenkins_workspace/${ITEM_FULLNAME}</workspaceDir>
<buildsDir>/wd-red1/jenkins_builds/${ITEM_FULL_NAME}</buildsDir>
Then I have copied all files from /var/lib/jenkins/workspace and /var/lib/jenkins/jobs respectively to the locations above. All went well except for the job build histories.
My question is: Can I somehow import the job build history too?
If your old jenkins instance is still running, install "thinbackup" and take a full backup. Copying the backup into the new server and restoring the backup will give you all the configurations as it is in old server. I tried this someday back and it worked liked a charm.
Note : Thinbackup plug in supports both backup and restore.
I have a Jenkins setup running in production, I want to automate jenkins setup(installation) along with all the jobs that are setup in jenkins.
One crude way I can think of is to copy the whole jobs directory to the new Jenkins setup.
I want to know how other people in industry do deal with this problem.
I have used the plugin Thinbackup to move jobs, users, and plugins. You can make a full backup and restore it to the new server. The plugin is not perfect and is up for adoption. I had issues with the restore. I ended up using the plugin only for creating the archive, but then I copied manually the folders (users, jobs, plugins, nodes, email-templates, secrets, JENKINS_HOME files) from the archive to the new server.
Before creating the archive or copying the jobs, ensure that no more than 30 builds per job are kept, this will keep your archive small. I have seen 5000+ builds per job, which were totally unnecessary and were blocking the creation of the archive.
When you create or restore the archive, or copy files, the server should be in quiet mode, no builds should be executed.
http://<jenkins.server>/quietDown
After you copy the files or restore the archive, you should restart Jenkins or even better, restart the server.
Another option is to use RSync as mentioned here. I am not sure what is the OS of your Jenkins server. If it is Linux you can check out this guide that I have written.
I am trying to automate Jenkins Upgrades so they do not have to be hands on. Some documentation recommends creating a batch file with instructions on the machine running Jenkins, and create a scheduled task to run the batch job. The site I found with a batch file is here, where it says:
It does delete the complete exploded war file from the deployment location, so be careful if you save any configuration files to that directory.
What configuration file would I have to worry about? No one I've talked to at my company knows of any configuration files held there, and they seem to think we have a pretty default setup, so what could I look for manually that would tell me whether or not I should be concerned?
We are running Jenkins on a Windows virtual box, I believe with Jenkins running as a service.
Alternatively, if the above method is not the easiest or best way to automate Jenkins upgrades, does anyone know a better way?
You can ignore this warning. I've never seen anything storing configuration files in that directory. It is intended to be used as a cache only.
If unsure, check your existing war directory for any files with timestamps newer than the installation time.
Here, on a busy Jenkins master, no files have been added or modified there over a period of several months (since initial war file explosion at installation time).
I have a Jenkins server hosted, which has a master node and couple of other slave configurations. Last night, the job that triggers the matrix based build configuration failed. I did a restart and performed clean up jobs via Jenkins but none of those fixed the issue. The initial error that was logged was:
FATAL: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
Following which I performed a reload configuration from disk,followed by a manual restart via <jenkins_job_url>/restart, which even worsened the build system. The master went offline due to unavailability of space in /tmp folder, which I fixed by cleaning up the space. Following which I observed that the original slave server configuration is no longer seen. I had slave-0 and slave-1 still there, but slave-2 was no longer present. Instead, it got replaced with slave-3 configuration. Now the slave 0's and 1's seems to be working fine. However, slave 3's build are failing due to Failed to mkdirs. Is there a way I could revert back to the original configuration from where I started, since the steps I performed seems to make sense initially, but I had no idea it had so many repercussions? Any help is appreciated.
UPDATE1: I guess I should have used some of the configuration backup plugins available in Jenkins, but is there some specific directory other than $JENKINS_HOME where these configurations gets stored?
You should always backup ${JENKINS_HOME} before doing major changes.
Even better is to have a job based on time trigger that will do this for you once in a while.
Other than that - only physically restoring the hard drive to a previous state will get back your old configs. Once a config is overwritten in Jenkins - it is gone. Except when you are using Job Config History plugin. Though keeping manually created backups is better in my opinion: where's the insurance that JobConfigHistory won't disappear along with the job configs? :)
Aside from that, the mentioned plugin tracks system config too.
As mentioned by #Zloj, there is no easy way to repair once the changes gets overwritten. I ended up fixing the issues by deleting the slaves that were not working, remapping the existing builds to the newer slaves that I created via Copy of the existing slaves that were working, reducing the number of builds(by removing the ones from the matrix that aren't required) and finally, taking a backup via https://wiki.jenkins-ci.org/display/JENKINS/thinBackup plugin and backing up the configuration at Stash :)
for windows just delete .jenkins folder in your home directory. This will revert you to the original settings.
We have been using SCM Sync Configuration plugin and that has saved our butt many times. It stores all job configuration including global config in bitbucket. But the latest plugin will say that its no longer maintained. but I was able to pull the source code from github and rebuild it ourselves.
one word of caution...don't use global variables for storing passwords and keys...this plugin will sync them all to github. Strictly use Jenkins Credentials.