I want to do an automatically backup of the JENKINS_HOME but due to file size I just want to copy the config.xml and /jobs.
Is it possible to restore Jenkins if needed with just those two things? How is the process to do it? Just replacing the config.xml and /jobs and restarting the service?
Thanks.
Consider using the Jenkins SCM Sync configuration plugin - it allows to back up all your Jenkins settings (including job configurations) to git or SVN. Also, version control means you'll have an easier time debugging problems caused by configuration changes or to revert to an older configuration.
Related
I have a Jenkins setup running in production, I want to automate jenkins setup(installation) along with all the jobs that are setup in jenkins.
One crude way I can think of is to copy the whole jobs directory to the new Jenkins setup.
I want to know how other people in industry do deal with this problem.
I have used the plugin Thinbackup to move jobs, users, and plugins. You can make a full backup and restore it to the new server. The plugin is not perfect and is up for adoption. I had issues with the restore. I ended up using the plugin only for creating the archive, but then I copied manually the folders (users, jobs, plugins, nodes, email-templates, secrets, JENKINS_HOME files) from the archive to the new server.
Before creating the archive or copying the jobs, ensure that no more than 30 builds per job are kept, this will keep your archive small. I have seen 5000+ builds per job, which were totally unnecessary and were blocking the creation of the archive.
When you create or restore the archive, or copy files, the server should be in quiet mode, no builds should be executed.
http://<jenkins.server>/quietDown
After you copy the files or restore the archive, you should restart Jenkins or even better, restart the server.
Another option is to use RSync as mentioned here. I am not sure what is the OS of your Jenkins server. If it is Linux you can check out this guide that I have written.
If I need to backup jenkins configurations, do you have good recommendations? I'm researching and found https://wiki.jenkins.io/display/JENKINS/Backup+Plugin but I want to be able to automate the whole process
The way I like to do it is as a git repo. Init the repo in the JENKINS_HOME directory. Add a .gitignore file to eliminate the workspace directory(s). Optionally, you can ignore the log and/or logs directories. The plugins directory can probably be rebuilt entirely and doesn't need to be backed up, but it might make sense to dump a directory listing of the plugins directory to a file, or better yet, hit this url and dump that content to a file before you commit changes:
http://jenkinsurl/pluginManager/api/json?depth=1&pretty=true.
You can also ignore the caches directory and the war directory.
Setup a cron job, or a job in Jenkins to commit the changes the git repo on a regular basis. Even put a Jenkinsfile in the root of JENKINS_HOME and use that as your pipeline job to backup itself.
You can use Gearman plugin as an alternative, it supports backup as well as failover in the evnt of an outage.
I have a Jenkins server hosted, which has a master node and couple of other slave configurations. Last night, the job that triggers the matrix based build configuration failed. I did a restart and performed clean up jobs via Jenkins but none of those fixed the issue. The initial error that was logged was:
FATAL: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
Following which I performed a reload configuration from disk,followed by a manual restart via <jenkins_job_url>/restart, which even worsened the build system. The master went offline due to unavailability of space in /tmp folder, which I fixed by cleaning up the space. Following which I observed that the original slave server configuration is no longer seen. I had slave-0 and slave-1 still there, but slave-2 was no longer present. Instead, it got replaced with slave-3 configuration. Now the slave 0's and 1's seems to be working fine. However, slave 3's build are failing due to Failed to mkdirs. Is there a way I could revert back to the original configuration from where I started, since the steps I performed seems to make sense initially, but I had no idea it had so many repercussions? Any help is appreciated.
UPDATE1: I guess I should have used some of the configuration backup plugins available in Jenkins, but is there some specific directory other than $JENKINS_HOME where these configurations gets stored?
You should always backup ${JENKINS_HOME} before doing major changes.
Even better is to have a job based on time trigger that will do this for you once in a while.
Other than that - only physically restoring the hard drive to a previous state will get back your old configs. Once a config is overwritten in Jenkins - it is gone. Except when you are using Job Config History plugin. Though keeping manually created backups is better in my opinion: where's the insurance that JobConfigHistory won't disappear along with the job configs? :)
Aside from that, the mentioned plugin tracks system config too.
As mentioned by #Zloj, there is no easy way to repair once the changes gets overwritten. I ended up fixing the issues by deleting the slaves that were not working, remapping the existing builds to the newer slaves that I created via Copy of the existing slaves that were working, reducing the number of builds(by removing the ones from the matrix that aren't required) and finally, taking a backup via https://wiki.jenkins-ci.org/display/JENKINS/thinBackup plugin and backing up the configuration at Stash :)
for windows just delete .jenkins folder in your home directory. This will revert you to the original settings.
We have been using SCM Sync Configuration plugin and that has saved our butt many times. It stores all job configuration including global config in bitbucket. But the latest plugin will say that its no longer maintained. but I was able to pull the source code from github and rebuild it ourselves.
one word of caution...don't use global variables for storing passwords and keys...this plugin will sync them all to github. Strictly use Jenkins Credentials.
Is there a way to recover builds that were shift+deleted in Jenkins?
I saw the other thread but I think it was only for delete and not for shift+delete
Please help. I lost a few builds from my production job
Even if the build was deleted from disk, you should have the build log and you can get the Changelist at which the build was made and rebuild it.
Assuming you're running Jenkins on Windows and you performed the Shift+Delete on the job folder itself in Windows Explorer:
No.* Jenkins stores all job data in that directory and Windows deletes the files immediately using that method. You will need to restore the jobs from backup if you have them.
*Note: Unless you use a file system level recovery tool, which is out of scope for this exchange. You could ask over on Superuser.
We are trying to troubleshoot why our job config page is not loading.
If we uninstall each plugin as we troubleshoot, would Hudson or Jenkins modify the job config files that had previously used those installed plugins
and remove those plugins tags just because we uninstalled them temporarily ?
We ask because we like to know if we need to make job config backups as we do not want to lose the old installed plugin tags used in our jobs.
Thank you
Sincerely
Yes, you should be doing job config backups! Yes, you probably should rsync your job config data and jenkins directories sans workspaces to a different system for debugging.
No, when you uninstall a plugin, the data that is associated with that plugin is generally left behind, unless you modify a job afterwards.