I want to take scheduled backup of Jenkins jobs form Jenkins server to some remote machine. I tried exploring several Jenkins plugins for that but none of that are not able to take backup to the remote machine.
I successfully take backup of the Jenkins workspace using the shell script but in that case it's hard to restore the backup.
Any Suggestions?
If I can suggest a different approach - if you're using any kind of source control it'll be better to backup your files and configuration there. for example - if you will work with git you can open a repository for your Jenkins configuration.
Back up your:
jobs folder
nodes folder
parent folder files (config.xml, all plugins configurations, etc.)
then it is only a matter of running a scheduled job from Jenkins every 12 hours running:
cd $JENKINS_HOME
git add --all
git commit -m "Automated backup commit: $BUILD_TIMESTAMP"
git push
* Make sure you have the right permissions to run those command on the master
This will enable you to:
Keep backups for you Jenkins configuration
Manage versions of backups
View history of changes you made to your configurations
Hope it helps.
Related
I am new to Jenkins. I want to build a jenkins job which replaces files in SVN repository and commits them. The SVN repository has folders and files in it which we replace and commit manually right now to SVN. Using the Jenkins job itself, I can place the files on a server, not sure if that will be helpful for Jenkins to replace and commit those files to SVN.
Is this possible with Jenkins? (using Jenkins Enterprise) I have mostly seen posts and questions about triggering build after commit in SVN using Jenkins. So, not sure if this is even possible.
Jenkins allows you to execute shell commands in a pipline. So you can execute the same shell commands as you would on your local machine:
sh 'svn commit -m "my message"'
or if you are on a windows node:
powershell/cmd 'svn commit -m "my message"'
Personally I have no experience with svn, but this approach works well with git and I strongly assume it also works with svn.
I want to know how to configure Jenkins with my live preprod instance server ?
Let me to explain you my process and tell me if I'm right and if that's not the good way to do.
1) I have my project project-1 in a server: /var/www/preprod/project-1, this project is in Magento Cms so it contains many files.
2) I copied this project project-1 in a repo Git, repo = project-1.
3) I cloned this project from this repo Git to my local machine: MAMP/htdocs/project-1.
4) I installed Jenkins, and I configured it with git, So when I do some push, Jenkins do a build automatically.
Now what I want to do is after the build, I want Jenkins to upload these changes to my live preprod server, whether automatically and manually.(I want to know the method to do it manually and automatically).
With this method, I develop in my local server, so when I finish some task and it's done, I push it to Git to have the changes history, and after that my need is to push it to the live server.
So tell me please if I'm using the right method, if it's a good practice and what I miss for this continuous deployment & delivery.
You can push it to the server using the Publish over SSH command if your doing a freestyle job https://wiki.jenkins.io/display/JENKINS/Publish+Over+SSH+Plugin, If you are doing an pipeline then you can do a simple scp command...
you can run this after the build is completed and it will run it automatically for you ...
Manually you will be notified when a build is done then you would copy it into your server using the normal way you would do it i.e.copy and paste...
Jenkins is a automation server. The whole point of using jenkins is to automate things so that you "manual" intervention is not required. So automate it where ever possible.
Hope it helps :)
I'm trying to a run a simple windows batch command (say copy) on Master inside a job that's set to run on a particular slave.
What i'm trying to accomplish with this is that copy the build log that gets saved on a master to a shared drive that's accessible from master. Please advise.
You are going to have to make the jenkins file system visible on the client independently of Jenkins. Since you have a Windows client, you are probably going to have to setup sharing from the Jenkins master using samba or something of the sort.
What I do instead: When I need assets from master is I use curl or wget to download the assets to the clients. You can use the FSTrigger plugin to start builds when the file changes on the Jenkins master. Once curl or wget has run, your asset is then in the %WORKSPACE% directory and you can proceed.
I would recommend to handle the logfile copying (and maybe further tasks) as a dedicated job (let's call it "SaveLog"). SaveLog should be tied to run only on master.
You should then configure SaveLog to be triggered after completion of your primary job.
The Logfile is already available on master, even if you do not save any artifacts.
Should you need further files from the slave workspace, then you should save them as artifacts. SaveLog (on master) can then still decide whether to do anything useful with those artifacts.
I'm running jenkins on my local machine. I need to build and run a project on remote machine as a slave.
I have my project(Java code) in my local machine where the Jenkins is running. So Now when I build the project it looks for the code in my slave machine and says it's not able to find the build.xml in my slave machine.
I want to be able to have Jenkins looks for the code in my local machine and build it in my slave machine and run in the slave machine.
Is that possible? if yes pls help me.
You do not state which version control system you are using? Jenkins has version control plugins for Git, Subversion, etc...
The idea is that you commit and push your changes to your source code repository and the Jenkins slave will automatically pull down the code and compile if anything has changed.
Try the copy artifact plugin. You can create 2 jobs - 1st one to just check out the source code from your repo/version control/workspace. The second job (which runs on the slave restrict where this job can be run inside the job config) can copy those files automatically to the ...remote slave/workspace and then you do whatever you want with them.
This will do what you want, but as Mark O'Çonnor mentions, the idea is you commit the changes to your repository and then the other machine checks it out.
I am currently using Jenkins on my development PC. I installed it on my development PC, because I had limited knowledge on this tool; so I tested on it in my development PC. Now, I feel comfortable with Jenkins as my long term "partner" in the build process and would like to "move" this Jenkins to a dedicated server.
Before this I have done few builds and have the artifacts archived from each build. In particular, the build number is very important to me for version control.
How can I export all the Jenkins information from my current PC to my new server?
Following the Jenkins wiki, you'll have to:
Install a fresh Jenkins instance on the new server
Be sure the old and the new Jenkins instances are stopped
Archive all the content of the JENKINS_HOME of the old Jenkins instance
Extract the archive into the new JENKINS_HOME directory
Do not forget to change the owner of the new Jenkins files : chown -R jenkins:jenkins $JENKINS_HOME
Launch the new Jenkins instance
Do not forget to change documentation/links to your new instance of Jenkins :)
JENKINS_HOME is by default located in ~/.jenkins on a Linux installation, yet to exactly find where it is located, go on the http://your_jenkins_url/configure page and check the value of the first parameter: Home directory; this is the JENKINS_HOME.
In case your JENKINS_HOME directory is too large to copy, and all you need is to set up same jobs, Jenkins Plugins and Jenkins configurations (and don't need old Job artifacts and reports), then you can use the ThinBackup Plugin:
Install ThinBackup on both the source and the target Jenkins servers
Configure the backup directory on both (in Manage Jenkins → ThinBackup → Settings)
On the source Jenkins, go to ThinBackup → Backup Now
Copy from Jenkins source backup directory to the Jenkins target backup directory
On the target Jenkins, go to ThinBackup → Restore, and then restart the Jenkins service.
If some plugins or jobs are missing, copy the backup content directly to the target JENKINS_HOME.
If you had user authentication on the source Jenkins, and now locked out on the target Jenkins, then edit Jenkins config.xml, set <useSecurity> to false, and restart Jenkins.
This worked for me to move from Ubuntu 12.04 (Jenkins ver. 1.628) to Ubuntu 16.04 (Jenkins ver. 1.651.2). I first installed Jenkins from the repositories.
Stop both Jenkins servers
Copy JENKINS_HOME (e.g. /var/lib/jenkins) from the old server to the new one. From a console in the new server:
rsync -av username#old-server-IP:/var/lib/jenkins/ /var/lib/jenkins/
Start your new Jenkins server
You might not need this, but I had to
Manage Jenkins and Reload Configuration from Disk.
Disconnect and connect all the nodes again.
Check that in the Configure System > Jenkins Location, the Jenkins URL is correctly assigned to the new Jenkins server.
Jenkins Server Automation:
Step 1:
Set up a repository to store the Jenkins home (jobs, configurations, plugins, etc.) in a GitLab local or on GitHub private repository and keep it updated regularly by pushing any new changes to Jenkins jobs, plugins, etc.
Step 2:
Configure a Puppet host-group/role for Jenkins that can be used to spin up new Jenkins servers. Do all the basic configuration in a Puppet recipe and make sure it installs the latest version of Jenkins and sets up a separate directory/mount for JENKINS_HOME.
Step 3:
Spin up a new machine using the Jenkins-puppet configuration above. When everything is installed, grab/clone the Jenkins configuration from the Git repository to the Jenkins home direcotry and restart Jenkins.
Step 4:
Go to the Jenkins URL, Manage Jenkins → Manage Plugins and update all the plugins that require an update.
Done
You can use Docker Swarm or Kubernetes to auto-scale the slave nodes.
Sometimes we may not have access to a Jenkins machine to copy a folder directly into another Jenkins instance. So I wrote a menu driven utility which uses Jenkins REST API calls to install plugins and jobs from one Jenkins instance to another.
For plugin migration:
GET request: {SOURCE_JENKINS_SERVER}/pluginManager/api/json?depth=1 will get you the list of plugins installed with their version.
You can send a POST request with the following parameters to install these plugins.
final_url=`{DESTINATION_JENKINS_SERVER}/pluginManager/installNecessaryPlugins`
data=`<jenkins><install plugin="{PLUGIN_NAME}#latest"/></jenkins>` (where, latest will fetch the latest version of the plugin_name)
auth=`(destination_jenkins_username, destination_jenkins_password)`
header=`{crumb_field:crumb_value,"Content-Type":"application/xml”}` (where crumb_field=Jenkins-Crumb and get crumb value using API call {DESTINATION_JENKINS_SERVER}/crumbIssuer/api/json
For job migration:
You can get the list of jobs installed on {SOURCE_JENKINS_URL} using a REST call, {SOURCE_JENKINS_URL}/view/All/api/json
Then you can get each job config.xml file from the jobs on {SOURCE_JENKINS_URL} using the job URL {SOURCE_JENKINS_URL}/job/{JOB_NAME}.
Use this config.xml file to POST the content of the XML file on {DESTINATION_JENKINS_URL} and that will create a job on {DESTINATION_JENKINS_URL}.
I have created a menu-driven utility in Python which asks the user to start plugin or Jenkins migration and uses Jenkins REST API calls to do it.
You can refer the JenkinsMigration.docx from this URL
jenkinsjenkinsmigrationjenkinsrestapi
Let us say we are migrating Jenkins LTS from PC1 to PC2 (irrispective of LTS version is same of upgraded).
It is easy to use ThinBackUp Plugin for migration or Upgrade of Jenkins version.
Step1: Prepare PC1 for migration
Manage Jenkins -> ThinbackUp -> Setting
Select correct options and directory for backup
If you need a job history and artifacts need to be added then please select 'Back build results' option as well.
Go back click on Backup Now.
Note: This Thinbackup will also take Plugin Backup which is optional.
Check the ThinbackUp folder must have a folder with current date and timestamp.
(wait for couple of minutes it might take some time.)
You are ready with your back, .zip it and copy to PARTICULAR (which will be 'Backup directory') directory in PC2.
Unzip ThinbackUp zipped folder.
Stop Jenkins Service in PC1.
Step2: Install Jenkins (Install using .war file or Paste archived version) in PC2.
Create Jenkins Service using command sc create <Jenkins_PC2Servicename> binPath="<Path_to_Jenkinsexe>/jenkins.exe"
Modify JENKINS_HOME/jenkins.xml if needed in PC2.
Run windows service <Jenkins_PC2Servicename> in PC2
Manage Jenkins -> ThinbackUp -> Setting
Make sure that you PARTICULAR path from step1 as Backup Directory in ThinBackup settings.
ThinbackUp -> Restore will give you a Dropdown list, choose a right backup (identify with date and timestamp).
Wait for some minutes and you have latest backup configurations including jobs history and plugins in PC2.
In case if there are additional changes needed in JENKINS_HOME/Jenkins.xml (coming from PC1 ThinbackUp which is not needed) then this modification need to do manually.
NOTE: If you are using Database setting of SCM in your Jenkins jobs then you need to take extra care as all SCM plugins do not support to carry Database settings with the help of ThinbackUp plugin.
e.g. If you are using PTC Integrity SCM Plugin, and some Jenkins jobs are using DB using Integrity, then it will create a directory JENKINS_Home/IntegritySCM, ThinbackUp will not include this DB while taking backup.
Solution: Directly Copy this JENKINS_Home/IntegritySCM folder from PC1 to PC2.