Jenkins's home directory - jenkins

My Jenkins is installed on the default location: /var/lib/jenkins. Every time it builds, it changes my root directory of my workspace (on my local machine /home/john/p4) to /var/lib/jenkins/..., which shouldn't happen?
How to specify my root directory of my client(workspace) so that the build won't change its location? Should I change $JENKINS_HOME? If I should change it, then that's equavilent to the fact that I have to re-install Jenkins to the location I want, because $JENKINS_HOME is supposed to be the root directory for all jenkins files and builds.
What should be the correct behavior of Jenkins and P4 client? Also, does it have anything to do with the user who starts the builds in Jenkins? Does Jenkins user have anything to do with the Linux user who installs Jenkins?

The p4-plugin Jenkins requires it's own Perforce workspace and it WILL set the Perforce workspace root to match the Jenkins workspace root.
Let Jenkins create a new Perforce workspace (use a name that does not exist, I generally prefix it with jenkins-). If you want to be dynamic use a name like:
jenkins-${NODE_NAME}-${JOB_NAME}
...as ${NODE_NAME} and ${JOB_NAME} will expand.
Next define a view mapping (or streams path) to specify the location of the files in Perforce and how you want them to appear in the workspace. e.g.:
View:
//depot/myProj/main/... //jenkins-${NODE_NAME}-${JOB_NAME}/...
As for the user that connects to Perforce, that is defined in the Perforce Credentials, but the files sync'ed to the Jenkins Master (or Slave if you have a build farm) will use the UID/GID of the Jenkins service.
You can find the documentation for the p4-plugin here.

Related

How to use workspace from local machine on Jenkins instance

I have workspace in my local, I want to use it as workspace in jenkins instance. How do I mention that path location which is present in local? How do I make all the execution to run locally and use Jenkins just to trigger the jobs?
Connect your machine as an agent to Jenkins. Here is a good wiki. And in Remote root directory specify your workspace.

How to control the workspace Jenkins starts the build on?

I have one Jenkins master node and 2 Jenkins slave nodes. All my job builds happen in the slave node. When I configured my slaves, I set the Remote root directory as /data/home/jenkins/jenkins-slave. Also, I give the custom workspace option as DEVELOP_BRANCH in the job configuration page of the respective job.
However, at the start of job, I get the following log information:
Building remotely on linux in workspace /data/home/jenkins/jenkins-slave/workspace/DEVELOP_BRANCH
I want to start my builds in this location.
/data/home/jenkins/jenkins-slave/DEVELOP_BRANCH
Why does the extra workspace directory come into the picture? How do I remove it? I do not have access to Jenkins master node. So, if there is a workaround that can match my requirements, it would be awesome.
Note: By node, I refer to a Linux OS computer with redHat distribution.
In project configuration, under Advanced Project Options, you can check Use custom workspace and put a path there.
If you put an absolute path, it will be used without any extra workspace/ directory. (at least that's the behavior I can see on a windows server.)

Perforce plugin in jenkins creates a workspace

I have Perforce managing our source. I have an application that uses Perforce as the back end. I have also setup an automated test tool that runs my application and performs automated tests. I want Jenkins to trigger the test every time there is a change in the source code. However, my Jenkins instance messes up the workspace root. It creates its own workspace root and that causes my application to fail. Jenkins actually overwrites the Perforce clients workspace root. So every time I try to get Jenkins work, I have to go and edit the workspace root in Perforce and reset it to the required value. I have tried getting Jenkins to manage the workspace and clearing the option to do the same but have failed. Is there anyway that Jenkins will use my workspace (root) settings and not change it?
Jenkins 'owns' the Perforce workspace used for the build, hence it sets the root.
Your application should ideally build and run independently of its location. However, there is the 'Advanced Project Options' --> 'Use custom workspace' configuration option in Jenkins.
Jenkins must have it's own Perforce workspace separate to your own development workspace. Use the Template workspace to mirror the options or create a Manual workspace for Jenkins own use.
Please note, there are two Perforce plugins for Jenkins p4 and perforce, documentation for the p4 plugin is located here/

Change SVN URL repository in Jenkins

Our SVN repository has moved to a new location. I need to reconfigure the jenkins so that it points to the new location and start building from there. Jenkins runs in a remote UNIX server.
Is it sufficient if I change the SVN URL in the source code management section in the jenkins UI page? or do I need to grep for the SVN url in the jenkins directory by logging into the unix server and start replacig every occurence of it ?
It is sufficient to change the SVN URL in the job configuration. Jenkins will remove the existing checkout and start anew. If the job has fixed slave, you can delete the workspace manually in the slave to fasten the process.

How to move Jenkins from one PC to another

I am currently using Jenkins on my development PC. I installed it on my development PC, because I had limited knowledge on this tool; so I tested on it in my development PC. Now, I feel comfortable with Jenkins as my long term "partner" in the build process and would like to "move" this Jenkins to a dedicated server.
Before this I have done few builds and have the artifacts archived from each build. In particular, the build number is very important to me for version control.
How can I export all the Jenkins information from my current PC to my new server?
Following the Jenkins wiki, you'll have to:
Install a fresh Jenkins instance on the new server
Be sure the old and the new Jenkins instances are stopped
Archive all the content of the JENKINS_HOME of the old Jenkins instance
Extract the archive into the new JENKINS_HOME directory
Do not forget to change the owner of the new Jenkins files : chown -R jenkins:jenkins $JENKINS_HOME
Launch the new Jenkins instance
Do not forget to change documentation/links to your new instance of Jenkins :)
JENKINS_HOME is by default located in ~/.jenkins on a Linux installation, yet to exactly find where it is located, go on the http://your_jenkins_url/configure page and check the value of the first parameter: Home directory; this is the JENKINS_HOME.
In case your JENKINS_HOME directory is too large to copy, and all you need is to set up same jobs, Jenkins Plugins and Jenkins configurations (and don't need old Job artifacts and reports), then you can use the ThinBackup Plugin:
Install ThinBackup on both the source and the target Jenkins servers
Configure the backup directory on both (in Manage Jenkins → ThinBackup → Settings)
On the source Jenkins, go to ThinBackup → Backup Now
Copy from Jenkins source backup directory to the Jenkins target backup directory
On the target Jenkins, go to ThinBackup → Restore, and then restart the Jenkins service.
If some plugins or jobs are missing, copy the backup content directly to the target JENKINS_HOME.
If you had user authentication on the source Jenkins, and now locked out on the target Jenkins, then edit Jenkins config.xml, set <useSecurity> to false, and restart Jenkins.
This worked for me to move from Ubuntu 12.04 (Jenkins ver. 1.628) to Ubuntu 16.04 (Jenkins ver. 1.651.2). I first installed Jenkins from the repositories.
Stop both Jenkins servers
Copy JENKINS_HOME (e.g. /var/lib/jenkins) from the old server to the new one. From a console in the new server:
rsync -av username#old-server-IP:/var/lib/jenkins/ /var/lib/jenkins/
Start your new Jenkins server
You might not need this, but I had to
Manage Jenkins and Reload Configuration from Disk.
Disconnect and connect all the nodes again.
Check that in the Configure System > Jenkins Location, the Jenkins URL is correctly assigned to the new Jenkins server.
Jenkins Server Automation:
Step 1:
Set up a repository to store the Jenkins home (jobs, configurations, plugins, etc.) in a GitLab local or on GitHub private repository and keep it updated regularly by pushing any new changes to Jenkins jobs, plugins, etc.
Step 2:
Configure a Puppet host-group/role for Jenkins that can be used to spin up new Jenkins servers. Do all the basic configuration in a Puppet recipe and make sure it installs the latest version of Jenkins and sets up a separate directory/mount for JENKINS_HOME.
Step 3:
Spin up a new machine using the Jenkins-puppet configuration above. When everything is installed, grab/clone the Jenkins configuration from the Git repository to the Jenkins home direcotry and restart Jenkins.
Step 4:
Go to the Jenkins URL, Manage Jenkins → Manage Plugins and update all the plugins that require an update.
Done
You can use Docker Swarm or Kubernetes to auto-scale the slave nodes.
Sometimes we may not have access to a Jenkins machine to copy a folder directly into another Jenkins instance. So I wrote a menu driven utility which uses Jenkins REST API calls to install plugins and jobs from one Jenkins instance to another.
For plugin migration:
GET request: {SOURCE_JENKINS_SERVER}/pluginManager/api/json?depth=1 will get you the list of plugins installed with their version.
You can send a POST request with the following parameters to install these plugins.
final_url=`{DESTINATION_JENKINS_SERVER}/pluginManager/installNecessaryPlugins`
data=`<jenkins><install plugin="{PLUGIN_NAME}#latest"/></jenkins>` (where, latest will fetch the latest version of the plugin_name)
auth=`(destination_jenkins_username, destination_jenkins_password)`
header=`{crumb_field:crumb_value,"Content-Type":"application/xml”}` (where crumb_field=Jenkins-Crumb and get crumb value using API call {DESTINATION_JENKINS_SERVER}/crumbIssuer/api/json
For job migration:
You can get the list of jobs installed on {SOURCE_JENKINS_URL} using a REST call, {SOURCE_JENKINS_URL}/view/All/api/json
Then you can get each job config.xml file from the jobs on {SOURCE_JENKINS_URL} using the job URL {SOURCE_JENKINS_URL}/job/{JOB_NAME}.
Use this config.xml file to POST the content of the XML file on {DESTINATION_JENKINS_URL} and that will create a job on {DESTINATION_JENKINS_URL}.
I have created a menu-driven utility in Python which asks the user to start plugin or Jenkins migration and uses Jenkins REST API calls to do it.
You can refer the JenkinsMigration.docx from this URL
jenkinsjenkinsmigrationjenkinsrestapi
Let us say we are migrating Jenkins LTS from PC1 to PC2 (irrispective of LTS version is same of upgraded).
It is easy to use ThinBackUp Plugin for migration or Upgrade of Jenkins version.
Step1: Prepare PC1 for migration
Manage Jenkins -> ThinbackUp -> Setting
Select correct options and directory for backup
If you need a job history and artifacts need to be added then please select 'Back build results' option as well.
Go back click on Backup Now.
Note: This Thinbackup will also take Plugin Backup which is optional.
Check the ThinbackUp folder must have a folder with current date and timestamp.
(wait for couple of minutes it might take some time.)
You are ready with your back, .zip it and copy to PARTICULAR (which will be 'Backup directory') directory in PC2.
Unzip ThinbackUp zipped folder.
Stop Jenkins Service in PC1.
Step2: Install Jenkins (Install using .war file or Paste archived version) in PC2.
Create Jenkins Service using command sc create <Jenkins_PC2Servicename> binPath="<Path_to_Jenkinsexe>/jenkins.exe"
Modify JENKINS_HOME/jenkins.xml if needed in PC2.
Run windows service <Jenkins_PC2Servicename> in PC2
Manage Jenkins -> ThinbackUp -> Setting
Make sure that you PARTICULAR path from step1 as Backup Directory in ThinBackup settings.
ThinbackUp -> Restore will give you a Dropdown list, choose a right backup (identify with date and timestamp).
Wait for some minutes and you have latest backup configurations including jobs history and plugins in PC2.
In case if there are additional changes needed in JENKINS_HOME/Jenkins.xml (coming from PC1 ThinbackUp which is not needed) then this modification need to do manually.
NOTE: If you are using Database setting of SCM in your Jenkins jobs then you need to take extra care as all SCM plugins do not support to carry Database settings with the help of ThinbackUp plugin.
e.g. If you are using PTC Integrity SCM Plugin, and some Jenkins jobs are using DB using Integrity, then it will create a directory JENKINS_Home/IntegritySCM, ThinbackUp will not include this DB while taking backup.
Solution: Directly Copy this JENKINS_Home/IntegritySCM folder from PC1 to PC2.

Resources