I'm new to Jenkins, and I like to know if it is possible to have one Jenkins server to deploy / update code on multiple web servers.
Currently, I have two web servers, which are using python Fabric for deployment.
Any good tutorials, will be greatly welcomed.
One solution could be to declare your web servers as slave nodes.
First thing, give jenkins credentials to your servers (login/password or ssh login+private key or certificate. This can be configured in the "Manage credentials" menu
Then configure the slave nodes. Read the doc
Then, create a multi-configuration job. First you have to install the matrix-project plugin. This will allow you to send the same deployment intructions to both your servers at once
Since you are already using Fabic for deployment, I would suggest installing Fabric on the Jenkins master and have Jenkins kick off the Fabric commands to deploy to the remote servers. You could set up the hostnames or IPs of the remote servers as parameters to the build and just have shell commands that iterate over them and run the Fabric commands. You can take this a step further and have the same job deploy to dev/test/prod just by using a different set of hosts.
I would not make the webservers slave nodes. Reserve slave nodes for build jobs. For example, if you need to build a windows application, you will need a windows Jenkins slave. IF you have a problem with installing Fabric on your Jenkins master, you could create a slave node that is responsible for running Fabric deploys and force anything that runs a fabric command to use that slave. I feel like this is overly complex but if you have a ton of builds on your master, you might want to go this route.
Related
I would like to set up jenkins server that would run test scripts based on successful build deployments on other Jenkins servers. for example, if the QA jenkins server is named JQA1OnMachine1 and i have three others that are named
J2OnMachine2, J3OnMachine3, J4OnMachine4 (different jenkins server on different boxes) can the JQA1OnMachine1 (QA jenkis) poll the others at regular interval to see if a build was deployed successfully? if so can anyone tell me how?
Jenkins master slave along with Jenkins Pipeline Plugin would be one of the better ways to implement this however, since you don't want to use that approach you can explore PSTools to remotely capture processes or files on different server.
Your builds may update a file on the build server post completion of the build and your QA machine can run script with PSTools to monitor and trigger the QA testing based on the file content
I am trying to scale Jenkins for a large organization. Is there a way to have multiple Jenkins masters share a slave pool? For example, if I had 200 Jenkins Masters and I want them to share the same set of 50 Linux slaves.
That is, assuming each slave only has 1 executor, if Master A submits a job to the slave pool and it is running on Slave 1, if Master B submits a job to the slave pool, it would try to run on one of the other free slaves, since Slave 1 is already occupied.
I know multiple masters could share a single slave if I configured the slave to have a new workspace and executor for each master. However, I want to be able to set the slave up once, instead of having a slave.jar running on the slave for each master.
Cloudbees Op Center appears to provide this functionality, but looking for a way to do this with the Open Source version. If not, how difficult do you think it would be to extend Jenkins to have this functionality? I have Java development experience and have done a little work with Jenkins plugin development.
Thanks,
As you've noted it's not hard to share slaves between masters, just setup multiple workspaces and each master will install it's own slave jar. The trick is to share resources properly.
One such resource manager is Apache Mesos. A Jenkins Mesos plugin exists enabling the creation of slaves on a managed cluster.
This approach is very new and Ebay have blogged on how they've evolved their Jenkins setup to use Mesos:
Ebay CI solutin part 1
Ebay CI solution part 2
Hope this helps.
There is a Gearman Plugin developed by Open Stack to handle sharing slaves by multiple masters.
If it were me, I'd set up all the masters with a cloud plugin for slaves. For example, you could install the kubernetes plugin or the nomad plugin and connect all masters up to the same kubernetes or nomad cluster. Nomad or Kubernetes would take care of resource management, and the masters would only be submitting jobs to a shared pool of resources. This concept can easily be applied to other cloud providers, like AWS, but IMHO if you just want to set up an on-prem pool of resources for your jenkins masters from scratch, nomad is the easiest option.
I want to configure Jenkins to build my code on 1 server. Then want to deploy it on another server using Jenkins.Both servers are using Linux I want to automate the entire process as much as possible. I went through some of plugins like pipeline, Job Import Plugin, etc
Can anyone guide me how to go about it ? Which plugins will be useful ? Any example or tutorial somewhere will be useful. The configuration of build pipeline plugin on jenkins was not seamless for me.
Thanks,
Bhargav
I would work it this way :
Install jenkins on your first server
Install the following plugins : ssh credentials, ssh slaves, copy to
slave, and restart jenkins
Go to Manage jenkins -> Manage credentials, and add ssh credentials
for your second server
Go to Manage jenkins -> Manage nodes, and create a passive slave.
The launch method should be "Launch slave agents on Unix machines
via ssh". You should use the credentials that you have added in step
3
Create a job to build your code. In the advanded options of job, you
should indicate that the job must only be built on master node.
Create a job to deploy your code on the second server. In the
avanded options of job, you should indicate that the job must only
be built on slave node.
In the "Build Environment" section, check the "Copy files into workspace before building" box and configure what files you want to copy from first server (https://wiki.jenkins-ci.org/display/JENKINS/Copy+To+Slave+Plugin)
The code will be copied into the jenkins slave's workspace.
I've been reading about Jenkins master/slave configurations but I still have some questions:
Is it so that the slave Jenkins is not actually installed and started up the way master Jenkins is? I assumed I would install one master Jenkins and another slave Jenkins in the same way, and then master Jenkins would control the slave e.g. through SSH? So I cannot view the slave Jenkins through a GUI?
The reason why I have thought about adding a slave Jenkins on another VM is because the VM contains our application servers (many test environments). Deploying and starting/stopping application servers from master Jenkins is a pain because master Jenkins and application servers are on different machines. Therefore, if I would add a slave Jenkins to the machine where our application servers are, these would actually be deployed and started/stopped locally (by slave Jenkins). I wonder if I have missed something, of if my presumptions are still valid.
In a standard Jenkins master/slave setup, Jenkins is only installed on the master. That is where you see the user interface and start/configure build jobs.
The slaves execute the jobs. There is no Jenkins installation here other than a small Java app to have Jenkins communicate to/from the slave. Jenkins talks to these slaves through the slave.jar app over e.g. SSH via the SSH Slaves Plugin and can monitor if the slave is running, etc.
So in your case, you can start jobs from the master that will execute on the application servers.
The master/slave setup also allows you to host all whole bunch of different slaves, with different OSes, different hardware, etc. You can communicate job results (artifacts) from one slave to another via the Copy Artifacts Plugin.
There are also ways to duplicate the actual Jenkins master with load balancing in a heavy use scenario. That is not what you seem to be looking for.
Travis CI has a really nice feature, builds are run within VirtualBox VMs. Each time a build is started, the box is refreshed from a snapshot and the code copied on to it. Any problems with the build cannot affect the host, and you can use any OS to run your builds on.
This would be really good, for example, compiling and testing code on a guest OS that matches your production env. Also, you can keep your host free of any installation dependencies you might need (e.g. a database server) and run ITs without worrying about things like port conflicts.
Does such a thing exist for Jenkins?
Check out the Vagrant Plugin https://wiki.jenkins-ci.org/display/JENKINS/Vagrant-plugin
This plugin allows booting of Vagrant virtual machines, provisioning them and also executing scripts inside of them
You can run Jenkins in a Master Slave Setup. Your Master instance manages all the jobs but lets all the slaves do the actual work. These Slaves can be VMs or physical machines. Go To Manage Jenkins -> Manage Nodes -> New Node to add Nodes to your Jenkins Setup.
There is the vSphere Cloud Plugin and the Scripted Cloud Plugin that can be used for this purpose.