My requirement is to trigger a CI & CD on a kubernetes on prem infra, whenever a PR has been raised. Jenkins X is an ideal candidate but unfortunately due to few proxy issues it didnt come to fruitition.
Coming to kubernetes-operator, looking for few clarifications.
I've 4 nodes cluster, with one node being the leader.
Do I've to set up a new instance of Jenkins before hand on my K8s cluster or kubernetes-operator does that for me ?
Looking to access the Jenkins instance under the domain : jenkins.mybusinessunit.myorg.com/jenkins
Do I have to do any addtional configurations to enable master - slave set up.
Does kubernetes-operator provides a feature to support CI/CD model like Jenkins X ?
Thanks in advance.
As per your comments you are actually interessted in more of a cloud-native solution to operating a Jenkins, so here goes.
Since you already have a Kubernetes cluster and would like to use the Jenkins Kubernetes operator, then I would recommend you use the Jenkins Kubernetes Plugin for managing your workloads.
The Jenkins Kubernetes plugin enables you to run each of your pipelines in a separate Pod in your Kubernetes cluster, and once the required Service resources are setup, then the communication between master and slave pods is completely regulated by the plugin. I would recommend that you look into their documentation, which is quite good ( in comparison to other plugins ).
Now since you are also using the Jenkins Kubernetes operator you should know that the plugin is installed as one of the default plugins and is available as soon as your Jenkins instance is spun up. I would recommend you read through the Jenkins Kubernetes operator documentation to get a better grasp of what happens when that is running.
So now I will move onto your questions.
Do I've to set up a new instance of Jenkins before hand on my K8s cluster or kubernetes-operator does that for me ?
If you install the Jenkins Kubernetes operator via Helm chart, then no, you get a Jenkins master instance included. Otherwise if you install the controller into your cluster manually, then you will need to create a Jenkins CRD, which will create a Jenkins instance for you.
Looking to access the Jenkins instance under the domain : jenkins.mybusinessunit.myorg.com/jenkins
Use Ingress + Load Balancer + DNS Service or expose the Pod via NodePort. Note that exposing your master Pod via NodePort may require you to make your Jenkins Master instance publicly available ( and that may not be wise ).
Do I have to do any addtional configurations to enable master - slave set up.
Please refer to the documentation of Jenkins Kubernetes plugin and Jenkins Kubernetes operator. All details are provided there, but configuration is rather minimal.
Does kubernetes-operator provides a feature to support CI/CD model like Jenkins X
No. The Jenkins Kubernetes operator is there only to manage your Jenkins instance and backups in immutable fashion. Jenkins X can be used in combination with Jenkins, but neither replaces the other completely.
Related
I have inherited a Jenkins installation which is used by multiple remote teams, and running on an Amazon EKS cluster. For reasons that are not really relevant, I need to move this Jenkins workload to a new EKS cluster.
Deploying jenkins itself is not a major issue, I am doing so using helm. The persistence of the existing jenkins deployment is bound to an Amazon EBS volume, the persistence of the new deployment will also be. The mount point will be /var/jenkins_home
I'm trying to find a simple way of migrating everything from the current jenkins installation and configuration to the new one. This includes mainly:
Authorization Strategy (RBAC)
Jobs
Plugins
Cloud and Agent Config
I know that everything required is most likely in Jenkins Home. Could I in theory just dump out the current Jenkins Home folder and import into the new running jenkins container using kubetl cp or something like that? Is there an easier way? Is this unsafe in some way?
I am trying to setup Kubernetes for my company. I have looked a good amount into Jenkins X and, while I really like the roadmap, I have come the realization that it is likely not mature enough for my company to use at this time. (UI in preview, flaky command line, random IP address needs and poor windows support are a few of the issues that have lead me to that conclusion.)
But I understand that the normal Jenkins is very mature and can run on Kubernetes. I also understand that it can have dynamically created build agents run in the cluster.
But I am not sure about gitops support. When I try to google it (gitops jenkins) I get a bunch of information that includes Jenkins X.
Is there an easy(ish) way for normal Jenkins to use GitOps? If so, how?
Update:
By GitOps, I mean something similar to what Jenkins X supports. (Meaning changes to the cluster stored in a Git repository. And merging causes a deployment.)
I mean something similar to what Jenkins X supports. (Meaning changes to the cluster stored in a Git repository. And merging causes a deployment.)
Yes, this is the what Jenkins (or other CICD tools) do. You can declare a deployment pipeline in a Jenkinsfile that is triggered on merge (commit to master) and have other steps for other branches (if you want).
I recommend to deploy with kubectl using kustomize and store the config files in your Git repository. You parameterize different environments e.g. staging and production with overlays. You may e.g. deploy with only 2 replicas in staging but with 6 replicas and more memory resources in production.
Using Jenkins for this, I would create a docker agent image with kubectl, so your steps can use the kubectl command line tool.
Jenkins on Kubernetes
But I understand that the normal Jenkins is very mature and can run on Kubernetes. I also understand that it can have dynamically created build agents run in the cluster.
I have not had the best experience with this. It may work - or it may not work so well. I currently host Jenkins outside the Kubernetes cluster. I think that Jenkins X together with Tekton may be an upcoming promising solution for this, but I have not tried that setup.
I am trying to install Jenkins in my kubernetes cluster. When I am exploring I found that in two ways.
The first way that I understood is that, To install Jenkins master and slave. Here I found documentation for installing Jenkins master and slave agent on top of my kubernetes cluster.
The second way that I found that usage of Kubernetes plugin for Jenkins.If we using this way , Installing only master and configuring the plugin. And Slave pod will automatically working when one deployment is creating.
Confusion
Here my confusion is that,
In first method Do we need to define the worker machine for installing both master and slave ?
In second method, Is this proper way of installing Jenkins , Since we only installing master and configuring the plugin to use the Jenkins slave agent? Is this standard way of using Jenkins in top of kubernetes cluster?
Can anyoen give clarification for my confusions please?
I found this tutorial to be pretty helpful in getting jenkins running on my kubernetes cluster: https://github.com/GoogleCloudPlatform/continuous-deployment-on-kubernetes
It relies on the jenkins kubernetes plugin you mentioned. And if Google is doing it this way, its probably pretty safe to assume it is a valid method. It is the method I use on my cluster, where the jenkins master can provision slave pods as needed, which makes much more sense than keeping slaves alive that aren't being used.
I've been struggling with setting up the Jenkins Kubernetes Plugin on the Google Container Engine.
I have the plugin installed but I think all my builds are still running on master.
I haven't found any good documentation or guides on configuring this.
UPDATE
I removed the master executor from my Jenkins image. So now my builds aren't running on master but now they have no executor so they don't run at all. Just waits in the queue forever.
You'll need to tell Jenkins how and where to run your builds by adding your Kubernetes cluster as a 'cloud' in the Jenkins configuration. Go to Manage Jenkins -> Configure System -> Cloud -> Add new cloud and select 'Kubernetes'. You'll find the server certificate key, user name and password in your local kubectl configuration (usually in ~/.kube/config). The values for 'Kubernetes URL' and 'Jenkins URL' depend on your cluster setup.
Next, you'll need to configure the docker images that should be used to run your builds by selecting 'Add Docker Template'. Use labels to define which tasks should be run with which image!
Here's a good video tutorial and here you'll find a nice tutorial which explains everything in detail.
The important bit after you've installed the plugin, set up access to your Kubernetes cluster, and set up your first Kubernetes Pod Template with a label like jnlp-slave, is that in your Jenkinsfile you need to begin with something like node('jnlp-slave') {}. Then the pod will be started when you trigger a build.
There's also a helm chart for easy deployment if that helps :)
This example might also help once you've set the plugin up too.
I am trying to scale Jenkins for a large organization. Is there a way to have multiple Jenkins masters share a slave pool? For example, if I had 200 Jenkins Masters and I want them to share the same set of 50 Linux slaves.
That is, assuming each slave only has 1 executor, if Master A submits a job to the slave pool and it is running on Slave 1, if Master B submits a job to the slave pool, it would try to run on one of the other free slaves, since Slave 1 is already occupied.
I know multiple masters could share a single slave if I configured the slave to have a new workspace and executor for each master. However, I want to be able to set the slave up once, instead of having a slave.jar running on the slave for each master.
Cloudbees Op Center appears to provide this functionality, but looking for a way to do this with the Open Source version. If not, how difficult do you think it would be to extend Jenkins to have this functionality? I have Java development experience and have done a little work with Jenkins plugin development.
Thanks,
As you've noted it's not hard to share slaves between masters, just setup multiple workspaces and each master will install it's own slave jar. The trick is to share resources properly.
One such resource manager is Apache Mesos. A Jenkins Mesos plugin exists enabling the creation of slaves on a managed cluster.
This approach is very new and Ebay have blogged on how they've evolved their Jenkins setup to use Mesos:
Ebay CI solutin part 1
Ebay CI solution part 2
Hope this helps.
There is a Gearman Plugin developed by Open Stack to handle sharing slaves by multiple masters.
If it were me, I'd set up all the masters with a cloud plugin for slaves. For example, you could install the kubernetes plugin or the nomad plugin and connect all masters up to the same kubernetes or nomad cluster. Nomad or Kubernetes would take care of resource management, and the masters would only be submitting jobs to a shared pool of resources. This concept can easily be applied to other cloud providers, like AWS, but IMHO if you just want to set up an on-prem pool of resources for your jenkins masters from scratch, nomad is the easiest option.