I've been struggling with setting up the Jenkins Kubernetes Plugin on the Google Container Engine.
I have the plugin installed but I think all my builds are still running on master.
I haven't found any good documentation or guides on configuring this.
UPDATE
I removed the master executor from my Jenkins image. So now my builds aren't running on master but now they have no executor so they don't run at all. Just waits in the queue forever.
You'll need to tell Jenkins how and where to run your builds by adding your Kubernetes cluster as a 'cloud' in the Jenkins configuration. Go to Manage Jenkins -> Configure System -> Cloud -> Add new cloud and select 'Kubernetes'. You'll find the server certificate key, user name and password in your local kubectl configuration (usually in ~/.kube/config). The values for 'Kubernetes URL' and 'Jenkins URL' depend on your cluster setup.
Next, you'll need to configure the docker images that should be used to run your builds by selecting 'Add Docker Template'. Use labels to define which tasks should be run with which image!
Here's a good video tutorial and here you'll find a nice tutorial which explains everything in detail.
The important bit after you've installed the plugin, set up access to your Kubernetes cluster, and set up your first Kubernetes Pod Template with a label like jnlp-slave, is that in your Jenkinsfile you need to begin with something like node('jnlp-slave') {}. Then the pod will be started when you trigger a build.
There's also a helm chart for easy deployment if that helps :)
This example might also help once you've set the plugin up too.
Related
My requirement is to trigger a CI & CD on a kubernetes on prem infra, whenever a PR has been raised. Jenkins X is an ideal candidate but unfortunately due to few proxy issues it didnt come to fruitition.
Coming to kubernetes-operator, looking for few clarifications.
I've 4 nodes cluster, with one node being the leader.
Do I've to set up a new instance of Jenkins before hand on my K8s cluster or kubernetes-operator does that for me ?
Looking to access the Jenkins instance under the domain : jenkins.mybusinessunit.myorg.com/jenkins
Do I have to do any addtional configurations to enable master - slave set up.
Does kubernetes-operator provides a feature to support CI/CD model like Jenkins X ?
Thanks in advance.
As per your comments you are actually interessted in more of a cloud-native solution to operating a Jenkins, so here goes.
Since you already have a Kubernetes cluster and would like to use the Jenkins Kubernetes operator, then I would recommend you use the Jenkins Kubernetes Plugin for managing your workloads.
The Jenkins Kubernetes plugin enables you to run each of your pipelines in a separate Pod in your Kubernetes cluster, and once the required Service resources are setup, then the communication between master and slave pods is completely regulated by the plugin. I would recommend that you look into their documentation, which is quite good ( in comparison to other plugins ).
Now since you are also using the Jenkins Kubernetes operator you should know that the plugin is installed as one of the default plugins and is available as soon as your Jenkins instance is spun up. I would recommend you read through the Jenkins Kubernetes operator documentation to get a better grasp of what happens when that is running.
So now I will move onto your questions.
Do I've to set up a new instance of Jenkins before hand on my K8s cluster or kubernetes-operator does that for me ?
If you install the Jenkins Kubernetes operator via Helm chart, then no, you get a Jenkins master instance included. Otherwise if you install the controller into your cluster manually, then you will need to create a Jenkins CRD, which will create a Jenkins instance for you.
Looking to access the Jenkins instance under the domain : jenkins.mybusinessunit.myorg.com/jenkins
Use Ingress + Load Balancer + DNS Service or expose the Pod via NodePort. Note that exposing your master Pod via NodePort may require you to make your Jenkins Master instance publicly available ( and that may not be wise ).
Do I have to do any addtional configurations to enable master - slave set up.
Please refer to the documentation of Jenkins Kubernetes plugin and Jenkins Kubernetes operator. All details are provided there, but configuration is rather minimal.
Does kubernetes-operator provides a feature to support CI/CD model like Jenkins X
No. The Jenkins Kubernetes operator is there only to manage your Jenkins instance and backups in immutable fashion. Jenkins X can be used in combination with Jenkins, but neither replaces the other completely.
I am trying to move the complete eco-system of our SAAS product to Kubernetes (and use Docker containers).
I am supposed to give a bash script which will set up everything. Only manual intervention should be setting up the Kubernetes cluster and mounting Persistent Volumes.
We were using Jenkins for code deployment and cron jobs. I am able to create the Jenkins service but I can not find ways to configure it using the command line. Tried finding ways online but can not find any good documentation.
First welcome to kubernetes, second, there are a lot of tools, templates over there, I would recommend you to check what is Helm
This is the Jenkins chart if you want to check:
https://github.com/helm/charts/tree/master/stable/jenkins
There is also a "fork" of jenkins for containerized environments, that I like, you can check more about Jenkins-X here
You can use helm package manager and simply install the Jenkin stable version.
Before using helm you have to setup tiller on kubernetes cluster.
$ helm install --name my-release stable/jenkins
here stable version of jenkin using helm.
https://github.com/helm/charts/tree/master/stable/jenkins
I can add that you can store Jenkins home folder as well as plugins and artifacts folder on persistent volume and mount that volume to Jenkins pod as a part of Helm installation. You can also make daily snapshots/backups of Jenkins disk. In this way Jenkins deployment becomes very smooth, quick and reliable.
I am wondering what is the pattern to proper integrate Jenkins and Kubernetes to satisfy the following scenario :
A developer checks in some code relative to a new feature
Jenkins build the container creating a pod for Kubernetes
Kubenetes assigns a proper dns name to the pod, this is to allow the
tester to connect exactly to the pod containing such feature to test
Carry on tests
I may be able to configure steps 1 to 2 but I am wondering if there is a way to automatically connect exactly to the pod that has the new feature
I need to test.
Just to be more clear, system builds the code automatically, a message is sent to the tester telling him which pod has that feature he is looking to test, in some way he tests the container with such a feature and if everything is ok the feature is merged in master.
cheers
Sorry, not a complete answer but what you describe sounds like the Auto Dev Ops Feature of Gitlab. You deploy a new "environment" to k8 for each branch, isolated into a namespace. I think you will be able to copy the procedure that Gitlab takes:
https://www.youtube.com/watch?v=uWC2QKv15mk&t=1730s
useful links:
Dns
http://xip.io/
Gitlab File (dont be scared by length) https://gitlab.com/gitlab-org/gitlab-ce/blob/master/lib/gitlab/ci/templates/Auto-DevOps.gitlab-ci.yml
Helm Chart used
https://gitlab.com/charts/auto-deploy-app
The helm chart expects your app as a docker container exposed on port 5000 and brings Postgres.
Jenkins X will deploy your pull requests to a new preview environment and give you a URL you can connect to, so you can carry your tests
https://jenkins-x.io/about/features/#preview-environments
I am trying to install Jenkins in my kubernetes cluster. When I am exploring I found that in two ways.
The first way that I understood is that, To install Jenkins master and slave. Here I found documentation for installing Jenkins master and slave agent on top of my kubernetes cluster.
The second way that I found that usage of Kubernetes plugin for Jenkins.If we using this way , Installing only master and configuring the plugin. And Slave pod will automatically working when one deployment is creating.
Confusion
Here my confusion is that,
In first method Do we need to define the worker machine for installing both master and slave ?
In second method, Is this proper way of installing Jenkins , Since we only installing master and configuring the plugin to use the Jenkins slave agent? Is this standard way of using Jenkins in top of kubernetes cluster?
Can anyoen give clarification for my confusions please?
I found this tutorial to be pretty helpful in getting jenkins running on my kubernetes cluster: https://github.com/GoogleCloudPlatform/continuous-deployment-on-kubernetes
It relies on the jenkins kubernetes plugin you mentioned. And if Google is doing it this way, its probably pretty safe to assume it is a valid method. It is the method I use on my cluster, where the jenkins master can provision slave pods as needed, which makes much more sense than keeping slaves alive that aren't being used.
I want to try out continuous deployment with Jenkins on Google Container Engine.
I have gone through https://cloud.google.com/solutions/jenkins-on-container-engine-tutorial and then https://cloud.google.com/solutions/continuous-delivery-jenkins-container-engine.
I have done it all twice to make sure I didn't miss anything in the tutorials, but still I have the same problem. The problem is in the second tutorial, under the section "Creating a Jenkins job". I follow all the steps, but my build job never starts. All I get in Jenkins is Waiting for next available executor.
Do you have any ideas what is wrong?
Things started working once I added jnlp here:
Manage Jenkins -> Configure System -> Cloud -> Kubernetes -> Defaults Provider Template Name