I wanted to setup a single source of truth to my Jenkins running in different DC's (Data Centers), so I converted all my jenkins jobs to pipeline jobs - Jenkinsfile taken from Github repo.
I'm looking for a method to create/delete/update Jenkins jobs in UI, for the multiple Jenkins running in different DC's automatically. so I am looking to auto create/delete jobs in all Jenkins upon updating the Job configurations in the GitHub Repository.
Any recommendations or help for this workflow would be appreciated.
Related
I'm using recent Jenkins version 2.286 and since this update there is an security hint: "You should set up distributed builds. Building on the controller node can be a security issue. See the documentation."
But I'm already doing so with three Jenkins nodes and I also fully understand the security implications.
The problem here is, that there are two jobs that need to run an the master, since they are the jobs that deploy those Jenkins nodes. That means I can not reduce the build executors to 0.
I've also tried using the Job Restrictions plugin to restrict which jobs can run on the master. This problem here is that all my jobs are waiting for the master queue do have a free slot available. I wonder why, because they all are declarative pipelines and define something like:
agent {
label 'some-different-node-label'
}
Which means they aren't really executed on the master node.
Questions here are:
Is this intentionally that all jobs require the master node before switching the agent?
Is there any configuration option to change that?
Is there a way to execute the deploy jobs on master, even if there aren't any executed defined (to bypass that behavior)?
Thanks.
With declarative pipelines the lightweight code checkout is done on the Master node to get a Jenkinsfile for that job. While this doesnt use an executor on the Master perhaps the Job Restriction Plugin is still blocking this (I havent used it before so cannot comment)
Also certain pipeline actions are delegated back to the Master node as well (e.g. the withAWSParameterStore step.
If you look at the console output for a Declarative pipeline job, you will see lots of output (mainly around library checkouts or git checkouts) before you see the start of the pipeline [Pipeline] Start of Pipeline. All that is done on the Master.
Unfortunately this cannot be changed as the Master needs to do this work to find out which agent type to delegate the job to.
Depending on how you are running you agents, you could use something like the EC2 Cloud Plugin to generate you agent nodes which wouldn't require a job to do it
I have automated Jenkins master and slaves deployment and redeployment successfully.
I know how to manually create pipeline jobs and add github repos to use their Jenkinsfiles for the steps.
my issue is how can I automate the pipeline jobs addition to jenkins after its been destroyed and redeployed without having to manually create the pipeline jobs and point to Jenkinsfile each time.
I have seen this done before in a container environment with chef and docker when redeployed or updated it re-adds all the pipelines automatically again.
I want to not use the UI at all only to confirm job status progress and verify settings.
I would recommend looking at the JobDSL Plugin to create jobs, using a seed job to create them on initial Jenkins startup. The Jenkins Configuration-as-Code plugin can be used to setup any other configuration outside the jobs.
I am currently using Cloudbees Jenkins Coreas my Jenkins solution.
I am using Jenkins Pipelines to write our Jenkins job configuration. These pipelines are stored in GitHub repositories. Each Jenkins job when created is connected to a GitHub Repository where the source code is pulled from, and that's where the Jenkinsfile is stored and Jenkins reads from.
Below are some high-level photos for how our Jenkins jobs are configured.
The advantage of the way these jobs are configured is the Jenkinsfile is always read from the master branch. Meaning if a rouge developer tries to remove stages from the Jenkinsfile from within there own branch, it doesn't matter because the Jenkinsfile is always read from the master branch (which is always protected).
However, the one massive drawback to this - is how do teams and developers who are devops engineerings make changes to the Jenkinsfile? For example, let's say a developer creates a branch called feature-jenkins-search and they edit the Jenkinsfile adding a new stage in the pipeline. Whenever they push these changes to GitHub to test - they can't test as it's always read from the master branch? Meaning devops engineerings have to work directly on the master branch? Surely this is not the best way to go and there is a better configuration to set?
We do want to still provide the security that if a developer is rougue and
You should really look into the Jenkins multi-branch pipeline feature. The Jenkins multi-branch pipeline allows to create a single configuration item in Jenkins (a bit like a folder) that can detect all the branches and pull requests in a GitHub repository with a Jenkinsfile and build them using automatically created jobs. Inside this multi-branch pipeline object when it is configured in Jenkins, you will find a number of jobs to build the various branches and pull-requests in the GitHub repository.
So your developers should maintain a Jenkinsfile in every branch they work on in GitHub to build that branch in your Jenkins server.
It is possible to make the Jenkinsfile do branch specific handling if required with conditional stages / when conditions in the Jenkinsfile pipelines in each branch.
You can lock down the master branch so that code and Jenkinsfile changes from other branches can only be merged with an approved PR (pull request). There is good integration between Jenkins and GitHub such that you can configure the master branch to only allow a PR to be merged if the PR is buildable in Jenkins. So if developers add new stages / processing to a Jenkinsfile on a branch being merged to master, it should be validated so that builds of your master branch are not broken.
There is a lot of configurability in the Jenkins multi-branch pipeline object for detection and handling of branches and it may be necessary to experiment to get it right for what you need with your team. If you cannot find this feature in Jenkins, it is probably because the correct Jenkins pipeline and GitHub related plugins are not installed.
You could also have a look at a similar Jenkins feature called the Jenkins GitHub Organization Folder which allows to detect and build all repos and branches at a GitHub Organization level. But when starting out, I would suggest to look into the multi-branch pipeline at the single repo level first.
These features are discussed in the Jenkins pipeline documentation. We use these features with our internal GitHub and Jenkins server and it works very well.
I think you will find the idea of using a single Jenkinsfile in the master branch to be used for building all branches is unworkable, as you have seen!
I am trying to understand how to best deploy an instance of Jenkins, complete with plugins, users and jobs using Chef. I am currently using the Chef Jenkins Supermarket cookbook.
I am attempting to achieve automated deployment of our Pipelines as part of the project. From what I have gathered, the best way to go about this is to have Chef configure a seed job in Jenkins initial setup and configuration.
The seed job should specify, among other things, the git repository from which to find and use a Jenkinsfile for a given job. I've found this resource by Daniel Spilker to be helpful in explaining seed jobs.
So the seed Jenkins job would be run, which would then generate the Jenkins job we have just scripted with it (in this case the seed job would be to pull the Jenkinsfile from source control and configure a new Jenkins job (our pipeline), with the details of the Jenkinsfile).
Am I understanding this correctly as the proper way to not only automate Jenkins job configuration, but also as the proper way to always have an up to date job configuration for any given job in the event the job configuration were to change?
If we used a seed job to setup our pipeline, what are some possible solutions to having the initial seed job run automatically once Jenkins is fully configured by Chef?
As for job configuration changes that may occur over time, would we need to setup the seed job to poll source control for any changes in the Jenkinsfile periodically in the event the Jenkinsfile has been modified? (It may be helpful to note that we are currently using BitBucket for source control).
Just getting started with pipeline as code. Thanks to everybody in advance for their patience and guidance.
I've mentioned this a bit in your other questions, but the least painful approach is to treat Jenkins as a database, not a web service. Have Chef do the basic install, but then configure the initial bits by hand. For DR, rely on your backups rather than Chef.
How do you maintain the Jenkins job configuration in SCM along side the source code?
As source code evolves, so does the job configuration. It would be ideal to be able to keep the job configuration in SCM, for the following benefits:
easy to see who a history of the changes, including the author and the description
able to rebuild old branch/tag by checking out the revision and build just work
not having to scroll through the UI to find the appropriate section and make change
I see there is a Jenkins Job Builder plugin. I prefer a solution along the lines of Travis CI, where the job configuration is maintained in a YAML file (.travis.yml). Any good suggestions?
Note: Most of our projects are using Java & Maven.
Update 2016: Jenkins now provides a Jenkinsfile which provides exactly this. This is supported by the core Jenkins developers and actively developed.
Benefits:
Creating a Jenkinsfile, which is checked into source control, provides a number of immediate benefits:
Code review/iteration on the Pipeline
Audit trail for the Pipeline
Single source of truth for the Pipeline, which can be viewed and edited by multiple members of the project.
I've written a plugin that does this!
Other than my plugin, you have some (limited) options with existing Jenkins plugins:
Use a single test script
If you configure your Jenkins to simply run:
$ bash run_tests.sh
You can then check in a run_tests.sh file into your SCM repo and you're now tracking changes for how you run tests. However, this won't track configuration of any plugins.
Similarly, if you're using Maven, the Maven Project Plugin simply runs a specified goal for your repo.
The Literate Plugin does allow Jenkins to run the commands in your README.md, but it hasn't yet been released.
Track changes to Jenkins configuration
You can use the SCM Sync configuration plugin to write configuration changes to SCM, so you at least have a persistent record. This is global, across all projects on your Jenkins instance.
There's also the job config history plugin, which stores config history on the filesystem.
Write Jenkins configuration from SCM
The Jenkins job builder project you mentioned lets you check config changes into SCM and have them applied to your Jenkins instance. Again, this is across all projects on your Jenkins instance.
Write Jenkins configuration from another job
You can use the Job DSL Plugin with a repo of groovy scripts. Jenkins then polls that repo, executes the groovy scripts, which create job configurations.
Discussions
Issue 996 (now closed) discusses this, and it has also been discussed on the mailing list: 'Keeping track of Hudson's configuration changes', and 'save hudson config in svn'.
you can do this all with the workflow plugin and a lot more. Workflow is one of the most advanced technics to use jenkins and it has a very strong support.
It is based on a groovy DSL and allows you to keep the whole configuration in the SCM of your choise (e.g. GIT, SVN...).