I have 15 Jenkins jobs configured in order to implement a specific flow. I am improving and editing these jobs as time goes by.
Is there a way to save all these jobs and its configurations to a repository or at least in the form of export jobs, save it and import when needed?
There are two plugins that will help you save Jenkins jobs, "SCM Sync configuration" and "JobConfigHistory" defined at the wiki.jenkins-ci.org website.
SCM Sync Configuration Plugin (which keeps the config in a SCM repository)
or
Job Config History Plugin (Saves copies of all job and system configurations)
The Job DSL Plugin allows to define the jobs in a DSL and store the DSL scripts in a SCM repo. The DSL increases the readability of the config files in contrast to the XML format.
For an intro, see the slides and video from the Configuration as Code: The Job DSL Plugin talk at the Jenkins User Conference 2015 in London.
You can move/copy jobs to another destination simply by copying directory with job (default path for jobs directories is /var/lib/jenkins/jobs).
You can get more info here - https://wiki.jenkins-ci.org/display/JENKINS/Administering+Jenkins
The Workflow feature may let you write your entire process as one (Groovy) script, which you can then maintain in your version control system alongside other sources.
Related
I would like to set up many jobs automatically. In the build step "Process Job DSLs", the job-dsl plugin allows to either "Use the provided DSL script" or "Look on Filesystem" for doing its thing. While "Look on Filesystem" looks like what I'm looking for, since it allows to define multiple Job DSL files at once, it sources them from a directory on the Jenkins server. I would rather have it fetch those job specifications from a remote Git repository.
Therefore, is it possible to source all jobs from a remote Git repository? Somehow like this pseudo code?
Apache Beam uses the Jenkins Job DSL plugin with a seed job responsible for generating the rest of our Jenkins jobs from groovy source files (sample run).
The job definitions are authored as Groovy scripts and maintained in source control just like other source code. I would like to more easily verify changes to Jenkins job definitions during code review.
We already have the ability to run our Seed Job on a GitHub pull request, however the updated jobs get published right away and replace the existing jobs. Is it possible to run a Seed Job on Jenkins but not publish the generated job? Ideally it would run the Seed Job script and output the generated XML somewhere without actually committing it to Jenkins. Perhaps it could even show a diff against the existing job definition.
I am currently using the Jenkins supermarket cookbook to deploy an instance of Jenkins running as a service on my Chef managed node. So far I have modified my _master_war_ recipe file to install Jenkins and start Jenkins as a service using runit, install all plugin versions I need as well as enable matrix based security and create a base administrator account. It should also be noted we are using BitBucket for source control.
I also want to add my jobs as part of the automated Jenkins deployment. From what I understand, the way to do this is to copy the job configuration files from a directory on my chef workstation to a directory (Chef's cache path), on the managed node.
Per the Jenkins public supermarket cookbook readme:
The :create action requires a Jenkins job config.xml. This config file
must exist on the target node and contain a valid Jenkins job
configuration file. Because the Jenkins CLI actually reads and
generates its own copy of this file, do NOT write this configuration
inside of the Jenkins job. We recommend putting them in Chef's file
cache path.
As these job configurations do change periodically, I'm wondering what is the best way to maintain the most recent copy of the job configuration file(s) on my chef workstation for deployment to my managed node?
Am I understanding the cookbook documentation correctly in that we will need a local copy of the job configuration file (on the Chef workstation), that is then copied to Chef's cache file path on the managed node?
Thanks in advance for any help anyone is able to provide.
Personally, I count setting up Jenkins jobs a lot more into the domain of Jenkins instead of Chef. The Jenkins community has developed several "jobs as code" approaches, the most popular ones being the Job DSL and the Jenkins Pipelines, with the latter one being the probably better starting point.
What remains up to the Chef cookbook is to define the seed job, either e.g. for the "Bitbucket Organisation Folder" plugin (and the one job that points to your organisation at BitBucket) or a so-called "Seed job" for the Job DSL.
Regarding automated setup of pipelines, I recommend a look at Torben Knerr's examples. This uses
Pipeline (as defined in Jenkinsfiles) for the actual build jobs
a Job DSL seed job to set up the pipeline job(s)
Regarding the actual implementation in Chef, you can see an example in a cookbook of mine. The template resource copies a file from the cookbook (in the templates/ subdirectory) into some temporary path, from where the jenkins_job resource picks it up (on the Jenkins server).
So I'm not sure, if you got it right regarding:
will need a local copy of the job configuration file (on the Chef workstation)
So you just need it once on your workstation to add it to the cookbook, yes.
I am trying to understand how to best deploy an instance of Jenkins, complete with plugins, users and jobs using Chef. I am currently using the Chef Jenkins Supermarket cookbook.
I am attempting to achieve automated deployment of our Pipelines as part of the project. From what I have gathered, the best way to go about this is to have Chef configure a seed job in Jenkins initial setup and configuration.
The seed job should specify, among other things, the git repository from which to find and use a Jenkinsfile for a given job. I've found this resource by Daniel Spilker to be helpful in explaining seed jobs.
So the seed Jenkins job would be run, which would then generate the Jenkins job we have just scripted with it (in this case the seed job would be to pull the Jenkinsfile from source control and configure a new Jenkins job (our pipeline), with the details of the Jenkinsfile).
Am I understanding this correctly as the proper way to not only automate Jenkins job configuration, but also as the proper way to always have an up to date job configuration for any given job in the event the job configuration were to change?
If we used a seed job to setup our pipeline, what are some possible solutions to having the initial seed job run automatically once Jenkins is fully configured by Chef?
As for job configuration changes that may occur over time, would we need to setup the seed job to poll source control for any changes in the Jenkinsfile periodically in the event the Jenkinsfile has been modified? (It may be helpful to note that we are currently using BitBucket for source control).
Just getting started with pipeline as code. Thanks to everybody in advance for their patience and guidance.
I've mentioned this a bit in your other questions, but the least painful approach is to treat Jenkins as a database, not a web service. Have Chef do the basic install, but then configure the initial bits by hand. For DR, rely on your backups rather than Chef.
How do you maintain the Jenkins job configuration in SCM along side the source code?
As source code evolves, so does the job configuration. It would be ideal to be able to keep the job configuration in SCM, for the following benefits:
easy to see who a history of the changes, including the author and the description
able to rebuild old branch/tag by checking out the revision and build just work
not having to scroll through the UI to find the appropriate section and make change
I see there is a Jenkins Job Builder plugin. I prefer a solution along the lines of Travis CI, where the job configuration is maintained in a YAML file (.travis.yml). Any good suggestions?
Note: Most of our projects are using Java & Maven.
Update 2016: Jenkins now provides a Jenkinsfile which provides exactly this. This is supported by the core Jenkins developers and actively developed.
Benefits:
Creating a Jenkinsfile, which is checked into source control, provides a number of immediate benefits:
Code review/iteration on the Pipeline
Audit trail for the Pipeline
Single source of truth for the Pipeline, which can be viewed and edited by multiple members of the project.
I've written a plugin that does this!
Other than my plugin, you have some (limited) options with existing Jenkins plugins:
Use a single test script
If you configure your Jenkins to simply run:
$ bash run_tests.sh
You can then check in a run_tests.sh file into your SCM repo and you're now tracking changes for how you run tests. However, this won't track configuration of any plugins.
Similarly, if you're using Maven, the Maven Project Plugin simply runs a specified goal for your repo.
The Literate Plugin does allow Jenkins to run the commands in your README.md, but it hasn't yet been released.
Track changes to Jenkins configuration
You can use the SCM Sync configuration plugin to write configuration changes to SCM, so you at least have a persistent record. This is global, across all projects on your Jenkins instance.
There's also the job config history plugin, which stores config history on the filesystem.
Write Jenkins configuration from SCM
The Jenkins job builder project you mentioned lets you check config changes into SCM and have them applied to your Jenkins instance. Again, this is across all projects on your Jenkins instance.
Write Jenkins configuration from another job
You can use the Job DSL Plugin with a repo of groovy scripts. Jenkins then polls that repo, executes the groovy scripts, which create job configurations.
Discussions
Issue 996 (now closed) discusses this, and it has also been discussed on the mailing list: 'Keeping track of Hudson's configuration changes', and 'save hudson config in svn'.
you can do this all with the workflow plugin and a lot more. Workflow is one of the most advanced technics to use jenkins and it has a very strong support.
It is based on a groovy DSL and allows you to keep the whole configuration in the SCM of your choise (e.g. GIT, SVN...).