I want to use Jenkins and store the configuration and the pipeline in my SCM(e.g. git). To do so, I created a directory, let's say "jobs" in the root of my project where I will store jobs.groovy files written as JobDSL plugin files.
Should I do all the things in a single job file, like fetching the source code, testing it, maybe building Docker images if necessary, then deploying on AWS cloud? Or for each operation, should I create different jobs? If so, then how can I create a pipeline using these job files?
look at jenkins configuration as code plugin. following link would be helpful
https://github.com/tomasbjerre/jenkins-configuration-as-code-sandbox
Related
I am currently using the Jenkins supermarket cookbook to deploy an instance of Jenkins running as a service on my Chef managed node. So far I have modified my _master_war_ recipe file to install Jenkins and start Jenkins as a service using runit, install all plugin versions I need as well as enable matrix based security and create a base administrator account. It should also be noted we are using BitBucket for source control.
I also want to add my jobs as part of the automated Jenkins deployment. From what I understand, the way to do this is to copy the job configuration files from a directory on my chef workstation to a directory (Chef's cache path), on the managed node.
Per the Jenkins public supermarket cookbook readme:
The :create action requires a Jenkins job config.xml. This config file
must exist on the target node and contain a valid Jenkins job
configuration file. Because the Jenkins CLI actually reads and
generates its own copy of this file, do NOT write this configuration
inside of the Jenkins job. We recommend putting them in Chef's file
cache path.
As these job configurations do change periodically, I'm wondering what is the best way to maintain the most recent copy of the job configuration file(s) on my chef workstation for deployment to my managed node?
Am I understanding the cookbook documentation correctly in that we will need a local copy of the job configuration file (on the Chef workstation), that is then copied to Chef's cache file path on the managed node?
Thanks in advance for any help anyone is able to provide.
Personally, I count setting up Jenkins jobs a lot more into the domain of Jenkins instead of Chef. The Jenkins community has developed several "jobs as code" approaches, the most popular ones being the Job DSL and the Jenkins Pipelines, with the latter one being the probably better starting point.
What remains up to the Chef cookbook is to define the seed job, either e.g. for the "Bitbucket Organisation Folder" plugin (and the one job that points to your organisation at BitBucket) or a so-called "Seed job" for the Job DSL.
Regarding automated setup of pipelines, I recommend a look at Torben Knerr's examples. This uses
Pipeline (as defined in Jenkinsfiles) for the actual build jobs
a Job DSL seed job to set up the pipeline job(s)
Regarding the actual implementation in Chef, you can see an example in a cookbook of mine. The template resource copies a file from the cookbook (in the templates/ subdirectory) into some temporary path, from where the jenkins_job resource picks it up (on the Jenkins server).
So I'm not sure, if you got it right regarding:
will need a local copy of the job configuration file (on the Chef workstation)
So you just need it once on your workstation to add it to the cookbook, yes.
does anyone know if its possible to add a Jenkins pipeline build into a Jenkins docker image? For example, I may have a Jenkinsfile that defines my pipeline in groovie, and would like to ADD that into my image when building from the Jenkins image.
something like:
FROM jenkins:latest
ADD ./jobs/Jenkinsfile-pipeline-example $JENKINS_HOME/${someplace}
And have that pipeline ready to go when i run it.
Thanks.
It's a lot cleaner to use Jenkinsfile for this instead. This way, as your repositories develop you can change the build process without needing to recompile and redeploy your Jenkins instance everytime. (less work, and less CI downtime) Also, having the Jenkinsfile in source code allows a simpler decoupling.
If you have any questions about extending Jenkins on Docker further to handle building NodeJS, Ruby or something else I go into how to do all that in an article.
You can create any job in Jenkins by passing in an XML file that describes the job. See https://support.cloudbees.com/hc/en-us/articles/220857567-How-to-create-a-job-using-the-REST-API-and-cURL
The way I've done this is to manually create the job I want in Jenkins, then append config.xml to the URL and it shows you the XML content needed to generate the pipeline job. Save that XML and you can deliver it to your newly deployed Jenkins instance.
I use a system similar to this to generate several hundred jobs based on our external build specifications.
I have several Jenkins Pipeline jobs set up on my Jenkins installation all of them with a Jenkinsfile inside the repository.
These pipelines are run for all branches, and contains all steps necessary to build and deploy the branch. However, there are some differences for the different branches with regards to building and deploying them, and I would like to be able to configure different environment variables for the different branches.
Is that possible with Jenkins, or do I need to reevaluate my approach or use another CI system?
#rednax answer works if you're using a branch-per-environment git strategy. But if you're using git-flow (or any strategy where you assume that changes will be propogated up, possibly without human intervention, to master/production) you'll run into headaches where a merge will overwrite scripts/variables.
We use a set of folders which match the environment names : infrastructure/Jenkinsfile contains the common steps, and infrastructure/test/Jenkinsfile contains the steps specific to the test environment (the folders also contain Dockerfiles and cloudformation scripts). You could make that very complex with cascading includes or file merges, or simply have almost-identical copies of each file in each folder.
When configuring the job you can specify for Jenkins to grab the script (Jenkins file) from the branch on which you are running. This mean that technically you can adjust the script on each of your branches to set up parameters there. Or you can grab the script from the same source control location, but commit a configuration file in each of your branches and have the script read that file after the checkout.
I want to use a config file provided by config file provider plugin in a pipeline project.
However when I run a build step inside a slave. I get a "PermissionDenied" exception, The same runs in master however.
So question is thats the best possible way to share files between master and slaves. I may not be able to Copy to slave plugin as there doesn't seem to be pipeline support.
If you want to share files between stages or nodes you can use the stash - unstash methods. see the example here
If you want to share files between builds you can use the archive method and the Copy Artifact Plugin
I have 15 Jenkins jobs configured in order to implement a specific flow. I am improving and editing these jobs as time goes by.
Is there a way to save all these jobs and its configurations to a repository or at least in the form of export jobs, save it and import when needed?
There are two plugins that will help you save Jenkins jobs, "SCM Sync configuration" and "JobConfigHistory" defined at the wiki.jenkins-ci.org website.
SCM Sync Configuration Plugin (which keeps the config in a SCM repository)
or
Job Config History Plugin (Saves copies of all job and system configurations)
The Job DSL Plugin allows to define the jobs in a DSL and store the DSL scripts in a SCM repo. The DSL increases the readability of the config files in contrast to the XML format.
For an intro, see the slides and video from the Configuration as Code: The Job DSL Plugin talk at the Jenkins User Conference 2015 in London.
You can move/copy jobs to another destination simply by copying directory with job (default path for jobs directories is /var/lib/jenkins/jobs).
You can get more info here - https://wiki.jenkins-ci.org/display/JENKINS/Administering+Jenkins
The Workflow feature may let you write your entire process as one (Groovy) script, which you can then maintain in your version control system alongside other sources.