I'm a newbie to Jenkins Templating Engine, and trying to implement a CI process using JTE. Usually, when using Jenkins, i have a repo named DevOps in which i keep all the utility scripts that i'm using in the CI process instead of placing them in the developers' source code repositories. So in each Jenkinsfile i check out the source code as well as the DevOps repo into the workspace and use it.
I was wondering if this is considered a best practice in general. If not, what is? and if is, what is the best way to imitate that kind of pattern in JTE, given that my fresh new Jenkinsfile looks like:
checkout scm
build()
deploy()
Checkout scm elegantly checks out the source code repository defined in the SCM plugin in the configuration page. Where can i embed another checkout step that will clone the DevOps repo and still keep the JTE infrastructure generic?
coming in JTE 2.0 you'll be able to store and access library resources.
merged PR for the feature: https://github.com/jenkinsci/templating-engine-plugin/pull/102
Related
Short explanation:
There are many repos in our Git
Each repo has it's own Jenkinsfile who has it's own separate Job at Jenkins
All Jenkins files are doing 99% the same thing!
What we want to achieve at the moment:
Build one Jenkinsfile for all repo
Maintain branches in between repos
Delete if we can the current Jenkins files of each repo and use the only generic new file.
Have the versatile to use and manipulate parameters so Jenkins file won't be affected by any other repo config
Solutions for now:
Remove all Jenkins files in all repos
Re-configure the Jenkinsfile PATH in Jenkins gui website to direct to our new file
Put a config .yaml file in each repo who will contains all the relevant information of each repo (like key-value)
So, when each repo will be triggered, our new Jenkinsfile will load the config file and use the parameters to proceed all the stages related to the config file.
I would be happy to hear ideas / examples / snippets from you guys! It will highly help me!
Regards
Niv
(Answering due to lack of reputation for comment. Please excuse)
Hi #n1vgabay
If you are using Bitbucket for your SCM, then you may try these:
create a Organization Folder/Bitbucket team Project inside Jenkins (From Jenkins --> New Item--> Organization Folder or Bitbucket team/Project)
Update the config to filter all the repos (or regex them) under the Project inside your SCM. This will create all the repos as individual MB pipelines with all the branches under them as individual jobs. Also with Bitbucket Server Integration plugin, it automatically creates Webhooks for all the repos to trigger the jobs accordingly upon the events (Push, Commit, PR opened etc)
Using Remote JenkinsFile provider plugin, you may choose to place your Jenkinsfile elsewhere in another Proj/repo and call them from this config.
This Jenkinsfile can have all the steps you need and will run the same for all the branches which run as individual MB jobs.
More details on the same can be obtained from here.
Now if you want to use individual jenkinsfiles, then you might have to come up with having Jenkinsfiles specific to each repo which might make it complicated and your Jenkinsfile at the root folder level will have to call the Jenkinsfile present in your repo/branch level across all the repos and branches.
Hopefully this helps! :)
I'm in the process of migrating git-based projects to use a shared Pipeline definition from a governance tier built with Jenkins Templating Engine.
In the process of testing I cloned the project and pushed it to a new repository in Bitbucket where it was recognized by Jenkins and the template was used immediately based on the definitions in pipeline_config.groovy. However, this is not a sane migration path for existing projects. How do I get Jenkins to start using the template on branches without Jenkinsfile and the Jenkinsfile on branches with a Jenkinsfile.
The result of of "Scan Multibranch Pipeline Now" according to the logs is ‘Jenkinsfile’ not found. Skipping. I assume that a new project regonizer is provided by the Jenkins Templating Plugin.
I assume that every project with Git Flow has to perform this migration, so I'm confused there's no documentation.
I'm using Jenkins 2.306 and JTE plugin 2.3.
It seems that newer versions of JTE or Jenkins allow a mixture of branches with JTE and without.
In case you have to deal with versions that don't do the following:
Once I removed Jenkinsfile from every branch and put a pipeline_config.groovy on every branch the Multibranch Plugin started recognizing the project first as removed and at the next scan as present with all branches which were all using the Jenkins Template Engine.
Not the best migration imo, but a great opportunity to cleanup old branches. Since my project was using Git Flow I needed to make a technical hotfix release to also migrate away from Jenkinsfile on master.
We are starting to develop CI workflow for our systems in my company.
Currently we just making few basic tasks like build, tests, and upload to Nexus.
The tech stack is a Java project which build in Gradle and Jenkins makes our build.
Currently i'm working with some basic Groovy script to make what we need, but each time i'm copy and paste my updated code to Jenkins and running the job from Jenkins UI to see the results, and to me it seems like not a very good approach for developing such automation code.
My question is, what is the best practice to build and run Jenkins jobs?
Is it possible to run it straight from Intellij ?
Do we need to create a Jenkins project which should be saved as a repository and then deploy it to Jenkins machine?
Do we need to use some Intellij plugins in order to work with Jenkins?
More best practices are welcome :)
Jenkins has an API - so you can do whatever you want!
But in general, for small to medium teams it's better to use Jenkinsfile and let Jenkins pull code changes (or pull-requests) from SCM and trigger builds. You can also configure hooks to trigger builds if your SCM supports this (Github & bitbucket supports this).
If you are eventually pushing your artifacts to a docker image, I would highly recommend docker multi-stage builds.
If you are completely new to CI/CD stuff - Atlassian has a lot of good resources https://www.atlassian.com/continuous-delivery/principles/continuous-integration-vs-delivery-vs-deployment
I am keeping all my code in SVN repository within my on-premise server. And also I am trying to implement the CI/CD pipeline for deploying my application. I am trying to use Kubernetes and Jenkins tools for implementing this. When I am exploring the implementation examples of CI/CD pipeline using Jenkins and Kubernetes, I am only seeing example with GIT repository and managing code commits using Webhooks.
Here my confusion is that, I am using SVN code repository. So How I can use my SVN code repository with Jenkins Pipeline Job ? Do I need to install any additional plugin for SVN ? My requirement is that, when I am committing into my SVN code repository, Jenkins need to pull code from code repo and need to build project and need to deploy in test environment.
Hooks to trigger Jenkins from SVN are also possible. Or you can poll the repository for changes - the Jenkins SVN plugin supports both methods (https://wiki.jenkins.io/display/JENKINS/Subversion+Plugin). The examples you are looking at will have a step that does a build from the source code of a particular repo. You should be fine to swap git for SVN and still follow the examples as where and how the source is hosted is not normally related to how to use Jenkins to build and deploy it.
How do you maintain the Jenkins job configuration in SCM along side the source code?
As source code evolves, so does the job configuration. It would be ideal to be able to keep the job configuration in SCM, for the following benefits:
easy to see who a history of the changes, including the author and the description
able to rebuild old branch/tag by checking out the revision and build just work
not having to scroll through the UI to find the appropriate section and make change
I see there is a Jenkins Job Builder plugin. I prefer a solution along the lines of Travis CI, where the job configuration is maintained in a YAML file (.travis.yml). Any good suggestions?
Note: Most of our projects are using Java & Maven.
Update 2016: Jenkins now provides a Jenkinsfile which provides exactly this. This is supported by the core Jenkins developers and actively developed.
Benefits:
Creating a Jenkinsfile, which is checked into source control, provides a number of immediate benefits:
Code review/iteration on the Pipeline
Audit trail for the Pipeline
Single source of truth for the Pipeline, which can be viewed and edited by multiple members of the project.
I've written a plugin that does this!
Other than my plugin, you have some (limited) options with existing Jenkins plugins:
Use a single test script
If you configure your Jenkins to simply run:
$ bash run_tests.sh
You can then check in a run_tests.sh file into your SCM repo and you're now tracking changes for how you run tests. However, this won't track configuration of any plugins.
Similarly, if you're using Maven, the Maven Project Plugin simply runs a specified goal for your repo.
The Literate Plugin does allow Jenkins to run the commands in your README.md, but it hasn't yet been released.
Track changes to Jenkins configuration
You can use the SCM Sync configuration plugin to write configuration changes to SCM, so you at least have a persistent record. This is global, across all projects on your Jenkins instance.
There's also the job config history plugin, which stores config history on the filesystem.
Write Jenkins configuration from SCM
The Jenkins job builder project you mentioned lets you check config changes into SCM and have them applied to your Jenkins instance. Again, this is across all projects on your Jenkins instance.
Write Jenkins configuration from another job
You can use the Job DSL Plugin with a repo of groovy scripts. Jenkins then polls that repo, executes the groovy scripts, which create job configurations.
Discussions
Issue 996 (now closed) discusses this, and it has also been discussed on the mailing list: 'Keeping track of Hudson's configuration changes', and 'save hudson config in svn'.
you can do this all with the workflow plugin and a lot more. Workflow is one of the most advanced technics to use jenkins and it has a very strong support.
It is based on a groovy DSL and allows you to keep the whole configuration in the SCM of your choise (e.g. GIT, SVN...).