How can we configure a Jenkins job for different environment. We are using a git repository with different branch like master , test and devl and these environments are deployed in different servers. I have configured build with parameter option, so I can build any of the branch from this job using the radio button.
When I choose devl branch the job need to take latest code from devl branch and build it then need to deploy into devl server. If we choose test, it will need to deploy into test server. How do we configure this multiple deployment within this same job?
You can use the https://wiki.jenkins.io/display/JENKINS/Pipeline+Multibranch+Plugin.
This plugin will watch all your branches of your repo and builds the different branches based on a Jenkinsfile.
In the Jenkinsfile you can use the when expression:
stage('Deploy devl') {
when {
expression { env.BRANCH_NAME == 'devl' }
}
steps {
sh 'deploy devl'
}
}
Why did you choose to do it in 1 job, and not different jobs?
What tool do you use for deployment?
What I can suggest - not getting too deep inside the details (as I don't have them).
Have a job for each of the different environments, connect each job to the relevant git repository and branch, whenever this job is built, it will perform an scm checkout (you can also trigger it on change using hooks) and deploy to the relevant environment.
If you would answer #2 I may be able to suggest other options to manage this (possibly under the same job - if you require this for some reason).
Related
We have our pipelines groovy scripts for Jenkins in SCM (git). I believe it currently gets the scripts from master as default.
Are we able to specify the branch we want to use for the groovy scripts?
There is a setting in the particular section but that seems to be for build branch if I understand correctly (as it allows for setting of multiple branches)
It appears that the branch option is the branch for the pipeline script.
I assume that multiple branches means you can run multiple branches of the different versions of that script. I suspect I am missing when that would be used but it does answer my original question.
We are currently using Windows \ Jenkins 2.107.1 (no pipeline), and I am researching going to pipeline. We have a nightly build job, that fetches from repositories, and submits and waits on other jobs. I see 9 jobs running on the same Master node (we only have a master), at the same time. I am not clear on if we should have one Jenkinsfile or multiple Jenkinsfiles. It will not be a multibranch pipeline, as we do not create test branches and then merge back to a master. In the repository we have product1.0 branch, product2.0 branch etc, and build only one branch (the latest one). While I do like the Blue Ocean editor, it is only for MultiBranch pipelines.
Do I combine all the jobs into one Jenkinsfile, or create multiple jenkins files for each of the existing jobs (Jenkinsfilestart, JenkinsfileFetchCVs, JenkinsFileFetchGit, Jenkinsfilenextjob,etc., and have one call the other)?. Do I create all the old jobs as Jenkinsfiles, or scripts executed by the one master Jenkinsfile? Do I do this in Declarative or script ?
Have set up Jenkins pipeline on test VM, but not clear on which way to go yet.
Looking for directions and\or examples. Is there documentation on how to convert existing Jenkins non-pipeline systems?
I found this after doing the initial post...https://wiki.jenkins.io/display/JENKINS/Convert+To+Pipeline+Plugin.
It does help a little in that it gives you some converted steps, but cannot convert all the steps, and will give comments in the pipeline script "//Unable to convert a build step referring to...please verify and convert manually if required." There is an option "Recursively convert downstream jobs if any" and if you select that, it appears to add all the downstream jobs to the same pipeline script, and really confuses the job parameters. There is also an option to "Commit JenkinsFile". I will play with this some more, but it is not the be all and end all of converting to pipeline, and I still am not sure of whether I should be have one or more scripts.
Added 07/26/19 -
Let’s see if I have my research to date correct…
A Declarative pipeline (Pipeline Script from SCM), is stored in a Jenkinsfile in the repository. Every time that this Jenkins job is executed, a fetch from the repository is done (to get the latest version of the Jenkinsfile).
A Pipeline script is stored as part of the config.xml file in the Jenkins\Jobs folder (it is not stored in the repository, or in a separate Jenkinsfile in the jobs folder). There is a fetch from the repository only if the job requires it (you do not need to do a repository fetch to get the Pipeline script).
Besides our nightly product build, we also have other jobs. I could create a separate Declarative Jenkinsfile for each of them (JenkinsfileA, JenkinsfileB, etc.) for each of the other jobs and store then in the repository also (in the same branch as the main Jenkinsfile), but that would mean that every one of those additional jobs, to get the particular Jenkinsfile for that job, would also need to do a repository fetch (basically fetching\cloning the repository branch for each job, and have multiple versions of the repository branch unnecessarily downloaded to the workspace of each job).
That does not make sense to me (unless my understanding of things to date is incorrect). Because the main product build does require a fetch every time it is run (to get any possible developer check-ins), I do not see a problem doing Declarative Jenkinsfile for that job. For the other jobs (if we do not leave then for the time being in the classic (non-pipeline) format)), they will be Pipeline scripts.
Is there any way of (or plans for), being able to do Declarative pipeline without having to store in the repository and doing a fetch every time (lessening the need to become a Groovy developer)? The Blue Ocean script editor appears to be an easier tool to use to create pipeline scripts, but it is only for MultiBranch pipelines (which we don’t do).
Serialization (restarting a job), is that only for when a node goes down, or can you restart a pipeline job (Declarative or Scripted), from any point if it fails?
I see that there are places to look to see what Jenkins plugin’s have been ported to pipeline, but is there anything that can be run to take a look at the classic jobs that you have, to determine up front which jobs are going to have problems being converted to pipeline?
08/02/19...
Studying and playing with pipelines. I see that you can use Declarative in the Pipeline Scrip window, but it still stores it in the config.xml file. And I have played with the combination of both Declarative and non Declarative in the same script.
I am trying to understand the Blue Ocean interface, the word "MultiBranch" is throwing me a little. We do not create test branches, and them merge them back into the master. In the repository, we have branches for each release of the product, and we rarely go back to previous branches\versions. So, if I am working on branchV9 right now, do I also need a Jenkinsfile in the Master branch, or any other of the previous version branches?
I have been playing with Blue Ocean (which only does MultiBranch pipelines). I am on a Windows system, Jenkins 2.176.2, and have all the latest Blue Ocean plugins as of today (1.18.0). I am accessing a local Git repository (not GitHub), and am running into the following...
If I try to use use “c:\GitRepos\Pipelines1.git”, i get "not a valid name"...
Why does it do this?
If you have a single job that you would be executed on multiple branches (with possibly optional stages, depending on the branch name or tag or other) then you still could utilize multi branch pipeline.
In general I would say that paradigm shift focuses mainly on converting the old jobs to stages in order to automate your build process. If you would have semi/fully automated CI/CD flow this could look like
Multibranch pipeline project (all branches) with the following stages (1st jenkinsfile)
build (all branches)
unit tests (all branches) publish report
publish artifacts (master and release branches)
build and publish docker (master and release branches)
deploy to test (master and release branches)
run integration tests (master and release branches)
deploy to staging (master and release branches) possibly ending with manual step if result of deployment was as expected
deploy to production (release branches)
Pipeline job for nightly tests (other jenkinsfile), what's the result here? Would it break CI/CD flow?
Can i use Jenkins integration for DevOps Continuos Delivery Pipeline for JavaScript code builds?
I am trying to build/propose a solution for integrating Jenkins tool and to remove manual code build and deployments to remove the manual effort for my team.
Generally in Jenkins, you can use the below:
Build -> Automated Test -> Dev Deploy -> QA Approval -> QA Deploy
The Seed Job is the one which will be creating other Jenkins job automatically but the seed job itself will be configured manually. The Seed Job will read the DSL script, parse those and create appropriate Job configurations in Jenkins.
After the seed job runs successfully, we will have a job created for our sample app.
The Seed job will create the following set of jobs which will eventually be part of the pipeline. The Seed job will also create a Jenkins pipeline view.
Build: This job includes the configuration for the building project, job triggers, scm location, jdk version to use, maven goals, artifact upload to repo like Artifactory.
Test: This job can call test suites and decide to call a downstream job or not.
Dev Deploy: Simple job with a trigger to the Promotion Job if the deployment was successful.
This job can call the script to perform a deployment or use tools like Bamboo or Urbancode.
Usually a Dev deploy doesn’t need promotion, but we can add that step if required.
QA Promotion: This job includes a send email notification to the person/group responsible for approval. The email contains a link for promotion.
The Promotion Email link can look like this: http://localhost:8080/XXX/XXXXX/XXX
The same can be done for the UAT and Prod:
We can chain multiple Promotion Jobs and Deploy jobs to accomplish the need for another environment, for e.g. UAT Promotion -> UAT Deploy ->PreProd Promotion -> PreProd Deploy -> Prod Promotion -> Prod Deploy
And here for all the above mentioned processes it can be done via Jenkins tool
Also yes, to answer your question you definitely can use Jenkins integration in Devops Pipeline.
If you are building your solution in cloud or in any domain servers then you might have to get the Jenkins integrated in the same environment.
I am currently using Cloudbees Jenkins Coreas my Jenkins solution.
I am using Jenkins Pipelines to write our Jenkins job configuration. These pipelines are stored in GitHub repositories. Each Jenkins job when created is connected to a GitHub Repository where the source code is pulled from, and that's where the Jenkinsfile is stored and Jenkins reads from.
Below are some high-level photos for how our Jenkins jobs are configured.
The advantage of the way these jobs are configured is the Jenkinsfile is always read from the master branch. Meaning if a rouge developer tries to remove stages from the Jenkinsfile from within there own branch, it doesn't matter because the Jenkinsfile is always read from the master branch (which is always protected).
However, the one massive drawback to this - is how do teams and developers who are devops engineerings make changes to the Jenkinsfile? For example, let's say a developer creates a branch called feature-jenkins-search and they edit the Jenkinsfile adding a new stage in the pipeline. Whenever they push these changes to GitHub to test - they can't test as it's always read from the master branch? Meaning devops engineerings have to work directly on the master branch? Surely this is not the best way to go and there is a better configuration to set?
We do want to still provide the security that if a developer is rougue and
You should really look into the Jenkins multi-branch pipeline feature. The Jenkins multi-branch pipeline allows to create a single configuration item in Jenkins (a bit like a folder) that can detect all the branches and pull requests in a GitHub repository with a Jenkinsfile and build them using automatically created jobs. Inside this multi-branch pipeline object when it is configured in Jenkins, you will find a number of jobs to build the various branches and pull-requests in the GitHub repository.
So your developers should maintain a Jenkinsfile in every branch they work on in GitHub to build that branch in your Jenkins server.
It is possible to make the Jenkinsfile do branch specific handling if required with conditional stages / when conditions in the Jenkinsfile pipelines in each branch.
You can lock down the master branch so that code and Jenkinsfile changes from other branches can only be merged with an approved PR (pull request). There is good integration between Jenkins and GitHub such that you can configure the master branch to only allow a PR to be merged if the PR is buildable in Jenkins. So if developers add new stages / processing to a Jenkinsfile on a branch being merged to master, it should be validated so that builds of your master branch are not broken.
There is a lot of configurability in the Jenkins multi-branch pipeline object for detection and handling of branches and it may be necessary to experiment to get it right for what you need with your team. If you cannot find this feature in Jenkins, it is probably because the correct Jenkins pipeline and GitHub related plugins are not installed.
You could also have a look at a similar Jenkins feature called the Jenkins GitHub Organization Folder which allows to detect and build all repos and branches at a GitHub Organization level. But when starting out, I would suggest to look into the multi-branch pipeline at the single repo level first.
These features are discussed in the Jenkins pipeline documentation. We use these features with our internal GitHub and Jenkins server and it works very well.
I think you will find the idea of using a single Jenkinsfile in the master branch to be used for building all branches is unworkable, as you have seen!
In Jenkins, I have a "Build" job setup to poll my git repo and automatically build on change. Then, I have separate "Deploy to DEV", "Deploy to QA", etc. jobs that will call an Ant build that deploys appropriately. Currently, this configuration works great.
However, this process favors deploying the latest build on the latest development branch. I use the Copy Artifact plugin to allow the user to choose which build to deploy. Also, the Ant scripts for build/deploy are part of the repo and are subject to change. This means it's possible the artifact could be incompatible between versions. So, it's ideal that I ensure that the build and deploy jobs are run using the same git checkout.
Is there an easier way? It ought to be possible for the Deploy job to obtain the git checkout hash used from the selected build and checkout. However, I don't see any options or plugins that do this.
Any ideas on how to simplify this configuration?
You can use Parameterized Trigger Plugin to do this for you. The straight way is to prepare file with parameters as a build step and pass this parameters to the downstream job using the plugin. You can pass git revision as a parameter for example or other settings.
The details would vary for a Git repo (see https://stackoverflow.com/a/13117975/466874), but for our SVN-based jobs, what we do is have the build job (re)create an SVN tag (with a static name like "LatestSuccessfulBuild") at successful completion, and then we configure the deployment jobs to use that tag as their repo URL rather than the trunk location. This ensures that deployments are always of whatever revision was successfully built by the build job (meaning all unit tests passed, etc.) rather than allowing newer trunk commits to sneak into the deployment build.