I am working on a multibranch pipeline Jenkins setup and build is triggered using webhook in Git.. Here I have selected Git Branch source as - Git.
When I push any change in git, webhook creates a request body with all push event details. How can I parse "git_http_url" value from this( which will have my git repo url). This value I can then use as ${myrepourl} in jenkins console. Basically I want avoid hardcoding the repo url, it should dynamically take using this parameter.
Please guide.
![webhook request body screenshots][2]][2]
[![attached my jenkins console branch source][1]][1]
[1]: https://i.stack.imgur.com/sdb0l.png
[2]: https://i.stack.imgur.com/icPP9.png
This looks like a not-very-good idea to begin with. I will start by explaining why, outlining the alternatives, and in the end suggesting a solution that might still work if you insist on doing this.
When you configure your pipeline, you need to provide it with Jenkinsfile. It can be pasted inside the configuration ("Pipeline script"), or you can provide a path to it so Jenkins can perform a checkout ("Pipeline script from SCM"). When doing the former, you have one Jenkinsfile, so different branches can't alter it (and so missing the point of having a multibranch). When doing the latter, even if you can parametrize the git repo, you still need to provide a path (as it won't arrive in github notification). In addition, I can trigger your build with my repo, but chances are your pipeline won't be able to properly build my repo anyway. So your pipeline can only build your repo, at which point it's a bit unclear why you insist on not providing your specific pipeline with a path to your specific repo that it specifically knows how to build.
Most people who need multibranch pipelines over Github use one of the plugins specifically written for this purpose, e.g. Github Multibranch plugin or Github organization. These plugins do all the job themselves: they sign up for notifications, process them, and start builds. They also update build status in Github for you.
Finally, if you insist on processing Github notifications yourself, you can use Generic Webhook Trigger plugin that will allow you to trigger the job by POST-ing to a specified URL with a token. This may look like this:
pipeline {
agent { node { label 'master'} }
triggers {
GenericTrigger(causeString: 'Generic Cause',
genericVariables: [
[key: 'DATA', value: '$'], //JSONPath expression meaning "everything"
[key: 'GITHUB_URL', value: '$.project.git_http_url']
],
printContributedVariables: false, // change to 'true' to see all the available variables in the console
printPostContent: false, // change to 'true' to see the dump of the POST data
silentResponse: false,
token: 'my_token')
}
As per the first configuration line, any JSON posted will get flattened and turned into pipeline variables, with a prefix you define (in this case, "DATA_"). E.g. the field git_http_url inside the field project in Github payload will be defined in the pipeline and available to you as DATA_project_git_http_url. As per second configuration line, it will also be available as GITHUB_URL.
You can test your pipeline with e.g.
curl -XPOST -H "Content-Type: application/json" 'http://<jenkins>/generic-webhook-trigger/invoke?token=my_token' --data '{"hello": "world"}'
In this case, the contributed variable will be DATA_hello and it will have the value of world. (The GITHUB_URL variable, naturally, won't be defined.)
If you want to turn this into real Github webhook processor, you need to make sure that Github notifs arrive to <jenkins>/generic-webhook-trigger/invoke?token=my_token. We use nginx for that, but there are many other options.
Related
I currently have a git repo which has a text file. I want to load its contents as a build parameter for a jenkins job.
One way would be to manually copy the contents of this file in Jenkins multi-line string parameter. But, since the content is in git already I want to keep it coupled.
Not sure, if this is even possible using Jenkins?
I am using Jenkins Job DSL to generate the job.
EDIT : You can find several different ways of achieving this in the following answer Jenkins dynamic declarative pipeline parameters
I think you can achieve it the following way (scripted pipeline).
node {
stage("read file") {
sh('echo -n "some text" > afile.txt')
def fileContent = readFile('afile.txt')
properties([
parameters([
string(name: 'FILE_CONTENT', defaultValue: fileContent)
])
])
}
stage("Display properties") {
echo("${params.FILE_CONTENT}")
}
}
The first time you execute it there will be no parameter choice. The second time, you'll have the option to build with parameter and the content will be the content of your file.
The bad thing with this approach is that it's always in sync with the previous execution, i.e. when you start the build on a commit where you changed the content of your file, it will prefill the parameter with the content of the file as per the last execution of the build.
The only way I know around this, is to split your pipeline into two pipelines. The first one reads the content of the file and then triggers the second one with the file content as build parameter with the build step.
If you find a better way let us know.
Why don't you have jenkins pull the repo as part of the Job, and then parse the contents of the parameters (in say, json, for example from a file within the repo) and then continue executing with those parameters?
On Mercurial I've implemented a hook in my hgrc file that activates when some sort of change occurs in Jenkins(i.e tagging or committing). Here is my hook code:
curl -X POST http://tokenusername:115d59a462df750d4f12347975b3d691cf#127.0.0.1:8080/job/pipelinejob/buildWithParameters/mercurial/notifyCommit?url=http://127.0.0.1:85/hg/experimentrepoistory?token=1247
So there's no issue with my hook notifying Jenkins that a change has occurred and the pipeline executes but for some reason I am having trouble getting the commit id or any or the author name's who made the commit etc. I went to the script console in jenkins and wrote the following code in groovy to see if the changeset data from Mercurial transferred over to Jenkins. Also all the libraries are imported
def job = hudson.model.Hudson.instance.getItem("pipelinejob")
def builds = job.getBuilds()
def thisBuild = builds[0]
println('Lets test Mercurial fields ' + thisBuild.getEnvironment()['MERCURIAL_REVISION']) //Lets test Mercurial fields null
It makes me think that MERCURIAL_REVISION for some reason wasnt defined even though I provided a job that has the changeset info. I was reading this documentation https://javadoc.jenkins.io/plugin/mercurial/hudson/plugins/mercurial/MercurialChangeSet.html#MercurialChangeSet-- that lists a bunch of functions that have alot of functions like getCommitId() getNode() etc that gets the information that I need. Problem is I'm not entirely sure how to instantiate MercurialChangeSet with the jenkin jobs pipelinejob that in theory should have the Mercurial commitId information. Thats why I wanted to know if I perhaps missed something obvious regarding accessing MERCURIAL_REVISION
So I found out that I need to enable the Pipeline script from SCM and that I need to put the Jenkinsfile with the pipeline code inside my workspace directory in order to get the changeset information. I am not entirely sure why this works since I would think the Jenkinsfile needs to be in the repo directory of the SCM
I want to set up a pipeline that is triggered automatically by pull requests for a GitHub project and then builds all the repositories in it. I found this article and followed the instructions as it was similar to what I required, but I'm currently stuck in getting the pipeline to trigger and build multiple repositories in the same GitHub project every time a PR is made for even one of the repositories.
I've attached this diagram to bring more clarity into my issue.
So the goal is when a pull request is made on the Branch 3 of Repository 1 the pipeline is triggered which builds that branch and all the other repositories in a specified order, i.e. Repository 2, Repository 3, etc. of the Working Project.
Your help would be much appreciated and I think a solution for this would benefit the CI DevOps community very much. Thanks!
Give the following a try - can't promise the below will be exact but should get you in the right direction.
The first thing you want to do is have a consistent Jenkinsfile across each of the repositories, now you could do this by a number of different manners but one way to accomplish this would be to use external groovy pipelines so that the logic can be kept consistent across the repos. An example of this is located here.. Copying the Jenkinsfile across each of the repositories would also work, however a single source of truth is generally a better approach.
node{
deleteDir()
git env.flowScm
def flow = load 'pipeline.groovy'
stash includes: '**', name: 'flowFiles'
stage 'Checkout'
checkout scm // short hand for checking out the "from scm repository"
flow.runFlow()
}
Where the pipeline.groovy file would contain the actual pipeline would look like this:
def runFlow() {
// your pipeline code
}
// Has to exit with 'return this;' in order to be used as library
return this;
Once you've got each of your triggers using the same pipeline logic you can take advantage of the dir command to clone and work with repositories that are not the one that triggered the build. An example of this is located here.
node('ATLAS && Linux') {
dir('CalibrationResults') {
git url: 'https://github.com/AtlasBID/CalibrationResults.git'
}
dir('Combination') {
git url: 'https://github.com/AtlasBID/Combination.git'
}
dir('CombinationBuilder') {
git url: 'https://github.com/AtlasBID/CombinationBuilder.git'
}
sh('ls')
sh('. CombinationBuilder/build.sh')
}
Putting the two steps together should achieve what you are after in this instance.
I've configured a service hook in VSTS to connect to Jenkins. I'm able to use the Jenkins extension to trigger a Jenkins job if it's not in a subfolder. E.g. http://myhost/job/Always%20succeed/
In that case, I can simply connect like this and run my job:
If my job is nested, however, I can't figure out how to connect. Here's an example: http://myhost/view/Production/job/Automation/job/Test/job/My%20Job
I've tried using just the name (e.g. "My Job"), the whole url, and a dot notation (Production.Automation.Test.My Job). How can I make this run and where can I find more documentation?
It's pretty nuanced and one could argue buggy. First off, I can reach the same job with two urls:
http://myhost/view/Production/job/Automation/job/Test/job/My%20Job
http://myhost/job/Automation/job/Test/job/My%20Job
Turns out the latter is the way to go.
I tried the following name, and it tried reaching the corresponding endpoint:
Automation/job/Test/job/My%20Job <- name used in VSTS "Job name" field
/job/Automation/job/job/Test/job/My%20Job/build <- url attempted, failed (404)
Note the double job/. Then I tried the following with better results:
Automation/Test/job/My%20Job <- name used
/job/Automation/job/Test/job/My%20Job/build <- url tried, success
It's a bit concerning that the pattern isn't consistent regarding the double "job/" part but whatever.
I have projects in Jenkins, by the names triger-Job-DEV, triger-Job-SYS, etc. I need to invoke these jobs when I get approval in the jira. The jira ticket contains the environment name(DEV, SYS, etc.), and there is seperate jira ticket for each environment. How to use the environment value in jira as variable in web hooks for calling the Jenkins job. I have tried trigger-Job-${Environment} as the job name but it is not invoking the Jenkins job.
Check out the Submitting jobs section of the Remote Access API
You can do a POST to get a job started. Try this first:
curl -X POST JENKINS_URL/job/triger-Job-DEV/build?token=TOKEN
If that works, its simply a matter of parameterizing the job name.
NOTE: I noticed that your jira jobs are misspelled as triger-Job-XXX (trigger with 1 g), and you are calling your Jenkins job with the correct spelling of trigger with 2 gs :)
EDIT: From your comment, I understand that DEV, SYS etc is stored as a custom defined variable for each jira task. In that case, this doc has a variable substitution section: https://developer.atlassian.com/static/connect/docs/latest/modules/common/webhook.html
and here they specify that you can use something like {variable.name} to get that value.
https://developer.atlassian.com/static/connect/docs/latest/concepts/context-parameters.html
So I assume your URL would look something like :
curl -X POST JENKINS_URL/job/triger-Job-{variable.<your_custom_field_name>}/build?token=TOKEN
I have never tried this, so let me know if it works