How can i read the payload received from gitlab webhook?
I know there is plugin which store some info from json in variables, but thats not enough for me. For some weird reasons im not allowed to use generic webhook plugin.
gitlabBranch
gitlabSourceBranch
gitlabActionType
gitlabUserName
gitlabUserUsername
gitlabUserEmail
gitlabSourceRepoHomepage
gitlabSourceRepoName
gitlabSourceNamespace
gitlabSourceRepoURL
gitlabSourceRepoSshUrl
gitlabSourceRepoHttpUrl
gitlabMergeRequestTitle
gitlabMergeRequestDescription
gitlabMergeRequestId
gitlabMergeRequestIid
gitlabMergeRequestState
gitlabMergedByUser
gitlabMergeRequestAssignee
gitlabMergeRequestLastCommit
gitlabMergeRequestTargetProjectId
gitlabTargetBranch
gitlabTargetRepoName
gitlabTargetNamespace
gitlabTargetRepoSshUrl
gitlabTargetRepoHttpUrl
gitlabBefore
gitlabAfter
gitlabTriggerPhrase
Just wondering, if anyone has ever tried to read the whole payload received from gitlab webhook
Related
I am trying to configure Jenkins integration in GitLab, as it is recommended in favor of webhooks.
On the input form ([Gitlab project url]/-/integrations/jenkins/edit), I can check: Merge request, which has this note:
Trigger event when a merge request is created, updated, or merged.
I would like to distinguish between merge requests being created or updated on one hand and being merged on the other hand. Only when they are being merged, I want Jenkins to perform a deployment.
Can I detect in Jenkins what type of merge request event triggered the pipeline?
You should be able to figure this out with the variables the Gitlab plugin exposes after the Webhook request. Here is the list of variables available.
gitlabBranch
gitlabSourceBranch
gitlabActionType
gitlabUserName
gitlabUserUsername
gitlabUserEmail
gitlabSourceRepoHomepage
gitlabSourceRepoName
gitlabSourceNamespace
gitlabSourceRepoURL
gitlabSourceRepoSshUrl
gitlabSourceRepoHttpUrl
gitlabMergeRequestTitle
gitlabMergeRequestDescription
gitlabMergeRequestId
gitlabMergeRequestIid
gitlabMergeRequestState
gitlabMergedByUser
gitlabMergeRequestAssignee
gitlabMergeRequestLastCommit
gitlabMergeRequestTargetProjectId
gitlabTargetBranch
gitlabTargetRepoName
gitlabTargetNamespace
gitlabTargetRepoSshUrl
gitlabTargetRepoHttpUrl
gitlabBefore
gitlabAfter
gitlabTriggerPhrase
I want to setup Jenkins pipeline trigger when PR is merged to master branch. I have setup Webhook in GitHub repo pointing to Jenkins url http://jenkins.example.com:8080/github-webhook/ and selected following events
Pull request review comments
Pull request reviews
Pull requests
in my Jenkinsfile I use this
triggers {
pullRequestReview(reviewStates: ['approved'])
}
But it failed with this error
WorkflowScript: 6: Invalid trigger type "pullRequestReview". Valid trigger types: [upstream, cron, parameterizedCron, GenericTrigger, githubPush, pollSCM] # line 6, column 9.
If I want to trigger the build when PR is merged to master, what I should user in triggers ?
Here is what you need:
GenericTrigger(
genericVariables: [
[key: 'action', value: '$.action'],
[key: ‘merged, value: '$.pull_request.merged]
],
causeString: 'Triggered on pr merge,
token: ‘<your-token>’,
printContributedVariables: true,
printPostContent: true,
silentResponse: false,
regexpFilterText: '$action#$merged,
regexpFilterExpression: ‘closed#true'
)
}
And you don't need to select Pull request review comments and
Pull request reviews events. Just Pull requests is enough for this case.
I have bin configuring a globally shared library correctly as it is shown to me in the global configuration of the Jenkins Server. In a Pipeline Job I try to use the specified library, but I get the error message, that no credentials were used to checkout the library from SCM.
Library Configuration
Name: LIBRARY
Default version: trunk -- Currently maps to revision: 125
Load implicitly: [ ]
Allow default version to be overridden: [x]
Include #Library changes in job recent changes: [x]
Modern SCM (o)
Legacy SCM ( )
Git ( )
GitHub ( )
Mercurial ( )
Subversion (o)
Project Repository Base: http://scrambled/url/repo/LIBRARY
Credentials: Scrambly/******
Include branches: trunk, tags/*
Pipeline
#Library('LIBRARY#trunk')
pipeline {
// Imagine some content //
}
Error Message
Checking out http://scrambled/url/repo/LIBRARY at revision 123
Using sole credentials <none> in realm ‘<http://scrambled> Scrambler’
ERROR: Subversion checkout has been canceled
org.tmatesoft.svn.core.SVNCancelException: svn: E200015: ISVNAuthentication provider did not provide credentials; HTTP authorization cancelled.
at org.tmatesoft.svn.core.internal.io.dav.http.HTTPConnection.request(HTTPConnection.java:694)
Caused: org.tmatesoft.svn.core.SVNCancelException: svn: E200015: E200015: ISVNAuthentication provider did not provide credentials; HTTP authorization cancelled.
svn: E200015: ISVNAuthentication provider did not provide credentials; HTTP authorization cancelled.
I do not understand it says there were <none> credentials in the error message, although I configured the shared library to use specific credentials.
Thank you all in advice.
i fix this problem by using https git url :
https://{username}:{passwd}#xxxxx.git
I have been following the below links in order to integrate Gitlab with Jenkins using web hooks. All the below links mention to receive 'JSON' or 'payload' or token over at Jenkins side, but I do not see anything when I try to echo or print these parameters in the Shell script of Jenkins configurations.
In shell script I have this, but I never receive any payloads:
echo "the build worked! The payload is $payload"
I do see some JSON coming through on /var/log/Jenkins/Jenkins.logs, but I want to see the messages coming in inside my 'Console Output', so that I can use the messages coming in from Gitlab to whether trigger a build or not.
Most of these links mention options are not available via Gitlab. One article was mentioning to convert web hook format to application/json, but there are no such options on the Gitlab UI.
How to process a github webhook payload in Jenkins?
http://chloky.com/github-json-payload-in-jenkins/
Jenkins Settings:
Gitlab webhook:
http://xx.xx.xx.xxx:8080/job/Interim_Build/buildWithParameters?token=TOKEN_NAME
Any help would be great. Thanks.
I suggest you to try two solutions (both working for me):
convert json data from Gitlab webhook using this elegant proxy written in Go https://github.com/akira/githookproxy .
It will take the webhook request, and translate it to a request to the target_url in the format of:
payload: JSON body
START: Start commit hash
END: End commit hash
REFNAME: Ref name
emulate jenkins as a Gitlab CI using this Jenkins plugin https://github.com/jenkinsci/gitlab-plugin
For me the best is the first because it is simple and more transparent.
GitLab and GitHub are two separate products. So, the documentation or links for GitHub webhooks that you are referring will not apply to GitLab webhooks.
GitLab invokes the webhook URL with a JSON payload in the request body that carries a lot of information about the GitLab event that led to the webhook invocation. For example, the GitLab webhook push event payload carries the following information in it:
{
"object_kind": "push",
"before": "95790bf891e76fee5e1747ab589903a6a1f80f22",
"after": "da1560886d4f094c3e6c9ef40349f7d38b5d27d7",
"ref": "refs/heads/master",
"checkout_sha": "da1560886d4f094c3e6c9ef40349f7d38b5d27d7",
"user_id": 4,
"user_name": "John Smith",
"user_username": "jsmith",
"user_email": "john#example.com",
"user_avatar": "https://s.gravatar.com/avatar/d4c74594d841139328695756648b6bd6?s=8://s.gravatar.com/avatar/d4c74594d841139328695756648b6bd6?s=80",
"project_id": 15,
"project":{
"id": 15,
"name":"Diaspora",
"description":"",
"web_url":"http://example.com/mike/diaspora",
"avatar_url":null,
"git_ssh_url":"git#example.com:mike/diaspora.git",
"git_http_url":"http://example.com/mike/diaspora.git",
"namespace":"Mike",
"visibility_level":0,
"path_with_namespace":"mike/diaspora",
"default_branch":"master",
"homepage":"http://example.com/mike/diaspora",
"url":"git#example.com:mike/diaspora.git",
"ssh_url":"git#example.com:mike/diaspora.git",
"http_url":"http://example.com/mike/diaspora.git"
},
"repository":{
"name": "Diaspora",
"url": "git#example.com:mike/diaspora.git",
"description": "",
"homepage": "http://example.com/mike/diaspora",
"git_http_url":"http://example.com/mike/diaspora.git",
"git_ssh_url":"git#example.com:mike/diaspora.git",
"visibility_level":0
},
"commits": [
{
"id": "b6568db1bc1dcd7f8b4d5a946b0b91f9dacd7327",
"message": "Update Catalan translation to e38cb41.",
"timestamp": "2011-12-12T14:27:31+02:00",
"url": "http://example.com/mike/diaspora/commit/b6568db1bc1dcd7f8b4d5a946b0b91f9dacd7327",
"author": {
"name": "Jordi Mallach",
"email": "jordi#softcatala.org"
},
"added": ["CHANGELOG"],
"modified": ["app/controller/application.rb"],
"removed": []
},
{
"id": "da1560886d4f094c3e6c9ef40349f7d38b5d27d7",
"message": "fixed readme",
"timestamp": "2012-01-03T23:36:29+02:00",
"url": "http://example.com/mike/diaspora/commit/da1560886d4f094c3e6c9ef40349f7d38b5d27d7",
"author": {
"name": "GitLab dev user",
"email": "gitlabdev#dv6700.(none)"
},
"added": ["CHANGELOG"],
"modified": ["app/controller/application.rb"],
"removed": []
}
],
"total_commits_count": 4
}
The Jenkins GitLab plugin makes this webhook payload information available in the Jenkins Global Variable env. The available env variables are as follows:
gitlabBranch
gitlabSourceBranch
gitlabActionType
gitlabUserName
gitlabUserEmail
gitlabSourceRepoHomepage
gitlabSourceRepoName
gitlabSourceNamespace
gitlabSourceRepoURL
gitlabSourceRepoSshUrl
gitlabSourceRepoHttpUrl
gitlabMergeRequestTitle
gitlabMergeRequestDescription
gitlabMergeRequestId
gitlabMergeRequestIid
gitlabMergeRequestState
gitlabMergedByUser
gitlabMergeRequestAssignee
gitlabMergeRequestLastCommit
gitlabMergeRequestTargetProjectId
gitlabTargetBranch
gitlabTargetRepoName
gitlabTargetNamespace
gitlabTargetRepoSshUrl
gitlabTargetRepoHttpUrl
gitlabBefore
gitlabAfter
gitlabTriggerPhrase
Just as you would read Jenkins job parameters from Jenkins Global Variable params in your job pipeline script, you could read webhook payload fields from Jenkins Global Variable env:
echo "My Jenkins job parameter is ${params.MY_PARAM_NAME}"
echo "One of Jenkins job webhook payload field is ${env.gitlabMergedByUser}"
Hope, the above information helps solve your problem.
I followed this tutorial to setup a Jenkins job to run whenever a push is made to the gitlab repository. I tested the webhook and I can see that the job is triggered. However, I don't see anything in the payload.
Just wondering, if anyone has ever tried to read the payload received from gitlab webhook?
Jenkins Gitlab Plugin sends these POST parameters to Jenkins whenever any event occurs in the Gitlab repo.
You can add env in the Jenkins console to get what all Gitlab parameters are exported to the Jenkins environment. Then you can print or use the required variables.
e.g
echo $gitlabSourceRepoURL
echo $gitlabAfter
echo $gitlabTargetBranch
echo $gitlabSourceRepoHttpUrl
echo $gitlabMergeRequestLastCommit
echo $gitlabSourceRepoSshUrl
echo $gitlabSourceRepoHomepage
echo $gitlabBranch
echo $gitlabSourceBranch
echo $gitlabUserEmail
echo $gitlabBefore
echo $gitlabSourceRepoName
echo $gitlabSourceNamespace
echo $gitlabUserName
The tutorial you have mentioned talks about GitHub webhooks. GitLab and GitHub are two separate products. So, the documentation or links for GitHub webhooks will not apply to GitLab webhooks.
GitLab invokes the webhook URL with a JSON payload in the request body that carries a lot of information about the GitLab event that led to the webhook invocation. For example, the GitLab webhook push event payload carries the following information in it:
{
"object_kind": "push",
"before": "95790bf891e76fee5e1747ab589903a6a1f80f22",
"after": "da1560886d4f094c3e6c9ef40349f7d38b5d27d7",
"ref": "refs/heads/master",
"checkout_sha": "da1560886d4f094c3e6c9ef40349f7d38b5d27d7",
"user_id": 4,
"user_name": "John Smith",
"user_username": "jsmith",
"user_email": "john#example.com",
"user_avatar": "https://s.gravatar.com/avatar/d4c74594d841139328695756648b6bd6?s=8://s.gravatar.com/avatar/d4c74594d841139328695756648b6bd6?s=80",
"project_id": 15,
"project":{
"id": 15,
"name":"Diaspora",
"description":"",
"web_url":"http://example.com/mike/diaspora",
"avatar_url":null,
"git_ssh_url":"git#example.com:mike/diaspora.git",
"git_http_url":"http://example.com/mike/diaspora.git",
"namespace":"Mike",
"visibility_level":0,
"path_with_namespace":"mike/diaspora",
"default_branch":"master",
"homepage":"http://example.com/mike/diaspora",
"url":"git#example.com:mike/diaspora.git",
"ssh_url":"git#example.com:mike/diaspora.git",
"http_url":"http://example.com/mike/diaspora.git"
},
"repository":{
"name": "Diaspora",
"url": "git#example.com:mike/diaspora.git",
"description": "",
"homepage": "http://example.com/mike/diaspora",
"git_http_url":"http://example.com/mike/diaspora.git",
"git_ssh_url":"git#example.com:mike/diaspora.git",
"visibility_level":0
},
"commits": [
{
"id": "b6568db1bc1dcd7f8b4d5a946b0b91f9dacd7327",
"message": "Update Catalan translation to e38cb41.",
"timestamp": "2011-12-12T14:27:31+02:00",
"url": "http://example.com/mike/diaspora/commit/b6568db1bc1dcd7f8b4d5a946b0b91f9dacd7327",
"author": {
"name": "Jordi Mallach",
"email": "jordi#softcatala.org"
},
"added": ["CHANGELOG"],
"modified": ["app/controller/application.rb"],
"removed": []
},
{
"id": "da1560886d4f094c3e6c9ef40349f7d38b5d27d7",
"message": "fixed readme",
"timestamp": "2012-01-03T23:36:29+02:00",
"url": "http://example.com/mike/diaspora/commit/da1560886d4f094c3e6c9ef40349f7d38b5d27d7",
"author": {
"name": "GitLab dev user",
"email": "gitlabdev#dv6700.(none)"
},
"added": ["CHANGELOG"],
"modified": ["app/controller/application.rb"],
"removed": []
}
],
"total_commits_count": 4
}
The Jenkins GitLab plugin makes this webhook payload information available in the Jenkins Global Variable env. The available env variables are as follows:
gitlabBranch
gitlabSourceBranch
gitlabActionType
gitlabUserName
gitlabUserEmail
gitlabSourceRepoHomepage
gitlabSourceRepoName
gitlabSourceNamespace
gitlabSourceRepoURL
gitlabSourceRepoSshUrl
gitlabSourceRepoHttpUrl
gitlabMergeRequestTitle
gitlabMergeRequestDescription
gitlabMergeRequestId
gitlabMergeRequestIid
gitlabMergeRequestState
gitlabMergedByUser
gitlabMergeRequestAssignee
gitlabMergeRequestLastCommit
gitlabMergeRequestTargetProjectId
gitlabTargetBranch
gitlabTargetRepoName
gitlabTargetNamespace
gitlabTargetRepoSshUrl
gitlabTargetRepoHttpUrl
gitlabBefore
gitlabAfter
gitlabTriggerPhrase
Just as you would read Jenkins job parameters from Jenkins Global Variable params in your job pipeline script, you could read webhook payload fields from Jenkins Global Variable env:
echo "My Jenkins job parameter is ${params.MY_PARAM_NAME}"
echo "One of Jenkins job webhook payload field is ${env.gitlabTargetBranch}"
Hope, the above information helps solve your problem.
Yes, I did it. And it works for some scenarios.
If you use /gitlab/buildnow, you can have access to payload objects. All of them.
But you have to name them under "this build is parametrized".
Then you can access them by name, like ${AUTHOR_NAME}.
Doc: https://github.com/elvanja/jenkins-gitlab-hook-plugin#parameterized-projects
But please note that if you use /gitlab/notifycommit, it will not work, since there is a gap (the poll) between triggering jenkins, and actually starting the job. All payload data in this situation is empty.
But be carefull to use /gitlab/buildnow, because you cannot control if you want or not to build, like when Maven commit back some files, and build is not supposed to be triggered.
What I did was to write a little tool in Python to receive all gitlab notification, and this tool talks back to GitLab and Jenkins, to fire (or not) jobs, and collect back statuses.
My start point:
How do I receive Github Webhooks in Python (last answer, not the choosen one).
I started developing it 2 days ago. It's done, but I am still validating it.