How to build with parameters in Jenkins from Gitlab push? - jenkins

I have GitLab Community Edition 8.15.2 successfully trigger pipeline projects in Jenkins 2.32.1 using a webhook. I want the gitlab push to trigger a build with parameters but the parameter value is null when it comes through so the build fails.
The gitlab webhook looks like:
http://jenkins.server:8080/project/project-a/buildWithParameters?MYPARAM=foo
In my pipeline project I echo the parameter value out with
echo "MYPARAM: ${MYPARAM}"
and it's not set to anything. Any ideas on where I've gone wrong?
UPDATE
The actual code I'm using in the pipeline is:
node {
try {
echo "VM_HOST: ${VM_HOST}"
echo "VM_NAME: ${VM_NAME}"
stage('checkout') {
deleteDir()
git 'http://git-server/project/automated-build.git'
}
stage('build') {
bat 'powershell -nologo -file Remove-MyVM.ps1 -VMHostName %VM_HOST% -VMName "%VM_NAME%" -Verbose'
}
...
}
}
The parameter VM_HOST has a default value but VM_NAME doesn't. In my Console output in Jenkins I can see:
[Pipeline] echo
VM_HOST: HyperVHost
[Pipeline] echo
VM_NAME:

I have been struggling with this for weeks. I had it working once, but I couldn't get it to work again, untill today. And the solution was mindblowingly obvious ofcourse...
Automatically for each pipeline job I ticked the following box:
Build when a change is pushed to GitLab. GitLab CI Service URL:
http://jenkins.dev:8080/project/MyProject
Then from GitLab I used the webhook to trigger the above.
Like you I tried to add /buildWithParameters and tried many other things that didn't work.
The problem was, I ticked the wrong checkbox!
Since I trigger the build from a GitLab webhook, the above checkbox (build when a...) does not have to be checked at all.
What needs to be checked is:
Trigger builds remotely (e.g., from scripts)
That checkbox provides you with a new URL:
Use the following URL to trigger build remotely:
JENKINS_URL/job/MyProject/build?token=TOKEN_NAME or
/buildWithParameters?token=TOKEN_NAME
Like all the documentation I came along states and as you can see, the URL now no longer starts with /project, but with /job instead!
So tick that box and change your URL accordingly:
http://jenkins.server:8080/**job**/project-a/buildWithParameters?token=TOKEN_NAME&MYPARAM=foo
Least I want to mention the token:
In the GitLab webhook there is a seperate field for "token", which states:
Use this token to validate received payloads. It will be sent with the request in the X-Gitlab-Token HTTP header.
So, the token provided there will be sent along the request as a HTTP header.
This is the token which can be provided globally in the Jenkins setup.
The token you must provide in the Jenkins job when ticking the box Use the following URL to trigger build remotely must be send in the URL as GET parameter, just like the example shows.
Final note: personally I have never got this working completely, because I don't get the Jenkins CSRF protection off my back. Disabling it gives me another error. However, hopefully the above does fix the problem for you and others.

GitLab plugin does not allow you to pass arbitrary parameters. In their project there is an open issue for it that deserves to be upvoted.
My convoluted solution was to use the desired values for the push trigger as the default parameters of the job. Then I used the Parameterized Scheduler plugin to use other values in the scheduled executions.
The problem is that I got a bad usability for the job when it was manually run, since the default parameters were appropriate for the push hook.

I found the solution here https://www.jittagornp.me/blog/jenkins-gitlab-webhook/
I verified it with Jenkins 2.263.1 and GitLab Community Edition 13.6.1
Your webhook url will look like
https://hunter:11a403302a4f01b9b4975c0ac27441a5cc#jenkinsservername.com/job/yourjenkinsproject/buildWithParameters?token=Aju9ryHUu6t7W8wLSeCWtY2bWjzQduYNPyY7B3gs&yourparam=yourvalue
"hunter" ist your username in Jenkins.
The following is the Jenkins API Token you have to create in your Jenkins User Managment independent of the project.
The last Token is the one you specify in the jenkins project options under "Trigger builds remotely (e.g., from scripts)"
The last thing is to add your Parameter and value to the url with &param=value

Related

How to invoke a jenkins pipeline A in another jenkins pipeline B in separate Jenkins master?

Important is separate Jenkins master.
I expecting something like
script { build job: 'https://second_instance_jenkins.mycompany.com/jobs/alt_build_job', parameters: [] }
However this does not work, I cannot connect to other master server, neither I know how to properly AUTH using this syntax. There are tons of examples how to invoke another job within same Jenkins, but not at separate independent Jenkins master!
Thanks.
You could enable the option Trigger builds remotely to trigger your job through the Jenkins API. Once you check the option, you will get the url you need to call in order to trigger the build
Use the following URL to trigger build remotely: JENKINS_URL/job/BranchDiffTests/build?token=TOKEN_NAME
Detailed instructions can be found here.
Once you have the job properly configured, you can send a GET request with the token as a parameter, and have the job executed remotely. Use Bash, Powershell, or whatever scripting language you use within your pipeline to create an API call to this endpoint, and once the call is made, the job should start running.
Found it.
I have to use Parameterized-Remote-Trigger as described here
https://www.jenkins.io/doc/pipeline/steps/Parameterized-Remote-Trigger/
I also going to post my working pipeline step. I wish there are more examples in manual.
steps {
triggerRemoteJob auth: CredentialsAuth(credentials: '7f89634523-1b42-440d-8053-aa3d523523441d'),
enhancedLogging: true,
job: 'Build/my-integration',
remoteJenkinsUrl: 'https://second_instance_jenkins.mycompany.com/',
useCrumbCache: true,
httpGetReadTimeout: 600,
httpPostReadTimeout: 600,
useJobInfoCache: true,
parameters: '''
PROJECT_NAME=core
BUILD_TYPE=Debug
PLATFORM=Linux
'''
}

How to notify Github after merge job is finished? [duplicate]

Is there a way to show the Jenkins build status on my project's GitHub Readme.md?
I use Jenkins to run continuous integration builds. After each commit it ensures that everything compiles, as well as executes unit and integration tests, before finally producing documentation and release bundles.
There's still a risk of inadvertently committing something that breaks the build. It would be good for users visiting the GitHub project page to know the current master is in that state.
Ok, here's how you can set up Jenkins to set GitHub build statuses. This assumes you've already got Jenkins with the GitHub plugin configured to do builds on every push.
Go to GitHub, log in, go to Settings, Developer Settings, Personal access tokens and click on Generate new token.
Check repo:status (I'm not sure this is necessary, but I did it, and it worked for me).
Generate the token, copy it.
Make sure the GitHub user you're going to use is a repository collaborator (for private repos) or is a member of a team with push and pull access (for organization repos) to the repositories you want to build.
Go to your Jenkins server, log in.
Manage Jenkins → Configure System
Under GitHub Web Hook select Let Jenkins auto-manage hook URLs, then specify your GitHub username and the OAuth token you got in step 3.
Verify that it works with the Test Credential button. Save the settings.
Find the Jenkins job and add Set build status on GitHub commit to the post-build steps
That's it. Now do a test build and go to GitHub repository to see if it worked. Click on Branches in the main repository page to see build statuses.
You should see green checkmarks:
In the meanwhile the UI of Jenkins and GitHub has changed a bit and it took me a while to figure out how to configure Jenkins now correctly. The explanation here is based on Jenkins version 2.121.1.
I also assume that you have already configured your Jenkins Job be triggered by a webhook or by polling.
Those are the steps that I have taken to get it working:
Configure Github: Create Personal Access Token with OAuth Scope repo:status
Configure Jenkins: Configure System and add the OAuth Secret as a GitHub Server - use Secret Text as an authentication method to put the OAuth Secret in there.
Configure your Jenkins Job: Add Set GitHub commit status as Post-build action. Set the Status Result to One of the default messages and statuses.
Check your result on GitHub: Check if you get the build status and build execution duration on your GitHub commit.
Configure Github
Configure Jenkins
Configure Jenkins Job
Result
You will now see the status for your commits and branches:
What I did is quite simple:
Install the Hudson Post Task Plugin
Create a Personal Access Token here : https://github.com/settings/tokens
Add a Post Task Plugin that always put success
curl -XPOST -H "Authorization: token OAUTH TOKEN" https://api.github.com/repos/:organization/:repos/statuses/$(git rev-parse HEAD) -d "{
\"state\": \"success\",
\"target_url\": \"${BUILD_URL}\",
\"description\": \"The build has succeeded!\"
}"
Add a Post Task Plugin that will put failure if "marked build as failure"
curl -XPOST -H "Authorization: token OAUTH TOKEN" https://api.github.com/repos/:organization/:repos/statuses/$(git rev-parse HEAD) -d "{
\"state\": \"failure\",
\"target_url\": \"${BUILD_URL}\",
\"description\": \"The build has failed!\"
}"
You can also add a call to pending at the beginning of tests
curl -XPOST -H "Authorization: token OAUTH TOKEN" https://api.github.com/repos/:organization/:repos/statuses/$(git rev-parse HEAD) -d "{
\"state\": \"pending\",
\"target_url\": \"${BUILD_URL}\",
\"description\": \"The build is pending!\"
}"
This plugin should work: https://wiki.jenkins-ci.org/display/JENKINS/Embeddable+Build+Status+Plugin
You should be able to embed badges like this into your README.md file:
The Commit Status API allows you to see the "Repo Statuses API".
And since April 26th 2013, you now can see the build status on your GitHub repo branch page:
That means it is another way, by visiting the GitHub project page, to see those statuses instead of having only Jenkins.
Starting April 30th, 2013, the API endpoint for commit statuses has been extended to allow branch and tag names, as well as commit SHAs.
There's also the plugin Embeddable Build Status that will give you a badge URL that you can post in your README.md file, and it looks like this:
If you have the GitHub plugin installed on your Jenkins, you can do it in the Post build actions like this:
Add the below line in your README.md file and change both URLs according to your Jenkins project.
[![Build Status](https://jenkins../..project/lastBuild/buildStatus)](https://jenkins../..project/lastBuild/)
In regards with setting up Jenkins and GitHub's protected branch. I'm using Jenkins 2.6, and these are the steps I did to make it work:
On your repository's GitHub webpage:
Navigate to Settings > Branches.
Under Protect branches, click on
the Choose a branch drown down menu and select the branch you want
to set as a Protected branch.
Enable the options as needed.
On the Jenkins Server:
(Make sure you have the Git and GitHub Plugin installed)
Navigate to Manage Jenkins > Configure System.
Under GitHub, set the API URL to https://api.github.com. Though this is the default value.
Select your generated token for the Credentials. If you haven't generated a token yet, click on Advanced... then on Additional actions, you can convert your login and password to token and use it as your credential.
Also, make sure the GitHub account that your Jenkins is using is a collaborator for the repository. I've set it with write permission level.
Hope this helps.
I followed the directions from Alex and it worked.
But, for GitHub Enterprise you need to modify the API URL when adding the server to Jenkins.
For example, if your company is creditcard.com, your URL would be
https://github.creditcard.com/api/v3/
Edit:
I'm no longer using this approach, please use one of the other answers.
Update: what I ended up doing, for our specific case: (above answers were great - thanks!)
Because our build server is not on the internet, we have a script to publish the build status to the gh-pages branch in github.
Start of build stamps failing
End of build stamps success
Project runs after main project to publish results -> build-status, API docs, test reports and test coverage.
GitHub caches images, so we created .htaccess file, that instructs a short cache timeout for the build-status image.
Put this in the directory with the build-status image:
ExpiresByType image/png "access plus 2 minutes"
Here's the build script. The target that publishes to gh-pages is '--publish.site.dry.run'
With less than 400 lines of config, we have:
Compile checks
unit & integration tests
Test Reports
Code Coverage Reports
API Docs
Publishing to Github
. . and this script can be run in or outside of Jenkins, so that:
Developers can run this script before commit, reducing the chance of a broken build that impacts others.
A failure is easy to reproduce locally.
The Results:
Project main page has the build status, updated after each build, along with latest API Docs, test results and test coverage.
Jently updates your GitHub commit status (as described by vonc), but unfortunately they have yet to implement the Repo Status API.
For pipeline projects, you can use the post section like so:
void setBuildStatus(String message, String state) {
step([
$class: "GitHubCommitStatusSetter",
reposSource: [$class: "ManuallyEnteredRepositorySource", url: "https://github.com/my-user/my-repo"],
contextSource: [$class: "ManuallyEnteredCommitContextSource", context: "ci/jenkins/build-status"],
errorHandlers: [[$class: "ChangingBuildStatusErrorHandler", result: "UNSTABLE"]],
statusResultSource: [ $class: "ConditionalStatusResultSource", results: [[$class: "AnyBuildResult", message: message, state: state]] ]
]);
}
pipeline {
agent any
triggers {
githubPush()
}
stages {
stage('Hello') {
steps {
echo 'Hello World'
}
}
}
post {
success {
setBuildStatus("Build succeeded", "SUCCESS");
}
failure {
setBuildStatus("Build failed", "FAILURE");
}
}
}
Just change "https://github.com/my-user/my-repo" to meet your GitHub repository.
Reference: How to set GitHub commit status with Jenkinsfile NOT using a pull request builder
I am adding to this answer, and also to this one. They have answered the question, but they didn't give us a proper intuition about the problem
So, here are my intuitions:
We need to add status to our GitHub commit. This status is based upon the results of our Jenkins build.
To do this, we need to give Jenkins access to the GitHub API, not to the repo. We do this through OAuth, and we can create the token going to the GitHub settings → Developer settings → Personal access tokens. Then for a public GitHub repository just check repo:status, and for a private repository, check the whole repository section
After creating your access token you need to add it to your Jenkins server:
Copy and paste the access token to the GitHub plugin section settings, under your Jenkins configurations
Create a GitHub server. It defaults to api.github.com. And add the access token as a secret text credentials.
The last step is to add a post build settings when you create your pipeline.
Resources:
Here is a section of a YouTube video in which he goes over the above steps.
Read about OAuth

Jenkins: how to trigger pipeline on git tag

We want to use Jenkins to generate releases/deployments on specific project milestones. Is it possible to trigger a Jenkins Pipeline (defined in a Jenkinsfile or Groovy script) when a tag is pushed to a Git repository?
We host a private Gitlab server, so Github solutions are not applicable to our case.
This is currently something that is sorely lacking in the pipeline / multibranch workflow. See a ticket around this here: https://issues.jenkins-ci.org/browse/JENKINS-34395
If you're not opposed to using release branches instead of tags, you might find that to be easier. For example, if you decided that all branches that start with release- are to be treated as "release branches", you can go...
if( env.BRANCH_NAME.startsWith("release-") ) {
// groovy code on release goes here
}
And if you need to use the name that comes after release-, such as release-10.1 turning into 10.1, just create a variable like so...
if( env.BRANCH_NAME.startsWith("release-") ) {
def releaseName = env.BRANCH_NAME.drop(8)
}
Both of these will probably require some method whitelisting in order to be functional.
I had the same desire and rolled my own, maybe not pretty but it worked...
In your pipeline job, mark that "This project is parameterized" and add a parameter for your tag. Then in the pipeline script checkout the tag if it is present.
Create a freestyle job that runs a script to:
Checkout
Run git describe --tags --abbrev=0 to get the latest tag.
Check that tag against a running list of builds (like in a file).
If the build hasn't occurred, trigger the pipeline job via a url passing your tag as a parameter (in your pipeline job under "Build Triggers" set "Trigger builds remotely (e.g. from scripts) and it will show the correct url.
Add the tag to your running list of builds so it doesn't get triggered again.
Have this job run frequently.
if you use multibranch pipeline, there is a discover tag. Use that plus Spencer solution

Jenkins: Access bitbucket payload in shell

According to the documentation of the bitbucket plugin for Jenkins (https://wiki.jenkins-ci.org/display/JENKINS/BitBucket+Plugin) it should be possible to access the payload infos through the environment varaible $BITBUCKET_PAYLOAD.
However when in my build I run the command printenv, there is no environment variable called $BITBUCKET_PAYLOAD, and nothing related to it.
So it's impossible for me to access informations I need to configure my build.
You can trigger Jenkins with Generic Webhook Trigger instead.
Then you can create a variable everything having the JSONPath $. Then everything will resolve to the entire JSON payload. So you can have a shell script build step like:
echo $everything

How to trigger Jenkins builds remotely and to pass parameters

I am invoking a Jenkins job remotely using:
wget http://<ServerIP>:8080/job/Test-Jenkins/build?token=DOIT
Here Test-Jenkins job is invoked and DOIT is the security token that I have used.
Now I need to pass some parameters to the build.xml file of this job i.e. Test-Jenkins.
I have not yet figured out how to pass the variables yet.
See Jenkins documentation: Parameterized Build
Below is the line you are interested in:
http://server/job/myjob/buildWithParameters?token=TOKEN&PARAMETER=Value
In your Jenkins job configuration, tick the box named "This build is parameterized", click the "Add Parameter" button and select the "String Parameter" drop down value.
Now define your parameter - example:
Now you can use your parameter in your job / build pipeline, example:
Next to trigger the build with own/custom parameter, invoke the following URL (using either POST or GET):
http://JENKINS_SERVER_ADDRESS/job/YOUR_JOB_NAME/buildWithParameters?myparam=myparam_value
To add to this question, I found out that you don't have to use the /buildWithParameters endpoint.
In my scenario, I have a script that triggers Jenkins to run tests after a deployment. Some of these tests require extra info about the deployment to work correctly.
If I tried to use /buildWithParameters on a job that does not expect parameters, the job would not run. I don't want to go in and edit every job to require fake parameters just to get the jobs to run.
Instead, I found you can pass parameters like this:
curl -X POST --data-urlencode "token=${TOKEN}" --data-urlencode json='{"parameter": [{"name": "myParam", "value": "TEST"}]}' https://jenkins.corp/job/$JENKINS_JOB/build
With this json=... it will pass the param myParam with value TEST to the job whenever the call is made. However, the Jenkins job will still run even if it is not expecting the parameter myParam.
The only scenario this does not cover is if the job has a parameter that is NOT passed in the json. Even if the job has a default value set for the parameter, it will fail to run the job. In this scenario you will run into the following error message / stack trace when you call /build:
java.lang.IllegalArgumentException: No such parameter definition: myParam
I realize that this answer is several years late, but I hope this may be useful info for someone else!
Note: I am using Jenkins v2.163
You can simply try it with a jenkinsfile. Create a Jenkins job with following pipeline script.
pipeline {
agent any
parameters {
booleanParam(defaultValue: true, description: '', name: 'userFlag')
}
stages {
stage('Trigger') {
steps {
script {
println("triggering the pipeline from a rest call...")
}
}
}
stage("foo") {
steps {
echo "flag: ${params.userFlag}"
}
}
}
}
Build the job once manually to get it configured & just create a http POST request to the Jenkins job as follows.
The format is http://server/job/myjob/buildWithParameters?PARAMETER=Value
curl http://admin:test123#localhost:30637/job/apd-test/buildWithParameters?userFlag=false --request POST
To pass/use the variables, first create parameters in the configure section of Jenkins. Parameters that you use can be of type text, String, file, etc.
After creating them, use the variable reference in the fields you want to.
For example: I have configured/created two variables for Email-subject and Email-recipentList, and I have used their reference in the EMail-ext plugin (attached screenshot).
When we have to send multiple trigger parameters to jenkins job, the following commands works.
curl -X POST -i -u "auto_user":"xxxauthentication_tokenxxx" "JENKINS_URL/view/tests/job/helloworld/buildWithParameters?param1=162&param2=store"
You can trigger Jenkins builds remotely and to pass parameters by using the following query.
JENKINS_URL/job/job-name/buildWithParameters?token=TOKEN_NAME&param_name1=value&param_name1=value
JENKINS_URL (can be) = https://<your domain name or server address>
TOKE_NAME can be created using configure tab
curl -X POST -u Admin:<api_token> http://localhost:8080/job/<job_name>/buildWithParameters\?ColorValue\=sddddsd
curl -H "Jenkins-Crumb: <your_crumb_data>" -u "<username>:<password>" "http://<your_jenkins_url>?buildWithParameters?token=<your_remote_api_name>?<parameterA>=<val_parameter_A>&<parameterB>=<val_parameterB>"
You can change following parameters as you want:
<your_crumb_data>
<username>
<password>
<your_jenkins_url>
<your_remote_api_name>
<parameterA>
<parameterB>
<val_parameter_A>
<val_parameter_B>
Note: Placing double quotes may be critical. Pay attention, please.
2023 Update
It has now changed how Jenkins Authenticates.
You need to pass 2 tokens to execute your job remotely.
You need:
apiToken to authenticate your identity. This value is created from JENKINS_URL/me/configure . Also check here for documentation
Another Job authentication token which you create when you enable 'Trigger builds remotely'.
Below is a sample you can tweak to get it done.
PARAM1_VALUE=<param1_value>
PARAM2_VALUE=<param2_vale>
USERNAME=dummy_user_name
JENKINS_URL="http://10.xxx.x.xxx:8080"
JOB_TOKEN="<value>" # you create this token when you enable Job>Configure>Build Triggers>Trigger builds remotely
LOGIN_API_TOKEN="<value>" #get this value from JENKINS_URL/me/configure
curl -L --user $USERNAME:$LOGIN_API_TOKEN "$JENKINS_URL/job/JobName/buildWithParameters?token=$JOB_TOKEN&param1_name=$PARAM1_VALUE&param2_name=$PARAM2_VALUE"

Resources