Can you please help, I have the following scenario and I went through many videos, blogs but could not find anything matching with my use-case
Requirement:
To write a CI\CD pipeline in GitLab, which can facilitate the following stages in this order
- verify # unit test, sonarqube, pages
- build # package
- publish # copy artifact in repository
- deploy # Deploy artifact on runtime in an test environment
- integration # run postman\integration tests
All other stages are fine and working but for the deploy stage, because of a few restrictions I have to submit an existing Jenkins job using Jenkin remote API with the following script but the problem that script returns an asynchronous response and start the Jenkins job and deploy stage completes and it moves to next stage (integration).
Run Jenkins Job:
image: maven:3-jdk-8
tags:
- java
environment: development
stage: deploy
script:
- artifact_no=$(grep -m1 '<version>' pom.xml | grep -oP '(?<=>).*(?=<)')
- curl -X POST http://myhost:8081/job/fpp/view/categorized/job/fpp_PREP_party/build --user mkumar:1121053c6b6d19bf0b3c1d6ab604f22867 --data-urlencode json="{\"parameter\":[{\"name\":\"app_version\",\"value\":\"$artifact_no\"}]}"
Note: Using GitLab CE edition and Jenkins CI project service is not available.
I am looking for a possible way of triggering the Jenkins job from the pipeline and only on successful completion of the Jenkins job my integration stage starts executing.
Thanks for the help!
Retrieving the status of a Jenkins job that is triggered programmatically through the remote access API is notorious for not being quite convoluted.
Normally you would expect to receive in the response header, under the Location attribute, a url that you can poll to get the status of your request, but unfortunately there are some in-between steps to reach that point. You can find a guide in this post. You may also have a look in this older post.
Once you have the url, you can pool and parse the status job and either sh "exit 1" or sh "exit 0" in your script to force the job that is invoking the external job to fail or succeed, depending on how you want to assert the result of the remote job
I want to set up notifications in slack through Jenkins using pabot(parallel run of robot framework tests).
I have already configured Jenkins to output the pass/failure/skip notifications when using robot, but once I change it to pabot the notification no long appears in my slack channel.
This command runs successfully and my slack channel is notified of the tests
that pass/fail/skip
robot -A Config/Config.args -d results -v REMOTE_URL:http://dummy:0000/wd/hub Tests
On the other hand this command alerts my channel that the build has commenced but nothing after that
pabot -A Config/Config.args -d results -v REMOTE_URL:http://dummy:0000/wd/hub Tests
In the directory the a pabot_results file is created which is different to the build with the robot command. But the output.xml, log.html and report.html files include the expected results.
I want the basic pass/fail/skip to appear in the slack channel for parallel run test. At the minute no test results are appearing in the slack channel.
I am using Jenkins declarative pipeline and I have solved problem following way in post section after running robot tests using Token Macro plugin:
post {
always {
robot(
outputPath: 'Reports/',
outputFileName: 'output.xml',
reportFileName: 'report.html',
logFileName: 'log.html',
disableArchiveOutput: false,
passThreshold: 100.0,
unstableThreshold: 95.0,
otherFiles: '*.png,debug.log'
)
script {
String robotReportSummary = tm('ROBOT_FAILEDCASES: ${ROBOT_FAILEDCASES}, ROBOT_PASSPERCENTAGE: ${ROBOT_PASSPERCENTAGE}, ROBOT_PASSRATIO: ${ROBOT_PASSRATIO}, ROBOT_REPORTLINK: ${ROBOT_REPORTLINK}')
// Instead of echo step you can use whatever you want with variable robotReportSummary
echo "${robotReportSummary}"
}
}
}
Further info:
From documentation of Robot Framework plugin is visible plugin exports following token macros:
${ROBOT_FAILEDCASES}
${ROBOT_PASSPERCENTAGE, onlyCritical}
${ROBOT_PASSRATIO, onlyCritical}
${ROBOT_REPORTLINK}
As well from source code of Robot Framework plugin it can be found other undocumented token macros:
${ROBOT_FAILED}
${ROBOT_PASSED}
${ROBOT_TOTAL}
In order to access these macros in declarative pipeline you can use Token Macro plugin and it's pipeline step tm. From documentation it's not clear that it returns string with replaced variables. But usage can be found e.g. here in source code of Token Macro plugin.
Is there a way to show the Jenkins build status on my project's GitHub Readme.md?
I use Jenkins to run continuous integration builds. After each commit it ensures that everything compiles, as well as executes unit and integration tests, before finally producing documentation and release bundles.
There's still a risk of inadvertently committing something that breaks the build. It would be good for users visiting the GitHub project page to know the current master is in that state.
Ok, here's how you can set up Jenkins to set GitHub build statuses. This assumes you've already got Jenkins with the GitHub plugin configured to do builds on every push.
Go to GitHub, log in, go to Settings, Developer Settings, Personal access tokens and click on Generate new token.
Check repo:status (I'm not sure this is necessary, but I did it, and it worked for me).
Generate the token, copy it.
Make sure the GitHub user you're going to use is a repository collaborator (for private repos) or is a member of a team with push and pull access (for organization repos) to the repositories you want to build.
Go to your Jenkins server, log in.
Manage Jenkins → Configure System
Under GitHub Web Hook select Let Jenkins auto-manage hook URLs, then specify your GitHub username and the OAuth token you got in step 3.
Verify that it works with the Test Credential button. Save the settings.
Find the Jenkins job and add Set build status on GitHub commit to the post-build steps
That's it. Now do a test build and go to GitHub repository to see if it worked. Click on Branches in the main repository page to see build statuses.
You should see green checkmarks:
In the meanwhile the UI of Jenkins and GitHub has changed a bit and it took me a while to figure out how to configure Jenkins now correctly. The explanation here is based on Jenkins version 2.121.1.
I also assume that you have already configured your Jenkins Job be triggered by a webhook or by polling.
Those are the steps that I have taken to get it working:
Configure Github: Create Personal Access Token with OAuth Scope repo:status
Configure Jenkins: Configure System and add the OAuth Secret as a GitHub Server - use Secret Text as an authentication method to put the OAuth Secret in there.
Configure your Jenkins Job: Add Set GitHub commit status as Post-build action. Set the Status Result to One of the default messages and statuses.
Check your result on GitHub: Check if you get the build status and build execution duration on your GitHub commit.
Configure Github
Configure Jenkins
Configure Jenkins Job
Result
You will now see the status for your commits and branches:
What I did is quite simple:
Install the Hudson Post Task Plugin
Create a Personal Access Token here : https://github.com/settings/tokens
Add a Post Task Plugin that always put success
curl -XPOST -H "Authorization: token OAUTH TOKEN" https://api.github.com/repos/:organization/:repos/statuses/$(git rev-parse HEAD) -d "{
\"state\": \"success\",
\"target_url\": \"${BUILD_URL}\",
\"description\": \"The build has succeeded!\"
}"
Add a Post Task Plugin that will put failure if "marked build as failure"
curl -XPOST -H "Authorization: token OAUTH TOKEN" https://api.github.com/repos/:organization/:repos/statuses/$(git rev-parse HEAD) -d "{
\"state\": \"failure\",
\"target_url\": \"${BUILD_URL}\",
\"description\": \"The build has failed!\"
}"
You can also add a call to pending at the beginning of tests
curl -XPOST -H "Authorization: token OAUTH TOKEN" https://api.github.com/repos/:organization/:repos/statuses/$(git rev-parse HEAD) -d "{
\"state\": \"pending\",
\"target_url\": \"${BUILD_URL}\",
\"description\": \"The build is pending!\"
}"
This plugin should work: https://wiki.jenkins-ci.org/display/JENKINS/Embeddable+Build+Status+Plugin
You should be able to embed badges like this into your README.md file:
The Commit Status API allows you to see the "Repo Statuses API".
And since April 26th 2013, you now can see the build status on your GitHub repo branch page:
That means it is another way, by visiting the GitHub project page, to see those statuses instead of having only Jenkins.
Starting April 30th, 2013, the API endpoint for commit statuses has been extended to allow branch and tag names, as well as commit SHAs.
There's also the plugin Embeddable Build Status that will give you a badge URL that you can post in your README.md file, and it looks like this:
If you have the GitHub plugin installed on your Jenkins, you can do it in the Post build actions like this:
Add the below line in your README.md file and change both URLs according to your Jenkins project.
[](https://jenkins../..project/lastBuild/)
In regards with setting up Jenkins and GitHub's protected branch. I'm using Jenkins 2.6, and these are the steps I did to make it work:
On your repository's GitHub webpage:
Navigate to Settings > Branches.
Under Protect branches, click on
the Choose a branch drown down menu and select the branch you want
to set as a Protected branch.
Enable the options as needed.
On the Jenkins Server:
(Make sure you have the Git and GitHub Plugin installed)
Navigate to Manage Jenkins > Configure System.
Under GitHub, set the API URL to https://api.github.com. Though this is the default value.
Select your generated token for the Credentials. If you haven't generated a token yet, click on Advanced... then on Additional actions, you can convert your login and password to token and use it as your credential.
Also, make sure the GitHub account that your Jenkins is using is a collaborator for the repository. I've set it with write permission level.
Hope this helps.
I followed the directions from Alex and it worked.
But, for GitHub Enterprise you need to modify the API URL when adding the server to Jenkins.
For example, if your company is creditcard.com, your URL would be
https://github.creditcard.com/api/v3/
Edit:
I'm no longer using this approach, please use one of the other answers.
Update: what I ended up doing, for our specific case: (above answers were great - thanks!)
Because our build server is not on the internet, we have a script to publish the build status to the gh-pages branch in github.
Start of build stamps failing
End of build stamps success
Project runs after main project to publish results -> build-status, API docs, test reports and test coverage.
GitHub caches images, so we created .htaccess file, that instructs a short cache timeout for the build-status image.
Put this in the directory with the build-status image:
ExpiresByType image/png "access plus 2 minutes"
Here's the build script. The target that publishes to gh-pages is '--publish.site.dry.run'
With less than 400 lines of config, we have:
Compile checks
unit & integration tests
Test Reports
Code Coverage Reports
API Docs
Publishing to Github
. . and this script can be run in or outside of Jenkins, so that:
Developers can run this script before commit, reducing the chance of a broken build that impacts others.
A failure is easy to reproduce locally.
The Results:
Project main page has the build status, updated after each build, along with latest API Docs, test results and test coverage.
Jently updates your GitHub commit status (as described by vonc), but unfortunately they have yet to implement the Repo Status API.
For pipeline projects, you can use the post section like so:
void setBuildStatus(String message, String state) {
step([
$class: "GitHubCommitStatusSetter",
reposSource: [$class: "ManuallyEnteredRepositorySource", url: "https://github.com/my-user/my-repo"],
contextSource: [$class: "ManuallyEnteredCommitContextSource", context: "ci/jenkins/build-status"],
errorHandlers: [[$class: "ChangingBuildStatusErrorHandler", result: "UNSTABLE"]],
statusResultSource: [ $class: "ConditionalStatusResultSource", results: [[$class: "AnyBuildResult", message: message, state: state]] ]
]);
}
pipeline {
agent any
triggers {
githubPush()
}
stages {
stage('Hello') {
steps {
echo 'Hello World'
}
}
}
post {
success {
setBuildStatus("Build succeeded", "SUCCESS");
}
failure {
setBuildStatus("Build failed", "FAILURE");
}
}
}
Just change "https://github.com/my-user/my-repo" to meet your GitHub repository.
Reference: How to set GitHub commit status with Jenkinsfile NOT using a pull request builder
I am adding to this answer, and also to this one. They have answered the question, but they didn't give us a proper intuition about the problem
So, here are my intuitions:
We need to add status to our GitHub commit. This status is based upon the results of our Jenkins build.
To do this, we need to give Jenkins access to the GitHub API, not to the repo. We do this through OAuth, and we can create the token going to the GitHub settings → Developer settings → Personal access tokens. Then for a public GitHub repository just check repo:status, and for a private repository, check the whole repository section
After creating your access token you need to add it to your Jenkins server:
Copy and paste the access token to the GitHub plugin section settings, under your Jenkins configurations
Create a GitHub server. It defaults to api.github.com. And add the access token as a secret text credentials.
The last step is to add a post build settings when you create your pipeline.
Resources:
Here is a section of a YouTube video in which he goes over the above steps.
Read about OAuth
I'm building a declarative Jenkins Pipeline and after I have completed a few testing steps, I would like to have a step that notifies a slack channel that a build is ready to proceed to production. If that slack post is not responded to for a fixed amount of time, I would like Slack to notify Jenkins Pipeline to continue to the next step. If a fixed reply "ABORT" replies to the Slack post, I would like for Slack to communicate back to the Jenkins pipeline to abort the build. Any ideas on how to accomplish this two way interaction between Slack and Jenkins Pipeline?
node {
stage('build'){
echo "building"
}
}
stage('Deploy approval'){
input "Deploy to prod?"
}
node {
stage('deploy to prod'){
echo "deploying"
}
}
There is no way to do it directly . What you can do it is you can divide your jobs into two or three jobs and send notification at the end of first job using a function like this :
def notifySlack(text, channel) {
def slackURL = ' https://hooks.slack.com/services/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'
def payload = JsonOutput.toJson([text : text,
channel : channel,
username : "jenkins",
icon_emoji: ":jenkins:"])
sh "curl -X POST --data-urlencode \'payload=${payload}\' ${slackURL}"
}
then in slack you can write a script to build another part of your job (which is actually a new job) using slash commands :https://github.com/auth0/slash#inputs-and-outputs
This could be accomplished writing a php or any server side code and sending it paramaters where you can send ABORT and in the job you have to build , have a parameter to check the value of this passed parameter and have a condition according to it .
Executing upstream job called "A". On success of A executing test cases which is downstream project "B". But while sending mail from B we have to incorporate upstream project details (upstream project name, build no) in mail. So we can easily map / corelate the test run with respective upstream job.
In downstream project dashboard below details are displaying.
Started by upstream project Dev_RM_3.0_CI_Test build number 10
originally caused by:
I checked in https://wiki.jenkins-ci.org/display/JENKINS/Building+a+software+project. but couldnt find anything to inherit in downstream.
Created sample job with below details to display the current job details.
echo $BUILD_NUMBER
echo $JOB_NAME
echo $BUILD_ID
But the output is
Building on master in workspace /var/lib/jenkins/workspace/env
[env] $ /bin/sh -xe /tmp/hudson970280339057643719.sh
+ echo 1
1
+ echo env
env
+ echo 1
1
Finished: SUCCESS
Any help to inherit upstream details in downstream job?
How to get current job details?
The message that you refer to your question "Started by upstream project "Chained/1-First" build number 34" for example, is available in the jenkins Cause.
Jenkins keeps the upstream build info in it's cause object. If your are using build DSL or Pipelines you may get it in groovy. Alternatively you can curl the job url and use jq to get the Cause
For example curl http://localhost:8080/job/Chained/job/2-Second/17/api/json
"_class": "org.jenkinsci.plugins.workflow.job.WorkflowRun",
"actions": [{
"_class": "hudson.model.CauseAction",
"causes": [{
"_class": "hudson.model.Cause$UpstreamCause",
"shortDescription": "Started by upstream project \"Chained/1-First\" build number 34",
"upstreamBuild": 34,
"upstreamProject": "Chained/1-First",
"upstreamUrl": "job/Chained/job/1-First/"
}]
}
Or from the pipeline for example:
node() {
stage('downstream') {
def upstream = currentBuild.rawBuild.getCause(hudson.model.Cause$UpstreamCause)
echo upstream?.shortDescription
}
}
You can get a bunch of information out of Cause, pending all the script approvals or a global shared step. You will get a null if a different cause triggers this build, eg commit, or user.
You can pass in the upstream variables via build parameters to the downstream job and then you can access them (in the downstream job) using things such as ${MyParameter1} and ${MyParameter2}.
You would need to:
Add build parameters to the downstream job. For example, a string parameter named "ParentJobName".
Add a post build "Trigger downstream parameterized builds on other projects" to the upstream job.
Add something like "Current Build parameters" or "Predefined parameters" to the #2 and pass in whatever you need. For example:
ParentJobName=${JOB_NAME}
Access the parameters as you would other build variables. e.g. ${ParentJobName}
You should be able to pass in the basic stuff that way. Anything more complicated than that and you will probably be better off using a plugin like Copy Artifacts Plugin to copy files or using the Jenkins API in a system groovy step to get/modify the upstream build, etc.
You can simply use params.variableName in your downstream job to retrieve the parameters passed from your upstream parameter job. Your downstream job need not necessarily be a parameterized job.
Extending #razboy answer:
this is good way if Cause cannot be whitelisted in sandbox. I forgot about Jenkins API and used current build console to look for string about trigger cause. You can try to fetch data from API as #razboy or get current console and grep it if you need simple stuff. Jenkins API is more flexible for more complex logic. To get API help,append /api to your build url: <jenkins_url>/job/<buildUrl>/<buildNumber>/api
def buildUrl = env.BUILD_URL
sh "wget $buildUrl -O currentConsole.txt"
statusCode = sh returnStatus: true,script: 'cat currentConsole.txt | grep -q "Started by upstream project"'
boolean startedByUpstream= statusCode==0
MeowRude's answer helped me. To repcap it, in upstream job:
build job: 'mail-test', parameters: [[$class: 'StringParameterValue', name: 'VERSION_NUMBER', value: '1.0.0.0']]
And in downstream job:
echo "${params.VERSION_NUMBER}"
You may have to have certain plugins installed, but
def causes = currentBuild.getBuildCauses()
will return an ArrayList of objects that will most likely provide the necessary details, for example upstreamProject for the full project name and upstreamBuild for the build number. Then you can correlate results between up- and downstream builds easily.
Source: link to pipeline-examples in razboy's comment above