How to do polling for REST endpoint on Jenkins? - jenkins

On the Jenkins pipeline that I've created, I make a POST curl request once. However, since this job goes into a queue first, I need to wait till the job is completed. This is an example of my command
curl -X POST -H "Content-Type: application/json" \
-d '{"source_path": "source/path/example", "destination_path": "destination/path/example"}' \
http://example.com/v1/postJob
This returns a JSON response {"jobID": 1234}
I want the Jenkins job to stay in 'running' state till the GET endpoint returns "status": success. The GET endpoint returns a JSON body with a field 'status' that indicates whether the job is done. This is the GET endpoint
curl http://example.com/v1/jobs/1234
This returns a JSON response
{
"job": 1234,
"status": success
}
Can anyone provide me with an example of polling that can be implemented directly on Jenkins without using a Jenkinsfile?

For polling an endpoint on bash, this code snippet should do the trick
GETURL="http://exmple.com/v1/tasks/${TASK_ID}"
while true;
do
STATUS=$(curl $GETURL | jq -r '.status'); //'status' is to check JSON response field
echo ${STATUS}
if [[ "$STATUS" == "succeeded" ]]; then
break;
fi;
sleep 2700; //The endpoint will be hit again after 2700 seconds
done

Related

Excluding Draft PRs from Jenkins

I will like to be able to choose to only run Jenkins build on PRs who are not marked as draft. Is there currently a way to do that?
I found something like this: https://github.com/jenkinsci/github-branch-source-plugin/pull/416, but cannot seem to find any place in the Jenkins dashboard that will allow me to exclude draft PRs.
Thanks!
The GitHub API allows you to view details of a PR and see if the PR is a draft:
Request:
curl \
-H "Accept: application/vnd.github+json" \
-H "Authorization: token <TOKEN>" \
https://api.github.com/repos/OWNER/REPO/pulls/PULL_NUMBER
Response:
{
"url": "https://api.github.com/repos/octocat/Hello-World/pulls/1347",
"id": 1,
"node_id": "MDExOlB1bGxSZXF1ZXN0MQ==",
...
"auto_merge": null,
"draft": false, <---- This is what you want
"merged": false,
...
"deletions": 3,
"changed_files": 5
}
You will have to modify the Jenkins job itself to perform this API call, parse the response and get the value of draft, and continue or abort the build depending on the value.

Resource not found - Triggering BitBucket Pipeline using curl

I created a new project and added a repository to it in my workspace. Further, I added a bitbucket-pipelines.yml to build a pipeline. I am able to trigger the pipeline manually however while trying to execute it using BitBucket API using curl, I get the below error every time:
Can anyone suggest what I am missing here?
NOTE: The same curl command (below) is able to run other pipelines in different repositories in the same workspace, so do I need to enable something in my current repository to access pipeline using BitBucket APIs. TIA
Error:
{"type": "error", "error": {"message": "Resource not found"}}%
cURL command:
curl -X POST -is -u username:password \
-H 'Content-Type: application/json' \
https://api.bitbucket.org/2.0/repositories/workspace-name/repo-name/pipelines/ \
-d '
{
"target": {
"type": "pipeline_ref_target",
"ref_type": "branch",
"ref_name": "master",
"selector": {
"type": "custom",
"pattern": "create-tenant"
}
}}'
So the tokens I was using in the curl weren't added to Repository Settings -> User and group access. Got them added and was able to execute it successfully.

Unable to get the payload from GitHub web hook trigger in jenkins pipeline

I have configured a Github web hook with the below settings:
Payload URL: https:///github-webhook/
Content Type: application/x-www-form-urlencoded
Events : Pushes, Pull Requests
The Jenkins job that I have, is a pipeline job that has the below enabled:
Build Trigger: GitHub hook trigger for GITScm polling
With the above configuration, I see that in response to an event ie; push/PR in GitHub, the jenkins job gets triggered successfully. In GitHub, under Recent Deliveries for the web hook, I see the details of the payload and a successful Response of 200.
I am trying to get the payload in Jenkins Pipeline for further processing. I need some details eg: PR URL/PR number, refs type, branch name etc for conditional processing in the Jenkins Pipeline.
I tried accessing the "payload" variable (as mentioned in other stack overflow posts and the documentations available around) and printing it as part of the pipeline, but I have had no luck yet.
So my question is, How can I get the payload from GitHub web hook trigger in my Jenkins Pipeline ?
You need to select Content type: application/json in your webhook in GitHub. Then you would be able to access any variable from the payload GitHub sends as follows: $. pull_request.url for pr url, for example.
Unsure if this is possible.
With the GitHub plugin we use (Pipeline Github), PR number is stored in the variable CHANGE_ID.
PR URL is pretty easy to generate given the PR number. Branch name is stored in the variable BRANCH_NAME. In case of pull requests, the global variable pullRequest is populated with lots of data.
Missing information can be obtained from Github by using their API. Here's an example of checking if PR is "behind", you can modify that to your specific requirements:
def checkPrIsNotBehind(String repo) {
withCredentials([usernamePassword(credentialsId: "<...>",
passwordVariable: 'TOKEN',
usernameVariable: 'USER')]) {
def headers = ' -H "Content-Type: application/json" -H "Authorization: token $TOKEN" '
def url = "https://api.github.com/repos/<...>/<...>/pulls/${env.CHANGE_ID}"
def head_sha = sh (label: "Check PR head SHA",
returnStdout: true,
script: "curl -s ${url} ${headers} | jq -r .head.sha").trim().toUpperCase()
println "PR head sha is ${head_sha}"
headers = ' -H "Accept: application/vnd.github.v3+json" -H "Authorization: token $TOKEN" '
url = "https://api.github.com/repos/<...>/${repo}/compare/${pullRequest.base}...${head_sha}"
def behind_by = sh (label: "Check PR commits behind",
returnStdout: true,
script: "curl -s ${url} ${headers} | jq -r .behind_by").trim().toUpperCase()
if (behind_by != '0') {
currentBuild.result = "ABORTED"
currentBuild.displayName = "#${env.BUILD_NUMBER}-Out of date"
error("The head ref is out of date. Please update your branch.")
}
}
}

Jenkins Webhook Header as Argument to Shell script

I'm trying to trigger a Jenkins job through the webhook using the curl command whenever there is EC2 Spot Instance Interruption Warning with below sample event. All this will be done in AWS lambda from CloudWatch Event Trigger.
{
"version": "0",
"id": "1e5527d7-bb36-4607-3370-4164db56a40e",
"detail-type": "EC2 Spot Instance Interruption Warning",
"source": "aws.ec2",
"account": "123456789012",
"time": "1970-01-01T00:00:00Z",
"region": "us-east-1",
"resources": [
"arn:aws:ec2:us-east-1b:instance/i-0b662ef9931388ba0"
],
"detail": {
"instance-id": "i-0b662ef9931388ba0",
"instance-action": "terminate"
}
}
My aim is to get the instance-id from the event and pass it as a header to the jenkins webhook where the jenkins job gets triggered and this header has to be sent as an argument to the underlying python script in the jenkins job.
I tried the below approach which didn't give me success. And am not sure if this is how it is done. :)
curl -H 'param: instance' https://jenkins.url/generic-webhook-trigger/invoke\?token\=jenkins-job
Generic Jenkins webhook trigger is configured as below.
The final python script in the job is configured as below.
python maintenance.py $.param
I'm trying to get the final script similar to below. Please let me know if you know of any approach how can I get this done. TIA
python maintenance.py i-0b662ef9931388ba
I fixed this by adding the following.
Add string parameter under This project is parameterized as shown below.
Next under Generic Webhook Trigger I have added Header parameters
Now, we can directly pass this param in the build command using $param.
The curl command is now modified to below.
curl --location --request POST 'https://jenkins.url/generic-webhook-trigger/invoke\?token\=jenkins-job' --header 'Content-Type: application/json' --header "param: $EVENT_DATA" --data-raw ''
What did I learn from this?
I was too lazy to read the documentation and finally figured it out only after reading it.

Executing Dataflow Template via Google Cloud Scheduler

I am trying to execute a custom Dataflow Template via the Google Cloud Scheduler but when I execute the Job I get an UNAUTHENTICATED exception.
How do I give Google Cloud Scheduler access to use the Dataflow google API?
Here is the URL and POST body I am using:
https://dataflow.googleapis.com/v1b3/projects/<<PROJECT>>/templates:launch?gcsPath=gs://<<GCS_BUCKET>>/template
{
"jobName": "job-name-scheduled",
"parameters": {
"param1" : "parmval1"
"environment": {
"tempLocation": "gs://<<BUCKET>>/temp",
"region": "us-east1"
}
}
The Cloud Scheduler documentation points out that "Targeted HTTP endpoints must be publicly accessible".
Normally, for creating that kind of Dataflow job, you would submit something like this:
curl -X POST \
'https://dataflow.googleapis.com/v1b3/projects/<project>/templates:launch?gcsPath=gs://dataflow-templates/latest/Word_Count' \
-H 'Authorization: Bearer '$(gcloud auth application-default print-access-token) \
-H 'Content-Type: application/json' \
--data '{
"jobName": "scheduled_job",
"parameters": {
"inputFile" : "gs://dataflow-samples/shakespeare/kinglear.txt",
"output": "gs://<bucket>/output/my_output"
},
"environment": { "zone": "us-central1-f" }
}'
But, you can't send the authorization token through Cloud Scheduler.
For scheduling Dataflow jobs, you can see this answer instead.

Resources