How can use File Spec in an API Call in Jfrog - jenkins

I have a question about how to use File Sepc in an API Call in JFrog.
I used Jenkins Artifactory Plugin to upload or download artifacts to JFrog, I try to rewrite the function using JFrog API (GET/PUT) to do the same thing.
but I have now a problem, for some artifacts I used file Spec to set some properties and finally I upload this file spec.
"files": [
{
"pattern": "${file}",
"target": "${target}" """
if (runID) {
uploadSpec += """,
"props": "artifactId=${runID}"
"""
}
uploadSpec += """
}
]
as you can see this artifactId.
in this case when I use JFrog API to upload artifacts how should I set properties?
sh """
curl -sSf -u user:pw -X PUT -T ${zipFile} 'https://${config.artifactory.name}.xxxx:443/artifactory/${path}'
"""
How can I call put api and set "props": "artifactId=${runID}"
any solutions??

First - if you can use the JFrog CLI - you should use it, because it makes it simpler and provides some advanced features out-of-the-box, such as batch parallel uploads/downloads, file-specs, attaching properties, build-info, authentication, etc.
If you still want to use the Artifactory API directly for setting properties, which is indeed a viable good option, you can do one of the following:
Add the properties as matrix parameters as part of the upload (deploy) API call.
In your case, it should be something like:
sh """
curl -sSf -u user:pw -X PUT -T ${zipFile} 'https://${config.artifactory.name}.xxxx:443/artifactory/${path};artifactId=${runID}'
"""
Note the ;key=value in the end of the URL.
Do a second call, after the upload, to set the item properties.
In your case, it should be something like:
Using the set item properties API -
sh """
curl -sSf -u user:pw -X PUT 'https://${config.artifactory.name}.xxxx:443/artifactory/api/storage/${path}?properties=artifactId=${runID}'
"""
or, using the update item properties API-
sh """
curl -sSf -u user:pw -X PATCH 'https://${config.artifactory.name}.xxxx:443/artifactory/api/metadata/${path}' -d '{ "props": { "artifactId" : "${runID}" } }'
"""
For more information, see:
Working with JFrog Properties
Using Properties in Deployment and Resolution
Artifactory REST API - Item Properties

Related

Resource not found - Triggering BitBucket Pipeline using curl

I created a new project and added a repository to it in my workspace. Further, I added a bitbucket-pipelines.yml to build a pipeline. I am able to trigger the pipeline manually however while trying to execute it using BitBucket API using curl, I get the below error every time:
Can anyone suggest what I am missing here?
NOTE: The same curl command (below) is able to run other pipelines in different repositories in the same workspace, so do I need to enable something in my current repository to access pipeline using BitBucket APIs. TIA
Error:
{"type": "error", "error": {"message": "Resource not found"}}%
cURL command:
curl -X POST -is -u username:password \
-H 'Content-Type: application/json' \
https://api.bitbucket.org/2.0/repositories/workspace-name/repo-name/pipelines/ \
-d '
{
"target": {
"type": "pipeline_ref_target",
"ref_type": "branch",
"ref_name": "master",
"selector": {
"type": "custom",
"pattern": "create-tenant"
}
}}'
So the tokens I was using in the curl weren't added to Repository Settings -> User and group access. Got them added and was able to execute it successfully.

Unable to get the payload from GitHub web hook trigger in jenkins pipeline

I have configured a Github web hook with the below settings:
Payload URL: https:///github-webhook/
Content Type: application/x-www-form-urlencoded
Events : Pushes, Pull Requests
The Jenkins job that I have, is a pipeline job that has the below enabled:
Build Trigger: GitHub hook trigger for GITScm polling
With the above configuration, I see that in response to an event ie; push/PR in GitHub, the jenkins job gets triggered successfully. In GitHub, under Recent Deliveries for the web hook, I see the details of the payload and a successful Response of 200.
I am trying to get the payload in Jenkins Pipeline for further processing. I need some details eg: PR URL/PR number, refs type, branch name etc for conditional processing in the Jenkins Pipeline.
I tried accessing the "payload" variable (as mentioned in other stack overflow posts and the documentations available around) and printing it as part of the pipeline, but I have had no luck yet.
So my question is, How can I get the payload from GitHub web hook trigger in my Jenkins Pipeline ?
You need to select Content type: application/json in your webhook in GitHub. Then you would be able to access any variable from the payload GitHub sends as follows: $. pull_request.url for pr url, for example.
Unsure if this is possible.
With the GitHub plugin we use (Pipeline Github), PR number is stored in the variable CHANGE_ID.
PR URL is pretty easy to generate given the PR number. Branch name is stored in the variable BRANCH_NAME. In case of pull requests, the global variable pullRequest is populated with lots of data.
Missing information can be obtained from Github by using their API. Here's an example of checking if PR is "behind", you can modify that to your specific requirements:
def checkPrIsNotBehind(String repo) {
withCredentials([usernamePassword(credentialsId: "<...>",
passwordVariable: 'TOKEN',
usernameVariable: 'USER')]) {
def headers = ' -H "Content-Type: application/json" -H "Authorization: token $TOKEN" '
def url = "https://api.github.com/repos/<...>/<...>/pulls/${env.CHANGE_ID}"
def head_sha = sh (label: "Check PR head SHA",
returnStdout: true,
script: "curl -s ${url} ${headers} | jq -r .head.sha").trim().toUpperCase()
println "PR head sha is ${head_sha}"
headers = ' -H "Accept: application/vnd.github.v3+json" -H "Authorization: token $TOKEN" '
url = "https://api.github.com/repos/<...>/${repo}/compare/${pullRequest.base}...${head_sha}"
def behind_by = sh (label: "Check PR commits behind",
returnStdout: true,
script: "curl -s ${url} ${headers} | jq -r .behind_by").trim().toUpperCase()
if (behind_by != '0') {
currentBuild.result = "ABORTED"
currentBuild.displayName = "#${env.BUILD_NUMBER}-Out of date"
error("The head ref is out of date. Please update your branch.")
}
}
}

Jenkins Webhook Header as Argument to Shell script

I'm trying to trigger a Jenkins job through the webhook using the curl command whenever there is EC2 Spot Instance Interruption Warning with below sample event. All this will be done in AWS lambda from CloudWatch Event Trigger.
{
"version": "0",
"id": "1e5527d7-bb36-4607-3370-4164db56a40e",
"detail-type": "EC2 Spot Instance Interruption Warning",
"source": "aws.ec2",
"account": "123456789012",
"time": "1970-01-01T00:00:00Z",
"region": "us-east-1",
"resources": [
"arn:aws:ec2:us-east-1b:instance/i-0b662ef9931388ba0"
],
"detail": {
"instance-id": "i-0b662ef9931388ba0",
"instance-action": "terminate"
}
}
My aim is to get the instance-id from the event and pass it as a header to the jenkins webhook where the jenkins job gets triggered and this header has to be sent as an argument to the underlying python script in the jenkins job.
I tried the below approach which didn't give me success. And am not sure if this is how it is done. :)
curl -H 'param: instance' https://jenkins.url/generic-webhook-trigger/invoke\?token\=jenkins-job
Generic Jenkins webhook trigger is configured as below.
The final python script in the job is configured as below.
python maintenance.py $.param
I'm trying to get the final script similar to below. Please let me know if you know of any approach how can I get this done. TIA
python maintenance.py i-0b662ef9931388ba
I fixed this by adding the following.
Add string parameter under This project is parameterized as shown below.
Next under Generic Webhook Trigger I have added Header parameters
Now, we can directly pass this param in the build command using $param.
The curl command is now modified to below.
curl --location --request POST 'https://jenkins.url/generic-webhook-trigger/invoke\?token\=jenkins-job' --header 'Content-Type: application/json' --header "param: $EVENT_DATA" --data-raw ''
What did I learn from this?
I was too lazy to read the documentation and finally figured it out only after reading it.

Fetching github payload in jenkinsfile

Need some help on fetching the GitHub payload into the Jenkins file without installing any plugin.
If anyone can provide the Jenkins file code snippet to access the GitHub payload from the webhook. it would be of great help.
I am able to call the Jenkins job from GitHub webhook. but need the payload as well to process further.
Any help would be appreciated. Thanks.
Please find the below groovy script:
stage('Pull Request Info') {
agent {
docker {
image 'maven'
args '-v $HOME/.m2:/home/jenkins/.m2 -ti -u 496 -e MAVEN_CONFIG=/home/jenkins/.m2 -e MAVEN_OPTS=-Xmx2048m'
}
}
steps {
script {
withCredentials([usernameColonPassword(credentialsId: "${env.STASH_CREDENTIAL_ID}",
variable: 'USERPASS')]) {
def hasChanges = maven.updateDependencies()
if (hasChanges != '') {
def pr1 = sh(
script: "curl -s -u ${"$USERPASS" } -H 'Content-Type: application/json' https://xx.example/rest/some/endpoint",
returnStdout: true
)
def pr = readJSON(text: pr1)
println pr.fromRef
}
}
}
}
}
Above script uses, curl to fetch the details about Pull request. I have stored the credentials in the jenkins and created an environment variable for the credentialId.
You can replace the url with your endpoint. You can also modify the script to use jq if you have jq installed in the machine.
In this case, I'm using readJSON to parse the JSON which is part of pipeline utilities plugin. My question would be, why not use the plugin as it provides the needed functionalities?
If you still dont want to use plugin, take a look at json parsing with groovy.

How to update Jenkins Job config.xml file using curl

How can I edit jenkins job's parameter by updating config.xml of jenkins job using curl?
You can use:
curl -X POST 'http://my-cool-jenkins.com:8080/createItem?name=mycooljob' -u username:password --data-binary #config.xml -H "Content-Type:text/xml"
Update:
That url for creating job, for updating use:
curl -X POST 'http://my-cool-jenkins.com:8080/job/mycooljob/config.xml' -u username:password --data-binary #config.xml -H "Content-Type:text/xml"
Just updating content of config.xml file is probably not enough to change in-memory state of Jenkins job. You still need to reload configuration from disk, which can be done either in GUI with jenkins/manage/, using groovy script or simply rebooting the server. After that your example should work.
This really goes down to the fact that Jenkins config.xml are XStream serialized java objects, not actual configuration files. So changing job parameters by manually editing xml files is likely not the best solution. Instead, you could change the job configuration using Jenkins script console. For example to change default parameter value for String parameter, you can run below script in Jenkins console (e.g. http://localhost:8080/jenkins/script):
import hudson.model.ParametersDefinitionProperty
def jobName = "job_name"
def paramName = "param_to_be_changed"
def newParamValue = "param_new_value"
def job = Jenkins.instance.getItem(jobName)
def params = job.getAction(ParametersDefinitionProperty)
def paramToModify = params.getParameterDefinitions().find { param -> param.getName() == paramName }
paramToModify.setDefaultValue(newParamValue)
job.save()
If the job is inside the folder or organization, it is necessary to go one level further, i.e.:
def folderName = "folder_name"
def job = Jenkins.instance.getItem(folderName).getItem(jobName)
After that job state will be stored in config.xml file. After that you can execute the script remotely using curl. Assuming you saved above script to script.groovy file:
# Get breadcrumb from Jenkins
curl -u <username>:<password> 'http://localhost:8080/jenkins/crumbIssuer/api/xml?xpath=concat(//crumbRequestField,":",//crumb)'
# Send script to Jenkins console
curl -X POST -u <username>:<password> -H 'Jenkins-Crumb: <crumb>' -H 'Content: text/plain' --data-urlencode "script=$(< script.groovy)" http://localhost:8080/jenkins/scriptText
More details on Parameter API in javadoc
Here is a link of a script that I've been using in order to modify a job's pipeline for the shell: https://raw.githubusercontent.com/iocanel/presentations/382074b5012d6c3ed87042298114e688424eeaed/workspace/editor/jenkins-run-pipeline

Resources