I am trying to run jenkins with code to remove old build from artifactory ,i only need to keep last 10 builds in artifactory however the builds are not getting deleted frm artifacty` below is the snip of pipeline using
stage("Artifactory Upload") {
def directory = "generic-local/ABC/XYZ/${params['version']}/${currDate}/${buildNo}/"
def server = Artifactory.server 'art-int'
def buildInfo = Artifactory.newBuildInfo()
buildInfo.retention maxBuilds: 10, deleteBuildArtifacts: true
def upload_spec_bin = """{
"files": [
{
"pattern": "*",
"target": "${directory}",
"exclusions": ["*.txt"]
}
]
}"""
server.upload spec: upload_spec_bin
server.publishBuildInfo buildInfo
}
I'm getting same issue listed as fixed here : https://issues.jenkins.io/browse/JENKINS-58643
We are using Jenkins 2.190.3.2
stage('upload artefactory') {
steps {
sh "touch /tmp/blabla"
sh "gzip /tmp/blabla"
script {
server = Artifactory.server('myid')
server.credentialsId = 'my-cred'
def uploadSpec = """{
"files": [
{
"pattern": "/tmp/blabla.gz",
"target": "pkg/com/myentreprise/mystuff/scm/dumps/solr/"
}
]
}"""
server.upload spec: uploadSpec, failNoOp: true
}
}
}
[Pipeline] artifactoryUpload
expected to call org.jfrog.hudson.pipeline.common.types.ArtifactoryServer.upload but wound up catching artifactoryUpload; see: https://jenkins.io/redirect/pipeline-cps-method-mismatches/
this was fixed in Pipeline Groovy 2.75
We have 2.74, thus why we still ahve this bug.
Only solution seems to be to upgrade.
pipeline {
agent any
stages {
stage("foo") {
steps {
script {
env.RELEASE_SCOPE = input message: 'User input required', ok: 'Release!',
parameters: [choice(name: 'RELEASE_SCOPE', choices: 'patch\nminor\nmajor',
description: 'What is the release scope?')]
}
echo "${env.RELEASE_SCOPE}"
}
}
}
}
In this above code, The choice are hardcoded (patch\nminor\nmajor) -- My requirement is to dynamically give choice values in the dropdown.
I get the values from calling api - Artifacts list (.zip) file names from artifactory
In the above example, It request input when we do the build, But i want to do a "Build with parameters"
Please suggest/help on this.
Depends how you get data from API there will be different options for it, for example let's imagine that you get data as a List of Strings (let's call it releaseScope), in that case your code be following:
...
script {
def releaseScopeChoices = ''
releaseScope.each {
releaseScopeChoices += it + '\n'
}
parameters: [choice(name: 'RELEASE_SCOPE', choices: ${releaseScopeChoices}, description: 'What is the release scope?')]
}
...
hope it will help.
This is a cutdown version of what we use. We separate stuff into shared libraries but I have consolidated a bit to make it easier.
Jenkinsfile looks something like this:
#!groovy
#Library('shared') _
def imageList = pipelineChoices.artifactoryArtifactSearchList(repoName, env.BRANCH_NAME)
imageList.add(0, 'build')
properties([
buildDiscarder(logRotator(numToKeepStr: '20')),
parameters([
choice(name: 'ARTIFACT_NAME', choices: imageList.join('\n'), description: '')
])
])
Shared library that looks at artifactory, its pretty simple.
Essentially make GET Request (And provide auth creds on it) then filter/split result to whittle down to desired values and return list to Jenkinsfile.
import com.cloudbees.groovy.cps.NonCPS
import groovy.json.JsonSlurper
import java.util.regex.Pattern
import java.util.regex.Matcher
List artifactoryArtifactSearchList(String repoKey, String artifact_name, String artifact_archive, String branchName) {
// URL components
String baseUrl = "https://org.jfrog.io/org/api/search/artifact"
String url = baseUrl + "?name=${artifact_name}&repos=${repoKey}"
Object responseJson = getRequest(url)
String regexPattern = "(.+)${artifact_name}-(\\d+).(\\d+).(\\d+).${artifact_archive}\$"
Pattern regex = ~ regexPattern
List<String> outlist = responseJson.results.findAll({ it['uri'].matches(regex) })
List<String> artifactlist=[]
for (i in outlist) {
artifactlist.add(i['uri'].tokenize('/')[-1])
}
return artifactlist.reverse()
}
// Artifactory Get Request - Consume in other methods
Object getRequest(url_string){
URL url = url_string.toURL()
// Open connection
URLConnection connection = url.openConnection()
connection.setRequestProperty ("Authorization", basicAuthString())
// Open input stream
InputStream inputStream = connection.getInputStream()
#NonCPS
json_data = new groovy.json.JsonSlurper().parseText(inputStream.text)
// Close the stream
inputStream.close()
return json_data
}
// Artifactory Get Request - Consume in other methods
Object basicAuthString() {
// Retrieve password
String username = "artifactoryMachineUsername"
String credid = "artifactoryApiKey"
#NonCPS
credentials_store = jenkins.model.Jenkins.instance.getExtensionList(
'com.cloudbees.plugins.credentials.SystemCredentialsProvider'
)
credentials_store[0].credentials.each { it ->
if (it instanceof org.jenkinsci.plugins.plaincredentials.StringCredentials) {
if (it.getId() == credid) {
apiKey = it.getSecret()
}
}
}
// Create authorization header format using Base64 encoding
String userpass = username + ":" + apiKey;
String basicAuth = "Basic " + javax.xml.bind.DatatypeConverter.printBase64Binary(userpass.getBytes());
return basicAuth
}
I could achieve it without any plugin:
With Jenkins 2.249.2 using a declarative pipeline,
the following pattern prompt the user with a dynamic dropdown menu
(for him to choose a branch):
(the surrounding withCredentials bloc is optional, required only if your script and jenkins configuration do use credentials)
node {
withCredentials([[$class: 'UsernamePasswordMultiBinding',
credentialsId: 'user-credential-in-gitlab',
usernameVariable: 'GIT_USERNAME',
passwordVariable: 'GITLAB_ACCESS_TOKEN']]) {
BRANCH_NAMES = sh (script: 'git ls-remote -h https://${GIT_USERNAME}:${GITLAB_ACCESS_TOKEN}#dns.name/gitlab/PROJS/PROJ.git | sed \'s/\\(.*\\)\\/\\(.*\\)/\\2/\' ', returnStdout:true).trim()
}
}
pipeline {
agent any
parameters {
choice(
name: 'BranchName',
choices: "${BRANCH_NAMES}",
description: 'to refresh the list, go to configure, disable "this build has parameters", launch build (without parameters)to reload the list and stop it, then launch it again (with parameters)'
)
}
stages {
stage("Run Tests") {
steps {
sh "echo SUCCESS on ${BranchName}"
}
}
}
}
The drawback is that one should refresh the jenkins configration and use a blank run for the list be refreshed using the script ...
Solution (not from me): This limitation can be made less anoying using an aditional parameters used to specifically refresh the values:
parameters {
booleanParam(name: 'REFRESH_BRANCHES', defaultValue: false, description: 'refresh BRANCH_NAMES branch list and launch no step')
}
then wihtin stage:
stage('a stage') {
when {
expression {
return ! params.REFRESH_BRANCHES.toBoolean()
}
}
...
}
this is my solution.
def envList
def dockerId
node {
envList = "defaultValue\n" + sh (script: 'kubectl get namespaces --no-headers -o custom-columns=":metadata.name"', returnStdout: true).trim()
}
pipeline {
agent any
parameters {
choice(choices: "${envList}", name: 'DEPLOYMENT_ENVIRONMENT', description: 'please choose the environment you want to deploy?')
booleanParam(name: 'SECURITY_SCAN',defaultValue: false, description: 'container vulnerability scan')
}
The example of Jenkinsfile below contains AWS CLI command to get the list of Docker images from AWS ECR dynamically, but it can be replaced with your own command. Active Choices Plug-in is required.
Note! You need to approve the script specified in parameters after first run in "Manage Jenkins" -> "In-process Script Approval", or open job configuration and save it to approve
automatically (might require administrator permissions).
properties([
parameters([[
$class: 'ChoiceParameter',
choiceType: 'PT_SINGLE_SELECT',
name: 'image',
description: 'Docker image',
filterLength: 1,
filterable: false,
script: [
$class: 'GroovyScript',
fallbackScript: [classpath: [], sandbox: false, script: 'return ["none"]'],
script: [
classpath: [],
sandbox: false,
script: '''\
def repository = "frontend"
def aws_ecr_cmd = "aws ecr list-images" +
" --repository-name ${repository}" +
" --filter tagStatus=TAGGED" +
" --query imageIds[*].[imageTag]" +
" --region us-east-1 --output text"
def aws_ecr_out = aws_ecr_cmd.execute() | "sort -V".execute()
def images = aws_ecr_out.text.tokenize().reverse()
return images
'''.stripIndent()
]
]
]])
])
pipeline {
agent any
stages {
stage('First stage') {
steps {
sh 'echo "${image}"'
}
}
}
}
choiceArray = [ "patch" , "minor" , "major" ]
properties([
parameters([
choice(choices: choiceArray.collect { "$it\n" }.join(' ') ,
description: '',
name: 'SOME_CHOICE')
])
])
I'm having trouble downloading a build from my artifactory server to my windows jenkins slave node using the jenkins pipeline plugin. It all appears to be going fine, but it doesn't actually download the file. Am I doing something wrong?
I don't see any requests in my Artifactory system logs to download, just to upload.
(2017-04-25 18:39:48,096 [http-nio-8081-exec-2] [INFO ] (o.a.e.UploadServiceImpl:516) - Deploy to 'BUILDS:windows/5840/build.tar.gz' Content-Length: 278600525)
I've been using this as a reference: https://wiki.jenkins-ci.org/pages/viewpage.action?pageId=99910084
Here's the output from my jenkins pipeline:
For pattern: build.tar.gz 1 artifacts were found.
Deploying artifact: http://myartifactory:8081/artifactory/BUILDS/windows/5840/build.tar.gz
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] timeout
Timeout set to expire in 3 min 0 sec
[Pipeline] {
[Pipeline] node
Running on test-windows-0 in C:/jenkinsroot/workspace/test-windows
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Test)
[Pipeline] echo
{
"files": [
{
"pattern": "BUILDS/windows/5840/build.tar.gz",
"target": "download/",
}
]
}
[Pipeline] echo
Artifactory Download: BUILDS/windows/5840/build.tar.gz -> download/
The file exists on artifactory.
Here's my jenkins code:
#NonCPS
def downloadArtifactory(String localPath, String repository, String remotePath) {
def downloadSpec = """{
"files": [
{
"pattern": "${repository}/${remotePath}",
"target": "${localPath}",
}
]
}"""
echo "${downloadSpec}"
echo "Artifactory Download: ${repository}/${remotePath} -> ${localPath}"
def server = Artifactory.server("MYARTIFACTORYSERVER")
def buildInfo = server.download spec: downloadSpec
return buildInfo
}
Called with:
downloadArtifactory("download/", "BUILDS", "windows/5840/build.tar.gz")
Removing the NonCPS annotation should solve the problem.
As you can see in this Jenkins issue, Artifactory Jenkins plugin does not support NonCPS.
Please remove the ,(comma) from the line "target": "${localPath}"
,
It works
make it,
def downloadArtifactory(String localPath, String repository, String remotePath) {
def downloadSpec = """{
"files": [
{
"pattern": "${repository}/${remotePath}",
"target": "${localPath}"
}
]
}"""
echo "${downloadSpec}"
echo "Artifactory Download: ${repository}/${remotePath} -> ${localPath}"
def server = Artifactory.server("MYARTIFACTORYSERVER")
def buildInfo = server.download spec: downloadSpec
return buildInfo
}
i have a groovy method that now creates XML files. I have verified it using the groovyConsole. But if i use this snippet in my jenkinsfile, the XML file is not seen to be created in the workspace, although the job completes successfully.
Question: how do i make sure that the XML file is generated in the workspace? I will be using this XML for subsequent stages in the jenkinsfile
Here is how the jenkinsfile looks like:
import groovy.xml.*
node('master') {
deleteDir()
stage('Checkout') {
// checks out the code
}
generateXML("deploy.xml") //This calls the method to generate the XML file
//stage for packaging
//Stage to Publish
//Stage to Deploy
}
#NonCPS
def generateXML(file1) {
println "Generating the manifest XML........"
def workflows = [
[ name: 'A', file: 'fileA', objectName: 'wf_A', objectType: 'workflow', sourceRepository: 'DEV2', folderNames: [ multifolder: '{{multifolderTST}}', multifolder2: '{{multifolderTST2}}' ]],
[ name: 'B', file: 'fileB', objectName: 'wf_B', objectType: 'workflow', sourceRepository: 'DEV2', folderNames: [ multifolder3: '{{multifolderTST3}}', multifolder4: '{{multifolderTST4}}']]
]
def builder = new StreamingMarkupBuilder()
builder.encoding = 'UTF-8'
new File(file1).newWriter() << builder.bind {
mkp.xmlDeclaration()
mkp.declareNamespace(udm :'http://www.w3.org/2001/XMLSchema')
mkp.declareNamespace(powercenter:'http://www.w3.org/2001/XMLSchema')
delegate.udm.DeploymentPackage(version:'$BUILD_NUMBER', application: "informaticaApp"){
delegate.deployables {
workflows.each { item ->
delegate.powercenter.PowercenterXml(name:item.name, file:item.file) {
delegate.scanPlaceholders(true)
delegate.sourceRepository(item.sourceRepository)
delegate.folderNameMap {
item.folderNames.each { name, value ->
it.entry(key:name, value)
}
}
delegate.objectNames {
delegate.value(item.objectName)
}
delegate.objectTypes {
delegate.value(item.objectType)
}
}
}
}
delegate.dependencyResolution('LATEST')
delegate.undeployDependencies(false)
}
}
}
I found the file in the / dir.. As i haven't entered any path in the filewriter.
UPDATE:
this is not the right solution for a distributed env. It appears that the java file io operations only works in master and not in the agent machines.