Upload multiple files using s3upload in Jenkins pipeline - jenkins

Question

Can we upload multiple files (not entire folder) to S3 using s3Upload in Jenkins file?
I was trying to upload all rpm files (*.rpm) in the root directory to S3 using the s3Upload function.

Answer 1

You can upload all the files with following command in one line.
s3Upload(bucket:"my-bucket", path:'path/to/targetFolder/', includePathPattern:'**/*.svg', workingDir:'dist')
Further explaining, You can create own filtering based on following two possibilities;
1.Include all the files of a certain extention.
s3Upload(bucket:"my-bucket", path:'path/to/targetFolder/', includePathPattern:'**/*.svg', workingDir:'dist')
2.Include all the files except certain file extention.
s3Upload(bucket:"my-bucket", path:'path/to/targetFolder/', includePathPattern:'**/*', workingDir:'dist', excludePathPattern:'**/*.svg')
Reference: https://github.com/jenkinsci/pipeline-aws-plugin (Check under s3Upload)

Answer 2

findFiles solved the issue. Below is the snippet used for the same.
files = findFiles(glob: '*.rpm')
files.each {
println "RPM: ${it}"
withAWS(credentials: '****************'){
s3Upload(file:"${it}", bucket:'rpm-repo', path:"${bucket_path}")
}
}

Answer 3

Refer to the following link AWS s3 documentation. In that, refer section 'Use of Exclude and Include Filters'
Here is a way to upload multiple files of a particular type.
If you only want to upload files with a particular extension, you need to first exclude all files, then re-include the files with the particular extension. This command will upload only files ending with .jpg:
aws s3 cp /tmp/foo/ s3://bucket/ --recursive --exclude "*" --include "*.jpg"
This works for AWS Command Line Interface.

Answer 4

For pipelines, you need to wrap the iteration in script, like
pipeline {
environment {
// Extract concise branch name.
BRANCH = GIT_BRANCH.substring(GIT_BRANCH.lastIndexOf('/') + 1, GIT_BRANCH.length())
}
...
post {
success {
script {
def artifacts = ['file1', 'dir2/file3']
artifacts.each {
withAWS(credentials:'my-aws-token', region:'eu-west-1') {
s3Upload(
file: "build/${it}",
bucket: 'my-artifacts',
path: 'my-repo/',
metadatas: ["repo:${env.JOB_NAME}", "branch:${env.BRANCH}", "commit:${env.GIT_COMMIT}"]
)
}
}
}
}
}
}