jenkins pipeline giving inexplicable NullPointerException at GenericDownloadExecutor.java: 52 - jenkins

I followed the example given in this page to download artifacts from JFrog artifactory:
https://www.jfrog.com/confluence/display/JFROG/Scripted+Pipeline+Syntax#ScriptedPipelineSyntax-UploadingandDownloadingFiles
But I have been getting NullPointerException. Below is the script output and error:
[Pipeline] echo
download spec {
"files": [
{
"pattern": "aod-libs-release/path-to-artifact/deployment/DM-DP-V3A.24-RELEASE/deployment-DM-DP-V3A.24-RELEASE.zip",
"target": "./artifacts"
}, {
"pattern": "aod-libs-release/path-to-artifact/dm-auth-web-services/DM-AU-V3A.88-RELEASE/dm-auth-web-services-DM-AU-V3A.88-RELEASE.war",
"target": "./artifacts"
}
]
}
[Pipeline] echo
artifactory server org.jfrog.hudson.pipeline.common.types.ArtifactoryServer#66026154
[Pipeline] newBuildInfo
[Pipeline] artifactoryDownload
[Pipeline] End of Pipeline
java.lang.NullPointerException
at org.jfrog.hudson.pipeline.common.executors.GenericDownloadExecutor.execute(GenericDownloadExecutor.java:52)
at org.jfrog.hudson.pipeline.scripted.steps.DownloadStep$Execution.runStep(DownloadStep.java:67)
at org.jfrog.hudson.pipeline.scripted.steps.DownloadStep$Execution.runStep(DownloadStep.java:53)
at org.jfrog.hudson.pipeline.ArtifactorySynchronousNonBlockingStepExecution.run(ArtifactorySynchronousNonBlockingStepExecution.java:54)
at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Finished: FAILURE
Below is my jenkins pipeline script
def artifactoryServer = Artifactory.server("Artifactory-instance-id")
artifactoryServer.connection.timeout = 300
def artifactDownloadSpec = """{
"files": [
{
"pattern": "aod-libs-release/path-to-artifact/deployment/${env.DP_VERSION}/deployment-${env.DP_VERSION}.zip",
"target": "./artifacts"
}, {
"pattern": "aod-libs-release/path-to-artifact/dm-auth-web-services/${env.VERSION}/dm-auth-web-services-${env.VERSION}.war",
"target": "./artifacts"
}
]
}"""
echo "download spec " + artifactDownloadSpec
echo "artifactory server " + artifactoryServer
artifactoryServer.download spec: artifactDownloadSpec
I googled for GenericDownloadExecutor.java and found below code, but cannot figure out what I am missing that line 52 needed. Looks like a field is not set.
https://github.com/jenkinsci/artifactory-plugin/blob/master/src/main/java/org/jfrog/hudson/pipeline/common/executors/GenericDownloadExecutor.java

Running artifactoryServer.download step requires using an agent.
For example:
Jenkins scripted pipeline
node {
...
artifactoryServer.download spec: artifactDownloadSpec
...
}
Jenkins declarative pipeline
The recommended way is to use the rtDownload step, however, if you still want to use artifactoryServer.download:
pipeline {
agent any // <- Make sure there is an active agent
stages {
stage('Example') {
steps {
script {
...
artifactoryServer.download spec: artifactDownloadSpec
...
}
}
}
}
}
Read more:
Uploading and Downloading Files
Scripted pipeline example
Declarative pipeline example
Scripted pipeline inside declarative pipeline example - Less recommended, but this example is like your case

Related

JFrog Artifactory upload tar via Jenkins failes with 404

Since a few days we got an error on a Jenkins Pipeline when we want to upload a tar archive via spec to the Artifactory.
All the Maven builds with goal deploy works well and the artifacts gets deployed.
I checked the docs, logs and Stackoverflow.
At the moment I have no further ideas except asking here.
This is the error message of the console output from the Jenkins Pipeline:
[...]
[Pipeline] script
[Pipeline] {
[Pipeline] newBuildInfo
[Pipeline] artifactoryUpload
Executing command: /bin/sh -c git log --pretty=format:%s -1
[consumer_0] Deploying artifact: http://artifactory.name.de/artifactory/snapshots/dir/app-name/release-bundles/1.62.0/app-name-release-bundle-1.62.0.tar.gz
Failed to upload file
[consumer_0] An exception occurred during execution:
java.lang.RuntimeException: java.io.IOException: JFrog service failed. Received 404: <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>404 Not Found</title>
</head><body>
<h1>Not Found</h1>
<p>The requested URL was not found on this server.</p>
</body></html>
at org.jfrog.build.extractor.clientConfiguration.util.spec.SpecDeploymentConsumer.consumerRun(SpecDeploymentConsumer.java:44)
at org.jfrog.build.extractor.producerConsumer.ConsumerRunnableBase.run(ConsumerRunnableBase.java:11)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.io.IOException: JFrog service failed. Received 404: <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>404 Not Found</title>
</head><body>
<h1>Not Found</h1>
<p>The requested URL was not found on this server.</p>
</body></html>
at org.jfrog.build.extractor.clientConfiguration.client.JFrogService.throwException(JFrogService.java:49)
at org.jfrog.build.extractor.clientConfiguration.client.artifactory.services.Upload.handleUnsuccessfulResponse(Upload.java:59)
at org.jfrog.build.extractor.clientConfiguration.client.JFrogService.execute(JFrogService.java:121)
at org.jfrog.build.extractor.clientConfiguration.client.artifactory.services.Upload.execute(Upload.java:77)
at org.jfrog.build.extractor.clientConfiguration.client.artifactory.ArtifactoryManager.upload(ArtifactoryManager.java:262)
at org.jfrog.build.extractor.clientConfiguration.client.artifactory.ArtifactoryManager.upload(ArtifactoryManager.java:257)
at org.jfrog.build.extractor.clientConfiguration.util.spec.SpecDeploymentConsumer.consumerRun(SpecDeploymentConsumer.java:39)
... 2 more
[...]
That's the relevant part of the pipeline with the upload spec:
def artifactoryServer = Artifactory.server 'art1'
[...]
pipeline {
stages {
stage('Transfer release bundle to artifactory.') {
steps {
script {
parentArtifactId = parentPom.artifactId
def repository = "snapshots"
if(params.branchName == "master") {
repository = "releases";
}
def uploadReleaseArchiveSpec = """{
"files": [
{
"pattern": "release-bundle-${releaseBundleVersion}.tar.gz",
"target": "${repository}/path/to/file/${parentArtifactId}/release-bundles/${parentPom.version}/"
}
]
}"""
artifactoryServer.upload spec: uploadReleaseArchiveSpec, failNoOp: true
}
}
}
}
}
In the router-request.log I found this:
{
"ClientAddr": "127.0.0.1:56828",
"DownstreamContentSize": 95,
"DownstreamStatus": 404,
"Duration": 2531738,
"RequestMethod": "GET",
"RequestPath": "/access/api/v1/users/jffe#000?expand=groups",
"ServiceAddr": "localhost:8040",
"StartUTC": "2022-09-08T09:39:50.224317922Z",
"level": "info",
"msg": "",
"request_Uber-Trace-Id": "745779c57b007818:30fa1347695348a7:062656bb9a6ef6c9:0",
"request_User-Agent": "JFrog Access Java Client/7.43.6 74306900 Artifactory/7.38.10 73810900",
"time": "2022-09-08T11:39:50+02:00"
}
There is also an error message in the catalina log from Tomcat of Artifactory:
08-Sep-2022 13:52:42.392 SEVERE [http-nio-127.0.0.1-8091-Acceptor] org.apache.tomcat.util.net.Acceptor.run Socket accept failed
java.io.IOException: Duplicate accept detected. This is a known OS bug. Please consider reporting that you are affected: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1924298
at org.apache.tomcat.util.net.NioEndpoint.serverSocketAccept(NioEndpoint.java:548)
at org.apache.tomcat.util.net.NioEndpoint.serverSocketAccept(NioEndpoint.java:78)
at org.apache.tomcat.util.net.Acceptor.run(Acceptor.java:129)
at java.base/java.lang.Thread.run(Thread.java:829)
But we are using RedHat Enterprise Linux and not Ubuntu. Otherwise there is an error.
Our environment:
Jenkins 2.346.3
Artifactory OSS 7.38.10
Jenkins Artifactory Plug-in 3.17.0 (and 3.16.2 former attempts)
RedHat Enterprise Linux 8.6
Has anybody an idea? Did I overseen something in the logs above? Where else should I look?
I saw also this post. Unfortunately without answer.
UPDATE - 2022-09-13
I found the issue:
The upload works with a directly in the Jenkins Job (via the corresponding text area) inserted Pipeline script and it doesn't work with the same Pipeline script requested from a Git repository.
Now, it would be interesting to know why the second approach fails.
I guess you are using your own private Jfrog? In that case, does the Jenkins machine have access to your JFrog? Can you ssh into your Jenkins machine and upload a random artifact using curl?
This may not be the best answer, but when dealing with Jfrog, I always simply use curl within the Jenkins pipeline to upload artifacts:
curl -u <user>:<password> -X PUT "<jfrog_url>/path/to/upload/artifact.zip" -T "path\on\jenkins\artifact.zip"
I hade the same issue. The problem is Jenkins injects GIT_ environment variables when checking out Pipeline from SCM.
I worked around the problem by resetting GIT_ environment variables.
pipeline {
agent {
label("some-agent")
}
environment {
GIT_BRANCH = ""
GIT_COMMIT = ""
GIT_PREVIOUS_COMMIT = ""
GIT_PREVIOUS_SUCCESSFUL_COMMIT = ""
GIT_URL = ""
}
stages {
stage("Push to Artifactory") {
steps {
rtUpload(
serverId: "your-artifactory-instance-id",
"buildName": "Your-Build-Name",
"failNoOp": true,
spec: """
{
"files": [
{
"pattern": "target/app.jar",
"target": "libs-release-local/some/path/in/artifactory/app.jar"
}
]
}
""".stripIndent()
)
rtPublishBuildInfo(
serverId: "your-artifactory-instance-id",
"buildName": "Your-Build-Name"
)
}
}
}
}

How do I download an Artifactory artifact that contains parenthesis in the name?

Using Jenkins declarative pipeline and an Artifactory file spec, how do I download an Artifactory artifact that contains parenthesis in the artifact name? Is there a way to escape the parenthesis?
For example, I have two artifacts in my Artifactory repository:
default-generic-local/one/two/aaabbbccc(1234).txt
default-generic-local/one/two/aaabbbccc1234.txt
When I run the pipeline defined below, it downloads aaabbbccc1234.txt. I would expect it to download aaabbbccc(1234).txt instead.
Here's an example of the pipeline script and file spec I'm using with my pipeline job:
pipeline {
agent any
stages {
stage('Download') {
steps {
rtServer(
id: 'my-art-server',
url: 'https://my.artifactory.url',
credentialsId: 'my-artifactory-creds')
rtDownload(
serverId: 'my-art-server',
spec: '''
{
"files": [
{
"pattern": "default-generic-local/one/two/aaabbbccc(1234).txt",
"target": "output/",
"flat": "true"
}
]
}''',
failNoOp: true)
}
}
}
post {
always {
cleanWs()
}
}
}

How to get the payload information from jfrog webhook trigger on jenkins

I'm setting up a jenkins job which will be triggered whenever a artifact is deployed in jfrog. I have followed the steps present in the below documentation and i was able to trigger the job.But unfortunately, I'm not able to get the variables values.I'm not sure how to see the payload which we are receiving on the jenkins side to pull the required variables.
https://www.eficode.com/blog/triggering-jenkins-pipelines-on-artifactory-events
My pipeline:
pipeline {
agent any
triggers {
GenericTrigger(
genericVariables: [
[key: 'ARTIFACT_NAME', value: '$.artifactory.webhook.data.name'],
[event: 'EVENT_NAME', value: '$.artifactory.webhook.event']
],
causeString: 'Triggered on $ARTIFACT_NAME'
)
}
stages {
stage('Hello') {
steps {
echo 'Hello World'
}
}
}
}
Jenkins job logs:
java.lang.NullPointerException: Variable name
at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:204)
at org.jenkinsci.plugins.gwt.GenericVariable.<init>(GenericVariable.java:31)
Caused: java.lang.reflect.InvocationTargetException
at sun.reflect.GeneratedConstructorAccessor6020.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.jenkinsci.plugins.structs.describable.DescribableModel.instantiate(DescribableModel.java:330)
Caused: java.lang.IllegalArgumentException: Could not instantiate {event=EVENT_NAME, value=$.artifactory.webhook.event} for org.jenkinsci.plugins.gwt.GenericVariable
at org.jenkinsci.plugins.structs.describable.DescribableModel.instantiate(DescribableModel.java:334)
at org.jenkinsci.plugins.structs.describable.DescribableModel.coerce(DescribableModel.java:474)
at org.jenkinsci.plugins.structs.describable.DescribableModel.coerceList(DescribableModel.java:585)
at org.jenkinsci.plugins.structs.describable.DescribableModel.coerce(DescribableModel.java:458)
at org.jenkinsci.plugins.structs.describable.DescribableModel.buildArguments(DescribableModel.java:409)
at org.jenkinsci.plugins.structs.describable.DescribableModel.instantiate(DescribableModel.java:329)
at org.jenkinsci.plugins.structs.describable.DescribableModel.instantiate(DescribableModel.java:272)
at org.jenkinsci.plugins.pipeline.modeldefinition.CommonUtils.instantiateDescribable(CommonUtils.java:131)
at org.jenkinsci.plugins.pipeline.modeldefinition.CommonUtils$instantiateDescribable.callStatic(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallStatic(CallSiteArray.java:56)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callStatic(AbstractCallSite.java:194)
at org.kohsuke.groovy.sandbox.impl.Checker$2.call(Checker.java:194)
at org.kohsuke.groovy.sandbox.GroovyInterceptor.onStaticCall(GroovyInterceptor.java:35)
at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onStaticCall(SandboxInterceptor.java:186)
at org.kohsuke.groovy.sandbox.impl.Checker$2.call(Checker.java:192)
at org.kohsuke.groovy.sandbox.impl.Checker.checkedStaticCall(Checker.java:196)
at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:103)
at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:17)
Caused: java.lang.IllegalArgumentException: Could not instantiate {genericVariables=[{key=ARTIFACT_NAME, value=$.artifactory.webhook.data.name}, {event=EVENT_NAME, value=$.artifactory.webhook.event}], causeString=Triggered on $ARTIFACT_NAME} for org.jenkinsci.plugins.gwt.GenericTrigger
I have made it. We have to enable print post contents in configure jenkins job under generic webhook trigger option. Once done it started to display json payload in console log. After that we can add variables using jsonpath filter from payload
I'd like to suggest a different approach using the Jenkins Artifactory plugin:
You can configure a build trigger in the UI:
Or you can configure the build trigger in the pipeline:
stages {
stage('Artifactory configuration') {
steps {
rtServer(
id: "ARTIFACTORY_SERVER",
url: SERVER_URL,
credentialsId: CREDENTIALS
)
}
}
stage('Add build trigger') {
steps {
rtBuildTrigger(
serverId: "ARTIFACTORY_SERVER",
spec: "*/10 * * * *",
paths: "generic-libs-local/builds/starship"
)
}
}
}
To get the path in Artifactory that caused the trigger you can do the following:
environment {
// The URL of the artifact in Artifactory, caused the job to be triggered.
// May be empty if the build isn't triggered by a change in Artifactory.
RT_TRIGGER_URL = "${currentBuild.getBuildCauses('org.jfrog.hudson.trigger.ArtifactoryCause')[0]?.url}"
}
Resources:
Triggering builds
Triggering Builds in Declarative pipeline
Example

How to add timestamp for artifacts in Jenkins

I have following Jenkisfile and I'm trying to upload the artifacts with a timestamp.
import groovy.transform.Field
#Field def timeStamp = Calendar.getInstance().getTime().format('YYYYMMdd-hhmmss',TimeZone.getTimeZone('CST'))
node {
stage('Creating some artifacts') {
sh 'touch hello.txt hi.txt'
}
stage('Uploading artifacts') {
def server = Artifactory.server ('art-1')
def uploadSpec = """{
"files": [
{
"pattern": "*.txt",
"target": "repo1/Dev/${env.BUILD_NUMBER}/*.txt.${timeStamp}"
}
]
}"""
def buildInfo1 = server.upload(uploadSpec)
server.publishBuildInfo(buildInfo1)
}
}
However, I'm getting the following error while trying this.
[consumer_1] Deploying artifact: http://learner.blr.example.com:8081/artifactory/repo1/Dev/12/*.txt.20180913-044451
[Thread consumer_1] An exception occurred during execution:
java.lang.RuntimeException: java.io.IOException: Failed to deploy file. Status code: 500 Response message: Artifactory returned the following errors:
Invalid path. '*' is not a valid name character: repo1/Dev/12/*.txt.20180913-044451 Status code: 500
at org.jfrog.build.extractor.clientConfiguration.util.spec.SpecDeploymentConsumer.consumerRun(SpecDeploymentConsumer.java:44)
at org.jfrog.build.extractor.producerConsumer.ConsumerRunnableBase.run(ConsumerRunnableBase.java:11)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: Failed to deploy file. Status code: 500 Response message: Artifactory returned the following errors:
Invalid path. '*' is not a valid name character: repo1/Dev/12/*.txt.20180913-044451 Status code: 500
at org.jfrog.build.extractor.clientConfiguration.client.ArtifactoryBuildInfoClient.uploadFile(ArtifactoryBuildInfoClient.java:692)
at org.jfrog.build.extractor.clientConfiguration.client.ArtifactoryBuildInfoClient.doDeployArtifact(ArtifactoryBuildInfoClient.java:374)
at org.jfrog.build.extractor.clientConfiguration.client.ArtifactoryBuildInfoClient.deployArtifact(ArtifactoryBuildInfoClient.java:362)
at org.jfrog.build.extractor.clientConfiguration.util.spec.SpecDeploymentConsumer.consumerRun(SpecDeploymentConsumer.java:39)
... 2 more
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
java.lang.Exception: Error occurred during operation, please refer to logs for more information.
at org.jfrog.build.extractor.producerConsumer.ProducerConsumerExecutor.start(ProducerConsumerExecutor.java:84)
at org.jfrog.build.extractor.clientConfiguration.util.spec.SpecsHelper.uploadArtifactsBySpec(SpecsHelper.java:71)
at org.jfrog.hudson.generic.GenericArtifactsDeployer$FilesDeployerCallable.invoke(GenericArtifactsDeployer.java:190)
Caused: java.lang.RuntimeException: Failed uploading artifacts by spec
at org.jfrog.hudson.generic.GenericArtifactsDeployer$FilesDeployerCallable.invoke(GenericArtifactsDeployer.java:194)
at org.jfrog.hudson.generic.GenericArtifactsDeployer$FilesDeployerCallable.invoke(GenericArtifactsDeployer.java:131)
at hudson.FilePath.act(FilePath.java:1042)
at hudson.FilePath.act(FilePath.java:1025)
at org.jfrog.hudson.pipeline.executors.GenericUploadExecutor.execution(GenericUploadExecutor.java:52)
at org.jfrog.hudson.pipeline.steps.UploadStep$Execution.run(UploadStep.java:65)
at org.jfrog.hudson.pipeline.steps.UploadStep$Execution.run(UploadStep.java:46)
at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1$1.call(AbstractSynchronousNonBlockingStepExecution.java:47)
at hudson.security.ACL.impersonate(ACL.java:290)
at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1.run(AbstractSynchronousNonBlockingStepExecution.java:44)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Finished: FAILURE
Are there any alternative/simple way to add timestamp in artifacts in Jenkins?
P.S.: I'm new to Jenkins groovy scripting and JFrog
The error message says that * is an invalid character for a file name so I don't think you can use it in the target field. However, the artifactory docs say you can do this instead (links for docs bellow):
def uploadSpec = """{
"files": [
{
"pattern": "(*).txt",
"target": "repo1/Dev/${env.BUILD_NUMBER}/{1}.txt.${timeStamp}"
}
]
In this code, {1} stands for "whatever got matched inside the first parenthesis in the pattern" (every open+close parenthesis in a regex defines a capture group).
Note: I don't use artifactory so I didn't test the above code, I am going off of the artifactory docs:
https://www.jfrog.com/confluence/display/RTF/Using+File+Specs
https://www.jfrog.com/confluence/display/RTF/Using+File+Specs#UsingFileSpecs-UsingPlaceholders
I'd also suggest you move the timestamp to the file name instead of the file extension, so that when you download the file, your computer knows which program to use to open it. So i'd change target to something like:
files shorted fist by name then by timestamp: repo1/Dev/${env.BUILD_NUMBER}/{1}-${timeStamp}.txt
files shorted fist by timestamp then by name:
repo1/Dev/${env.BUILD_NUMBER}/${timeStamp}-{1}.txt

Create Artifact in Archiva through Jenkins Pipeline / Gradle

I'm looking to create and upload an artifact to Archiva through a Jenkins pipeline node. I've found plenty of documentation for doing this in Artifactory, but I am having trouble getting footing on how to handle this with Archiva.
For reference, the Artifactory equivalent of what I'm trying to do is something on the order of:
node {
def server = Artifactory.server 'my-server-id'
stage('Build') {
// ...
}
stage('Test') {
// ...
}
// ...
stage('Archive') {
def uploadSpec = """{
"files": [
{
"pattern": "build/files",
"target": "repo/path/"
}
]
}"""
server.upload(uploadSpec)
}
}
But now I want to handle this in Archiva instead (or with a generic Maven repository in general). For what it's worth, I'm using a Gradle build system, if getting Jenkins to tell Gradle to upload to Archiva would be an easier prospect.

Resources