Since a few days we got an error on a Jenkins Pipeline when we want to upload a tar archive via spec to the Artifactory.
All the Maven builds with goal deploy works well and the artifacts gets deployed.
I checked the docs, logs and Stackoverflow.
At the moment I have no further ideas except asking here.
This is the error message of the console output from the Jenkins Pipeline:
[...]
[Pipeline] script
[Pipeline] {
[Pipeline] newBuildInfo
[Pipeline] artifactoryUpload
Executing command: /bin/sh -c git log --pretty=format:%s -1
[consumer_0] Deploying artifact: http://artifactory.name.de/artifactory/snapshots/dir/app-name/release-bundles/1.62.0/app-name-release-bundle-1.62.0.tar.gz
Failed to upload file
[consumer_0] An exception occurred during execution:
java.lang.RuntimeException: java.io.IOException: JFrog service failed. Received 404: <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>404 Not Found</title>
</head><body>
<h1>Not Found</h1>
<p>The requested URL was not found on this server.</p>
</body></html>
at org.jfrog.build.extractor.clientConfiguration.util.spec.SpecDeploymentConsumer.consumerRun(SpecDeploymentConsumer.java:44)
at org.jfrog.build.extractor.producerConsumer.ConsumerRunnableBase.run(ConsumerRunnableBase.java:11)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.io.IOException: JFrog service failed. Received 404: <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>404 Not Found</title>
</head><body>
<h1>Not Found</h1>
<p>The requested URL was not found on this server.</p>
</body></html>
at org.jfrog.build.extractor.clientConfiguration.client.JFrogService.throwException(JFrogService.java:49)
at org.jfrog.build.extractor.clientConfiguration.client.artifactory.services.Upload.handleUnsuccessfulResponse(Upload.java:59)
at org.jfrog.build.extractor.clientConfiguration.client.JFrogService.execute(JFrogService.java:121)
at org.jfrog.build.extractor.clientConfiguration.client.artifactory.services.Upload.execute(Upload.java:77)
at org.jfrog.build.extractor.clientConfiguration.client.artifactory.ArtifactoryManager.upload(ArtifactoryManager.java:262)
at org.jfrog.build.extractor.clientConfiguration.client.artifactory.ArtifactoryManager.upload(ArtifactoryManager.java:257)
at org.jfrog.build.extractor.clientConfiguration.util.spec.SpecDeploymentConsumer.consumerRun(SpecDeploymentConsumer.java:39)
... 2 more
[...]
That's the relevant part of the pipeline with the upload spec:
def artifactoryServer = Artifactory.server 'art1'
[...]
pipeline {
stages {
stage('Transfer release bundle to artifactory.') {
steps {
script {
parentArtifactId = parentPom.artifactId
def repository = "snapshots"
if(params.branchName == "master") {
repository = "releases";
}
def uploadReleaseArchiveSpec = """{
"files": [
{
"pattern": "release-bundle-${releaseBundleVersion}.tar.gz",
"target": "${repository}/path/to/file/${parentArtifactId}/release-bundles/${parentPom.version}/"
}
]
}"""
artifactoryServer.upload spec: uploadReleaseArchiveSpec, failNoOp: true
}
}
}
}
}
In the router-request.log I found this:
{
"ClientAddr": "127.0.0.1:56828",
"DownstreamContentSize": 95,
"DownstreamStatus": 404,
"Duration": 2531738,
"RequestMethod": "GET",
"RequestPath": "/access/api/v1/users/jffe#000?expand=groups",
"ServiceAddr": "localhost:8040",
"StartUTC": "2022-09-08T09:39:50.224317922Z",
"level": "info",
"msg": "",
"request_Uber-Trace-Id": "745779c57b007818:30fa1347695348a7:062656bb9a6ef6c9:0",
"request_User-Agent": "JFrog Access Java Client/7.43.6 74306900 Artifactory/7.38.10 73810900",
"time": "2022-09-08T11:39:50+02:00"
}
There is also an error message in the catalina log from Tomcat of Artifactory:
08-Sep-2022 13:52:42.392 SEVERE [http-nio-127.0.0.1-8091-Acceptor] org.apache.tomcat.util.net.Acceptor.run Socket accept failed
java.io.IOException: Duplicate accept detected. This is a known OS bug. Please consider reporting that you are affected: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1924298
at org.apache.tomcat.util.net.NioEndpoint.serverSocketAccept(NioEndpoint.java:548)
at org.apache.tomcat.util.net.NioEndpoint.serverSocketAccept(NioEndpoint.java:78)
at org.apache.tomcat.util.net.Acceptor.run(Acceptor.java:129)
at java.base/java.lang.Thread.run(Thread.java:829)
But we are using RedHat Enterprise Linux and not Ubuntu. Otherwise there is an error.
Our environment:
Jenkins 2.346.3
Artifactory OSS 7.38.10
Jenkins Artifactory Plug-in 3.17.0 (and 3.16.2 former attempts)
RedHat Enterprise Linux 8.6
Has anybody an idea? Did I overseen something in the logs above? Where else should I look?
I saw also this post. Unfortunately without answer.
UPDATE - 2022-09-13
I found the issue:
The upload works with a directly in the Jenkins Job (via the corresponding text area) inserted Pipeline script and it doesn't work with the same Pipeline script requested from a Git repository.
Now, it would be interesting to know why the second approach fails.
I guess you are using your own private Jfrog? In that case, does the Jenkins machine have access to your JFrog? Can you ssh into your Jenkins machine and upload a random artifact using curl?
This may not be the best answer, but when dealing with Jfrog, I always simply use curl within the Jenkins pipeline to upload artifacts:
curl -u <user>:<password> -X PUT "<jfrog_url>/path/to/upload/artifact.zip" -T "path\on\jenkins\artifact.zip"
I hade the same issue. The problem is Jenkins injects GIT_ environment variables when checking out Pipeline from SCM.
I worked around the problem by resetting GIT_ environment variables.
pipeline {
agent {
label("some-agent")
}
environment {
GIT_BRANCH = ""
GIT_COMMIT = ""
GIT_PREVIOUS_COMMIT = ""
GIT_PREVIOUS_SUCCESSFUL_COMMIT = ""
GIT_URL = ""
}
stages {
stage("Push to Artifactory") {
steps {
rtUpload(
serverId: "your-artifactory-instance-id",
"buildName": "Your-Build-Name",
"failNoOp": true,
spec: """
{
"files": [
{
"pattern": "target/app.jar",
"target": "libs-release-local/some/path/in/artifactory/app.jar"
}
]
}
""".stripIndent()
)
rtPublishBuildInfo(
serverId: "your-artifactory-instance-id",
"buildName": "Your-Build-Name"
)
}
}
}
}
Related
node("DevHub && WinServer2019"){
cleanWs()
stage ("in windows docker") {
withDockerRegistry(credentialsId: "ABC_Technical_User", url: "https://abc-swc-build-toolchains-docker-dev") {
dockerImage = docker.image("abc-swc-build-toolchains-docker-dev/ci-snapshot/mt4/windows_vc100:image-13-77d3d3b")
dockerImage.pull()
String rtDocker = ''
def server = Artifactory.newServer url: 'https://abc/artifactory/', credentialsId: 'ABC_Technical_User'
def buildInfo = Artifactory.newBuildInfo()
// Step 2: Create an Artifactory Docker instance:
def bdDocker = Artifactory.docker server: server
buildInfo localBuildInfo = bdDocker.pull 'abc-swc-build-toolchains-docker-dev/ci-snapshot/mt4/windows_vc100:base-image-13-77d3d3b', 'abc-swc-build-toolchains-docker-dev'
// Step 4: Publish the build-info to Artifactory:
buildInfo.append localBuildInfo
echo "buildInfo : ${buildInfo}"
server.publishBuildInfo buildInfo
}
}
}
I am getting below error
INFO: Pulling image: abc-swc-build-toolchains-docker-dev/ci-snapshot/mt4/windows_vc100:base-image-13-77d3d3b
**java.lang.RuntimeException: Unsupported OS**
at com.github.dockerjava.netty.NettyDockerCmdExecFactory$UnixDomainSocketInitializer.init(NettyDockerCmdExecFactory.java:147)
at com.github.dockerjava.netty.NettyDockerCmdExecFactory.init(NettyDockerCmdExecFactory.java:116)
at com.github.dockerjava.core.DockerClientImpl.withDockerCmdExecFactory(DockerClientImpl.java:193)
at com.github.dockerjava.core.DockerClientBuilder.build(DockerClientBuilder.java:45)
at org.jfrog.build.extractor.docker.DockerJavaWrapper.getDockerClient(DockerJavaWrapper.java:77)
at org.jfrog.build.extractor.docker.DockerJavaWrapper.pullImage(DockerJavaWrapper.java:150)
at org.jfrog.build.extractor.docker.extractor.DockerPull.execute(DockerPull.java:76)
at org.jfrog.build.extractor.packageManager.PackageManagerExtractor.executeAndSaveBuildInfo(PackageManagerExtractor.java:33)
at org.jfrog.build.extractor.docker.extractor.DockerPull.main(DockerPull.java:61)
java.lang.RuntimeException: docker build failed
Can someone help me on this?
Would it be possible for you to test the scenario with the latest Jenkins Artifactory plugin and Java 11? I remember a similar issue reported in the past due to a lower Java version/old plugin as well. Upgrading the plugin to the latest version along with the respective Java version mentioned, should help in resolving the issue.
I am trying to write a pipeline script to publish *.war/*.jar file to JFrogArtifactory. I don't find any syntax for the same.
Anyone can help me out on the same.
please help me with a sample script.
JFrog has a dedicated GitHub repository with many examples for such cases.
There are Jenkins Pipelines examples there.
First, you must install Artifactory Plugin and config it in Jenkins server.
Refer: https://www.jfrog.com/confluence/display/JFROG/Configuring+Jenkins+Artifactory+Plug-in
And then try add below script to Jenkinsfile:
script {
def server = Artifactory.server '<artifactory id>'
def uploadSpec = '''{
"files": [{
"pattern": "<name of war or jar file>",
"target": "<artifactory repo>/path-to/war-or-jar/file/in-Artifactory"
}]
}'''
server.upload(uploadSpec)
}
Don't forget replace <artifactory id> <name of war or jar file> and <artifactory repo>/path-to/war-or-jar/file/in-Artifactory
More information: https://www.jfrog.com/confluence/display/JFROG/Declarative+Pipeline+Syntax
The scripted pipeline syntax for deploying war files to JFrog artifactory is :
env.ARTIFACTORY = 'True'
if(env.ARTIFACTORY == 'True')
{
stage('Deploying to Artifactory')
{
FAILED_STAGE = env.STAGE_NAME
bat 'mvn deploy'
}
}
Note :
1.) 'bat' command is for Windows batch file. If you're using Linux, replace 'bat' with 'sh'
2.) env.ARTIFACTORY is used to give you control over whether or not you want to execute this particular stage in your pipeline job. if you don't want this stage to execute, simply set env.ARTIFACTORY = 'False'
3.) Also note, you've to configure JFrog in : Manage Jenkins -> Configure system -> JFrog Platform Instances
4.) Include JFrog in your pom.xml file under distributionManagement tag.
I have following Jenkisfile and I'm trying to upload the artifacts with a timestamp.
import groovy.transform.Field
#Field def timeStamp = Calendar.getInstance().getTime().format('YYYYMMdd-hhmmss',TimeZone.getTimeZone('CST'))
node {
stage('Creating some artifacts') {
sh 'touch hello.txt hi.txt'
}
stage('Uploading artifacts') {
def server = Artifactory.server ('art-1')
def uploadSpec = """{
"files": [
{
"pattern": "*.txt",
"target": "repo1/Dev/${env.BUILD_NUMBER}/*.txt.${timeStamp}"
}
]
}"""
def buildInfo1 = server.upload(uploadSpec)
server.publishBuildInfo(buildInfo1)
}
}
However, I'm getting the following error while trying this.
[consumer_1] Deploying artifact: http://learner.blr.example.com:8081/artifactory/repo1/Dev/12/*.txt.20180913-044451
[Thread consumer_1] An exception occurred during execution:
java.lang.RuntimeException: java.io.IOException: Failed to deploy file. Status code: 500 Response message: Artifactory returned the following errors:
Invalid path. '*' is not a valid name character: repo1/Dev/12/*.txt.20180913-044451 Status code: 500
at org.jfrog.build.extractor.clientConfiguration.util.spec.SpecDeploymentConsumer.consumerRun(SpecDeploymentConsumer.java:44)
at org.jfrog.build.extractor.producerConsumer.ConsumerRunnableBase.run(ConsumerRunnableBase.java:11)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: Failed to deploy file. Status code: 500 Response message: Artifactory returned the following errors:
Invalid path. '*' is not a valid name character: repo1/Dev/12/*.txt.20180913-044451 Status code: 500
at org.jfrog.build.extractor.clientConfiguration.client.ArtifactoryBuildInfoClient.uploadFile(ArtifactoryBuildInfoClient.java:692)
at org.jfrog.build.extractor.clientConfiguration.client.ArtifactoryBuildInfoClient.doDeployArtifact(ArtifactoryBuildInfoClient.java:374)
at org.jfrog.build.extractor.clientConfiguration.client.ArtifactoryBuildInfoClient.deployArtifact(ArtifactoryBuildInfoClient.java:362)
at org.jfrog.build.extractor.clientConfiguration.util.spec.SpecDeploymentConsumer.consumerRun(SpecDeploymentConsumer.java:39)
... 2 more
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
java.lang.Exception: Error occurred during operation, please refer to logs for more information.
at org.jfrog.build.extractor.producerConsumer.ProducerConsumerExecutor.start(ProducerConsumerExecutor.java:84)
at org.jfrog.build.extractor.clientConfiguration.util.spec.SpecsHelper.uploadArtifactsBySpec(SpecsHelper.java:71)
at org.jfrog.hudson.generic.GenericArtifactsDeployer$FilesDeployerCallable.invoke(GenericArtifactsDeployer.java:190)
Caused: java.lang.RuntimeException: Failed uploading artifacts by spec
at org.jfrog.hudson.generic.GenericArtifactsDeployer$FilesDeployerCallable.invoke(GenericArtifactsDeployer.java:194)
at org.jfrog.hudson.generic.GenericArtifactsDeployer$FilesDeployerCallable.invoke(GenericArtifactsDeployer.java:131)
at hudson.FilePath.act(FilePath.java:1042)
at hudson.FilePath.act(FilePath.java:1025)
at org.jfrog.hudson.pipeline.executors.GenericUploadExecutor.execution(GenericUploadExecutor.java:52)
at org.jfrog.hudson.pipeline.steps.UploadStep$Execution.run(UploadStep.java:65)
at org.jfrog.hudson.pipeline.steps.UploadStep$Execution.run(UploadStep.java:46)
at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1$1.call(AbstractSynchronousNonBlockingStepExecution.java:47)
at hudson.security.ACL.impersonate(ACL.java:290)
at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1.run(AbstractSynchronousNonBlockingStepExecution.java:44)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Finished: FAILURE
Are there any alternative/simple way to add timestamp in artifacts in Jenkins?
P.S.: I'm new to Jenkins groovy scripting and JFrog
The error message says that * is an invalid character for a file name so I don't think you can use it in the target field. However, the artifactory docs say you can do this instead (links for docs bellow):
def uploadSpec = """{
"files": [
{
"pattern": "(*).txt",
"target": "repo1/Dev/${env.BUILD_NUMBER}/{1}.txt.${timeStamp}"
}
]
In this code, {1} stands for "whatever got matched inside the first parenthesis in the pattern" (every open+close parenthesis in a regex defines a capture group).
Note: I don't use artifactory so I didn't test the above code, I am going off of the artifactory docs:
https://www.jfrog.com/confluence/display/RTF/Using+File+Specs
https://www.jfrog.com/confluence/display/RTF/Using+File+Specs#UsingFileSpecs-UsingPlaceholders
I'd also suggest you move the timestamp to the file name instead of the file extension, so that when you download the file, your computer knows which program to use to open it. So i'd change target to something like:
files shorted fist by name then by timestamp: repo1/Dev/${env.BUILD_NUMBER}/{1}-${timeStamp}.txt
files shorted fist by timestamp then by name:
repo1/Dev/${env.BUILD_NUMBER}/${timeStamp}-{1}.txt
I have a private GitHub Rust project that depends on another private GitHub Rust project and I want to build the main one with Jenkins. I have called the organization Organization and the dependency package subcrate in the below code.
My Jenkinsfile looks something like
pipeline {
agent {
docker {
image 'rust:latest'
}
}
stages {
stage('Build') {
steps {
sh "cargo build"
}
}
etc...
}
}
I have tried the following in Cargo.toml to reference the dependency, it works fine on my machine
[dependencies]
subcrate = { git = "ssh://git#ssh.github.com/Organization/subcrate.git", tag = "0.1.0" }
When Jenkins runs I get the following error
+ cargo build
Updating registry `https://github.com/rust-lang/crates.io-index`
Updating git repository `ssh://git#github.com/Organization/subcrate.git`
error: failed to load source for a dependency on `subcrate`
Caused by:
Unable to update ssh://git#github.com/Organization/subcrate.git?tag=0.1.0#0623c097
Caused by:
failed to clone into: /usr/local/cargo/git/db/subcrate-3e391025a927594e
Caused by:
failed to authenticate when downloading repository
attempted ssh-agent authentication, but none of the usernames `git` succeeded
Caused by:
error authenticating: no auth sock variable; class=Ssh (23)
script returned exit code 101
How can I get Cargo to access this GitHub repository? Do I need to inject the GitHub credentials onto the slave? If so, how can I do this? Is it possible to use the same credentials Jenkins uses to checkout the main crate in the first place?
I installed the ssh-agent plugin and updated my Jenkinsfile to look like this
pipeline {
agent {
docker {
image 'rust:latest'
}
}
stages {
stage('Build') {
steps {
sshagent(credentials: ['id-of-github-credentials']) {
sh "ssh -vvv -T git#github.com"
sh "cargo build"
}
}
}
etc...
}
}
I get the error
+ ssh -vvv -T git#github.com
No user exists for uid 113
script returned exit code 255
Okay, I figured it out, No user exists for uid error is because of a mismatch between the users in the host /etc/passwd and the container /etc/passwd. This can be fixed by mounting /etc/passwd.
agent {
docker {
image 'rust:latest'
args '-v /etc/passwd:/etc/passwd'
}
}
Then
sshagent(credentials: ['id-of-github-credentials']) {
sh "cargo build"
}
Works just fine
I am trying to use Nexus Jenkins Plugin to upload a release artifact with its pom file.
The Nexus repository is configured with a deployment policy of "disable redeploy" so that releases can't be overriden.
To simplify the problem imagine I only want to upload my custom pom:
pipeline {
agent any
stages {
stage('Publish') {
steps {
nexusPublisher nexusInstanceId: 'nexusJose', nexusRepositoryId: 'nexusJose',
packages: [
[$class: 'MavenPackage',
mavenAssetList: [
[classifier: '',
extension: 'pom',
filePath: "/libs/mylib-4.6.0.pom"],
],
mavenCoordinate: [
artifactId: "mylib",
groupId: "com.codependent.libs",
packaging: "pom", version: "4.6.0"]
]
]
}
}
}
}
For some reason when nexusPublisher executes, Nexus is creating a default pom for those coordinates before performing the upload of the pom, so the upload of the actual pom fails as it already exists in the repository:
Uploading Maven asset with groupId: mylib artifactId: com.codependent.libs version: 4.6.0 To repository: thirdparty
Upload of /libs/mylib-4.6.0.pom failed
Failing build due to failure to upload file to Nexus Repository Manager Publisher
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
com.sonatype.nexus.api.exception.RepositoryManagerException: Unable to upload component: Bad Request <html><body><error>Repository with ID='thirdparty' does not allow updating artifacts.</error></body></html>
How can I upload an artifact with its own pom?
Actually, you don't need to configure uploading pom.xml file - Nexus Jenkins Plugin will upload your pom.xml file automatically. You just need to configure uploading of your release artifact.
So, in your case you can use the following configuration and your 'mylib-4.6.0.pom' should be uploaded automatically:
pipeline {
agent any
stages {
stage('Publish') {
steps {
nexusPublisher nexusInstanceId: 'nexusJose', nexusRepositoryId: 'nexusJose',
packages: [
[$class: 'MavenPackage',
mavenAssetList: [
[classifier: '',
extension: '',
filePath: "${path-to-artifact}/mylib-4.6.0.jar"],
],
mavenCoordinate: [
artifactId: "mylib",
groupId: "com.codependent.libs",
packaging: "jar", version: "4.6.0"]
]
]
}
}
}
}
This page can be useful:
https://support.sonatype.com/hc/en-us/articles/227256688-How-do-I-configure-the-Nexus-Jenkins-Plugin