Unable to push Docker image to Artifactory - docker

I am running a Jenkins Pipeline and trying to upload a docker image but it is failing using the Artifactory command.
This is a snippet of my Jenkinsfile stage:
stage("Build docker image") {
steps {
container('docker') {
sh 'docker -v'
script {
def rtServer = Artifactory.server "artifactory"
def rtDocker = Artifactory.docker server: rtServer
docker.build("app", "--build-arg JAR_FILE=app.jar -f Dockerfile .")
def buildInfo = rtDocker.push '<companyname>.jfrog.io/app','docker-snapshot-local'
}
}
}
}
This fails after the docker.build with the following message:
[Pipeline] newBuildInfo
[Pipeline] dockerPushStep
expected to call org.jfrog.hudson.pipeline.common.types.Docker.push but wound up catching dockerPushStep; see: https://jenkins.io/redirect/pipeline-cps-method-mismatches/
[Pipeline] }
Jenkins EOF log:
Also: hudson.remoting.Channel$CallSiteStackTrace: Remote call to JNLP4-connect connection from ip-XX-XX-XX-XX.ec2.internal/XX.XX.XX.XX:42790
at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1743)
at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
at hudson.remoting.Channel.call(Channel.java:957)
at org.jfrog.hudson.pipeline.common.docker.utils.DockerAgentUtils.getImageIdFromAgent(DockerAgentUtils.java:291)
at org.jfrog.hudson.pipeline.common.executors.DockerExecutor.execute(DockerExecutor.java:59)
at org.jfrog.hudson.pipeline.scripted.steps.DockerPushStep$Execution.run(DockerPushStep.java:104)
at org.jfrog.hudson.pipeline.scripted.steps.DockerPushStep$Execution.run(DockerPushStep.java:71)
at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1$1.call(AbstractSynchronousNonBlockingStepExecution.java:47)
at hudson.security.ACL.impersonate(ACL.java:290)
at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1.run(AbstractSynchronousNonBlockingStepExecution.java:44)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at ...io.netty.channel.epoll.AbstractEpollChannel.doConnect(AbstractEpollChannel.java:713)
at io.netty.channel.epoll.EpollDomainSocketChannel.doConnect(EpollDomainSocketChannel.java:87)
at io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.connect(AbstractEpollChannel.java:555)
at io.netty.channel.DefaultChannelPipeline$HeadContext.connect(DefaultChannelPipeline.java:1366)
at io.netty.channel.AbstractChannelHandlerContext.invokeConnect(AbstractChannelHandlerContext.java:545)
at io.netty.channel.AbstractChannelHandlerContext.connect(AbstractChannelHandlerContext.java:530)
at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.connect(CombinedChannelDuplexHandler.java:497)
at io.netty.channel.ChannelOutboundHandlerAdapter.connect(ChannelOutboundHandlerAdapter.java:47)
at io.netty.channel.CombinedChannelDuplexHandler.connect(CombinedChannelDuplexHandler.java:298)
at io.netty.channel.AbstractChannelHandlerContext.invokeConnect(AbstractChannelHandlerContext.java:545)
at io.netty.channel.AbstractChannelHandlerContext.connect(AbstractChannelHandlerContext.java:530)
at io.netty.channel.AbstractChannelHandlerContext.connect(AbstractChannelHandlerContext.java:512)
at io.netty.channel.DefaultChannelPipeline.connect(DefaultChannelPipeline.java:1024)
at io.netty.channel.AbstractChannel.connect(AbstractChannel.java:259)
at io.netty.bootstrap.Bootstrap$3.run(Bootstrap.java:252)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404)
at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:335)
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:897)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)

The expected to call... is just a warning message and you can ignore it. This may disappear in the next version of workflow-cps-plugin. More information about it can be found here.
About the error - Try first to follow the instructions here HAP-1241.

Related

Conan Artifactory Jenkins Integration fails

We have a problem regarding the Jenkins Artifactory Plug-in using Conan.
Basically, we created our scripted pipeline (following the example in https://github.com/jfrog/project-examples/blob/master/jenkins-examples/pipeline-examples/scripted-examples/conan-example/Jenkinsfile), which set-up the artifactory server, creates the conanClient, performs the creation of a Conan Package (by running through the conanClient the command "conan create", since our project is built via a conanfile.py recipe), uploads the package to our artifactory instance, and finally we want to clear the conan cache (through the "conan remove * -f" command).
But this last step always fails: the problem seems to be caused by the conanClient, which implicity invokes the "conan_build_info" command after the "conan remove * -f". The "conan_build_info" command fails since the "conan_build_info" apparently requires the "package" cache, which is, of course, cleared. Is this a conanClient bug (maybe because, as we read in the documentation, the conan_build_info command is not recommended to be used: https://docs.conan.io/en/latest/reference/commands/misc/conan_build_info.html), or are we missing something?
Is there a way to perform a package creation and upload via the ConanClient, and clear the conan cache without causing the pipeline to fail?
It seems to me this is a big bogus bug, since package creation and upload via CI - Jenkins is a fondumental aspect... and of course, at the end, the conan cache must be cleared...
Here our Jenkinsfile:
node()
{
// Obtain an Artifactory server instance, defined in Jenkins --> Manage:
def server = Artifactory.server "artifactory_server"
// Create a local build-info instance:
def buildInfo = Artifactory.newBuildInfo()
buildInfo.name = "our git master pipeline"
// Create a conan client instance:
def conanClient = Artifactory.newConanClient()
// Add a new repository named 'conan-local' to the conan client.
// The 'remote.add' method returns a 'serverName' string, which is used later in the script:
String serverName = conanClient.remote.add server: server, repo: "conan-local"
// We enable strict ABI dependency propagation
conanClient.run(command: "config set general.default_package_id_mode=package_revision_mode", buildInfo: buildInfo)
conanClient.run(command: "config set general.revisions_enabled=1", buildInfo: buildInfo)
conanClient.run(command: "config set general.full_transitive_package_id=1", buildInfo: buildInfo)
stage('Checkout')
{
// checkout from our repo...
}
stage('Build Release')
{
// Run a conan build. The 'buildInfo' instance is passed as an argument to the 'run' method:
conanClient.run(command: "create ./project_dir channel/channel", buildInfo: buildInfo)
}
stage('Upload')
{
// Create an upload command. The 'serverName' string is used as a conan 'remote', so that the artifacts are uploaded into it:
String command = "upload our_packet/*#*/master --all -r ${serverName} --confirm"
// Run the upload command, with the same build-info instance as an argument:
conanClient.run(command: command, buildInfo: buildInfo)
}
stage("Clear Conan Cache")
{
// Clean all conan cache
String command = "remove * -f"
// Run the remove command, with the same build-info instance as an argument:
conanClient.run(command: command, buildInfo: buildInfo)
}
stage('Publish build info')
{
// Publish the build-info to Artifactory:
server.publishBuildInfo buildInfo
}
}
The error from Jenkins log:
[out_project] $ cmd.exe /C "conan remove "*" -f && exit %%ERRORLEVEL%%"
[out_project] $ cmd.exe /C "conan_build_info D:\jenkins\workspace\out_project#tmp\artifactory\conan.tmp8862200414119894885\conan_log.log --output D:\jenkins\workspace\out_project#tmp\artifactory\conan1112119574977543529build-info && exit %%ERRORLEVEL%%"
[1m[31mERROR: [Errno 2] No such file or directory: 'D:\\.conan\\efa5c5\\1\\conaninfo.txt'[0m
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
java.lang.RuntimeException: Conan build failed
at org.jfrog.hudson.pipeline.common.Utils.launch(Utils.java:256)
at org.jfrog.hudson.pipeline.common.executors.ConanExecutor.execute(ConanExecutor.java:132)
at org.jfrog.hudson.pipeline.common.executors.ConanExecutor.collectConanBuildInfo(ConanExecutor.java:180)
at org.jfrog.hudson.pipeline.common.executors.ConanExecutor.execCommand(ConanExecutor.java:101)
at org.jfrog.hudson.pipeline.scripted.steps.conan.RunCommandStep$Execution.runStep(RunCommandStep.java:50)
at org.jfrog.hudson.pipeline.scripted.steps.conan.RunCommandStep$Execution.runStep(RunCommandStep.java:37)
at org.jfrog.hudson.pipeline.ArtifactorySynchronousNonBlockingStepExecution.run(ArtifactorySynchronousNonBlockingStepExecution.java:42)
at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Finished: FAILURE

docker push to artifactory gives error : "Target host is not specified "

I am trying to push docker image to artifactory using jenkins declarative pipeline. Below is my code-
stage('Push Image to Artifactory') {
steps {
script{
def buildInfo = Artifactory.newBuildInfo()
def server = Artifactory.newServer url: 'artifactory.org', credentialsId: 'artifactsId'
def rtDocker = Artifactory.docker server: server
def tagDockerApp = "artifactory.org/docker-repository/venus:1.0.3"
docker.build(tagDockerApp)
buildInfo = rtDocker.push(tagDockerApp,'docker-repository',buildInfo)
// Publish build info.
buildInfo.env.capture = true
buildInfo.env.collect()
server.publishBuildInfo buildInfo
}
}
}
I am getting below error after rtDocker.push command :
org.apache.http.ProtocolException: Target host is not specified
at org.apache.http.impl.conn.DefaultRoutePlanner.determineRoute(DefaultRoutePlanner.java:71)
at org.apache.http.impl.client.InternalHttpClient.determineRoute(InternalHttpClient.java:125)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
Caused: org.apache.http.client.ClientProtocolException
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:187)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at org.jfrog.build.client.PreemptiveHttpClient.execute(PreemptiveHttpClient.java:102)
at org.jfrog.build.extractor.clientConfiguration.client.ArtifactoryDependenciesClient.execute(ArtifactoryDependenciesClient.java:200)
at org.jfrog.build.extractor.clientConfiguration.client.ArtifactoryDependenciesClient.downloadArtifact(ArtifactoryDependenciesClient.java:170)
at org.jfrog.hudson.pipeline.docker.DockerImage.findAndSetManifestFromArtifactory(DockerImage.java:162)
at org.jfrog.hudson.pipeline.docker.DockerImage.generateBuildInfoModule(DockerImage.java:146)
at org.jfrog.hudson.pipeline.steps.DockerPushStep$Execution.run(DockerPushStep.java:135)
at org.jfrog.hudson.pipeline.steps.DockerPushStep$Execution.run(DockerPushStep.java:86)
at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1$1.call(AbstractSynchronousNonBlockingStepExecution.java:47)
at hudson.security.ACL.impersonate(ACL.java:367)
at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1.run(AbstractSynchronousNonBlockingStepExecution.java:44)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Although i can see the image pushed on to the artifactory , but due to the exception i am not able publish build info and execute other stages
Try providing a full url including protocol (http/https) such as https://artifactory.org in line:
def server = Artifactory.newServer url: 'artifactory.org', credentialsId: 'artifactsId'

Unable to run terraform from Jenkins pipeline in vagrant virtual machine

I have the below pipeline setup which uses terraform plugin to execute terraform.
Here is my environment. A centos7 vagrant vm has been installed terraform, jenkins using user vagrant. Terraform plugin is also installed to Jenkins. Terraform can be run manually. When terraform being used by pipeline, it is failed. Tried a couple of other ways to set up terraform path, all are failed.
Terraform is located at /usr/local/bin/terraform.
pipeline {
agent any
stages {
stage('Set Terraform path') {
steps {
script {
def tfHome = tool name: 'terraform'
env.PATH = "${tfHome}:${env.PATH}"
// def tfhome = tool name: 'terraform', type: 'org.jenkinsci.plugins.terraform.TerraformInstallation'
// env.PATH = "${tfhome}:${env.PATH}"
// def tfHome = tool name: 'terraform', type: 'com.cloudbees.jenkins.plugins.customtools.CustomTool'
// env.PATH = "${tfHome}:${env.PATH}"
echo "tfhome is ${tfhome}, env.PATH is ${env.PATH} "
}
// sh '/usr/local/bin/terraform -version'
sh 'terraform -version'
}
}
}
}
Here is error message:
Failed Node Use a tool from a predefined Tool Installation
hudson.AbortException: No org.jenkinsci.plugins.terraform.TerraformInstallation named terraform found
at org.jenkinsci.plugins.workflow.steps.ToolStep$Execution.run(ToolStep.java:162)
at org.jenkinsci.plugins.workflow.steps.ToolStep$Execution.run(ToolStep.java:133)
at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
I figure it out. The terraform tool cannot be used by user jenkins. So I create a symlink pointing to terraform command. The error is gone!

Build remote multibranch pipeline in Jenkins

I would like to use the parameterized remote trigger plugin to run the remote project(all branches) and monitor only one of the branches for the status.
node(''){
triggerRemoteJob abortTriggeredJob: true, auth: CredentialsAuth(credentials: 'E2E'), job: 'http://localhost:8080/job/test-projectF', maxConn: 5, useCrumbCache: true, useJobInfoCache: true
}
This is my code to trigger the remote project. But I end up with an exception
[Pipeline] {
[Pipeline] triggerRemoteJob################################################################################################################
Parameterized Remote Trigger Configuration:
- job: http://localhost:8080/job/test-projectF
- auth: 'Credentials Authentication' as user 'admin' (Credentials ID 'E2E')
- parameters:
- blockBuildUntilComplete: true
- connectionRetryLimit: 5
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
hudson.remoting.ProxyException: net.sf.json.JSONException: JSONObject["property"] is not a JSONArray.
at net.sf.json.JSONObject.getJSONArray(JSONObject.java:1986)
at org.jenkinsci.plugins.ParameterizedRemoteTrigger.RemoteBuildConfiguration.isRemoteJobParameterized(RemoteBuildConfiguration.java:1086)
at org.jenkinsci.plugins.ParameterizedRemoteTrigger.RemoteBuildConfiguration.performTriggerAndGetQueueId(RemoteBuildConfiguration.java:637)
at org.jenkinsci.plugins.ParameterizedRemoteTrigger.pipeline.RemoteBuildPipelineStep$Execution.run(RemoteBuildPipelineStep.java:263)
at org.jenkinsci.plugins.ParameterizedRemoteTrigger.pipeline.RemoteBuildPipelineStep$Execution.run(RemoteBuildPipelineStep.java:239)
at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution$1$1.call(SynchronousNonBlockingStepExecution.java:51)
at hudson.security.ACL.impersonate(ACL.java:290)
at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution$1.run(SynchronousNonBlockingStepExecution.java:48)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Finished: FAILURE
However, same command with a branch name ends in SUCCESS. http://localhost:8080/job/test-projectF/job/master
What is the better way to build the dynamically created branch on the remote Jenkins and track the status?

How to add timestamp for artifacts in Jenkins

I have following Jenkisfile and I'm trying to upload the artifacts with a timestamp.
import groovy.transform.Field
#Field def timeStamp = Calendar.getInstance().getTime().format('YYYYMMdd-hhmmss',TimeZone.getTimeZone('CST'))
node {
stage('Creating some artifacts') {
sh 'touch hello.txt hi.txt'
}
stage('Uploading artifacts') {
def server = Artifactory.server ('art-1')
def uploadSpec = """{
"files": [
{
"pattern": "*.txt",
"target": "repo1/Dev/${env.BUILD_NUMBER}/*.txt.${timeStamp}"
}
]
}"""
def buildInfo1 = server.upload(uploadSpec)
server.publishBuildInfo(buildInfo1)
}
}
However, I'm getting the following error while trying this.
[consumer_1] Deploying artifact: http://learner.blr.example.com:8081/artifactory/repo1/Dev/12/*.txt.20180913-044451
[Thread consumer_1] An exception occurred during execution:
java.lang.RuntimeException: java.io.IOException: Failed to deploy file. Status code: 500 Response message: Artifactory returned the following errors:
Invalid path. '*' is not a valid name character: repo1/Dev/12/*.txt.20180913-044451 Status code: 500
at org.jfrog.build.extractor.clientConfiguration.util.spec.SpecDeploymentConsumer.consumerRun(SpecDeploymentConsumer.java:44)
at org.jfrog.build.extractor.producerConsumer.ConsumerRunnableBase.run(ConsumerRunnableBase.java:11)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: Failed to deploy file. Status code: 500 Response message: Artifactory returned the following errors:
Invalid path. '*' is not a valid name character: repo1/Dev/12/*.txt.20180913-044451 Status code: 500
at org.jfrog.build.extractor.clientConfiguration.client.ArtifactoryBuildInfoClient.uploadFile(ArtifactoryBuildInfoClient.java:692)
at org.jfrog.build.extractor.clientConfiguration.client.ArtifactoryBuildInfoClient.doDeployArtifact(ArtifactoryBuildInfoClient.java:374)
at org.jfrog.build.extractor.clientConfiguration.client.ArtifactoryBuildInfoClient.deployArtifact(ArtifactoryBuildInfoClient.java:362)
at org.jfrog.build.extractor.clientConfiguration.util.spec.SpecDeploymentConsumer.consumerRun(SpecDeploymentConsumer.java:39)
... 2 more
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
java.lang.Exception: Error occurred during operation, please refer to logs for more information.
at org.jfrog.build.extractor.producerConsumer.ProducerConsumerExecutor.start(ProducerConsumerExecutor.java:84)
at org.jfrog.build.extractor.clientConfiguration.util.spec.SpecsHelper.uploadArtifactsBySpec(SpecsHelper.java:71)
at org.jfrog.hudson.generic.GenericArtifactsDeployer$FilesDeployerCallable.invoke(GenericArtifactsDeployer.java:190)
Caused: java.lang.RuntimeException: Failed uploading artifacts by spec
at org.jfrog.hudson.generic.GenericArtifactsDeployer$FilesDeployerCallable.invoke(GenericArtifactsDeployer.java:194)
at org.jfrog.hudson.generic.GenericArtifactsDeployer$FilesDeployerCallable.invoke(GenericArtifactsDeployer.java:131)
at hudson.FilePath.act(FilePath.java:1042)
at hudson.FilePath.act(FilePath.java:1025)
at org.jfrog.hudson.pipeline.executors.GenericUploadExecutor.execution(GenericUploadExecutor.java:52)
at org.jfrog.hudson.pipeline.steps.UploadStep$Execution.run(UploadStep.java:65)
at org.jfrog.hudson.pipeline.steps.UploadStep$Execution.run(UploadStep.java:46)
at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1$1.call(AbstractSynchronousNonBlockingStepExecution.java:47)
at hudson.security.ACL.impersonate(ACL.java:290)
at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1.run(AbstractSynchronousNonBlockingStepExecution.java:44)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Finished: FAILURE
Are there any alternative/simple way to add timestamp in artifacts in Jenkins?
P.S.: I'm new to Jenkins groovy scripting and JFrog
The error message says that * is an invalid character for a file name so I don't think you can use it in the target field. However, the artifactory docs say you can do this instead (links for docs bellow):
def uploadSpec = """{
"files": [
{
"pattern": "(*).txt",
"target": "repo1/Dev/${env.BUILD_NUMBER}/{1}.txt.${timeStamp}"
}
]
In this code, {1} stands for "whatever got matched inside the first parenthesis in the pattern" (every open+close parenthesis in a regex defines a capture group).
Note: I don't use artifactory so I didn't test the above code, I am going off of the artifactory docs:
https://www.jfrog.com/confluence/display/RTF/Using+File+Specs
https://www.jfrog.com/confluence/display/RTF/Using+File+Specs#UsingFileSpecs-UsingPlaceholders
I'd also suggest you move the timestamp to the file name instead of the file extension, so that when you download the file, your computer knows which program to use to open it. So i'd change target to something like:
files shorted fist by name then by timestamp: repo1/Dev/${env.BUILD_NUMBER}/{1}-${timeStamp}.txt
files shorted fist by timestamp then by name:
repo1/Dev/${env.BUILD_NUMBER}/${timeStamp}-{1}.txt

Resources