Unable to run terraform from Jenkins pipeline in vagrant virtual machine - jenkins

I have the below pipeline setup which uses terraform plugin to execute terraform.
Here is my environment. A centos7 vagrant vm has been installed terraform, jenkins using user vagrant. Terraform plugin is also installed to Jenkins. Terraform can be run manually. When terraform being used by pipeline, it is failed. Tried a couple of other ways to set up terraform path, all are failed.
Terraform is located at /usr/local/bin/terraform.
pipeline {
agent any
stages {
stage('Set Terraform path') {
steps {
script {
def tfHome = tool name: 'terraform'
env.PATH = "${tfHome}:${env.PATH}"
// def tfhome = tool name: 'terraform', type: 'org.jenkinsci.plugins.terraform.TerraformInstallation'
// env.PATH = "${tfhome}:${env.PATH}"
// def tfHome = tool name: 'terraform', type: 'com.cloudbees.jenkins.plugins.customtools.CustomTool'
// env.PATH = "${tfHome}:${env.PATH}"
echo "tfhome is ${tfhome}, env.PATH is ${env.PATH} "
}
// sh '/usr/local/bin/terraform -version'
sh 'terraform -version'
}
}
}
}
Here is error message:
Failed Node Use a tool from a predefined Tool Installation
hudson.AbortException: No org.jenkinsci.plugins.terraform.TerraformInstallation named terraform found
at org.jenkinsci.plugins.workflow.steps.ToolStep$Execution.run(ToolStep.java:162)
at org.jenkinsci.plugins.workflow.steps.ToolStep$Execution.run(ToolStep.java:133)
at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

I figure it out. The terraform tool cannot be used by user jenkins. So I create a symlink pointing to terraform command. The error is gone!

Related

Conan Artifactory Jenkins Integration fails

We have a problem regarding the Jenkins Artifactory Plug-in using Conan.
Basically, we created our scripted pipeline (following the example in https://github.com/jfrog/project-examples/blob/master/jenkins-examples/pipeline-examples/scripted-examples/conan-example/Jenkinsfile), which set-up the artifactory server, creates the conanClient, performs the creation of a Conan Package (by running through the conanClient the command "conan create", since our project is built via a conanfile.py recipe), uploads the package to our artifactory instance, and finally we want to clear the conan cache (through the "conan remove * -f" command).
But this last step always fails: the problem seems to be caused by the conanClient, which implicity invokes the "conan_build_info" command after the "conan remove * -f". The "conan_build_info" command fails since the "conan_build_info" apparently requires the "package" cache, which is, of course, cleared. Is this a conanClient bug (maybe because, as we read in the documentation, the conan_build_info command is not recommended to be used: https://docs.conan.io/en/latest/reference/commands/misc/conan_build_info.html), or are we missing something?
Is there a way to perform a package creation and upload via the ConanClient, and clear the conan cache without causing the pipeline to fail?
It seems to me this is a big bogus bug, since package creation and upload via CI - Jenkins is a fondumental aspect... and of course, at the end, the conan cache must be cleared...
Here our Jenkinsfile:
node()
{
// Obtain an Artifactory server instance, defined in Jenkins --> Manage:
def server = Artifactory.server "artifactory_server"
// Create a local build-info instance:
def buildInfo = Artifactory.newBuildInfo()
buildInfo.name = "our git master pipeline"
// Create a conan client instance:
def conanClient = Artifactory.newConanClient()
// Add a new repository named 'conan-local' to the conan client.
// The 'remote.add' method returns a 'serverName' string, which is used later in the script:
String serverName = conanClient.remote.add server: server, repo: "conan-local"
// We enable strict ABI dependency propagation
conanClient.run(command: "config set general.default_package_id_mode=package_revision_mode", buildInfo: buildInfo)
conanClient.run(command: "config set general.revisions_enabled=1", buildInfo: buildInfo)
conanClient.run(command: "config set general.full_transitive_package_id=1", buildInfo: buildInfo)
stage('Checkout')
{
// checkout from our repo...
}
stage('Build Release')
{
// Run a conan build. The 'buildInfo' instance is passed as an argument to the 'run' method:
conanClient.run(command: "create ./project_dir channel/channel", buildInfo: buildInfo)
}
stage('Upload')
{
// Create an upload command. The 'serverName' string is used as a conan 'remote', so that the artifacts are uploaded into it:
String command = "upload our_packet/*#*/master --all -r ${serverName} --confirm"
// Run the upload command, with the same build-info instance as an argument:
conanClient.run(command: command, buildInfo: buildInfo)
}
stage("Clear Conan Cache")
{
// Clean all conan cache
String command = "remove * -f"
// Run the remove command, with the same build-info instance as an argument:
conanClient.run(command: command, buildInfo: buildInfo)
}
stage('Publish build info')
{
// Publish the build-info to Artifactory:
server.publishBuildInfo buildInfo
}
}
The error from Jenkins log:
[out_project] $ cmd.exe /C "conan remove "*" -f && exit %%ERRORLEVEL%%"
[out_project] $ cmd.exe /C "conan_build_info D:\jenkins\workspace\out_project#tmp\artifactory\conan.tmp8862200414119894885\conan_log.log --output D:\jenkins\workspace\out_project#tmp\artifactory\conan1112119574977543529build-info && exit %%ERRORLEVEL%%"
[1m[31mERROR: [Errno 2] No such file or directory: 'D:\\.conan\\efa5c5\\1\\conaninfo.txt'[0m
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
java.lang.RuntimeException: Conan build failed
at org.jfrog.hudson.pipeline.common.Utils.launch(Utils.java:256)
at org.jfrog.hudson.pipeline.common.executors.ConanExecutor.execute(ConanExecutor.java:132)
at org.jfrog.hudson.pipeline.common.executors.ConanExecutor.collectConanBuildInfo(ConanExecutor.java:180)
at org.jfrog.hudson.pipeline.common.executors.ConanExecutor.execCommand(ConanExecutor.java:101)
at org.jfrog.hudson.pipeline.scripted.steps.conan.RunCommandStep$Execution.runStep(RunCommandStep.java:50)
at org.jfrog.hudson.pipeline.scripted.steps.conan.RunCommandStep$Execution.runStep(RunCommandStep.java:37)
at org.jfrog.hudson.pipeline.ArtifactorySynchronousNonBlockingStepExecution.run(ArtifactorySynchronousNonBlockingStepExecution.java:42)
at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Finished: FAILURE

Jenkins and terraform plugin, terraform file not found

I’m running Jenkins v2.263.2 with Terraform plugin on Debian 10.
I have created a freestyle project to trigger my terraform deployment, which is first pulling from my git repo and the trigger terraform, but at the terraform step I’m getting this error:
FATAL: java.io.FileNotFoundException: Configuration path not found [/var/lib/jenkins/workspace/Terraform/terraform/test.tf].
at org.jenkinsci.plugins.terraform.TerraformBuildWrapper.setupWorkspace(TerraformBuildWrapper.java:367)
at org.jenkinsci.plugins.terraform.TerraformBuildWrapper.executeGet(TerraformBuildWrapper.java:208)
at org.jenkinsci.plugins.terraform.TerraformBuildWrapper.setUp(TerraformBuildWrapper.java:256)
at hudson.model.Build$BuildExecution.doRun(Build.java:157)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:514)
at hudson.model.Run.execute(Run.java:1907)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Finished: FAILURE
Also if I look at the OS level I do have access to this file with jenkins user, so I’m not clear what is wrong with my setup.
jenkins#jenkins:~/workspace/Terraform/terraform$ cat test.tf
data "xenorchestra_template" "template" {
name_label = "Galaxy_Template"
}
data "xenorchestra_network" "net" {
name_label = "ETH"
}
resource "xenorchestra_vm" "bar" {
memory_max = 1073733632
cpus = 1
name_label = "MyName"
name_description = "Mydescription"
template = data.xenorchestra_template.template.id
network {
network_id = data.xenorchestra_network.net.id
}
disk {
sr_id = "c7e5cf80-2b3e-c0ed-6f1e-09f1417b7d24"
name_label = "WDisk"
size = 62212254720
}
}
First, check your setup, if your node path to Terraform binaries is correct
node {
env.PATH += ":/opt/terraform_0.7.xx/"
stage ('Terraform Plan') {
sh 'terraform plan -no-color -out=create.tfplan'
}
Second, check if the terraform binary is installed on the jenkins slave that is executing the pipeline, the binary must be installed to have the plugin work

Jenkinsfile is failing with error #tmp/durable-df843027/script.sh: line 1: terraform: command not found

I'm trying to run a terraform commands from Jenkinsfile stage. The code I'm using is as below:
node {
checkout(scm)
stage ('Templates Deployment'){
sh "terraform init"
}
}
This fails with the error as :
+terraform init
/var/lib/jenkins/workspace/Terraform-Code/#tmp/durable-df843027/script.sh: line 1: terraform: command not found
Terraform is installed on the Jenkins server. When I execute the terraform init command from the server(CLI), it works fine.
But while running it from the Jenkinsfile(console) it's throwing this error.
Can someone please suggest how this error can be resolved? Any help to execute terraform commands via Jenkinsfile is highly appreciated.
Configure Terraform
Go to Manage Jenkins > Global Tool Configuration > It will display Terraform on the list.
give full path of terraform binary or set PATH before terraform init
`node {
checkout(scm)
stage ('Templates Deployment'){
sh """
PATH=/bin/terraform
terraform init"
}
}`
You can set the PATH inside the environment block:
pipeline {
agent any
environment {
PATH = "/usr/local/bin/:$PATH"
}
stages{
stage("first stage"){
steps{
sh "cd /Users/<user>/Terraform/proj1"
sh "pwd"
sh "terraform"
}
}
}
}

Unable to push Docker image to Artifactory

I am running a Jenkins Pipeline and trying to upload a docker image but it is failing using the Artifactory command.
This is a snippet of my Jenkinsfile stage:
stage("Build docker image") {
steps {
container('docker') {
sh 'docker -v'
script {
def rtServer = Artifactory.server "artifactory"
def rtDocker = Artifactory.docker server: rtServer
docker.build("app", "--build-arg JAR_FILE=app.jar -f Dockerfile .")
def buildInfo = rtDocker.push '<companyname>.jfrog.io/app','docker-snapshot-local'
}
}
}
}
This fails after the docker.build with the following message:
[Pipeline] newBuildInfo
[Pipeline] dockerPushStep
expected to call org.jfrog.hudson.pipeline.common.types.Docker.push but wound up catching dockerPushStep; see: https://jenkins.io/redirect/pipeline-cps-method-mismatches/
[Pipeline] }
Jenkins EOF log:
Also: hudson.remoting.Channel$CallSiteStackTrace: Remote call to JNLP4-connect connection from ip-XX-XX-XX-XX.ec2.internal/XX.XX.XX.XX:42790
at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1743)
at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
at hudson.remoting.Channel.call(Channel.java:957)
at org.jfrog.hudson.pipeline.common.docker.utils.DockerAgentUtils.getImageIdFromAgent(DockerAgentUtils.java:291)
at org.jfrog.hudson.pipeline.common.executors.DockerExecutor.execute(DockerExecutor.java:59)
at org.jfrog.hudson.pipeline.scripted.steps.DockerPushStep$Execution.run(DockerPushStep.java:104)
at org.jfrog.hudson.pipeline.scripted.steps.DockerPushStep$Execution.run(DockerPushStep.java:71)
at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1$1.call(AbstractSynchronousNonBlockingStepExecution.java:47)
at hudson.security.ACL.impersonate(ACL.java:290)
at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1.run(AbstractSynchronousNonBlockingStepExecution.java:44)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at ...io.netty.channel.epoll.AbstractEpollChannel.doConnect(AbstractEpollChannel.java:713)
at io.netty.channel.epoll.EpollDomainSocketChannel.doConnect(EpollDomainSocketChannel.java:87)
at io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.connect(AbstractEpollChannel.java:555)
at io.netty.channel.DefaultChannelPipeline$HeadContext.connect(DefaultChannelPipeline.java:1366)
at io.netty.channel.AbstractChannelHandlerContext.invokeConnect(AbstractChannelHandlerContext.java:545)
at io.netty.channel.AbstractChannelHandlerContext.connect(AbstractChannelHandlerContext.java:530)
at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.connect(CombinedChannelDuplexHandler.java:497)
at io.netty.channel.ChannelOutboundHandlerAdapter.connect(ChannelOutboundHandlerAdapter.java:47)
at io.netty.channel.CombinedChannelDuplexHandler.connect(CombinedChannelDuplexHandler.java:298)
at io.netty.channel.AbstractChannelHandlerContext.invokeConnect(AbstractChannelHandlerContext.java:545)
at io.netty.channel.AbstractChannelHandlerContext.connect(AbstractChannelHandlerContext.java:530)
at io.netty.channel.AbstractChannelHandlerContext.connect(AbstractChannelHandlerContext.java:512)
at io.netty.channel.DefaultChannelPipeline.connect(DefaultChannelPipeline.java:1024)
at io.netty.channel.AbstractChannel.connect(AbstractChannel.java:259)
at io.netty.bootstrap.Bootstrap$3.run(Bootstrap.java:252)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404)
at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:335)
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:897)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)
The expected to call... is just a warning message and you can ignore it. This may disappear in the next version of workflow-cps-plugin. More information about it can be found here.
About the error - Try first to follow the instructions here HAP-1241.

Running SonarQube Scanner on a Jenkins remote slave

I have a Docker container running Jenkins (2.150.1) and another Docker container running SonarQube (7.4). Jenkins is using the SonarQube Scanner for Jenkins plugin and the scanning is done on the Jenkins container. The Jenkinsfile for the project looks like this:
pipeline {
agent any
stages {
stage('Build') {
steps {
echo 'building...'
}
}
stage('Test') {
steps {
echo 'testing...'
withSonarQubeEnv ('SonarQube') {
sh '/var/jenkins_home/sonar-scanner/sonar-scanner-3.2.0.1227-linux/bin/sonar-scanner'
}
echo 'really finished testing2'
}
}
stage("Quality Gate") {
steps {
timeout(time: 1, unit: 'MINUTES') {
waitForQualityGate abortPipeline: true
}
}
}
stage('Deployment') {
steps {
echo 'deploying...'
}
}
}
}
To get the scanning to work as part of a Jenkins pipeline job, I manually installed sonar-scanner on the Jenkins container by downloading the zip file and unzipping it to: /var/jenkins_home/sonar-scanner/sonar-scanner-3.2.0.1227-linux
This is working well, but I want to improve it by:
taking out the harcoded path to sonar-scanner from my Jenkinsfile
specify a non local location of sonar-scanner because I now need to run the scan on another VM/container instead of on the Jenkins container
I tried using Manage Jenkins > Global Tool Configuration > SonarQube Scanner and updated my Jenkinsfile to use SONAR_RUNNER_HOME instead of the hard coded path, but that didn't work and I got an error that sonar-scanner can't be found.
In Manage Jenkins > Global Tool Configuration > SonarQube Scanner check install automatically.
Then go to Manage Jenkins > Configure System and add the following
The Name should be the same as the parameter in the line in your Jenkinsfile: withSonarQubeEnv('SonarQube')

Resources