I’m running Jenkins v2.263.2 with Terraform plugin on Debian 10.
I have created a freestyle project to trigger my terraform deployment, which is first pulling from my git repo and the trigger terraform, but at the terraform step I’m getting this error:
FATAL: java.io.FileNotFoundException: Configuration path not found [/var/lib/jenkins/workspace/Terraform/terraform/test.tf].
at org.jenkinsci.plugins.terraform.TerraformBuildWrapper.setupWorkspace(TerraformBuildWrapper.java:367)
at org.jenkinsci.plugins.terraform.TerraformBuildWrapper.executeGet(TerraformBuildWrapper.java:208)
at org.jenkinsci.plugins.terraform.TerraformBuildWrapper.setUp(TerraformBuildWrapper.java:256)
at hudson.model.Build$BuildExecution.doRun(Build.java:157)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:514)
at hudson.model.Run.execute(Run.java:1907)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Finished: FAILURE
Also if I look at the OS level I do have access to this file with jenkins user, so I’m not clear what is wrong with my setup.
jenkins#jenkins:~/workspace/Terraform/terraform$ cat test.tf
data "xenorchestra_template" "template" {
name_label = "Galaxy_Template"
}
data "xenorchestra_network" "net" {
name_label = "ETH"
}
resource "xenorchestra_vm" "bar" {
memory_max = 1073733632
cpus = 1
name_label = "MyName"
name_description = "Mydescription"
template = data.xenorchestra_template.template.id
network {
network_id = data.xenorchestra_network.net.id
}
disk {
sr_id = "c7e5cf80-2b3e-c0ed-6f1e-09f1417b7d24"
name_label = "WDisk"
size = 62212254720
}
}
First, check your setup, if your node path to Terraform binaries is correct
node {
env.PATH += ":/opt/terraform_0.7.xx/"
stage ('Terraform Plan') {
sh 'terraform plan -no-color -out=create.tfplan'
}
Second, check if the terraform binary is installed on the jenkins slave that is executing the pipeline, the binary must be installed to have the plugin work
Related
I am trying to integrate Jenkins with Terraform and installed the terraform plugin. https://plugins.jenkins.io/terraform/
I setup my terraform plugin in the global configuration like this :-
In my build environment, i chose terraform ( Terraform test from the screenshot ) -> Configuration text and wrote a simple tf file for testing.
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 4.0"
}
}
}
provider "aws" {
region = "us-east-1"
access_key="XXXXX"
secret_access_key="XXXXX"
}
resource "aws_instance" "web" {
ami = "ami-XXX"
instance_type = "t2.micro"
tags = {
Name = "HelloWorld"
}
}
But everytime i do build now, i keep running into the following error :-
Started by user admin
Running as SYSTEM
Building in workspace C:\ProgramData\Jenkins\.jenkins\workspace\terraform
[terraform-plugin] $ "C:\Program Files\Terraform\terraform.exe" get
[terraform-plugin] $ "C:\Program Files\Terraform\terraform.exe" apply -input=false -state=C:\ProgramData\Jenkins\.jenkins\workspace\terraform\terraform-plugin\terraform-plugin.tfstate
[31mâ•·[0m[0m
[31m│[0m [0m[1m[31mError: [0m[0m[1mInconsistent dependency lock file[0m
[31m│[0m [0m
[31m│[0m [0m[0mThe following dependency selections recorded in the lock file are
[31m│[0m [0minconsistent with the current configuration:
[31m│[0m [0m - provider registry.terraform.io/hashicorp/aws: required by this configuration but no version is selected
[31m│[0m [0m
[31m│[0m [0mTo make the initial dependency selections that will initialize the
[31m│[0m [0mdependency lock file, run:
[31m│[0m [0m terraform init
[31m╵[0m[0m
FATAL: java.lang.Exception: Terraform Apply failed: 1
at org.jenkinsci.plugins.terraform.TerraformBuildWrapper.executeApply(TerraformBuildWrapper.java:249)
at org.jenkinsci.plugins.terraform.TerraformBuildWrapper.setUp(TerraformBuildWrapper.java:269)
at hudson.model.Build$BuildExecution.doRun(Build.java:158)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:526)
at hudson.model.Run.execute(Run.java:1900)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:44)
at hudson.model.ResourceController.execute(ResourceController.java:107)
at hudson.model.Executor.run(Executor.java:449)
Finished: FAILURE
Any help will be appreciated on this. Thank you.
I have the below pipeline setup which uses terraform plugin to execute terraform.
Here is my environment. A centos7 vagrant vm has been installed terraform, jenkins using user vagrant. Terraform plugin is also installed to Jenkins. Terraform can be run manually. When terraform being used by pipeline, it is failed. Tried a couple of other ways to set up terraform path, all are failed.
Terraform is located at /usr/local/bin/terraform.
pipeline {
agent any
stages {
stage('Set Terraform path') {
steps {
script {
def tfHome = tool name: 'terraform'
env.PATH = "${tfHome}:${env.PATH}"
// def tfhome = tool name: 'terraform', type: 'org.jenkinsci.plugins.terraform.TerraformInstallation'
// env.PATH = "${tfhome}:${env.PATH}"
// def tfHome = tool name: 'terraform', type: 'com.cloudbees.jenkins.plugins.customtools.CustomTool'
// env.PATH = "${tfHome}:${env.PATH}"
echo "tfhome is ${tfhome}, env.PATH is ${env.PATH} "
}
// sh '/usr/local/bin/terraform -version'
sh 'terraform -version'
}
}
}
}
Here is error message:
Failed Node Use a tool from a predefined Tool Installation
hudson.AbortException: No org.jenkinsci.plugins.terraform.TerraformInstallation named terraform found
at org.jenkinsci.plugins.workflow.steps.ToolStep$Execution.run(ToolStep.java:162)
at org.jenkinsci.plugins.workflow.steps.ToolStep$Execution.run(ToolStep.java:133)
at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
I figure it out. The terraform tool cannot be used by user jenkins. So I create a symlink pointing to terraform command. The error is gone!
I'm trying to automate the creation of a Jenkins Pipeline build from within a pipeline.
I have a pipeline which creates a Bitbucket repository and commits some code to it, including a Jenkinsfile.
I need to add another step to this pipeline to then create the Pipeline build for it, which would run the steps in the Jenkinsfile.
I think the Jobs DSL should be able to handle this but the documentation I've found for it has been very sparse, and I'm still not entirely sure if it's possible or how to do it.
Any help would be appreciated. The generated Pipeline build I would imagine just needs to have a link to the repository and be told to run the Jenkinsfile there?
Yes, Job DSL is what you need for your use case.
See this and this to help you get started.
EDIT
pipeline {
agent {
label 'slave'
}
stages{
stage('stage'){
steps {
// some other steps
jobDsl scriptText: '''pipelineJob(\'new-job\') {
def repo = \'https://xxxxx#bitbucket.org/xxxx/dummyrepo.git\'
triggers {
scm(\'H/5 * * * *\')
}
definition {
cpsScm {
scm {
git {
remote {
url(repo)
credentials('bitbucket-jenkins-access')
}
branches(\'master\')
scriptPath(\'Jenkinsfile\')
extensions { }
}
}
}
}
}'''
}
}
}
}
Documentation - https://jenkinsci.github.io/job-dsl-plugin/#path/pipelineJob-scm-git
By using this python library jenins-job-builder you can easily create your expected pipeline or free-style job from another pipeline or from any other remote location.
Example:
steps-1
python3 -m venv .venv
source .venv/bin/activate
pip install --user jenkins-job-builder
steps-2
Once you have done above, Create 2 file, one with name config.ini and the other one is job.yml. Please note - there are no strict rules about the file name. It can be up to you.
The config.ini file contain can looks like
[job_builder]
allow_duplicates = False
keep_descriptions = False
ignore_cache = True
recursive = False
update = all
[jenkins]
password = jenkins-password
query_plugins_info = False
url = http://jenkins-url.net
user = jenkins-username
If you are creating a pipeline job , then your job.yml file can look like
- job:
name: pipeline01
display-name: 'pipeline01'
description: 'Do not edit this job through the web!'
project-type: pipeline
dsl: |
node(){
stage('hello') {
sh 'echo "Hellow World!"'
}
}
steps-3
after all the above. Invoke below command
jenkins-jobs --conf config.ini update job.yml
Note- jenkins-jobs command can only be available if you have followed steps-1
I have a private GitHub Rust project that depends on another private GitHub Rust project and I want to build the main one with Jenkins. I have called the organization Organization and the dependency package subcrate in the below code.
My Jenkinsfile looks something like
pipeline {
agent {
docker {
image 'rust:latest'
}
}
stages {
stage('Build') {
steps {
sh "cargo build"
}
}
etc...
}
}
I have tried the following in Cargo.toml to reference the dependency, it works fine on my machine
[dependencies]
subcrate = { git = "ssh://git#ssh.github.com/Organization/subcrate.git", tag = "0.1.0" }
When Jenkins runs I get the following error
+ cargo build
Updating registry `https://github.com/rust-lang/crates.io-index`
Updating git repository `ssh://git#github.com/Organization/subcrate.git`
error: failed to load source for a dependency on `subcrate`
Caused by:
Unable to update ssh://git#github.com/Organization/subcrate.git?tag=0.1.0#0623c097
Caused by:
failed to clone into: /usr/local/cargo/git/db/subcrate-3e391025a927594e
Caused by:
failed to authenticate when downloading repository
attempted ssh-agent authentication, but none of the usernames `git` succeeded
Caused by:
error authenticating: no auth sock variable; class=Ssh (23)
script returned exit code 101
How can I get Cargo to access this GitHub repository? Do I need to inject the GitHub credentials onto the slave? If so, how can I do this? Is it possible to use the same credentials Jenkins uses to checkout the main crate in the first place?
I installed the ssh-agent plugin and updated my Jenkinsfile to look like this
pipeline {
agent {
docker {
image 'rust:latest'
}
}
stages {
stage('Build') {
steps {
sshagent(credentials: ['id-of-github-credentials']) {
sh "ssh -vvv -T git#github.com"
sh "cargo build"
}
}
}
etc...
}
}
I get the error
+ ssh -vvv -T git#github.com
No user exists for uid 113
script returned exit code 255
Okay, I figured it out, No user exists for uid error is because of a mismatch between the users in the host /etc/passwd and the container /etc/passwd. This can be fixed by mounting /etc/passwd.
agent {
docker {
image 'rust:latest'
args '-v /etc/passwd:/etc/passwd'
}
}
Then
sshagent(credentials: ['id-of-github-credentials']) {
sh "cargo build"
}
Works just fine
Using the Jenkins Artifactory Plugin and Gradle, I am able to deploy to my Artifactory instance successfully. However, I have not been able to use the credentials entered into Jenkins configuration to resolve the artifact from the same repository.
Here is the build.gradle, adopted right from Artifactory's "Generate Build Script" feature in Artifactory.
apply plugin: 'java'
sourceCompatibility = 1.5
version = '1.0'
buildscript {
repositories {
maven {
url 'http://artifactory.myorg.com:8081/artifactory/plugins-release'
credentials {
username = "${artifactory_user}"
password = "${artifactory_password}"
}
}
}
dependencies {
//Check for the latest version here: http://plugins.gradle.org/plugin/com.jfrog.artifactory
classpath "org.jfrog.buildinfo:build-info-extractor-gradle:3.0.3"
}
}
allprojects {
apply plugin: "com.jfrog.artifactory"
}
artifactory {
contextUrl = "${artifactory_contextUrl}" //The base Artifactory URL if not overridden by the publisher/resolver
publish {
repository {
repoKey = 'libs-release-local'
username = "${artifactory_user}"
password = "${artifactory_password}"
maven = true
}
}
resolve {
repository {
repoKey = 'libs-release'
username = "${artifactory_user}"
password = "${artifactory_password}"
maven = true
}
}
}
dependencies {
compile group: "com.myorg", name: "internal-library", version: '1.0'
testCompile group: 'junit', name: 'junit', version: '4.11'
}
Job configuration on Jenkins:
Jenkins Artifactory plugin configuration:
Error:
FAILURE: Build failed with an exception.
* Where:
Build file '/var/lib/jenkins/jobs/deleteme/workspace/build.gradle' line: 10
* What went wrong:
A problem occurred evaluating root project 'untitled2'.
> Could not find property 'artifactory_user' on Credentials [username: null].
Locally, resolving artifacts works (artifactory_user etc are resolved, because I configured them in my local ~/.gradle/gradle.properties). However, builds fail, because artifactory_user is not defined. Of course, I can configure a gradle.properties for Jenkins to use (and this works), however how is this supposed to work? It appears the Jenkins Artifactory Plugin Gradle integration is supposed to somehow communicate resolver credentials. Is there a way to do this without replicating the credentials in two places on Jenkins?
tl;dr username = "${project.getProperty('artifactory.publish.username')}"
Note: I haven't tested this on a Jenkins machine yet
The Jenkins artifactory plugin's gradle integration appears to do two things
Inject the artifactory configuration using a [gradle init script]
Provide build and artifactory configuration information
Both involve writing files to a temporary folder (i.e. /tmp on linux). If you have access to your build server you will probably have a lot of buildInfo\d{19}.properties and init-artifactory\d{19}gradle files in your temp folder.
Looking inside one of these buildInfo files reveals the resolve and publish credentials are stored in artifactory.resolve.username and artifactory.publish.username, respectively.
Trying to setting and trying to use ${artifactory.publish.username} directly doesn't work on my local machine; I assume it has to do with gradle trying to access the property before it is set. But the project.getProperty method works.
buildscript {
repositories {
maven {
url 'http://artifactory.myorg.com:8081/artifactory/plugins-release'
credentials {
username = "${project.getProperty('artifactory.resolve.username')}"
password = "${project.getProperty('artifactory.resolve.password')}"
}
}
}
}
artifactory {
publish {
contextUrl = "${project.getProperty('artifactory.publish.contextUrl')}"
repository {
repoKey = 'libs-release-local'
username = "${project.getProperty('artifactory.publish.username')}"
password = "${project.getProperty('artifactory.publish.password')}"
maven = true
}
}
resolve {
repository {
contextUrl = "${project.getProperty('artifactory.resolve.contextUrl')}"
repoKey = 'libs-release'
username = "${project.getProperty('artifactory.resolve.username')}"
password = "${project.getProperty('artifactory.resolve.password')}"
maven = true
}
}
}
You will have to update your local ~/.gradle/gradle.properties accordingly
artifactory.publish.contextUrl=<artifactory-url>
artifactory.publish.username=<username>
artifactory.publish.password=<password>
artifactory.resolve.contextUrl=<artifactory-url>
artifactory.resolve.username=<username>
artifactory.resolve.password=<password>
I am also banging my head against the wall with this one...I tried all the combinations myself and the only one that seems to be working is adding the gradle.properties to the jenkins server.
Finally I ended up not using the artifactory gradle plugin from jenkins alltogether.R ather add it to the build.gradle and simply call artifactoryPublish from Jenkins Server gradle plugin.