I try to pass credentials from jenkins pipline to ansible playbook into docker as enviromental variable.
jenkins pipline:
steps {
withCredentials([usernamePassword(credentialsId: 'creds', passwordVariable: 'pass', usernameVariable: 'usr')]) {
ansiblePlaybook(
playbook: 'AnsiblePlaybook.yml',
inventory: 'AnsibleInventory.ini',
installation: 'ansible-latest',
disableHostKeyChecking: true,
become: 'yes',
extras: '-vvv',
extraVars: [
usr: ${usr},
pass: ${pass}
]
)
}
}
AnsiblePlaybook.yml:
docker_container:
name: mysql
image: mysql:5.6
state: started
env:
MYSQL_USER: "{{ usr }}"
MYSQL_PASSWORD: "{{ pass }}"
MYSQL_RANDOM_ROOT_PASSWORD: "yes"
I got error:
java.lang.NoSuchMethodError: No such DSL method '$' found among steps
You can declare parameters into variables and use it for building your ansible:
Jenkinsfile parameters: https://jenkins.io/doc/book/pipeline/syntax/#parameters-example
Ansible with args: https://docs.ansible.com/ansible/latest/user_guide/playbooks_environment.html
I had same issue.
You had used wrong syntax.
You forgot "
Here is right example:
steps {
withCredentials([usernamePassword(credentialsId: 'creds', passwordVariable: 'pass', usernameVariable: 'usr')]) {
ansiblePlaybook(
playbook: 'AnsiblePlaybook.yml',
inventory: 'AnsibleInventory.ini',
installation: 'ansible-latest',
disableHostKeyChecking: true,
become: 'yes',
extras: '-vvv',
extraVars: [
usr: "${usr}", // <================ Here you need extra "
pass: "${pass}" // <================ Here you need extra "
]
)
}
}
Related
I am trying to pass an aws creds I have stored within jenkins into an ansible playbook and it doesnt seems to be taking it. I've done research and seems like everybody is storing their creds in the jenkins file. Is it possible to pass the variable into an ansible-playbook? Below is my current situation
Jenkins Creds
Jenkinsfile
pipeline {
agent any
stages {
stage('GIT Code Checkout'){
steps{
git branch: 'ansible', credentialsId: 'test-pipeline', url: 'https://github.com/newbtech'
}
environment{
AWS_ACCESS_KEY_ID = credentials('aws-key')
AWS_SECRET_ACCESS_KEY = credentials('aws_secret_access_key')
}
stage('Run Tools Playbook'){
steps{
ansiblePlaybook credentialsId: 'root-key',
disableHostKeyChecking: true, installation: 'ansible',
extras: "-e HOST=${SERVER}",
inventory: 'ansible/host.inv',
playbook: 'ansible/cstest.yml'
}
}
}
}
Ansible-Playbook
---
- hosts: "{{ HOST }}"
tasks:
- name: "S3 Pull - Ubunutu"
aws_s3:
aws_access_key: "aws-key"
aws_secret_key: "aws_secret_access_key"
bucket: "images"
object: "ubuntu.deb"
dest: "/tmp/ubuntu.deb"
mode: get
when: ansible_facts['os_family'] == "Debian"
vars:
ansible_python_interpreter: /usr/bin/python3
Using the plugin Credentials Binding
I've tried to use snippet generator for the Pipeline Syntax for this plugin, but it wasn't helpful at all for me.
Define a withCredentials block in the pipeline under the step you want the credentials to be available:
withCredentials(
[[
$class: 'AmazonWebServicesCredentialsBinding',
accessKeyVariable: 'AWS_ACCESS_KEY_ID',
credentialsId: 'aws', # ID of AWS credentials in Jenkins
secretKeyVariable: 'AWS_SECRET_ACCESS_KEY'
]])
In your pipeline:
pipeline {
agent any
stages {
stage ('Git checkout')
{
steps
{
git branch: 'ansible', credentialsId: 'test-pipeline', url: 'https://github.com/newbtech'
}
}
stage('Run Tools Playbook')
{
steps
{
withCredentials(
[[
$class: 'AmazonWebServicesCredentialsBinding',
accessKeyVariable: 'AWS_ACCESS_KEY_ID',
credentialsId: 'aws-key',
secretKeyVariable: 'AWS_SECRET_ACCESS_KEY'
]])
{
ansiblePlaybook credentialsId: 'root-key',
disableHostKeyChecking: true, installation: 'ansible',
extras: "-e HOST=${SERVER}",
inventory: 'ansible/host.inv',
playbook: 'ansible/cstest.yml'
}
}
}
}
}
Then you can define credentials in your playbook as follows:
---
- hosts: "{{ HOST }}"
tasks:
- name: "S3 Pull - Ubunutu"
aws_s3:
aws_access_key: "AWS_ACCESS_KEY_ID"
aws_secret_key: "AWS_SECRET_ACCESS_KEY"
bucket: "images"
object: "ubuntu.deb"
dest: "/tmp/ubuntu.deb"
mode: get
when: ansible_facts['os_family'] == "Debian"
vars:
ansible_python_interpreter: /usr/bin/python3
I have Kubernetes Cluster where deployed and scaled Jenkins,
below podTemplate yaml file which I run in Jenkins pipeline:
podTemplate(yaml: """
apiVersion: v1
kind: Pod
spec:
containers:
- name: docker
image: docker/compose
command: ['cat']
tty: true
volumeMounts:
- name: dockersock
mountPath: /var/run/docker.sock
volumes:
- name: dockersock
hostPath:
path: /var/run/docker.sock
"""
) {
def image = "image/name"
node(POD_LABEL) {
stage('Build and run Docker image') {
git branch: 'test',
credentialsId: 'github-credentials',
url: 'https://github.com/project/project.git'
container('docker') {
sh "docker-compose up -d --build"
}
}
}
}
I have got an error:
[Pipeline] sh
+ docker-compose up -d --build
The ENV variable is not set. Defaulting to a blank string.
[Pipeline] }
Which is best practice to set env vars during the this kind of deployments?
Update:
yes, it's working outside of Jenkins,
I have listed env var files in docker compose yaml file:
...
context: ./validator
ports:
- '${VALIDATOR_PROD_ONE_PORT}:${VALIDATOR_PROD_ONE_PORT}'
volumes:
- ./validator/logs_1:/usr/src/app/logs
***env_file: .env.validator.test***
...
Of course I cat set env var in Jenkins pipeline before executing docker-compose build like this, for example:
container('docker') {
***sh ' echo "someEnvVar=value" > .env.validator.test'***
sh "docker-compose up -d --build"
}
This way also working, but not beautiful (:
You should be able to set environment variables on your stage as following:
stage('Build and run Docker image') {
environment {
SOME_ENV_VAR = "SOME_VAL"
}
git branch: 'test',
credentialsId: 'github-credentials',
url: 'https://github.com/project/project.git'
container('docker') {
sh "docker-compose up -d --build"
}
}
This would essentially set shell Environment variables, which should precede over those in .env file.
In General
I'm trying to use label when using kubernetes-plugin for Jenkins, but I get a little bit confused.
In my pipeline bellow I'm trying to build test job in parallel steps with different labels (agents).
I already have configured the plugin with pod template and container in my Jenkins config, where I use same settings as it's in the pipeline podTemplate defined.
Issue
The problem is that when I use agent label in stage 2 there is jnpl image running instead the image that I point in the config someimage:latest.
In stage 1 where I define the pod in pipeline there is no problem and the required images are running fine.
Question
What I'm doing wrong?
Here is my jenkinsfile and config of the kubernetes-plugin in Jenkins:
def podTemplate = """
apiVersion: v1
kind: Pod
spec:
containers:
- name: k8s
image: someimage:latest
command:
- sleep
args:
- infinity
volumeMounts:
- name: workspace-volume
mountPath: /home/jenkins/agent
workingDir: "/home/jenkins/agent"
volumes:
- name: "workspace-volume"
persistentVolumeClaim:
claimName: "jenkins-worker-pvc"
readOnly: false
"""
pipeline {
agent none
stages {
stage("Parallel") {
parallel {
stage("1.k8s") {
agent {
kubernetes {
yaml podTemplate
defaultContainer 'k8s'
}
}
steps {
sh """
mvn -version
"""
}
}
stage("2. k8s") {
agent { label 'k8s' }
steps {
sh """
mvn -version
"""
}
}
stage("win") {
agent { label 'windows' }
steps {
bat "dir"
}
}
}
}
}
}
You did not specified an image for stage with label k8s and windows.
You can read in the docs that:
The plugin creates a Kubernetes Pod for each agent started, defined by the Docker image to run, and stops it after each build.
Agents are launched using JNLP, so it is expected that the image connects automatically to the Jenkins master.
You are using podTemplate and I would advice setting up container , this might look like the following:
podTemplate(containers: [
containerTemplate(name: 'maven', image: 'maven:3.3.9-jdk-8-alpine', ttyEnabled: true, command: 'cat'),
containerTemplate(name: 'golang', image: 'golang:1.8.0', ttyEnabled: true, command: 'cat')
]) {
node(POD_LABEL) {
stage('Get a Maven project') {
git 'https://github.com/jenkinsci/kubernetes-plugin.git'
container('maven') {
stage('Build a Maven project') {
sh 'mvn -B clean install'
}
}
}
stage('Get a Golang project') {
git url: 'https://github.com/hashicorp/terraform.git'
container('golang') {
stage('Build a Go project') {
sh """
mkdir -p /go/src/github.com/hashicorp
ln -s `pwd` /go/src/github.com/hashicorp/terraform
cd /go/src/github.com/hashicorp/terraform && make core-dev
"""
}
}
}
}
}
You can read more about Container Configuration and Container Group Support
I am trying to create a pipeline job for Angular code to deploy the application into k8 cluster. Below there is a code for pipeline container podTemplate, during the build I get the next error.
def label = "worker-${UUID.randomUUID().toString()}"
podTemplate(
cloud: 'kubernetes',
namespace: 'test',
imagePullSecrets: ['regcred'],
label: label,
containers: [
containerTemplate(name: 'nodejs', image: 'nodejscn/node:latest', ttyEnabled: true, command: 'cat'),
containerTemplate(name: 'docker', image: 'nodejscn/node:latest', ttyEnabled: true, command: 'cat'),
containerTemplate(name: 'kubectl', image: 'k8spoc1/kubctl:latest', ttyEnabled: true, command: 'cat')
],
volumes: [
hostPathVolume(hostPath: '/var/run/docker.sock', mountPath: '/var/run/docker.sock'),
hostPathVolume(hostPath: '/root/.m2/repository', mountPath: '/root/.m2/repository')
]
) {
node(label) {
def scmInfo = checkout scm
def image_tag
def image_name
sh 'pwd'
def gitCommit = scmInfo.GIT_COMMIT
def gitBranch = scmInfo.GIT_BRANCH
def commitId
commitId= scmInfo.GIT_COMMIT[0..7]
image_tag = "${scmInfo.GIT_BRANCH}-${scmInfo.GIT_COMMIT[0..7]}"
stage('NPM Install') {
container ('nodejs') {
withEnv(["NPM_CONFIG_LOGLEVEL=warn"]) {
sh 'npm install'
}
}
}
}
}
Error from Jenkins:
[Pipeline] Start of Pipeline
[Pipeline] podTemplate
[Pipeline] // podTemplate
[Pipeline] End of Pipeline
ERROR: Labels must follow required specs - https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#syntax-and-character-set: Ubuntu-82f3782f-b5aa-4029-9c51-57610153747c
Finished: FAILURE
Do I need to mention a spec value of my Jenkins file?
The error message you get:
ERROR: Labels must follow required specs - https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#syntax-and-character-set: Ubuntu-82f3782f-b5aa-4029-9c51-57610153747c
points out quite precisely what can be wrong with your Pod template. As you can see in link to kubernetes documentation given in the ERROR message, you need to follow certain rules when defining a Pod. labels element is a dictionary/map field that requires you to provide at least one key-value pair so you cannot just write label: label in your specification.
You can try to define your PodTemplate in yaml format (which is mostly used in kubernetes) like in this example:
podTemplate(yaml: """
apiVersion: v1
kind: Pod
metadata:
labels:
some-label: some-label-value
spec:
containers:
- name: busybox
image: busybox
command:
- cat
tty: true
"""
) {
node(POD_LABEL) {
container('busybox') {
sh "hostname"
}
}
}
As you can read here:
label The label of the pod. Can be set to a unique value to avoid
conflicts across builds, or omitted and POD_LABEL will be defined
inside the step.
label field can be omitted at all so first you can try without it and you shouldn't get any error message.
ERROR: Labels must follow required specs - https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#syntax-and-character-set: Ubuntu-82f3782f-b5aa-4029-9c51-57610153747c
Your label Ubuntu-82f3782f-b5aa-4029-9c51-57610153747c has a space before Ubuntu which is not valid
But that error message doesn't seem to match the pod definition you posted as there is no mention of Ubuntu anywhere. Maybe inherited
I am using the Jenkins kubernetes plugin to run pipeline builds:
pipeline {
agent {
kubernetes {
label 'kind'
defaultContainer 'jnlp'
yaml """
apiVersion: v1
kind: Pod
metadata:
labels:
name: dind
...
I want to template a particular field of the yaml with an integer between 0 and 5 that is rotated in a round robin fashion (i.e. first build is templated with 0, second build templated with 1 etc. and goes back to 0 again after 4).
How can I achieve this?
You can use podTemplates next code is from https://github.com/jenkinsci/kubernetes-plugin, you can use variables to prepare any kind of pods you need.
If this is not what you need, can you provide an example of what you are trying to do?
def label = "mypod-${UUID.randomUUID().toString()}"
podTemplate(label: label, containers: [
containerTemplate(name: 'maven', image: 'maven:3.3.9-jdk-8-alpine', ttyEnabled: true, command: 'cat'),
containerTemplate(name: 'golang', image: 'golang:1.8.0', ttyEnabled: true, command: 'cat')
]) {
node(label) {
stage('Get a Maven project') {
git 'https://github.com/jenkinsci/kubernetes-plugin.git'
container('maven') {
stage('Build a Maven project') {
sh 'mvn -B clean install'
}
}
}
stage('Get a Golang project') {
git url: 'https://github.com/hashicorp/terraform.git'
container('golang') {
stage('Build a Go project') {
sh """
mkdir -p /go/src/github.com/hashicorp
ln -s `pwd` /go/src/github.com/hashicorp/terraform
cd /go/src/github.com/hashicorp/terraform && make core-dev
"""
}
}
}
}
}