I am trying to use environment variables to configure my Jenkins agent as follows:
pipeline {
environment {
TEST = "test"
}
agent {
kubernetes {
label 'kubernetes'
defaultContainer 'jnlp'
yaml """
apiVersion: v1
kind: Pod
metadata:
labels:
name: "${env.TEST}"
...
but ${env.TEST} is coming out as null. Using ${env.BUILD_NUMBER} works as expected so it seems the agent doesn't have access to environment variables defined in the pipeline.
Is there any way to get this to work?
You have it basically right. env.VALUE are used for specific user environment variables (e.g. If I run jenkins in an agent environment with KUBECONFIG set, as per an AMI or otherwise, that would be considered env.KUBECONFIG). It is confusing, but typically in a library you define global environment variables as follows:
env.MY_VALUE = "some value"
When referencing the env.VALUE, it is the actual user environment variables you are checking. For the values you set in the environment closure, you can just call them by MY_VALUE:
pipeline {
environment {
TEST = "test"
}
agent {
kubernetes {
label 'kubernetes'
defaultContainer 'jnlp'
yaml """
apiVersion: v1
kind: Pod
metadata:
labels:
name: "${TEST}"
...
Related
If I have a java configuration bean, saying:
package com.mycompany.app.configuration;
// whatever imports
public class MyConfiguration {
private String someConfigurationValue = "defaultValue";
// getters and setters etc
}
If I set that using jetty for local testing I can do so using a config.xml file in the following form:
<myConfiguration class="com.mycompany.app.configuration.MyConfiguration" context="SomeContextAttribute">
<someConfigurationValue>http://localhost:8080</someConfigurationValue>
</myConfiguration>
However in the deployed environment in which I need to test, I will need to use docker to set these configuration values, we use jboss.
Is there a way to directly set these JNDI values? I've been looking for examples for quite a while but cannot find any. This would be in the context of a yaml file which is used to configure a k8 cluster. Apologies for the psuedocode, I would post the real code but it's all proprietary so I can't.
What I have so far for the overrides.yaml snippet is of the form:
env:
'MyConfig.SomeContextAttribute':
class_name: 'com.mycompany.app.configuration.MyConfiguration'
someConfigurationValue: 'http://localhost:8080'
However this is a complete guess.
You can achieve it by using ConfigMap.
A ConfigMap is an API object used to store non-confidential data in key-value pairs. Pods can consume ConfigMaps as environment variables, command-line arguments, or as configuration files in a volume.
First what you need to create ConfigMap from your file using command as below:
kubectl create configmap <map-name> <data-source>
Where <map-name> is the name you want to assign to the ConfigMap and <data-source> is the directory, file, or literal value to draw the data from. You can read more about it here.
Here is an example:
Download the sample file:
wget https://kubernetes.io/examples/configmap/game.properties
You can check what is inside this file using cat command:
cat game.properties
You will see that there are some variables in this file:
enemies=aliens
lives=3
enemies.cheat=true
enemies.cheat.level=noGoodRotten
secret.code.passphrase=UUDDLRLRBABAS
secret.code.allowed=true
secret.code.lives=30r
Create the ConfigMap from this file:
kubectl create configmap game-config --from-file=game.properties
You should see output that ConfigMap has been created:
configmap/game-config created
You can display details of the ConfigMap using command below:
kubectl describe configmaps game-config
You will see output as below:
Name: game-config
Namespace: default
Labels: <none>
Annotations: <none>
Data
====
game.properties:
----
enemies=aliens
lives=3
enemies.cheat=true
enemies.cheat.level=noGoodRotten
secret.code.passphrase=UUDDLRLRBABAS
secret.code.allowed=true
secret.code.lives=30
You can also see how yaml of this ConfigMap will look using:
kubectl get configmaps game-config -o yaml
The output will be similar:
apiVersion: v1
data:
game.properties: |-
enemies=aliens
lives=3
enemies.cheat=true
enemies.cheat.level=noGoodRotten
secret.code.passphrase=UUDDLRLRBABAS
secret.code.allowed=true
secret.code.lives=30
kind: ConfigMap
metadata:
creationTimestamp: "2022-01-28T12:33:33Z"
name: game-config
namespace: default
resourceVersion: "2692045"
uid: 5eed4d9d-0d38-42af-bde2-5c7079a48518
Next goal is connecting ConfigMap to Pod. It could be added in yaml file of Podconfiguration.
As you can see under containersthere is envFrom section. As name is a name of ConfigMapwhich I created in previous step. You can read about envFrom here
apiVersion: v1
kind: Pod
metadata:
name: test-pod
spec:
containers:
- name: test-container
image: nginx
envFrom:
- configMapRef:
name: game-config
Create a Pod from yaml file using:
kubectl apply -f <name-of-your-file>.yaml
Final step is checking environment variables in this Pod using below command:
kubectl exec -it test-pod -- env
As you can see below, there are environment variables from simple file which I downloaded in the first step:
game.properties=enemies=aliens
lives=3
enemies.cheat=true
enemies.cheat.level=noGoodRotten
secret.code.passphrase=UUDDLRLRBABAS
secret.code.allowed=true
secret.code.lives=30
The way to do this is as follows:
If you are attempting to set a value that looks like this in terms of fully qualified name:
com.mycompany.app.configuration.MyConfiguration#someConfigurationValue
Then that will look like the following in a yaml file:
com_mycompany_app_configuration_MyConfiguration_someConfigurationValue: 'blahValue'
It really is that simple. It does need to be set as an environment variable in the yaml, but I'm not sure whether it needs to be under env: or if that's specific to us.
I don't think there's a way of setting something in YAML that in XML would be an attribute, however. I've tried figuring that part out, but I haven't been able to.
I am using Jenkins with the kubernetes plugin to run my jobs and I need to run a pipeline that:
builds a docker image
submit it to the registry
Uses that same image in the following steps to perform the tests.
Container(image:A): build image B
Container(image:B) : test image B
So I would like to use variables and substitute them inside the kubernetes podtemplate as here:
pipeline {
agent none
stages {
stage("Build image"){
// some script that builds the image
steps{
script{
def image_name = "busybox"
}
}
}
stage('Run tests') {
environment {
image = "$image_name"
}
agent {
kubernetes {
yaml """\
apiVersion: v1
kind: Pod
metadata:
labels:
some-label: some-label-value
spec:
containers:
- name: busybox
image: "${env.image}"
command:
- cat
tty: true
""".stripIndent()
}
}
steps {
container('busybox') {
sh 'echo "I am alive!!"'
}
}
}
}
}
but the variable is empty as I get:
[Normal][ci/test-10-g91lr-xtc20-s1ng1][Pulling] Pulling image "null"
[Warning][ci/test-10-g91lr-xtc20-s1ng1][Failed] Error: ErrImagePull
[Warning][ci/test-10-g91lr-xtc20-s1ng1][Failed] Failed to pull image "null": rpc error: code = Unknown desc = Error response from daemon: pull access denied for null, repository does not exist or may require 'docker login': denied: requested access to the resource is denied
Do you know how I can achieve a similar behaviour ?
Thank you zett42 for your answer, I was able to achieve my objective with your suggestions.
Basically the solution was to set in the build stage a global environment variable. I post here the full solution to help others in my same problem:
pipeline {
agent none
stages {
stage("Build image"){
// some script that builds the image
steps{
script{
env.image_name = "busybox"
}
}
}
stage('Run tests') {
agent {
kubernetes {
yaml """\
apiVersion: v1
kind: Pod
metadata:
labels:
some-label: some-label-value
spec:
containers:
- name: busybox
image: "${env.image_name}"
command:
- cat
tty: true
""".stripIndent()
}
}
steps {
container('busybox') {
sh 'echo "I am alive!!"'
}
}
}
}
}
To better understand it it was useful to read this article:
https://e.printstacktrace.blog/jenkins-pipeline-environment-variables-the-definitive-guide/
I'm running perfomance tests on Jenkins. Test that may include multiple instances of the same container to generate necessary load. I can't hardcode number of instances as it varies based on params for tests.
I've tried to use the following code:
pipeline {
agent any
stages {
stage('Running Jmeter') {
agent {
kubernetes {
label "jmeter_tests_executor"
yaml '''
apiVersion: batch/v1
kind: Job
metadata:
name: jmeter
namespace: jenkins
spec:
parallelism: 2
backoffLimit: 1
ttlSecondsAfterFinished: 100
...
But it doesn't work. It's hanging on pod scheduling(jobs works ok if you apply this manifest directly on kubernetes cluster without Jenkins).
If someone had experience with it, please share your workarounds or ideas how to implement this idea.
Maybe try something like this
stage("RUN LOAD TEST") {
steps {
script {
//params.each creates an array of stages
paramsToTest.each {param ->
load["load test"] = {
stage("Executing run ${param}") {
agent {
kubernetes {
label "jmeter_tests_executor"
yaml '''
apiVersion: batch/v1
kind: Job
metadata:
name: jmeter
namespace: jenkins
spec:
parallelism: 2
backoffLimit: 1
ttlSecondsAfterFinished: 100
...
'''
}
}
steps {
<EXECUTE LOAD TEST COMMAND>
}
}
}
parallel(load) //actually executes the parallel stages
}
}
}
}
What this does is use an array of something and then generates stages based on that array. The agent params in the stage should tell Jenkins to create a new pod with each execution in parallel.
I have a pipeline that needs a modified yaml file for different environments. For that I read the template, overwrite the parameter and save it again before the pipeline { ... } part starts.
node {
stage('Adjust serviceAccountName to env') {
checkout scm
def valuesYaml = readYaml (file: 'build_nodes.yaml')
valuesYaml.spec.serviceAccountName = 'user-test'
sh 'rm -f build_nodes_new.yaml'
writeYaml file: 'build_nodes_new.yaml', data: valuesYaml
}
}
Once I want to load the file the problem is that it can't be found:
pipeline {
environment {
ENV_VAR=....
}
agent {
kubernetes {
label 'some_label'
yamlFile 'build_nodes_new.yaml'
}
}
stages {
stage('Assume Role') { ... }
Throws an error:
java.io.FileNotFoundException: URL:
/rest/api/1.0/projects/PROJECT/repos/backend/browse/build_nodes_new.yaml?at=feature%2Fmy-branch-name&start=0&limit=500
Do I have to save the yaml file somewhere else? If I ls -la it is displayed.
This is because you wrote the yaml file on a regular node, and then try to read it from a container in k8s. It's like they're on different machines. In fact, they very likely are. You could pass the contents as a string to the k8s node, or you could write it to a filesystem that the k8s pod can mount
I had similar issue and below worked for me. thanks #sam_ste
def get_yaml() {
node {
sh 'env'
echo GERRIT_PATCHSET_REVISION
echo "${GERRIT_PATCHSET_REVISION}"
return """
apiVersion: v1
kind: Pod
metadata:
labels:
some-label: some-label-value
spec:
containers:
- name: simplekube
image: dhub.net/jenkins/simplekube:${GERRIT_PATCHSET_REVISION}
command:
- cat
tty: true
securityContext:
runAsUser: 0
"""
}
}
I am trying to run a test docker image in kubernetes which will test my application. The application container and test container have the same version which is incremented if any of tests or application changes. How can I define pod yaml dynamically for kubernetes plugin so that I can get the version in the first stage(which is outside the kubernetes cluster) and then update pod yaml with the right version of the container?
APP_VERSION = ""
pod_yaml = """
apiVersion: v1
kind: Pod
metadata:
labels:
some-label: ci--my-app
spec:
containers:
- name: test-runner
image: my.docker.registry/app-tester:${-> APP_VERSION}
imagePullPolicy: Always
command:
- cat
tty: true
"""
pipeline {
agent none
stages {
stage('Build and Upload') {
agent { node { label 'builder' } }
steps {
script {
APP_VERSION = sh(
script: "cat VERSION",
returnStdout: true
).trim()
}
}
}
stage('Deploy and Test application') {
agent {
kubernetes {
label 'ci--data-visualizer-kb'
defaultContainer 'jnlp'
yaml pod_yml
}
}
steps {
container('test-runner') {
sh "echo ${APP_VERSION}"
sh "ls -R /workspace"
}
}
}
}
}
The kubernetes block in pipeline do not accept lazy evaluation of string pod_yaml which contains ${-> APP_VERSION}. Is there any workaround for this or I am doing it totally wrong?
PS: I cannot use the scripted pipeline for other reasons. So, I have to stick to the declarative pipeline.
It might be a bit odd, but if you're out of other options, you can use jinja2 template engine and python to dynamically generate the file you want.
Check it out - it's quite robust.