I was wondering what is the best approach to reading in the yaml config and setting environment variables.
For example, my yaml config looks like this:
amps-ml:
models:
- name: app-sample
type: sagemaker
inference:
image: project_id.dkr.ecr.us-west-2.amazonaws.com/template-model-bert:test_1
data: s3://my_project/sagemaker/huggingface-pytorch-inference-recommender/sentiment-analysis/model/model.tar.gz
endpoint: amp-app-endpoint-test
model_name: sample-model
endpoint_config_name: amp-app-config
model_package_group_name: sample-package-group
endpoint_instance_count: 1,
endpoint_instance_type: ml.m5.large
I essentially want to set environment variables in my Jenkins pipeline for all the variables under inference.
Try
def yaml = readYAML file: "your-file.yaml"
yaml["amps-ml"]["models"][0]["inference"].each {name, value ->
env["$name"] = value
}
You can also iterate the models instead of using explicit index (0)
Related
I am trying to write to a yaml file, specifically overwrite, in a declarative pipeline in groovy.
transfers:
- name: xyz
cloud: aws
subheading:
impact: Low
reason: ---
artifacts:
- name: name1
type: type1
source:
hash: a1b2C3dd4 ---> VALUE TO OVERWRITE
HASH = sh(returnStdout:true, script: 'git rev-parse HEAD').trim()
data.transfers['artifacts'].source['hash'] = HASH
writeYaml file: filename, data: data, overwrite = true
However there is a type mismatch when doing the second line of the code , and it looks like it's because data.transfers['artifacts'].source['hash'] is of type arraylist and I am trying to set it = to HASH which is of type string. I know that to solve this I can just convert the HASH to arrayList, but I don't understand why data.transfers['artifacts'].source['hash'] = HASH doesn't overwrite the value inside the array? Is there anyway to avoid converting the HASH to array list?
In the jenkinsfile I have started using environment context in one of the stage and now there is a requirement to concatenate with one static value which is 'grafana-' with the variable declared in the Jenkins configuration and assign the output to a new variable.
CLUSTER_NAME is the key/variable and value is TEST under jenkins > configuration
Tried different ways but couldn't get value for variable CLUSTER_NAME
environment {
def INSTANCE_NAME='grafana-${env.CLUSTER_NAME}'
}
Use doudble quotes instead of single quotes.
environment {
def INSTANCE_NAME = "grafana-${env.CLUSTER_NAME}"
}
I am completely new to Groovy and Jenkins. I have some pre defined variable in Groovy Script (of Jenkins pipeline) and need to pick any one variable from them dynamically based on job/user input.
The example context of requirement is as provided below.
Here variable env is my input and based on that I should get correct userid.
env = "dev" //Input
def stg_userid = "abc"
def dev_userid = "xyz"
uid_var_name = "${env}_userid"
print "${uid_var_name}" // It is giving "dev_userid"
print 'abc' if we give 'stg' for env ;
print 'xyz' if we give 'dev' for env
Tried searching online for dynamic variable name use case in Groovy , but didn't got anything useful.
usually it's question of complex variable (Map) that holds parameters for all possible environments
and you could get section of this configuration by environment name
env = "dev"
def config = [
dev: [
user: 'aaa',
url: 'aaa-url'
],
stg: [
user: 'zzz',
url: 'zzz-url'
]
]
def uid_var_name = config[env].user // returns "aaa"
In my Jenkins pipeline, I've got a yaml file that I need to apply to multiple environments, and separate environment specific yaml files that I'd like to inject or merge into the default file and write as a new file.
I've looked at readYaml and writeYaml here: https://jenkins.io/doc/pipeline/steps/pipeline-utility-steps/ But I'm not finding a good way of merging multiple files.
A simple example of what I'd like to achieve is here:
# config.yaml
config:
num_instances: 3
instance_size: large
# dev-overrides.yaml
config:
instance_size: small
# dev-config.yaml (desired output after merging dev-overrides.yaml in config.yaml)
config
num_instances: 3
instance_size: small
The Jenkins implementation of readYaml uses SnakeYAML as processor and supports YAML 1.1. You could possibly use the merge operator to accomplish your goal. But the merge operator has been removed in YAML 1.2. Thus I would not advise using this feature even it's currently available.
I would instead merge the objects with some Groovy code like this:
Map merge(Map... maps) {
Map result = [:]
maps.each { map ->
map.each { k, v ->
result[k] = result[k] instanceof Map ? merge(result[k], v) : v
}
}
result
}
def config = readYaml text: """
config:
num_instances: 3
instance_size: large
"""
def configOverrides = readYaml text: """
config:
instance_size: small
"""
// Showcasing what the above code does:
println "merge(config, configOverrides): " + merge(config, configOverrides)
// => [config:[num_instances:3, instance_size:small]]
println "merge(configOverrides, config): " + merge(configOverrides, config)
// => [config:[instance_size:large, num_instances:3]]
// Write to file
writeYaml file: 'dev-config.yaml', data: merge(config, configOverrides)
Inspired by https://stackoverflow.com/a/27476077/1549149
I am trying to create a custom Environment variables in Jenkins by doing this
List of key-value pairs
Name: Build_Date
Value: Date()
The value part is not resolving to a date - any ideas?
Here is where I am trying to config the above
Thanks
I confirm that you can use the EnvInject plugin with a Groovy script:
Here is the Groovy script:
// Generate a global BUILD_ID_LONG variable with date and time
// =======================================
TimeZone.setDefault(TimeZone.getTimeZone('UTC'))
def now = new Date()
def map = [BUILD_ID_LONG: now.format("yyyyMMdd_HHmm")]
return map
Next, you can use the ${BUILD_ID_LONG} variable in your build steps.