Dropwizard takes .yml configurations which of course allow lists. It also has the ${FOO:-bar} syntax which allows reading environment variables with default fallback.
Is there a way to read lists of strings from an env variable and parse automatically with Dropwizard or do I have to do it manually? What is the syntax to have a default fallback list if this exists? I could not find this in their documentation.
Thic can be achived by using a json format in the default parameter:
supported_locales: ${SUPPORTED_LOCALES:-[en,da_DK,pt_BR,es]}
This will make a config parameter called supported_locales which takes a SUPPORTED_LOCALES env variable (json array of strings format), and defaults to the array provided after the - character in this case [en,da_DK,pt_BR,es]
Related
I'm creating a custom Fluent-Bit image and I want a "generic" configuration file that can work on multiple cases, i.e.: it should work with a forward input sometimes and with tail input some other times.
I thought about using environment variables so to only have one input but it seems we cannot set variables in the key part only on the value side (see following code).
When I set the corresponding environment variables in a docker-entrypoint file with corresponding conditions
export INPUT_PATH="/myLogPath"
export INPUT_PATH_TYPE="path"
export INPUT_NAME="tail"
[INPUT]
Name ${INPUT_NAME}
${INPUT_PATH_TYPE} ${INPUT_PATH}
This is the error message I got
[error] [config] tail: unknown configuration property '${INPUT_PATH_TYPE}'. The following properties are allowed: path, exclude_path, key, read_from_head, refresh_interval, watcher_interval, rotate_wait, docker_mode, docker_mode_flush, docker_mode_parser, path_key, ignore_older, buffer_chunk_size, buffer_max_size, skip_long_lines, exit_on_eof, parser, tag_regex, db, db.sync, db.locking, multiline, multiline_flush, parser_firstline, and parser_.
I'm looking for a way to make it dynamic so either to have a single file with dynamic configuration or multiple files which can be included dynamically (#Include requires a static filepath from what I've seen).
EDIT: the only option I see is to have multiple input files (for each use case) and call it dynamically when starting fluent-bit in the docker-entrypoint file
I use a docker-entrypoint and split the input, filters to different files and then depending of the environment variables in the entrypoint I create a symbolic link to the corresponding file
I've run into a specific problem while trying to automate a tagging process in terraform. I've set an environment variable that is essentially a list of all the tags we'd be using for all resources provisioned in the apply. It looks like this...
export TF_VAR_taglist='{JiraEpic = "ETOS-56","AssignedResearcherPri" = "Isaac",AssignedResearcherSec = "Matt"}'
After setting the environment variable I added a variable called "taglist" in the variables.tf file that grabs the aforementioned environment variable. It looks like this...
variable "taglist"{}
Lastly, I have another locals.tf file where i set a common_tags variable. Like so...
locals { common_tags ="${var.taglist}" }
When i terraform apply, the build fails while trying to map the tags properly. This is the error i receive...
Error: Incorrect attribute value type
on kube_master_worker_nodes_ec2.tf line 9, in resource "aws_instance" "master":
9: tags = local.common_tags
|----------------
| local.common_tags is "{JiraEpic = \"ETOS-56\",AssignedResearcherPri = \"Isaac \",AssignedResearcherSec = \"Matt\"}"
Inappropriate value for attribute "tags": map of string required.
I then decided to define the type of the variable as map(string in the variables.tf file like this
variable "taglist"{ type = map(string) }
I had hoped that this would allow terraform to recognize this variable as a map of strings and not just a string literal, but I was wrong, and these are the errors I get when that definition is applied.
Error: Missing attribute separator
on <value for var.taglist> line 1:
(source code not available)
Expected a newline or comma to mark the beginning of the next attribute.
Error: No value for required variable
on variables.tf line 11:
11: variable "taglist"{
The root module input variable "taglist" is not set, and has no default value.
Use a -var or -var-file command line argument to provide a value for this
variable.
I'm really stuck on this, and I feel like I'm close. Can anyone provide some insight into this and how I should go about solving it?
I want to first thank Martin Atkins for giving me the idea of using colons instead of equal signs in my environment variable, because that was the ONLY issue. The variable was not properly represented as a JSON object so terraform was interpreting it as a string.
I changed
export TF_VAR_taglist='{JiraEpic = "ETOS-56","AssignedResearcherPri" = "Isaac",AssignedResearcherSec = "Matt"}'
to this
export TF_VAR_taglist='{"JiraEpic":"ETOS-56","AssignedResearcherPri":"Isaac", "AssignedResearcherSec":"Matt"}'
The build purrs like a kitten on catnip now.
Terraform uses the type constraint of a variable to decide how to interpret a string representation of its value. By default, Terraform will assume the value expects a primitive type such as a string or number, because that's the most typical case for variables set via the command line or environment variables.
Since your tag list is a list you need Terraform to interpret it as a map expression rather than as a string. You can tell Terraform to do that by telling Terraform which type of value you expect:
variable "taglist" {
type = map(string)
}
You can read more about this in the Terraform documentation section Complex-typed Values.
You then need to make sure that the value in the environment variable is a valid object expression in order to avoid a syntax error. If you're setting the environment variable from the shell command line then you need to be mindful of escaping/quoting to ensure that Terraform will see the value with all of the quotes intact, and without any extra metacharacters.
The result is often hard to read clearly, which is why the Terraform documentation recommends using a .tfvars file to set complex-typed variables, instead of the -var command line option or environment variables. However, since you are using automation you might find it easier to generate a .tfvars.json file instead, which uses standard JSON format and is therefore easier to generate using JSON libraries available in most programming languages. Here's the .tfvars.json equivalent of the value you showed in your question:
{
"taglist": {
"JiraEpic": "ETOS-56",
"AssignedResearcherPri": "Isaac",
"AssignedResearcherSec": "Matt"
}
}
Note that subjectively I'd find it pretty confusing to have a variable whose name ends in list when it actually expects a map. A more typical name for this variable would be just tags, though if it's useful to mention its type in order to distinguish it from other variables then I'd suggest tag_map instead, to make it less confusing.
I want to read a variable from CSV and use that value into another variable.
Example:
I have a variable as:
${url}: wwww.$(value_from_csv}.com
and secondary url ${url}/xyz
In my Jmeter script, ${value_from_csv} is not passed.
What I am missing?
Observed that CSV Dataset Config values are not passed (not available) to any of the Config Elements irrespective of the order of the components (Config Elements) in JMeter Test Plan (checked with User Defined Config & MongoDB Source Config), though passed to Samplers.
so, suggested the OP to define the value in jmeter.properties instead of a CSV file, so we can access user.host in MongoDB Source Config.
Steps:
Add user.host=address in jmeter.properties
Restart Jmeter
Add ${__P(user.host,)} in Server Address List field in MongoDB Source Config
Note: In case of running JMeter script from Jenkins, property will be picked by the script, from jmeter.properites file.
MongoDB Source Config is initialized before any JMeter Variables so the only way to make it dynamic is using JMeter Properties instead.
You can set a JMeter Property in 2 ways:
Define it in user.properties file like:
server.address.1=some.ip.or.hostname.1
server.address.2=some.ip.or.hostname.2
Pass the properties via -J command-line arguments like:
jmeter -Jserver.address.1=some.ip.or.hostname.1 -Jserver.address.2=some.ip.or.hostname.2 ....
See Apache JMeter Properties Customization Guide for more information on using JMeter Properties
We are using DropWizard v0.8.1 and we're wondering if we can have a YAML files with default values that will then get overridden by the specific environment file (such as dev.xml).
Spring boot works this way, where the application.yml file act as a template for default values and then application-dev.yml will override duplicate properties.
We don't want to duplicate all the repetitive properties and only want to update in one file the defaults.
You can write your own ConfigurationProvider that combines 2 InputStreams and use yaml merge directives
You can use a configuration management tool, such as Ansible, to manage your configurations files.
Set up a template .yml file, and substitute the variables per environment as needed.
I have an environment variable injected by Jenkins like:
CUSTOMERS="foo,bar"
Now i need to loop over these values.
Is there any way to access these values AS items in ansible?
Any help including other suggestions how to to solve this is welcome.
You can pass in the environment variable to ansible with --extra-vars, but that's only part of the solution, you need to get the string value into a data format that ansible understands.
One straightforward option is to write a simple Python (or your preferred language) script to convert the environment variable to a JSON list and pass the JSON file to ansible as extra vars with --extra-vars "#customers.json" (JSON file input is available in ansible 1.3), see Ansible Variable documentation.
import sys
import os
import json
DEFAULT_VAR="CUSTOMERS"
def var_to_json(var_name, list_sep = ','):
var_dict = {var_name: os.environ[var_name].split(list_sep)}
return json.dumps(var_dict)
var_name=DEFAULT_VAR
if len(sys.argv) > 1:
var_name = sys.argv[1]
print var_to_json(var_name)
The script above could be generalized further (or customized to your situation). I'll have to leave it to you to hook the pieces together in your build environment.
Alternatively, as this previous answer explains, you can define a custom filter in ansible to process the input value. You could create a filter that converts the variable value to a list, then use the filter in your playbook when referencing the variable (presumably passed in via --extra-vars).