New to Ansible. There has to be a simpler way to do this: I need to issue a command (e.g. ps), and go through the return elements one by one, skipping the first element which has the column headings (e.g. PID, etc. in this case). I am only interested in the process names, not the other column values.
What's the easiest way of doing this? Below is my approach (which may not be the best way). I finally whittled down the list to omit the first element. Do I read into a separate array, loop through each item, and use awk or sed to filter on the process name?
name: Get process
hosts: localhost
tasks:
- name: Get process
command: "ps"
register: ps_output
- name: Set list length
set_fact:
array_end_index: "{{ ps_output.stdout_lines | length - 1 }}"
- name: List all but last item
debug:
msg: "{{ ps_output.stdout_lines[item] }}"
loop: "{{ range(1, (array_end_index | int)) | list }}"
Your text says to skip the "first element which has the column headings", but when I run your example it skips the first AND last lines. Not sure which you want, but since you already have a list in the ps_output.stdout_lines variable, so use the Python slice notation to return entries to easily remove from either or both ends of the list. (Note that the list is a zero-indexed list.)
To drop both the first and last list entries:
- name: Get process
hosts: localhost
tasks:
- name: Get process
command: "ps"
register: ps_output
- name: List all but last item
debug:
msg: "{{ item }}"
loop: "{{ ps_output.stdout_lines[1:-1] }}"
If you just want to drop the first line, change the loop: slice to [1:]. If you want to drop just the last line, use [:-1].
See Understanding Slice Notation for more information.
Related
I'm trying to replace the hardcode in the Helm3 template with variables.
the name of the variable must match the value of the string name from the Values.yaml file. (name: test-rrrr-blabla-file) without spaces and only the last 2 blocks, i.e. blablafile.
Adequate examples in the
https://helm.sh/docs/chart_best_practices/templates/
I didn't find how to do it, tried the following expression:
{{- default .Chart.Name .Values | (split “-” .Values.nameOverride)._3 }}-{{ (split “-” .Values.nameOverride)._4 }}
but it didn't work.
Also found undocumented capabilities here:
https://github.com/Masterminds/sprig/blob/master/docs/strings.md#regexfind
Not sure exactly, maybe need to use or regexSplit
or regex_replace, but I don't understand how to properly compose the expression... maybe you have come across this in practice?
Any help would be appreciated.
Thank you!
The <<: operator in YAML is usable to import the contents of one mapping into another, similarly to the ** double-splat operator in Python or ... object destructuring operator in JavaScript. For example,
foo:
a: b
<<:
c: d
e: f
is equivalent to
foo:
a: b
c: d
e: f
This is useful when used along with node anchors to include some common default properties in many objects, as illustrated in, for example, the Learn YAML in Y minutes tutorial:
# Anchors can be used to duplicate/inherit properties
base: &base
name: Everyone has same name
foo: &foo
<<: *base
age: 10
bar: &bar
<<: *base
age: 20
However, I am confused about where this syntax comes from or why it works. CTRL + Fing the YAML specification for << reveals that it doesn't appear anywhere in the specification. Yet it's supported by, at the very least, PyYAML and Online YAML Parser.
What is this syntax, and how come it doesn't seem to appear in the specification?
It is called the Merge Key Language-Independent Type for YAML version 1.1. and specified here.
It is something that parsers can optionally support, and it is essentially an interpretation of the key-value pair with the special key <<, where the value is either a mapping (usually indicated via an alias as in the spec, and although that doesn't seem to be required, it makes little sense not to use an alias) or a list of mappings (i.e., aliases of mappings), and gets interpreted in a special way.
To add on to other answers:
IMO, the example from "Learn YAML in Y Minutes" is incomplete, because it doesn't show what happens when the keys are the same. For example:
base: &base
name: Everyone has same name
age: 5
foo: &foo
<<: *base
bar: &bar
<<: *base
age: 20
For the bottom two items, it yields:
foo:
name: Everyone has same name
age: 5
bar:
name: Everyone has same name
age: 20
bar overrides the age while foo does not. According to the spec, the entries of the object merging in have a lower priority than those on the object receiving them.
The “<<” merge key is used to indicate that all the keys of one or more specified maps should be inserted into the current map. If the value associated with the key is a single mapping node, each of its key/value pairs is inserted into the current mapping, unless the key already exists in it.
I have a file which has the following format:
LINK|Grouping_Indicator|ID_Dat|HASH_Akey|HASH_HUKey|
FALSE|75768163|XY100|c5157cba1b5f20|817f8b50bc9
FALSE|75768409|XY102|9f3de314a224f2|b686e4760f5
TRUE|75769393|XY1005|ce0a50207cc86c|f9233c0b8e7
TRUE|75769885|XY1012|ce0a50207cc86c|ef9eb8ea13f
TRUE|75723124|XY1111|df0q45677ee89v|gt8qc9fb24g
I am trying to count the numbers of TRUE where the HASH_Akey is unique.
I've managed to count the numbers of TRUE in total with the following command:
grep -c "TRUE" file.psv
However, I am unsure on how to count "TRUE" where the HASH_Akey is unique.
So the count for "TRUE" from the table above should only return 2
Thanks
I would do it with awk:
awk -F'|' '$1=="TRUE"{a[$(NF-1)]}END{print length(a)}' file
with your example, the above one-liner will print 2
You can also do it with:
awk -F'|' '$1=="TRUE"&&!a[$(NF-1)]++' file|wc -l
the line is a little bit shorter but it starts another process (wc) to do the counting.
Setup
Prometheus node exporter is registered as a service with consul agent with various tags. Example service definition provided to consul agent:
{
"service":{
"id": "server-stats",
"name": "server-stats",
"tags": [
"a=1_meow",
"b=2_woof",
"c=3_moo",
"monkey"
],
"port": 9100,
"checks": [
{
"name": "Process #1",
"script": "/path/to/healthcheck/script.sh",
"interval": "5s"
}
]
}
}
Prometheus is set to look for this server-stats service and use the configuration (host address and port) provided by Consul to scrape stats from servers. The above tags are available as a comma separated list in __meta_consul_tags that can be used for relabeling.
Prometheus relabeling configuration:
relabel_configs:
- source_labels: [__meta_consul_tags]
separator: ','
#regex: '(.+)=(.+)'
regex: '([a-z_]+)=([a-z_]+|\d+)'
target_label: ${1}
replacement: ${2}
Issue
I am trying to expose tags to Prometheus so that we can get stats and graphs based on labels. Keeping the above service configuration in mind, I would like each metric to have following labels in addition to whatever Prometheus does internally:
a=1_meow, b=2_woof, c=3_moo and ignore monkey because it is just a string. I can remove monkey from my list of tags if there is a solution that requires = to be there. The relabel configuration written above is not leading to exposing any tag at all and seems to be getting ignored. Running Prometheus with log level set to debug is also not yielding anything.
Relevant docs
https://prometheus.io/docs/operating/configuration/#%3Crelabel_config%3E
https://www.robustperception.io/extracting-full-labels-from-consul-tags/
Incorrect understanding
I think there was a mistake in my understanding of how labeling in prometheus works. My incorrect understanding was:
before applying regex, string would be first split on separator (otherwise what is its purpose?),
each substring has regex evaluated against it,
if match groups are declared and found, they will be available as indexed values available to use in target_label and replacement fields.
if regex does not match, then that substring will be ignored.
because regex is expected to be applied to each substring after the split, it will lead to multiple labels from multiple substrings.
Correct understanding
However, from brian-brazil's post linked in his answer and Prometheus's documentation, it seems the following happening:
All __meta tags are combined into one long separator separated line.
regex is applied on that line only once.
If regex matches and includes groups, they are indexed beginning from 1 and available for use in target_label and replacement.
separator seems to be getting ignored in this section even if you mention it.
Config from corrected understanding
From this idea and following from example in the question, I was able to make the following config that works
relabel_configs:
- source_labels: [__meta_consul_tags]
regex: '.*,a=([a-z0-9_]+),.+'
target_label: 'a'
replacement: ${1}
- source_labels: [__meta_consul_tags]
regex: '.*,b=([a-z0-9_]+),.+'
target_label: 'b'
replacement: ${1}
- source_labels: [__meta_consul_tags]
regex: '.*,c=([a-z0-9_]+),.+'
target_label: 'c'
replacement: ${1}
- source_labels: [__meta_consul_tags]
regex: '.*,d=([a-z0-9_]+),.+'
target_label: 'd'
replacement: ${1}
Caveats
I believe both approaches (the approach brian-brazil wrote in his blogpost, and what I am using above) have caveats - we either need to know all the labels we want beforehand, or have a set number of them. This means if a developer wants to associate different, or more labels with his/her service, s/he would need to work with ops as general flow will not be able to handle it. I think it is a minor caveat that should be addressed.
https://www.robustperception.io/extracting-full-labels-from-consul-tags/ shows how to do this, in particular the last example.
The following relabeling rule can be used for extracting a particular tag from __meta_consul_tags and placing it to destination_label:
relabel_configs:
- source_labels: [__meta_consul_tags]
regex: '.*,tag=([^,]+),.*'
target_label: destination_label
There is no need to specify the replacement option because it defaults to $1.
If multiple tags must be extracted from __meta_consul_tags into multiple labels, then just repeat the relabeling rule per each needed tag. For example, the following relabeling rules extract a tag to a label and b tag to b label:
relabel_configs:
- source_labels: [__meta_consul_tags]
regex: '.*,a=([^,]+),.*'
target_label: a
- source_labels: [__meta_consul_tags]
regex: '.*,b=([^,]+),.*'
target_label: b
More information about relabeling rules in Prometheus is available in this article.
Update (2022-12-19): VictoriaMetrics and vmagent (these are Prometheus-like monitoring systems and scrapers I work on) expose __meta_consul_tag_<tagname> and __meta_consul_tagpresent_<tagname> labels for discovered targets starting from v1.85.2. This allows copying all the tags defined at Consul service into the scrape target labels with a single relabeling rule:
- action: labelmap
regex: __meta_consul_tag_(.+)
Try playing with this relabeling rule at Prometheus relabel debugger.
I'm trying to replace the html entity for a blank space with an actual space between occurrences of {{ and }}
Example example
"this is a gap {{ for user in users }}" =>
"this is a gap {{ for user in users }}"
I've found answers similar which had led me to write something like this (which doesn't work)
.gsub(/(?<=\{\{).*( ?).*(?=\}\})/,' ')
Any help with such a regex would be appreciated thanks!
You can do this with a complex regular expression I agree, but I find it simpler to use nested substitution. First use gsub to find the bracketed substrings and then use another to replace the entity.
string = 'this is a gap {{ for user in users }}'
result = string.gsub(/{{.*?}}/){ |s| s.gsub(/ /, ' ') }
# => "this is a gap {{ for user in users }}"
Single call to gsub using \K and \G
It's a bit tricky, but in Ruby 2.0+, we can do it with a compact regex thanks to the \K and \G tokens. Use this regex and replace with a space:
[^{}]*\K(?:{{|\G).*?\K (?=[^{]*})
See demo.
In Ruby:
result = subject.gsub(/[^{}]*\K(?:{{|\G).*?\K (?=[^{]*})/, ' ')
Explanation
[^{}]* matches any chars that are not braces
\K tells the engine to drop what was just matched
(?:{{|\G) either matches the opening curlies or asserts that we are positioned after the last match
.*?\K lazily matches chars (and drops them), up to...
Our match!
which, as the lookakead (?=[^{]*}) asserts, must be followed by any chars that are not an opening brace before meeting a closing brace