How to change my instance names on Prometheus - monitoring

Question

I'm monitoring multiple computers in the same cluster, for that I'm using prometheus.
Here is my config file prometheus.yml:
# my global config
global:
scrape_interval: 15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
# scrape_timeout is set to the global default (10s).
# Alertmanager configuration
alerting:
alertmanagers:
- static_configs:
- targets:
# - alertmanager:9093
# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
rule_files:
# - "first_rules.yml"
# - "second_rules.yml"
# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
# The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
- job_name: "Server-monitoring-Api"
# metrics_path defaults to '/metrics'
# scheme defaults to 'http'.
static_configs:
- targets: ["localhost:9090"]
- targets: ["localhost:9182"]
- targets: ["192.168.1.71:9182"]
- targets: ["192.168.1.84:9182"]
I'm new to Prometheus, I want to show the name of my target, i.e: rather than using for example 192.168.1.71:9182 I only want the target name to be shown, I have a research, I found this:
relabel_configs:
- source_labels: [__meta_ec2_tag_Name]
target_label: instance
But I dont know how to use to relabel my targets(instances), any help will be appreciated, thanks for your help.

Answer 1

The snippet that you found should work only if you're using the EC2 service discover features of Prometheus (which doesn't seem your case since you're using some static targets).
I see a couple of options. You could expose directly in your metrics a different metrics (hostname) with the value of the hostname. Or you could use the textfile collector to expose the same metric as a static value (on a different port).
I recommend reading this post which explains why having a different metric for the "name" or "role" of the machine is usually a better approach than having a hostname label in your metrics.
It is also possible to add a custom label in the Prometheus config directly, something like (since you have your static targets anyhow). Finally, if you are already using the Prometheus node exporter you could use the node_uname_info metric (the nodename label).
- job_name: 'Kafka'
metrics_path: /metrics
static_configs:
- targets: ['10.0.0.4:9309']
labels:
hostname: hostname-a