I'm trying to send this very simple JSON string to Telegraf to be saved into InfluxDB:
{ "id": "id_123", "value": 10 }
So the request would be this: curl -i -XPOST 'http://localhost:8080/telegraf' --data-binary '{"id": "id_123","value": 10}'
When I make that request, I get the following answer: HTTP/1.1 204 No Content Date: Tue, 20 Apr 2021 13:02:49 GMT but when I check what was written to database, there is only value field:
select * from http_listener_v2
time host influxdb_database value
---- ---- ----------------- -----
1618923747863479914 my.host.com my_db 10
What am I doing wrong?
Here's my Telegraf config:
[global_tags]
[agent]
interval = "10s"
round_interval = true
metric_batch_size = 1000
metric_buffer_limit = 10000
collection_jitter = "0s"
flush_interval = "10s"
flush_jitter = "0s"
precision = ""
hostname = ""
omit_hostname = false
# OUTPUTS
[[outputs.influxdb]]
urls = ["http://127.0.0.1:8086"]
database = "telegraf"
username = "xxx"
password = "xxx"
[outputs.influxdb.tagdrop]
influxdb_database = ["*"]
[[outputs.influxdb]]
urls = ["http://127.0.0.1:8086"]
database = "httplistener"
username = "xxx"
password = "xxx"
[outputs.influxdb.tagpass]
influxdb_database = ["httplistener"]
# INPUTS
## system
[[inputs.cpu]]
percpu = true
totalcpu = true
collect_cpu_time = false
report_active = false
[[inputs.disk]]
ignore_fs = ["tmpfs", "devtmpfs", "devfs", "iso9660", "overlay", "aufs", "squashfs"]
[[inputs.mem]]
[[inputs.swap]]
[[inputs.system]]
## http listener
[[inputs.http_listener_v2]]
service_address = ":8080"
path = "/telegraf"
methods = ["POST", "PUT"]
data_source = "body"
data_format = "json"
[inputs.http_listener_v2.tags]
influxdb_database = "httplistener"
Use json_string_fields = ["id"]
My Application is built using Ruby On Rails. I'm ask to use Microsoft Azure Key Vault to store our secrets string.
I know there is Gems available made By the Microsoft Teams:
https://rubygems.org/gems/azure_mgmt_key_vault
https://rubygems.org/gems/azure_key_vault
How do I "extract" or "reference" a Key and pass it to my application?
With a lot of Work and Sweat I figured out. Even more, I was able to use a Certificate to connect to Microsoft Azure Key Vault. So below I put all my code with 2 ways to get the token. One with a client secret id and the other with a certificate.
I found how to generate a self-sign certificate (for debuging purpose) and get the
encode thumbprint:
Certificat that was upload to Azure was generated with:
openssl req -x509 -newkey rsa:4096 -keyout private_key.pem -out public_certificate.pem -nodes -days 3650
To obtain the x5t encode base64 thumbprint of the certificate:
echo $(openssl x509 -in public_certificate.pem -fingerprint -noout) | sed 's/SHA1 Fingerprint=//g' | sed 's/://g' | xxd -r -ps | base64
I built a GEM.
I have a Configuration file lib\azurekeyvault\configuration.rb:
module AzureKeyVault
class Configuration
attr_accessor :azure_tenant_id, :azure_client_id, :azure_client_secret, :azure_subscription_id, :vault_base_url, :api_version, :resource, :azure_certificate_thumbprint, :azure_certificate_private_key_file
def initialize
#azure_tenant_id = nil
#azure_client_id = nil
#azure_client_secret = nil
#azure_subscription_id = nil
#vault_base_url = nil
#api_version = nil
#resource = nil
#azure_certificate_thumbprint = nil
#azure_certificate_private_key_file = nil
end
end
end
This is the file where the magic happen lib\azurekeyvault\extraction.rb:
module AzureKeyVault
require 'singleton'
class Extraction
include Singleton
def initialize
#configuration = AzureKeyVault.configuration
end
def get_value(secret_name, secret_version = nil)
get_secret(secret_name, secret_version)
end
private
### Get a Secret value from Microsoft Azure Vault
## secret_name: Name of the Key which contain the value
## secret_version (optional): Version of the key value we need, by omitting version the system to use the latest available version
def get_secret(secret_name, secret_version = nil)
# GET {vaultBaseUrl}/secrets/{secret-name}/{secret-version}?api-version=7.1
vault_base_url = #configuration.vault_base_url
api_version = #configuration.api_version
azure_certificate_thumbprint = #configuration.azure_certificate_thumbprint
auth_token = nil
if azure_certificate_thumbprint.nil?
auth_token = get_auth_token()
else
auth_token = get_auth_certificate_token()
end
return nil if auth_token.nil?
url = "#{vault_base_url}/secrets/#{secret_name}/#{secret_version}?api-version=#{api_version}"
headers = { 'Authorization' => "Bearer " + auth_token }
begin
response = HTTParty.get(url, {headers: headers})
return response.parsed_response['value']
rescue HTTParty::Error => e
puts "HTTParty ERROR: #{e.message}"
raise e
rescue Exception => e
puts "ERROR: #{e.message}"
raise e
end
end
def get_auth_token
#Microsoft identity platform and the OAuth 2.0 client credentials flow
# https://learn.microsoft.com/en-us/azure/active-directory/develop/v2-oauth2-client-creds-grant-flow
# https://learn.microsoft.com/en-us/azure/active-directory/azuread-dev/v1-oauth2-client-creds-grant-flow#request-an-access-token
azure_tenant_id = #configuration.azure_tenant_id
azure_client_id = #configuration.azure_client_id
azure_client_secret = #configuration.azure_client_secret
resource = #configuration.resource
authUrl = "https://login.microsoftonline.com/#{azure_tenant_id}/oauth2/token"
data = {
'grant_type': 'client_credentials',
'client_id': azure_client_id,
'client_secret': azure_client_secret,
'resource': resource
}
begin
response= HTTParty.post(authUrl, body: data)
token = nil
if response
#puts response.to_json
token = response.parsed_response['access_token']
end
return token
rescue HTTParty::Error => e
puts "HTTParty ERROR: #{e.message}"
raise e
rescue Exception => e
puts "ERROR: #{e.message}"
raise e
end
end
def get_auth_certificate_token
begin
# Microsoft identity platform and the OAuth 2.0 client credentials flow
#
# Certificat that was upload to Azure was generated with:
# openssl req -x509 -newkey rsa:4096 -keyout private_key.pem -out public_certificate.pem -nodes -days 3650
#
# To obtain the x5t encode base64 thumbprint of the certificate:
# echo $(openssl x509 -in public_certificate.pem -fingerprint -noout) | sed 's/SHA1 Fingerprint=//g' | sed 's/://g' | xxd -r -ps | base64
# https://learn.microsoft.com/en-us/azure/active-directory/develop/v2-oauth2-client-creds-grant-flow
# https://learn.microsoft.com/en-us/azure/active-directory/azuread-dev/v1-oauth2-client-creds-grant-flow#request-an-access-token
azure_tenant_id = #configuration.azure_tenant_id
azure_client_id = #configuration.azure_client_id
resource = #configuration.resource
azure_certificate_thumbprint = #configuration.azure_certificate_thumbprint
azure_certificate_private_key_file = #configuration.azure_certificate_private_key_file
authUrl = "https://login.microsoftonline.com/#{azure_tenant_id}/oauth2/token"
exp = Time.now.to_i + 4 * 3600
nbf = Time.now.to_i - 3600
jti = SecureRandom.uuid
#//x5t THUMBPRINT of Cert
header = {
"alg": "RS256",
"typ": "JWT",
"x5t": azure_certificate_thumbprint
}
#Claim (payload)
payload = {
"aud": authUrl,
"exp": exp,
"iss": azure_client_id,
"jti": jti,
"nbf": nbf,
"sub": azure_client_id
}
token = "#{Base64.strict_encode64(header.to_json)}.#{Base64.strict_encode64(payload.to_json)}"
# Get the private key, from the file
azure_certificate_private_key = OpenSSL::PKey.read(File.read(azure_certificate_private_key_file))
# The hash algorithm, I assume SHA256 is being used
base64_signature = Base64.strict_encode64(azure_certificate_private_key.sign(OpenSSL::Digest::SHA256.new, token))
jwt_client_assertion = "#{token}.#{base64_signature}"
data = {
'grant_type': 'client_credentials',
'client_id': azure_client_id,
'client_assertion_type': 'urn:ietf:params:oauth:client-assertion-type:jwt-bearer',
'client_assertion': jwt_client_assertion,
'resource': resource
}
response = HTTParty.post(authUrl, body: data)
token = nil
if response
token = response.parsed_response['access_token']
end
return token
rescue HTTParty::Error => e
puts "HTTParty ERROR: #{e.message}"
raise e
rescue Exception => e
puts "ERROR: #{e.message}"
raise e
end
end
end
end
I have also an Initialiser where I assign value for my configuration variables
AzureKeyVault.configure do |config|
config.azure_tenant_id = ENV["AZURE_VAULT_TENANT_ID"]
config.azure_client_id = ENV["AZURE_VAULT_CLIENT_ID"]
config.azure_client_secret = ENV["AZURE_VAULT_CLIENT_SECRET"]
config.azure_subscription_id = ENV["AZURE_VAULT_SUBSCRIPTION_ID"]
config.vault_base_url = ENV["AZURE_VAULT_BASE_URL"]
config.api_version = ENV["AZURE_VAULT_API_VERSION"]
config.resource = ENV["AZURE_VAULT_RESOURCE"]
# To obtain the x5t encode base64 thumbprint of the certificate:
# echo $(openssl x509 -in public_certificate.pem -fingerprint -noout) | sed 's/SHA1 Fingerprint=//g' | sed 's/://g' | xxd -r -ps | base64
config.azure_certificate_thumbprint = ENV["AZURE_CERTIFICATE_THUMBPRINT"]
#Certificat that was upload to Azure was generated with:
# openssl req -x509 -newkey rsa:4096 -keyout private_key.pem -out public_certificate.pem -nodes
config.azure_certificate_private_key_file = ENV["AZURE_CERTIFICATE_PRIVATE_KEY_FILE"]
end
Note: This post and answer (#Jason Johnston) help me a lot to understand what was going on: Office 365 Rest API - Daemon week authentication
I'm using freeradius on docker and is trying to get a response back when access request is sent.
When radtest is run using: radtest bob testpw 127.0.0.1 1812 sharedSecret
Sent Access-Request Id 18 from 0.0.0.0:56219 to 127.0.0.1:1812 length 73
User-Name = "bob"
User-Password = "testpw"
NAS-IP-Address = 172.17.0.2
NAS-Port = 1812
Message-Authenticator = 0x00
Cleartext-Password = "testpw"
Sent Access-Request Id 18 from 0.0.0.0:56219 to 127.0.0.1:1812 length 73
User-Name = "bob"
User-Password = "testpw"
NAS-IP-Address = 172.17.0.2
NAS-Port = 1812
Message-Authenticator = 0x00
Cleartext-Password = "testpw"
Sent Access-Request Id 18 from 0.0.0.0:56219 to 127.0.0.1:1812 length 73
User-Name = "bob"
User-Password = "testpw"
NAS-IP-Address = 172.17.0.2
NAS-Port = 1812
Message-Authenticator = 0x00
Cleartext-Password = "testpw"
(0) No reply from server for ID 18 socket 3
I know that this normally happens when the shared secret is wrong, but it is the same as mentioned in the configuration files.
When debugged I get the bellow error.
Ignoring request to auth address * port 1812 bound to server default
from unknown client 172.17.0.3 port 60699 proto udp Ready to process
requests
error is given without any response back from the server.
The clients.conf file is as follows;
client dockernet {
ipaddr = 172.17.0.0/16
secret = sharedSecret
}
and authorise file;
bob Cleartext-Password := "testpw"
default file;
server default {
listen {
type = auth
ipv4addr = *
# ipv6addr = *
# ipaddr = *
port = 5
# interface = eth0
# clients = per_socket_clients
recv_buff = 65536
limit {
max_connections = 16
lifetime = 0
idle_timeout = 30
}
}
authorize {
update request {
&Tmp-String-0 := "%{string:User-Password}"
&User-Password := "%{string:Tmp-String-0}"
}
# filter_username
# filter_password
preprocess
# operator-name
# cui
# auth_log
chap
mschap
digest
# wimax
# IPASS
suffix
# ntdomain
eap {
ok = return
# updated = return
}
#
# unix
# Read the 'users' file. In v3, this is located in
# raddb/mods-config/files/authorize
files
-sql
# smbpasswd
-ldap
# daily
expiration
logintime
pap
# Autz-Type Status-Server {
# }
}
authenticate {
ntlm_auth
Auth-Type PAP {
pap
}
Auth-Type CHAP {
chap
}
Auth-Type MS-CHAP {
mschap
}
mschap
digest
# pam
# Auth-Type LDAP {
# ldap
# }
#
# Allow EAP authentication.
eap
# Auth-Type eap {
# eap {
# handled = 1
# }
# if (handled && (Response-Packet-Type == Access-Challenge)) {
# attr_filter.access_challenge.post-auth
# handled # override the "updated" code from attr_filter
# }
# }
}
#
# Pre-accounting. Decide which accounting type to use.
#
preacct {
preprocess
#
# Merge Acct-[Input|Output]-Gigawords and Acct-[Input-Output]-Octets
# into a single 64bit counter Acct-[Input|Output]-Octets64.
#
# acct_counters64
# update request {
# &FreeRADIUS-Acct-Session-Start-Time = "%{expr: %l - %{%{Acct-Session-Time}:-0} - %{%{Acct-Delay-Time}:-0}}"
# }
acct_unique
# IPASS
suffix
# ntdomain
#
# Read the 'acct_users' file
files
}
An Access-Accept or an Access-Reject is expected. But no response, when debugged and gives the above error in the logs.
It works! after I changed the clients.conf file to have a netmask and not ipaddr = 172.17.0.0/16
client dockernet {
ipaddr = 172.17.0.0
secret = sharedSecret
netmask = 24
shortname = dockernet
}
This article helped me https://linux.die.net/man/5/clients.conf
I'm trying to create a new UPS.conf file for telegraf to collect data from a batch of ups units via SNMP. Inputs such as hostname and upsType when queried via SNMPGet the OID's return a String, but when run using Telegraf I get only integer results.
My UPS.conf File
[[inputs.snmp]]
agents = [ "192.168.15.60", "192.168.15.64" , "192.168.15.65","192.168.15.66","192.168.15.67" ]
## Timeout for each SNMP query.
timeout = "10s"
## Number of retries to attempt within timeout.
retries = 3
## SNMP version, values can be 1, 2, or 3
version = 3
## SNMP community string.
community = "heabc"
#
# ## The GETBULK max-repetitions parameter
# max_repetitions = 10
#
# ## SNMPv3 auth parameters
sec_name = "grafana"
auth_protocol = "SHA" # Values: "MD5", "SHA", ""
auth_password = "redacted"
sec_level = "authPriv" # Values: "noAuthNoPriv", "authNoPriv", "authPriv"
# #context_name = ""
priv_protocol = "AES" # Values: "DES", "AES", ""
priv_password = "redacted"
#
# ## measurement name
[[inputs.snmp.field]]
name = "hostname"
oid = "iso.1.3.6.1.2.1.1.6.0"
conversion = ""
is_tag = true
[[inputs.snmp.field]]
name = "upsType"
oid = "iso.1.3.6.1.4.1.318.1.1.1.1.1.1.0"
is_tag = true
conversion = ""
[[inputs.snmp.field]]
name = "batteryCapacityPercent"
oid = "iso.1.3.6.1.4.1.318.1.1.1.2.2.1.0"
[[inputs.snmp.field]]
name = "batteryTemp"
oid = "iso.1.3.6.1.4.1.318.1.1.1.2.2.2.0"
[[inputs.snmp.field]]
name = "batteryRuntimeRemain"
oid = "iso.1.3.6.1.4.1.318.1.1.1.2.2.3.0"
[[inputs.snmp.field]]
name = "batteryReplace"
oid = "iso.1.3.6.1.4.1.318.1.1.1.2.2.4.0"
[[inputs.snmp.field]]
name = "inputVoltage"
oid = "iso.1.3.6.1.4.1.318.1.1.1.3.2.1.0"
[[inputs.snmp.field]]
name = "inputFreq"
oid = "iso.1.3.6.1.4.1.318.1.1.1.3.2.4.0"
[[inputs.snmp.field]]
name = "lastTransferReason"
oid = "iso.1.3.6.1.4.1.318.1.1.1.3.2.5.0"
[[inputs.snmp.field]]
name = "outputVoltage"
oid = "iso.1.3.6.1.4.1.318.1.1.1.4.2.1.0"
[[inputs.snmp.field]]
name = "outputFreq"
oid = "iso.1.3.6.1.4.1.318.1.1.1.4.2.2.0"
[[inputs.snmp.field]]
name = "outputLoad"
oid = "iso.1.3.6.1.4.1.318.1.1.1.4.2.3.0"
[[inputs.snmp.field]]
name = "ouputCurrent"
oid = "iso.1.3.6.1.4.1.318.1.1.1.4.2.4.0"
[[inputs.snmp.field]]
name = "lastSelfTestResult"
oid = "iso.1.3.6.1.4.1.318.1.1.1.7.2.3.0"
[[inputs.snmp.field]]
name = "lastSelfTestDate"
oid = "iso.1.3.6.1.4.1.318.1.1.1.7.2.4.0"
Ouput of telegraf --test --config UPS.conf - Notice the hostname on each, one is 121, one is 91, 82 etc. The upsType field also comes through as a string, but is being converted to a number.
* Plugin: inputs.snmp, Collection 1
> snmp,hostname=121,upsType=122,agent_host=192.168.15.60,host=HEAGrafana batteryTemp=124i,inputVoltage=127i,outputFreq=131i,outputLoad=132i,lastSelfTestDate=135i,outputVoltage=130i,ouputCurrent=133i,lastSelfTestResult=134i,batteryCapacityPercent=123i,batteryRuntimeRemain=125i,batteryReplace=126i,inputFreq=128i,lastTransferReason=129i 1527721763000000000
> snmp,host=HEAGrafana,hostname=103,upsType=104,agent_host=192.168.15.64 batteryCapacityPercent=105i,batteryReplace=108i,inputFreq=110i,lastTransferReason=111i,lastSelfTestResult=116i,ouputCurrent=115i,lastSelfTestDate=117i,batteryTemp=106i,batteryRuntimeRemain=107i,inputVoltage=109i,outputVoltage=112i,outputFreq=113i,outputLoad=114i 1527721764000000000
> snmp,hostname=91,upsType=92,agent_host=192.168.15.65,host=HEAGrafana lastSelfTestDate=105i,batteryTemp=94i,inputVoltage=97i,inputFreq=98i,outputFreq=101i,outputLoad=102i,ouputCurrent=103i,lastSelfTestResult=104i,batteryCapacityPercent=93i,batteryRuntimeRemain=95i,batteryReplace=96i,lastTransferReason=99i,outputVoltage=100i 1527721766000000000
> snmp,hostname=82,upsType=83,agent_host=192.168.15.66,host=HEAGrafana batteryReplace=87i,inputVoltage=88i,inputFreq=89i,lastTransferReason=90i,outputLoad=93i,batteryCapacityPercent=84i,batteryTemp=85i,batteryRuntimeRemain=86i,lastSelfTestResult=95i,lastSelfTestDate=96i,outputVoltage=91i,outputFreq=92i,ouputCurrent=94i 1527721768000000000
> snmp,hostname=61,upsType=62,agent_host=192.168.15.67,host=HEAGrafana lastTransferReason=69i,outputVoltage=70i,outputFreq=71i,outputLoad=72i,batteryTemp=64i,batteryReplace=66i,inputVoltage=67i,inputFreq=68i,lastSelfTestDate=75i,batteryCapacityPercent=63i,batteryRuntimeRemain=65i,ouputCurrent=73i,lastSelfTestResult=74i 1527721769000000000
Output of snmpget -v2c -c heabc 192.168.15.60 .1.3.6.1.4.1.318.1.1.1.1.1.1.0 - It returns a string.
iso.3.6.1.4.1.318.1.1.1.1.1.1.0 = STRING: "Smart-UPS X 3000"
I want encrypt data with AES 256bit ECB mode using PKCS5padding
My ruby method is as follows, how to use PKCS5Padding here
def encrypt(raw_data,key)
cipher = OpenSSL::Cipher::AES.new(256, :ECB)
cipher.encrypt
cipher.key = key
encrypted_data = cipher.update(raw_data) + cipher.final
end
here key is OpenSSL::PKey::RSA type, throwing no implicit conversion of OpenSSL::PKey::RSA into String exception
I think your key is in the wrong format. You're trying to pass an RSA key, when the key should just be a hash string ... something like:
key = SecureRandom.hex(32)
=> "b67f7a5bf031aaa730473e5a9612a94b157c43aed5f52a2e70c9573f2d5a4ecd"
You should use
key = cipher.random_key
instead of RSA key
I have used it in following way for my purpose
Generate cypher random keys
Do AES encryption of data with these keys
Before supply the keys encrypt it with RSA public key
At receiver end
Decrypt the cypher keys with RSA private key
Decrypt the data with resultant cypher keys
Note: We can not encrypt large data with RSA private/public key based technique
Super secured Example
# At sender side
public_key_file = 'public.pem'
message = 'Hey vishh you are awesome!!'
cipher = OpenSSL::Cipher::AES.new(128, :CBC)
cipher.encrypt
aes_key = cipher.random_key
encrypted_data = cipher.update(message) + cipher.final
# encrypted_data is ready to travel
rsa = OpenSSL::PKey::RSA.new(File.read(public_key_file))
rsa_cypher_key = rsa.public_encrypt(aes_key)
# rsa_cypher_key is ready to travel
# sending these data in encoded format is good idea
encrypted_data = Base64.encode64(encrypted_data)
rsa_cypher_key = Base64.encode64(rsa_cypher_key)
====> encrypted_data + rsa_cypher_key =====> Travelling
encrypted_data = Base64.decode64(encrypted_data)
rsa_cypher_key = Base64.decode64(rsa_cypher_key) # decode the data
# At recevier side
private_key_file = 'private.pem'
# Decrypt the cypher key with private key
rsp = OpenSSL::PKey::RSA.new(File.read('./config/private.pem'))
aes_key = private_key.private_decrypt(rsa_cypher_key)
decipher = OpenSSL::Cipher::AES.new(128, :CBC)
decipher.decrypt
decipher.key = aes_key
message = decipher.update(encrypted_data) + decipher.final
p message
'Hey vishh you are awesome!!'