I have Serilog.Sinks.Syslog, Serilog V2, Serilog.Sinks.Console and Serilog.Sinks.PeriodicBatching, I am pushing logs to a Nagios LS server using UDPSyslog but what I have noticed is that unlike other loggers the serilog udpsyslogger is not adding milliseconds to the event timestamp as such I cannot trace properly.
My code is in C#.
/*
* Logging parameters to the syslog
*/
var log = new LoggerConfiguration().Enrich.With().WriteTo.UdpSyslog(outputTemplate: syslogMessageTemplate, host: syslogServer, port: syslogServerPort, appName: applicationName, format:SyslogFormat.RFC3164 , facility: Facility.User).CreateLogger();
Below is the picture. Any idea what i am missing?
Related
I started kafka connector using following command:
./bin/connect-standalone etc/schema-registry/connect-avro-standalone.properties etc/kafka-connect-postgres/connect-postgres.properties
Serialization props in the connect-avro-standalone.properties is:
key.converter=io.confluent.connect.avro.AvroConverter
key.converter.schema.registry.url=http://localhost:8081
value.converter=io.confluent.connect.avro.AvroConverter
value.converter.schema.registry.url=http://localhost:8081
internal.key.converter=org.apache.kafka.connect.json.JsonConverter
internal.value.converter=org.apache.kafka.connect.json.JsonConverter
internal.key.converter.schemas.enable=false
internal.value.converter.schemas.enable=false
I've created a java backend which listen to this kafka stream topic and its able to get the data from postgres with each add/update/delete.
But the data is coming in some unknown encoding format and that's why ican't read the data correctly.
Here is the relevant code snippet:
properties.put(StreamsConfig.DEFAULT_KEY_SERDE_CLASS_CONFIG,
Serdes.String().getClass().getName());
properties.put(StreamsConfig.DEFAULT_VALUE_SERDE_CLASS_CONFIG,
Serdes.ByteArray().getClass().getName());
StreamsBuilder streamsBuilder = new StreamsBuilder();
final Serde<String> stringSerde = Serdes.String();
final Serde<byte[]> byteSerde = Serdes.ByteArray();
streamsBuilder.stream(Pattern.compile(getTopic()), Consumed.with(stringSerde, byteSerde))
.mapValues(data -> {
System.out.println("->"+new String(data));
return data;
});
I'm confused on where and what I need to change; in the avro connector prop or in the java side code
Your Kafka Connect config here means that the messages on the Kafka topic will be Avro serialised:
value.converter=io.confluent.connect.avro.AvroConverter
Which means that you need to deserialise using Avro in your Streams app. See here for more details: https://docs.confluent.io/current/streams/developer-guide/datatypes.html#avro
I followed the BlazeMeter article to monitor Gatling tests with Grafana and InfluxDB but no data is sent to InfluxDB and not any database created with the name "graphite".
InfluxDB is up and listen to port :2003. This is the log from InfluxDB:
2018-06-24T09:48:17Z Listening on TCP: [::]:2003 service=graphite addr=:2003
And I set gatling.conf fields to these:
data {
#writers = [console, file] # The list of DataWriters to which Gatling write simulation data (currently supported : console, file, graphite, jdbc)
console {
#light = false # When set to true, displays a light version without detailed request stats
}
file {
#bufferSize = 8192 # FileDataWriter's internal data buffer size, in bytes
}
leak {
#noActivityTimeout = 30 # Period, in seconds, for which Gatling may have no activity before considering a leak may be happening
}
graphite {
light = false # only send the all* stats
host = "localhost" # The host where the Carbon server is located
port = 2003 # The port to which the Carbon server listens to (2003 is default for plaintext, 2004 is default for pickle)
protocol = "tcp" # The protocol used to send data to Carbon (currently supported : "tcp", "udp")
rootPathPrefix = "gatling" # The common prefix of all metrics sent to Graphite
bufferSize = 8192 # GraphiteDataWriter's internal data buffer size, in bytes
writeInterval = 1 # GraphiteDataWriter's write interval, in seconds
}
}
gatling.conf is in src/test/resources folder and I ensured that this config file is loaded by Gatling by debugging it.
What I have missed?
You have invalid data writers configuration. Set it to:
writers = [console, file, graphite]
My question was already asked but I didn't succeed to solve my issue.
I don't succeed to send my data from Gatling in real time to InfluxDB.
I'm on Windows 10.
Gatling Version: 2.3.0 (the last one).
InfluxDB version: 1.3.5 (the last is 1.3.6).
My gatling.conf:
data {
writers = [console, file, graphite] # The list of DataWriters to which Gatling write simulation data (currently supported : console, file, graphite, jdbc)
console {
#light = false # When set to true, displays a light version without detailed request stats
}
file {
#bufferSize = 8192 # FileDataWriter's internal data buffer size, in bytes
}
leak {
#noActivityTimeout = 30 # Period, in seconds, for which Gatling may have no activity before considering a leak may be happening
}
graphite {
#light = false # only send the all* stats
host = "127.0.0.1" # The host where the Carbon server is located
port = "2003" # The port to which the Carbon server listens to (2003 is default for plaintext, 2004 is default for pickle)
protocol = "tcp" # The protocol used to send data to Carbon (currently supported : "tcp", "udp")
rootPathPrefix = "gatling" # The common prefix of all metrics sent to Graphite
#bufferSize = 8192 # GraphiteDataWriter's internal data buffer size, in bytes
#writeInterval = 1 # GraphiteDataWriter's write interval, in seconds
}
}
My influxdb.conf:
[http]
# Determines whether HTTP endpoint is enabled.
enabled = true
# The bind address used by the HTTP service.
bind-address = "127.0.0.1:8086"
###
### [[graphite]]
###
### Controls one or many listeners for Graphite data.
###
[[graphite]]
# Determines whether the graphite endpoint is enabled.
enabled = true
database = "gatlingdb"
# retention-policy = ""
bind-address = ":2003"
protocol = "tcp"
# consistency-level = "one"
templates = [
"gatling.*.*.*.*.measurement.simulation.request.status.field"
]
My gatlingdb database is created on InfluxDB, it stays empty.
When I try:
C:\InfluxDB-1.3.5-1>influx -host 127.0.0.1
I'm connected to InfluxDB
>USE gatlingdb
I'm connected to my database. Then:
>SHOW SERIES
and
>SELECT * FROM gatling
Don't return anything. It's empty.
Note: I put "FROM gatling" because I put that in my gatling.conf: rootPathPrefix = "gatling"
I didn't download Graphite but I saw that InfluxDB accept the graphite protocol. I assume I can send data from Gatling to InfluxDB. I certainly missed something.
I succeeded in connecting InfluxDB to Grafana and I display data from other databases. I just missed the connection between Gatling and InfluxDB.
Thanks in advance for your help, I definitely need it!
Anthony
I'm almost finished the article which shows all the steps required to create the whole monitoring infrastructure using the Gatling, Grafana and InfluxDB (btw, without Graphite installed separately) which worked very well for me.
I think I'll publish it in my blog on the blazemeter.com just in few days! So stay tuned there!
http://blazemeter.com/blog
There you will even find the ready solution to spin up everything inside the Docker.
But until this (if it is urgent for you), can share my InfluxDB config section:
[[graphite]]
enabled = true
bind-address = ":2003"
database = "graphite"
retention-policy = ""
protocol = "tcp"
batch-size = 5000
batch-pending = 10
batch-timeout = "1s"
consistency-level = "one"
separator = "."
udp-read-buffer = 0
gatling.conf:
graphite {
light = false # only send the all* stats
host = "localhost" # The host where the Carbon server is located
port = 2003 # The port to which the Carbon server listens to (2003 is default for plaintext, 2004 is default for pickle)
protocol = "tcp" # The protocol used to send data to Carbon (currently supported : "tcp", "udp")
rootPathPrefix = "gatling" # The common prefix of all metrics sent to Graphite
bufferSize = 8192 # GraphiteDataWriter's internal data buffer size, in bytes
writeInterval = 1 # GraphiteDataWriter's write interval, in seconds
}
The first thing you need to check is that InfluxDB actually accepts incoming metrics via graphite protocol. For example, during InfluxDB startup logs you should find this line:
influxdb_1 | [I] 2018-01-26T13:40:37Z Listening on TCP: [::]:2003 service=graphite addr=:2003
Is there a way of adding login security to the admin servlet?
Seems like in V0.7 you could add the following two to your yaml file :
adminUsername: user1234
adminPassword: pass5678
However I tried that in the latest version (0.9.2) and it gives me an error saying :
server.yaml has an error:
* Unrecognized field at: server.adminConnectors.[0].adminUsername
Did you mean?:
- soLingerTime
- bindHost
- idleTimeout
- useServerHeader
- useDateHeader
[14 more]
This is what I have :
adminConnectors:
- type: http
port: 9180
adminUsername: user1234
adminPassword: pass5678
I run an app on Heroku which only allows the application to make a single port available. I attach the AdminServlet to the main HTTP port (8080) with this in the run() method of my Application (Kotlin):
environment.applicationContext.apply {
setAttribute(MetricsServlet.METRICS_REGISTRY, environment.metrics())
setAttribute(HealthCheckServlet.HEALTH_CHECK_REGISTRY, environment.healthChecks())
addServlet(NonblockingServletHolder(AdminServlet()), "/admin/*")
}
Then, I protect this path with a BasicAuthFilter (still Kotlin, you should use it):
val basicAuthFilter = BasicAuthFilter("admin", configuration.adminUsername, configuration.adminPassword)
val adminFilter = environment.servlets().addFilter("AdminFilter", basicAuthFilter)
adminFilter.addMappingForUrlPatterns(EnumSet.of(DispatcherType.REQUEST), false, "/admin/*")
How can I increase the timeout value of a Worklight adapter procedure? My app crashes and I see the following in the exception details:
"response [/apps/services/api/index/common/query] success: /-secure-
{"responseID":"24","errors":["Invocation of procedure 'getFTTitle' has
timed out after 30
sec."],"isSuccessful":false,"warnings":[],"info":[]}/ "
There are several places in Worklight where a timeout value can be specified:
CLIENT ----> WORKLIGHT SERVER -- (adapter) --> BACKEND
You can increase the adapter procedure timeout (Worklight Server --> Backend) as follows:
<procedure name="nameHere" requestTimeoutInSeconds="valueHere"/>
I don't know what is your specific use case, so be sure to also increase the client-side timeout. Have them match each other.
WL.Client.invokeProcedure(invocationData,{
onSuccess : getDataSuccess,
onFailure : getDataFailure,
timeout : valueHere
});
Also note that if you need to increase your timeout to a whole minute, consider that something may not be right here...
var wlInitOptions =
{
// # Worklight server connection timeout
timeout: 60000,
...
}