this is my serilog configuration :
Log.Logger = new LoggerConfiguration()
.WriteTo.RollingFile(System.IO.Path.Combine(AppDomain.CurrentDomain.BaseDirectory,
#"Logs\log-{Date}.txt"))
.MinimumLevel.ControlledBy(LogLevelSwitch)
.CreateLogger();
i'm logging like this :
var log = Serilog.Log.ForContext<SomeClassName>();
log.Information("some log text");
My question :
I need a modification in my configuration to create separate logs for each SourceContext.
I need log files somthing like Logs\log-{Date}-{SourceContext}.txt
thanks pros...
One way to implement this is to create your own ILogEventSink implementation (the interface is very simple) that internally dispatches to one of several RollingFileSinks based on the SourceContext property of the LogEvent.
To configure it (let's say you call your sink SourceRollingFileSink) you can use WriteTo.Sink():
Log.Logger = new LoggerConfiguration()
.WriteTo.Sink(new SourceRollingFileSink())
.CreateLogger();
There's no out-of-the-box implementation so some work is required, but it should not be complicated. Be aware that implementations of ILogEventSink need to be thread-safe.
Related
We are trying to load test our infrastructure of logstash/elastic. Since the actual logs are generated by a software that uses hardware, we are unable to simulate it at scale.
I am wondering if we can store the logs using file sink and later write a program that reads the log files and send data through the actual sink. Since, we are trying different setup, it would be great if we can swap different sinks for testing. Say http sink and elastic sink.
I thought of reading the json file one line at a time and then invoking Write method on the Logger. However I am not sure how to get the properties array from the json. Also, it would be great to hear if there are better alternatives in Serilog world for my needs.
Example parsing
var events= File.ReadAllLines(#"C:\20210520.json")
.Select(line => JsonConvert.DeserializeObject<dynamic>(line));
foreach (var o in objects)
{
DateTime timeStamp = o.Timestamp;
LogEventLevel level = o.Level;
string messageTemplate = o.MessageTemplate;
string exception = o.Exception;
var properties = (o.Properties as JObject);
List<object> parameters = new List<object>();
foreach (var property in properties)
{
if(messageTemplate.Contains(property.Key))
parameters.Add(property.Value.ToString());
}
logInstance.Write(level, messageTemplate, parameters.ToArray());
count++;
}
Example Json Event written to the file
{"Timestamp":"2021-05-20T13:15:49.5565372+10:00","Level":"Information","MessageTemplate":"Text dialog with {Title} and {Message} displayed, user selected {Selected}","Properties":{"Title":"Unload Device from Test","Message":"Please unload the tested device from test jig","Selected":"Methods.Option","SinkRepository":null,"SourceRepository":null,"TX":"TX2937-002 ","Host":"Host1","Session":"Host1-2021.05.20 13.12.44","Seq":87321,"ThreadId":3}}
UPDATE
Though this works for simple events,
it is not able to handle Context properties (there is a work around though using ForContext),
also it forces all the properties to be of type string and
not to mention that destucturing (#property) is not handled properly
If you can change the JSON format to Serilog.Formatting.Compact's CLEF format, then you can use Serilog.Formatting.Compact.Reader for this.
In the source app:
// dotnet add package Serilog.Formatting.Compact
Log.Logger = new LoggerConfiguration()
.WriteTo.File(new CompactJsonFormatter(), "./logs/myapp.clef")
.CreateLogger();
In the load tester:
// dotnet add package Serilog.Formatting.Compact.Reader
using (var target = new LoggerConfiguration()
.MinimumLevel.Verbose()
.WriteTo.Console()
.CreateLogger())
{
using (var file = File.OpenText("./logs/myapp.clef"))
{
var reader = new LogEventReader(file);
while (reader.TryRead(out var evt))
target.Write(evt);
}
}
Be aware though that load testing results won't be accurate for many sinks if you use repeated timestamps. You should consider re-mapping the events you read in to use current timestamps.
E.g. once you've loaded up evt:
var current = new LogEvent(DateTimeOffset.Now,
evt.Level,
evt.Exception,
evt.MessageTemplate,
evt.Properties);
target.Write(current);
The Log.Information("Hello"); is not writing to the table. I have used the basic configuration here in the readme file and the sink created my table OK on first run. I am certain that my user has read/write permission.
I am expecting the Log.Information("Hello"); to add a row to the table.
Serilog.Debugging.SelfLog.Enable(msg => Debug.WriteLine(msg));
var logDB = #"data source=xxxxxx\SQLEXPRESS;initial catalog=Eng;integrated security=False;persist security info=True;user id=xxxx;password=xxxxxxxx";
var sinkOpts = new SinkOptions();
sinkOpts.TableName = "SL24AddInLogging";
sinkOpts.AutoCreateSqlTable = true;
var columnOpts = new ColumnOptions();
columnOpts.Store.Remove(StandardColumn.Properties);
columnOpts.Store.Add(StandardColumn.LogEvent);
columnOpts.LogEvent.DataLength = 2048;
columnOpts.PrimaryKey = columnOpts.TimeStamp;
columnOpts.TimeStamp.NonClusteredIndex = true;
var log = new LoggerConfiguration()
.WriteTo.MSSqlServer(
connectionString: logDB,
sinkOptions: sinkOpts,
columnOptions: columnOpts
).CreateLogger();
I must be doing something wrong. What?
There are a number of things you can do to troubleshoot issues in Serilog. Several are listed in this answer here on StackOverflow:
Serilog MSSQL Sink doesn't write logs to database
I found my issue. In the past when I have used Serilog I have used the static "Log.Information()" rather than my variable that is declared in my code when I create the logger. In this case my declared variable is also "log" - lower case.
Log.Information() - bad.
log.Information() - good.
Edit
In addition, in case you run into this. I have used the static version to have my logging available throughout the entire project. To define a logger and have it available you can use the following pattern:
var log = new LoggerConfiguration()
.WriteTo.MSSqlServer(
connectionString: logDB,
sinkOptions: sinkOpts,
columnOptions: columnOpts
).CreateLogger();
Log.Logger = log;
This is my default Serilog configuration
SeriLogLevelSwitch.MinimumLevel = LogEventLevel.Information;
Log.Logger = new LoggerConfiguration()
.MinimumLevel.ControlledBy(SeriLogLevelSwitch)
.MinimumLevel.Override("Microsoft", LogEventLevel.Warning)
.MinimumLevel.Override("Microsoft.Hosting.Lifetime", LogEventLevel.Information)
.Enrich.FromLogContext()
....
How can i change the loglevel to Debug for a specific namespace at runtime when the default is Information?
Each of your MinimumLevel.Override can have its own LoggingLevelSwitch, which allows you to control the log level for each particular override at run-time.
Create individual LoggingLevelSwitch for each override that you intend to modify whilst the app is running, and store these instances in a place that you can access from other parts of your application, which will allow you to change the MinimumLevel of these LoggingLevelSwitch(es).
e.g.
public class LoggingLevelSwitches
{
// Logging level switch that will be used for the "Microsoft" namespace
public static readonly LoggingLevelSwitch MicrosoftLevelSwitch
= new LoggingLevelSwitch(LogEventLevel.Warning);
// Logging level switch that will be used for the "Microsoft.Hosting.Lifetime" namespace
public static readonly LoggingLevelSwitch MicrosoftHostingLifetimeLevelSwitch
= new LoggingLevelSwitch(LogEventLevel.Information);
}
Configure your Serilog logging pipeline to use these LoggingLevelSwitch instances:
static void Main(string[] args)
{
Log.Logger = new LoggerConfiguration()
.MinimumLevel.Override("Microsoft", LoggingLevelSwitches.MicrosoftLevelSwitch)
.MinimumLevel.Override("Microsoft.Hosting.Lifetime",
LoggingLevelSwitches.MicrosoftHostingLifetimeLevelSwitch)
.Enrich.FromLogContext()
.CreateLogger();
// ...
}
Then somewhere in your application, for example, in the code that handles your application configuration that can be changed at run-time, update the LoggingLevelSwitch instance(s) to the new LogEventLevel that you want:
public class AppSettings
{
void ChangeLoggingEventLevel()
{
LoggingLevelSwitches.MicrosoftHostingLifetimeLevelSwitch
.MinimumLevel = LogEventLevel.Error;
LoggingLevelSwitches.MicrosoftHostingLifetimeLevelSwitch
.MinimumLevel = LogEventLevel.Warning;
// ...
}
}
As you can see, the LogEventLevel is controlled by the LoggingLevelSwitch instances, so it's up to you to decide where in your application (and how) these instances will be modified, to affect the logging pipeline.
The example above I'm assuming you have a screen (or API) in your application that a user would be able to configure the logging levels.
If you don't have that, then another approach is to have a background thread that periodically checks a configuration file, an environment variable, or query a database, etc. to determine what these logging levels should be.
If you're using a .NET Core Host, you can use the Options pattern which can handle the refresh of the configuration for you, and allow you to execute code when the configuration changes (where you'd change the MinimumLevel of your LoggingLevelSwitch(es) you have.
You can use an environment variable paired with another MinimumLevel.Override to change the loglevel to Debug for a specific namespace at runtime like so:
using System;
...
SeriLogLevelSwitch.MinimumLevel = LogEventLevel.Information;
Log.Logger = new LoggerConfiguration()
.MinimumLevel.ControlledBy(SeriLogLevelSwitch)
.MinimumLevel.Override("Microsoft", LogEventLevel.Warning)
.MinimumLevel.Override("Microsoft.Hosting.Lifetime", LogEventLevel.Information)
.MinimumLevel.Override(Environment.GetEnvironmentVariable("SPECIFIC_NAMESPACE"), LogEventLevel.Debug)
.Enrich.FromLogContext()
....
Then, ensure the environment variable SPECIFIC_NAMESPACE is accessible by your application at runtime. Note that "namespace" is synonymous with "source context prefix"
Environment.GetEnvironmentVariable
I was using the following in my static constructor of the base class
static ApplicationBase()
{
Log.Logger = new LoggerConfiguration()
.Enrich.FromLogContext()
.Enrich.WithMachineName()
.Enrich.WithThreadId()
.Enrich.FromLogContext()
.MinimumLevel.Debug()
.WriteTo.Async(a => a.RollingFile(
new RenderedCompactJsonFormatter(),
#"c:\logs\log-{Date}.txt", fileSizeLimitBytes: 4194304))
.CreateLogger();
}
Now I need to attach one of our own custom enricher. The enricher is expecting a function that must use a modifier of the actual Application class.
For example, I need to do
.Enrich.WithStoreData(()=>GetStoreData)
Well, it doesn't have to be a function, the bottom line is, the call of GetStoreData is using an object instantiated in the actual child application class (and I cannot change the lifecycle of that object), so I can't access the object from the static constructor.
That means I have to move the logger creation to the normal base constructor. Because it has many children, how can I ensure the logger creation is executed only once? That means I have to apply a lock and check if the logger has been created already. That's really ugly.
And I am not using any container like autofac, so I will not want to create a wrapper of the logger.
At this point, I can only think of the idea creating the logger in the base constructor, and protect it with a lock.
Any other suggestion?
You can only use enrichment while configuring Serilog, and if you are configuring your static logger in your base class, then you cannot change the enrichment later at run time.
But you can use Contextual loggers at run time to add additional properties to your logger: Serilog Context and Correlation
Adding Log Context
// Log.Logger is initialized in your static base
var StudentLogger = Log.Logger.ForContext<Student>();
StudentLogger.Error(/* log message */);
Adding correlation:
// Log.Logger is initialized in your static base
var orderId = "some value";
var corrLog = Log.Logger.ForContext("orderId", orderId)
corrLog.Error(/* log message */);
I want to write Serilog Warning events to a .NET collection so they can be included in a report to the user. How do I configure the static Log.Logger to write to something like a static List<string<> ?
I saw the Rx Observer in the list of provided sinks, this was the only one that seemed to readily make .NET objects available or is there an obvious alternative that I missed?
Alternatively, is this a dumb way to do it - is there a better way to collect just Warning events to massage into a user-facing report?
class CollectionSink : ILogEventSink {
public ICollection<LogEvent> Events { get; } = new ConcurrentBag<LogEvent>();
public void Emit(LogEvent le) {
Events.Add(le);
}
}
var col = new CollectionSink();
Log.Logger = new LoggerConfiguration()
.WriteTo.Sink(col, LogEventLevel.Warning)
.CreateLogger();
// read col.Events....
Not sure this one will typecheck but this is essentially how I've done it a few times.
I found the following implementations useful as references:
CollectionSink from the Serilog Timings tests
InMemorySink from Serilog.Sinks.InMemory