Remove new lines from Message and Exception - serilog

Question

I am using Serilog in an Asp dot net core application. I want my my log files to be human readable, but also be able to be parsed easily.
The problem that I have come across is that exceptions are logged with line breaks. Some Microsoft events have messages that include new lines.
I would like to be able to parse the log file with one event per line.
I could write my own implementation of ITextFormatter that replaces new lines with \r\n, but that we mean I would need to duplicate much of the logic in MessageTemplateTextFormatter and other classes.

Answer 1

After digging into this for a while, I was able to come up with an answer. Andy West's answer pointed me in the right direction.
There are two separate issues here: CRLFs in the message and CRLFs in the exception.
I was able to solve the message problem by changing "{Message}" to "{Message:j}" in the outputTemplate.
Changing the exception was a little trickier. I had to add an enricher:
class ExceptionEnricher : ILogEventEnricher
{
public void Enrich(LogEvent logEvent, ILogEventPropertyFactory propertyFactory)
{
if (logEvent.Exception == null)
return;
var logEventProperty = propertyFactory.CreateProperty("EscapedException", logEvent.Exception.ToString().Replace("\r\n", "\\r\\n"));
logEvent.AddPropertyIfAbsent(logEventProperty);
}
}
This adds and new property called EscapedException. This has to be added to the configuration with .Enrich.With().
Then I replaced "{Exception}" with "{EscapedException}" in the outputTemplate.

Answer 2

This technique will remove all CRLF. First a new ITextFormatter.
public class RemoveCrLf : ITextFormatter
{
private const int DefaultWriteBuffer = 256;
private readonly ITextFormatter _textFormatter;
/// <summary>
///
/// </summary>
/// <param name="textFormatter"></param>
public RemoveCrLf(ITextFormatter textFormatter)
{
_textFormatter = textFormatter;
}
/// <summary>
///
/// </summary>
/// <param name="logEvent"></param>
/// <param name="output"></param>
public void Format(LogEvent logEvent, TextWriter output)
{
var buffer = new StringWriter(new StringBuilder(DefaultWriteBuffer));
_textFormatter.Format(logEvent, buffer);
var logText = buffer.ToString();
output.WriteLine(logText.Trim().Replace("\n","\\n").Replace("\r","\\r"));
output.Flush();
}
}
Use it like this
configuration.WriteTo.Console(new RemoveCrLf(new MessageTemplateTextFormatter("[{Timestamp:HH:mm:ss} {Level:u3}] {Message:lj} {Exception}")));
Of course, customise the output template as needed.

Answer 3

In reference to the solution from #Andrew Radford, that solution was working for me only on windows, but not on linux docker environment as expected. Therefore
I have enhanced the solution for messages as well as exceptions.
I have used Environment.NewLine for case matching in Regex which will pick the case based on the hosted environment.
Add following class for Messages.
public class MessageEnricher : ILogEventEnricher
{
public void Enrich(LogEvent logEvent, ILogEventPropertyFactory propertyFactory)
{
if (logEvent.MessageTemplate == null)
return;
var logEventProperty = propertyFactory.CreateProperty("EscapedMessage", Regex.Replace(logEvent.MessageTemplate.ToString(), Environment.NewLine, "[Newline]"));
logEvent.AddPropertyIfAbsent(logEventProperty);
}
}
Use following class for Exceptions.
public void Enrich(LogEvent logEvent, ILogEventPropertyFactory propertyFactory)
{
if (logEvent.Exception == null)
return;
var logEventProperty = propertyFactory.CreateProperty("EscapedException", Regex.Replace(logEvent.Exception.ToString(), Environment.NewLine, "[Newline]"));
logEvent.AddPropertyIfAbsent(logEventProperty);
}
Then add these both helper classes into program.cs
Log.Logger = new LoggerConfiguration()
.ReadFrom.Configuration(Configuration)
.Enrich.With(new ExceptionEnricher())
.Enrich.With(new MessageEnricher())
.Enrich.FromLogContext()
.CreateLogger();
Update the appsettings output template
"outputTemplate": "{Timestamp:yyyy-MM-dd HH:mm:ss.fff} {Level:u4} {SourceContext:l} : {EscapedMessage}{NewLine}{EscapedException}{NewLine}"

Answer 4

You didn't specify how you are logging exceptions, but assuming you are using something like:
Log.Error("An error occurred: {Exception}", exception);
then the exception will be rendered using ToString(). In that case, couldn't you simply do this instead:
Log.Error("An error occurred: {Exception}", exception.ToString().Replace(System.Environment.NewLine, "\\r\\n"));
Of course, you could refactor this into a method (maybe an extension method) if you need to do it more than one place.