How to transfer MDC via gelf to graylog? - log4j2

Question

We have a logging flow where our java application fills an MDC which is transferred via log4j2 to syslog to a central rsyslog installation. Here we make extensive use of MDC. Our setup is thus:
<Syslog name="syslog" format="RFC5424" host="localhost" port="514" protocol="UDP"
appName="messaging_platform.${application}" mdcId="mdc" includeMDC="true" facility="LOCAL5" connectTimeoutMillis="100" ignoreExceptions="false">
<LoggerFields>
<KeyValuePair key="class" value="%c"/>
<KeyValuePair key="classname" value="%c{1}"/>
<KeyValuePair key="exception" value="%ex{full}"/>
<KeyValuePair key="method" value="%method"/>
<KeyValuePair key="line" value="%line"/>
<KeyValuePair key="application_name" value="${application}"/>
<KeyValuePair key="sequenceNumber" value="%sequenceNumber"/>
<KeyValuePair key="application_version" value="${application.version}"/>
<KeyValuePair key="marker" value="%marker"/>
<KeyValuePair key="thread" value="%thread"/>
<KeyValuePair key="system_nano_time" value="%nano"/>
<KeyValuePair key="app_uptime" value="%relative"/>
</LoggerFields>
</Syslog>
I'm trying to convert this to use graylog and it seems GELF is the recommended transport protocol for that. I've found multiple libraries to do this and started with the build-in GelfLayout of log4j2. But that does not support LoggerFields.
So what's the recommended way to get these fields into Graylog? If I do
<Socket name="Graylog" protocol="udp" host="localhost" port="12201">
<GelfLayout host="localhost" compressionType="GZIP" compressionThreshold="1024">
<KeyValuePair key="class" value="%c"/>
<KeyValuePair key="classname" value="%c{1}"/>
<KeyValuePair key="exception" value="%ex{full}"/>
<KeyValuePair key="method" value="%method"/>
<KeyValuePair key="line" value="%line"/>
<KeyValuePair key="application_name" value="${application}"/>
<KeyValuePair key="sequenceNumber" value="%sequenceNumber"/>
<KeyValuePair key="application_version" value="${application.version}"/>
<KeyValuePair key="marker" value="%marker"/>
<KeyValuePair key="thread" value="%thread"/>
<KeyValuePair key="system_nano_time" value="%nano"/>
<KeyValuePair key="app_uptime" value="%relative"/>
</GelfLayout>
</Socket>
I get a bunch of fields in Graylog but the values are not expanded.

Answer 1

I found a solution with logstash-gelf:
<Gelf name="gelf" host="udp:localhost" port="12201" version="1.1" extractStackTrace="true"
filterStackTrace="true" mdcProfiling="true" includeFullMdc="true" maximumMessageSize="8192"
originHost="%host{fqdn}">
<Field name="class" pattern="%c"/>
<Field name="classname" pattern="%c{1}"/>
<Field name="exception" pattern="%ex{full}"/>
<Field name="method" pattern="%method"/>
<Field name="line" pattern="%line"/>
<Field name="application_name" pattern="${application}"/>
<Field name="sequenceNumber" pattern="%sequenceNumber"/>
<Field name="application_version" pattern="${application.version}"/>
<Field name="marker" pattern="%marker"/>
<Field name="thread" pattern="%thread"/>
<Field name="system_nano_time" pattern="%nano"/>
<Field name="app_uptime" pattern="%relative"/>
<Field name="severity" pattern="%level{WARN=Warning, DEBUG=Debug, ERROR=Error, TRACE=Trace, INFO=Info}"/>
</Gelf>

Answer 2

I use logstash-gelf with the following logback configuration;
logback.xml configuration:
<appender name="NameOfAppender" class="biz.paluch.logging.gelf.logback.GelfLogbackAppender">
<host>udp:10.123.33.41</host> <!-- graylog endpoint ip here -->
<port>12201</port> <!-- graylog endpoint port here -->
<version>1.1</version>
<facility>java-prod</facility>
<extractStackTrace>true</extractStackTrace>
<filterStackTrace>true</filterStackTrace>
<mdcProfiling>true</mdcProfiling>
<timestampPattern>yyyy-MM-dd HH:mm:ss,SSSS</timestampPattern>
<maximumMessageSize>8192</maximumMessageSize>
<additionalFields>Env=prod,Operation=INTG_PERF</additionalFields>
<!-- This are fields using MDC -->
<mdcFields>RequestTime,ResponseTime,ElapsedTime,ServiceType,ServiceIdentifier,Endpoint,Method,Status</mdcFields>
<dynamicMdcFields>mdc.*,(mdc|MDC)fields</dynamicMdcFields>
<includeFullMdc>false</includeFullMdc>
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>INFO</level>
</filter>
</appender>
<logger name="NameOfLogger" level="INFO">
<appender-ref ref="NameOfAppender" />
</logger>
java logging sample:
org.slf4j.Logger logger = LoggerFactory.getLogger("NameOfLogger");
MDC.put("RequestTime", new Date().toString());
MDC.put("ResponseTime", new Date().toString());
MDC.put("ElapsedTime", "232");
MDC.put("ServiceType", "BANK");
MDC.put("ServiceIdentifier", "XBANK");
MDC.put("Endpoint", "https://www..");
MDC.put("Method", "doSale");
MDC.put("Status", "SUCCEED");
logger.info("WS INTEGRATION PERF");
maven dependency:
<dependency>
<groupId>biz.paluch.logging</groupId>
<artifactId>logstash-gelf</artifactId>
<version>1.11.1</version>
</dependency>