Change Log4J2 appender encoding dynamically - log4j2

Log4J2 2.17.2 (or newer)
I have a "console" application generating a report from a product. The product data are in UTF-8 format.
When I run the application in windows cmd shell (CMD), the following (simplified) appender is working fine,
and all the encoding looks fine. So also german Umlaute are converted properly without any additinally programming.
<Console name="STDOUT" target="SYSTEM_OUT">
<PatternLayout pattern="%m%n" charset="CP850" />
So CMD is using the CP850 encoding.
I also can run my application from within a product, so I get as runtime environment an UTF-8 encoded outputstream.
Testing with
<Console name="STDOUT" target="SYSTEM_OUT">
<PatternLayout pattern="%m%n" charset="UTF-8" />
is working fine from this environment.
My log4j2 configuration is part of the application, read via class path. More or less similar to log4j2.xml
So I'm looking for a dynamic way to switch. I can launch my program e.g with a swith/flag if necessary.
Any hints?
Uwe
EDIT after Piotr's answer:
The answer from Piotr (Thanks) brought me to the following solution,
using the variable substitution and the ThreadContext.
main
boolean cmdMode = ...
String charSet = cmdMode ? "CP850" : "UTF-8";
ThreadContext.put("CHARSET", charSet);
In the log4j xml, I used:
<Console name="STDOUT" target="SYSTEM_OUT">
<PatternLayout pattern="%m%n" charset="${ctx:CHARSET}"/>
Works fine.

The console appender can create a pattern layout with the correct encoding if none is provided. So if %m%n is enough for you, just omit the layout configuration:
<Console name="STDOUT" target="SYSTEM_OUT"/>
If you need a different pattern we incur in a limitation of the plugin builder (cf. LOG4J2-3372): the layout is created before the console appender, so the appender can not modify the encoding used by the layout. You can however apply the same logic used by Log4j2 in your configuration:
<Properties>
<Property name="sun.stdout.encoding" value="${sys:file.encoding:-UTF-8}"/>
</Properties>
<Appenders>
<Console name="STDOUT" target="SYSTEM_OUT">
<PatternLayout pattern="%d [%t] %p - %m%n" encoding="${sys:sun.stdout.encoding"}"/>
</Console>
</Appenders>
This should work because:
Java applications that run from the console, have the system property sun.stdout.encoding correctly set,
applications that don't run in the console will have file.encoding set to the default system encoding.
in the unlikely event that file.encoding is not set, UTF-8 will be used.

Related

Log4j2 rolling over file with wrong timestamp

I am using log4j2 -
org.apache.logging.log4j-log4j-api-2.14.1.jar
and for clean up policy, I am using CronbasedTriggerPolicy, after adding cron based trigger policy, whenever log4j2 rolls over the files, it replaces the date and time stamp with 0's
This is how my config looks like
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="INFO">
<Appenders>
<RollingFile fileName="data/appname-${date:yyyy-MM-dd_HH:mm:ss:SSS}.gz" filePattern="data/appname-%d{yyyy-MM-dd_HH:mm:ss.SSS}.%03i.gz" name="AppAppender">
<PatternLayout>
<pattern>%-6p %d{yyyy-MM-dd HH:mm:ss.SSS} %c{1}: %m%n
</pattern>
</PatternLayout>
<Policies>
<OnStartupTriggeringPolicy/>
<SizeBasedTriggeringPolicy size="10MB"/>
<CronTriggeringPolicy schedule="*/5 * * * *"/>
</Policies>
<DefaultRolloverStrategy>
<Delete basePath="data/" maxDepth="2">
<IfFileName glob="*appname-*.gz"/>
<IfLastModified age="1d"/>
</Delete>
</DefaultRolloverStrategy>
</RollingFile>
<Console name="console" target="SYSTEM_OUT">
<PatternLayout pattern="[%-5level] %d{yyyy-MM-dd HH:mm:ss.SSS} [%t] %c{1} - %msg%n"/>
</Console>
</Appenders>
<Loggers>
<Logger additivity="false" name="com.app.Test">
<AppenderRef ref="AppAppender"/>
</Logger>
</Loggers>
</Configuration>
When logs are being written to file file looks like this -
appname-2021-10-28_16:35:01:393.gz
After rolling over file changes to this-
appname-2021-10-28_00:00:00.000.001.gz
Why is log4j2 changing the timestamp to all 0s? and this started happening after I added the cron trigger policy, if I remove it, things work as expected. My final agenda is to have log4j2 delete all files that are older than 2 weeks. I am just playing with the config for a day right now.
Although, I am not still not clear on why cron trigger would change the timestamp to all 0s, I achieved what I wanted to achieve, my understanding of delete tag was not clear enough. I read through the log4j2 documentation and I realized I missed the part where they explained that delete works whenever roll over happens, for my needs I realized there was no need to have cron trigger policy in first place. The delete would have tried to run every time log4j2 rolls over the file.

Including classpath resource in external log4j configuration

I use external log4j configuration and load it in servlet context listener using org.apache.logging.log4j.core.config.Configurator
I would like to reference some application properties (like application version) that are generated on build time.
Here is what I would like to do:
/external-folder/log4j.xml
<?xml version="1.0" encoding="UTF-8"?>
<Configuration xmlns:xi="http://www.w3.org/2001/XInclude"
monitorInterval="60">
<xi:include href="log4j2-properties.xml" />
<Appenders>
<Console name="stdout" target="SYSTEM_OUT">
<PatternLayout pattern="%d %5p %t [${app-version}] %m (%F:%L) %n" />
</Console>
</Appenders>
<!-- ... -->
</Configuration>
log4j2-properties.xml
is supposed to be a classpath resource, #version# is filtered by gradle.
<?xml version="1.0" encoding="UTF-8"?>
<Properties>
<Property name="app-version">#version#</Property>
</Properties>
However this approach doesn't work. I can xinclude only resource from the same folder as top level log4j.xml
How can I solve my problem?
(I know how to do this using logback, but I would like to upgrade to log4j2)

How to configure port in Apache Chainsaw with Log4j2

I got log4j2.xml that generates log files. Here is the config:
<Configuration status="INFO" advertiser="multicastdns">
<Properties>
<Property name="layout">%d | %-5p | [%t] | %c{2} | %M | %m%n
</Property>
</Properties>
<Appenders>
<RollingFile name="LogFile" fileName="${sys:user.home}/logs/webapp.log"
filePattern="${sys:user.home}/logs/webapp-%d{yyyy-MM-dd}-%i.log"
bufferedIO="false" advertiseURI="file:///C://users/bilguuna/logs/webapp.log"
advertise="true">
<PatternLayout pattern="${layout}" />
<Policies>
<TimeBasedTriggeringPolicy />
<SizeBasedTriggeringPolicy size="1 MB" />
</Policies>
<DefaultRolloverStrategy max="10" />
</RollingFile>
</Appenders>
<Loggers>
<Root level="INFO">
<AppenderRef ref="LogFile" level="INFO" />
</Root>
</Loggers>
</Configuration>
I'm able to see a log file using
file:///C://users/bilguuna/logs/webapp.log
on the browser.
A problem is that on the ZeroConf tab, my appender "LogFile" is appeared with Connection status "Connected". But when I double click on that row, it's just changed to disconnected/connected. When I check Chainsaw-log. It said that Connection refused as image below:
I guess it's because of that Chainsaw chooses 4555 as a default port which is not open on my machine. So what port should choose and how to set it in my log4j2.xml file?
Thanks
UPDATE: After I used the developer snapshot version as #Scott suggested, it seems like "connection refused" problem went away. But still I can't see the logs. When I click on entry at ZeroConf tab, it still changes the Connection Status to Connected/Not Connected. Here is the log from chainsaw-log tab:
Again, was I supposed to see the actual logs when I double click on entry at ZeroConf tab?
Update: I got following exception on my console:
WARNING: SocketListener(WS00943.local.).run() exception
java.io.IOException: DNSIncoming corrupted message
at javax.jmdns.impl.DNSIncoming.<init>(DNSIncoming.java:239)
at javax.jmdns.impl.SocketListener.run(SocketListener.java:50)
Caused by: java.lang.IllegalStateException: Can't overwrite cause with java.lang.IllegalStateException: Illegal access: this web application instance has been stopped already. Could not load [javax.jmdns.impl.constants.DNSResultCode]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.
at java.lang.Throwable.initCause(Unknown Source)
at org.apache.catalina.loader.WebappClassLoaderBase.checkStateForClassLoading(WebappClassLoaderBase.java:1344)
at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1206)
at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1167)
at javax.jmdns.impl.DNSIncoming.readAnswer(DNSIncoming.java:342)
at javax.jmdns.impl.DNSIncoming.<init>(DNSIncoming.java:229)
... 1 more
Caused by: java.lang.ClassNotFoundException
at org.apache.catalina.loader.WebappClassLoaderBase.checkStateForClassLoading(WebappClassLoaderBase.java:1343)
... 5 more
The problem is you need to use the latest developer snapshot of Chainsaw to work with log4j2. The webstart-runnable version works only with log4j1.
Developer snapshot is available at: http://home.apache.org/~sdeboy/

Log4j2 not logging statements from WAR and Non-EJB jars in an EAR

I have an unpacked EAR deployed at $JBOSS_HOME/standalone/deployments in Wildfly-8.2.0.Final AS. It contains unpacked WAR and application jar files.
HelloWorldEar-0.0.1-SNAPSHOT.ear
-HelloWorldServlet-0.0.1-SNAPSHOT.war
-lib
-META-INF
-aspectjrt-1.8.6.jar
-HelloWorldAnnotation-0.0.1-SNAPSHOT.jar
-HelloWorldAspect-0.0.1-SNAPSHOT.jar
-HelloWorldCommonLib-0.0.1-SNAPSHOT.jar
-HelloWorldEJB-0.0.1-SNAPSHOT.jar
-log4j2.xml
lib folder of EAR contains log4j2 related APIs for providing logging feature.
lib
-commons-logging-1.2.jar
-HelloWorldLog4jPlugin-0.0.1-SNAPSHOT.jar (Jar containing custom log4j2 plugins)
-log4j-api-2.4.1.jar
-log4j-core-2.4.1.jar
-log4j-jcl-2.4.1.jar
-log4j-web-2.4.1.jar
I am initializing log4j2 configuration by initializing log4j2.xml using in a startup singleton bean in HelloWorldEJB-0.0.1-SNAPSHOT.jar.
String path = System.getProperty("jboss.home.dir")
+ "/standalone/deployments/HelloWorldEar-0.0.1-SNAPSHOT.ear/log4j2.xml";
ConfigurationSource source;
File configFile = new File(path);
try {
source = new ConfigurationSource(new FileInputStream(configFile), configFile);
Configurator.initialize(null, source);
} catch (FileNotFoundException e) {
e.printStackTrace();
}
log4j2.xml content
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="trace">
<Appenders>
<RollingFile name="SERVER_FILE" fileName="${sys:LOGS}/sample.log" filePattern="${sys:LOGS}/sample.log.%i" append="true">
<PatternLayout>
<pattern>%d %-5p [%c{1}] [EventId: %e] [%t] %m%n</pattern>
</PatternLayout>
<Policies>
<SizeBasedTriggeringPolicy size="500 KB" />
</Policies>
<DefaultRolloverStrategy max="5"/>
</RollingFile>
<Console name="CONSOLE" target="SYSTEM_OUT">
<PatternLayout>
<pattern>%d{ABSOLUTE} %-5p [%c{1}] %m%n</pattern>
</PatternLayout>
</Console>
</Appenders>
<Loggers>
<Logger name="com.test.prototype.log4j" level="debug">
<AppenderRef ref="SERVER_FILE"/>
</Logger>
<Root level="debug" additivity="false">
<AppenderRef ref="CONSOLE"/>
</Root>
</Loggers>
</Configuration>
All the sub-packages and classes are defined under the package com.test.prototype.log4j.
Log statements added in the classes of HelloWorldEJB-0.0.1-SNAPSHOT.jar gets logged in the log file.
My issue is that 1. log statements from other application jars and WAR are not getting logged into the log file.
2. Logging not working through commons-logging API though required system parameters for log4j2 is added.
System.setProperty("org.apache.commons.logging.Log",
"org.apache.commons.logging.impl.Log4JLogger");
I have tried adding the class-path in the application jars and below listener in the web.xml of WAR file but nothing worked.
org.apache.logging.log4j.web.Log4jServletContextListener
I need to have log4j2.xml under EAR and it should server for the all the application jars and WARs added in the EAR. Could anyone please help me about how to resolve this issue.
Have you tried specifying the log4j2.xml config file location with system property -Dlog4j.configurationFile=path/to/log4j2.xml?
I suspect that the way you're initializing log4j now is not visible from other WARs.
If this fails, please raise this issue on the log4j2 Jira issue tracker. https://issues.apache.org/jira/browse/LOG4J2

log4j2 to chainsaw hello world not working... what am I doing wrong?

I'm trying to stream a basic hello world log message to show up in chainsaw from log4j2. I don't care if it uses "Zeroconf" or not, I just want something that works. I know that my test program is logging messages since they show on the console, and I know it's finding my config file because I can change the format of the messages that get printed in the console, but that's all I know.
My config file (containing various failed guesses):
<?xml version="1.0" encoding="UTF-8"?>
<configuration advertiser="org.apache.logging.log4j.core.net.MulticastDNSAdvertiser">
<appenders>
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout pattern="%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %m%n"/>
</Console>
<File name="testFile" fileName="logs/test.log" bufferedIO="false" advertiseURI="file://localhost/home/matt/code/ade/logs/test.log" advertise="true">
<XMLLayout />
</File>
<SocketAppender name="socketTest" host="localhost" immediateFlush="true" port="4560" protocol="TCP" advertiseURI="http://localhost" advertise="true">
<XMLLayout />
</SocketAppender>
</appenders>
<loggers>
<root level="TRACE">
<appender-ref ref="Console"/>
<appender-ref ref="testFile"/>
<appender-ref ref="socketTest"/>
</root>
</loggers>
</configuration>
I've tried various combinations of: including jmdns.jar on the classpath, restarting chainsaw at various points, and getting frustrated, but nothing has helped.
Any ideas?
Edit: I figured out why it couldn't read the log files I was saving to disk, (I hadn't been using XMLLayout) so I've updated the question to reflect that I now only need to get streaming working.
The 'advertiser' uses the log4j2 plugin mechanism, so you must provide the 'name' defined on the Advertiser in the configuration - not the fully-qualified class name.
The log4j2 advertisement mechanism currently supports advertisement of FileAppenders and SocketAppenders. However, Chainsaw only supports discovery of log4j2 FileAppenders which are advertised with a PatternLayout. XMLLayout support will show up in the near future.
The latest developer snapshot of Chainsaw must be used in order to leverage log4j2's advertiser mechanism. Chainsaw tarball and DMG available at: http://people.apache.org/~sdeboy/
Chainsaw will discover the advertised fileappender configuration and parse (and tail if you get the latest Chainsaw developer build) the log file - no Chainsaw configuration required.
Note, you need to use a PatternLayout, and JMDNS must be on the classpath of the application using this appender configuration.
Here is an example Log4j2 -appender- configuration that will advertise a fileappender configuration:
<?xml version="1.0" encoding="UTF-8"?>
<configuration advertiser="multicastdns">
<appenders>
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout pattern="%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %m%n"/>
</Console>
<File name="testFile" fileName="logs/test.log" bufferedIO="false" advertiseURI="file:///localhost/home/matt/code/ade/logs/test.log" advertise="true">
<PatternLayout pattern="%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %m%n"/>
</File>
</appenders>
<loggers>
<root level="TRACE">
<appender-ref ref="Console"/>
<appender-ref ref="testFile"/>
</root>
</loggers>
</configuration>
Once you have started your app that is using the appender
configuration, open the 'Zeroconf' tab in Chainsaw.
You should see a row with your appender's name (assuming you added
jmdns to the classpath for the app using the fileappender
configuration).
You can click 'Autoconnect' if you'd like to always start Chainsaw
with this configuration if it is available.
Next, double-click
on the row with the appender name and Chainsaw will start parsing and tailing your log file.
The advertised URL provided in your file appender
configuration must be accessible network-wise to Chainsaw (looks like you are working
locally with Chainsaw and your fileappender, so file:/// paths will
work fine - note the three slashes).
Chainsaw will work best if there are delimiters around each field - square brackets, dashes, etc. as long as that character won't be present in the field you are delimiting.

Resources