Log4j2 rolling over file with wrong timestamp - log4j2

I am using log4j2 -
org.apache.logging.log4j-log4j-api-2.14.1.jar
and for clean up policy, I am using CronbasedTriggerPolicy, after adding cron based trigger policy, whenever log4j2 rolls over the files, it replaces the date and time stamp with 0's
This is how my config looks like
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="INFO">
<Appenders>
<RollingFile fileName="data/appname-${date:yyyy-MM-dd_HH:mm:ss:SSS}.gz" filePattern="data/appname-%d{yyyy-MM-dd_HH:mm:ss.SSS}.%03i.gz" name="AppAppender">
<PatternLayout>
<pattern>%-6p %d{yyyy-MM-dd HH:mm:ss.SSS} %c{1}: %m%n
</pattern>
</PatternLayout>
<Policies>
<OnStartupTriggeringPolicy/>
<SizeBasedTriggeringPolicy size="10MB"/>
<CronTriggeringPolicy schedule="*/5 * * * *"/>
</Policies>
<DefaultRolloverStrategy>
<Delete basePath="data/" maxDepth="2">
<IfFileName glob="*appname-*.gz"/>
<IfLastModified age="1d"/>
</Delete>
</DefaultRolloverStrategy>
</RollingFile>
<Console name="console" target="SYSTEM_OUT">
<PatternLayout pattern="[%-5level] %d{yyyy-MM-dd HH:mm:ss.SSS} [%t] %c{1} - %msg%n"/>
</Console>
</Appenders>
<Loggers>
<Logger additivity="false" name="com.app.Test">
<AppenderRef ref="AppAppender"/>
</Logger>
</Loggers>
</Configuration>
When logs are being written to file file looks like this -
appname-2021-10-28_16:35:01:393.gz
After rolling over file changes to this-
appname-2021-10-28_00:00:00.000.001.gz
Why is log4j2 changing the timestamp to all 0s? and this started happening after I added the cron trigger policy, if I remove it, things work as expected. My final agenda is to have log4j2 delete all files that are older than 2 weeks. I am just playing with the config for a day right now.

Although, I am not still not clear on why cron trigger would change the timestamp to all 0s, I achieved what I wanted to achieve, my understanding of delete tag was not clear enough. I read through the log4j2 documentation and I realized I missed the part where they explained that delete works whenever roll over happens, for my needs I realized there was no need to have cron trigger policy in first place. The delete would have tried to run every time log4j2 rolls over the file.

Related

Log4j2 automatic reconfiguration when file size exceeds

In log4j2, I wanted to reconfigure automatically when file size exceeds. I have seen the attribute monitorinterval but it is defined for time based triggering policy. But I am looking for size based triggering policy.
I am going to assume that you want to know how to roll over a file when its size exceeds some threshold since reconfiguring automatically when a file exceeds some size is not something I have ever been asked before.
To roll over the file you would use the RollingFileAppender with the SizeBasedTriggeringPolicy. The configuration below will roll over the file once it hits 250 MB.
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="warn" name="MyApp" packages="">
<Appenders>
<RollingFile name="RollingFile" fileName="logs/app.log"
filePattern="logs/$${date:yyyy-MM}/app-%i.log">
<PatternLayout>
<Pattern>%d %p %c{1.} [%t] %m%n</Pattern>
</PatternLayout>
<Policies>
<SizeBasedTriggeringPolicy size="250 MB"/>
</Policies>
</RollingFile>
</Appenders>
<Loggers>
<Root level="error">
<AppenderRef ref="RollingFile"/>
</Root>
</Loggers>
</Configuration>

Names of log archives generated using Log4j2

Here i am trying to implement Log4j2 archival strategy using DefaultRollOverStrategy. Basically i want to have number of archive files being generated once size of actual file crosses 10Mb.
With current programmatic configuration i am able to do so, but problem here is archived file name is not reflecting accurate time part.
For example, over the period log archives are generated but time part(highlighted) is not changing
Sample-2019-12-02_17-09-37-1.log
Sample-2019-12-02_17-09-37-2.log
Sample-2019-12-02_17-09-37-3.log
Sample-2019-12-02_17-09-37-4.log
I am sharing XML representation of configuration generated grammatically, rather than XML based config.
<Configuration name="RollingBuilder" status="DEBUG">
<Appenders>
<RollingFile name="XYZRollingFileAppender" fileName="logs/Sample.log" filePattern="logs/Sample-${date:yyyy-MM-dd_HH-mm-ss}-%i.log" createOnDemand="false">
<PatternLayout pattern="%d{ISO8601} %5p %c{1} - %m%n"/>
<Policies>
<CronTriggeringPolicy schedule="0 0 0 * * ?"/>
<SizeBasedTriggeringPolicy size="10MB"/>
</Policies>
<DefaultRolloverStrategy max="100"/>
</RollingFile>
</Appenders>
<Loggers>
<Root level="ERROR">
<AppenderRef ref="XYZRollingFileAppender"/>
</Root>
<Logger name="com.sample.test" level="ERROR">
<AppenderRef ref="XYZRollingFileAppender"/>
</Logger>
</Loggers>
</Configuration>```

Log4j2 daily log files

I want my log files created by log4j2 to have the date pattern in their file name, including the current active file. That is, if todays date is 2016-12-15, I want the current log-file to be lager-2016-12-15.log. When the date changes, I want a new file to be created named lager-2016-12-16.log.
With RollingFileAppender I am not able to get the current active log file to have date pattern in the filename. My Log4j2.xml:
<?xml version="1.0" encoding="UTF-8"?>
<Configuration>
<Properties>
<Property name="layoutPattern">%d{ISO8601} [%t] %-5p [%X{REQUEST_ID}] [%X{CLIENT_ID}] [%X{USER_ID}] %c- %m%n</Property>
<Property name="logDir">${sys:catalina.home}/logs/</Property>
<Property name="fileName">${logDir}lager-${date:yyyy-MM-dd-HHmm}.log</Property>
<Property name="filePattern">${logDir}lager-%d{yyyy-MM-dd-HHmm}.log</Property>
</Properties>
<Appenders>
<RollingFile name="LAGER" append="true"
fileName="${fileName}"
filePattern="${filePattern}">
<PatternLayout pattern="${layoutPattern}" charset="UTF-8"/>
<Policies>
<TimeBasedTriggeringPolicy interval="1" modulate="true"/>
</Policies>
</RollingFile>
</Appenders>
<loggers>
<Logger name="org.apache.catalina.core.ContainerBase.[Catalina].[localhost]" level="WARN" additivity="false">
<AppenderRef ref="LAGER" />
<AppenderRef ref="CONSOLE"/>
</Logger>
<Root level="INFO">
<AppenderRef ref="LAGER"/>
<AppenderRef ref="CONSOLE"/>
</Root>
</loggers>
</Configuration>
With this config the copying when it rolls over is messed up. Removing the date pattern from the fileName property fixes this, but the the current file does not have the date in its name.
I am running this on a tomee 7.0.1.
There may already be an outstanding feature request for this. Would this match your requirements? https://issues.apache.org/jira/browse/LOG4J2-1101
If so, please comment on that JIRA ticket. If you're able to contribute a patch (ideally with unit test) it is likely to get resolved quickly.

Log4j2 RollingFile appenders clashing

Log4j2 RollingFile appenders clashing
Below is a simplified debug version of our log4j2 configuration file (We rollover nightly not every minute!).This configuration, instead of it creating a Rollover file each minute (as per theTimeBasedTriggeringPolicy) will create one rollover file (non-tarred), containing JSON formatted logging, which will be overwritten every 20KB(although it will end up being slightly greater than 20KB (See Screenshot).We also get the following errors (abbreviated with "..."):-
2016-10-07 08:47:34,433 default-workqueue-4 ERROR Unable to copy file /.../logs/logFile-2016-10-07-08:47:11.log to /.../logs/logFile-2016-10-07-08:47:11.log: java.nio.file.NoSuchFileException /.../logs/logFile-2016-10-07-08:47:11.log
If we switch the order of the timeBasedRollingFileJsonLayout appender and the sizeBasedRollingFilePatternLayoutWithZippedArchive appender then no rollover occurs at all.
If we remove the sizeBasedRollingFilePatternLayoutWithZippedArchive appender then the timeBasedRollingFileJsonLayout appender works as expected.
We have the two different appenders for different environments, where the logs may or may not be hooked up to ELK.In our real log4j2 config file we use properties to select the appropriate appender for the environment.I have removed the properties from this file for clarity and to rule them out as a possible cause of the problem.
We are using log4j 2.6.2.
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN">
<Appenders>
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout pattern="%d{HH:mm:ss.SSS} [%t] %logger{36} - %msg %n" />
</Console>
<RollingFile name="timeBasedRollingFileJsonLayout" append="true" fileName="logs/logFile.log" filePattern="logs/logFile-%d{yyyy-MM-dd-HH:mm}.log">
<JSONLayout properties="true" compact="true" eventEol="true" />
<Policies>
<TimeBasedTriggeringPolicy />
</Policies>
</RollingFile>
<RollingFile name="sizeBasedRollingFilePatternLayoutWithZippedArchive" append="true" fileName="logs/logFile.log" filePattern="logs/logFile-%d{yyyy-MM-dd}-%i.log.gz">
<PatternLayout pattern="%d{HH:mm:ss.SSS} [%t] %logger{36} - %msg %n" />
<Policies>
<SizeBasedTriggeringPolicy size="20KB" />
</Policies>
</RollingFile>
</Appenders>
<Loggers>
<logger name="logger.one" level="info" additivity="false">
<AppenderRef ref="timeBasedRollingFileJsonLayout" />
</logger>
<Logger name="logger.two" level="info" additivity="false">
<AppenderRef ref="timeBasedRollingFileJsonLayout" />
</Logger>
<Logger name="logger.three" level="info" additivity="false">
<AppenderRef ref="timeBasedRollingFileJsonLayout" />
</Logger>
<Root level="info">
<AppenderRef ref="timeBasedRollingFileJsonLayout" level="all" />
</Root>
</Loggers>
</Configuration>
I am at a loss to understand why you think this should work. You have two appenders trying to write to the same file trying to rollover based on different criteria and rollover to files with different names. It is no surprise that you are getting the file is in use error since two things have it open at once.

log4j2 to chainsaw hello world not working... what am I doing wrong?

I'm trying to stream a basic hello world log message to show up in chainsaw from log4j2. I don't care if it uses "Zeroconf" or not, I just want something that works. I know that my test program is logging messages since they show on the console, and I know it's finding my config file because I can change the format of the messages that get printed in the console, but that's all I know.
My config file (containing various failed guesses):
<?xml version="1.0" encoding="UTF-8"?>
<configuration advertiser="org.apache.logging.log4j.core.net.MulticastDNSAdvertiser">
<appenders>
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout pattern="%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %m%n"/>
</Console>
<File name="testFile" fileName="logs/test.log" bufferedIO="false" advertiseURI="file://localhost/home/matt/code/ade/logs/test.log" advertise="true">
<XMLLayout />
</File>
<SocketAppender name="socketTest" host="localhost" immediateFlush="true" port="4560" protocol="TCP" advertiseURI="http://localhost" advertise="true">
<XMLLayout />
</SocketAppender>
</appenders>
<loggers>
<root level="TRACE">
<appender-ref ref="Console"/>
<appender-ref ref="testFile"/>
<appender-ref ref="socketTest"/>
</root>
</loggers>
</configuration>
I've tried various combinations of: including jmdns.jar on the classpath, restarting chainsaw at various points, and getting frustrated, but nothing has helped.
Any ideas?
Edit: I figured out why it couldn't read the log files I was saving to disk, (I hadn't been using XMLLayout) so I've updated the question to reflect that I now only need to get streaming working.
The 'advertiser' uses the log4j2 plugin mechanism, so you must provide the 'name' defined on the Advertiser in the configuration - not the fully-qualified class name.
The log4j2 advertisement mechanism currently supports advertisement of FileAppenders and SocketAppenders. However, Chainsaw only supports discovery of log4j2 FileAppenders which are advertised with a PatternLayout. XMLLayout support will show up in the near future.
The latest developer snapshot of Chainsaw must be used in order to leverage log4j2's advertiser mechanism. Chainsaw tarball and DMG available at: http://people.apache.org/~sdeboy/
Chainsaw will discover the advertised fileappender configuration and parse (and tail if you get the latest Chainsaw developer build) the log file - no Chainsaw configuration required.
Note, you need to use a PatternLayout, and JMDNS must be on the classpath of the application using this appender configuration.
Here is an example Log4j2 -appender- configuration that will advertise a fileappender configuration:
<?xml version="1.0" encoding="UTF-8"?>
<configuration advertiser="multicastdns">
<appenders>
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout pattern="%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %m%n"/>
</Console>
<File name="testFile" fileName="logs/test.log" bufferedIO="false" advertiseURI="file:///localhost/home/matt/code/ade/logs/test.log" advertise="true">
<PatternLayout pattern="%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %m%n"/>
</File>
</appenders>
<loggers>
<root level="TRACE">
<appender-ref ref="Console"/>
<appender-ref ref="testFile"/>
</root>
</loggers>
</configuration>
Once you have started your app that is using the appender
configuration, open the 'Zeroconf' tab in Chainsaw.
You should see a row with your appender's name (assuming you added
jmdns to the classpath for the app using the fileappender
configuration).
You can click 'Autoconnect' if you'd like to always start Chainsaw
with this configuration if it is available.
Next, double-click
on the row with the appender name and Chainsaw will start parsing and tailing your log file.
The advertised URL provided in your file appender
configuration must be accessible network-wise to Chainsaw (looks like you are working
locally with Chainsaw and your fileappender, so file:/// paths will
work fine - note the three slashes).
Chainsaw will work best if there are delimiters around each field - square brackets, dashes, etc. as long as that character won't be present in the field you are delimiting.

Resources