I am using log4j2.xml for logging. 3 instances of my application are simultaneously writing to the debug.log file (with Rollingfile appender and CronTriggeringPolicy). But debug.log files are not getting generated sequentially and some sequences and logs are missing during multiple access. If I test with single access, it is working as expected.
<Appenders>
<RollingFile name="application" fileName="${log-path}/debug.log"
filePattern="${log-path}/debug.log.%i">
<PatternLayout>
<pattern>[%d{yyyy-MM-dd HH:mm:ss,SSS}]%5p [%t] (%F:%M:%L) - %m%n</pattern>
</PatternLayout>
<Policies>
<CronTriggeringPolicy schedule="0 0/2 * 1/1 * ? *" />
</Policies>
<DefaultRolloverStrategy fileindex="min" min="1" max="5" />
</RollingFile>
That's expected behavior. You could (artificially) get serialized lines, if you separate your log file by thread (the %t you are printing in your pattern).
The lines written with the same %t value, will be in the correct order.
On the other hand, if you are serious about logging properly, you might want to consider using LOG4J's MDC.
When 3 instances of your application are running simultaneously and logging in the same file, you can not expect sequential logs.
Reason is when 3 instances are running, it means 3 separate processes are running and these 3 separate processes are trying to write to a single file.
Writing to a file involves locking the file resource, write and then release the lock. So, sometimes, 1 process will get the lock and write it. Meanwhile, other 2 process will keep waiting and if there buffer will get filled, some logs will get missed and this cycle will keep on going for other processes of the application.
For such situations, you should use some log daemon like rsyslog using log4j2 SyslogAppender or easy solution is to log in separate files by each parallel instance of the application.
Related
I have the following RollingFileAppender setup:
<RollingFile name="RollingFile">
<FileName>${LOG_DIR}system.log</FileName>
<FilePattern>${LOG_DIR}system%d{yyyy-MM-dd HH.mm.ss}.log.zip</FilePattern>
<JsonLayout compact="true" eventEol="true" />
<Policies>
<OnStartupTriggeringPolicy />
<CronTriggeringPolicy schedule="0 0 0 * * ?"/>
</Policies>
<!-- roll over strategy, with unlimited files created -->
<DefaultRolloverStrategy fileIndex="nomax" />
</RollingFile>
It does nearly exactly what i want (new log file at midnight or on startup/restart), but unfortunately the timestamps that get assigned to the rolled over log files are sometimes simply incorrect. So for example:
system.log // current log, started at 2022-08-10 at midnight
system2022-08-09 00.00.00.log.zip // started at 2022-08-09 at midnight, so this is fine
system2022-08-08 00.00.00.log.zip // started at 2022-08-08 10.09 am, so this is not fine
system2022-08-08 10.04.57.log.zip // started at 2022-08-08 at 8.55 am, so this is not fine
As you can see system2022-08-08 00.00.00.log.zip is using a completely wrong timestamp and system2022-08-08 10.04.57.log.zip is using the timestamp when the log ended.
I'm not sure what i'm doing wrong here. Appreciate any suggestions on how to properly combine OnStartupTriggeringPolicy and CronTriggeringPolicy to achieve my goal here.
Seems like this behavior is unfortunately a bug in log4j2, and an old one. See my comment here: https://issues.apache.org/jira/browse/LOG4J2-2148
I also posted a solution for log4j2 2.18.0 in there, which will atleast does what i desire - there are still several things wrong in log4j2 itself though.
I inherited an anylogic project where parts of the model logic have been implemented in a java project, from which this model is also launched.
This project has multiple logger configurations since in most parts of the project log4j2 is used, in few parts it is slf4j which uses the logback backend.
In the anylogic UI/Browser-Window, you can view some Console in which some output is printed:
Interestingly there are no logs printed, that haven been recorded with the log4j-library.
Those beeing recorded with slf4j and logback however, are also shown in the Console.
The appender configurations read like this:
# log4j2 appender
appender.console.type = Console
appender.console.name = consoleLogger
appender.console.target = SYSTEM_OUT
appender.console.layout.type = PatternLayout
<!--- logback.xml appender --->
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<pattern>%highlight(%d{yyyy.MM.dd HH:mm:ss.SSS} [%thread] %-5level %logger{36}:%L - %msg%n)</pattern>
</encoder>
</appender>
Now for the question: What is difference between the two logging frameworks/appenders, that can cause the observed behavior?
It seems, the SYSTEM_OUT is reassigned after configuration.
If I set the Log4J2 Console Appender to follow
appender.console.follow = true
The log is also appended to the console in the AnyLogic UI.
Apparently logback follows by default.
I have in my test automation project problem with logging. I'm using log4j2 logger with FileAppender. The way I'm using it is:
Logger logger = (Logger) LogManager.getLogger(loggerName);
Appender appender = FileAppender.newBuilder()
.withAppend(false)
.withBufferedIo(true)
.withFileName(DIR_NAME + File.separator + loggerName + ".log")
.withIgnoreExceptions(false)
.withImmediateFlush(true)
.withLocking(false)
.withLayout(PatternLayout.newBuilder().withPattern("%d{HH:mm:ss.SSS} [%-5level] %msg%n").withCharset(Charset.forName("UTF-8")).build())
.withName(loggerName)
.build();
appender.start();
logger.addAppender(appender);
It works when I'm running single test. All data are visible in console, the file is created and test log is written in the file. Problem occurs in case of tests are running in parallel - in different threads.
In this case, two different loggers and file appenders are created. Log files from both file appenders are created too and logs from both tests are visible in console. Everything seems to be fine, but every time one of these log files is empty. The empty log belongs to test which started later.
I suspect problem with caching. The first file appender holds all cache for writing so the second one is not able to write. Am I right? What is the solution for this?
Thank you.
You should be able to achieve what you want without using programmatic configuration. There are many reasons not to configure log4j2 programmatically, but the best one, in my opinion, is that in doing so you would make your code dependent on aspects of log4j2 that are not part of the public API. This means that if the implementation of log4j2 changes your code has to change as well. This creates more work for you in the long run.
So, with that in mind I will provide a demo of how to set up log4j2 using an XML config file such that it will generate separate logs for each test. I am assuming, since it was not specified in your question, that your goal is to create a log for each method with a Test annotation and that each of these methods is executed in parallel.
First, here is my TestNG class:
package testpkg;
import java.lang.reflect.Method;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.apache.logging.log4j.ThreadContext;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.Test;
public class NewTest {
private static final Logger log = LogManager.getLogger();
#BeforeMethod
public void setThreadName(Method method){
ThreadContext.put("threadName", method.getName());
}
#Test
public void test1() {
log.info("This is the first test!");
log.warn("Something may be wrong, better take a look.");
}
#Test
public void test2() {
log.info("Here's the second test!");
log.error("There's a problem, better fix it");
}
}
As you can see here I have two Test methods and a BeforeMethod called setThreadName. The setThreadName method is, obviously, executed before each of the Test methods. It places a key named threadName into the log4j2 ThreadContext using the name of the method that is about to be run. This will be used as part of the log file name in the log4j2 config file.
Here is the log4j2.xml file:
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN">
<Appenders>
<Routing name="MyRoutingAppender">
<Routes pattern="$${ctx:threadName}">
<Route>
<File
fileName="logs/${ctx:threadName}.log"
name="appender-${ctx:threadName}"
append="false">
<PatternLayout>
<Pattern>[%date{ISO8601}][%-5level][%t] %m%n</Pattern>
</PatternLayout>
</File>
</Route>
</Routes>
</Routing>
<Console name="STDOUT" target="SYSTEM_OUT">
<PatternLayout pattern="[%date{ISO8601}][%-5level][%t] %m%n" />
</Console>
</Appenders>
<Loggers>
<Logger name="testpkg" level="TRACE" additivity="false">
<AppenderRef ref="STDOUT" />
<AppenderRef ref="MyRoutingAppender" />
</Logger>
<Root level="WARN">
<AppenderRef ref="STDOUT" />
</Root>
</Loggers>
</Configuration>
As you can see I've set up the config file to use a RoutingAppender to dynamically generate appenders at runtime based on the ThreadContext key threadName and that threadName is also used in the fileName attribute of the FileAppender.
Here is my testNG config file:
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
<suite name="My suite" parallel="methods" thread-count="5" verbose="1">
<test name="testpkg" >
<classes>
<class name="testpkg.NewTest" />
</classes>
</test>
</suite>
As you can see here I've set it up so that each Test method within my class is run in parallel.
When executed this results in the following console output:
[RemoteTestNG] detected TestNG version 6.14.3
[2018-05-04T21:54:54,703][INFO ][TestNG-test=testpkg-2] Here's the second test!
[2018-05-04T21:54:54,703][INFO ][TestNG-test=testpkg-1] This is the first test!
[2018-05-04T21:54:54,709][WARN ][TestNG-test=testpkg-1] Something may be wrong, better take a look.
[2018-05-04T21:54:54,709][ERROR][TestNG-test=testpkg-2] There's a problem, better fix it
===============================================
My suite
Total tests run: 2, Failures: 0, Skips: 0
===============================================
You can clearly see that the output of the two methods is interleaved, so we know that the methods are indeed running in parallel.
The execution of the test class also creates two log files as expected. They are named test1.log and test2.log
Here are their contents:
test1.log:
[2018-05-04T21:54:54,703][INFO ][TestNG-test=testpkg-1] This is the first test!
[2018-05-04T21:54:54,709][WARN ][TestNG-test=testpkg-1] Something may be wrong, better take a look.
test2.log:
[2018-05-04T21:54:54,703][INFO ][TestNG-test=testpkg-2] Here's the second test!
[2018-05-04T21:54:54,709][ERROR][TestNG-test=testpkg-2] There's a problem, better fix it
So we see here that as expected the logs from the first method went to test1.log and the logs from the second method went to test2.log
Enjoy!
Below is the sample used in log4j 1.x. I am not getting any example to convert the same in log4j2.
<appender name="CoalescingStatisticsAppender"
class="org.perf4j.log4j.AsyncCoalescingStatisticsAppender">
<!--
The TimeSlice option is used to determine the time window for which
all received StopWatch logs are aggregated to create a single
GroupedTimingStatistics log. Here we set it to 10 seconds, overriding
the default of 30000 ms
-->
<param name="TimeSlice" value="30000" />
<appender-ref ref="perf4jFileAppender" />
</appender>
The Appender won't work as is in Log4j 2. It would have to be rewritten.
You may be interested to know that Log4j 2 supports nanoTime timestamps in PatternLayout. This, in combination with the low overhead Async Loggers, allow you to use Log4j as a rough profiling tool.
Similar question but i'm using log4j2.
I need a way to disable All logs from freemarker, in their documentation they say we can do it by calling Logger.selectLoggerLibrary(Logger.LIBRARY_NONE) but they say
selectLoggerLibrary must be called early, before FreeMarker could log anything, or else it will not have (consistent) effect.
Where do I call this in a struts2 application? (I tried calling it in prepare() method in my action class but its not working.) or is there any other way to disable the logs?
Question is, why do you need to disable it like that?
You shouldn't need that, so I guess that's where the real problem lies. Is there some kind of malfunction? Because if there isn't, why not just set the freemarker logger category to be ignored in your logger configuration? That's the normal way of doing this, FreeMarker or not.
Anyway, in 2.3.22 (release expected in early 2015) you can use the -Dorg.freemarker.loggerLibrary=none where you start the JVM (that is, you set the org.freemarker.loggerLibrary system property). Otherwise, if you could call that method in a ServletContextListener that's certainly early enough.
Update:
Reacting to the comments... in most applications you will have 3rd party libraries that use various logging "frameworks", like SLF4J, commons-logging, JUL, Log4j, Log4j2. Thus you have to ensure that all these get redirected into the same logger library, which is certainly Log4j2 in your case. I suspect that wasn't properly done in your case, so now multiple logger libraries log to the console, each with its own configuration settings.
FreeMarker 2.3.x uses Log4j 1.x if it detects that org.apache.log4j.Logger is present. Other logger libraries that it also can detect and use (Log4j2 is not amongst them) have lower priority. (FreeMarker 2.4.x will always use SLF4J if it's present.) Thus, if you add org.apache.logging.log4j:log4j-1.2-api to your dependencies, then FM will use org.apache.log4j.Logger, and so log4j-1.2.-api will redirect the FM log messages to Log4j2. That worked for me, with this Log4j2 configuration:
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN">
<Appenders>
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout pattern="%d{HH:mm:ss.SSS} %-5level %logger{36} - %msg%n" />
</Console>
</Appenders>
<Loggers>
<Root level="debug">
<AppenderRef ref="Console" />
</Root>
<Logger name="freemarker" level="off">
<AppenderRef ref="Console" />
</Logger>
</Loggers>
</Configuration>
This came up as the first search result for "disable freemarker logging" which I searched for because I got double error logs for template errors, one from within the Freemarker library and one from my own code catching the same exception and logging it. The solution to this is simple and different from the answers already given: call setLogTemplateExceptions(false) on the Freemarker Configuration. In full:
Configuration configuration = new Configuration(Configuration.VERSION_2_3_31);
configuration.setLogTemplateExceptions(false);
The default behavior of logging the exception even though it propagates out of the Freemarker library is mentioned as a quirk on the Freemarker Logging documentation.
Use this statement:
freemarker.log.Logger.selectLoggerLibrary(freemarker.log.Logger.LIBRARY_NONE);