Tomcat application server logs are not printed in a rolling file other than catalina.log using log4j2.properties - log4j2

I have recently migrated application from log4j1.x to log4j2.17.1. Application logs are printed well in console as well as in Rolling file when start the tomcat server. I could see Tomcat server start up logs are all printing in catalina.log but the same server startup logs and any other server internal logs are not printed in rolling file. Rolling file prints only those logs are coming from class files. Could you anyone of you suggest what can be done to print all application server start up logs like some logs gets printed in console before server start or connection refused logs etc. also get printed into Rolling file? Same things was working well when I had log4j1.x version.It means when I had log4j1.x server start up logs are printed well in catalina as well as in a DailyRollingFileAppender but Now issue is only with log4j2, it means server start up logs are printed in catalain only not in a rolling file which I want. Please suggest.
Here is my log4j2.properties
rootLogger.level = info
property.filename =D:/log/TEST.log
appenders = R, console
appender.console.type = Console
appender.console.name = STDOUT
appender.console.layout.type = PatternLayout
appender.console.layout.pattern = %d %5p [%t] (%F:%L) - %m%n
appender.R.type = RollingFile
appender.R.name = File
appender.R.fileName = ${filename}
appender.R.filePattern = ${filename}.%d{yyyy-MM-dd}
appender.R.layout.type = PatternLayout
appender.R.layout.pattern = %d{yyyy-MM-dd HH:mm:ss} %-5p [%15.15t]: %c{1}:%L - %m%n
appender.R.policies.type = Policies
appender.R.policies.time.type = TimeBasedTriggeringPolicy
appender.R.policies.time.interval = 1
rootLogger.appenderRef.console.ref = STDOUT
rootLogger.appenderRef.R.ref = File

Related

Grails 2.5.4 does not log anything when running with run-app after upgrading log4j

I am trying to upgrade the version of log4j in my grails 2.5.4 application from 1.2.17 to 2.17.2. I am able to build and deploy the war and have all the logs work as expected, but the app does not log anything when running with grails run-app locally.
To upgrade the version, I made the following changes to BuildConfig.groovy:
inherits("global") {
excludes "grails-plugin-log4j", "log4j"
}
dependencies {
runtime 'org.apache.logging.log4j:log4j-core:2.17.2'
runtime 'org.apache.logging.log4j:log4j-api:2.17.2'
runtime 'org.apache.logging.log4j:log4j-jcl:2.17.2'
runtime 'org.apache.logging.log4j:log4j-slf4j-impl:2.17.2'
runtime 'org.apache.logging.log4j:log4j-1.2-api:2.17.2'
log4j2.properties:
rootLogger.level = INFO
rootLogger.additivity = false
rootLogger.appenderRefs = stdout
rootLogger.appenderRef.stdout.ref = STDOUT
appender.console.type = Console
appender.console.name = STDOUT
appender.console.layout.type = PatternLayout
appender.console.layout.pattern = %d [%t] %-5p %c{2} %x %m %n
If I remove the log4j-1.2-api dependency, the logs work locally with run-app but the app is unable to start when deploying the war. The primary error log is:
Caused by: java.lang.ClassNotFoundException: org.apache.log4j.Level
Using Log4j 2.17.1 appears to have resolved this.

Can't add a rolling file appender with Log4j2.properties

I have a java application running in a docker container. I can't change the program but I can change the log4j2.properties file. I want to add a rolling file appender but when I do it I get this error:
ERROR An exception occurred processing Appender file_appender java.security.AccessControlException: access denied ("java.io.FilePermission" "shared_logs" "read")
Then I gave all permissions to my shared_logs folder
chmod 777 shared_logs
But the error is still there.
This is how I added the the appender:
appender.fa.type = RollingFile
appender.fa.name = file_appender
appender.fa.fileName = shared_logs/elastic.log
appender.fa.filePattern = shared_logs/elastic-%d{yyyy-dd-MM}-%i.log.gz
appender.fa.layout.type = PatternLayout
appender.fa.layout.pattern = [%-5level] %d{yyyy-MM-dd HH:mm:ss.SSS} [%t] %c{1} - %msg%n
appender.fa.policies.type = Policies
appender.fa.policies.time.type = TimeBasedTriggeringPolicy
appender.fa.policies.size.type = SizeBasedTriggeringPolicy
appender.fa.policies.size.size = 500
appender.fa.strategy.type = DefaultRolloverStrategy
appender.fa.strategy.max = 5
"fa" stands for fileappender
Well, you obviously still have a permissions problem. Your fileName is a relative file so where it is being created will be based on the currently directory of your app. That would typically be either the directory your app is in or the root directory. If it is the root directory and you are expecting it to be in your app this would explain the problem. If you add -Dlog4j2.debug to the startup options you would see where Log4j is trying to create the file.
Writing to log files in Docker Containers is also not usually considered a best practice. If you haven't already, I would suggest taking a look at Logging in the Cloud on the Log4j web site.

Docker, Tomee, Logging, STDOUT, AWS

Let me start by saying I am not a Tomee/TomCat expert.
I have an application (.war) running in a Tomee based Docker container on ECS/Fargate on AWS. I am trying to get Tomee to send all logs to STDOUT so that logs from the application will be sent to CloudWatch in AWS. I have tried the suggestions/answers in this question but I am still not seeing application logs even when testing locally:
docker logs -f myapp
Here is the logging.properties file with the added java.util.logging.ConsoleHandler:
handlers = 1catalina.org.apache.juli.AsyncFileHandler, 2localhost.org.apache.juli.AsyncFileHandler, 3manager.org.apache.juli.AsyncFileHandler, 4host-manager.org.apache.juli.AsyncFileHandler, org.apache.tomee.jul.formatter.AsyncConsoleHandler
.handlers = 1catalina.org.apache.juli.AsyncFileHandler, org.apache.tomee.jul.formatter.AsyncConsoleHandler, java.util.logging.ConsoleHandler
############################################################
# Handler specific properties.
# Describes specific configuration info for Handlers.
############################################################
1catalina.org.apache.juli.AsyncFileHandler.level = FINE
1catalina.org.apache.juli.AsyncFileHandler.directory = ${catalina.base}/logs
1catalina.org.apache.juli.AsyncFileHandler.prefix = catalina.
1catalina.org.apache.juli.AsyncFileHandler.maxDays = 90
1catalina.org.apache.juli.AsyncFileHandler.encoding = UTF-8
2localhost.org.apache.juli.AsyncFileHandler.level = FINE
2localhost.org.apache.juli.AsyncFileHandler.directory = ${catalina.base}/logs
2localhost.org.apache.juli.AsyncFileHandler.prefix = localhost.
2localhost.org.apache.juli.AsyncFileHandler.maxDays = 90
2localhost.org.apache.juli.AsyncFileHandler.encoding = UTF-8
3manager.org.apache.juli.AsyncFileHandler.level = FINE
3manager.org.apache.juli.AsyncFileHandler.directory = ${catalina.base}/logs
3manager.org.apache.juli.AsyncFileHandler.prefix = manager.
3manager.org.apache.juli.AsyncFileHandler.maxDays = 90
3manager.org.apache.juli.AsyncFileHandler.encoding = UTF-8
4host-manager.org.apache.juli.AsyncFileHandler.level = FINE
4host-manager.org.apache.juli.AsyncFileHandler.directory = ${catalina.base}/logs
4host-manager.org.apache.juli.AsyncFileHandler.prefix = host-manager.
4host-manager.org.apache.juli.AsyncFileHandler.maxDays = 90
4host-manager.org.apache.juli.AsyncFileHandler.encoding = UTF-8
org.apache.tomee.jul.formatter.AsyncConsoleHandler.level = FINE
org.apache.tomee.jul.formatter.AsyncConsoleHandler.formatter = org.apache.juli.OneLineFormatter
org.apache.tomee.jul.formatter.AsyncConsoleHandler.encoding = UTF-8
############################################################
# Facility specific properties.
# Provides extra control for each logger.
############################################################
org.apache.catalina.core.ContainerBase.[Catalina].[localhost].level = INFO
org.apache.catalina.core.ContainerBase.[Catalina].[localhost].handlers = 2localhost.org.apache.juli.AsyncFileHandler, java.util.logging.ConsoleHandler
org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/manager].level = INFO
org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/manager].handlers = 3manager.org.apache.juli.AsyncFileHandler, java.util.logging.ConsoleHandler
org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/host-manager].level = INFO
org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/host-manager].handlers = 4host-manager.org.apache.juli.AsyncFileHandler, java.util.logging.ConsoleHandler
I have added an environment variable as suggested in /usr/local/tomee/bin/:
#!/bin/bash
CATALINA_OUT=/dev/stdout
All of the logs sent to STDOUT are automatically sent to AWS Cloudwatch (I can see the information from the Catalina logs there) but the access/error logs are not.
Have I missed something here? Or is there an easier way to make sure the application's logs are seen in CloudWatch?
After much research, I have found that using the following statements in server.xml will send the logs to STDOUT using AccessLogValve:
<Valve className="org.apache.catalina.valves.AccessLogValve" directory="/dev/stdout"
prefix="" suffix="" rotatable="false"
pattern="%h %l %u %t "%r" %s %b" />
OR
<Valve className="[...].AccessLogValve" directory="/dev"
prefix="stdout" suffix="" fileDateFormat=""
pattern="%h %l %u %t "%r" %s %b"
/>
Dependent upon the version of Tomcat/Tomee both of these work well without the need for soft linking the actual logs to /dev/stdout

How to generate log files having datetime stamp at each run using log4j2 Properties file

I want to create log files for my Testing framework having datetime stamp for each run of test suites. I trying below code but it is not working.
name=PropertiesConfig
property.filename = logs/Temp
appenders = console, file
appender.console.type = Console
appender.console.name = STDOUT
appender.console.layout.type = PatternLayout
appender.console.layout.pattern = [%-5level] %d{yyyy-MM-dd HH:mm:ss.SSS} - %msg%n
appender.file.type = File
appender.file.name = LOGFILE
appender.file.fileName=${filename}/TestRun_%d{yyyy-MM-dd HH:mm:ss.SSS}.log
appender.fileout = com.veerasundar.dynamiclogger.NewLogForEachRunFileAppender
appender.file.layout.type=PatternLayout
appender.file.layout.pattern=[%-5level] %d{yyyy-MM-dd HH:mm:ss.SSS} - %msg%n
loggers=file
logger.file.name=utilities
logger.file.level = debug
logger.file.appenderRefs = file
logger.file.appenderRef.file.ref = LOGFILE
rootLogger.level = debug
rootLogger.appenderRefs = stdout
rootLogger.appenderRef.stdout.ref = STDOUT
It is throwing below error when I execute the test suite.
2019-06-06 21:57:54,919 main ERROR FileManager (logs/Temp/TestRun_%d{yyyy-MM-dd HH:mm:ss.SSS}.log) java.io.FileNotFoundException: logs\Temp\TestRun_%d{yyyy-MM-dd HH:mm:ss.SSS}.log (The filename, directory name, or volume label syntax is incorrect) java.io.FileNotFoundException: logs\Temp\TestRun_%d{yyyy-MM-dd HH:mm:ss.SSS}.log (The filename, directory name, or volume label syntax is incorrect)
at java.io.FileOutputStream.open0(Native Method)
at java.io.FileOutputStream.open(Unknown Source)
at java.io.FileOutputStream.<init>(Unknown Source)
Can someone please help me?
The error is self-explanatory, the filename you have specified was not found.
One can provide fileName as some filename and the filepattern can be specified explicitly, like this:
appender.file.fileName = Testrun.log
appender.file.filePattern = Testrun_%d{yyyy-MM-dd HH:mm:ss.SSS}.log

apache flume log directory

Just begin to learn Apache Flume. I follow the instructions on Flume official getting started website:
https://cwiki.apache.org/confluence/display/FLUME/Getting+Started
Almost everything is fine after follow the instructions on above link. But I could not find any log file afterwards. I suppose log file is under {flume.directory}/logs. Any idea to find flume log files?
Here comes my log4j.properties:
flume.root.logger=INFO,LOGFILE
flume.log.dir=./logs
flume.log.file=flume.log
log4j.logger.org.apache.flume.lifecycle = INFO
log4j.logger.org.jboss = WARN
log4j.logger.org.mortbay = INFO
log4j.logger.org.apache.avro.ipc.NettyTransceiver = WARN
log4j.logger.org.apache.hadoop = INFO
log4j.logger.org.apache.hadoop.hive = ERROR
# Define the root logger to the system property "flume.root.logger".
log4j.rootLogger=${flume.root.logger}
# Stock log4j rolling file appender
# Default log rotation configuration
log4j.appender.LOGFILE=org.apache.log4j.RollingFileAppender
log4j.appender.LOGFILE.MaxFileSize=100MB
log4j.appender.LOGFILE.MaxBackupIndex=10
log4j.appender.LOGFILE.File=${flume.log.dir}/${flume.log.file}
log4j.appender.LOGFILE.layout=org.apache.log4j.PatternLayout
log4j.appender.LOGFILE.layout.ConversionPattern=%d{dd MMM yyyy HH:mm:ss,SSS} %-5p [%t] (%C.%M:%L) %x - %m%n
# Warning: If you enable the following appender it will fill up your disk if you don't have a cleanup job!
# This uses the updated rolling file appender from log4j-extras that supports a reliable time-based rolling policy.
# See http://logging.apache.org/log4j/companions/extras/apidocs/org/apache/log4j/rolling/TimeBasedRollingPolicy.html
# Add "DAILY" to flume.root.logger above if you want to use this
log4j.appender.DAILY=org.apache.log4j.rolling.RollingFileAppender
log4j.appender.DAILY.rollingPolicy=org.apache.log4j.rolling.TimeBasedRollingPolicy
log4j.appender.DAILY.rollingPolicy.ActiveFileName=${flume.log.dir}/${flume.log.file}
log4j.appender.DAILY.rollingPolicy.FileNamePattern=${flume.log.dir}/${flume.log.file}.%d{yyyy-MM-dd}
log4j.appender.DAILY.layout=org.apache.log4j.PatternLayout
log4j.appender.DAILY.layout.ConversionPattern=%d{dd MMM yyyy HH:mm:ss,SSS} %-5p [%t] (%C.%M:%L) %x - %m%n
# console
# Add "console" to flume.root.logger above if you want to use this
log4j.appender.console=org.apache.log4j.ConsoleAppender
log4j.appender.console.target=System.err
log4j.appender.console.layout=org.apache.log4j.PatternLayout
log4j.appender.console.layout.ConversionPattern=%d (%t) [%p - %l] %m%n
Cause your configuration use a relative path :
flume.log.dir=./logs
log4j.appender.LOGFILE.File=${flume.log.dir}/${flume.log.file}
Flume use your current directory to log.
(the directory where you are when you launch flume)
You can use an absolute path for flume.log.dir if you want to force flume to log in that path, no matter where you launch it.
Based on your log4j.properties, it would be in the same directory where you start your flume-agent, thats what ./ would do.
So, if you run from /home directory, it would be
/home/logs/flume.log
Also, you can use linux commands anytime to find that out,
find / -name flume.log ##search in whole disk
find . -name flume.log ##search in current directory

Resources