How to properly rotate and archive rundeck logs - log4j2

Question

I am using 2.6.4-1. Since rundeck logs dont archive or rotate themselves or don't drop a conf into logrotate with the installation; what is the best way to rotate and archive it..?
I can manually drop a config into logrotate so it can rotate and archive it, may be I can use copytruncate since Java apps (such as Rundeck) usually ignore UNIX signals like SIGHUP and SIGUSR1 but what stopping me using logrotate is, rundeck is trying to rotate the logs with appending a date stamp and keeping it there without archiving.This makes logrotate not working as expected. Since the size of the logs keep on growing, any alternatives for rundeck log rotation,archiving.
Or help me on how to avoid rundeck to try rotate the logs with appending a date stamp into the log name. Like rundeck.access.log turns to rundeck.access.log.2017-06-02 which makes logrotate to not working on archiving it.
Thanks in advance.
Leo Prince

Answer 1

As you already know that Rundeck uses log4f as its application logging facility. All logging is controlled by log4j. The log config file is RD_BASE/log4j.properties or RD_BASE/etc/log4j.properties if you are using launcher.
If you look into your log4j.properties file. You will find for the config for all files that appear under /var/log/rundeck/ directory have following setting:
log4j.appender.cmd-logger=org.apache.log4j.DailyRollingFileAppender
log4j.appender.cmd-logger.file=/var/log/rundeck/command.log
log4j.appender.cmd-logger.datePattern='.'yyyy-MM-dd
log4j.appender.cmd-logger.append=true
log4j.appender.cmd-logger.layout=org.apache.log4j.PatternLayout
log4j.appender.cmd-logger.layout.ConversionPattern=%d{ISO8601} [%t] %-5p %c - %m%n
org.apache.log4j.DailyRollingFileAppender makes log file rotate dailly.
The log file name will have a dateParttern as '.'yyyy-MM-dd
log4j log rotation and archive
TimeBasedRollingPolicy
The setting you are looking for is FileNamePattern
Here is an example on how to log rotate and archive using log4j
How to make log files rotate per day and zip old rotated logs using log4j.properties

Answer 2

The previous answer doesn't seem to work in log4j2. Here is my log4j2.properties file that rotates the archived logs every 2 days:
name = Rundeck Logging Configuration
property.baseDir = /var/log/rundeck
property.classLength = 2
property.noConsoleNoAnsi = true
property.prefix = [%style{%d{ISO8601}}{dim, noConsoleNoAnsi=${noConsoleNoAnsi}}] %highlight{%-5p}{noConsoleNoAnsi=${noConsoleNoAnsi}} %style{%c{${classLength}}}{cyan,noConsoleNoAnsi=${noConsoleNoAnsi}}
appender.console.type = Console
appender.console.name = STDOUT
appender.console.layout.type = PatternLayout
appender.console.layout.pattern = ${prefix} - %m%n
appender.rundeck.type = RollingFile
appender.rundeck.name = rundeck
appender.rundeck.fileName = ${baseDir}/rundeck.log
appender.rundeck.append = true
appender.rundeck.bufferedIO = true
appender.rundeck.filePattern = ${baseDir}/rundeck.log.%d{yyyy-MM-dd}.gz
appender.rundeck.layout.type = PatternLayout
appender.rundeck.layout.pattern = ${prefix} [%t] - %m%n
appender.rundeck.policies.type = Policies
appender.rundeck.policies.time.type = TimeBasedTriggeringPolicy
appender.rundeck.policies.time.interval = 1
appender.rundeck.strategy.type = DefaultRolloverStrategy
appender.rundeck.strategy.delete.type = Delete
appender.rundeck.strategy.delete.basePath = ${baseDir}
appender.rundeck.strategy.delete.maxDepth = 1
appender.rundeck.strategy.delete.ifFileName.type = IfFileName
appender.rundeck.strategy.delete.ifFileName.glob = rundeck.log.*.gz
appender.rundeck.strategy.delete.ifLastModified.type = IfLastModified
appender.rundeck.strategy.delete.ifLastModified.age = 2d
appender.audit.type = RollingFile
appender.audit.name = audit
appender.audit.fileName = ${baseDir}/rundeck.audit.log
appender.audit.append = true
appender.audit.bufferedIO = true
appender.audit.filePattern = ${baseDir}/rundeck.audit.log.%d{yyyy-MM-dd}.gz
appender.audit.layout.type = PatternLayout
appender.audit.layout.pattern = ${prefix} - %m%n
appender.audit.policies.type = Policies
appender.audit.policies.time.type = TimeBasedTriggeringPolicy
appender.audit.policies.time.interval = 1
appender.audit.strategy.type = DefaultRolloverStrategy
appender.audit.strategy.delete.type = Delete
appender.audit.strategy.delete.basePath = ${baseDir}
appender.audit.strategy.delete.maxDepth = 1
appender.audit.strategy.delete.ifFileName.type = IfFileName
appender.audit.strategy.delete.ifFileName.glob = rundeck.audit.log.*.gz
appender.audit.strategy.delete.ifLastModified.type = IfLastModified
appender.audit.strategy.delete.ifLastModified.age = 2d
appender.options.type = RollingFile
appender.options.name = options
appender.options.fileName = ${baseDir}/rundeck.options.log
appender.options.append = true
appender.options.bufferedIO = true
appender.options.filePattern = ${baseDir}/rundeck.options.log.%d{yyyy-MM-dd}.gz
appender.options.layout.type = PatternLayout
appender.options.layout.pattern = ${prefix} %X{httpStatusCode} %X{contentLength}B %X{durationTime}ms %X{lastModifiedDateTime} [%X{jobName}] %X{url} %X{contentSHA1}%n
appender.options.policies.type = Policies
appender.options.policies.time.type = TimeBasedTriggeringPolicy
appender.options.policies.time.interval = 1
appender.options.strategy.type = DefaultRolloverStrategy
appender.options.strategy.delete.type = Delete
appender.options.strategy.delete.basePath = ${baseDir}
appender.options.strategy.delete.maxDepth = 1
appender.options.strategy.delete.ifFileName.type = IfFileName
appender.options.strategy.delete.ifFileName.glob = rundeck.options.log.*.gz
appender.options.strategy.delete.ifLastModified.type = IfLastModified
appender.options.strategy.delete.ifLastModified.age = 2d
appender.storage.type = RollingFile
appender.storage.name = storage
appender.storage.fileName = ${baseDir}/rundeck.storage.log
appender.storage.append = true
appender.storage.bufferedIO = true
appender.storage.filePattern = ${baseDir}/rundeck.storage.log.%d{yyyy-MM-dd}.gz
appender.storage.layout.type = PatternLayout
appender.storage.layout.pattern = ${prefix} %X{action} %X{type} %X{path} %X{status} %X{metadata}%n
appender.storage.policies.type = Policies
appender.storage.policies.time.type = TimeBasedTriggeringPolicy
appender.storage.policies.time.interval = 1
appender.storage.strategy.type = DefaultRolloverStrategy
appender.storage.strategy.delete.type = Delete
appender.storage.strategy.delete.basePath = ${baseDir}
appender.storage.strategy.delete.maxDepth = 1
appender.storage.strategy.delete.ifFileName.type = IfFileName
appender.storage.strategy.delete.ifFileName.glob = rundeck.storage.log.*.gz
appender.storage.strategy.delete.ifLastModified.type = IfLastModified
appender.storage.strategy.delete.ifLastModified.age = 2d
appender.jobchanges.type = RollingFile
appender.jobchanges.name = jobchanges
appender.jobchanges.fileName = ${baseDir}/rundeck.jobs.log
appender.jobchanges.append = true
appender.jobchanges.bufferedIO = true
appender.jobchanges.filePattern = ${baseDir}/rundeck.jobs.log.%d{yyyy-MM-dd}.gz
appender.jobchanges.layout.type = PatternLayout
appender.jobchanges.layout.pattern = ${prefix} %X{user} %X{change} [%X{id}] %X{project} "%X{groupPath}/%X{jobName}" (%X{method})%X{extraInfo}%n
appender.jobchanges.policies.type = Policies
appender.jobchanges.policies.time.type = TimeBasedTriggeringPolicy
appender.jobchanges.policies.time.interval = 1
appender.jobchanges.strategy.type = DefaultRolloverStrategy
appender.jobchanges.strategy.delete.type = Delete
appender.jobchanges.strategy.delete.basePath = ${baseDir}
appender.jobchanges.strategy.delete.maxDepth = 1
appender.jobchanges.strategy.delete.ifFileName.type = IfFileName
appender.jobchanges.strategy.delete.ifFileName.glob = rundeck.jobs.log.*.gz
appender.jobchanges.strategy.delete.ifLastModified.type = IfLastModified
appender.jobchanges.strategy.delete.ifLastModified.age = 2d
appender.execevents.type = RollingFile
appender.execevents.name = execevents
appender.execevents.fileName = ${baseDir}/rundeck.executions.log
appender.execevents.append = true
appender.execevents.bufferedIO = true
appender.execevents.filePattern = ${baseDir}/rundeck.executions.log.%d{yyyy-MM-dd}.gz
appender.execevents.layout.type = PatternLayout
appender.execevents.layout.pattern = ${prefix} %X{eventUser} %X{event} [%X{id}:%X{state}] %X{project} %X{user}/%X{abortedby} \"%X{groupPath}/%X{jobName} %X{argString}\"[%X{uuid}] %n
appender.execevents.policies.type = Policies
appender.execevents.policies.time.type = TimeBasedTriggeringPolicy
appender.execevents.policies.time.interval = 1
appender.execevents.strategy.type = DefaultRolloverStrategy
appender.execevents.strategy.delete.type = Delete
appender.execevents.strategy.delete.basePath = ${baseDir}
appender.execevents.strategy.delete.maxDepth = 1
appender.execevents.strategy.delete.ifFileName.type = IfFileName
appender.execevents.strategy.delete.ifFileName.glob = rundeck.executions.log.*.gz
appender.execevents.strategy.delete.ifLastModified.type = IfLastModified
appender.execevents.strategy.delete.ifLastModified.age = 2d
appender.apirequests.type = RollingFile
appender.apirequests.name = apirequests
appender.apirequests.fileName = ${baseDir}/rundeck.api.log
appender.apirequests.append = true
appender.apirequests.bufferedIO = true
appender.apirequests.filePattern = ${baseDir}/rundeck.api.log.%d{yyyy-MM-dd}.gz
appender.apirequests.layout.type = PatternLayout
appender.apirequests.layout.pattern = ${prefix} "%X{method} %X{uri}" %X{remoteHost} %X{secure} %X{remoteUser} %X{authToken} %X{duration} %X{project} (%X{userAgent})%n
appender.apirequests.policies.type = Policies
appender.apirequests.policies.time.type = TimeBasedTriggeringPolicy
appender.apirequests.policies.time.interval = 1
appender.apirequests.strategy.type = DefaultRolloverStrategy
appender.apirequests.strategy.delete.type = Delete
appender.apirequests.strategy.delete.basePath = ${baseDir}
appender.apirequests.strategy.delete.maxDepth = 1
appender.apirequests.strategy.delete.ifFileName.type = IfFileName
appender.apirequests.strategy.delete.ifFileName.glob = rundeck.api.log.*.gz
appender.apirequests.strategy.delete.ifLastModified.type = IfLastModified
appender.apirequests.strategy.delete.ifLastModified.age = 2d
appender.access.type = RollingFile
appender.access.name = access
appender.access.fileName = ${baseDir}/rundeck.access.log
appender.access.append = true
appender.access.bufferedIO = true
appender.access.filePattern = ${baseDir}/rundeck.access.log.%d{yyyy-MM-dd}.gz
appender.access.layout.type = PatternLayout
appender.access.layout.pattern = ${prefix} "%X{method} %X{uri}" %X{remoteHost} %X{secure} %X{remoteUser} %X{authToken} %X{duration} %X{project} [%X{contentType}] (%X{userAgent})%n
appender.access.policies.type = Policies
appender.access.policies.time.type = TimeBasedTriggeringPolicy
appender.access.policies.time.interval = 1
appender.access.strategy.type = DefaultRolloverStrategy
appender.access.strategy.delete.type = Delete
appender.access.strategy.delete.basePath = ${baseDir}
appender.access.strategy.delete.maxDepth = 1
appender.access.strategy.delete.ifFileName.type = IfFileName
appender.access.strategy.delete.ifFileName.glob = rundeck.access.log.*.gz
appender.access.strategy.delete.ifLastModified.type = IfLastModified
appender.access.strategy.delete.ifLastModified.age = 2d
appender.project.type = RollingFile
appender.project.name = project
appender.project.fileName = ${baseDir}/rundeck.project.log
appender.project.append = true
appender.project.bufferedIO = true
appender.project.filePattern = ${baseDir}/rundeck.project.log.%d{yyyy-MM-dd}.gz
appender.project.layout.type = PatternLayout
appender.project.layout.pattern = ${prefix} - %m%n
appender.project.policies.type = Policies
appender.project.policies.time.type = TimeBasedTriggeringPolicy
appender.project.policies.time.interval = 1
appender.project.strategy.type = DefaultRolloverStrategy
appender.project.strategy.delete.type = Delete
appender.project.strategy.delete.basePath = ${baseDir}
appender.project.strategy.delete.maxDepth = 1
appender.project.strategy.delete.ifFileName.type = IfFileName
appender.project.strategy.delete.ifFileName.glob = rundeck.project.log.*.gz
appender.project.strategy.delete.ifLastModified.type = IfLastModified
appender.project.strategy.delete.ifLastModified.age = 2d
appender.cleanup.type = RollingFile
appender.cleanup.name = cleanup
appender.cleanup.fileName = ${baseDir}/rundeck.cleanup.log
appender.cleanup.append = true
appender.cleanup.bufferedIO = true
appender.cleanup.filePattern = ${baseDir}/rundeck.cleanup.log.%d{yyyy-MM-dd}.gz
appender.cleanup.layout.type = PatternLayout
appender.cleanup.layout.pattern = ${prefix} - %m%n
appender.cleanup.policies.type = Policies
appender.cleanup.policies.time.type = TimeBasedTriggeringPolicy
appender.cleanup.policies.time.interval = 1
appender.cleanup.strategy.type = DefaultRolloverStrategy
appender.cleanup.strategy.delete.type = Delete
appender.cleanup.strategy.delete.basePath = ${baseDir}
appender.cleanup.strategy.delete.maxDepth = 1
appender.cleanup.strategy.delete.ifFileName.type = IfFileName
appender.cleanup.strategy.delete.ifFileName.glob = rundeck.cleanup.log.*.gz
appender.cleanup.strategy.delete.ifLastModified.type = IfLastModified
appender.cleanup.strategy.delete.ifLastModified.age = 2d
appender.webhooks.type = RollingFile
appender.webhooks.name = webhooks
appender.webhooks.fileName = ${baseDir}/rundeck.webhooks.log
appender.webhooks.append = true
appender.webhooks.bufferedIO = true
appender.webhooks.filePattern = ${baseDir}/rundeck.webhooks.log.%d{yyyy-MM-dd}.gz
appender.webhooks.layout.type = PatternLayout
appender.webhooks.layout.pattern = ${prefix} - %m%n
appender.webhooks.policies.type = Policies
appender.webhooks.policies.time.type = TimeBasedTriggeringPolicy
appender.webhooks.policies.time.interval = 1
appender.webhooks.strategy.type = DefaultRolloverStrategy
appender.webhooks.strategy.delete.type = Delete
appender.webhooks.strategy.delete.basePath = ${baseDir}
appender.webhooks.strategy.delete.maxDepth = 1
appender.webhooks.strategy.delete.ifFileName.type = IfFileName
appender.webhooks.strategy.delete.ifFileName.glob = rundeck.webhooks.log.*.gz
appender.webhooks.strategy.delete.ifLastModified.type = IfLastModified
appender.webhooks.strategy.delete.ifLastModified.age = 2d
rootLogger.level = warn
rootLogger.appenderRef.stdout.ref = STDOUT
rootLogger.appenderRef.rundeck.ref = rundeck
logger.interceptors.name = rundeck.interceptors
logger.interceptors.level = info
logger.interceptors.additivity = false
logger.interceptors.appenderRef.stdout.ref = STDOUT
logger.rundeckapp.name = rundeckapp
logger.rundeckapp.level = info
logger.rundeckapp.additivity = false
logger.rundeckapp.appenderRef.stdout.ref = STDOUT
logger.bootstrap.name = rundeckapp.BootStrap
logger.bootstrap.level = info
logger.bootstrap.additivity = false
logger.bootstrap.appenderRef.stdout.ref = STDOUT
logger.grails.name = grails
logger.grails.level = warn
logger.grails.additivity = false
logger.grails.appenderRef.stdout.ref = STDOUT
logger.grails_env.name = grails.util.Environment
logger.grails_env.level = error
logger.grails_env.additivity = false
logger.grails_env.appenderRef.stdout.ref = STDOUT
logger.prjmanager.name = grails.app.services.rundeck.services.ProjectManagerService
logger.prjmanager.level = info
logger.prjmanager.additivity = false
logger.prjmanager.appenderRef.stdout.ref = STDOUT
logger.authorization.name = com.dtolabs.rundeck.core.authorization
logger.authorization.level = info
logger.authorization.additivity = false
logger.authorization.appenderRef.stdout.ref = audit
logger.options.name = com.dtolabs.rundeck.remoteservice.http.options
logger.options.level = info
logger.options.additivity = false
logger.options.appenderRef.stdout.ref = options
logger.jobchanges.name = com.dtolabs.rundeck.data.jobs.changes
logger.jobchanges.level = info
logger.jobchanges.additivity = false
logger.jobchanges.appenderRef.stdout.ref = jobchanges
logger.execevents.name = org.rundeck.execution.status
logger.execevents.level = info
logger.execevents.additivity = false
ogger.execevents.appenderRef.stdout.ref = execevents
logger.apirequests.name = org.rundeck.api.requests
logger.apirequests.level = info
logger.apirequests.additivity = false
logger.apirequests.appenderRef.stdout.ref = apirequests
logger.access.name = org.rundeck.web.requests
logger.access.level = info
logger.access.additivity = false
logger.access.appenderRef.access.ref = access
logger.project.name = org.rundeck.project.events
logger.project.level = info
logger.project.additivity = false
logger.project.appenderRef.stdout.ref = project
logger.storage.name = org.rundeck.storage.events
logger.storage.level = info
logger.storage.additivity = false
logger.storage.appenderRef.storage.ref = storage
logger.webhook_events.name = org.rundeck.webhook.events
logger.webhook_events.level = info
logger.webhook_events.additivity = false
logger.webhook_events.appenderRef.webhooks.ref = webhooks
logger.webhook_plugins.name = org.rundeck.plugin.webhook
logger.webhook_plugins.level = debug
logger.webhook_plugins.additivity = false
logger.webhook_plugins.appenderRef.webhooks.ref = webhooks
logger.cleanup.name = rundeck.quartzjobs.ExecutionsCleanUp
logger.cleanup.level = debug
logger.cleanup.additivity = false
logger.cleanup.appenderRef.cleanup.ref = cleanup
logger.jetty.name = org.mortbay.log
logger.jetty.level = warn
logger.jetty.additivity = false
logger.jetty.appenderRef.stdout.ref = STDOUT
logger.hibernate.name = org.hibernate.orm.deprecation
logger.hibernate.level = error
logger.hibernate.additivity = false
logger.hibernate.appenderRef.stdout.ref = STDOUT
logger.rundeck_jaas.name = com.dtolabs.rundeck.jetty.jaas
logger.rundeck_jaas.level = debug
logger.rundeck_jaas.additivity = false
logger.rundeck_jaas.appenderRef.stdout.ref = STDOUT
logger.spring_security.name = grails.plugin.springsecurity.web.authentication.GrailsUsernamePasswordAuthenticationFilter
logger.spring_security.level = debug
logger.spring_security.additivity = false
logger.spring_security.appenderRef.stdout.ref = STDOUT
logger.jaas.name = org.rundeck.jaas
logger.jaas.level = debug
logger.jaas.additivity = false
logger.jaas.appenderRef.stdout.ref = STDOUT
logger.springBeanPropertyDescriptor.name = org.springframework.beans.GenericTypeAwarePropertyDescriptor
logger.springBeanPropertyDescriptor.level = error
logger.springBeanPropertyDescriptor.additivity = false
logger.springBeanPropertyDescriptor.appenderRef.stdout.ref = STDOUT