Using ActiveMQ in Grails through JMS plugin I cannot figure how to set KahaDb persistence properties.
Tried something like:
amq.broker(useJmx: true, persistent: true) {
amq.transportConnectors() {
amq.transportConnector(uri: "tcp://0.0.0.0:61616")
}
amq.persistenceAdapter() {
amq.kahaDB(directory:${application.config.grails.moviesxd.activemq.kahadb},
checksumJournalFiles:true,
checkForCorruptJournalFiles:true,
ignoreMissingJournalfiles:true)
}
}
But i get
groovy.lang.GroovyRuntimeException: Namespace prefix: kahadb is not bound to a URI
Seems like kahadb prefix is not recognized but i cannot find documentation anywhere on how to do this.
You're just missing the equivalent of
xmlns:amq='http://activemq.apache.org/schema/core'
which would be
xmlns amq:'http://activemq.apache.org/schema/core'
(see the reference docs for more info - search for "Using Spring Namespaces")
so the whole conversion would be
import org.springframework.jms.connection.SingleConnectionFactory
xmlns amq:'http://activemq.apache.org/schema/core'
amq.broker(useJmx: true, persistent: true) {
amq.transportConnectors {
amq.transportConnector(uri: 'tcp://0.0.0.0:61616')
}
amq.persistenceAdapter {
amq.kahaDB(directory: application.config.grails.moviesxd.activemq.kahadb,
checksumJournalFiles: true,
checkForCorruptJournalFiles: true,
ignoreMissingJournalfiles: true)
}
}
amq.connectionFactory(id: 'amqConnectionFactory', brokerURL: 'vm://localhost')
jmsConnectionFactory(SingleConnectionFactory, ref('amqConnectionFactory'))
As a workaround what i've done is to include an additional resources.xml and use standard and well documented XML properties as:
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:amq="http://activemq.apache.org/schema/core" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
http://activemq.apache.org/schema/core http://activemq.apache.org/schema/core/activemq-core.xsd">
<amq:broker useJmx="true" persistent="true">
<amq:transportConnectors>
<amq:transportConnector uri="tcp://0.0.0.0:61616" />
</amq:transportConnectors>
<amq:persistenceAdapter>
<amq:kahaDB directory="${grails.moviesxd.activemq.kahadb.path}" />
</amq:persistenceAdapter>
</amq:broker>
<amq:connectionFactory id="amqConnectionFactory"
brokerURL="vm://localhost" />
<bean id="jmsConnectionFactory" class="org.springframework.jms.connection.SingleConnectionFactory">
<constructor-arg ref="amqConnectionFactory" />
</bean>
</beans>
... but it's a piti i have to do this ... still will be great if i get an answer.
Related
I've been trying to handle security of log4j2 in our spring application to pass in Veracode. Especially CWE 117 - log injection vulnerability.
We have a spring application with spring-boot-starter-log4j2. I have tried to configure log4j2 pattern:
<PatternLayout pattern="%d{DEFAULT} [%t] %-5level %logger{36} - %encode{%m}%n" />
but it doesn't work.
I also tried something like this:
<PatternLayout pattern="%d{ISO8601} %-5p - %encode{ %.-500m }{CRLF}%n" />
or
<PatternLayout pattern="%d{HH:mm:ss.SSS} %marker [%t] %-5level %logger{36} - %encode{%msg}{CRLF}%n"/>
I am still getting the veracode result:
117 Improper Output Neutralization for Logs WelcomeResource.java: 15
117 Improper Output Neutralization for Logs WelcomeResource.java: 16
We don't want use ESAPI nor any log facade, we don't want to change all log rows in our code, there are thousands of occurrences.
We would like to use the straigt setting as in the snippet below or here:
https://owasp.org/www-project-cheat-sheets/cheatsheets/Injection_Prevention_Cheat_Sheet_in_Java.html#Log_Injection
or
https://github.com/javabeanz/owasp-security-logging/wiki/Log-Forging
But it doesn't work. Where could be the problem?
Here is a snippet of our code:
build.gradle:
plugins {
id 'org.springframework.boot' version '2.2.0.RELEASE'
id 'io.spring.dependency-management' version '1.0.8.RELEASE'
id 'java'
id 'maven'
}
group = 'com.example'
version = '0.0.2-SNAPSHOT'
repositories {
mavenCentral()
}
configurations {
all {
exclude group: 'org.springframework.boot', module: 'spring-boot-starter-logging'
}
}
dependencies {
implementation 'org.springframework.boot:spring-boot-starter-actuator'
implementation 'org.springframework.boot:spring-boot-starter-web'
implementation 'org.springframework.boot:spring-boot-starter-log4j2'
}
App.java:
package com.example.demoLog4j2;
import org.slf4j.LoggerFactory;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
#SpringBootApplication
public class App {
final static org.slf4j.Logger Logger = LoggerFactory.getLogger("App");
public static void main(String[] args) {
SpringApplication.run(App.class, args);
System.out.println(" //---------------------->> DemoLog4j2 Application started... ");
Logger.info(" Logger implementation: " + Logger.getClass().getName());
}
}
WelcomeResource.java:
package com.example.demoLog4j2;
import org.slf4j.LoggerFactory;
import org.springframework.web.bind.annotation.*;
#RestController
public class WelcomeResource {
private static final String welcomeMessage = "Welcome...";
final org.slf4j.Logger Logger = LoggerFactory.getLogger(this.getClass());
#GetMapping("/name")
public String getName(#RequestParam(name="name", required = false, defaultValue = "Josef") String name) {
Logger.info( "----- name: " + name);
Logger.debug( "--- name: " + name );
return "name: " + name;
}
}
log4j2.xml:
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="INFO ">
<Appenders>
<Console name="Console" target="SYSTEM_OUT">
<!-- <PatternLayout pattern="%d{DEFAULT} [%t] %-5level %logger{36} - %encode{%m}%n" /> -->
<!-- <PatternLayout pattern="%d{HH:mm:ss.SSS} %marker [%t] %-5level %logger{36} - %encode{%msg}{CRLF}%n" /> -->
<PatternLayout pattern="%d{ISO8601} %-5p - %encode{ %.-500m }{CRLF}%n" />
</Console>
</Appenders>
<Loggers>
<Root level="INFO">
<AppenderRef ref="Console" />
</Root>
</Loggers>
</Configuration>
Finally we have solved the logging injection threat with "%encode{%msg}" in log4j2 config file. It solved the threat, but it didn't solve the veracode report. We ignored it, because it was false report. I don't know if veracode repaired it.
I'm having issues getting phpunit to positively test a method i created. im using a zend application and the ApplicationTest works fine. Here's my current command line output:
ApplicationTest\Controller\IndexController
[x] Sample not active
[x] Index action can be accessed
[x] Index action view model template rendered within layout
[x] Invalid route does not crash
PlutoTest\Stdlib\ArrayUtils
[ ] Sample not active
As you can see, there is no X inside my ArrayUtils test. I even put the method inside the ApplicationTest and you can see it executed correctly.
Here's the phpunit class:
namespace PlutoTest\Stdlib;
use Pluto\Stdlib\ArrayUtils;
use Pluto\pluto;
use PHPUnit\Framework\TestCase;
class ArrayUtilsTest extends TestCase
{
public function setUp()
{
$configOverrides = [];
$phpunitConfigFile=$this->getNormalizedPhpunitConfigFile();
$this->setApplicationConfig(ArrayUtils::merge(
include $phpunitConfigFile,
$configOverrides
));
parent::setUp();
}
private function getNormalizedPhpunitConfigFile()
{
$file = sprintf('%s/application.phpunit.php',pluto::path('zfconfig'));
return $file;
}
public function testSampleNotActive()
{
$stack = [];
$this->assertEquals(0, count($stack));
array_push($stack, 'foo');
$this->assertEquals('foo', $stack[count($stack)-1]);
$this->assertEquals(1, count($stack));
$this->assertEquals('foo', array_pop($stack));
$this->assertEquals(0, count($stack));
}
}
Here's my phpunit.xml:
<?xml version="1.0" encoding="UTF-8"?>
<phpunit
colors="true"
bootstrap="bootstrap.phpunit.php"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="https://schema.phpunit.de/6.3/phpunit.xsd"
backupGlobals="true"
backupStaticAttributes="false"
cacheTokens="false"
convertErrorsToExceptions="true"
convertNoticesToExceptions="true"
convertWarningsToExceptions="true"
forceCoversAnnotation="true"
processIsolation="false"
stopOnError="false"
stopOnFailure="false"
stopOnIncomplete="false"
stopOnSkipped="false"
timeoutForSmallTests="1"
timeoutForMediumTests="10"
timeoutForLargeTests="60"
beStrictAboutTestsThatDoNotTestAnything="true"
beStrictAboutOutputDuringTests="true"
beStrictAboutTestSize="true"
verbose="true">
<testsuites>
<testsuite name="Application Module Test Suite">
<directory phpVersion="7.0.0" phpVersionOperator=">=">./module/Application/test</directory>
<directory phpVersion="7.0.0" phpVersionOperator=">=">./test/pluto/src</directory>
</testsuite>
</testsuites>
<php>
<env name="APPLICATION_ENV" value="development"/>
<ini name="display_errors" value="1"/>
<ini name="display_startup_errors" value="1"/>
<ini name="error_reporting" value="32767"/>
<ini name="report_memleaks" value="1"/>
</php>
</phpunit>
And here's my composer:
"autoload-dev" : {
"psr-4" : {
"ApplicationTest\\" : "module/Application/test/",
"PlutoTest\\" : "test/pluto/src"
}
},
Try changing as the following:
"autoload-dev": {
"psr-4": {
...
"YourModuleNameTest\\": "module/YourModuleName/test/"
}
},
If done, do not forget to run dump-autoload through composer in your terminal.
What if you modify your phpunit.xml this way:
<testsuites>
<testsuite name="Application Module Test Suite">
<directory phpVersion="7.0.0" phpVersionOperator=">=">./module/Application/test</directory>
</testsuite>
<testsuite name="YourModuleName">
<directory phpVersion="7.0.0" phpVersionOperator=">=">./module/YourModuleName/test</directory>
</testsuite>
</testsuites>
Next up, run the following command:
./vendor/bin/phpunit --testsuite YourModuleName
To list all available suites, run the following command:
./vendor/bin/phpunit --list-suites
To list all available tests, run the following command:
./vendor/bin/phpunit --list-tests
Using the Grails config DSL, how can I configure the appender-ref property on an appender?
I am trying to translate a log4j.xml from perf4j into the Grails config DSL (Grails 2.x).
In the log4j.xml file that I am trying to imitate, there is a segment like this:
log4j.xml
<appender name="CoalescingStatistics"
class="org.perf4j.log4j.AsyncCoalescingStatisticsAppender">
<param name="TimeSlice" value="10000"/>
<appender-ref ref="fileAppender"/>
</appender>
I want to imitate this appender's config in Grails, but I can't figure out the <appender-ref> tag.
Here is my latest attempt:
log4j = {
appenders {
//main appender for the app
rollingFile ...
//perf4j coalescing stats appender
appender new AsyncCoalescingStatisticsAppender(name: "coalescingStatsAppender", timeSlice: 10000)
//perf4j stats file appender
rollingFile name: "perfLogFileAppender", file: "perfStats.log", layout: new PatternLayout("%m%n")
}
root {
error 'stdout', 'rollingFileAppender'
}
info additivity: false, coalescingStatsAppender: [StopWatch.DEFAULT_LOGGER_NAME]
}
Additional details
Here is the source log4j.xml I am imitating:
https://web.archive.org/web/20150508124059/http://perf4j.codehaus.org/devguide.html#Using_the_log4j_Appenders_to_Generate_Real-Time_Performance_Information
I was able to make it work with some Groovy magic:
log4j = {
appenders {
//...regular appenders
//perf4j appenders
Appender perfLogFileAppender = delegate.rollingFile name: "perfLogFileAppender", file: "perfStats.log", layout: new PatternLayout("%m%n")
def statsAppender = new AsyncCoalescingStatisticsAppender(name: "coalescingStatsAppender", timeSlice:
10000)
statsAppender.addAppender(perfLogFileAppender)
appender statsAppender
}
root {
error 'stdout', 'rollingFileAppender'
}
info additivity: false, coalescingStatsAppender: [StopWatch.DEFAULT_LOGGER_NAME]
}
A more elegant solution is always welcome.
I had gone through this blog to make use of 2nd level caching in grails
link
& I made the following changes
in resources.groovy
userCache(
org.springframework.cache.ehcache.EhCacheFactoryBean) {
}
Inside service class
def userCache
public def userCachedList(){
if (userCache.get("userList")) {
...
}
else {
...
userCache.put(
new net.sf.ehcache.Element("userList",
list)
)
}
...
}
Caching works fine but the issue this cached list is stored in physical location. When I start the application, I see .ehcache-diskstore.lock is created in tmp directory.
My settings are
hibernate {
cache.use_second_level_cache = false
cache.use_query_cache = false
//cache.use_structured_entities = false
cache.provider_class = 'net.sf.ehcache.hibernate.EhCacheProvider'
hibernate.format_sql=false
hibernate.connection.release_mode='after_transaction'
}
ehcache.xml is
<ehcache name="myapp">
<!-- Necessary for spring-security-acl plugin-->
<!--<diskStore path="java.io.tmpdir"/>-->
<defaultCache
maxElementsInMemory="20000"
eternal="false"
timeToIdleSeconds="3600"
timeToLiveSeconds="3600"
overflowToDisk="false">
<!-- <cacheEventListenerFactory
class="net.sf.ehcache.distribution.RMICacheReplicatorFactory"/> -->
</defaultCache>
<cache
name="owsoo.User"
maxEntriesLocalHeap="50"
eternal="false"
timeToLiveSeconds="600"
overflowToDisk="false">
<!-- <cacheEventListenerFactory
class="net.sf.ehcache.distribution.RMICacheReplicatorFactory"/> -->
</cache>
<cache name="aclCache"
maxElementsInMemory="100000"
eternal="true"
overflowToDisk="false"
memoryStoreEvictionPolicy="LRU"
statistics="true"
/>
What do I need to change in the setting to use in-memory cache instead of physical disk storage?
it seems default values provided in ehcache.xml does not work for beans mentioned in resources.groovy so I added properties here to solve this issue
userCache(org.springframework.cache.ehcache.EhCacheFactoryBean) { bean ->
maxElementsInMemory="1000"
eternal="true"
overflowToDisk="false"
memoryStoreEvictionPolicy="LRU"
}
I am trying to manually reconfigure log4j2 at runtime but getting partial success.
Here is the relevant code:
package examples.test;
public class ABCImpl implements XYX{
static Logger logger;
public void initialize(){
LoggerContext ctx = null;
Configuration config = null;
Map mp = null;
ctx = (LoggerContext) LogManager.getContext(false);
config = (Configuration)ctx.getConfiguration();
mp = config.getAppenders();
System.out.println("***<Provider o/p follows> Before logger re-configuration:");
System.out.println("\tAppenders:" + mp.keySet());
//reconfiguration attempt - starts
try{
URI configuration = this.getClass().getResource("/log4j2.xml").toURI();
ctx = Configurator.initialize(this.getClass().getName(), null, configuration);
}catch(Exception e){
System.out.println("\t-------Exception encountered-------");
e.printStackTrace();
}
config = (Configuration) ctx.getConfiguration();
mp = config.getAppenders();
System.out.println("***<Provider o/p follows> After logger re-configuration:");
System.out.println("\tAppenders:" + mp.keySet());
//reconfiguration attempt - ends
logger = LogManager.getLogger(this.getClass().getName());
}
public void myBusinessMethod(){
logger.info("Entry..............");
logger.info("Exit..............");
}
}
This class is actually part of a jar file and am running it inside an application server which guarantees that my initialize method would be called as soon as my class is instantiated.
Here is my log4j2.xml, which I have packed into jar's root:
<?xml version="1.0" encoding="UTF-8"?>
<!-- Don't forget to set system property
-DLog4jContextSelector=org.apache.logging.log4j.core.async.AsyncLoggerContextSelector
to make all loggers asynchronous. -->
<Configuration status="info">
<Appenders>
<!-- Async Loggers will auto-flush in batches, so switch off immediateFlush. -->
<RollingRandomAccessFile name="Appender1" fileName="servers/${sys:weblogic.Name}/logs/Auditing_${sys:weblogic.Name}.log" immediateFlush="true" append="false" filePattern="servers/${sys:weblogic.Name}/logs/archive/Auditing_${sys:weblogic.Name}-%d{yyyy-MM-dd-HH}-%i.log.gz">
<PatternLayout>
<Pattern>%d %p %c{1.} [%t] %m %ex%n</Pattern>
</PatternLayout>
<Policies>
<TimeBasedTriggeringPolicy interval="1" modulate="true"/>
<SizeBasedTriggeringPolicy size="250 MB"/>
</Policies>
<DefaultRolloverStrategy max="20"/>
</RollingRandomAccessFile>
<Async name="Async1">
<AppenderRef ref="Appender1"/>
</Async>
</Appenders>
<Loggers>
<Logger name="examples.test.ABCImpl" level="info" includeLocation="false" additivity="false">
<AppenderRef ref="Appender1"/>
</Logger>
<Root level="info" includeLocation="false">
<AppenderRef ref="Appender1"/>
</Root>
</Loggers>
</Configuration>
The problem is even though the log file gets created, but nothing gets logged in it. The output in std out that I get is :
ABCImpl.initialize
activeHandlerEntries.length=1
***<Provider o/p follows> Before logger re-configuration:
Appenders:[Console]
***<Provider o/p follows> After logger re-configuration:
Appenders:[Async1, Appender1]
Just to make sure that my log4j2.xml and the piece of code is okay without runtime reconfiguration stuff, if I use -Dlog4j.configurationFile=file:SOME_PATH_OUTSIDE_JAR/log4j2.xml log gets populated as expected.
Please help.
I wondered, if deriving the context for initialization by
ctx = (LoggerContext) LogManager.getContext(false);
may be the problem, since you have to access the "current" context with
ctx = (LoggerContext) LogManager.getContext(true);
according to the Log4j2 API. I would really appreciate a clarification, since i'm trying to programmatically configuring Log4J2, too.
Hours of helpless staring at the code and trying my hands here and there, I found that if I get the Logger from newly initialized context (which is obtained from Configurator.initialize) instead of LogManager, log gets populated with data.
Without this trick, not sure if I needed to call any additional API methods (just after Configurator.initialize) to hook the new context into LogManager so that I could have continued traditional approach of getting Logger from LogManager.
...
config = (Configuration) ctx.getConfiguration();
mp = config.getAppenders();
System.out.println("***<Provider o/p follows> After logger re-configuration:");
System.out.println("\tAppenders:" + mp.keySet());
//reconfiguration attempt - ends
//Following line screwed me up for over three weeks
//logger = LogManager.getLogger(this.getClass().getName());
//workaround:
logger = ctx.getLogger(this.getClass().getName());
}
public void myBusinessMethod(){
logger.info("Entry..............");
logger.info("Exit..............");
}
....
}