When i run a
mvn test
Maven runs all the tests successfully and then goes on a wait infinite loop
I see a java process in the task manager and I have to kill it before maven returns with this error message
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1:33.949s
[INFO] Finished at: Fri Jun 17 14:50:01 EDT 2011
[INFO] Final Memory: 5M/247M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.
4:test (default-test) on project EDIFileHandling: There are test failures.
[ERROR]
[ERROR] Please refer to c:\code\target\surefire-reports for the individual test
results.
Any help appreciated.
Thanks
R
C:\Users\Raghuveer.Rao>jstack 7256
2011-06-17 15:24:55
Full thread dump Java HotSpot(TM) Client VM (20.0-b11 mixed mode, sharing):
"Thread-2" prio=6 tid=0x03ed3000 nid=0x14d0 waiting for monitor entry [0x04ddf00
0]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.hsqldb.Session.close(Unknown Source)
- waiting to lock <0x23ba76f0> (a org.hsqldb.Session)
at org.hsqldb.SessionManager.closeAllSessions(Unknown Source)
- locked <0x292ce828> (a org.hsqldb.SessionManager)
at org.hsqldb.Database.close(Unknown Source)
at org.hsqldb.StatementCommand.getResult(Unknown Source)
at org.hsqldb.StatementCommand.execute(Unknown Source)
at org.hsqldb.Session.executeCompiledStatement(Unknown Source)
at org.hsqldb.Session.executeDirectStatement(Unknown Source)
at org.hsqldb.Session.execute(Unknown Source)
- locked <0x23b8efe8> (a org.hsqldb.Session)
at org.hsqldb.jdbc.JDBCStatement.fetchResult(Unknown Source)
at org.hsqldb.jdbc.JDBCStatement.execute(Unknown Source)
- locked <0x23b90f30> (a org.hsqldb.jdbc.JDBCStatement)
at org.springframework.jdbc.datasource.embedded.AbstractEmbeddedDatabase
Configurer.shutdown(AbstractEmbeddedDatabaseConfigurer.java:42)
at org.springframework.jdbc.datasource.embedded.EmbeddedDatabaseFactory.
shutdownDatabase(EmbeddedDatabaseFactory.java:184)
at org.springframework.jdbc.datasource.embedded.EmbeddedDatabaseFactoryB
ean.destroy(EmbeddedDatabaseFactoryBean.java:57)
at org.springframework.beans.factory.support.DisposableBeanAdapter.destr
oy(DisposableBeanAdapter.java:184)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistr
y.destroyBean(DefaultSingletonBeanRegistry.java:487)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistr
y.destroySingleton(DefaultSingletonBeanRegistry.java:463)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistr
y.destroySingletons(DefaultSingletonBeanRegistry.java:431)
- locked <0x2951f1c0> (a java.util.LinkedHashMap)
at org.springframework.context.support.AbstractApplicationContext.destro
yBeans(AbstractApplicationContext.java:1048)
at org.springframework.context.support.AbstractApplicationContext.doClos
e(AbstractApplicationContext.java:1022)
at org.springframework.context.support.AbstractApplicationContext$3.run(
AbstractApplicationContext.java:940)
"Thread-1" prio=6 tid=0x03ed2c00 nid=0x1fe0 waiting for monitor entry [0x04d8f00
0]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.hsqldb.SessionManager.closeAllSessions(Unknown Source)
- waiting to lock <0x292ce828> (a org.hsqldb.SessionManager)
at org.hsqldb.Database.close(Unknown Source)
at org.hsqldb.StatementCommand.getResult(Unknown Source)
at org.hsqldb.StatementCommand.execute(Unknown Source)
at org.hsqldb.Session.executeCompiledStatement(Unknown Source)
at org.hsqldb.Session.executeDirectStatement(Unknown Source)
at org.hsqldb.Session.execute(Unknown Source)
- locked <0x23ba76f0> (a org.hsqldb.Session)
at org.hsqldb.jdbc.JDBCStatement.fetchResult(Unknown Source)
at org.hsqldb.jdbc.JDBCStatement.execute(Unknown Source)
- locked <0x23ba9568> (a org.hsqldb.jdbc.JDBCStatement)
at org.springframework.jdbc.datasource.embedded.AbstractEmbeddedDatabase
Configurer.shutdown(AbstractEmbeddedDatabaseConfigurer.java:42)
at org.springframework.jdbc.datasource.embedded.EmbeddedDatabaseFactory.
shutdownDatabase(EmbeddedDatabaseFactory.java:184)
at org.springframework.jdbc.datasource.embedded.EmbeddedDatabaseFactoryB
ean.destroy(EmbeddedDatabaseFactoryBean.java:57)
at org.springframework.beans.factory.support.DisposableBeanAdapter.destr
oy(DisposableBeanAdapter.java:184)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistr
y.destroyBean(DefaultSingletonBeanRegistry.java:487)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistr
y.destroySingleton(DefaultSingletonBeanRegistry.java:463)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistr
y.destroySingletons(DefaultSingletonBeanRegistry.java:431)
- locked <0x2910d510> (a java.util.LinkedHashMap)
at org.springframework.context.support.AbstractApplicationContext.destro
yBeans(AbstractApplicationContext.java:1048)
at org.springframework.context.support.AbstractApplicationContext.doClos
e(AbstractApplicationContext.java:1022)
at org.springframework.context.support.AbstractApplicationContext$3.run(
AbstractApplicationContext.java:940)
"Low Memory Detector" daemon prio=6 tid=0x01bdac00 nid=0x1eb0 runnable [0x000000
00]
java.lang.Thread.State: RUNNABLE
"C1 CompilerThread0" daemon prio=10 tid=0x01bc8400 nid=0x18e8 waiting on conditi
on [0x00000000]
java.lang.Thread.State: RUNNABLE
"Attach Listener" daemon prio=10 tid=0x01bc5400 nid=0x1ac0 waiting on condition
[0x00000000]
java.lang.Thread.State: RUNNABLE
"Signal Dispatcher" daemon prio=10 tid=0x01bc2400 nid=0x1fbc runnable [0x0000000
0]
java.lang.Thread.State: RUNNABLE
"Finalizer" daemon prio=8 tid=0x01bbb800 nid=0x1c00 in Object.wait() [0x03cdf000
]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x290702b0> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
- locked <0x290702b0> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)
"Reference Handler" daemon prio=10 tid=0x01bba000 nid=0x810 in Object.wait() [0x
03c8f000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x29070340> (a java.lang.ref.Reference$Lock)
at java.lang.Object.wait(Object.java:485)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
- locked <0x29070340> (a java.lang.ref.Reference$Lock)
"main" prio=6 tid=0x00e29800 nid=0x1e20 in Object.wait() [0x001bf000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x294c4300> (a org.springframework.context.support.Abstrac
tApplicationContext$3)
at java.lang.Thread.join(Thread.java:1186)
- locked <0x294c4300> (a org.springframework.context.support.AbstractApp
licationContext$3)
at java.lang.Thread.join(Thread.java:1239)
at java.lang.ApplicationShutdownHooks.runHooks(ApplicationShutdownHooks.
java:79)
at java.lang.ApplicationShutdownHooks$1.run(ApplicationShutdownHooks.jav
a:24)
at java.lang.Shutdown.runHooks(Shutdown.java:79)
at java.lang.Shutdown.sequence(Shutdown.java:123)
at java.lang.Shutdown.exit(Shutdown.java:168)
- locked <0x38b53ef8> (a java.lang.Class for java.lang.Shutdown)
at java.lang.Runtime.exit(Runtime.java:90)
at java.lang.System.exit(System.java:904)
at org.apache.maven.surefire.booter.SurefireBooter.main(SurefireBooter.j
ava:986)
"VM Thread" prio=10 tid=0x01b7e400 nid=0x1e10 runnable
"VM Periodic Task Thread" prio=10 tid=0x01bec800 nid=0x1bac waiting on condition
JNI global references: 1776
Found one Java-level deadlock:
"Thread-2":
waiting to lock monitor 0x0467c1cc (object 0x23ba76f0, a org.hsqldb.Session),
which is held by "Thread-1"
"Thread-1":
waiting to lock monitor 0x01bbf56c (object 0x292ce828, a org.hsqldb.SessionMan
ager),
which is held by "Thread-2"
Java stack information for the threads listed above:
"Thread-2":
at org.hsqldb.Session.close(Unknown Source)
- waiting to lock <0x23ba76f0> (a org.hsqldb.Session)
at org.hsqldb.SessionManager.closeAllSessions(Unknown Source)
- locked <0x292ce828> (a org.hsqldb.SessionManager)
at org.hsqldb.Database.close(Unknown Source)
at org.hsqldb.StatementCommand.getResult(Unknown Source)
at org.hsqldb.StatementCommand.execute(Unknown Source)
at org.hsqldb.Session.executeCompiledStatement(Unknown Source)
at org.hsqldb.Session.executeDirectStatement(Unknown Source)
at org.hsqldb.Session.execute(Unknown Source)
- locked <0x23b8efe8> (a org.hsqldb.Session)
at org.hsqldb.jdbc.JDBCStatement.fetchResult(Unknown Source)
at org.hsqldb.jdbc.JDBCStatement.execute(Unknown Source)
- locked <0x23b90f30> (a org.hsqldb.jdbc.JDBCStatement)
at org.springframework.jdbc.datasource.embedded.AbstractEmbeddedDatabase
Configurer.shutdown(AbstractEmbeddedDatabaseConfigurer.java:42)
at org.springframework.jdbc.datasource.embedded.EmbeddedDatabaseFactory.
shutdownDatabase(EmbeddedDatabaseFactory.java:184)
at org.springframework.jdbc.datasource.embedded.EmbeddedDatabaseFactoryB
ean.destroy(EmbeddedDatabaseFactoryBean.java:57)
at org.springframework.beans.factory.support.DisposableBeanAdapter.destr
oy(DisposableBeanAdapter.java:184)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistr
y.destroyBean(DefaultSingletonBeanRegistry.java:487)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistr
y.destroySingleton(DefaultSingletonBeanRegistry.java:463)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistr
y.destroySingletons(DefaultSingletonBeanRegistry.java:431)
- locked <0x2951f1c0> (a java.util.LinkedHashMap)
at org.springframework.context.support.AbstractApplicationContext.destro
yBeans(AbstractApplicationContext.java:1048)
at org.springframework.context.support.AbstractApplicationContext.doClos
e(AbstractApplicationContext.java:1022)
at org.springframework.context.support.AbstractApplicationContext$3.run(
AbstractApplicationContext.java:940)
"Thread-1":
at org.hsqldb.SessionManager.closeAllSessions(Unknown Source)
- waiting to lock <0x292ce828> (a org.hsqldb.SessionManager)
at org.hsqldb.Database.close(Unknown Source)
at org.hsqldb.StatementCommand.getResult(Unknown Source)
at org.hsqldb.StatementCommand.execute(Unknown Source)
at org.hsqldb.Session.executeCompiledStatement(Unknown Source)
at org.hsqldb.Session.executeDirectStatement(Unknown Source)
at org.hsqldb.Session.execute(Unknown Source)
- locked <0x23ba76f0> (a org.hsqldb.Session)
at org.hsqldb.jdbc.JDBCStatement.fetchResult(Unknown Source)
at org.hsqldb.jdbc.JDBCStatement.execute(Unknown Source)
- locked <0x23ba9568> (a org.hsqldb.jdbc.JDBCStatement)
at org.springframework.jdbc.datasource.embedded.AbstractEmbeddedDatabase
Configurer.shutdown(AbstractEmbeddedDatabaseConfigurer.java:42)
at org.springframework.jdbc.datasource.embedded.EmbeddedDatabaseFactory.
shutdownDatabase(EmbeddedDatabaseFactory.java:184)
at org.springframework.jdbc.datasource.embedded.EmbeddedDatabaseFactoryB
ean.destroy(EmbeddedDatabaseFactoryBean.java:57)
at org.springframework.beans.factory.support.DisposableBeanAdapter.destr
oy(DisposableBeanAdapter.java:184)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistr
y.destroyBean(DefaultSingletonBeanRegistry.java:487)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistr
y.destroySingleton(DefaultSingletonBeanRegistry.java:463)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistr
y.destroySingletons(DefaultSingletonBeanRegistry.java:431)
- locked <0x2910d510> (a java.util.LinkedHashMap)
at org.springframework.context.support.AbstractApplicationContext.destro
yBeans(AbstractApplicationContext.java:1048)
at org.springframework.context.support.AbstractApplicationContext.doClos
e(AbstractApplicationContext.java:1022)
at org.springframework.context.support.AbstractApplicationContext$3.run(
AbstractApplicationContext.java:940)
Found 1 deadlock.
C:\Users\Raghuveer.Rao>
Maven surefire plugin by default spawns a new process to run the tests. This is probably the process you are killing. When maven notices its child process was interrupted, it probably assummes that the tests failed.
Have a look at /target/surefire-reports for confirmation. What you can do to help us diagnose the problem is to find the surefire process id (the one you are killing) and run:
$ jstack ID
Stack dump will tell you what is causing the test to hung.
Related
My client application is a Spring-shell application, running SpringBoot 2.1.1.
We are creating rabbitmq connection on demand CachingConnectionFacotry
CachingConnectionFactory connectionFactory = new CachingConnectionFactory(uri);
connectionFactory.setPublisherReturns(true);
When exiting the application, calling cachingConnectionFactory.destroy(); on #PreDestroy and/or onContextClosed events is not shutting down cachingConnectionFactory .
It is taking a minute for the application to exit.
Thread dump below indicates there is an executor inside cachingConnectionFactory that hasnt shutdown yet.
"spring-rabbit-deferred-pool-12" #29 prio=5 os_prio=31 tid=0x00007ff61245e000 nid=0x7503 waiting on condition [0x000070000fa57000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x000000076c928300> (a java.util.concurrent.SynchronousQueue$TransferStack)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
"spring-rabbit-deferred-pool-11" #28 prio=5 os_prio=31 tid=0x00007ff616461000 nid=0x7303 waiting on condition [0x000070000f954000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x000000076c928300> (a java.util.concurrent.SynchronousQueue$TransferStack)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
"process reaper" #14 daemon prio=10 os_prio=31 tid=0x00007ff614cd0800 nid=0x590b waiting on condition [0x000070000ec2d000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000006c078c1e0> (a java.util.concurrent.SynchronousQueue$TransferStack)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Anything I can do to exit the application immediately?
I don't think it's related to that thread pool...
public final void destroy() {
super.destroy();
resetConnection();
if (getContextStopped()) {
this.stopped = true;
if (this.channelsExecutor != null) {
this.channelsExecutor.shutdownNow();
}
}
}
shutDownNow() is called after resetting the connection and, clearly, all of the threads are interruptible.
Best guess is something (perhaps network related) is delaying the resetConnection().
Post the complete thread dump while the delay is enountered, perhaps in a GitHub gist, or on pastebin, or similar.
my spring boot app with amqp rabbitMQ hangs without errors in one of the environments. The same config works ok in another.
The thread dump shows,
"main" - Thread t#32
java.lang.Thread.State: WAITING
at java.lang.Object.wait(Native Method)
- waiting on <e62a7fa> (a java.lang.Object)
at java.lang.Object.wait(Object.java:502)
at org.springframework.util.ConcurrencyThrottleSupport.beforeAccess(ConcurrencyThrottleSupport.java:124)
at org.springframework.core.task.SimpleAsyncTaskExecutor$ConcurrencyThrottleAdapter.beforeAccess(SimpleAsyncTaskExecutor.java:243)
at org.springframework.core.task.SimpleAsyncTaskExecutor.execute(SimpleAsyncTaskExecutor.java:184)
at org.springframework.core.task.SimpleAsyncTaskExecutor.execute(SimpleAsyncTaskExecutor.java:167)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.doStart(SimpleMessageListenerContainer.java:815)
- locked <5d300728> (a java.lang.Object)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.start(AbstractMessageListenerContainer.java:550)
at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:173)
at org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:51)
at org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:346)
at org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:149)
at org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:112)
at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:874)
at org.springframework.boot.context.embedded.EmbeddedWebApplicationContext.finishRefresh(EmbeddedWebApplicationContext.java:144)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:544)
- locked <1c4259b2> (a java.lang.Object)
at org.springframework.boot.context.embedded.EmbeddedWebApplicationContext.refresh(EmbeddedWebApplicationContext.java:122)
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:761)
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:371)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:315)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1186)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1175)
and then all the AMQP consumer threads are in BLOCKED state,
"AMQPConsumerThread_20" - Thread t#100
java.lang.Thread.State: BLOCKED
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.isActive(SimpleMessageListenerContainer.java:870)
- waiting to lock <5d300728> (a java.lang.Object) owned by "main" t#32
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.access$1000(SimpleMessageListenerContainer.java:95)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer.run(SimpleMessageListenerContainer.java:1310)
at org.springframework.core.task.SimpleAsyncTaskExecutor$ConcurrencyThrottlingRunnable.run(SimpleAsyncTaskExecutor.java:268)
at java.lang.Thread.run(Thread.java:745)
Locked ownable synchronizers:
- None
The SpringBoot app then waits with the last line on console,
2017-02-08T17:48:26.91-0500 [APP/0] OUT 2017-02-08 17:48:26 [AMQPConsumerThread_1] INFO o.s.a.r.c.CachingConnectionFactory -
Created new connection: SimpleConnection#64f05682
[delegate=amqp://b489be6b-b6e5-41b5-8f31-3be3acac4518#x.y.z.w:5672/daad99cf-fa24-40fe-b0d1-f9312fb583be, localPort= 57509]
EDIT
when I drop the consumer count down to 1, the BLOCKED state goes away and the main thread state is as follows,
"main" - Thread t#32
java.lang.Thread.State: WAITING
at java.lang.Object.wait(Native Method)
- waiting on <79908042> (a java.lang.Object)
at java.lang.Object.wait(Object.java:502)
at org.springframework.util.ConcurrencyThrottleSupport.beforeAccess(ConcurrencyThrottleSupport.java:124)
at org.springframework.core.task.SimpleAsyncTaskExecutor$ConcurrencyThrottleAdapter.beforeAccess(SimpleAsyncTaskExecutor.java:243)
at org.springframework.core.task.SimpleAsyncTaskExecutor.execute(SimpleAsyncTaskExecutor.java:184)
at org.springframework.core.task.SimpleAsyncTaskExecutor.execute(SimpleAsyncTaskExecutor.java:167)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.doStart(SimpleMessageListenerContainer.java:815)
- locked <4f3dcc9> (a java.lang.Object)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.start(AbstractMessageListenerContainer.java:550)
at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:173)
at org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:51)
at org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:346)
at org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:149)
at org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:112)
at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:874)
at org.springframework.boot.context.embedded.EmbeddedWebApplicationContext.finishRefresh(EmbeddedWebApplicationContext.java:144)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:544)
- locked <1a60fd96> (a java.lang.Object)
at org.springframework.boot.context.embedded.EmbeddedWebApplicationContext.refresh(EmbeddedWebApplicationContext.java:122)
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:761)
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:371)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:315)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1186)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1175)
But the app still does not start.
the rabbitmq webconsole shows the consumer connected with the last line in logs,
2017-02-08T18:36:09.99-0500 [APP/0] OUT 2017-02-08 18:36:09 [AMQPConsumerThread_1] INFO o.s.a.r.c.CachingConnectionFactory - Created new connection: SimpleConnection#39271079
[delegate=amqp://b489be6b-b6e5-41b5-8f31-3be3acac4518#10.146.54.74:5672/daad99cf-fa24-40fe-b0d1-f9312fb583be, localPort= 32880]
what am I missing?
thanks
This...
"main" - Thread t#32
java.lang.Thread.State: WAITING
at java.lang.Object.wait(Native Method)
- waiting on <79908042> (a java.lang.Object)
at java.lang.Object.wait(Object.java:502)
at org.springframework.util.ConcurrencyThrottleSupport.beforeAccess(ConcurrencyThrottleSupport.java:124)
at org.springframework.core.task.SimpleAsyncTaskExecutor$ConcurrencyThrottleAdapter.beforeAccess(SimpleAsyncTaskExecutor.java:243)
at org.springframework.core.task.SimpleAsyncTaskExecutor.execute(SimpleAsyncTaskExecutor.java:184)
Indicates the SimpleAsyncTaskExecutor is at its thread limit (setConcurrencyLimit). This check is only performed if you have set a limit.
This won't proceed until some task terminates. Since the consumer threads are long-lived, that will likely not happen.
This question already has an answer here:
omnifaces starting before openwebbeans 1.5.0?
(1 answer)
Closed 7 years ago.
I'm converting my existing JSF application with Primefaces on TomCat7 to be using CDI and Omnifaces 2.0. I already used a previous version of Omnifaces in the webapp before.
TomEE is not an option and after multiple tries with Weld, I got started with OpenWebbeans. However I receive an error at server startup.
Any help on this?
My configuration (JSF 2.2.8):
pom.xml
Note: Adding openwebbeans-jsf did not solve anything
<dependency>
<groupId>org.apache.geronimo.specs</groupId>
<artifactId>geronimo-atinject_1.0_spec</artifactId>
<version>1.0</version>
</dependency>
<dependency>
<groupId>org.apache.geronimo.specs</groupId>
<artifactId>geronimo-jcdi_1.1_spec</artifactId>
<version>1.0</version>
</dependency>
<dependency>
<groupId>org.apache.geronimo.specs</groupId>
<artifactId>geronimo-interceptor_1.2_spec</artifactId>
<version>1.0</version>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>org.apache.geronimo.specs</groupId>
<artifactId>geronimo-annotation_1.2_spec</artifactId>
<version>1.0</version>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>org.apache.openwebbeans</groupId>
<artifactId>openwebbeans-spi</artifactId>
<version>1.5.0</version>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>org.apache.openwebbeans</groupId>
<artifactId>openwebbeans-impl</artifactId>
<version>1.5.0</version>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>org.apache.openwebbeans</groupId>
<artifactId>openwebbeans-web</artifactId>
<version>1.5.0</version>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>org.apache.openwebbeans</groupId>
<artifactId>openwebbeans-tomcat7</artifactId>
<version>1.5.0</version>
<scope>compile</scope>
</dependency>
context.xml
<Context>
<Resource name="BeanManager"
auth="Container"
type="javax.enterprise.inject.spi.BeanManager"
factory="org.apache.webbeans.container.ManagerObjectFactory" />
</Context>
I am getting the following error:
apr 25, 2015 4:50:34 PM org.apache.catalina.core.StandardContext listenerStart
SEVERE: Exception sending context initialized event to listener instance of class org.omnifaces.ApplicationListener
java.lang.NullPointerException
at org.omnifaces.ApplicationListener.contextInitialized(ApplicationListener.java:55)
at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4992)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5490)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1575)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1565)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
apr 25, 2015 4:50:34 PM com.sun.faces.config.ConfigureListener contextInitialized
INFO: Initializing Mojarra 2.2.8 ( 20140814-1418 https://svn.java.net/svn/mojarra~svn/tags/2.2.8#13507) for context '/admin'
apr 25, 2015 4:50:34 PM com.sun.faces.spi.InjectionProviderFactory createInstance
INFO: JSF1048: PostConstruct/PreDestroy annotations present. ManagedBeans methods marked with these annotations will have said annotations processed.
apr 25, 2015 4:50:35 PM com.sun.faces.config.ConfigureListener contextInitialized
SEVERE: Critical error during deployment:
com.sun.faces.config.ConfigurationException: Factory 'javax.faces.lifecycle.ClientWindowFactory' was not configured properly.
at com.sun.faces.config.processor.FactoryConfigProcessor.verifyFactoriesExist(FactoryConfigProcessor.java:330)
at com.sun.faces.config.processor.FactoryConfigProcessor.process(FactoryConfigProcessor.java:236)
at com.sun.faces.config.ConfigManager.initialize(ConfigManager.java:435)
at com.sun.faces.config.ConfigureListener.contextInitialized(ConfigureListener.java:221)
at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4990)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5490)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1575)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1565)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: javax.faces.FacesException: com.sun.faces.lifecycle.ClientWindowFactoryImpl
at javax.faces.FactoryFinderInstance.getImplGivenPreviousImpl(FactoryFinderInstance.java:417)
at javax.faces.FactoryFinderInstance.getImplementationInstance(FactoryFinderInstance.java:248)
at javax.faces.FactoryFinderInstance.getFactory(FactoryFinderInstance.java:488)
at javax.faces.FactoryFinder.getFactory(FactoryFinder.java:287)
at com.sun.faces.config.processor.FactoryConfigProcessor.verifyFactoriesExist(FactoryConfigProcessor.java:328)
... 12 more
Caused by: java.lang.UnsupportedOperationException: java.lang.reflect.InvocationTargetException
at org.omnifaces.config.BeanManager.getReference(BeanManager.java:154)
at org.omnifaces.application.OmniApplication.<init>(OmniApplication.java:69)
at org.omnifaces.application.OmniApplicationFactory.createOmniApplication(OmniApplicationFactory.java:88)
at org.omnifaces.application.OmniApplicationFactory.getApplication(OmniApplicationFactory.java:54)
at com.sun.faces.application.InjectionApplicationFactory.getApplication(InjectionApplicationFactory.java:93)
at com.sun.faces.config.InitFacesContext.getApplication(InitFacesContext.java:144)
at com.sun.faces.lifecycle.ClientWindowFactoryImpl.<init>(ClientWindowFactoryImpl.java:62)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
at java.lang.reflect.Constructor.newInstance(Unknown Source)
at java.lang.Class.newInstance(Unknown Source)
at javax.faces.FactoryFinderInstance.getImplGivenPreviousImpl(FactoryFinderInstance.java:415)
... 16 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.omnifaces.config.BeanManager.getReference(BeanManager.java:143)
... 28 more
Caused by: java.lang.IllegalStateException: It's not allowed to call getBeans(Type, Annotation...) before AfterBeanDiscovery
at org.apache.webbeans.container.InjectableBeanManager.checkAfterBeanDiscoveryProcessed(InjectableBeanManager.java:402)
at org.apache.webbeans.container.InjectableBeanManager.getBeans(InjectableBeanManager.java:121)
... 33 more
[EDIT]
I followed the blog on the installation of Weld. I added SP1 from Weld before, but I retried now with Final version to catch the necessary traces.
Just adding the weld-servlet dependency gives me the following stacktrace:
25 apr 2015 21:37:10 INFO servletWeldServlet - WELD-ENV-001008: Initialize Weld using ServletContainerInitializer
25 apr 2015 21:37:11 INFO Version - WELD-000900: 2.2.10 (Final)
25 apr 2015 21:37:11 INFO Bootstrap - WELD-ENV-000020: Using jandex for bean discovery
25 apr 2015 21:37:13 INFO Bootstrap - WELD-000101: Transactional services not available. Injection of #Inject UserTransaction not available. Transactional observers will be invoked synchronously.
25 apr 2015 21:37:14 WARN Interceptor - WELD-001700: Interceptor annotation class javax.ejb.PostActivate not found, interception based on it is not enabled
25 apr 2015 21:37:14 WARN Interceptor - WELD-001700: Interceptor annotation class javax.ejb.PrePassivate not found, interception based on it is not enabled
25 apr 2015 21:37:15 WARN Event - WELD-000411: Observer method [BackedAnnotatedMethod] public org.omnifaces.VetoAnnotatedTypeExtension.processAnnotatedType(#Observes ProcessAnnotatedType<Object>) receives events for all annotated types. Consider restricting events using #WithAnnotations or a generic type with bounds.
apr 25, 2015 9:37:15 PM org.apache.catalina.core.ContainerBase startInternal
SEVERE: A child container failed during start
java.util.concurrent.ExecutionException: org.apache.catalina.LifecycleException: Failed to start component [StandardEngine[Catalina].StandardHost[localhost].StandardContext[/admin]]
at java.util.concurrent.FutureTask.report(Unknown Source)
at java.util.concurrent.FutureTask.get(Unknown Source)
at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:1123)
at org.apache.catalina.core.StandardHost.startInternal(StandardHost.java:816)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1575)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1565)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: org.apache.catalina.LifecycleException: Failed to start component [StandardEngine[Catalina].StandardHost[localhost].StandardContext[/admin]]
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:154)
... 6 more
Caused by: org.jboss.weld.exceptions.DeploymentException: org.jboss.jandex.ClassInfo.hasNoArgsConstructor()Z
at org.jboss.weld.executor.AbstractExecutorServices.checkForExceptions(AbstractExecutorServices.java:66)
at org.jboss.weld.executor.AbstractExecutorServices.invokeAllAndCheckForExceptions(AbstractExecutorServices.java:43)
at org.jboss.weld.executor.AbstractExecutorServices.invokeAllAndCheckForExceptions(AbstractExecutorServices.java:51)
at org.jboss.weld.bootstrap.ConcurrentBeanDeployer.addClasses(ConcurrentBeanDeployer.java:62)
at org.jboss.weld.bootstrap.BeanDeployment.createClasses(BeanDeployment.java:203)
at org.jboss.weld.bootstrap.WeldStartup.startInitialization(WeldStartup.java:372)
at org.jboss.weld.bootstrap.WeldBootstrap.startInitialization(WeldBootstrap.java:76)
at org.jboss.weld.environment.servlet.WeldServletLifecycle.initialize(WeldServletLifecycle.java:141)
at org.jboss.weld.environment.servlet.EnhancedListener.onStartup(EnhancedListener.java:61)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5479)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
... 6 more
Caused by: java.lang.NoSuchMethodError: org.jboss.jandex.ClassInfo.hasNoArgsConstructor()Z
at org.jboss.weld.environment.deployment.discovery.jandex.JandexClassFileInfo.<init>(JandexClassFileInfo.java:65)
at org.jboss.weld.environment.deployment.discovery.jandex.JandexClassFileServices.getClassFileInfo(JandexClassFileServices.java:82)
at org.jboss.weld.bootstrap.FastAnnotatedTypeLoader.loadAnnotatedType(FastAnnotatedTypeLoader.java:61)
at org.jboss.weld.bootstrap.BeanDeployer.addClass(BeanDeployer.java:97)
at org.jboss.weld.bootstrap.ConcurrentBeanDeployer$1.doWork(ConcurrentBeanDeployer.java:65)
at org.jboss.weld.bootstrap.ConcurrentBeanDeployer$1.doWork(ConcurrentBeanDeployer.java:62)
at org.jboss.weld.executor.IterativeWorkerTaskFactory$1.call(IterativeWorkerTaskFactory.java:60)
at org.jboss.weld.executor.IterativeWorkerTaskFactory$1.call(IterativeWorkerTaskFactory.java:53)
... 4 more
I had this error before, I had to add Jandex to the classpath, version 1.2.4.Final
With this library added it seems (and seemed) like the server and application is starting correctly, but it isn't. I never get the "application started" message.
I'm running TomCat through Eclipse localhost on a Windows 7, also tried on Mac but same behaviour. Eclipse never stops showing the message "Starting Tomcat v7.0 ...".
Looking at the threaddump doesn't learn me anything more:
c:\Program Files\Java\jdk1.7.0_67\bin>jstack -l 4476
2015-04-25 22:11:41
Full thread dump Java HotSpot(TM) Client VM (24.65-b04 mixed mode):
"Hikari Housekeeping Timer (pool HikariPool-0)" daemon prio=6 tid=0x4e5dd000 nid=0x126c waiting on condition [0x5013f000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1bc5aed0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Locked ownable synchronizers:
- None
"Abandoned connection cleanup thread" daemon prio=6 tid=0x4f205400 nid=0x119c in Object.wait() [0x4e48f000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x1bbe5af8> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(Unknown Source)
- locked <0x1bbe5af8> (a java.lang.ref.ReferenceQueue$Lock)
at com.mysql.jdbc.AbandonedConnectionCleanupThread.run(AbandonedConnectionCleanupThread.java:41)
Locked ownable synchronizers:
- None
"weld-worker-1" daemon prio=6 tid=0x4eb2f800 nid=0x11bc waiting on condition [0x4ed9f000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x1b5a8570> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(Unknown Source)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)
at java.util.concurrent.LinkedBlockingQueue.take(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Locked ownable synchronizers:
- None
"localhost-startStop-1" daemon prio=6 tid=0x4e1b2400 nid=0xb18 runnable [0x03dce000]
java.lang.Thread.State: RUNNABLE
at org.omnifaces.application.OmniApplicationFactory.createOmniApplication(OmniApplicationFactory.java:84)
at org.omnifaces.application.OmniApplicationFactory.getApplication(OmniApplicationFactory.java:54)
at com.sun.faces.application.InjectionApplicationFactory.getApplication(InjectionApplicationFactory.java:93)
at com.sun.faces.config.InitFacesContext.getApplication(InitFacesContext.java:144)
at com.sun.faces.lifecycle.ClientWindowFactoryImpl.<init>(ClientWindowFactoryImpl.java:62)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
at java.lang.reflect.Constructor.newInstance(Unknown Source)
at java.lang.Class.newInstance(Unknown Source)
at javax.faces.FactoryFinderInstance.getImplGivenPreviousImpl(FactoryFinderInstance.java:415)
at javax.faces.FactoryFinderInstance.getImplementationInstance(FactoryFinderInstance.java:248)
at javax.faces.FactoryFinderInstance.getFactory(FactoryFinderInstance.java:488)
at javax.faces.FactoryFinder.getFactory(FactoryFinder.java:287)
at com.sun.faces.config.processor.FactoryConfigProcessor.verifyFactoriesExist(FactoryConfigProcessor.java:328)
at com.sun.faces.config.processor.FactoryConfigProcessor.process(FactoryConfigProcessor.java:236)
at com.sun.faces.config.ConfigManager.initialize(ConfigManager.java:435)
at com.sun.faces.config.ConfigureListener.contextInitialized(ConfigureListener.java:221)
at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4990)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5490)
- locked <0x1953f410> (a org.apache.catalina.core.StandardContext)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
- locked <0x1953f410> (a org.apache.catalina.core.StandardContext)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1575)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1565)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Locked ownable synchronizers:
- <0x0601e4e8> (a java.util.concurrent.locks.ReentrantReadWriteLock$FairSync)
- <0x195beea8> (a java.util.concurrent.ThreadPoolExecutor$Worker)
"Catalina-startStop-1" daemon prio=6 tid=0x4e1b1c00 nid=0x4c4 waiting on condition [0x4e4ff000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x195e0380> (a java.util.concurrent.FutureTask)
at java.util.concurrent.locks.LockSupport.park(Unknown Source)
at java.util.concurrent.FutureTask.awaitDone(Unknown Source)
at java.util.concurrent.FutureTask.get(Unknown Source)
at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:1123)
- locked <0x1951a790> (a org.apache.catalina.core.StandardHost)
at org.apache.catalina.core.StandardHost.startInternal(StandardHost.java:816)
- locked <0x1951a790> (a org.apache.catalina.core.StandardHost)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
- locked <0x1951a790> (a org.apache.catalina.core.StandardHost)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1575)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1565)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Locked ownable synchronizers:
- <0x195bf000> (a java.util.concurrent.ThreadPoolExecutor$Worker)
"GC Daemon" daemon prio=2 tid=0x4e15bc00 nid=0xc90 in Object.wait() [0x4e42f000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x19570e38> (a sun.misc.GC$LatencyLock)
at sun.misc.GC$Daemon.run(Unknown Source)
- locked <0x19570e38> (a sun.misc.GC$LatencyLock)
Locked ownable synchronizers:
- None
"Service Thread" daemon prio=6 tid=0x00dcc800 nid=0x7e0 runnable [0x00000000]
java.lang.Thread.State: RUNNABLE
Locked ownable synchronizers:
- None
"C1 CompilerThread0" daemon prio=10 tid=0x00dbf000 nid=0x1248 waiting on condition [0x00000000]
java.lang.Thread.State: RUNNABLE
Locked ownable synchronizers:
- None
"Attach Listener" daemon prio=10 tid=0x00dbdc00 nid=0x7b0 waiting on condition [0x00000000]
java.lang.Thread.State: RUNNABLE
Locked ownable synchronizers:
- None
"Signal Dispatcher" daemon prio=10 tid=0x0070e400 nid=0x980 runnable [0x00000000]
java.lang.Thread.State: RUNNABLE
Locked ownable synchronizers:
- None
"Finalizer" daemon prio=8 tid=0x00d93c00 nid=0x11c0 in Object.wait() [0x007ef000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x193c2cc0> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(Unknown Source)
- locked <0x193c2cc0> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(Unknown Source)
at java.lang.ref.Finalizer$FinalizerThread.run(Unknown Source)
Locked ownable synchronizers:
- None
"Reference Handler" daemon prio=10 tid=0x0070c800 nid=0x1618 in Object.wait() [0x00d8f000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x193c2d50> (a java.lang.ref.Reference$Lock)
at java.lang.Object.wait(Object.java:503)
at java.lang.ref.Reference$ReferenceHandler.run(Unknown Source)
- locked <0x193c2d50> (a java.lang.ref.Reference$Lock)
Locked ownable synchronizers:
- None
"main" prio=6 tid=0x00614800 nid=0x50c waiting on condition [0x0055f000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x195bef98> (a java.util.concurrent.FutureTask)
at java.util.concurrent.locks.LockSupport.park(Unknown Source)
at java.util.concurrent.FutureTask.awaitDone(Unknown Source)
at java.util.concurrent.FutureTask.get(Unknown Source)
at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:1123)
- locked <0x194dbaf8> (a org.apache.catalina.core.StandardEngine)
at org.apache.catalina.core.StandardEngine.startInternal(StandardEngine.java:300)
- locked <0x194dbaf8> (a org.apache.catalina.core.StandardEngine)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
- locked <0x194dbaf8> (a org.apache.catalina.core.StandardEngine)
at org.apache.catalina.core.StandardService.startInternal(StandardService.java:443)
- locked <0x194dbaf8> (a org.apache.catalina.core.StandardEngine)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
- locked <0x194c6188> (a org.apache.catalina.core.StandardService)
at org.apache.catalina.core.StandardServer.startInternal(StandardServer.java:739)
- locked <0x19542560> (a [Lorg.apache.catalina.Service;)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
- locked <0x19434bd0> (a org.apache.catalina.core.StandardServer)
at org.apache.catalina.startup.Catalina.start(Catalina.java:689)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:321)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:455)
Locked ownable synchronizers:
- None
"VM Thread" prio=10 tid=0x00709000 nid=0x160c runnable
"VM Periodic Task Thread" prio=10 tid=0x00de7400 nid=0x1460 waiting on condition
JNI global references: 221
That's non portable behaviour in Omnifaces. Bauke needs to work on this code.
The CDI spec says nothing about the CDI container having to be available in ServletContainerInitializer. So it is simply not guaranteed in a portable way. It will also not work on Google App Engine, jetty, etc...
I would simply remove that check code from Omnifaces and your application will 'just work' (tm). It's really a false positive...
I have a VPS with Centos running with the following details:
[root#XXXXXXX~]# uname -a
Linux xxxxxxxx2.6.32-042stab055.10 #1 SMP Thu May 10 15:38:32 MSD 2012 i686 i686 i386 GNU/Linux
I am trying to run the ecommerce system shopizer from the tomcat installed on it.
I had tried to build it on the VPS but did not succeed so I built it else where and copied the war files on the VPS's tomcat.
The issue I am facing now and during the build too was that the build was hanging when i ran the built ant scripts. now when i am launching the tomcat server whose details are as follows :
Server version: Apache Tomcat/6.0.35
Server built: Nov 28 2011 11:20:06
Server number: 6.0.35.0
OS Name: Linux
OS Version: 2.6.32-042stab055.10
Architecture: i386
JVM Version: 1.6.0_24-b24
JVM Vendor: Sun Microsystems Inc.
the server hangs up at start and when I take thread dump (both in this case and the build case) I get the following :
"GC Daemon" daemon prio=10 tid=0xa09fd000 nid=0x36ef in Object.wait() [0xa08f7000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0xa6c35a88> (a sun.misc.GC$LatencyLock)
at sun.misc.GC$Daemon.run(GC.java:117)
- locked <0xa6c35a88> (a sun.misc.GC$LatencyLock)
"Low Memory Detector" daemon prio=10 tid=0xb7686000 nid=0x36ed runnable [0x00000000]
java.lang.Thread.State: RUNNABLE
"C1 CompilerThread0" daemon prio=10 tid=0xb7684000 nid=0x36ec runnable [0x00000000]
java.lang.Thread.State: RUNNABLE
"Signal Dispatcher" daemon prio=10 tid=0xb7682800 nid=0x36eb waiting on condition [0x00000000]
java.lang.Thread.State: RUNNABLE
"Finalizer" daemon prio=10 tid=0xb7673000 nid=0x36ea in Object.wait() [0xa0ffe000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0xa6ad0b58> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:133)
- locked <0xa6ad0b58> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:149)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:177)
"Reference Handler" daemon prio=10 tid=0xb7671800 nid=0x36e9 in Object.wait() [0xa1198000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0xa6ad0a58> (a java.lang.ref.Reference$Lock)
at java.lang.Object.wait(Object.java:502)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133)
- locked <0xa6ad0a58> (a java.lang.ref.Reference$Lock)
"main" prio=10 tid=0xb7605c00 nid=0x36e7 runnable [0xb775d000]
java.lang.Thread.State: RUNNABLE
at java.lang.Byte$ByteCache.<clinit>(Byte.java:79)
at java.lang.Byte.valueOf(Byte.java:102)
- waiting on <0xa6ad0a58> (a java.lang.ref.Reference$Lock)
at java.lang.Object.wait(Object.java:502)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133)
- locked <0xa6ad0a58> (a java.lang.ref.Reference$Lock)
"main" prio=10 tid=0xb7605c00 nid=0x36e7 runnable [0xb775d000]
java.lang.Thread.State: RUNNABLE
at java.lang.Byte$ByteCache.<clinit>(Byte.java:79)
at java.lang.Byte.valueOf(Byte.java:102)
at com.opensymphony.xwork2.conversion.impl.DefaultTypeConverter.<init>(DefaultTypeConverter.java:59)
at com.opensymphony.xwork2.conversion.impl.XWorkConverter.<init>(XWorkConverter.java:186)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:532)
at com.opensymphony.xwork2.inject.ContainerImpl$ConstructorInjector.construct(ContainerImpl.java:419)
at com.opensymphony.xwork2.inject.ContainerBuilder$5.create(ContainerBuilder.java:207)
at com.opensymphony.xwork2.inject.Scope$2$1.create(Scope.java:51)
- locked <0xa18cf408> (a com.opensymphony.xwork2.inject.ContainerImpl)
at com.opensymphony.xwork2.inject.ContainerImpl$ParameterInjector.inject(ContainerImpl.java:462)
at com.opensymphony.xwork2.inject.ContainerImpl.getParameters(ContainerImpl.java:477)
at com.opensymphony.xwork2.inject.ContainerImpl.access$000(ContainerImpl.java:34)
at com.opensymphony.xwork2.inject.ContainerImpl$MethodInjector.inject(ContainerImpl.java:293)
at com.opensymphony.xwork2.inject.ContainerImpl$ConstructorInjector.construct(ContainerImpl.java:431)
at com.opensymphony.xwork2.inject.ContainerBuilder$5.create(ContainerBuilder.java:207)
at com.opensymphony.xwork2.inject.Scope$2$1.create(Scope.java:51)
- locked <0xa18cf408> (a com.opensymphony.xwork2.inject.ContainerImpl)
at com.opensymphony.xwork2.inject.ContainerImpl$ParameterInjector.inject(ContainerImpl.java:462)
at com.opensymphony.xwork2.inject.ContainerImpl.getParameters(ContainerImpl.java:477)
at com.opensymphony.xwork2.inject.ContainerImpl.access$000(ContainerImpl.java:34)
at com.opensymphony.xwork2.inject.ContainerImpl$MethodInjector.inject(ContainerImpl.java:293)
at com.opensymphony.xwork2.inject.ContainerImpl$ConstructorInjector.construct(ContainerImpl.java:431)
at com.opensymphony.xwork2.inject.ContainerBuilder$5.create(ContainerBuilder.java:207)
at com.opensymphony.xwork2.inject.Scope$2$1.create(Scope.java:51)
- locked <0xa18cf408> (a com.opensymphony.xwork2.inject.ContainerImpl)
at com.opensymphony.xwork2.inject.ContainerImpl$ParameterInjector.inject(ContainerImpl.java:462)
at com.opensymphony.xwork2.inject.ContainerImpl.getParameters(ContainerImpl.java:477)
at com.opensymphony.xwork2.inject.ContainerImpl.access$000(ContainerImpl.java:34)
at com.opensymphony.xwork2.inject.ContainerImpl$MethodInjector.inject(ContainerImpl.java:293)
at com.opensymphony.xwork2.inject.ContainerImpl$ConstructorInjector.construct(ContainerImpl.java:431)
at com.opensymphony.xwork2.inject.ContainerBuilder$5.create(ContainerBuilder.java:207)
at com.opensymphony.xwork2.inject.Scope$2$1.create(Scope.java:51)
- locked <0xa18cf408> (a com.opensymphony.xwork2.inject.ContainerImpl)
at com.opensymphony.xwork2.inject.ContainerBuilder$3.create(ContainerBuilder.java:93)
at com.opensymphony.xwork2.inject.ContainerBuilder$7.call(ContainerBuilder.java:487)
at com.opensymphony.xwork2.inject.ContainerBuilder$7.call(ContainerBuilder.java:484)
at com.opensymphony.xwork2.inject.ContainerImpl.callInContext(ContainerImpl.java:574)
at com.opensymphony.xwork2.inject.ContainerBuilder.create(ContainerBuilder.java:484)
at com.opensymphony.xwork2.config.impl.DefaultConfiguration.createBootstrapContainer(DefaultConfiguration.java:252)
at com.opensymphony.xwork2.config.impl.DefaultConfiguration.reloadContainer(DefaultConfiguration.java:193)
- locked <0xa37c05b0> (a com.opensymphony.xwork2.config.impl.DefaultConfiguration)
....
....
..
"VM Thread" prio=10 tid=0xb766d800 nid=0x36e8 runnable
"VM Periodic Task Thread" prio=10 tid=0xb7688400 nid=0x36ee waiting on condition
JNI global references: 911
Heap
def new generation total 39424K, used 5740K [0xa1580000, 0xa4040000, 0xa6ad0000)
eden space 35072K, 12% used [0xa1580000, 0xa19a8e88, 0xa37c0000)
from space 4352K, 34% used [0xa37c0000, 0xa3932570, 0xa3c00000)
to space 4352K, 0% used [0xa3c00000, 0xa3c00000, 0xa4040000)
tenured generation total 87424K, used 1432K [0xa6ad0000, 0xac030000, 0xb1580000)
the space 87424K, 1% used [0xa6ad0000, 0xa6c36038, 0xa6c36200, 0xac030000)
compacting perm gen total 12288K, used 11825K [0xb1580000, 0xb2180000, 0xb5580000)
the space 12288K, 96% used [0xb1580000, 0xb210c588, 0xb210c600, 0xb2180000)
No shared spaces configured.
What can I do to resolve the issue or is it a bug/issue with java running on VPS??
B) the jvm sometimes crashes on the same VPS with the following error :
[root#xxxxxx ~]# java
Error occurred during initialization of VM
Could not reserve enough space for object heap
Could not create the Java virtual machine.
[root#xxxxxx ~]# free
total used free shared buffers cached
Mem: 1155072 561320 593752 0 0 317124
-/+ buffers/cache: 244196 910876
Swap: 0 0 0
Very strange to me, can anyone explain me this behaviour.
I suggest that you try to update your operating system and your JVM.
I think this crash is due to a bug in your actual configuration.
We have an intermittent problem with slaves hanging AFTER the job itself is finished. In the post processing step (?) what we see is that the console log has this line:
Description set: vap_current_iter-2012_03_29_19_01_03
And then nothing. Usually, it will look like this:
Description set: prod_pull-2012_03_28_19_01_03
Notifying upstream build armada_Launch_prod_pull #13 of job completion
Project armada_Launch_prod_pull still waiting for 1 builds to complete
Notifying upstream projects of job completion
Notifying upstream of completion: armada_Launch_prod_pull #13
Finished: SUCCESS
I setup a logger for hudson.model.Run, and it currently has this :
at java.lang.Thread.run(Thread.java:619)
Mar 30, 2012 12:44:00 PM hudson.model.Run run
INFO: galleon_allUnit #1134 main build action completed: SUCCESS
Mar 30, 2012 12:44:00 PM hudson.model.Run setResult
FINE: galleon_allUnit #1134 : result is set to SUCCESS
java.lang.Exception
at hudson.model.Run.setResult(Run.java:352)
at hudson.model.Run.run(Run.java:1410)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
at hudson.model.ResourceController.execute(ResourceController.java:88)
at hudson.model.Executor.run(Executor.java:238)
Repeated for every hung slave.
The main hudson log doesn't have any additional information.
Disconnecting the slave has no effect.
Trying to do an orderly shutdown of Jenkins has no effect (jenkins actually appears to hang on shutdown).
The only way we have found to recover is to kill -9 the tomcat process.
The tread dump for one of the slaves (they are all the same) is:
Thread Dump
Channel reader thread: channel
"Channel reader thread: channel" Id=9 Group=main RUNNABLE (in native)
at java.io.FileInputStream.readBytes(Native Method)
at java.io.FileInputStream.read(FileInputStream.java:199)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
at java.io.BufferedInputStream.read(BufferedInputStream.java:237)
- locked java.io.BufferedInputStream#1ae615a
at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2249)
at java.io.ObjectInputStream$BlockDataInputStream.peek(ObjectInputStream.java:2542)
at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2552)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1297)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
at hudson.remoting.Channel$ReaderThread.run(Channel.java:1030)
main
"main" Id=1 Group=main WAITING on hudson.remoting.Channel#e1d5ea
at java.lang.Object.wait(Native Method)
- waiting on hudson.remoting.Channel#e1d5ea
at java.lang.Object.wait(Object.java:485)
at hudson.remoting.Channel.join(Channel.java:766)
at hudson.remoting.Launcher.main(Launcher.java:420)
at hudson.remoting.Launcher.runWithStdinStdout(Launcher.java:366)
at hudson.remoting.Launcher.run(Launcher.java:206)
at hudson.remoting.Launcher.main(Launcher.java:168)
Ping thread for channel hudson.remoting.Channel#e1d5ea:channel
"Ping thread for channel hudson.remoting.Channel#e1d5ea:channel" Id=10 Group=main TIMED_WAITING
at java.lang.Thread.sleep(Native Method)
at hudson.remoting.PingThread.run(PingThread.java:86)
Pipe writer thread: channel
"Pipe writer thread: channel" Id=12 Group=main WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject#14263ed
at sun.misc.Unsafe.park(Native Method)
- waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject#14263ed
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
pool-1-thread-267
"pool-1-thread-267" Id=285 Group=main RUNNABLE
at sun.management.ThreadImpl.dumpThreads0(Native Method)
at sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:374)
at hudson.Functions.getThreadInfos(Functions.java:872)
at hudson.util.RemotingDiagnostics$GetThreadDump.call(RemotingDiagnostics.java:93)
at hudson.util.RemotingDiagnostics$GetThreadDump.call(RemotingDiagnostics.java:89)
at hudson.remoting.UserRequest.perform(UserRequest.java:118)
at hudson.remoting.UserRequest.perform(UserRequest.java:48)
at hudson.remoting.Request$2.run(Request.java:287)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Number of locked synchronizers = 1
- java.util.concurrent.locks.ReentrantLock$NonfairSync#1186f88
Finalizer
"Finalizer" Id=3 Group=system WAITING on java.lang.ref.ReferenceQueue$Lock#1798fdd
at java.lang.Object.wait(Native Method)
- waiting on java.lang.ref.ReferenceQueue$Lock#1798fdd
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:132)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)
Reference Handler
"Reference Handler" Id=2 Group=system WAITING on java.lang.ref.Reference$Lock#1d40442
at java.lang.Object.wait(Native Method)
- waiting on java.lang.ref.Reference$Lock#1d40442
at java.lang.Object.wait(Object.java:485)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
Signal Dispatcher
"Signal Dispatcher" Id=4 Group=system RUNNABLE
Any ideas on how to better recover or prevent this would be greatly appreciated.
We honestly just wrote a script that restarts jenkins every night at 4pm. We found that our breakages were happening at 3am give or take a half hour or so. Since restarting the server at this time, we haven't seen any further hangs. This is a way to prevent it as you asked, though it doesn't "Fix" the problem obviously!