The title of the exception is "java.util.concurrent.TimeoutException: Timed out(5000) waiting for operation"!
How can I set a approprate time for not exceeding the time out limit?!
The whole exeption stack is:
**java.util.concurrent.TimeoutException: Timed out waiting for operation
at net.rubyeye.xmemcached.XMemcachedClient.getMulti0(XMemcachedClient.java:1155)
at net.rubyeye.xmemcached.XMemcachedClient.get(XMemcachedClient.java:1026)
at net.rubyeye.xmemcached.XMemcachedClient.get(XMemcachedClient.java:1064)
at net.rubyeye.xmemcached.XMemcachedClient.get(XMemcachedClient.java:1051)
......
A lot of reason can cause the ttserver client throw TimeoutException.
In my situation, it was because of the replication to a node which did not exist.So when delete the startup parameters mhost and mport,the problem was fixed.....
Hope this can help someone else....
Related
I have been seeing below error message for quite some time now but could not figure out what leads to the failure.
Error:
concurrent.futures._base.CancelledError: ('sort_index-f23b0553686b95f2d91d4a3fda85f229', 7)
On restart of dask cluster it runs successfully.
If running a dask-cloudprovider ECSCluster or FargateCluster the concurrent.futures._base.CancelledError can result from a long-running step in computation where there is no output (logging or otherwise) to the Client. In these cases, due to the lack of interaction with the client, the scheduler regards itself as "idle" and times out after the configured cloudprovider.ecs.scheduler_timeout period, which defaults to 5 minutes. The CancelledError error message is misleading, but if you look in the logs for the scheduler task itself it will record the idle timeout.
The solution is to set scheduler_timeout to a higher value, either via config or by passing directly to the ECSCluster/FargateCluster constructor.
I'm afraid I'm a little bit of a noob at this, but here goes.
I have an issue with Infinispan and timeout during synch.
I'm running two instances of JBoss AS7 with Infinispan 5.2.7-FINAL. TCP for synch.
Sometimes when deleting an entry from a cache on one of the nodes, the synchronization seems to fail, but checking the log on both nodes tells med the entry is gone and all the required cleanup involved from application side (tearing down Camel routes, etc) has been done. The initiating node claims to time-out when notifying it's peer:
09:53:33,182 INFO [ConfigurationService] (http-/10.217.xx.yy:8080-3) ConfigurationService: deleteSmsAccountConfigurationData: got smsAccountID of SmsAccountIdentifier{smsAccountId=tel:46xxxxxx}
09:53:33,183 INFO [com.ex.control.throttling.controller.ThrottlingService] (http-/10.217.xx.yy:8080-3) ThrottlingService: deleteThrottleForResource: deleting throttle from resource tel:46xxxxxx
09:53:45,232 INFO [com.ex.control.routing.startup.DynamicRouteBootStrap] (OOB-21,smapexs01-54191) DynamnicRouteBootStrap: stopRoute: Attempting to stop sendSmppRoute: sendsmpproutetel:46xxxxxx recieveSmppRoute: recievesmpproutetel:46xxxxxx
09:53:45,233 INFO [org.apache.camel.impl.DefaultShutdownStrategy] (OOB-21,smapexs01-54191) Starting to graceful shutdown 1 routes (timeout 300 seconds)
09:53:45,235 INFO [org.apache.camel.impl.DefaultShutdownStrategy] (Camel (camel-2) thread #1 - ShutdownTask) Route: sendsmpproutetel:46xxxxxx shutdown complete, was consuming from: Endpoint[direct://tel:46xxxxxx]
09:53:45,235 INFO [org.apache.camel.impl.DefaultShutdownStrategy] (OOB-21,smapexs01-54191) Graceful shutdown of 1 routes completed in 0 seconds
09:53:48,190 ERROR [org.infinispan.interceptors.InvocationContextInterceptor] (http-/10.217.xx.yy:8080-3) ISPN000136: Execution error: org.infinispan.CacheException: org.jgroups.TimeoutException: timeout sending message to smapexs03-38104
On this node it takes less than a second (I guess) to tear it down, on the other node it takes 6 seconds, is there a timeout of say 5 seconds somewhere that I have overseen?
I've searched this forum and others, but have not been able to sort it out.
Like I said, I'm a bit of a noob, I hope this makes sense to you. Anything that points me in the right direction is very welcome!
Thanks,
Peter
I have an open UDP connection that streams video for several hours between two machines on different vlans.
After several hours I get the following exception on the server side (the transmitter):
System.Net.Sockets.SocketException: A blocking operation was
interrupted by a call to WSACancelBlockingCall at
System.Net.Sockets.Socket.Send(Byte[] buffer, Int32 offset, Int32
size, SocketFlags socketFlags)
From that moment on, from time to time (not on every send), I see the following:
System.Net.Sockets.SocketException: A non-blocking socket operation
could not be completed immediately at
System.Net.Sockets.Socket.Send(Byte[] buffer, Int32 offset, Int32
size, SocketFlags socketFlags)
On the client side I see no exception or abnormal behavior.
Is it possible that I get this exception due to a N/W problem, for example, something in the switch?
Any other ideas what can cause these exceptions?
Thanks
I will make a wild guess about WSACancelBlockingCall exception.
Probably you are trying to close the socket from another thread or your socket is getting disposed somehow with garbage collector.
am connectting the MQ with below code. I am able connected to MQ successfully. My case is i place the messages to MQ every 1 min once. After disconnecting the cable i get a ResonCode error but IsConnected property still show true. Is this is the right way to check if the connection is still connected ? Or there any best pratcices around that.
I would like to open the connection when applicaiton is started keep it open for ever.
public static MQQueueManager ConnectMQ()
{
if ((queueManager == null) || (!queueManager.IsConnected)||(queueManager.ReasonCode == 2009))
{
queueManager = new MQQueueManager();
}
return queueManager;
}
The behavior of the WMQ client connection is that when idle it will appear to be connected until an API call fails or the connection times out. So isConnected() will likely report true until a get, put or inquire call is attempted and fails, at which point QMgr will then report disconnected.
The other thing to consider here is that 2009 is not the only code you might get. It happens to be the one you get when the connection is severed but there are connection codes for QMgr shutting down, channel shutting down, and a variety of resource and other errors.
Typically for a requirement to maintain a constant connection you would want to wrap the connect and message processing loop inside a try/catch block nested inside a while statement. When you catch an exception other than an intentional exit, close the objects and QMgr, sleep at least 5 seconds, then loop around to the top of the while. The sleep is crucial because if you get caught in a tight reconnect loop and throw hundreds of connection attempts at the QMgr, you can bring even a mainframe QMgr to its knees.
An alternative is to use a v7 WMQ client and QMgr. With this combination, automatic reconnection is configurable as a channel configuration.
I changed the JTA transaction timeout from admin console and set to 300, even after changing it fails saying JTA transaction unexpectedly rolled back (maybe due to a timeout) with a:
weblogic.transaction.RollbackException: Transaction timed out after 181 seconds`
To make sure whether my changes (timeout value 300) got reflected for that domain or not I checked under domain config.xml it got reflected with 300.
My question is, is there any other place also do I need to update the transaction timeout value and do I need to restart the server ?
Full stack trace after the exception from server below:
Caused by: org.springframework.transaction.UnexpectedRollbackException: JTA transaction unexpectedly rolled back (maybe due to a timeout); nested exception is weblogic.transaction.RollbackException: Transaction
timed out after 180 seconds
BEA1-160A800A149091F72E5E
at org.springframework.transaction.jta.JtaTransactionManager.doCommit(JtaTransactionManager.java:1031)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:709)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:678)
at org.springframework.transaction.interceptor.TransactionAspectSupport.completeTransactionAfterThrowing(TransactionAspectSupport.java:359)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:110)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at $Proxy103.saveRegistryData(Unknown Source)
at gov.cms.pqri.arch.submission.registry.bean.RegDataAccessManager.persistRegistry(RegDataAccessManager.java:54)
... 14 more
Caused by: weblogic.transaction.RollbackException: Transaction timed out after 180 seconds
BEA1-160A800A149091F72E5E
at weblogic.transaction.internal.TransactionImpl.throwRollbackException(TransactionImpl.java:1818)
at weblogic.transaction.internal.ServerTransactionImpl.internalCommit(ServerTransactionImpl.java:333)
at weblogic.transaction.internal.ServerTransactionImpl.commit(ServerTransactionImpl.java:227)
at weblogic.transaction.internal.TransactionManagerImpl.commit(TransactionManagerImpl.java:281)
at org.springframework.transaction.jta.JtaTransactionManager.doCommit(JtaTransactionManager.java:1028)
... 22 more
after changing the stuck Thread Max time to 300 under servers -> configuration -> tuning (tab) from admin console it is getting updated and working fine.
I have also came across this issue and have resolved the same, since this is related to JTA transaction so we need to increase the timeout of JTA as well along with the time out for stuck max thread. Please click on JTA from the weblogic console home and increase the JTA timeout from 30(by default) to 300.
We met same issue on Weblogic 12.1.2 [JTA transaction unexpectedly rolled back (maybe due to a timeout)] after all investigation we found the root cause of the problem.In my opinion it occurs due to huge dataset processing transactional and near the end of the process If an exception is thrown, JTA is rolling back data as expected.But it does not give the details of the error.In our case ,it mostly cause because of the database integrity (e.g we try to insert data a column with smaller size than data.)
In summary,it will be the best way to investigate db logs instead of increasing stuck Thread Max time.Thread max time can be a solution,but not a proper solution for real enterprise systems.
Also this issue discussed on another stackover link and hibernate jira issue
And solution suggested:
This is a default behaviour of Weblogic JTA realization. To obtain
root exception you should set system property
weblogic.transaction.allowOverrideSetRollbackReason to true.
One of the solution is add this line into
/bin/setDomainEnv.cmd:
set JAVA_OPTIONS=%JAVA_OPTIONS% -Dweblogic.transaction.allowOverrideSetRollbackReason=true
I got my JTA timeouts increased by adding jta.properties file into config folder of my app with lines:
com.atomikos.icatch.default_jta_timeout=600000
com.atomikos.icatch.max_timeout=600000