Nitrogen SimpleCache Mutex signkey issue - erlang

Nitrogen seems to be running properly until it starts throwing these warning messages.
It throws these warnings.
=WARNING REPORT==== 2-Jul-2022::10:18:25 ===
SimpleCache: Mutex "{nitrogen,signkey}" timed out after 10 seconds. Something might be
wrong. Trying again.
=WARNING REPORT==== 2-Jul-2022::10:18:39 ===
SimpleCache: Mutex "{nitrogen,{ensure_loaded,dot}}" timed out after 10 seconds.
Something might be wrong. Trying again.
=WARNING REPORT==== 2-Jul-2022::10:19:16 ===
SimpleCache: Mutex "{nitrogen,signkey}" timed out after 10 seconds. Something might be
wrong. Trying again.
The crashe handler crashes with this message
=INFO REPORT==== 2-Jul-2022::10:18:19 ===
Crash Handler Crashed:
{throw,{simple_cache_failed,nitrogen,1000,signkey,#Fun<wf_pickle.0.36987981>},
[{simple_cache,get,5,[{file,"src/simple_cache.erl"},{line,92}]},
{default_cache_handler,get_cached,5,
[{file,"src/handlers/cache/default_cache_handler.erl"},
{line,59}]},
{wf_handler,call,3,[{file,"src/lib/wf_handler.erl"},{line,37}]},
{cache_handler,get_cached,3,
[{file,"src/handlers/cache/cache_handler.erl"},
{line,43}]},
{wf,cache,3,[{file,"src/wf.erl"},{line,430}]},
{wf_pickle,pickle,1,[{file,"src/lib/wf_pickle.erl"},{line,28}]},
{wf_event,generate_postback_script,7,
[{file,"src/lib/wf_event.erl"},{line,72}]},
{action_event,render_action,1,
[{file,"src/actions/action_event.erl"},{line,32}]}]}
I also keep getting this error even though simple cache is already started
=INFO REPORT==== 2-Jul-2022::10:41:41 ===
{error,first_request,
{url,"servers/"},
{exit,
{failure_to_deserialize_page_context,
[{serialized_page_context,
<<"GBqhfsgYvoUIZZt7E4ouOknMW1i9TrEXWG2Gp9Uxrll7lnLlRDsjAku3_QC0CyQ1km8ZLgZgUbY18ehPzkBzvoKLfa1S7LXfQnCdO1seo9NvqEExZP9Cxz...">>},
{suggestion,
"The most common cause of this is that simple_cache is not started. Try running: application:start(simple_cache)."}]},
[{wf_core,deserialize_context,1,[{file,"src/wf_core.erl"},{line,187}]},
{wf_core,run_catched,0,[{file,"src/wf_core.erl"},{line,84}]},
{wf_core,run,0,[{file,"src/wf_core.erl"},{line,23}]},
{yaws_server,deliver_dyn_part,8,
[{file,"src/yaws_server.erl"},{line,2882}]},
{yaws_server,aloop,4,[{file,"src/yaws_server.erl"},{line,1251}]},
{yaws_server,acceptor0,2,[{file,"src/yaws_server.erl"},{line,1074}]},
{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,247}]}]}}
What could be the cause of this?

There is a known bug in earlier versions of SimpleCache where in some cases, a cache request that crashes is not caught by the mutex server.
This has been fixed in the latest versions - what I'd do is pull the most recent simple_cache master (which will pull nitro_cache as a dependency).
simple_cache was renamed nitro_cache a little over a year ago to avoid namespace clashes, with the original simple_cache being converted to be an alias of nitro_cache.

Related

What causes dask job failure with CancelledError exception

I have been seeing below error message for quite some time now but could not figure out what leads to the failure.
Error:
concurrent.futures._base.CancelledError: ('sort_index-f23b0553686b95f2d91d4a3fda85f229', 7)
On restart of dask cluster it runs successfully.
If running a dask-cloudprovider ECSCluster or FargateCluster the concurrent.futures._base.CancelledError can result from a long-running step in computation where there is no output (logging or otherwise) to the Client. In these cases, due to the lack of interaction with the client, the scheduler regards itself as "idle" and times out after the configured cloudprovider.ecs.scheduler_timeout period, which defaults to 5 minutes. The CancelledError error message is misleading, but if you look in the logs for the scheduler task itself it will record the idle timeout.
The solution is to set scheduler_timeout to a higher value, either via config or by passing directly to the ECSCluster/FargateCluster constructor.

what is the causes of Exhausted 2 retries in sidekiq.log

On my sidekiq.log I have a warn message WARN: {:message=>"Exhausted 2 retries"}. I would like to know how to fix that issues.
complete gist at line 16.
The problem is here:
EXECABORT Transaction discarded because of previous errors.
Your Redis instance is broken somehow, most likely out of memory.

How to disable crash dump for the gen_server

There is a gen_server that keeps some sensible information in it's state (password and so on)
Lagger is enabled,
So in case of crash gen_server's state is dump to the crash log like:
yyyy-mm-dd hh:mm:ss =ERROR REPORT====
** Generic server XXX terminating
** Last message in was ...
** When Server state == {state, ...}
** Reason for termination ==
As a result sensible information is written to the log file.
Are there any way to prevent state of the gen_server to be written into the log files/crash dumps?
You could implement the optional format_status callback function. That means that whenever the gen_server crashes, you get the chance to format the state data to your liking before it gets logged, for example by removing sensitive information.
You can add this into your app.config:
{lager, [{error_logger_redirect, false}]}
to prevent lager from redirecting error logs. You should also try to catch error (that causes gen_server to crash) and handle it in some graceful way. Our you can keep passwords salted and just let it crash.

Infinispan synchronization timeout exception

I'm afraid I'm a little bit of a noob at this, but here goes.
I have an issue with Infinispan and timeout during synch.
I'm running two instances of JBoss AS7 with Infinispan 5.2.7-FINAL. TCP for synch.
Sometimes when deleting an entry from a cache on one of the nodes, the synchronization seems to fail, but checking the log on both nodes tells med the entry is gone and all the required cleanup involved from application side (tearing down Camel routes, etc) has been done. The initiating node claims to time-out when notifying it's peer:
09:53:33,182 INFO [ConfigurationService] (http-/10.217.xx.yy:8080-3) ConfigurationService: deleteSmsAccountConfigurationData: got smsAccountID of SmsAccountIdentifier{smsAccountId=tel:46xxxxxx}
09:53:33,183 INFO [com.ex.control.throttling.controller.ThrottlingService] (http-/10.217.xx.yy:8080-3) ThrottlingService: deleteThrottleForResource: deleting throttle from resource tel:46xxxxxx
09:53:45,232 INFO [com.ex.control.routing.startup.DynamicRouteBootStrap] (OOB-21,smapexs01-54191) DynamnicRouteBootStrap: stopRoute: Attempting to stop sendSmppRoute: sendsmpproutetel:46xxxxxx recieveSmppRoute: recievesmpproutetel:46xxxxxx
09:53:45,233 INFO [org.apache.camel.impl.DefaultShutdownStrategy] (OOB-21,smapexs01-54191) Starting to graceful shutdown 1 routes (timeout 300 seconds)
09:53:45,235 INFO [org.apache.camel.impl.DefaultShutdownStrategy] (Camel (camel-2) thread #1 - ShutdownTask) Route: sendsmpproutetel:46xxxxxx shutdown complete, was consuming from: Endpoint[direct://tel:46xxxxxx]
09:53:45,235 INFO [org.apache.camel.impl.DefaultShutdownStrategy] (OOB-21,smapexs01-54191) Graceful shutdown of 1 routes completed in 0 seconds
09:53:48,190 ERROR [org.infinispan.interceptors.InvocationContextInterceptor] (http-/10.217.xx.yy:8080-3) ISPN000136: Execution error: org.infinispan.CacheException: org.jgroups.TimeoutException: timeout sending message to smapexs03-38104
On this node it takes less than a second (I guess) to tear it down, on the other node it takes 6 seconds, is there a timeout of say 5 seconds somewhere that I have overseen?
I've searched this forum and others, but have not been able to sort it out.
Like I said, I'm a bit of a noob, I hope this makes sense to you. Anything that points me in the right direction is very welcome!
Thanks,
Peter

why Erlang's Reason of termination is normal?

all
the part of log :
** Reason for termination == **
{normal,
{gen_server,call,
[<0.9723.458>,
{create_jtxn_mon,
{player,34125,0,"gulexi",
why does it report error log when the reason is normal?
thanks for your help~~~
It seems like you made a call to a gen_server that exited with reason normal before it sent a response to the caller.
In general, if a gen_server exits with reason ServerExitReason during a call, gen_server:call will exit with the exit reason {ServerExitReason, {gen_server, call, [...]}}, even if ServerExitReason is normal. (See the source)
That is, the exit reason is not normal but {normal, ...}, and that's why you get a log message.

Resources