My neo4j server is automatically stopping and starting, - neo4j

I'm running my neo4j community edition 3.5.5 version with 8GB ram in aws instance.
Initially for few months it ran very fine and got results in millis of time, but now a days it's getting stopping automatically and starting automatically. Sometimes it's not at all starting for hours,even we started it manually also.
Can anyone please help me with this. I'm getting the below logs.
tail -100f /var/log/neo4j/neo4j.log
2019-07-29 13:17:52.570+0000 WARN The client is unauthorized due to authentication failure.
2019-09-04 05:33:52.328+0000 WARN The client is unauthorized due to authentication failure.
2019-10-17 15:18:14.652+0000 INFO Transaction with id 2683388 has been automatically rolled back due to transaction timeout.
nohup: ignoring input
OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x00000006e5400000, 3670016000, 0) failed; error='Cannot allocate memory' (errno=12)
There is insufficient memory for the Java Runtime Environment to continue.
Native memory allocation (mmap) failed to map 3670016000 bytes for committing reserved memory.
An error report file with more information is saved as:
/home/ubuntu/hs_err_pid8965.log
nohup: ignoring input
OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x00000006e5400000, 3670016000, 0) failed; error='Cannot allocate memory' (errno=12)
There is insufficient memory for the Java Runtime Environment to continue.
Native memory allocation (mmap) failed to map 3670016000 bytes for committing reserved memory.
An error report file with more information is saved as:
/home/ubuntu/hs_err_pid9050.log
nohup: ignoring input
OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x00000006e5400000, 3670016000, 0) failed; error='Cannot allocate memory' (errno=12)
2019-10-17 17:14:44.651+0000 INFO Transaction with id 2689294 has been automatically rolled back due to transaction timeout.

this can be because you are running lot of merge operations and dont have proper indices created or try increasing the heap size in config file .

Related

Jenkins 2.304: Failed to monitor for Free Swap Space

Jenkins 2.304, Java 11.
When launching a job to a remote machine, observed a Free Swap Space failure. What could cause it?
Remoting version: 4.10
This is a Unix agent
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by jenkins.slaves.StandardOutputSwapper$ChannelSwapper to constructor java.io.FileDescriptor(int)
WARNING: Please consider reporting this to the maintainers of jenkins.slaves.StandardOutputSwapper$ChannelSwapper
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Evacuated stdout
ERROR: Failed to monitor for Architecture
ERROR: Failed to monitor for Free Disk Space
ERROR: ERROR: Failed to monitor for Clock Difference
Failed to monitor for Free Swap Space
java.util.concurrent.TimeoutException
at hudson.remoting.Request$1.get(Request.java:321)
at hudson.remoting.Request$1.get(Request.java:240)
at hudson.remoting.FutureAdapter.get(FutureAdapter.java:65)
at hudson.node_monitors.AbstractAsyncNodeMonitorDescriptor.monitorDetailed(AbstractAsyncNodeMonitorDescriptor.java:113)
at hudson.node_monitors.AbstractAsyncNodeMonitorDescriptor.monitor(AbstractAsyncNodeMonitorDescriptor.java:77)
at hudson.node_monitors.AbstractNodeMonitorDescriptor$Record.run(AbstractNodeMonitorDescriptor.java:305)

GPU Acceleration with WSL 2

I'm trying to setup tensorflow to use GPU acceleration with WSL 2 running Ubuntu 20.04. I'm following this tutorial and am running into the error seen here. However, when I follow the solution there and try to start docker with sudo service docker start I get told docker is an unrecognized service. However, considering I can access the help menu and whatnot, I know docker is installed. While I can get docker to work with the desktop tool, since it doesn't support Cuda as mentioned in the SO post from earlier, it's not very helpful. It's not really giving me error logs or anything, so please ask if you need more details.
Edit:
Considering the lack of details, here are a list of solutions I've tried to no avail. 1 2 3
Update: I used sudo dockerd to get the container started and tried running the nvidia benchmark container only to be met with
INFO[2020-07-18T21:04:05.875283800-04:00] shim containerd-shim started address=/containerd-shim/021834ef5e5600bdf62a6a9e26dff7ffc1c76dd4ec9dadb9c1fcafb6c88b6e1b.sock debug=false pid=1960
INFO[2020-07-18T21:04:05.899420200-04:00] shim reaped id=70316df254d6b2633c743acb51a26ac2d0520f6f8e2f69b69c4e0624eaac1736
ERRO[2020-07-18T21:04:05.909710600-04:00] stream copy error: reading from a closed fifo
ERRO[2020-07-18T21:04:05.909753500-04:00] stream copy error: reading from a closed fifo
ERRO[2020-07-18T21:04:06.001006700-04:00] 70316df254d6b2633c743acb51a26ac2d0520f6f8e2f69b69c4e0624eaac1736 cleanup: failed to delete container from containerd: no such container
ERRO[2020-07-18T21:04:06.001045100-04:00] Handler for POST /v1.40/containers/70316df254d6b2633c743acb51a26ac2d0520f6f8e2f69b69c4e0624eaac1736/start returned error: OCI runtime create failed: container_linux.go:349: starting container process caused "process_linux.go:449: container init caused \"process_linux.go:432: running prestart hook 0 caused \\\"error running hook: exit status 1, stdout: , stderr: nvidia-container-cli: initialization error: driver error: failed to process request\\\\n\\\"\"": unknown
docker: Error response from daemon: OCI runtime create failed: container_linux.go:349: starting container process caused "process_linux.go:449: container init caused \"process_linux.go:432: running prestart hook 0 caused \\\"error running hook: exit status 1, stdout: , stderr: nvidia-container-cli: initialization error: driver error: failed to process request\\\\n\\\"\"": unknown.
ERRO[0000] error waiting for container: context canceled
Update 2: After installing windows insider and making everything as up to date as possible, I encountered a different error.
Run "nbody -benchmark [-numbodies=<numBodies>]" to measure performance.
-fullscreen (run n-body simulation in fullscreen mode)
-fp64 (use double precision floating point values for simulation)
-hostmem (stores simulation data in host memory)
-benchmark (run benchmark to measure performance)
-numbodies=<N> (number of bodies (>= 1) to run in simulation)
-device=<d> (where d=0,1,2.... for the CUDA device to use)
-numdevices=<i> (where i=(number of CUDA devices > 0) to use for simulation)
-compare (compares simulation results running once on the default GPU and once on the CPU)
-cpu (run n-body simulation on the CPU)
-tipsy=<file.bin> (load a tipsy model file for simulation)
NOTE: The CUDA Samples are not meant for performance measurements. Results may vary when GPU Boost is enabled.
Error: only 0 Devices available, 1 requested. Exiting.
I have a GTX 970, so I'm not sure why it's not being detected. After running sudo lshw -C display, it was confirmed that my graphics card isn't being detected. I got:
*-display UNCLAIMED
description: 3D controller
product: Microsoft Corporation
vendor: Microsoft Corporation
physical id: 4
bus info: pci#941e:00:00.0
version: 00
width: 32 bits
clock: 33MHz
capabilities: bus_master cap_list
configuration: latency=0

Jenkins build faild on slave node with `java.io.EOFException`

The stacktrace is below:
Evacuated stdout
Starting Selenium nodes on ci2
March 18, 2019 11:04:00 AM org.jenkinsci.remoting.util.AnonymousClassWarnings warn
WARN: Attempt to (de-)serialize anonymous class org.jenkinsci.plugins.gitclient.Git$1; see: https://jenkins.io/redirect/serialization-of-anonymous-classes/
March 18, 2019 11:04:03 AM org.jenkinsci.remoting.util.AnonymousClassWarnings warn
WARN: Attempt to (de-)serialize anonymous class org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1; see: https://jenkins.io/redirect/serialization-of-anonymous-classes/
Slave JVM has not reported exit code. Is it still running?
[03/18/19 11:04:06] Launch failed - cleaning up connection
ERROR: Connection terminated
java.io.EOFException
at java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2681)
at java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:3156)
at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:862)
at java.io.ObjectInputStream.<init>(ObjectInputStream.java:358)
at hudson.remoting.ObjectInputStreamEx.<init>(ObjectInputStreamEx.java:49)
at hudson.remoting.Command.readFrom(Command.java:140)
at hudson.remoting.Command.readFrom(Command.java:126)
at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:36)
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:63)
Caused: java.io.IOException: Unexpected termination of the channel
at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:77)
[03/18/19 11:04:06] [SSH] Connection closed。
I use JDK8 and jenkins 2.164.1 on Ubuntu 16.04.6
How to fix this?
This issue has been bugging us as well. There seems to be a temporary work around, not a solution. Just set the remote root directory of the slave node configuration on jenkins server to a new path, and remoting will take care of the rest.. but, the issue seem to reoccur now and then. We don't know the root cause yet. Any word is more than welcome.

dockerized cassandra: java.lang.OutOfMemoryError: Java heap space

I have cassandra docker container, which did not start after hard restart pc.
The command line output is very long. This are some snippets
java.lang.OutOfMemoryError: Java heap space
ERROR 20:08:26 JVM state determined to be unstable. Exiting forcefully due to:
java.lang.OutOfMemoryError: Java heap space
Caused by: java.io.IOException: Corrupt (negative) value length encountered
ERROR 20:08:29 JVM state determined to be unstable. Exiting forcefully due to:
java.lang.OutOfMemoryError: Java heap space
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)ERROR 20:08:29 JVM state determined to be unstable. Exiting forcefully due to:
java.lang.OutOfMemoryError: Java heap space
ERROR 20:08:29 JVM state determined to be unstable. Exiting forcefully due to:
java.lang.OutOfMemoryError: Java heap space
ERROR 20:08:29 JVM state determined to be unstable. Exiting forcefully due to:
java.lang.OutOfMemoryError: Java heap space
ERROR 20:08:29 JVM state determined to be unstable. Exiting forcefully due to:
java.lang.OutOfMemoryError: Java heap space
ERROR 20:08:29 JVM state determined to be unstable. Exiting forcefully due to:
java.lang.OutOfMemoryError: Java heap space
ERROR 20:08:29 JVM state determined to be unstable. Exiting forcefully due to:
java.lang.OutOfMemoryError: Java heap space
ERROR 20:08:29 JVM state determined to be unstable. Exiting forcefully due to:
java.lang.OutOfMemoryError: Java heap space
ERROR 20:08:29 Unknown exception caught while attempting to update MaterializedView! findkita.kitas
java.lang.OutOfMemoryError: Java heap space
ERROR 20:08:29 Unknown exception caught while attempting to update MaterializedView! findkita.kitas
java.lang.OutOfMemoryError: Java heap space
ERROR 20:08:29 Unknown exception caught while attempting to update MaterializedView! findkita.kitas
java.lang.OutOfMemoryError: Java heap space
ERROR 20:08:29 Unknown exception caught while attempting to update MaterializedView! findkita.kitas
java.lang.OutOfMemoryError: Java heap space
WARN 20:08:29 ConcurrentMarkSweep GC in 5961ms. CMS Old Gen: 885922704 -> 895736336; Par Survivor Space: 18224024 -> 0
ERROR 20:08:26 Unknown exception caught while attempting to update MaterializedView! findkita.kitas
Thats are some of the error I could pick out. But I do not know how to solve theme. Because of error cointainer do not start, so I could not touch cassandra directly.
One error is repead every time:
ERROR 20:08:26 Unknown exception caught while attempting to update MaterializedView! findkita.kitas

ejabberd incompatiblity with p1_yaml library

I have installed erlang/OTP 17.1 from source & I have installed ejabberd from GitHub. Both have installed successfully. But when i am starting ejabberd server, i am getting the following error in error.log.
2016-01-08 16:13:57.700 [error] <0.100.0> unable to load p1_yaml NIF: {error,{bad_lib,"Library version (2.8) not compatible (with 2.6)."}}
2016-01-08 16:13:57.700 [error] <0.99.0> CRASH REPORT Process <0.99.0> with 0 neighbours exited with reason: {{bad_lib,"Library version (2.8) not compatible (with 2.6)."},{p1_yaml_app,start,[normal,[]]}} in application_master:init/4 line 133
2016-01-08 16:13:57.769 [critical] <0.38.0>#ejabberd:exit_or_halt:133 failed to start application 'p1_yaml': {error,
{{bad_lib,
"Library version (2.8) not compatible (with 2.6)."},
{p1_yaml_app,start,[normal,[]]}}}
You likely have a p1_yaml.so file around that was build with a different version of Erlang runtime than the one you are using.
You should rebuild ejabberd (or just p1_yaml) with the correct Erlang version.

Resources