i'm having a real issue trying start neo4j at the moment. My setup consists in 2 servers which are in HA mode. Last night, server 1 logged the fact that there was a timeout trying to elect the master, and it was not available anymore. Server 2 seemed to work fine.
So i tried to restart server 1, and here's the log message i have :
Starting Neo4j Server console-mode...
Detected incorrectly shut down database, performing recovery..
05/07/13 09:52:57 org.neo4j.server.CommunityBootstrapper SEVERE:
java.lang.RuntimeException: org.neo4j.kernel.lifecycle.LifecycleException: Component 'org.neo4j.kernel.impl.transaction.XaDataSourceManager#17f0d3b8' was successfully initialized, but failed to start. Please see attached cause exception.
at org.neo4j.kernel.InternalAbstractGraphDatabase.run(InternalAbstractGraphDatabase.java:319)
at org.neo4j.kernel.EmbeddedGraphDatabase.<init>(EmbeddedGraphDatabase.java:100)
at org.neo4j.graphdb.factory.GraphDatabaseFactory$1.newDatabase(GraphDatabaseFactory.java:92)
at org.neo4j.graphdb.factory.GraphDatabaseBuilder.newGraphDatabase(GraphDatabaseBuilder.java:197)
at org.neo4j.kernel.impl.recovery.StoreRecoverer.recover(StoreRecoverer.java:115)
at org.neo4j.server.preflight.PerformRecoveryIfNecessary.run(PerformRecoveryIfNecessary.java:59)
at org.neo4j.server.preflight.PreFlightTasks.run(PreFlightTasks.java:70)
at org.neo4j.server.AbstractNeoServer.runPreflightTasks(AbstractNeoServer.java:299)
at org.neo4j.server.AbstractNeoServer.start(AbstractNeoServer.java:132)
at org.neo4j.server.Bootstrapper.start(Bootstrapper.java:86)
at org.neo4j.server.Bootstrapper.main(Bootstrapper.java:49)
Caused by: org.neo4j.kernel.lifecycle.LifecycleException: Component 'org.neo4j.kernel.impl.transaction.XaDataSourceManager#17f0d3b8' was successfully initialized, but failed to start. Please see attached cause exception.
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:504)
at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:115)
at org.neo4j.kernel.InternalAbstractGraphDatabase.run(InternalAbstractGraphDatabase.java:296)
... 10 more
Caused by: org.neo4j.kernel.lifecycle.LifecycleException: Component 'org.neo4j.kernel.impl.nioneo.xa.NeoStoreXaDataSource#3cbcef07' was successfully initialized, but failed to start. Please see attached cause exception.
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:504)
at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:115)
at org.neo4j.kernel.impl.transaction.XaDataSourceManager.start(XaDataSourceManager.java:126)
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:498)
... 12 more
Caused by: java.lang.IllegalStateException: No such store[/home/neo-data/graph.db/neostore.relationshiptypestore.db.names] for org.neo4j.kernel.DefaultFileSystemAbstraction#7d6bab68
at org.neo4j.kernel.impl.nioneo.store.CommonAbstractStore.checkStorage(CommonAbstractStore.java:149)
at org.neo4j.kernel.impl.nioneo.store.CommonAbstractStore.<init>(CommonAbstractStore.java:110)
at org.neo4j.kernel.impl.nioneo.store.AbstractDynamicStore.<init>(AbstractDynamicStore.java:75)
at org.neo4j.kernel.impl.nioneo.store.DynamicStringStore.<init>(DynamicStringStore.java:43)
at org.neo4j.kernel.impl.nioneo.store.StoreFactory.newDynamicStringStore(StoreFactory.java:153)
at org.neo4j.kernel.impl.nioneo.store.StoreFactory.newRelationshipTypeTokenStore(StoreFactory.java:159)
at org.neo4j.kernel.impl.nioneo.store.StoreFactory.attemptNewNeoStore(StoreFactory.java:125)
at org.neo4j.kernel.impl.nioneo.store.StoreFactory.newNeoStore(StoreFactory.java:114)
at org.neo4j.kernel.impl.nioneo.xa.NeoStoreXaDataSource.start(NeoStoreXaDataSource.java:277)
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:498)
... 15 more
05/07/13 09:52:57 org.neo4j.server.CommunityBootstrapper SEVERE: Failed to start Neo Server on port [7474]
I understand that there may be troubles with the data, but i can't find how to fix that.
I'm running neo4j 2.0M03 on ubuntu server appliances.
Thanks in advance for any help !
Gouffe
Related
I am currently unable to start a neo4j instance on a server where it was previously working. Neo4j produces an error saying one of the files is externally locked. From reading answers to similar questions I know this usually means another instance is currently using the store.
I have used lsof to look for any open files with 'neo' in their name but there are no open files.
I have used ps to look for any process with 'neo' or 'java' it it's name but no luck with that either.
I am not sure where to go from here to get neo4j working again. How do I figure out why this file is locked?
[site#x neo4j]$ neo4jstart-234.sh
[site#x neo4j]$ WARNING: Max 1024 open files allowed, minimum of 40 000 recommended. See the Neo4j manual.
Starting Neo4j Server console-mode...
2017-07-20 10:37:51.073-0400 INFO Successfully shutdown Neo4j Server
2017-07-20 10:37:51.074-0400 ERROR Failed to start Neo4j: Starting Neo4j failed: Component 'org.neo4j.server.database.LifecycleManagingDatabase#79b877d2' was successfully initialized, but failed to start. Please see attached cause exception. Starting Neo4j failed: Component 'org.neo4j.server.database.LifecycleManagingDatabase#79b877d2' was successfully initialized, but failed to start. Please see attached cause exception.
org.neo4j.server.ServerStartupException: Starting Neo4j failed: Component 'org.neo4j.server.database.LifecycleManagingDatabase#79b877d2' was successfully initialized, but failed to start. Please see attached cause exception.
at org.neo4j.server.exception.ServerStartupErrors.translateToServerStartupError(ServerStartupErrors.java:67)
at org.neo4j.server.AbstractNeoServer.start(AbstractNeoServer.java:235)
at org.neo4j.server.Bootstrapper.start(Bootstrapper.java:97)
at org.neo4j.server.CommunityBootstrapper.start(CommunityBootstrapper.java:48)
at org.neo4j.server.CommunityBootstrapper.main(CommunityBootstrapper.java:35)
Caused by: org.neo4j.kernel.lifecycle.LifecycleException: Component 'org.neo4j.server.database.LifecycleManagingDatabase#79b877d2' was successfully initialized, but failed to start. Please see attached cause exception.
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:462)
at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:111)
at org.neo4j.server.AbstractNeoServer.start(AbstractNeoServer.java:195)
... 3 more
Caused by: java.lang.RuntimeException: Error starting org.neo4j.kernel.impl.factory.CommunityFacadeFactory, /scratch/neo4j/neo4j-community-2.3.4/data/graph.db
at org.neo4j.kernel.impl.factory.GraphDatabaseFacadeFactory.newFacade(GraphDatabaseFacadeFactory.java:143)
at org.neo4j.kernel.impl.factory.CommunityFacadeFactory.newFacade(CommunityFacadeFactory.java:43)
at org.neo4j.kernel.impl.factory.GraphDatabaseFacadeFactory.newFacade(GraphDatabaseFacadeFactory.java:108)
at org.neo4j.server.CommunityNeoServer$1.newGraphDatabase(CommunityNeoServer.java:66)
at org.neo4j.server.database.LifecycleManagingDatabase.start(LifecycleManagingDatabase.java:95)
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:452)
... 5 more
Caused by: org.neo4j.kernel.lifecycle.LifecycleException: Component 'org.neo4j.kernel.NeoStoreDataSource#14c7f72f' was successfully initialized, but failed to start. Please see attached cause exception.
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:462)
at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:111)
at org.neo4j.kernel.impl.transaction.state.DataSourceManager.start(DataSourceManager.java:112)
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:452)
at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:111)
at org.neo4j.kernel.impl.factory.GraphDatabaseFacadeFactory.newFacade(GraphDatabaseFacadeFactory.java:139)
... 10 more
Caused by: org.neo4j.kernel.impl.store.UnderlyingStorageException: Unable to open store file: /scratch/neo4j/neo4j-community-2.3.4/data/graph.db/neostore.nodestore.db.labels
at org.neo4j.kernel.impl.store.CommonAbstractStore.checkStorage(CommonAbstractStore.java:161)
at org.neo4j.kernel.impl.store.CommonAbstractStore.initialise(CommonAbstractStore.java:102)
at org.neo4j.kernel.impl.store.NeoStores.initialize(NeoStores.java:429)
at org.neo4j.kernel.impl.store.NeoStores$StoreType$1.open(NeoStores.java:85)
at org.neo4j.kernel.impl.store.NeoStores$StoreType$1.open(NeoStores.java:78)
at org.neo4j.kernel.impl.store.NeoStores.openStore(NeoStores.java:422)
at org.neo4j.kernel.impl.store.NeoStores.getOrCreateStore(NeoStores.java:467)
at org.neo4j.kernel.impl.store.NeoStores.<init>(NeoStores.java:365)
at org.neo4j.kernel.impl.store.StoreFactory.openNeoStores(StoreFactory.java:174)
at org.neo4j.kernel.impl.store.StoreFactory.openAllNeoStores(StoreFactory.java:138)
at org.neo4j.kernel.NeoStoreDataSource.buildNeoStore(NeoStoreDataSource.java:661)
at org.neo4j.kernel.NeoStoreDataSource.start(NeoStoreDataSource.java:535)
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:452)
... 15 more
Caused by: org.neo4j.io.pagecache.impl.FileLockException: Externally locked: /scratch/neo4j/neo4j-community-2.3.4/data/graph.db/neostore.nodestore.db.labels
at org.neo4j.io.pagecache.impl.SingleFilePageSwapper.acquireLock(SingleFilePageSwapper.java:209)
at org.neo4j.io.pagecache.impl.SingleFilePageSwapper.<init>(SingleFilePageSwapper.java:160)
at org.neo4j.io.pagecache.impl.SingleFilePageSwapperFactory.createPageSwapper(SingleFilePageSwapperFactory.java:64)
at org.neo4j.io.pagecache.impl.muninn.MuninnPagedFile.<init>(MuninnPagedFile.java:95)
at org.neo4j.io.pagecache.impl.muninn.MuninnPageCache.map(MuninnPageCache.java:357)
at org.neo4j.kernel.impl.store.CommonAbstractStore.checkStorage(CommonAbstractStore.java:140)
... 27 more
today I´ve updated my Neo4j Server (running on Ubuntu) from version 3.1.0 to 3.1.3! Everything works so far, so good.
After restarting my server, it crashes instantly. This happens everytime when I start/restart the server...
This is the part from the error log neo4.log
2017-05-01 15:18:30.896+0000 INFO Starting...
2017-05-01 15:18:31.950+0000 INFO Bolt enabled on localhost:7687.
2017-05-01 15:18:35.181+0000 ERROR Failed to start Neo4j: Starting Neo4j failed: Component 'org.neo4j.server.database.LifecycleManagingDatabase#486c3088' was successfully initialized, but failed to start. Please see attached cause exception. Starting Neo4j failed: Component 'org.neo4j.server.database.LifecycleManagingDatabase#486c3088' was successfully initialized, but failed to start. Please see attached cause exception.
org.neo4j.server.ServerStartupException: Starting Neo4j failed: Component 'org.neo4j.server.database.LifecycleManagingDatabase#486c3088' was successfully initialized, but failed to start. Please see attached cause exception.
at org.neo4j.server.exception.ServerStartupErrors.translateToServerStartupError(ServerStartupErrors.java:68)
at org.neo4j.server.AbstractNeoServer.start(AbstractNeoServer.java:227)
at org.neo4j.server.ServerBootstrapper.start(ServerBootstrapper.java:91)
at org.neo4j.server.ServerBootstrapper.start(ServerBootstrapper.java:68)
at org.neo4j.server.CommunityEntryPoint.main(CommunityEntryPoint.java:28)
Caused by: org.neo4j.kernel.lifecycle.LifecycleException: Component 'org.neo4j.server.database.LifecycleManagingDatabase#486c3088' was successfully initialized, but failed to start. Please see attached cause exception.
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:443)
at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:107)
at org.neo4j.server.AbstractNeoServer.start(AbstractNeoServer.java:199)
... 3 more
Caused by: java.lang.RuntimeException: Error starting org.neo4j.kernel.impl.factory.GraphDatabaseFacadeFactory, /var/lib/neo4j/data/databases/graph.db
at org.neo4j.kernel.impl.factory.GraphDatabaseFacadeFactory.initFacade(GraphDatabaseFacadeFactory.java:193)
at org.neo4j.kernel.impl.factory.GraphDatabaseFacadeFactory.newFacade(GraphDatabaseFacadeFactory.java:124)
at org.neo4j.server.CommunityNeoServer.lambda$static$0(CommunityNeoServer.java:57)
at org.neo4j.server.database.LifecycleManagingDatabase.start(LifecycleManagingDatabase.java:89)
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:433)
... 5 more
Caused by: org.neo4j.kernel.lifecycle.LifecycleException: Component 'org.neo4j.kernel.impl.proc.Procedures#78b283fb' was successfully initialized, but failed to start. Please see attached cause exception.
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:443)
at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:107)
at org.neo4j.kernel.impl.factory.GraphDatabaseFacadeFactory.initFacade(GraphDatabaseFacadeFactory.java:189)
... 9 more
Caused by: org.neo4j.kernel.api.exceptions.ProcedureException: Unable to register procedure, because the name `apoc.algo.betweenness` is already in use.
at org.neo4j.kernel.impl.proc.ProcedureRegistry.register(ProcedureRegistry.java:81)
at org.neo4j.kernel.impl.proc.Procedures.register(Procedures.java:103)
at org.neo4j.kernel.impl.proc.Procedures.register(Procedures.java:76)
at org.neo4j.kernel.impl.proc.Procedures.start(Procedures.java:209)
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:433)
... 11 more
nohup: ignoring input
2017-05-01 15:19:58.689+0000 INFO Starting...
2017-05-01 15:19:59.725+0000 INFO Bolt enabled on localhost:7687.
2017-05-01 15:20:01.504+0000 ERROR Cannot register procedure AssertSchemaProcedure org.neo4j.kernel.api.proc.ProcedureSignature$Builder.mode(Lorg/neo4j/procedure/Mode;)Lorg/neo4j/kernel/api/proc/ProcedureSignature$Builder;
java.lang.NoSuchMethodError: org.neo4j.kernel.api.proc.ProcedureSignature$Builder.mode(Lorg/neo4j/procedure/Mode;)Lorg/neo4j/kernel/api/proc/ProcedureSignature$Builder;
at apoc.schema.AssertSchemaProcedure.signature(AssertSchemaProcedure.java:154)
at org.neo4j.kernel.impl.proc.ProcedureRegistry.register(ProcedureRegistry.java:53)
at org.neo4j.kernel.impl.proc.Procedures.register(Procedures.java:103)
at org.neo4j.kernel.impl.proc.Procedures.register(Procedures.java:76)
at apoc.ApocKernelExtensionFactory$ApocLifecycle.registerCustomProcedures(ApocKernelExtensionFactory.java:79)
at apoc.ApocKernelExtensionFactory$ApocLifecycle.start(ApocKernelExtensionFactory.java:67)
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:433)
at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:107)
at org.neo4j.kernel.extension.KernelExtensions.start(KernelExtensions.java:86)
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:433)
at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:107)
at org.neo4j.kernel.impl.factory.GraphDatabaseFacadeFactory.initFacade(GraphDatabaseFacadeFactory.java:189)
at org.neo4j.kernel.impl.factory.GraphDatabaseFacadeFactory.newFacade(GraphDatabaseFacadeFactory.java:124)
at org.neo4j.server.CommunityNeoServer.lambda$static$0(CommunityNeoServer.java:57)
at org.neo4j.server.database.LifecycleManagingDatabase.start(LifecycleManagingDatabase.java:89)
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:433)
at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:107)
at org.neo4j.server.AbstractNeoServer.start(AbstractNeoServer.java:199)
Does anyone had this problem after updating Neo4j or can help me with "understanding" that error log?
Thanks in advance ;)
UPDATE
When I start Neo4j I´ll get this message:
Starting Neo4j.
Started neo4j (pid 28724). By default, it is available at http://localhost:7474/
/usr/share/neo4j/bin/neo4j: line 176: /usr/share/neo4j/logs /var/log/neo4j/neo4j.log: No such file or directory
There may be a short delay until the server is ready.
See /usr/share/neo4j/logs /var/log/neo4j/neo4j.log for current status.
I´ve highlighted the 'suspect' part, which doesn´t appear before updating...
This is the ultimate cause in your call stack:
Caused by: org.neo4j.kernel.api.exceptions.ProcedureException: Unable to register procedure, because the name `apoc.algo.betweenness` is already in use.
This type of exception, where an APOC procedure name is already in use, can be seen when you install a new APOC plugin .jar without removing an existing APOC .jar.
If this applies to you, delete the old APOC .jar and try starting neo4j again.
I have been using Neo4j for more than 3 months on this server. Not sure what changed, but today morning I am unable to start Neo4j server.
Till today everything was fine and there were not changes that I know of were made to Neo4j or anything else.
I am currently using neo4j 3.1.0.
The following is the error log.
xxx#dataserver:~$ sudo neo4j console
Starting Neo4j.
WARNING: Max 1024 open files allowed, minimum of 40000 recommended. See the Neo4j manual.
2017-05-01 03:09:17.948+0000 INFO Starting...
2017-05-01 03:09:18.616+0000 INFO Bolt enabled on 0.0.0.0:7687.
2017-05-01 03:09:20.766+0000 ERROR Failed to start Neo4j: Starting Neo4j failed: Component 'org.neo4j.server.database.LifecycleManagingDatabase#2241b683' was successfully initialized, but failed to start. Please see attached cause exception. Starting Neo4j failed: Component 'org.neo4j.server.database.LifecycleManagingDatabase#2241b683' was successfully initialized, but failed to start. Please see attached cause exception.
org.neo4j.server.ServerStartupException: Starting Neo4j failed: Component 'org.neo4j.server.database.LifecycleManagingDatabase#2241b683' was successfully initialized, but failed to start. Please see attached cause exception.
at org.neo4j.server.exception.ServerStartupErrors.translateToServerStartupError(ServerStartupErrors.java:68)
at org.neo4j.server.AbstractNeoServer.start(AbstractNeoServer.java:227)
at org.neo4j.server.ServerBootstrapper.start(ServerBootstrapper.java:91)
at org.neo4j.server.ServerBootstrapper.start(ServerBootstrapper.java:68)
at org.neo4j.server.CommunityEntryPoint.main(CommunityEntryPoint.java:28)
Caused by: org.neo4j.kernel.lifecycle.LifecycleException: Component 'org.neo4j.server.database.LifecycleManagingDatabase#2241b683' was successfully initialized, but failed to start. Please see attached cause exception.
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:443)
at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:107)
at org.neo4j.server.AbstractNeoServer.start(AbstractNeoServer.java:199)
... 3 more
Caused by: java.lang.RuntimeException: Error starting org.neo4j.kernel.impl.factory.GraphDatabaseFacadeFactory, /var/lib/neo4j/data/databases/default.graphdb
at org.neo4j.kernel.impl.factory.GraphDatabaseFacadeFactory.initFacade(GraphDatabaseFacadeFactory.java:193)
at org.neo4j.kernel.impl.factory.GraphDatabaseFacadeFactory.newFacade(GraphDatabaseFacadeFactory.java:124)
at org.neo4j.server.CommunityNeoServer.lambda$static$0(CommunityNeoServer.java:57)
at org.neo4j.server.database.LifecycleManagingDatabase.start(LifecycleManagingDatabase.java:89)
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:433)
... 5 more
Caused by: org.neo4j.kernel.lifecycle.LifecycleException: Component 'org.neo4j.kernel.recovery.Recovery#4b1ec7fa' failed to initialize. Please see attached cause exception.
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.init(LifeSupport.java:415)
at org.neo4j.kernel.lifecycle.LifeSupport.init(LifeSupport.java:62)
at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:98)
at org.neo4j.kernel.NeoStoreDataSource.start(NeoStoreDataSource.java:517)
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:433)
at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:107)
at org.neo4j.kernel.impl.transaction.state.DataSourceManager.start(DataSourceManager.java:100)
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:433)
at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:107)
at org.neo4j.kernel.impl.factory.GraphDatabaseFacadeFactory.initFacade(GraphDatabaseFacadeFactory.java:189)
... 9 more
Caused by: java.lang.IllegalArgumentException: Unknown entry type -1 for version -1. At position LogPosition{logVersion=94, byteOffset=157773910} and entry version V2_1
at org.neo4j.kernel.impl.transaction.log.entry.LogEntryVersion.entryParser(LogEntryVersion.java:141)
at org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader.readLogEntry(VersionAwareLogEntryReader.java:84)
at org.neo4j.kernel.impl.transaction.log.LogEntryCursor.next(LogEntryCursor.java:54)
at org.neo4j.kernel.recovery.LatestCheckPointFinder.find(LatestCheckPointFinder.java:78)
at org.neo4j.kernel.recovery.PositionToRecoverFrom.apply(PositionToRecoverFrom.java:53)
at org.neo4j.kernel.recovery.DefaultRecoverySPI.getPositionToRecoverFrom(DefaultRecoverySPI.java:80)
at org.neo4j.kernel.recovery.Recovery.init(Recovery.java:80)
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.init(LifeSupport.java:405)
... 18 more
Stop service
Change dbms.allow_upgrade to true in neo4j config file
Start service again
I had the same problem, and upgrading the db fixed it. I would also suggest changing the password if you are still using default password. Once you start the service it should upgrade the database, once upgraded, you can change the dbms.allow_upgrade to false again if you dont want neo4j to update automatically.
Following the documentation we can set up a cluster with an empty DB.
However, following the instructions to set up a cluster with an existing DB we are met with failure, after 10 minutes all three cores throw an exception.
This could be due to: causal_clustering.join_catch_up_timeout=10m
However the databases are already in sync, so "start up should be instantaneous"
Any information pointing us in the right direction would be appreciated.
Exception:
2017-01-27 16:28:56.128+0000 INFO Starting...
2017-01-27 16:28:56.787+0000 INFO Bolt enabled on 192.168.252.231:7687.
2017-01-27 16:28:56.794+0000 INFO Initiating metrics...
2017-01-27 16:28:56.885+0000 INFO Waiting for other members to join cluster before continuing...
2017-01-27 17:00:28.225+0000 ERROR Failed to start Neo4j: Starting Neo4j failed: Component 'org.neo4j.server.database.LifecycleManagingDatabase#1db1d0b8' was successfully initialized, but failed to start. Please see attached cause exception. Starting Neo4j failed: Component 'org.neo4j.server.database.LifecycleManagingDatabase#1db1d0b8' was successfully initialized, but failed to start. Please see attached cause exception.
org.neo4j.server.ServerStartupException: Starting Neo4j failed: Component 'org.neo4j.server.database.LifecycleManagingDatabase#1db1d0b8' was successfully initialized, but failed to start. Please see attached cause exception.
at org.neo4j.server.exception.ServerStartupErrors.translateToServerStartupError(ServerStartupErrors.java:68)
at org.neo4j.server.AbstractNeoServer.start(AbstractNeoServer.java:227)
at org.neo4j.server.ServerBootstrapper.start(ServerBootstrapper.java:91)
at org.neo4j.server.ServerBootstrapper.start(ServerBootstrapper.java:68)
at org.neo4j.server.enterprise.EnterpriseEntryPoint.main(EnterpriseEntryPoint.java:32)
Caused by: org.neo4j.kernel.lifecycle.LifecycleException: Component 'org.neo4j.server.database.LifecycleManagingDatabase#1db1d0b8' was successfully initialized, but failed to start. Please see attached cause exception.
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:443)
at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:107)
at org.neo4j.server.AbstractNeoServer.start(AbstractNeoServer.java:199)
... 3 more
Caused by: java.lang.RuntimeException: Error starting org.neo4j.kernel.impl.factory.GraphDatabaseFacadeFactory, /var/lib/neo4j/data/databases/graph.db
at org.neo4j.kernel.impl.factory.GraphDatabaseFacadeFactory.initFacade(GraphDatabaseFacadeFactory.java:193)
at org.neo4j.causalclustering.core.CoreGraphDatabase.<init>(CoreGraphDatabase.java:56)
at org.neo4j.causalclustering.core.CoreGraphDatabase.<init>(CoreGraphDatabase.java:47)
at org.neo4j.server.enterprise.EnterpriseNeoServer.lambda$static$2(EnterpriseNeoServer.java:95)
at org.neo4j.server.database.LifecycleManagingDatabase.start(LifecycleManagingDatabase.java:89)
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:433)
... 5 more
Caused by: org.neo4j.kernel.lifecycle.LifecycleException: Component 'org.neo4j.causalclustering.core.state.CoreState#278df960' was successfully initialized, but failed to start. Please see attached cause exception.
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:443)
at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:107)
at org.neo4j.kernel.impl.factory.GraphDatabaseFacadeFactory.initFacade(GraphDatabaseFacadeFactory.java:189)
... 10 more
Caused by: java.lang.RuntimeException: This machine failed to get the start state in time.
at org.neo4j.causalclustering.core.state.CoreState.start(CoreState.java:176)
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:433)
... 12 more
Please check if the following configurations are in order for your cluster.
1) neo4j.conf
"dbms.connectors.default_advertised_address" for each causal cluster is configured correctly and these IP addresses are reachable to each other.
also, set "dbms.connectors.default_listen_address" to 0.0.0.0
2) unbind the nodes and restart them.
Let me know if you still see the issue.
Neo4j 3.0.3 HA cluster fails to start with the following error message from the log:
ERROR Failed to start Neo4j: Starting Neo4j failed: Component 'org.neo4j.server.database.LifecycleManagingDatabase#1b38563a' was successfully initialized, but failed to start. Please see attached cause exception. Starting Neo4j failed: Component 'org.neo4j.server.database.LifecycleManagingDatabase#1b38563a' was successfully initialized, but failed to start. Please see attached cause exception.
org.neo4j.server.ServerStartupException: Starting Neo4j failed: Component 'org.neo4j.server.database.LifecycleManagingDatabase#1b38563a' was successfully initialized, but failed to start. Please see attached cause exception.
at org.neo4j.server.exception.ServerStartupErrors.translateToServerStartupError(ServerStartupErrors.java:68)
at org.neo4j.server.AbstractNeoServer.start(AbstractNeoServer.java:217)
at org.neo4j.server.ServerBootstrapper.start(ServerBootstrapper.java:87)
at org.neo4j.server.ServerBootstrapper.start(ServerBootstrapper.java:66)
at org.neo4j.server.enterprise.EnterpriseEntryPoint.main(EnterpriseEntryPoint.java:32)
Caused by: org.neo4j.kernel.lifecycle.LifecycleException: Component 'org.neo4j.server.database.LifecycleManagingDatabase#1b38563a' was successfully initialized, but failed to start. Please see attached cause exception.
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:444)
at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:107)
at org.neo4j.server.AbstractNeoServer.start(AbstractNeoServer.java:189)
... 3 more
Caused by: java.lang.RuntimeException: Error starting org.neo4j.kernel.ha.factory.HighlyAvailableFacadeFactory, /var/lib/neo4j/data/databases/graph.db
at org.neo4j.kernel.impl.factory.GraphDatabaseFacadeFactory.newFacade(GraphDatabaseFacadeFactory.java:144)
at org.neo4j.kernel.ha.factory.HighlyAvailableFacadeFactory.newFacade(HighlyAvailableFacadeFactory.java:42)
at org.neo4j.kernel.ha.HighlyAvailableGraphDatabase.(HighlyAvailableGraphDatabase.java:41)
at org.neo4j.server.enterprise.EnterpriseNeoServer.lambda$static$0(EnterpriseNeoServer.java:80)
at org.neo4j.server.database.LifecycleManagingDatabase.start(LifecycleManagingDatabase.java:89)
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:434)
... 5 more
Caused by: org.neo4j.kernel.lifecycle.LifecycleException: Component 'org.neo4j.cluster.client.ClusterJoin#27564996' was successfully initialized, but failed to start. Please see attached cause exception.
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:444)
at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:107)
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:434)
at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:107)
at org.neo4j.kernel.impl.factory.GraphDatabaseFacadeFactory.newFacade(GraphDatabaseFacadeFactory.java:140)
... 10 more
Caused by: java.util.concurrent.TimeoutException
at org.neo4j.cluster.statemachine.StateMachineProxyFactory$ResponseFuture.get(StateMachineProxyFactory.java:300)
at org.neo4j.cluster.client.ClusterJoin.joinByConfig(ClusterJoin.java:143)
at org.neo4j.cluster.client.ClusterJoin.start(ClusterJoin.java:82)
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:434)
... 14 more
Since there's a timeout, it's probably an error in the network configuration of the cluster: wrong IP addresses, TCP port filtered by a firewall, etc.
I have the same problem on 3.0.1 in an 2 node ha cluster with an arbiter. However, my arbiter fails first, and if I restart it two more times while the nodes are still trying to start, I'm able to start entire cluster. This is a ridiculous procedure, but this is the only on I've found so far to start the cluster. And not, it is not a networking issue, because after that elaborate restart, everything is working.