HikariCP - Integration with Teradata - grails

I am trying to integrate Teradata with HikariCP in a Grails application and facing issues. I am able to connect to Teradata with the same configuration using the default Grails connection pool or datasource.
I tried tweaking the connection timeout also.
libraries used:
Teradata:
runtime (group: 'com.teradata', name:'terajdbc4', version:'13.10.0.8')
runtime (group: 'com.teradata', name:'tdgssconfig', version:'13.10.0.8')
HikariCp:
compile (group: 'com.zaxxer', name: 'HikariCP-java6', version: '2.2.4')
Connection Properties Used:
[logSql:false, loggingSql:true, dialect:org.hibernate.dialect.TeradataDialect,
driverClassName:com.teradata.jdbc.TeraDriver, username:userId, password:pwd,
url:jdbc:teradata://DBName, pooled:true,
properties:[initialSize:5, minIdle:1, maxIdle:5, maxActive:3, maxWait:2000,
maxAge:60000, minEvictableIdleTimeMillis:30000, timeBetweenEvictionRunsMillis:30000,
abandonWhenPercentageFull:50, numTestsPerEvictionRun:3, testOnBorrow:true,
testWhileIdle:true, testOnReturn:true, validationQuery:SELECT 1,
validationInterval:500], jndiName:[:], readOnly:[:], passwordEncryptionCodec:[:]]
Error Log:
TERAJDBC4 ERROR [HikariCP connection filler (pool HikariPool-0)] com.teradata.jdbc.jdbc_4.TDSession#5f045975 Connection to DBName
Tue May 05 17:20:50 CDT 2015 socket orig=DBName cid=247608ab sess=0 com.teradata.jdbc.jdbc_4.util.JDBCException:
[Teradata JDBC Driver] [TeraJDBC 13.10.00.08] [Error 1276] [SQLState 08S01] Exceeded login timeout for DBName
at com.teradata.jdbc.jdbc_4.util.ErrorFactory.makeDriverJDBCException(ErrorFactory.java:92)
at com.teradata.jdbc.jdbc_4.util.ErrorFactory.makeDriverJDBCException(ErrorFactory.java:67)
at com.teradata.jdbc.jdbc.ConnectionBase.checkRemainingLoginTime(ConnectionBase.java:81)
at com.teradata.jdbc.jdbc_4.io.TDNetworkIOIF$Lookup.<init>(TDNetworkIOIF.java:168)
at com.teradata.jdbc.jdbc_4.io.TDNetworkIOIF.connectToHost(TDNetworkIOIF.java:289)
at com.teradata.jdbc.jdbc_4.io.TDNetworkIOIF.<init>(TDNetworkIOIF.java:108)
at com.teradata.jdbc.jdbc_4.TDSession.getIO(TDSession.java:584)
at com.teradata.jdbc.jdbc.GenericStateController.<init>(GenericStateController.java:41)
at com.teradata.jdbc.jdbc.GenericLogonController.<init>(GenericLogonController.java:40)
at com.teradata.jdbc.jdbc_4.TDSession.<init>(TDSession.java:202)
at com.teradata.jdbc.jdbc_3.ifjdbc_4.TeraLocalConnection.<init>(TeraLocalConnection.java:99)
at com.teradata.jdbc.jdbc.ConnectionFactory.createConnection(ConnectionFactory.java:58)
at com.teradata.jdbc.TeraDriver.doConnect(TeraDriver.java:218)
at com.teradata.jdbc.TeraDriver.connect(TeraDriver.java:151)
at java.sql.DriverManager.getConnection(DriverManager.java:571)
at java.sql.DriverManager.getConnection(DriverManager.java:187)
at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:56)
at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:62)
at com.zaxxer.hikari.pool.HikariPool.addConnection(HikariPool.java:415)
at com.zaxxer.hikari.pool.HikariPool.access$400(HikariPool.java:73)
at com.zaxxer.hikari.pool.HikariPool$1AddConnection.run(HikariPool.java:316)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Error | 2015-05-05 17:20:51,234 [HikariCP connection filler (pool HikariPool-0)] ERROR pool.HikariPool - JDBC4 Connection.isValid() method not supported, connection test query must be configured
Error | 2015-05-05 17:20:51,821 [HikariCP connection filler (pool HikariPool-0)] ERROR pool.HikariPool - JDBC4 Connection.isValid() method not supported, connection test query must be configured

I'm not sure if those are the connection properties that work, or those that don't work, but they surely are not HikariCP properties. I suggest enabling DEBUG level logging, and then examining the properties that HikariCP logs at startup to ensure that they match your expections.

Related

Apache ranger 2.1.0 - Installation failed

I am trying to install the latest version of Apache ranger 2.1.0 and getting this error while installing ranger admin. [E] Ranger all admins default password change request failed and there is not much error logs.
2020-09-05 20:21:14,541 [JISQL] /usr/lib/jvm/jre-11-openjdk-11.0.8.10-0.el7_8.x86_64/bin/java -cp /usr/share/java/mysql-connector-java.jar:/opt/apache-ranger/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver mysqlconj -cstring jdbc:mysql://localhost/ranger -u 'ranger' -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('DEFAULT_ALL_ADMIN_UPDATE', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'pas-bifrostx0.myntra.com','N') ;"
Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary.
2020-09-05 20:21:15,219 [I] Ranger all admins default password change request is in process..
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.eclipse.persistence.internal.security.PrivilegedAccessHelper (file:/opt/apache-ranger/ranger-2.1.0-admin/ews/webapp/WEB-INF/lib/eclipselink-2.5.2.jar) to method com.sun.org.apache.xerces.internal.dom.CoreDocumentImpl.getXmlEncoding()
WARNING: Please consider reporting this to the maintainers of org.eclipse.persistence.internal.security.PrivilegedAccessHelper
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
2020-09-05 20:21:19,448 [JISQL] /usr/lib/jvm/jre-11-openjdk-11.0.8.10-0.el7_8.x86_64/bin/java -cp /usr/share/java/mysql-connector-java.jar:/opt/apache-ranger/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver mysqlconj -cstring jdbc:mysql://localhost/ranger -u 'ranger' -p '********' -noheader -trim -c \; -query "delete from x_db_version_h where version = 'DEFAULT_ALL_ADMIN_UPDATE' and active = 'N' and updated_by='pas-bifrostx0.myntra.com';"
Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary.
2020-09-05 20:21:20,073 [E] Ranger all admins default password change request failed
Also, regardless of this error, I was able to start the service with sudo ranger-admin start and is able to do telnet localhost 6080 but web UI is not showing up.
Found the following error in ews/logs/catalina.out
Internal Exception: Exception [EclipseLink-7161] (Eclipse Persistence Services - 2.5.2.v20140319-9ad6abd): org.eclipse.persistence.exceptions.ValidationException
Exception Description: Entity class [class org.apache.ranger.entity.XXAccessAuditV4] has no primary key specified. It should define either an #Id, #EmbeddedId or an #IdClass. If you have defined PK using any of these annotations then make sure that you do not have mixed access-type (both fields and properties annotated) in your entity class hierarchy.
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1619)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:553)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:481)
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:312)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:308)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197)
at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1082)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:857)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:542)
at org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:443)
at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:325)
at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:107)
at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:5128)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5653)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1689)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1679)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: javax.persistence.PersistenceException: Exception [EclipseLink-28018] (Eclipse Persistence Services - 2.5.2.v20140319-9ad6abd): org.eclipse.persistence.exceptions.EntityManagerSetupException
Exception Description: Predeployment of PersistenceUnit [defaultPU] failed.
Internal Exception: Exception [EclipseLink-7161] (Eclipse Persistence Services - 2.5.2.v20140319-9ad6abd): org.eclipse.persistence.exceptions.ValidationException
Exception Description: Entity class [class org.apache.ranger.entity.XXAccessAuditV4] has no primary key specified. It should define either an #Id, #EmbeddedId or an #IdClass. If you have defined PK using any of these annotations then make sure that you do not have mixed access-type (both fields and properties annotated) in your entity class hierarchy.
at org.eclipse.persistence.internal.jpa.EntityManagerSetupImpl.createPredeployFailedPersistenceException(EntityManagerSetupImpl.java:1954)
at org.eclipse.persistence.internal.jpa.EntityManagerSetupImpl.predeploy(EntityManagerSetupImpl.java:1945)
at org.eclipse.persistence.jpa.PersistenceProvider.createContainerEntityManagerFactory(PersistenceProvider.java:322)
at org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean.createNativeEntityManagerFactory(LocalContainerEntityManagerFactoryBean.java:360)
at org.springframework.orm.jpa.AbstractEntityManagerFactoryBean.buildNativeEntityManagerFactory(AbstractEntityManagerFactoryBean.java:384)
at org.springframework.orm.jpa.AbstractEntityManagerFactoryBean.afterPropertiesSet(AbstractEntityManagerFactoryBean.java:371)
at org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean.afterPropertiesSet(LocalContainerEntityManagerFactoryBean.java:336)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1677)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1615)
... 21 more
Caused by: Exception [EclipseLink-28018] (Eclipse Persistence Services - 2.5.2.v20140319-9ad6abd): org.eclipse.persistence.exceptions.EntityManagerSetupException
Exception Description: Predeployment of PersistenceUnit [defaultPU] failed.
Internal Exception: Exception [EclipseLink-7161] (Eclipse Persistence Services - 2.5.2.v20140319-9ad6abd): org.eclipse.persistence.exceptions.ValidationException
Exception Description: Entity class [class org.apache.ranger.entity.XXAccessAuditV4] has no primary key specified. It should define either an #Id, #EmbeddedId or an #IdClass. If you have defined PK using any of these annotations then make sure that you do not have mixed access-type (both fields and properties annotated) in your entity class hierarchy.
at org.eclipse.persistence.exceptions.EntityManagerSetupException.predeployFailed(EntityManagerSetupException.java:230)
... 30 more
Caused by: Exception [EclipseLink-7161] (Eclipse Persistence Services - 2.5.2.v20140319-9ad6abd): org.eclipse.persistence.exceptions.ValidationException
Exception Description: Entity class [class org.apache.ranger.entity.XXAccessAuditV4] has no primary key specified. It should define either an #Id, #EmbeddedId or an #IdClass. If you have defined PK using any of these annotations then make sure that you do not have mixed access-type (both fields and properties annotated) in your entity class hierarchy.
at org.eclipse.persistence.exceptions.ValidationException.noPrimaryKeyAnnotationsFound(ValidationException.java:1422)
at org.eclipse.persistence.internal.jpa.metadata.accessors.classes.EntityAccessor.validatePrimaryKey(EntityAccessor.java:1536)
at org.eclipse.persistence.internal.jpa.metadata.accessors.classes.EntityAccessor.processMappingAccessors(EntityAccessor.java:1243)
at org.eclipse.persistence.internal.jpa.metadata.accessors.classes.EntityAccessor.process(EntityAccessor.java:697)
at org.eclipse.persistence.internal.jpa.metadata.MetadataProject.processStage2(MetadataProject.java:1793)
at org.eclipse.persistence.internal.jpa.metadata.MetadataProcessor.processORMMetadata(MetadataProcessor.java:576)
at org.eclipse.persistence.internal.jpa.deployment.PersistenceUnitProcessor.processORMetadata(PersistenceUnitProcessor.java:585)
at org.eclipse.persistence.internal.jpa.EntityManagerSetupImpl.predeploy(EntityManagerSetupImpl.java:1869)
... 28 more
Sep 05, 2020 2:56:32 PM org.apache.catalina.core.StandardContext startInternal
SEVERE: One or more listeners failed to start. Full details will be found in the appropriate container log file
Sep 05, 2020 2:56:32 PM org.apache.catalina.core.StandardContext startInternal
SEVERE: Context [] startup failed due to previous errors
Sep 05, 2020 2:56:32 PM org.apache.catalina.core.ApplicationContext log
INFO: Closing Spring root WebApplicationContext
Sep 05, 2020 2:56:32 PM org.apache.catalina.core.ApplicationContext log
INFO: Shutting down log4j
Sep 05, 2020 2:56:32 PM org.apache.catalina.loader.WebappClassLoaderBase clearReferencesJdbc
SEVERE: The web application [] registered the JDBC driver [com.mysql.cj.jdbc.Driver] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered.
Sep 05, 2020 2:56:32 PM org.apache.catalina.loader.WebappClassLoaderBase clearReferencesJdbc
SEVERE: The web application [] registered the JDBC driver [org.apache.solr.client.solrj.io.sql.DriverImpl] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered.
Sep 05, 2020 2:56:32 PM org.apache.catalina.loader.WebappClassLoaderBase clearReferencesThreads
SEVERE: The web application [] appears to have started a thread named [org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner] but has failed to stop it. This is very likely to create a memory leak.
Sep 05, 2020 2:56:32 PM org.apache.catalina.loader.WebappClassLoaderBase clearReferencesThreads
SEVERE: The web application [] appears to have started a thread named [mysql-cj-abandoned-connection-cleanup] but has failed to stop it. This is very likely to create a memory leak.
Sep 05, 2020 2:56:32 PM org.apache.catalina.loader.WebappClassLoaderBase checkThreadLocalMapForLeaks
SEVERE: The web application [] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal#2dfce76e]) and a value of type [org.apache.hadoop.fs.FileSystem.Statistics.StatisticsData] (value [9910 bytes read, 0 bytes written, 0 read ops, 0 large read ops, 0 write ops]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
Sep 05, 2020 2:56:32 PM org.apache.catalina.loader.WebappClassLoaderBase checkThreadLocalMapForLeaks
SEVERE: The web application [] created a ThreadLocal with key of type [org.apache.htrace.core.Tracer.ThreadLocalContext] (value [org.apache.htrace.core.Tracer$ThreadLocalContext#4073a8a9]) and a value of type [org.apache.htrace.core.Tracer.ThreadContext] (value [org.apache.htrace.core.Tracer$ThreadContext#42aecda1]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
Sep 05, 2020 2:56:32 PM org.apache.coyote.AbstractProtocol start
INFO: Starting ProtocolHandler ["http-bio-6080"]
Sep 05, 2020 2:56:37 PM org.apache.catalina.loader.WebappClassLoaderBase findResourceInternal
INFO: Illegal access: this web application instance has been stopped already. Could not load []. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.
Can someone please help here or redirect me to Ranger community so that I can reach out to someone for here.
Thanks in advance.
Issues were related to JAVA 11. I was trying to run ranger with JAVA 11
(https://issues.apache.org/jira/browse/RANGER-2575) but seems like it doesn't work. Building and running against java 8 worked.

"dse spark" fails after enabling DSE Unified Authentication

I updated DSE to 5.1.2 and enabled DSE Unified Authentication.
When I run "dse -u xxx -p xxx spark", I get a shell, but if i submit a query using "spark.sql" I get an error:
scala> val result = spark.sql("SELECT id FROM xxx.yyy")
java.lang.RuntimeException: com.datastax.bdp.fs.client.InternalServerException: Server returned HTTP error response 500 (Internal Server Error)
Caused by: com.datastax.bdp.fs.client.InternalServerException: Server returned HTTP error response 500 (Internal Server Error)
at com.datastax.bdp.fs.client.DseFsClient$stateMachine$macro$1$1.apply(DseFsClient.scala:132)
In /var/log/cassandra/system.log I see:
WARN [Native-Transport-Requests-2] 2017-08-31 12:05:29,057 DseAuthenticator.java:659 - Plain text authentication without client / server encryption is strongly discouraged
ERROR [dsefs-netty-worker-4] 2017-08-31 12:05:29,842 DSEFS RestServerHandler.scala:287 - [id: 0x1810ea2d, L:/10.225.155.31:5598 - R:/10.225.155.31:58006] RestHandler invocation failed
java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (no host was tried)
at com.google.common.util.concurrent.AbstractFuture$Sync.getValue(AbstractFuture.java:299) ~[guava-18.0.jar:na]
Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (no host was tried)
Any idea how to fix this. By the way, dse spark-submit works...
The problem was solved by enabling dsefs in dse.yaml.

Jenkins slaves are disconnecting from master with java.net.SocketException

Starting today some of our slaves in our Jenkins master started to disconnect with the following error:
INFO: Trying protocol: JNLP2-connect
May 31, 2016 3:27:07 PM hudson.remoting.jnlp.Main$CuiListener error
SEVERE: Software caused connection abort: recv failed
java.net.SocketException: Software caused connection abort: recv failed
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(Unknown Source)
at java.net.SocketInputStream.read(Unknown Source)
at java.io.BufferedInputStream.fill(Unknown Source)
at java.io.BufferedInputStream.read(Unknown Source)
at org.jenkinsci.remoting.engine.EngineUtil.readLine(EngineUtil.java:51)
at org.jenkinsci.remoting.engine.JnlpProtocol2.performHandshake(JnlpProtocol2.java:74)
I've read about this issue and tried to figure out what caused it. I have around 85% or the rest of my windows slaves still connected and running.
Slaves : Windows server 2012R2
Connected with JNLP protocol
Jenkins version : 1.609.3
If more information is needed let me know and I'll supply it.
This usually means that there was a network error, such as a TCP timeout. I would start by placing a sniffer (wireshark) on the connection to see if you can see any problems. If there is a TCP error, you should be able to see it. Also, you can check your router logs, if this is applicable. If wireless is involved anywhere, that is another source for these kind of errors.
Duplicate: java.net.SocketException: Software caused connection abort: recv failed

Neo4j HA backup fails after first succeeding

I have successfully setup a neo4j 3-instance HA cluster using version 2.0.2 enterprise, but I'm having a problem using the built in backup script (../bin/neo4j-backup).
I manually run:
./bin/neo4j-backup -from ha://10.6.10.48:5001 -to /usr/local/neo4j/backup
...on the master, and it works fine the first time, dumping the data into ../neo4j/backup.
Subsequent tries with the same command yields only this on the command line:
Could not find backup server in cluster neo4j.ha at 10.6.10.48:5001, operation timed out
and this in messages.log:
2014-04-29 17:08:00.919+0000 DEBUG [o.n.c.p.c.ClusterState$4]: ClusterState: entered-[configurationRequest]->entered from:cluster://10.6.10.48:5002 conversation-id:-1/8# payload:-1:cluster://0.0.0.0:5002/?name=Backup
2014-04-29 17:08:00.922+0000 ERROR [o.n.c.c.NetworkSender]: Receive exception:
java.nio.channels.ClosedChannelException: null
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.cleanUpWriteBuffer(AbstractNioWorker.java:409) ~[netty-3.6.3.Final.jar:na]
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.writeFromUserCode(AbstractNioWorker.java:127) ~[netty-3.6.3.Final.jar:na]
at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink.eventSunk(NioClientSocketPipelineSink.java:83) ~[netty-3.6.3.Final.jar:na]
at org.jboss.netty.channel.Channels.write(Channels.java:725) ~[netty-3.6.3.Final.jar:na]
at org.jboss.netty.handler.codec.oneone.OneToOneEncoder.doEncode(OneToOneEncoder.java:71) ~[netty-3.6.3.Final.jar:na]
at org.jboss.netty.handler.codec.oneone.OneToOneEncoder.handleDownstream(OneToOneEncoder.java:59) ~[netty-3.6.3.Final.jar:na]
at org.jboss.netty.channel.Channels.write(Channels.java:704) ~[netty-3.6.3.Final.jar:na]
at org.jboss.netty.channel.Channels.write(Channels.java:671) ~[netty-3.6.3.Final.jar:na]
at org.jboss.netty.channel.AbstractChannel.write(AbstractChannel.java:248) ~[netty-3.6.3.Final.jar:na]
at org.neo4j.cluster.com.NetworkSender$2.run(NetworkSender.java:266) ~[neo4j-cluster-2.0.2.jar:2.0.2]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) ~[na:1.7.0_15]
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) ~[na:1.7.0_15]
at java.util.concurrent.FutureTask.run(FutureTask.java:166) ~[na:1.7.0_15]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_15]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_15]
at java.lang.Thread.run(Thread.java:722) [na:1.7.0_15]
(exception repeats every 5 seconds for awhile)
relevant neo4j.properties values:
online_backup_enabled=true
online_backup_server=127.0.0.1:6362
ha.cluster_server=10.6.10.48:5001
I've checked all firewall settings for all instances.
Any help would be appreciated!
Running a online backup versus single://<host> in general way more easy compared to ha://<host>. From a functional view there is no advantage of ha://.
So you might change
online_backup_server=10.6.10.48:6362
and then run
/bin/neo4j-backup -single ha://10.6.10.48:6362 -to /usr/local/neo4j/backup

Neo4j startup timeout on Arch Linux

[This doesn't really feel like a StackOverflow question to me, but the Google Group explicitly redirected me here.]
I am attempting to run Neo4j on Arch Linux, with a spectacular lack of success. Running bin/neo4j console times out. I am using neo4j-community-1.9.5. All of this information is from running it for the first time immediately after untarring.
Output to the console:
Starting Neo4j Server console-mode...
Using additional JVM arguments: -server -XX:+DisableExplicitGC -Dorg.neo4j.server.properties=conf/neo4j-server.properties -Djava.util.logging.config.file=conf/logging.properties -Dlog4j.configuration=file:conf/log4j.properties -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled
2013-12-07 00:00:02.698+0000 DEBUG [API] You are using an unsupported Java runtime. Please use Oracle(R) Java(TM) Runtime Environment 7.
2013-12-07 00:00:03.225+0000 INFO [API] Setting startup timeout to: 120000ms based on -1
16:00:03.687 [main] WARN o.neo4j.kernel.EmbeddedGraphDatabase - You are using an unsupported Java runtime. Please use Oracle(R) Java(TM) Runtime Environment 7.
org.neo4j.kernel.lifecycle.LifecycleException: Component 'org.neo4j.kernel.impl.transaction.XaDataSourceManager#4d9ecb' failed to stop. Please see attached cause exception.
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.stop(LifeSupport.java:515)
at org.neo4j.kernel.lifecycle.LifeSupport.stop(LifeSupport.java:146)
at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:112)
at org.neo4j.kernel.InternalAbstractGraphDatabase.run(InternalAbstractGraphDatabase.java:262)
at org.neo4j.kernel.EmbeddedGraphDatabase.(EmbeddedGraphDatabase.java:106)
at org.neo4j.graphdb.factory.GraphDatabaseFactory$1.newDatabase(GraphDatabaseFactory.java:88)
at org.neo4j.graphdb.factory.GraphDatabaseBuilder.newGraphDatabase(GraphDatabaseBuilder.java:207)
at org.neo4j.server.database.CommunityDatabase.start(CommunityDatabase.java:54)
at org.neo4j.server.AbstractNeoServer.start(AbstractNeoServer.java:168)
at org.neo4j.server.Bootstrapper.start(Bootstrapper.java:87)
at org.neo4j.server.Bootstrapper.main(Bootstrapper.java:50)
Caused by: org.neo4j.kernel.lifecycle.LifecycleException: Component 'org.neo4j.kernel.impl.nioneo.xa.NeoStoreXaDataSource#1394a8d' failed to stop. Please see attached cause exception.
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.stop(LifeSupport.java:515)
at org.neo4j.kernel.lifecycle.LifeSupport.stop(LifeSupport.java:146)
at org.neo4j.kernel.impl.transaction.XaDataSourceManager.stop(XaDataSourceManager.java:149)
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.stop(LifeSupport.java:511)
... 10 more
Caused by: java.lang.RuntimeException: java.nio.channels.ClosedByInterruptException
at org.neo4j.kernel.impl.transaction.xaframework.LogBackedXaDataSource.stop(LogBackedXaDataSource.java:72)
at org.neo4j.kernel.impl.nioneo.xa.NeoStoreXaDataSource.stop(NeoStoreXaDataSource.java:301)
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.stop(LifeSupport.java:511)
... 13 more
Caused by: java.nio.channels.ClosedByInterruptException
at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202)
at sun.nio.ch.FileChannelImpl.writeInternal(FileChannelImpl.java:743)
at sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:723)
at org.neo4j.kernel.impl.transaction.xaframework.DirectMappedLogBuffer.writeOut(DirectMappedLogBuffer.java:159)
at org.neo4j.kernel.impl.transaction.xaframework.DirectMappedLogBuffer.force(DirectMappedLogBuffer.java:172)
at org.neo4j.kernel.impl.transaction.xaframework.XaLogicalLog.releaseCurrentLogFile(XaLogicalLog.java:678)
at org.neo4j.kernel.impl.transaction.xaframework.XaLogicalLog.close(XaLogicalLog.java:703)
at org.neo4j.kernel.impl.transaction.xaframework.LogBackedXaDataSource.stop(LogBackedXaDataSource.java:68)
... 15 more
org.neo4j.kernel.lifecycle.LifecycleException: Component 'org.neo4j.kernel.extension.KernelExtensions#6d3bea' was successfully initialized, but failed to start. Please see attached cause exception.
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:497)
at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:104)
at org.neo4j.kernel.InternalAbstractGraphDatabase.run(InternalAbstractGraphDatabase.java:262)
at org.neo4j.kernel.EmbeddedGraphDatabase.(EmbeddedGraphDatabase.java:106)
at org.neo4j.graphdb.factory.GraphDatabaseFactory$1.newDatabase(GraphDatabaseFactory.java:88)
at org.neo4j.graphdb.factory.GraphDatabaseBuilder.newGraphDatabase(GraphDatabaseBuilder.java:207)
at org.neo4j.server.database.CommunityDatabase.start(CommunityDatabase.java:54)
at org.neo4j.server.AbstractNeoServer.start(AbstractNeoServer.java:168)
at org.neo4j.server.Bootstrapper.start(Bootstrapper.java:87)
at org.neo4j.server.Bootstrapper.main(Bootstrapper.java:50)
Caused by: org.neo4j.kernel.lifecycle.LifecycleException: Component 'org.neo4j.index.lucene.LuceneKernelExtension#183608e' was successfully initialized, but failed to start. Please see attached cause exception.
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:497)
at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:104)
at org.neo4j.kernel.extension.KernelExtensions.start(KernelExtensions.java:113)
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:491)
... 9 more
Caused by: org.neo4j.kernel.lifecycle.LifecycleException: Component 'org.neo4j.index.impl.lucene.LuceneDataSource#12d3734' was successfully initialized, but failed to start. Please see attached cause exception.
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:497)
at org.neo4j.kernel.lifecycle.LifeSupport.bringToState(LifeSupport.java:404)
at org.neo4j.kernel.lifecycle.LifeSupport.add(LifeSupport.java:318)
at org.neo4j.kernel.impl.transaction.XaDataSourceManager.registerDataSource(XaDataSourceManager.java:196)
at org.neo4j.index.lucene.LuceneKernelExtension.start(LuceneKernelExtension.java:86)
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:491)
... 12 more
Caused by: java.lang.RuntimeException: java.nio.channels.ClosedByInterruptException
at org.neo4j.kernel.impl.index.IndexProviderStore.(IndexProviderStore.java:85)
at org.neo4j.index.impl.lucene.LuceneDataSource.newIndexStore(LuceneDataSource.java:299)
at org.neo4j.index.impl.lucene.LuceneDataSource.start(LuceneDataSource.java:223)
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:491)
... 17 more
Caused by: java.nio.channels.ClosedByInterruptException
at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202)
at sun.nio.ch.FileChannelImpl.position(FileChannelImpl.java:290)
at org.neo4j.kernel.impl.index.IndexProviderStore.write(IndexProviderStore.java:169)
at org.neo4j.kernel.impl.index.IndexProviderStore.create(IndexProviderStore.java:153)
at org.neo4j.kernel.impl.index.IndexProviderStore.(IndexProviderStore.java:61)
... 20 more
2013-12-07 00:02:15.217+0000 DEBUG [API] Failed to start database.
2013-12-07 00:02:15.221+0000 DEBUG [API]
org.neo4j.server.ServerStartupException: Startup took longer than 120000ms, and was stopped. You can disable this behavior by setting 'org.neo4j.server.startup_timeout' to 0. Error code: 1
at org.neo4j.server.AbstractNeoServer.start(AbstractNeoServer.java:204) ~[neo4j-server-1.9.5.jar:1.9.5]
at org.neo4j.server.Bootstrapper.start(Bootstrapper.java:87) [neo4j-server-1.9.5.jar:1.9.5]
at org.neo4j.server.Bootstrapper.main(Bootstrapper.java:50) [neo4j-server-1.9.5.jar:1.9.5]
2013-12-07 00:02:15.225+0000 DEBUG [API] Failed to start Neo Server on port [7474]
Starting Neo4j Server console-mode...
Using additional JVM arguments: -server -XX:+DisableExplicitGC -Dorg.neo4j.server.properties=conf/neo4j-server.properties -Djava.util.logging.config.file=conf/logging.properties -Dlog4j.configuration=file:conf/log4j.properties -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled
2013-12-07 00:00:02.698+0000 DEBUG [API] You are using an unsupported Java runtime. Please use Oracle(R) Java(TM) Runtime Environment 7.
2013-12-07 00:00:03.225+0000 INFO [API] Setting startup timeout to: 120000ms based on -1
16:00:03.687 [main] WARN o.neo4j.kernel.EmbeddedGraphDatabase - You are using an unsupported Java runtime. Please use Oracle(R) Java(TM) Runtime Environment 7.
org.neo4j.kernel.lifecycle.LifecycleException: Component 'org.neo4j.kernel.impl.transaction.XaDataSourceManager#4d9ecb' failed to stop. Please see attached cause exception.
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.stop(LifeSupport.java:515)
at org.neo4j.kernel.lifecycle.LifeSupport.stop(LifeSupport.java:146)
at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:112)
at org.neo4j.kernel.InternalAbstractGraphDatabase.run(InternalAbstractGraphDatabase.java:262)
at org.neo4j.kernel.EmbeddedGraphDatabase.(EmbeddedGraphDatabase.java:106)
at org.neo4j.graphdb.factory.GraphDatabaseFactory$1.newDatabase(GraphDatabaseFactory.java:88)
at org.neo4j.graphdb.factory.GraphDatabaseBuilder.newGraphDatabase(GraphDatabaseBuilder.java:207)
at org.neo4j.server.database.CommunityDatabase.start(CommunityDatabase.java:54)
at org.neo4j.server.AbstractNeoServer.start(AbstractNeoServer.java:168)
at org.neo4j.server.Bootstrapper.start(Bootstrapper.java:87)
at org.neo4j.server.Bootstrapper.main(Bootstrapper.java:50)
Caused by: org.neo4j.kernel.lifecycle.LifecycleException: Component 'org.neo4j.kernel.impl.nioneo.xa.NeoStoreXaDataSource#1394a8d' failed to stop. Please see attached cause exception.
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.stop(LifeSupport.java:515)
at org.neo4j.kernel.lifecycle.LifeSupport.stop(LifeSupport.java:146)
at org.neo4j.kernel.impl.transaction.XaDataSourceManager.stop(XaDataSourceManager.java:149)
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.stop(LifeSupport.java:511)
... 10 more
Caused by: java.lang.RuntimeException: java.nio.channels.ClosedByInterruptException
at org.neo4j.kernel.impl.transaction.xaframework.LogBackedXaDataSource.stop(LogBackedXaDataSource.java:72)
at org.neo4j.kernel.impl.nioneo.xa.NeoStoreXaDataSource.stop(NeoStoreXaDataSource.java:301)
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.stop(LifeSupport.java:511)
... 13 more
Caused by: java.nio.channels.ClosedByInterruptException
at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202)
at sun.nio.ch.FileChannelImpl.writeInternal(FileChannelImpl.java:743)
at sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:723)
at org.neo4j.kernel.impl.transaction.xaframework.DirectMappedLogBuffer.writeOut(DirectMappedLogBuffer.java:159)
at org.neo4j.kernel.impl.transaction.xaframework.DirectMappedLogBuffer.force(DirectMappedLogBuffer.java:172)
at org.neo4j.kernel.impl.transaction.xaframework.XaLogicalLog.releaseCurrentLogFile(XaLogicalLog.java:678)
at org.neo4j.kernel.impl.transaction.xaframework.XaLogicalLog.close(XaLogicalLog.java:703)
at org.neo4j.kernel.impl.transaction.xaframework.LogBackedXaDataSource.stop(LogBackedXaDataSource.java:68)
... 15 more
org.neo4j.kernel.lifecycle.LifecycleException: Component 'org.neo4j.kernel.extension.KernelExtensions#6d3bea' was successfully initialized, but failed to start. Please see attached cause exception.
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:497)
at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:104)
at org.neo4j.kernel.InternalAbstractGraphDatabase.run(InternalAbstractGraphDatabase.java:262)
at org.neo4j.kernel.EmbeddedGraphDatabase.(EmbeddedGraphDatabase.java:106)
at org.neo4j.graphdb.factory.GraphDatabaseFactory$1.newDatabase(GraphDatabaseFactory.java:88)
at org.neo4j.graphdb.factory.GraphDatabaseBuilder.newGraphDatabase(GraphDatabaseBuilder.java:207)
at org.neo4j.server.database.CommunityDatabase.start(CommunityDatabase.java:54)
at org.neo4j.server.AbstractNeoServer.start(AbstractNeoServer.java:168)
at org.neo4j.server.Bootstrapper.start(Bootstrapper.java:87)
at org.neo4j.server.Bootstrapper.main(Bootstrapper.java:50)
Caused by: org.neo4j.kernel.lifecycle.LifecycleException: Component 'org.neo4j.index.lucene.LuceneKernelExtension#183608e' was successfully initialized, but failed to start. Please see attached cause exception.
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:497)
at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:104)
at org.neo4j.kernel.extension.KernelExtensions.start(KernelExtensions.java:113)
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:491)
... 9 more
Caused by: org.neo4j.kernel.lifecycle.LifecycleException: Component 'org.neo4j.index.impl.lucene.LuceneDataSource#12d3734' was successfully initialized, but failed to start. Please see attached cause exception.
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:497)
at org.neo4j.kernel.lifecycle.LifeSupport.bringToState(LifeSupport.java:404)
at org.neo4j.kernel.lifecycle.LifeSupport.add(LifeSupport.java:318)
at org.neo4j.kernel.impl.transaction.XaDataSourceManager.registerDataSource(XaDataSourceManager.java:196)
at org.neo4j.index.lucene.LuceneKernelExtension.start(LuceneKernelExtension.java:86)
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:491)
... 12 more
Caused by: java.lang.RuntimeException: java.nio.channels.ClosedByInterruptException
at org.neo4j.kernel.impl.index.IndexProviderStore.(IndexProviderStore.java:85)
at org.neo4j.index.impl.lucene.LuceneDataSource.newIndexStore(LuceneDataSource.java:299)
at org.neo4j.index.impl.lucene.LuceneDataSource.start(LuceneDataSource.java:223)
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:491)
... 17 more
Caused by: java.nio.channels.ClosedByInterruptException
at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202)
at sun.nio.ch.FileChannelImpl.position(FileChannelImpl.java:290)
at org.neo4j.kernel.impl.index.IndexProviderStore.write(IndexProviderStore.java:169)
at org.neo4j.kernel.impl.index.IndexProviderStore.create(IndexProviderStore.java:153)
at org.neo4j.kernel.impl.index.IndexProviderStore.(IndexProviderStore.java:61)
... 20 more
2013-12-07 00:02:15.217+0000 DEBUG [API] Failed to start database.
2013-12-07 00:02:15.221+0000 DEBUG [API]
org.neo4j.server.ServerStartupException: Startup took longer than 120000ms, and was stopped. You can disable this behavior by setting 'org.neo4j.server.startup_timeout' to 0. Error code: 1
at org.neo4j.server.AbstractNeoServer.start(AbstractNeoServer.java:204) ~[neo4j-server-1.9.5.jar:1.9.5]
at org.neo4j.server.Bootstrapper.start(Bootstrapper.java:87) [neo4j-server-1.9.5.jar:1.9.5]
at org.neo4j.server.Bootstrapper.main(Bootstrapper.java:50) [neo4j-server-1.9.5.jar:1.9.5]
2013-12-07 00:02:15.225+0000 DEBUG [API] Failed to start Neo Server on port [7474]
Contents of the log file:
Dec 06, 2013 4:00:01 PM org.neo4j.server.logging.Logger log
WARNING: You are using an unsupported Java runtime. Please use Oracle(R) Java(TM) Runtime Environment 7.
Dec 06, 2013 4:00:03 PM org.neo4j.server.logging.Logger log
INFO: Setting startup timeout to: 120000ms based on -1
Dec 06, 2013 4:02:15 PM org.neo4j.server.logging.Logger log
SEVERE: Failed to start database.
Dec 06, 2013 4:02:15 PM org.neo4j.server.logging.Logger log
SEVERE:
org.neo4j.server.ServerStartupException: Startup took longer than 120000ms, and was stopped. You can disable this behavior by setting 'org.neo4j.server.startup_timeout' to 0. Error code: 1
at org.neo4j.server.AbstractNeoServer.start(AbstractNeoServer.java:204)
at org.neo4j.server.Bootstrapper.start(Bootstrapper.java:87)
at org.neo4j.server.Bootstrapper.main(Bootstrapper.java:50)
Dec 06, 2013 4:02:15 PM org.neo4j.server.logging.Logger log
SEVERE: Failed to start Neo Server on port [7474]
Java version information:
java version "1.7.0_45"
OpenJDK Runtime Environment (IcedTea 2.4.3) (ArchLinux build 7.u45_2.4.3-1-i686)
OpenJDK Server VM (build 24.45-b08, mixed mode)
Does anyone have any idea what's going on here? Or where I could dig for more information? Or what I could do instead that might work?
Neo4j is currently only supported on Oracle JDK, you're running on OpenJDK.
Also make sure your settings for number of files is correct, see http://docs.neo4j.org/chunked/stable/configuration-linux-notes.html#_setting_the_number_of_open_files.
Check as well if you got enough permissions for Neo4j to write to the data/graph.db folder.
I reverted to version 1.8.3, the second to latest stable. That version works fine and is sufficient to my purpose. Problem undiagnosed, but resolved enough.

Resources