I am using Neo4J 3.2.0 community edition and I was able to connect and establish connection properly using traditional java driver and running Cypher queries.
However, I wanted to create rich data model, so I imported the following dependencies:
<dependency>
<groupId>org.neo4j</groupId>
<artifactId>neo4j-ogm-core</artifactId>
<version>2.1.3</version>
</dependency>
<dependency>
<groupId>org.neo4j</groupId>
<artifactId>neo4j-ogm-bolt-driver</artifactId>
<version>2.1.3</version>
</dependency>
Post that, I wrote the following code to establish the connection:
Configuration configuration = new Configuration();
configuration.driverConfiguration().setDriverClassName("org.neo4j.ogm.drivers.bolt.driver.BoltDriver")
.setURI("bolt://localhost:7687");
SessionFactory sessionFactory = new SessionFactory(configuration, "org.neo4j.example.domain");
Session session = sessionFactory.openSession();
But irrespective of weather my Neo4j server is turned on or not, it establish connection, and the logs dont show any errors, just the following three lines for the logging system:
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Any help would be appreciated.
Adding the following dependency fixes it:
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-simple</artifactId>
<version>1.7.25</version>
</dependency>
Related
After starting the service discovery, When the proxy microservice is registered, the eureka is throwing the following error:
2020-06-11 21:07:19.778 ERROR 1040 --- [et_localhost-19] c.n.e.cluster.ReplicationTaskProcessor : Batch update failure with HTTP status code 404; discarding 1 replication tasks
2020-06-11 21:07:19.778 WARN 1040 --- [et_localhost-19] c.n.eureka.util.batcher.TaskExecutors : Discarding 1 tasks of TaskBatchingWorker-target_localhost-19 due to permanent error
2020-06-11 21:07:49.782 ERROR 1040 --- [et_localhost-19] c.n.e.cluster.ReplicationTaskProcessor : Batch update failure with HTTP status code 404; discarding 1 replication tasks
2020-06-11 21:07:49.782 WARN 1040 --- [et_localhost-19] c.n.eureka.util.batcher.TaskExecutors : Discarding 1 tasks of TaskBatchingWorker-target_localhost-19 due to permanent error
2020-06-11 21:08:11.629 INFO 1040 --- [a-EvictionTimer] c.n.e.registry.AbstractInstanceRegistry : Running the evict task with compensationTime 0ms
2020-06-11 21:08:19.796 ERROR 1040 --- [et_localhost-19] c.n.e.cluster.ReplicationTaskProcessor : Batch update failure with HTTP status code 404; discarding 1 replication tasks
2020-06-11 21:08:19.796 WARN 1040 --- [et_localhost-19] c.n.eureka.util.batcher.TaskExecutors : Discarding 1 tasks of TaskBatchingWorker-target_localhost-19 due to permanent error
Error Log
The dependencies are as follows:
<properties>
<java.version>1.8</java.version>
<spring-cloud.version>Hoxton.SR3</spring-cloud.version>
<springfox-swagger2-version>2.9.2</springfox-swagger2-version>
<springfox-swagger2-ui-version>2.9.2</springfox-swagger2-ui-version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-zuul</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-config</artifactId>
</dependency>
</dependencies>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-dependencies</artifactId>
<version>${spring-cloud.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
bootstrap.yml
spring:
application:
name: tng-crs-eureka
server:
port: 8761
servlet:
context-path: /eureka
eureka:
client:
registerWithEureka: false
fetchRegistry: false
Any help is highly appreciated. I have tried by disabling the batch update tasks also. But, it didn't worked.
Finally, The mistake that I have made was figured out. I have set the context-path of /eureka. And that is the reason for all this trouble. Removing the context path of eureka resolves the issue for my case.
I'm trying to create a pipeline task using beam java SDK and Google Dataflow to move data from Cloud SQL to Elastic search
I've created the following class main method:
public static void main(String[] args) throws Exception{
DataflowPipelineOptions options = PipelineOptionsFactory.as(DataflowPipelineOptions.class);
options.setProject("staging");
options.setTempLocation("gs://csv_to_sql_staging/temp");
options.setRunner(DataflowRunner.class);
options.setGcpTempLocation("gs://csv_to_sql_staging/temp");
options.setUsePublicIps(false);
options.setJobName("tamer-new");
options.setSubnetwork("regions/us-central1/subnetworks/new-network");
final List<String> SCOPES = Arrays.asList(
"https://www.googleapis.com/auth/cloud-platform",
"https://www.googleapis.com/auth/devstorage.full_control",
"https://www.googleapis.com/auth/userinfo.email",
"https://www.googleapis.com/auth/datastore",
"https://www.googleapis.com/auth/sqlservice.admin",
"https://www.googleapis.com/auth/pubsub");
options.setGcpCredential(ServiceAccountCredentials.fromStream(new ElasticSearchIO().getClass().getResourceAsStream("/staging-b648da5d2b9b.json")).createScoped(SCOPES)); options.setServiceAccount("data-flow#staging.iam.gserviceaccount.com");
Pipeline p = Pipeline.create(options);
p.begin();
PCollection < List < String >> rows = p.apply(JdbcIO. < List < String >> read().withQuery("select u.id, u.name from user_table").withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create("com.mysql.jdbc.Driver", "jdbc:mysql://google/nameDB_new?cloudSqlInstance=staging:europe-west1:sql-staging-instance&socketFactory=com.google.cloud.sql.mysql.SocketFactory&useUnicode=true&characterEncoding=UTF-8&user=user&password=password&useSSL=false")).withRowMapper(new RowMapper < List < String >> () {
#Override public List < String > mapRow(ResultSet resultSet) throws Exception {
List < String > addRow = new ArrayList < String > ();
for (int i = 1; i <= resultSet.getMetaData().getColumnCount(); i++) {
addRow.add(i - 1, String.valueOf(resultSet.getObject(i)));
}
//LOG.info(String.join(",", addRow));
return addRow;
}
})
.withCoder(ListCoder.of(StringUtf8Coder. < Object > of ()))
);
Write w = ElasticsearchIO.write().withConnectionConfiguration(
ElasticsearchIO.ConnectionConfiguration.create(new String[] {
"https://host:9243"
}, "user-temp", "String").withUsername("elastic").withPassword("password")
);
rows.apply(w.compose(new SerializableFunction() {
#Override public Object apply(Object input) {
// TODO Auto-generated method stub
return input;
}
}));
p.run().waitUntilFinish();
}
and below is the pom.xml file :
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.harmonica.dataflow</groupId>
<artifactId>com-harmonica-dataflow</artifactId>
<version>0.0.1-SNAPSHOT</version>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<maven-compiler-plugin.version>3.7.0</maven-compiler-plugin.version>
<exec-maven-plugin.version>1.6.0</exec-maven-plugin.version>
<slf4j.version>1.7.25</slf4j.version>
<beam.version>2.19.0</beam.version>
</properties>
<repositories>
<repository>
<id>ossrh.snapshots</id>
<name>Sonatype OSS Repository Hosting</name>
<url>https://oss.sonatype.org/content/repositories/snapshots/</url>
<releases>
<enabled>false</enabled>
</releases>
<snapshots>
<enabled>true</enabled>
</snapshots>
</repository>
</repositories>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>${maven-compiler-plugin.version}</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
</plugins>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>${exec-maven-plugin.version}</version>
<configuration>
<cleanupDaemonThreads>false</cleanupDaemonThreads>
</configuration>
</plugin>
</plugins>
</pluginManagement>
</build>
<dependencies>
<!-- Beam Lib -->
<dependency>
<groupId>org.apache.beam</groupId>
<artifactId>beam-sdks-java-core</artifactId>
<version>${beam.version}</version>
</dependency>
<dependency>
<groupId>org.apache.beam</groupId>
<artifactId>beam-runners-google-cloud-dataflow-java</artifactId>
<version>${beam.version}</version>
</dependency>
<dependency>
<groupId>org.apache.beam</groupId>
<artifactId>beam-sdks-java-io-elasticsearch</artifactId>
<version>${beam.version}</version>
</dependency>
<dependency>
<groupId>org.apache.beam</groupId>
<artifactId>beam-sdks-java-io-jdbc</artifactId>
<version>${beam.version}</version>
</dependency>
<dependency>
<groupId>org.apache.beam</groupId>
<artifactId>beam-sdks-java-io-google-cloud-platform</artifactId>
<version>${beam.version}</version>
</dependency>
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<version>8.0.19</version>
</dependency>
<dependency>
<groupId>com.google.cloud.sql</groupId>
<artifactId>mysql-socket-factory-connector-j-8</artifactId>
<version>1.0.15</version>
</dependency>
<!-- slf4j API frontend binding with JUL backend -->
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>${slf4j.version}</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-jdk14</artifactId>
<version>${slf4j.version}</version>
</dependency>
</dependencies>
</project>
and when I execute this command:
man exec mvn compile exec:java -Dexec.mainClass=com.dataflow.ElasticSearchIO
The worker started successfully but then cant connect to the the Cloud SQL:
even thought I've done the flowing:
I've created a service account with owner access to the project and passed it to the runner options
I've created a VPC network with a name of new-network with a range of IP of 190.10.0.0/16 and assigned it to the pipeline options and then whitlisted this range in the cloud SQL
and however Im still getting this error:
Error message from worker: java.lang.RuntimeException:
org.apache.beam.sdk.util.UserCodeException: java.sql.SQLException:
Cannot create PoolableConnectionFactory (Communications link failure
The last packet sent successfully to the server was 0 milliseconds
ago. The driver has not received any packets from the server.)
org.apache.beam.runners.dataflow.worker.IntrinsicMapTaskExecutorFactory$1.typedApply(IntrinsicMapTaskExecutorFactory.java:194)
org.apache.beam.runners.dataflow.worker.IntrinsicMapTaskExecutorFactory$1.typedApply(IntrinsicMapTaskExecutorFactory.java:165)
org.apache.beam.runners.dataflow.worker.graph.Networks$TypeSafeNodeFunction.apply(Networks.java:63)
org.apache.beam.runners.dataflow.worker.graph.Networks$TypeSafeNodeFunction.apply(Networks.java:50)
org.apache.beam.runners.dataflow.worker.graph.Networks.replaceDirectedNetworkNodes(Networks.java:87)
org.apache.beam.runners.dataflow.worker.IntrinsicMapTaskExecutorFactory.create(IntrinsicMapTaskExecutorFactory.java:125)
org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:352)
org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:305)
org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
java.util.concurrent.FutureTask.run(FutureTask.java:266)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
java.lang.Thread.run(Thread.java:748) Caused by:
org.apache.beam.sdk.util.UserCodeException: java.sql.SQLException:
Cannot create PoolableConnectionFactory (Communications link failure
The last packet sent successfully to the server was 0 milliseconds
ago. The driver has not received any packets from the server.)
org.apache.beam.sdk.util.UserCodeException.wrap(UserCodeException.java:34)
org.apache.beam.sdk.io.jdbc.JdbcIO$ReadFn$DoFnInvoker.invokeSetup(Unknown
Source)
org.apache.beam.runners.dataflow.worker.DoFnInstanceManagers$ConcurrentQueueInstanceManager.deserializeCopy(DoFnInstanceManagers.java:80)
org.apache.beam.runners.dataflow.worker.DoFnInstanceManagers$ConcurrentQueueInstanceManager.peek(DoFnInstanceManagers.java:62)
org.apache.beam.runners.dataflow.worker.UserParDoFnFactory.create(UserParDoFnFactory.java:95)
org.apache.beam.runners.dataflow.worker.DefaultParDoFnFactory.create(DefaultParDoFnFactory.java:75)
org.apache.beam.runners.dataflow.worker.IntrinsicMapTaskExecutorFactory.createParDoOperation(IntrinsicMapTaskExecutorFactory.java:264)
org.apache.beam.runners.dataflow.worker.IntrinsicMapTaskExecutorFactory.access$000(IntrinsicMapTaskExecutorFactory.java:86)
org.apache.beam.runners.dataflow.worker.IntrinsicMapTaskExecutorFactory$1.typedApply(IntrinsicMapTaskExecutorFactory.java:183)
... 14 more Caused by: java.sql.SQLException: Cannot create
PoolableConnectionFactory (Communications link failure The last packet
sent successfully to the server was 0 milliseconds ago. The driver has
not received any packets from the server.)
org.apache.commons.dbcp2.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:735)
org.apache.commons.dbcp2.BasicDataSource.createDataSource(BasicDataSource.java:605)
org.apache.commons.dbcp2.BasicDataSource.getConnection(BasicDataSource.java:809)
org.apache.beam.sdk.io.jdbc.JdbcIO$ReadFn.setup(JdbcIO.java:881)
Caused by: com.mysql.cj.jdbc.exceptions.CommunicationsException:
Communications link failure The last packet sent successfully to the
server was 0 milliseconds ago. The driver has not received any packets
from the server.
com.mysql.cj.jdbc.exceptions.SQLError.createCommunicationsException(SQLError.java:174)
com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:64)
com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:836)
com.mysql.cj.jdbc.ConnectionImpl.(ConnectionImpl.java:456)
com.mysql.cj.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:246)
com.mysql.cj.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:197)
org.apache.commons.dbcp2.DriverConnectionFactory.createConnection(DriverConnectionFactory.java:53)
org.apache.commons.dbcp2.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:355)
org.apache.commons.dbcp2.BasicDataSource.validateConnectionFactory(BasicDataSource.java:116)
org.apache.commons.dbcp2.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:731)
org.apache.commons.dbcp2.BasicDataSource.createDataSource(BasicDataSource.java:605)
org.apache.commons.dbcp2.BasicDataSource.getConnection(BasicDataSource.java:809)
org.apache.beam.sdk.io.jdbc.JdbcIO$ReadFn.setup(JdbcIO.java:881)
org.apache.beam.sdk.io.jdbc.JdbcIO$ReadFn$DoFnInvoker.invokeSetup(Unknown
Source)
org.apache.beam.runners.dataflow.worker.DoFnInstanceManagers$ConcurrentQueueInstanceManager.deserializeCopy(DoFnInstanceManagers.java:80)
org.apache.beam.runners.dataflow.worker.DoFnInstanceManagers$ConcurrentQueueInstanceManager.peek(DoFnInstanceManagers.java:62)
org.apache.beam.runners.dataflow.worker.UserParDoFnFactory.create(UserParDoFnFactory.java:95)
org.apache.beam.runners.dataflow.worker.DefaultParDoFnFactory.create(DefaultParDoFnFactory.java:75)
org.apache.beam.runners.dataflow.worker.IntrinsicMapTaskExecutorFactory.createParDoOperation(IntrinsicMapTaskExecutorFactory.java:264)
org.apache.beam.runners.dataflow.worker.IntrinsicMapTaskExecutorFactory.access$000(IntrinsicMapTaskExecutorFactory.java:86)
org.apache.beam.runners.dataflow.worker.IntrinsicMapTaskExecutorFactory$1.typedApply(IntrinsicMapTaskExecutorFactory.java:183)
org.apache.beam.runners.dataflow.worker.IntrinsicMapTaskExecutorFactory$1.typedApply(IntrinsicMapTaskExecutorFactory.java:165)
org.apache.beam.runners.dataflow.worker.graph.Networks$TypeSafeNodeFunction.apply(Networks.java:63)
org.apache.beam.runners.dataflow.worker.graph.Networks$TypeSafeNodeFunction.apply(Networks.java:50)
org.apache.beam.runners.dataflow.worker.graph.Networks.replaceDirectedNetworkNodes(Networks.java:87)
org.apache.beam.runners.dataflow.worker.IntrinsicMapTaskExecutorFactory.create(IntrinsicMapTaskExecutorFactory.java:125)
org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:352)
org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:305)
org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
java.util.concurrent.FutureTask.run(FutureTask.java:266)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
java.lang.Thread.run(Thread.java:748) Caused by:
com.mysql.cj.exceptions.CJCommunicationsException: Communications link
failure The last packet sent successfully to the server was 0
milliseconds ago. The driver has not received any packets from the
server. sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
java.lang.reflect.Constructor.newInstance(Constructor.java:423)
com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:61)
com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:105)
com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:151)
com.mysql.cj.exceptions.ExceptionFactory.createCommunicationsException(ExceptionFactory.java:167)
com.mysql.cj.protocol.a.NativeSocketConnection.connect(NativeSocketConnection.java:91)
com.mysql.cj.NativeSession.connect(NativeSession.java:144)
com.mysql.cj.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:956)
com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:826)
... 32 more Caused by: java.net.ConnectException: Connection timed out
(Connection timed out) java.net.PlainSocketImpl.socketConnect(Native
Method)
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
java.net.Socket.connect(Socket.java:589)
sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:673)
sun.security.ssl.BaseSSLSocketImpl.connect(BaseSSLSocketImpl.java:173)
com.google.cloud.sql.core.CoreSocketFactory.createSslSocket(CoreSocketFactory.java:233)
com.google.cloud.sql.core.CoreSocketFactory.connect(CoreSocketFactory.java:185)
com.google.cloud.sql.mysql.SocketFactory.connect(SocketFactory.java:48)
com.google.cloud.sql.mysql.SocketFactory.connect(SocketFactory.java:38)
com.mysql.cj.protocol.a.NativeSocketConnection.connect(NativeSocketConnection.java:65)
... 35 more
Plz any help will be highly appreciated!
Thanks in advance
You can use below piece of code to establish the connection:
Pipeline p = Pipeline.create(options);
//Increase pool size based on your records
ComboPooledDataSource dataSource = new ComboPooledDataSource();
dataSource.setDriverClass("com.mysql.jdbc.Driver");
dataSource.setJdbcUrl(
"jdbc:mysql://google/test?cloudSqlInstance=dataflowtest-:us-central1:sql-test&socketFactory=com.google.cloud.sql.mysql.SocketFactory");
dataSource.setUser("root");
dataSource.setPassword("root");
dataSource.setMaxPoolSize(10);
dataSource.setInitialPoolSize(6);
JdbcIO.DataSourceConfiguration config = JdbcIO.DataSourceConfiguration.create(dataSource);
// ADD rewriteBatchedStatements=true to improve write speed"
PCollection<KV<String, String>> sqlResult = p.apply(JdbcIO.<KV<String, String>>read()
.withDataSourceConfiguration(config)
.withQuery("select * from test_table").withCoder(KvCoder.of(StringUtf8Coder.of(), StringUtf8Coder.of()))
.withRowMapper(new JdbcIO.RowMapper<KV<String, String>>() {
private static final long serialVersionUID = 1L;
public KV<String, String> mapRow(ResultSet resultSet) throws Exception {
return KV.of(resultSet.getString(1), resultSet.getString(2));
}
}));
Add below dependency in pom.xml
<dependency>
<groupId>org.apache.beam</groupId>
<artifactId>beam-sdks-java-io-jdbc</artifactId>
<version>2.17.0</version>
</dependency>
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<version>5.1.25</version>
</dependency>
<dependency>
<groupId>com.google.cloud.sql</groupId>
<artifactId>mysql-socket-factory</artifactId>
<version>1.0.0</version>
</dependency>
This should work..
If possible try the below code for sql connection:
connection = connectToCloudSql(map.get(LiteralConstant.URL.toString()),
map.get(LiteralConstant.USERNAME.toString()), map.get(LiteralConstant.PASSWORD.toString()));
Then use the below piece of code to get result from sql connection:
statement = connection.prepareCall("query");
statement.execute();
resultSet = statement.getResultSet();
ResultSetMetaData rsmd = resultSet.getMetaData();
int count = rsmd.getColumnCount();
if(!resultSet.next() || count < 1)
throw new ConnectionFailureException("Failed to connect to Cloud SQL");
for (int k = 1; k <= count; k++) {
row.set(rsmd.getColumnName(k), resultSet.getString(k));
}
Get the above result in PCollection
Note: Don't forget to enable Cloud sql api and Cloud sql admin api.
Maven dependency:
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<version>5.1.25</version>
</dependency>
<dependency>
<groupId>com.google.cloud.sql</groupId>
<artifactId>mysql-socket-factory</artifactId> <!-- mysql-socket-factory-connector-j-6 if using 6.x.x -->
<version>1.0.0</version>
</dependency>
This above piece of code worked in my case. Let me know in case this solution works for you.
I am trying to implement Swagger for documentation, so that it picks both JSR 303 validation annotations and Spring Webflux based classes.
In Spring Boot project, I am currently using:
Spring Webflux framework for some classes
JSR 303 annotations on domain classes for Bean Validations and Constraints
Swagger integration with Spring Fox framework for generating OpenAPI documentation.
In build.gradle, following dependencies are used and I am able to make Spring Webflux + SpringFox work but only JSR 303 is giving issues:
springBootVersion = '2.0.2.RELEASE'
//JavaX-validation
'javax.validation:validation-api:2.0.0.Final',
'javax.el:javax.el-api:3.0.0',
'org.glassfish.web:javax.el:2.2.6'
//SpringFox-Swagger
'io.springfox:springfox-swagger2:2.9.2',
'io.springfox:springfox-spring-webflux:3.0.0-SNAPSHOT',
'io.github.swagger2markup:swagger2markup-gradle-plugin:1.3.3'
Here is the issue :
When below dependency is added to achieve "Spring Webflux + SpringFox + JSR 303",
**'io.springfox:springfox-bean-validators:2.9.2'**
I am getting the following error,
[WARN ] 2018-11-01 10:16:50.535 [main] SpringApplication - Unable to close ApplicationContext
java.lang.IllegalStateException: Failed to introspect Class
springfox.documentation.swagger2.configuration.Swagger2DocumentationConfiguration] from ClassLoader [sun.misc.Launcher$AppClassLoader#764c12b6]
at
org.springframework.util.ReflectionUtils.getDeclaredMethods(ReflectionUtils.java:659) ~[spring-core-5.0.6.RELEASE.jar:5.0.6.RELEASE]
at
org.springframework.util.ReflectionUtils.doWithMethods(ReflectionUtils.java:556 ) ~[spring-core-5.0.6.RELEASE.jar:5.0.6.RELEASE]
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.ge tTypeForFactoryMethod(AbstractAutowireCapableBeanFactory.java:724) ~[spring- beans-5.0.6.RELEASE.jar:5.0.6.RELEASE]
at
org.springframework.boot.SpringApplication.handleExitCode(SpringApplication.java:861) ~[spring-boot-2.0.2.RELEASE.jar:2.0.2.RELEASE]
at
org.springframework.boot.SpringApplication.handleRunFailure(SpringApplication.java:810) [spring-boot-2.0.2.RELEASE.jar:2.0.2.RELEASE]
at
org.springframework.boot.SpringApplication.run(SpringApplication.java:1255) [spring-boot-2.0.2.RELEASE.jar:2.0.2.RELEASE]
Caused by: java.lang.NoClassDefFoundError:
org/springframework/web/servlet/HandlerMapping
at java.lang.Class.getDeclaredMethods0(Native Method) ~[?:1.8.0_92]
at java.lang.Class.privateGetDeclaredMethods(Unknown Source) ~[?:1.8.0_92]
at java.lang.Class.getDeclaredMethods(Unknown Source) ~[?:1.8.0_92]
at org.springframework.util.ReflectionUtils.getDeclaredMethods(ReflectionUtils.java:641) ~[spring-core-5.0.6.RELEASE.jar:5.0.6.RELEASE]
... 20 more
Caused by: java.lang.ClassNotFoundException: org.springframework.web.servlet.HandlerMapping
at java.net.URLClassLoader.findClass(Unknown Source) ~[?:1.8.0_92]
at java.lang.ClassLoader.loadClass(Unknown Source) ~[?:1.8.0_92]
Several blogs say that,SpringBoot version 2x doesn't support JSR 303. I am not sure but I tried to see it doesn't work.
Please advise.
Also, If JSR 303 is not supported currently, what other Validation annotations can be used apart from JSR 303 ?
I don't want to use Swagger core annotations, only because it is messing up code readability.
Add in your pom.xml:
<dependency>
<groupId>io.springfox</groupId>
<artifactId>springfox-swagger2</artifactId>
<version>3.0.0</version>
<scope>compile</compile>
</dependency>
<dependency>
<groupId>io.springfox</groupId>
<artifactId>springfox-swagger-ui</artifactId>
<version>3.0.0</version>
</dependency>
<dependency>
<groupId>io.springfox</groupId>
<artifactId>springfox-spring-webflux</artifactId>
<version>3.0.0</version>
</dependency>
And use the annotation:
#EnableSwagger2
In you swagger configuration.
Officially springfox doesn't support spring webflux(more informations you can read here: springfox issues).
But you can try SNAPSHOT version like this: webflux-swagger.
As I can see version greater than 2.3.2, support for bean validation annotations:pringfox-support-for-jsr-303.
I have neo4j 2.2.5 community edition and java code on the same machine (windows 7), I an trying the following code to connect to neo4j (without HA)
Configuration config = new BaseConfiguration();
config.setProperty(Neo4jGraph.CONFIG_DIRECTORY, "C:\\Users\\mike\\Documents\\Neo4j");
config.setProperty("gremlin.neo4j.conf.cache_type", "none");
Graph toyGraph = Neo4jGraph.open(config);
but I am getting the following exception
Exception in thread "main" java.lang.RuntimeException: Error starting org.neo4j.kernel.EmbeddedGraphDatabase, C:\Users\mike\Documents\Neo4j
at org.neo4j.kernel.InternalAbstractGraphDatabase.run(InternalAbstractGraphDatabase.java:334)
at org.neo4j.kernel.EmbeddedGraphDatabase.<init>(EmbeddedGraphDatabase.java:59)
at org.neo4j.graphdb.factory.GraphDatabaseFactory.newDatabase(GraphDatabaseFactory.java:108)
at org.neo4j.graphdb.factory.GraphDatabaseFactory$1.newDatabase(GraphDatabaseFactory.java:95)
at org.neo4j.graphdb.factory.GraphDatabaseBuilder.newGraphDatabase(GraphDatabaseBuilder.java:176)
at org.neo4j.tinkerpop.api.impl.Neo4jFactoryImpl.newGraphDatabase(Neo4jFactoryImpl.java:44)
at org.neo4j.tinkerpop.api.Neo4jFactory$Builder.open(Neo4jFactory.java:32)
at org.apache.tinkerpop.gremlin.neo4j.structure.Neo4jGraph.<init>(Neo4jGraph.java:130)
at org.apache.tinkerpop.gremlin.neo4j.structure.Neo4jGraph.open(Neo4jGraph.java:144)
at ca.concordia.orchestrator.test.TestNeo4j.main(TestNeo4j.java:33)
Caused by: org.neo4j.kernel.lifecycle.LifecycleException: Component 'org.neo4j.kernel.extension.KernelExtensions#7c0edd8f' failed to initialize. Please see attached cause exception.
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.init(LifeSupport.java:489)
at org.neo4j.kernel.lifecycle.LifeSupport.init(LifeSupport.java:72)
at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:106)
at org.neo4j.kernel.InternalAbstractGraphDatabase.run(InternalAbstractGraphDatabase.java:330)
... 9 more
Caused by: java.lang.NoClassDefFoundError: org/apache/lucene/document/Fieldable
at org.neo4j.kernel.api.impl.index.NodeRangeDocumentLabelScanStorageStrategy.<init>(NodeRangeDocumentLabelScanStorageStrategy.java:71)
at org.neo4j.kernel.api.impl.index.LuceneLabelScanStoreExtension.newKernelExtension(LuceneLabelScanStoreExtension.java:73)
at org.neo4j.kernel.api.impl.index.LuceneLabelScanStoreExtension.newKernelExtension(LuceneLabelScanStoreExtension.java:39)
at org.neo4j.kernel.extension.KernelExtensions.init(KernelExtensions.java:66)
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.init(LifeSupport.java:483)
... 12 more
Caused by: java.lang.ClassNotFoundException: org.apache.lucene.document.Fieldable
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 17 more
I have following dependencies in pom.xml
<dependency>
<groupId>org.apache.tinkerpop</groupId>
<artifactId>gremlin-core</artifactId>
<version>3.0.1-incubating</version>
</dependency>
<dependency>
<groupId>org.apache.tinkerpop</groupId>
<artifactId>tinkergraph-gremlin</artifactId>
<version>3.0.1-incubating</version>
</dependency>
<!-- Neo4j -->
<dependency>
<groupId>org.apache.tinkerpop</groupId>
<artifactId>neo4j-gremlin</artifactId>
<version>3.0.1-incubating</version>
</dependency>
<!-- neo4j-tinkerpop-api-impl is NOT Apache 2 licensed - more information
below -->
<dependency>
<groupId>org.neo4j</groupId>
<artifactId>neo4j-tinkerpop-api-impl</artifactId>
<version>0.1-2.2</version>
</dependency>
Any idea what could be the problem ? Does Thinkerpop3 work with neo4j 2.2.5?
Looks like that there are problems with dependency versions in your setup.
If you look at neo4j-tinkerpop-api-impl - you can see that there is currently release for Neo4j 2.2.5 in master. But maven central has 2.2 version.
Ensure that you are using correct version of Neo4j dependencies everywhere.
In maven dependency tree can be helpful.
I have a simple EJB 3.1 project which uses Glassfish 3.1.2 as AS and Maven 3 for dependency management. In the pom.xml of ejb project I set the generateClient option to true. I successfully deployed my ear to the server and then I tried to create a simple standalone client. Here is the pom.xml of the client:
<project>
<modelVersion>4.0.0</modelVersion>
<groupId>duan</groupId>
<artifactId>ejb31-app-client</artifactId>
<version>1.0-SNAPSHOT</version>
<name>ejb31-app-client</name>
<description>my app client</description>
<dependencies>
<dependency>
<groupId>org.glassfish.main.appclient.client</groupId>
<artifactId>gf-client-module</artifactId>
<version>3.1.2</version>
<type>pom</type>
<scope>compile</scope>
<exclusions>
<exclusion>
<artifactId>tools</artifactId>
<groupId>com.sun</groupId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>duan</groupId>
<artifactId>ejb31</artifactId>
<version>1.0-SNAPSHOT</version>
<type>ejb-client</type>
</dependency>
</dependencies>
where ejb31 is my ejb project for which I client is generated.
The Java class used in my application client is:
public class Test {
private static HelloWorldBeanRemote helloWorldBean;
public static void main(String[] args) {
String jndiPath = "java:global/ejb31-ear-1.0/ejb31-ejb/HelloWorldBean";
try {
Context ctx = new InitialContext();
System.out.println("Looking up bean at: " + jndiPath);
helloWorldBean = (HelloWorldBeanRemote) ctx.lookup(jndiPath);
System.out.println("Found helloWorldBean:" + helloWorldBean);
System.out.println("Calling sayHello():");
String greeting = helloWorldBean.sayHello();
System.out.println("HelloWorldBean said:" + greeting);
} catch (NamingException e) {
System.err.println("Could not find HelloWorldBeanRemote!");
System.err.println("JNDI path used for lookup:" + jndiPath);
e.printStackTrace();
}
}
}
The application client is compiled without problem by Maven. If I run this from Eclipse I get this exception:
Caused by: java.lang.ClassNotFoundException: ro.duan.ejb.HelloWorldBeanRemote
at java.net.URLClassLoader$1.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at com.sun.ejb.EJBUtils.getBusinessIntfClassLoader(EJBUtils.java:688)
at com.sun.ejb.EJBUtils.loadGeneratedRemoteBusinessClasses(EJBUtils.java:463)
at com.sun.ejb.EJBUtils.lookupRemote30BusinessObject(EJBUtils.java:414)
... 7 more
But then if I manually add ejb-client.jar to the build path it works like a charm. So my conclusion is that somehow ejb-client.jar is available at compile time, but not at runtime. Any thoughts on how to solve this?
Remove the following line:
<scope>compile</scope>
Read more about dependency scope at:
http://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#Dependency_Scope