Spring cloud + Keyclock + Angular - CORS Error - spring-security

I'm trying to build a microservices application using spring-cloud + spring-gateway + keycloak + angular-frontend. In my application there is a api-gateway application that handle all the request and later will dispatch those request to the right microservice.
For the front-end I'm using angular and for test the endpoints I'm using postman. At the moment I'm having a CORS problem.
I've configured the api-gateway in this way:
spring:
cloud.gateway:
globalcors.corsConfigurations:
'[/**]':
allowedOrigins: '*'
allowedPorts: '*'
allowedHeaders: '*'
allowedMethods:
- GET
- POST
- PUT
- DELETE
- OPTIONS
security.oauth2:
client:
provider:
keycloak:
authorization-uri: ${app.config.keycloak.url}/realms/${app.config.keycloak.realm}/protocol/openid-connect/auth
token-uri: ${app.config.keycloak.url}/realms/${app.config.keycloak.realm}/protocol/openid-connect/token
userinfo-uri: ${app.config.keycloak.url}/realms/${app.config.keycloak.realm}/protocol/openid-connect/userinfo
jwk-set-uri: ${app.config.keycloak.url}/realms/${app.config.keycloak.realm}/protocol/openid-connect/certs
user-name-attribute: preferred_username
user-info-authentication-method: header
registration:
keycloak:
provider: keycloak
client-id: ${app.config.keycloak.client}
authorization-grant-type: authorization_code
redirect-uri: '{baseUrl}/login/oauth2/code/keycloak'
scope: openid
resourceserver:
jwt:
jwk-set-uri: ${app.config.keycloak.url}/realms/${app.config.keycloak.realm}/protocol/openid-connect/certs
Below the securityConfig of gateway:
#Configuration
public class SecurityConfig {
#Bean
SecurityWebFilterChain springSecurityFilterChain(ServerHttpSecurity http) {
http.authorizeExchange()
.anyExchange().authenticated()
.and().cors()
.and().csrf().disable().oauth2Login()
.and().oauth2ResourceServer().jwt();
return http.build();
}
}
And below the securityConfig of ms:
#EnableWebSecurity
public class SecurityConfig {
#Bean
protected SecurityFilterChain filterChain(HttpSecurity http) throws Exception {
http.authorizeRequests(authorizeRequests -> authorizeRequests
.antMatchers("/v3/api-docs/**").permitAll()
.antMatchers("/swagger-ui/**").permitAll()
.anyRequest().authenticated()
).oauth2ResourceServer(OAuth2ResourceServerConfigurer::jwt);
return http.build();
}
}
Said that, if for instance I make a request with postman to the path /api/controller I get the correct response. If I'm using the angular client and try to access an end-point I get this error:
Preflight: (failed)net::ERR_CONNECTION_REFUSED
Do you have any idea how to fix this? Any help will be appreciated. Thanks to all
Below my gateway pom dependencies:
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-gateway</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-security</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-oauth2-client</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-oauth2-resource-server</artifactId>
</dependency>

Related

Spring actuator and Spring Security to allow access - all. Whitelabel Error Page

Gradle configuration:
plugins {
id("org.springframework.boot") version "2.3.3.RELEASE"
}
Spring Security configuration:
#Configuration
#RequiredArgsConstructor
public class SecurityConfig extends WebSecurityConfigurerAdapter {
#Bean
public PasswordEncoder passwordEncoder() {
int rounds = 12;
return new BCryptPasswordEncoder(rounds);
}
#Override
protected void configure(HttpSecurity http) throws Exception {
http
.csrf().disable()
.sessionManagement()
.sessionCreationPolicy(SessionCreationPolicy.STATELESS)
.and()
.httpBasic()
.and()
.authorizeRequests()
.antMatchers("/actuator/**").permitAll()
.anyRequest().authenticated()
.and()
.formLogin()
.permitAll();
}
}
Spring Boot configuration:
server:
port: 9090
management:
server:
port: 90901
endpoints:
web:
exposure:
include: '*'
I would like to set up access without authorization to actuator endpoints. However, when I try to request information from the browser line, I get a display error.
Whitelabel Error Page
This application has no explicit mapping for /error, so you are seeing this as a fallback.
Can anyone explain why this is happening and how to fix it?

spring security oauth2 returning 404 after authorize

I have setup OAuth2 client application with the following configuration in my application.yml file
spring:
security:
oauth2:
client:
registration:
octa:
client-id: <confidential>
client-secret: <confidential>
scope: openid
redirect-uri: http://localhost:8555/oauth/callback
clientName: octa
provider: octa
provider:
octa:
issuer-uri: https://dev-7858070.okta.com/oauth2/default
user-name-atttibute: name
server:
port: 8555
My spring boot application is running on port 8555 and below is my security configuration
#Configuration
public class ApplicationSecurityConfiguration extends WebSecurityConfigurerAdapter {
#Override
public void configure(HttpSecurity http) throws Exception {
http.antMatcher("/**").authorizeRequests()
.antMatchers("/", "/login**","/callback/", "/webjars/**", "/error**", "/oauth/**")
.permitAll()
.anyRequest().authenticated()
.and()
.oauth2Login();
}
}
When the authorization server returns the url http://localhost:8555/oauth/callback?code=t8oJIWdR2lTgY1_06WPzR2d with 404 error.
I should ideally be getting the access token. Where am I going wrong?

Batch update failure with HTTP status code 404; discarding 1 replication tasks

After starting the service discovery, When the proxy microservice is registered, the eureka is throwing the following error:
2020-06-11 21:07:19.778 ERROR 1040 --- [et_localhost-19] c.n.e.cluster.ReplicationTaskProcessor : Batch update failure with HTTP status code 404; discarding 1 replication tasks
2020-06-11 21:07:19.778 WARN 1040 --- [et_localhost-19] c.n.eureka.util.batcher.TaskExecutors : Discarding 1 tasks of TaskBatchingWorker-target_localhost-19 due to permanent error
2020-06-11 21:07:49.782 ERROR 1040 --- [et_localhost-19] c.n.e.cluster.ReplicationTaskProcessor : Batch update failure with HTTP status code 404; discarding 1 replication tasks
2020-06-11 21:07:49.782 WARN 1040 --- [et_localhost-19] c.n.eureka.util.batcher.TaskExecutors : Discarding 1 tasks of TaskBatchingWorker-target_localhost-19 due to permanent error
2020-06-11 21:08:11.629 INFO 1040 --- [a-EvictionTimer] c.n.e.registry.AbstractInstanceRegistry : Running the evict task with compensationTime 0ms
2020-06-11 21:08:19.796 ERROR 1040 --- [et_localhost-19] c.n.e.cluster.ReplicationTaskProcessor : Batch update failure with HTTP status code 404; discarding 1 replication tasks
2020-06-11 21:08:19.796 WARN 1040 --- [et_localhost-19] c.n.eureka.util.batcher.TaskExecutors : Discarding 1 tasks of TaskBatchingWorker-target_localhost-19 due to permanent error
Error Log
The dependencies are as follows:
<properties>
<java.version>1.8</java.version>
<spring-cloud.version>Hoxton.SR3</spring-cloud.version>
<springfox-swagger2-version>2.9.2</springfox-swagger2-version>
<springfox-swagger2-ui-version>2.9.2</springfox-swagger2-ui-version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-zuul</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-config</artifactId>
</dependency>
</dependencies>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-dependencies</artifactId>
<version>${spring-cloud.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
bootstrap.yml
spring:
application:
name: tng-crs-eureka
server:
port: 8761
servlet:
context-path: /eureka
eureka:
client:
registerWithEureka: false
fetchRegistry: false
Any help is highly appreciated. I have tried by disabling the batch update tasks also. But, it didn't worked.
Finally, The mistake that I have made was figured out. I have set the context-path of /eureka. And that is the reason for all this trouble. Removing the context path of eureka resolves the issue for my case.

how to connect to Cloud SQL from Google DataFlow

I'm trying to create a pipeline task using beam java SDK and Google Dataflow to move data from Cloud SQL to Elastic search
I've created the following class main method:
public static void main(String[] args) throws Exception{
DataflowPipelineOptions options = PipelineOptionsFactory.as(DataflowPipelineOptions.class);
options.setProject("staging");
options.setTempLocation("gs://csv_to_sql_staging/temp");
options.setRunner(DataflowRunner.class);
options.setGcpTempLocation("gs://csv_to_sql_staging/temp");
options.setUsePublicIps(false);
options.setJobName("tamer-new");
options.setSubnetwork("regions/us-central1/subnetworks/new-network");
final List<String> SCOPES = Arrays.asList(
"https://www.googleapis.com/auth/cloud-platform",
"https://www.googleapis.com/auth/devstorage.full_control",
"https://www.googleapis.com/auth/userinfo.email",
"https://www.googleapis.com/auth/datastore",
"https://www.googleapis.com/auth/sqlservice.admin",
"https://www.googleapis.com/auth/pubsub");
options.setGcpCredential(ServiceAccountCredentials.fromStream(new ElasticSearchIO().getClass().getResourceAsStream("/staging-b648da5d2b9b.json")).createScoped(SCOPES)); options.setServiceAccount("data-flow#staging.iam.gserviceaccount.com");
Pipeline p = Pipeline.create(options);
p.begin();
PCollection < List < String >> rows = p.apply(JdbcIO. < List < String >> read().withQuery("select u.id, u.name from user_table").withDataSourceConfiguration(JdbcIO.DataSourceConfiguration.create("com.mysql.jdbc.Driver", "jdbc:mysql://google/nameDB_new?cloudSqlInstance=staging:europe-west1:sql-staging-instance&socketFactory=com.google.cloud.sql.mysql.SocketFactory&useUnicode=true&characterEncoding=UTF-8&user=user&password=password&useSSL=false")).withRowMapper(new RowMapper < List < String >> () {
#Override public List < String > mapRow(ResultSet resultSet) throws Exception {
List < String > addRow = new ArrayList < String > ();
for (int i = 1; i <= resultSet.getMetaData().getColumnCount(); i++) {
addRow.add(i - 1, String.valueOf(resultSet.getObject(i)));
}
//LOG.info(String.join(",", addRow));
return addRow;
}
})
.withCoder(ListCoder.of(StringUtf8Coder. < Object > of ()))
);
Write w = ElasticsearchIO.write().withConnectionConfiguration(
ElasticsearchIO.ConnectionConfiguration.create(new String[] {
"https://host:9243"
}, "user-temp", "String").withUsername("elastic").withPassword("password")
);
rows.apply(w.compose(new SerializableFunction() {
#Override public Object apply(Object input) {
// TODO Auto-generated method stub
return input;
}
}));
p.run().waitUntilFinish();
}
and below is the pom.xml file :
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.harmonica.dataflow</groupId>
<artifactId>com-harmonica-dataflow</artifactId>
<version>0.0.1-SNAPSHOT</version>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<maven-compiler-plugin.version>3.7.0</maven-compiler-plugin.version>
<exec-maven-plugin.version>1.6.0</exec-maven-plugin.version>
<slf4j.version>1.7.25</slf4j.version>
<beam.version>2.19.0</beam.version>
</properties>
<repositories>
<repository>
<id>ossrh.snapshots</id>
<name>Sonatype OSS Repository Hosting</name>
<url>https://oss.sonatype.org/content/repositories/snapshots/</url>
<releases>
<enabled>false</enabled>
</releases>
<snapshots>
<enabled>true</enabled>
</snapshots>
</repository>
</repositories>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>${maven-compiler-plugin.version}</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
</plugins>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>${exec-maven-plugin.version}</version>
<configuration>
<cleanupDaemonThreads>false</cleanupDaemonThreads>
</configuration>
</plugin>
</plugins>
</pluginManagement>
</build>
<dependencies>
<!-- Beam Lib -->
<dependency>
<groupId>org.apache.beam</groupId>
<artifactId>beam-sdks-java-core</artifactId>
<version>${beam.version}</version>
</dependency>
<dependency>
<groupId>org.apache.beam</groupId>
<artifactId>beam-runners-google-cloud-dataflow-java</artifactId>
<version>${beam.version}</version>
</dependency>
<dependency>
<groupId>org.apache.beam</groupId>
<artifactId>beam-sdks-java-io-elasticsearch</artifactId>
<version>${beam.version}</version>
</dependency>
<dependency>
<groupId>org.apache.beam</groupId>
<artifactId>beam-sdks-java-io-jdbc</artifactId>
<version>${beam.version}</version>
</dependency>
<dependency>
<groupId>org.apache.beam</groupId>
<artifactId>beam-sdks-java-io-google-cloud-platform</artifactId>
<version>${beam.version}</version>
</dependency>
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<version>8.0.19</version>
</dependency>
<dependency>
<groupId>com.google.cloud.sql</groupId>
<artifactId>mysql-socket-factory-connector-j-8</artifactId>
<version>1.0.15</version>
</dependency>
<!-- slf4j API frontend binding with JUL backend -->
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>${slf4j.version}</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-jdk14</artifactId>
<version>${slf4j.version}</version>
</dependency>
</dependencies>
</project>
and when I execute this command:
man exec mvn compile exec:java -Dexec.mainClass=com.dataflow.ElasticSearchIO
The worker started successfully but then cant connect to the the Cloud SQL:
even thought I've done the flowing:
I've created a service account with owner access to the project and passed it to the runner options
I've created a VPC network with a name of new-network with a range of IP of 190.10.0.0/16 and assigned it to the pipeline options and then whitlisted this range in the cloud SQL
and however Im still getting this error:
Error message from worker: java.lang.RuntimeException:
org.apache.beam.sdk.util.UserCodeException: java.sql.SQLException:
Cannot create PoolableConnectionFactory (Communications link failure
The last packet sent successfully to the server was 0 milliseconds
ago. The driver has not received any packets from the server.)
org.apache.beam.runners.dataflow.worker.IntrinsicMapTaskExecutorFactory$1.typedApply(IntrinsicMapTaskExecutorFactory.java:194)
org.apache.beam.runners.dataflow.worker.IntrinsicMapTaskExecutorFactory$1.typedApply(IntrinsicMapTaskExecutorFactory.java:165)
org.apache.beam.runners.dataflow.worker.graph.Networks$TypeSafeNodeFunction.apply(Networks.java:63)
org.apache.beam.runners.dataflow.worker.graph.Networks$TypeSafeNodeFunction.apply(Networks.java:50)
org.apache.beam.runners.dataflow.worker.graph.Networks.replaceDirectedNetworkNodes(Networks.java:87)
org.apache.beam.runners.dataflow.worker.IntrinsicMapTaskExecutorFactory.create(IntrinsicMapTaskExecutorFactory.java:125)
org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:352)
org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:305)
org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
java.util.concurrent.FutureTask.run(FutureTask.java:266)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
java.lang.Thread.run(Thread.java:748) Caused by:
org.apache.beam.sdk.util.UserCodeException: java.sql.SQLException:
Cannot create PoolableConnectionFactory (Communications link failure
The last packet sent successfully to the server was 0 milliseconds
ago. The driver has not received any packets from the server.)
org.apache.beam.sdk.util.UserCodeException.wrap(UserCodeException.java:34)
org.apache.beam.sdk.io.jdbc.JdbcIO$ReadFn$DoFnInvoker.invokeSetup(Unknown
Source)
org.apache.beam.runners.dataflow.worker.DoFnInstanceManagers$ConcurrentQueueInstanceManager.deserializeCopy(DoFnInstanceManagers.java:80)
org.apache.beam.runners.dataflow.worker.DoFnInstanceManagers$ConcurrentQueueInstanceManager.peek(DoFnInstanceManagers.java:62)
org.apache.beam.runners.dataflow.worker.UserParDoFnFactory.create(UserParDoFnFactory.java:95)
org.apache.beam.runners.dataflow.worker.DefaultParDoFnFactory.create(DefaultParDoFnFactory.java:75)
org.apache.beam.runners.dataflow.worker.IntrinsicMapTaskExecutorFactory.createParDoOperation(IntrinsicMapTaskExecutorFactory.java:264)
org.apache.beam.runners.dataflow.worker.IntrinsicMapTaskExecutorFactory.access$000(IntrinsicMapTaskExecutorFactory.java:86)
org.apache.beam.runners.dataflow.worker.IntrinsicMapTaskExecutorFactory$1.typedApply(IntrinsicMapTaskExecutorFactory.java:183)
... 14 more Caused by: java.sql.SQLException: Cannot create
PoolableConnectionFactory (Communications link failure The last packet
sent successfully to the server was 0 milliseconds ago. The driver has
not received any packets from the server.)
org.apache.commons.dbcp2.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:735)
org.apache.commons.dbcp2.BasicDataSource.createDataSource(BasicDataSource.java:605)
org.apache.commons.dbcp2.BasicDataSource.getConnection(BasicDataSource.java:809)
org.apache.beam.sdk.io.jdbc.JdbcIO$ReadFn.setup(JdbcIO.java:881)
Caused by: com.mysql.cj.jdbc.exceptions.CommunicationsException:
Communications link failure The last packet sent successfully to the
server was 0 milliseconds ago. The driver has not received any packets
from the server.
com.mysql.cj.jdbc.exceptions.SQLError.createCommunicationsException(SQLError.java:174)
com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:64)
com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:836)
com.mysql.cj.jdbc.ConnectionImpl.(ConnectionImpl.java:456)
com.mysql.cj.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:246)
com.mysql.cj.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:197)
org.apache.commons.dbcp2.DriverConnectionFactory.createConnection(DriverConnectionFactory.java:53)
org.apache.commons.dbcp2.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:355)
org.apache.commons.dbcp2.BasicDataSource.validateConnectionFactory(BasicDataSource.java:116)
org.apache.commons.dbcp2.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:731)
org.apache.commons.dbcp2.BasicDataSource.createDataSource(BasicDataSource.java:605)
org.apache.commons.dbcp2.BasicDataSource.getConnection(BasicDataSource.java:809)
org.apache.beam.sdk.io.jdbc.JdbcIO$ReadFn.setup(JdbcIO.java:881)
org.apache.beam.sdk.io.jdbc.JdbcIO$ReadFn$DoFnInvoker.invokeSetup(Unknown
Source)
org.apache.beam.runners.dataflow.worker.DoFnInstanceManagers$ConcurrentQueueInstanceManager.deserializeCopy(DoFnInstanceManagers.java:80)
org.apache.beam.runners.dataflow.worker.DoFnInstanceManagers$ConcurrentQueueInstanceManager.peek(DoFnInstanceManagers.java:62)
org.apache.beam.runners.dataflow.worker.UserParDoFnFactory.create(UserParDoFnFactory.java:95)
org.apache.beam.runners.dataflow.worker.DefaultParDoFnFactory.create(DefaultParDoFnFactory.java:75)
org.apache.beam.runners.dataflow.worker.IntrinsicMapTaskExecutorFactory.createParDoOperation(IntrinsicMapTaskExecutorFactory.java:264)
org.apache.beam.runners.dataflow.worker.IntrinsicMapTaskExecutorFactory.access$000(IntrinsicMapTaskExecutorFactory.java:86)
org.apache.beam.runners.dataflow.worker.IntrinsicMapTaskExecutorFactory$1.typedApply(IntrinsicMapTaskExecutorFactory.java:183)
org.apache.beam.runners.dataflow.worker.IntrinsicMapTaskExecutorFactory$1.typedApply(IntrinsicMapTaskExecutorFactory.java:165)
org.apache.beam.runners.dataflow.worker.graph.Networks$TypeSafeNodeFunction.apply(Networks.java:63)
org.apache.beam.runners.dataflow.worker.graph.Networks$TypeSafeNodeFunction.apply(Networks.java:50)
org.apache.beam.runners.dataflow.worker.graph.Networks.replaceDirectedNetworkNodes(Networks.java:87)
org.apache.beam.runners.dataflow.worker.IntrinsicMapTaskExecutorFactory.create(IntrinsicMapTaskExecutorFactory.java:125)
org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:352)
org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:305)
org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
java.util.concurrent.FutureTask.run(FutureTask.java:266)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
java.lang.Thread.run(Thread.java:748) Caused by:
com.mysql.cj.exceptions.CJCommunicationsException: Communications link
failure The last packet sent successfully to the server was 0
milliseconds ago. The driver has not received any packets from the
server. sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
java.lang.reflect.Constructor.newInstance(Constructor.java:423)
com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:61)
com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:105)
com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:151)
com.mysql.cj.exceptions.ExceptionFactory.createCommunicationsException(ExceptionFactory.java:167)
com.mysql.cj.protocol.a.NativeSocketConnection.connect(NativeSocketConnection.java:91)
com.mysql.cj.NativeSession.connect(NativeSession.java:144)
com.mysql.cj.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:956)
com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:826)
... 32 more Caused by: java.net.ConnectException: Connection timed out
(Connection timed out) java.net.PlainSocketImpl.socketConnect(Native
Method)
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
java.net.Socket.connect(Socket.java:589)
sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:673)
sun.security.ssl.BaseSSLSocketImpl.connect(BaseSSLSocketImpl.java:173)
com.google.cloud.sql.core.CoreSocketFactory.createSslSocket(CoreSocketFactory.java:233)
com.google.cloud.sql.core.CoreSocketFactory.connect(CoreSocketFactory.java:185)
com.google.cloud.sql.mysql.SocketFactory.connect(SocketFactory.java:48)
com.google.cloud.sql.mysql.SocketFactory.connect(SocketFactory.java:38)
com.mysql.cj.protocol.a.NativeSocketConnection.connect(NativeSocketConnection.java:65)
... 35 more
Plz any help will be highly appreciated!
Thanks in advance
You can use below piece of code to establish the connection:
Pipeline p = Pipeline.create(options);
//Increase pool size based on your records
ComboPooledDataSource dataSource = new ComboPooledDataSource();
dataSource.setDriverClass("com.mysql.jdbc.Driver");
dataSource.setJdbcUrl(
"jdbc:mysql://google/test?cloudSqlInstance=dataflowtest-:us-central1:sql-test&socketFactory=com.google.cloud.sql.mysql.SocketFactory");
dataSource.setUser("root");
dataSource.setPassword("root");
dataSource.setMaxPoolSize(10);
dataSource.setInitialPoolSize(6);
JdbcIO.DataSourceConfiguration config = JdbcIO.DataSourceConfiguration.create(dataSource);
// ADD rewriteBatchedStatements=true to improve write speed"
PCollection<KV<String, String>> sqlResult = p.apply(JdbcIO.<KV<String, String>>read()
.withDataSourceConfiguration(config)
.withQuery("select * from test_table").withCoder(KvCoder.of(StringUtf8Coder.of(), StringUtf8Coder.of()))
.withRowMapper(new JdbcIO.RowMapper<KV<String, String>>() {
private static final long serialVersionUID = 1L;
public KV<String, String> mapRow(ResultSet resultSet) throws Exception {
return KV.of(resultSet.getString(1), resultSet.getString(2));
}
}));
Add below dependency in pom.xml
<dependency>
<groupId>org.apache.beam</groupId>
<artifactId>beam-sdks-java-io-jdbc</artifactId>
<version>2.17.0</version>
</dependency>
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<version>5.1.25</version>
</dependency>
<dependency>
<groupId>com.google.cloud.sql</groupId>
<artifactId>mysql-socket-factory</artifactId>
<version>1.0.0</version>
</dependency>
This should work..
If possible try the below code for sql connection:
connection = connectToCloudSql(map.get(LiteralConstant.URL.toString()),
map.get(LiteralConstant.USERNAME.toString()), map.get(LiteralConstant.PASSWORD.toString()));
Then use the below piece of code to get result from sql connection:
statement = connection.prepareCall("query");
statement.execute();
resultSet = statement.getResultSet();
ResultSetMetaData rsmd = resultSet.getMetaData();
int count = rsmd.getColumnCount();
if(!resultSet.next() || count < 1)
throw new ConnectionFailureException("Failed to connect to Cloud SQL");
for (int k = 1; k <= count; k++) {
row.set(rsmd.getColumnName(k), resultSet.getString(k));
}
Get the above result in PCollection
Note: Don't forget to enable Cloud sql api and Cloud sql admin api.
Maven dependency:
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<version>5.1.25</version>
</dependency>
<dependency>
<groupId>com.google.cloud.sql</groupId>
<artifactId>mysql-socket-factory</artifactId> <!-- mysql-socket-factory-connector-j-6 if using 6.x.x -->
<version>1.0.0</version>
</dependency>
This above piece of code worked in my case. Let me know in case this solution works for you.

Maven 3 ejb-client dependency visible at compile time not visible at runtime for ejb 3.1 app client

I have a simple EJB 3.1 project which uses Glassfish 3.1.2 as AS and Maven 3 for dependency management. In the pom.xml of ejb project I set the generateClient option to true. I successfully deployed my ear to the server and then I tried to create a simple standalone client. Here is the pom.xml of the client:
<project>
<modelVersion>4.0.0</modelVersion>
<groupId>duan</groupId>
<artifactId>ejb31-app-client</artifactId>
<version>1.0-SNAPSHOT</version>
<name>ejb31-app-client</name>
<description>my app client</description>
<dependencies>
<dependency>
<groupId>org.glassfish.main.appclient.client</groupId>
<artifactId>gf-client-module</artifactId>
<version>3.1.2</version>
<type>pom</type>
<scope>compile</scope>
<exclusions>
<exclusion>
<artifactId>tools</artifactId>
<groupId>com.sun</groupId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>duan</groupId>
<artifactId>ejb31</artifactId>
<version>1.0-SNAPSHOT</version>
<type>ejb-client</type>
</dependency>
</dependencies>
where ejb31 is my ejb project for which I client is generated.
The Java class used in my application client is:
public class Test {
private static HelloWorldBeanRemote helloWorldBean;
public static void main(String[] args) {
String jndiPath = "java:global/ejb31-ear-1.0/ejb31-ejb/HelloWorldBean";
try {
Context ctx = new InitialContext();
System.out.println("Looking up bean at: " + jndiPath);
helloWorldBean = (HelloWorldBeanRemote) ctx.lookup(jndiPath);
System.out.println("Found helloWorldBean:" + helloWorldBean);
System.out.println("Calling sayHello():");
String greeting = helloWorldBean.sayHello();
System.out.println("HelloWorldBean said:" + greeting);
} catch (NamingException e) {
System.err.println("Could not find HelloWorldBeanRemote!");
System.err.println("JNDI path used for lookup:" + jndiPath);
e.printStackTrace();
}
}
}
The application client is compiled without problem by Maven. If I run this from Eclipse I get this exception:
Caused by: java.lang.ClassNotFoundException: ro.duan.ejb.HelloWorldBeanRemote
at java.net.URLClassLoader$1.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at com.sun.ejb.EJBUtils.getBusinessIntfClassLoader(EJBUtils.java:688)
at com.sun.ejb.EJBUtils.loadGeneratedRemoteBusinessClasses(EJBUtils.java:463)
at com.sun.ejb.EJBUtils.lookupRemote30BusinessObject(EJBUtils.java:414)
... 7 more
But then if I manually add ejb-client.jar to the build path it works like a charm. So my conclusion is that somehow ejb-client.jar is available at compile time, but not at runtime. Any thoughts on how to solve this?
Remove the following line:
<scope>compile</scope>
Read more about dependency scope at:
http://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#Dependency_Scope

Resources