I am developing a Service Account application to obtain Token for accessing downstream services. However, I am facing an issue with compilation. Eclipse prompts an error "The type com.google.api.client.auth.oauth2.Credential$Builder cannot be resolved. It is indirectly referenced from required .class files". Code snippet below:
import com.google.api.client.googleapis.auth.oauth2.GoogleCredential;
import com.google.api.client.http.HttpTransport;
import com.google.api.client.http.javanet.NetHttpTransport;
import com.google.api.client.json.JsonFactory;
import com.google.api.client.json.jackson.JacksonFactory;
public class AppMain {
public static void main(String[] args) {
// TODO Auto-generated method stub
final HttpTransport TRANSPORT = new NetHttpTransport();
final JsonFactory JSON_FACTORY = new JacksonFactory();
GoogleCredential credential = new GoogleCredential.Builder()
.setTransport(TRANSPORT)
.setJsonFactory(JSON_FACTORY)
.setServiceAccountId("SOMETHING#developer.gserviceaccount.com")
.setServiceAccountScopes(BigqueryScopes.BIGQUERY)
.setServiceAccountPrivateKeyFromP12File(new File("key.p12"))
.build();
}
}
Please help. Thanks in advance.
I resolved this error with overriding google-api-client dependency to this -
<dependency>
<groupId>com.google.api-client</groupId>
<artifactId>google-api-client</artifactId>
<version>1.22.0</version>
</dependency>
I resolved above issue and getting token by adding one more jar in library "google-oauth-client-1.22.0.jar". Below is code snippet
GoogleCredential googleCredentials = GoogleCredential.fromStream(serviceAccount);
GoogleCredential scoped = googleCredentials.createScoped(
Arrays.asList(
"https://www.googleapis.com/auth/firebase.messaging"
)
);
scoped.refreshToken();
String token = scoped.getAccessToken();
Related
I'm trying to run spock tests against a neo4j container using the TestContainers library. Previously, I used the test-harness library to run tests against an embedded neo4j database but the need to migrate to using TestContainers has come up. With test-harness I used a configuration file to include some necessary beans for callback functionality. Without these beans, some unit tests are failing.
Previous test-harness method
#ContextConfiguration(classes = [
Neo4jTestHarnessAutoConfiguration,
Neo4jDriverAutoConfiguration,
Neo4jDataAutoConfiguration,
CustomNeo4jConfiguration
])
#ImportAutoConfiguration(classes = [
Neo4jTestHarnessAutoConfiguration,
Neo4jDriverAutoConfiguration,
Neo4jDataAutoConfiguration
])
#Transactional
trait EmbeddedNeo4j {
// Neo4j embedded database is created in Neo4jTestHarnessAutoConfiguration
}
CustomNeo4jConfiguration
#Configuration
#EntityScan(value = {
"data.neo4j.nodes",
"data.neo4j.relationships",
"data.neo4j.queryresults"
})
#EnableNeo4jRepositories("data.neo4j.repository")
#EnableNeo4jAuditing(auditorAwareRef = "auditorAware")
public class CustomNeo4jConfiguration {
#Bean
public AuditorAware<String> auditorAware(){
return new CustomAuditorAwareImpl();
}
#Bean
public BeforeBindCallback neo4jCustomEntitiesCallback(AuditorAware<String> auditorAware) {
return new CustomNeo4jEntitiesCallback(auditorAware);
}
#Bean
public Neo4jConversions neo4jCustomConversions() {
Set<GenericConverter> additionalConverters = Collections.singleton(new InstantStringConverter());
return new Neo4jConversions(additionalConverters);
}
}
The above method works fine. All tests run properly and all beans are created.
TestContainers attempt
#Testcontainers
#Transactional
trait EmbeddedNeo4j {
#Shared
static final Neo4jContainer<?> neo4jContainer = new Neo4jContainer<>("neo4j:4.3.9")
.withReuse(true)
#DynamicPropertySource
static void neo4jProperties(DynamicPropertyRegistry registry) {
neo4jContainer.start()
registry.add("spring.neo4j.uri", neo4jContainer::getBoltUrl)
registry.add("spring.neo4j.authentication.username", () -> "neo4j")
registry.add("spring.neo4j.authentication.password", neo4jContainer::getAdminPassword)
}
}
With the above, all tests successfully run against the test container. However, any test that needs the functionality of the beans in the CustomNeo4jConfiguration fails.
What I've tried
I've attempted to use the same #ContextConfiguration annotations in various configurations, such as...
#ContextConfiguration(classes = [
Neo4jDriverAutoConfiguration,
Neo4jDataAutoConfiguration,
CustomNeo4jConfiguration
])
#ImportAutoConfiguration(classes = [
Neo4jDriverAutoConfiguration,
Neo4jDataAutoConfiguration
])
#Transactional
trait EmbeddedNeo4j {
TestContainer code here...
However, this fails with the following stack trace:
Failed to load ApplicationContext
java.lang.IllegalStateException: Failed to load ApplicationContext
at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContext(DefaultCacheAwareContextLoaderDelegate.java:132)
at org.springframework.test.context.support.DefaultTestContext.getApplicationContext(DefaultTestContext.java:124)
at org.springframework.test.context.web.ServletTestExecutionListener.setUpRequestContextIfNecessary(ServletTestExecutionListener.java:190)
at org.springframework.test.context.web.ServletTestExecutionListener.prepareTestInstance(ServletTestExecutionListener.java:132)
at org.springframework.test.context.TestContextManager.prepareTestInstance(TestContextManager.java:248)
at org.spockframework.spring.SpringTestContextManager.prepareTestInstance(SpringTestContextManager.java:56)
at org.spockframework.spring.SpringInterceptor.interceptInitializerMethod(SpringInterceptor.java:43)
at org.spockframework.runtime.extension.AbstractMethodInterceptor.intercept(AbstractMethodInterceptor.java:24)
at org.spockframework.runtime.extension.MethodInvocation.proceed(MethodInvocation.java:101)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$prepare$2(NodeTestTask.java:123)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.prepare(NodeTestTask.java:123)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:90)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
at org.spockframework.runtime.model.MethodInfo.invoke(MethodInfo.java:148)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:35)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:54)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:107)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:88)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:54)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:67)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:52)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:114)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:86)
at org.junit.platform.launcher.core.DefaultLauncherSession$DelegatingLauncher.execute(DefaultLauncherSession.java:86)
at org.junit.platform.launcher.core.SessionPerRequestLauncher.execute(SessionPerRequestLauncher.java:53)
at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.processAllTestClasses(JUnitPlatformTestClassProcessor.java:99)
at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.access$000(JUnitPlatformTestClassProcessor.java:79)
at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor.stop(JUnitPlatformTestClassProcessor.java:75)
at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.stop(SuiteTestClassProcessor.java:61)
at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
at org.gradle.api.internal.tasks.testing.worker.TestWorker.stop(TestWorker.java:133)
at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:182)
at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:164)
at org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:414)
at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:56)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'auditFieldRepository' defined in data.neo4j.repository.AuditFieldRepository defined in #EnableNeo4jRepositories declared on CustomNeo4jConfiguration: Cannot resolve reference to bean 'neo4jTemplate' while setting bean property 'neo4jOperations'; nested exception is org.springframework.beans.factory.NoSuchBeanDefinitionException: No bean named 'neo4jTemplate' available
at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:342)
at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:113)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyPropertyValues(AbstractAutowireCapableBeanFactory.java:1707)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1452)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:619)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542)
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:936)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:918)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:583)
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:734)
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:408)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:308)
at org.springframework.boot.test.context.SpringBootContextLoader.loadContext(SpringBootContextLoader.java:132)
at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContextInternal(DefaultCacheAwareContextLoaderDelegate.java:99)
at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContext(DefaultCacheAwareContextLoaderDelegate.java:124)
... 64 more
Caused by: org.springframework.beans.factory.NoSuchBeanDefinitionException: No bean named 'neo4jTemplate' available
at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBeanDefinition(DefaultListableBeanFactory.java:874)
at org.springframework.beans.factory.support.AbstractBeanFactory.getMergedLocalBeanDefinition(AbstractBeanFactory.java:1344)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:309)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)
at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:330)
... 82 more
I've also attempted in various ways to create the missing beans, but that doesn't feel like the right path to take and that didn't produce any results.
What's the right way to use a #Configuration file with TestContainers?
Appreciate any help!
If I interpret your mixture of Java and Scala code correctly, I would recommend the following approach, but only if you insist on not using #SpringBootTest.
Also: You are using the outdated neo4j-java-driver-spring-boot-starter classes, they aren't needed for a while now in recent Spring Boot and also will potentially break SDN6+.
Please note: You should never import auto configuration classes and use them as ContextConfiguration at the same time. They are only meant to be imported via #ImportAutoConfiguration, but even that is redundant or overly complicated most of the time.
First of all, here's the pom so you get the proper dependencies:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.7.2</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<groupId>com.example</groupId>
<artifactId>neowithapoc</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>neowithapoc</name>
<description>neowithapoc</description>
<properties>
<java.version>17</java.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-neo4j</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>junit-jupiter</artifactId>
<version>1.17.3</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>neo4j</artifactId>
<version>1.17.3</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
Solution 1
Here's a solution that goes into what you have there, with a trait - or in case of pure java - abstract test class:
package com.example.neowithapoc;
import org.springframework.boot.autoconfigure.ImportAutoConfiguration;
import org.springframework.boot.autoconfigure.data.neo4j.Neo4jDataAutoConfiguration;
import org.springframework.boot.autoconfigure.neo4j.Neo4jAutoConfiguration;
import org.springframework.boot.test.context.TestConfiguration;
import org.springframework.data.neo4j.repository.config.EnableNeo4jRepositories;
import org.springframework.test.context.DynamicPropertyRegistry;
import org.springframework.test.context.DynamicPropertySource;
import org.springframework.transaction.annotation.EnableTransactionManagement;
import org.testcontainers.containers.Neo4jContainer;
import org.testcontainers.containers.Neo4jLabsPlugin;
#TestConfiguration
#ImportAutoConfiguration({
Neo4jAutoConfiguration.class,
Neo4jDataAutoConfiguration.class
})
#EnableTransactionManagement
#EnableNeo4jRepositories(considerNestedRepositories = true)
public abstract class EmbeddedNeo4jConfig {
static final Neo4jContainer<?> neo4j = new Neo4jContainer<>("neo4j:4.4")
.withLabsPlugins(Neo4jLabsPlugin.APOC)
.withReuse(true);
#DynamicPropertySource
static void neo4jProperties(DynamicPropertyRegistry registry) {
neo4j.start();
registry.add("spring.neo4j.uri", neo4j::getBoltUrl);
registry.add("spring.neo4j.authentication.username", () -> "neo4j");
registry.add("spring.neo4j.authentication.password", neo4j::getAdminPassword);
}
}
Use like this:
package com.example.neowithapoc;
import static org.assertj.core.api.Assertions.assertThat;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.data.neo4j.core.Neo4jClient;
import org.springframework.data.neo4j.core.Neo4jTemplate;
import org.springframework.test.context.ContextConfiguration;
import org.springframework.test.context.junit.jupiter.SpringExtension;
#ExtendWith(SpringExtension.class)
#ContextConfiguration(classes = EmbeddedNeo4jConfig.class)
public class SomeTest extends EmbeddedNeo4jConfig {
#Test
void whatever(#Autowired Neo4jTemplate template, #Autowired Neo4jClient client) {
assertThat(template).isNotNull();
assertThat(client).isNotNull();
String apoc = client
.query("RETURN apoc.version() AS output")
.fetchAs(String.class)
.first().get();
assertThat(apoc).startsWith("4.4");
}
}
Solution 2
I do prefer this way simpler and less complicated approach:
package com.example.neowithapoc;
import static org.assertj.core.api.Assertions.assertThat;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.data.neo4j.core.Neo4jClient;
import org.springframework.data.neo4j.core.Neo4jTemplate;
import org.springframework.test.context.ContextConfiguration;
import org.springframework.test.context.DynamicPropertyRegistry;
import org.springframework.test.context.DynamicPropertySource;
import org.springframework.test.context.junit.jupiter.SpringExtension;
import org.springframework.transaction.annotation.Transactional;
import org.testcontainers.containers.Neo4jContainer;
import org.testcontainers.containers.Neo4jLabsPlugin;
import org.testcontainers.junit.jupiter.Testcontainers;
#SpringBootTest
#Transactional
#Testcontainers(disabledWithoutDocker = true)
public class SomeSpringBootTest {
static final Neo4jContainer<?> neo4j = new Neo4jContainer<>("neo4j:4.4")
.withLabsPlugins(Neo4jLabsPlugin.APOC)
.withReuse(true);
#DynamicPropertySource
static void neo4jProperties(DynamicPropertyRegistry registry) {
neo4j.start();
registry.add("spring.neo4j.uri", neo4j::getBoltUrl);
registry.add("spring.neo4j.authentication.username", () -> "neo4j");
registry.add("spring.neo4j.authentication.password", neo4j::getAdminPassword);
}
#Test
void whatever(#Autowired Neo4jTemplate template, #Autowired Neo4jClient client) {
assertThat(template).isNotNull();
assertThat(client).isNotNull();
String apoc = client
.query("RETURN apoc.version() AS output")
.fetchAs(String.class)
.first().get();
assertThat(apoc).startsWith("4.4");
}
}
Bonus questions custom conversions
Given the following class:
package com.example.neowithapoc;
public class SomeObject {
private final String value;
public SomeObject(String value) {
this.value = value;
}
public String getValue() {
return value;
}
}
It is not an entity and it needs some special treatment during conversions.
It can be converted back and forth like this:
import java.util.Set;
import org.neo4j.driver.Value;
import org.neo4j.driver.Values;
import org.springframework.core.convert.TypeDescriptor;
import org.springframework.core.convert.converter.GenericConverter;
public class SomeObjectConverter implements GenericConverter {
#Override public Set<ConvertiblePair> getConvertibleTypes() {
return Set.of(
new ConvertiblePair(Value.class, SomeObject.class),
new ConvertiblePair(SomeObject.class, Value.class)
);
}
#Override
public Object convert(Object source, TypeDescriptor sourceType, TypeDescriptor targetType) {
if (source == null) {
return Value.class.isAssignableFrom(targetType.getType()) ? Values.NULL : null;
} else if (source instanceof SomeObject someObject) {
return Values.value(someObject.getValue());
} else if (source instanceof Value value) {
return new SomeObject(value.asString());
} else
throw new IllegalArgumentException();
}
}
Configure it with a #TestConfiguration (or on your main config) like this
#TestConfiguration
static class AdditionalBeans {
#Bean
SomeObjectConverter someObjectConverter() {
return new SomeObjectConverter();
}
#Bean
Neo4jConversions neo4jConversions(SomeObjectConverter someObjectConverter) {
return new Neo4jConversions(List.of(someObjectConverter));
}
}
In both cases this class is a static nested class of either EmbeddedNeo4jConfig and SomeSpringBootTest.
In the former you normally would just declare the bean method directly, but as I tried to recreate your "trait" as much as possible in Java and I do extend that class, this is not possible. Tests must not have their own #Bean methods.
After that, test like this
#Node
static class ContainerNode {
#Id #GeneratedValue
Long id;
SomeObject someObject;
public Long getId() {
return id;
}
public SomeObject getSomeObject() {
return someObject;
}
}
#Test
void conversionsShouldWork(#Autowired Neo4jTemplate template) {
var container = template.find(ContainerNode.class)
.matching("CREATE (n:ContainerNode {someObject: 'Hello'}) RETURN n")
.one();
assertThat(container).map(ContainerNode::getSomeObject).map(SomeObject::getValue).hasValue("Hello");
}
will work just fine, also with the repository.
This test here will fail
#Test
void conversionsShouldWork(#Autowired Neo4jClient client) {
var anObject = client.query("RETURN 'whatever'").fetchAs(SomeObject.class).first();
assertThat(anObject).map(SomeObject::getValue).hasValue("whatever");
}
but the Spring Data Neo4j fixed this in 6.3.3 https://github.com/spring-projects/spring-data-neo4j/issues/2594 and you would be able to manually add it to the client like this:
#TestConfiguration
static class AdditionalBeans {
#Bean
SomeObjectConverter someObjectConverter() {
return new SomeObjectConverter();
}
#Bean
Neo4jConversions neo4jConversions(SomeObjectConverter someObjectConverter) {
return new Neo4jConversions(List.of(someObjectConverter));
}
#Bean
public Neo4jClient neo4jClient(
Driver driver,
DatabaseSelectionProvider databaseSelectionProvider,
Neo4jConversions neo4jConversions) {
return Neo4jClient.with(driver)
.withDatabaseSelectionProvider(databaseSelectionProvider)
.withNeo4jConversions(neo4jConversions)
.build();
}
}
Than this test will work just fine:
#Test
void clientConversionsShouldWork(#Autowired Neo4jClient client) {
var anObject = client.query("RETURN 'whatever'").fetchAs(SomeObject.class).first();
assertThat(anObject).map(SomeObject::getValue).hasValue("whatever");
}
We are using spring security oauth2 to obtain token using client credentials grant type. We are not using the application.properties file for specifying the client credentials, instead we are supplying them programmatically.
ClientRegistration clientRegistration = ClientRegistration
.withRegistrationId("test")
.clientId("testclientid")
.clientSecret("testclientsecret")
.authorizationGrantType(AuthorizationGrantType.CLIENT_CREDENTIALS)
.tokenUri("http://test.tokenuri.com")
.build();
ReactiveClientRegistrationRepository reactiveClientRegistrationRepository = new InMemoryReactiveClientRegistrationRepository(clientRegistration);
ServerOAuth2AuthorizedClientExchangeFilterFunction oauth =
new ServerOAuth2AuthorizedClientExchangeFilterFunction(
reactiveClientRegistrationRepository,
new UnAuthenticatedServerOAuth2AuthorizedClientRepository());
oauth.setDefaultClientRegistrationId("test");
this.webClient = webClientFactory.getBuilder()
.filter(oauth)
.build();
The code is working fine, but we see a warning that UnAuthenticatedServerOAuth2AuthorizedClientRepository is deprecated.
The api docs for UnAuthenticatedServerOAuth2AuthorizedClientRepository recommend to use AuthorizedClientServiceReactiveOAuth2AuthorizedClientManager instead, but AuthorizedClientServiceReactiveOAuth2AuthorizedClientManager does not implement the same interface as UnAuthenticatedServerOAuth2AuthorizedClientRepository. What is the recommendation on replacing the deprecated UnAuthenticatedServerOAuth2AuthorizedClientRepository in this case?
I found https://github.com/spring-projects/spring-security/issues/8016 but the issue does not give much detail.
With the help of #Jokers answer, I managed to solve this problem in the following way. I put the credentials in appliction.properties and seperated the RegistrationRepository for that.
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.security.oauth2.client.AuthorizedClientServiceReactiveOAuth2AuthorizedClientManager;
import org.springframework.security.oauth2.client.InMemoryReactiveOAuth2AuthorizedClientService;
import org.springframework.security.oauth2.client.registration.ClientRegistration;
import org.springframework.security.oauth2.client.registration.InMemoryReactiveClientRegistrationRepository;
import org.springframework.security.oauth2.client.registration.ReactiveClientRegistrationRepository;
import org.springframework.security.oauth2.client.web.reactive.function.client.ServerOAuth2AuthorizedClientExchangeFilterFunction;
import org.springframework.security.oauth2.core.AuthorizationGrantType;
import org.springframework.web.reactive.function.client.WebClient;
#Configuration
public class MyClientRequestConfig {
#Bean
ReactiveClientRegistrationRepository getRegistration(
#Value("${spring.security.oauth2.client.provider.myprovider.token-uri}") String token_uri,
#Value("${spring.security.oauth2.client.registration.myprovider.client-id}") String client_id,
#Value("${spring.security.oauth2.client.registration.myprovider.client-secret}") String client_secret
) {
ClientRegistration registration = ClientRegistration
.withRegistrationId("myprovider")
.tokenUri(token_uri)
.clientId(client_id)
.clientSecret(client_secret)
.authorizationGrantType(AuthorizationGrantType.CLIENT_CREDENTIALS)
.build();
return new InMemoryReactiveClientRegistrationRepository(registration);
}
#Bean(name = "myprovider")
WebClient webClient(ReactiveClientRegistrationRepository clientRegistrations) {
InMemoryReactiveOAuth2AuthorizedClientService clientService = new InMemoryReactiveOAuth2AuthorizedClientService(clientRegistrations);
AuthorizedClientServiceReactiveOAuth2AuthorizedClientManager authorizedClientManager = new AuthorizedClientServiceReactiveOAuth2AuthorizedClientManager(clientRegistrations, clientService);
ServerOAuth2AuthorizedClientExchangeFilterFunction oauth = new ServerOAuth2AuthorizedClientExchangeFilterFunction(authorizedClientManager);
oauth.setDefaultClientRegistrationId("myprovider");
return WebClient.builder()
.filter(oauth)
.build();
}
}
Allthough kotlin code (conversion should be easy to java), we ended up with something like that:
val clientRegistryRepo = InMemoryReactiveClientRegistrationRepository(ClientRegistration
.withRegistrationId("test")
.tokenUri("http://test.tokenuri.com")
.clientId("testClientId")
.clientSecret("testclientsecret")
.authorizationGrantType(AuthorizationGrantType.CLIENT_CREDENTIALS)
.build())
val clientService = InMemoryReactiveOAuth2AuthorizedClientService(clientRegistryRepo)
val authorizedClientManager =
AuthorizedClientServiceReactiveOAuth2AuthorizedClientManager(clientRegistryRepo, clientService)
val oauthFilter = ServerOAuth2AuthorizedClientExchangeFilterFunction(authorizedClientManager)
oauthFilter.setDefaultClientRegistrationId("test")
WebClient.builder()
.filter(oauthFilter)
.build()
We're working on a Spring Boot application that is an OIDC client. The identity provider (IdP) is a third-party service and fully OpenID Connect and OAuth 2.0 compliant (as far as we can tell). As it's built with high security in mind, its UserInfo endpoint returns a signed JWT (instead of a regular one).
It seems that Spring Security does not support it. The authentication flow ends with an error message (displayed in an HTML page generated by our Spring application):
[invalid_user_info_response] An error occurred while attempting to
retrieve the UserInfo Resource: Could not extract response: no
suitable HttpMessageConverter found for response type
[java.util.Map] and content type
[application/jwt;charset=UTF-8]
My questions:
Is it correct that Spring does not currently support UserInfo endpoints returning signed JWTs?
If so, how can we add support for signed JWTs (including verfication of the signature)?
Our analysis has shown that the DefaultOAuth2UserService requests (Accept: application/json) and expects a JSON response from the IdP. However, being configured for high security, the IdP returns a signed JWT with the content type application/jwt. The response looks like the example on jwt.io. As the RestTemplate has no message converter capable of handling the content type application/jwt, the authenticaton fails.
Our sample app is as simple as it gets:
build.gradle
plugins {
id 'java'
id 'org.springframework.boot' version '2.2.4.RELEASE'
id 'io.spring.dependency-management' version '1.0.9.RELEASE'
}
group = 'com.example'
version = '0.0.1-SNAPSHOT'
sourceCompatibility = '11'
repositories {
mavenCentral()
}
dependencies {
implementation 'org.springframework.boot:spring-boot-starter-web'
implementation 'org.springframework.boot:spring-boot-starter-oauth2-client'
}
DemoApplication.java
package demo;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
#SpringBootApplication
public class DemoApplication {
public static void main(String[] args) {
SpringApplication.run(DemoApplication.class, args);
}
}
application.yml
server:
port: 8081
spring:
security:
oauth2:
client:
registration:
demo:
client-id: our-client-id
client-secret: our-client-secret
clientAuthenticationMethod: post
provider: our-idp
scope:
- profile
- email
provider:
our-idp:
issuer-uri: https://login.idp.com:443/idp/oauth2
HomeController.java
package demo;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;
#RestController
public class HomeController {
#GetMapping("/")
String hello() { return "hello"; }
}
After more analysis, it seems Spring Boot does not support UserInfo endpoints returning signed JWTs. It's obviously an unusual setup (but still within the OAuth 2.0 / OIDC specification). What I haven't mentioned so far is that the JWT is signed with the client secret.
While Spring Boot doesn't support it, it can be added. The solution consists of:
A user service supporting signed JWTs (as a replacment for DefaultOAuth2UserService)
A HttpMessageConverter supporting JWTs (used in the user service's RestTemplate)
A JwtDecoder using the client secret
A security configuration that puts the pieces together
Note that we have changed from OAuth 2.0 to OIDC in the mean-time, thus our application.yml now includes the openid scope.
spring:
security:
oauth2:
client:
registration:
demo:
client-id: our-client-id
client-secret: our-client-secret
clientAuthenticationMethod: post
provider: our-idp
scope:
- profile
- email
provider:
our-idp:
issuer-uri: https://login.idp.com:443/idp/oauth2
The security configuration is:
package demoapp;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.security.config.annotation.web.builders.HttpSecurity;
import org.springframework.security.config.annotation.web.configuration.WebSecurityConfigurerAdapter;
import org.springframework.security.oauth2.client.oidc.userinfo.OidcUserService;
import org.springframework.security.oauth2.client.registration.ClientRegistration;
import org.springframework.security.oauth2.client.registration.ClientRegistrationRepository;
import org.springframework.security.oauth2.jwt.JwtDecoder;
import org.springframework.security.oauth2.jwt.NimbusJwtDecoder;
import javax.crypto.spec.SecretKeySpec;
import java.nio.charset.StandardCharsets;
#Configuration
public class SecurityConfig extends WebSecurityConfigurerAdapter {
private final ClientRegistrationRepository clientRegistrationRepository;
public SecurityConfig(ClientRegistrationRepository clientRegistrationRepository) {
this.clientRegistrationRepository = clientRegistrationRepository;
}
#Override
protected void configure(HttpSecurity httpSecurity) throws Exception {
httpSecurity
.authorizeRequests()
.anyRequest().authenticated()
.and()
.oauth2Login()
.userInfoEndpoint()
.oidcUserService(oidcUserService());
}
#Bean
OidcUserService oidcUserService() {
OidcUserService userService = new OidcUserService();
userService.setOauth2UserService(new ValidatingOAuth2UserService(jwtDecoderUsingClientSecret("demo")));
return userService;
}
JwtDecoder jwtDecoderUsingClientSecret(String registrationId) {
ClientRegistration registration = clientRegistrationRepository.findByRegistrationId(registrationId);
SecretKeySpec key = new SecretKeySpec(registration.getClientSecret().getBytes(StandardCharsets.UTF_8), "HS256");
return NimbusJwtDecoder.withSecretKey(key).build();
}
}
If you are using OAuth 2.0 instead of OIDC (i.e. you don't use the scope 'openid'), the configuration is simpler:
package demo;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.security.config.annotation.web.builders.HttpSecurity;
import org.springframework.security.config.annotation.web.configuration.WebSecurityConfigurerAdapter;
import org.springframework.security.oauth2.client.oidc.userinfo.OidcUserService;
import org.springframework.security.oauth2.client.registration.ClientRegistration;
import org.springframework.security.oauth2.client.registration.ClientRegistrationRepository;
import org.springframework.security.oauth2.jwt.JwtDecoder;
import org.springframework.security.oauth2.jwt.NimbusJwtDecoder;
import javax.crypto.spec.SecretKeySpec;
import java.nio.charset.StandardCharsets;
#Configuration
public class SecurityConfig extends WebSecurityConfigurerAdapter {
private final ClientRegistrationRepository clientRegistrationRepository;
public SecurityConfig(ClientRegistrationRepository clientRegistrationRepository) {
this.clientRegistrationRepository = clientRegistrationRepository;
}
#Override
protected void configure(HttpSecurity httpSecurity) throws Exception {
httpSecurity
.authorizeRequests()
.anyRequest().authenticated()
.and()
.oauth2Login()
.userInfoEndpoint()
.userService(new ValidatingOAuth2UserService(jwtDecoderUsingClientSecret("demo")));
}
JwtDecoder jwtDecoderUsingClientSecret(String registrationId) {
ClientRegistration registration = clientRegistrationRepository.findByRegistrationId(registrationId);
SecretKeySpec key = new SecretKeySpec(registration.getClientSecret().getBytes(StandardCharsets.UTF_8), "HS256");
return NimbusJwtDecoder.withSecretKey(key).build();
}
}
The ValidatingOAuth2UserService class is - for the most part - a copy of DefaultOAuth2UserService:
/*
* Copyright 2002-2018 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package demo;
import java.util.LinkedHashSet;
import java.util.Map;
import java.util.Set;
import org.springframework.core.convert.converter.Converter;
import org.springframework.http.RequestEntity;
import org.springframework.http.ResponseEntity;
import org.springframework.security.core.GrantedAuthority;
import org.springframework.security.core.authority.SimpleGrantedAuthority;
import org.springframework.security.oauth2.client.http.OAuth2ErrorResponseErrorHandler;
import org.springframework.security.oauth2.client.registration.ClientRegistration;
import org.springframework.security.oauth2.client.userinfo.OAuth2UserRequest;
import org.springframework.security.oauth2.client.userinfo.OAuth2UserRequestEntityConverter;
import org.springframework.security.oauth2.client.userinfo.OAuth2UserService;
import org.springframework.security.oauth2.core.OAuth2AccessToken;
import org.springframework.security.oauth2.core.OAuth2AuthenticationException;
import org.springframework.security.oauth2.core.OAuth2AuthorizationException;
import org.springframework.security.oauth2.core.OAuth2Error;
import org.springframework.security.oauth2.core.user.DefaultOAuth2User;
import org.springframework.security.oauth2.core.user.OAuth2User;
import org.springframework.security.oauth2.core.user.OAuth2UserAuthority;
import org.springframework.security.oauth2.jwt.Jwt;
import org.springframework.security.oauth2.jwt.JwtDecoder;
import org.springframework.util.Assert;
import org.springframework.util.StringUtils;
import org.springframework.web.client.ResponseErrorHandler;
import org.springframework.web.client.RestClientException;
import org.springframework.web.client.RestOperations;
import org.springframework.web.client.RestTemplate;
/**
* An implementation of an {#link OAuth2UserService} that supports standard OAuth 2.0 Provider's.
* <p>
* This provider supports <i>UserInfo</i> endpoints returning user details
* in signed JWTs (content-type {#code application/jwt}).
* </p>
* <p>
* For standard OAuth 2.0 Provider's, the attribute name used to access the user's name
* from the UserInfo response is required and therefore must be available via
* {#link ClientRegistration.ProviderDetails.UserInfoEndpoint#getUserNameAttributeName() UserInfoEndpoint.getUserNameAttributeName()}.
* <p>
* <b>NOTE:</b> Attribute names are <b>not</b> standardized between providers and therefore will vary.
* Please consult the provider's API documentation for the set of supported user attribute names.
*
* #see org.springframework.security.oauth2.client.userinfo.OAuth2UserService
* #see OAuth2UserRequest
* #see OAuth2User
* #see DefaultOAuth2User
*/
public class ValidatingOAuth2UserService implements OAuth2UserService<OAuth2UserRequest, OAuth2User> {
private static final String MISSING_USER_INFO_URI_ERROR_CODE = "missing_user_info_uri";
private static final String MISSING_USER_NAME_ATTRIBUTE_ERROR_CODE = "missing_user_name_attribute";
private static final String INVALID_USER_INFO_RESPONSE_ERROR_CODE = "invalid_user_info_response";
private Converter<OAuth2UserRequest, RequestEntity<?>> requestEntityConverter = new OAuth2UserRequestEntityConverter();
private RestOperations restOperations;
private JwtDecoder jwtDecoder;
public ValidatingOAuth2UserService(JwtDecoder jwtDecoder) {
this.jwtDecoder = jwtDecoder;
RestTemplate restTemplate = new RestTemplate();
restTemplate.setErrorHandler(new OAuth2ErrorResponseErrorHandler());
restTemplate.getMessageConverters().add(new JwtHttpMessageConverter());
this.restOperations = restTemplate;
}
#Override
public OAuth2User loadUser(OAuth2UserRequest userRequest) throws OAuth2AuthenticationException {
Assert.notNull(userRequest, "userRequest cannot be null");
if (!StringUtils.hasText(userRequest.getClientRegistration().getProviderDetails().getUserInfoEndpoint().getUri())) {
OAuth2Error oauth2Error = new OAuth2Error(
MISSING_USER_INFO_URI_ERROR_CODE,
"Missing required UserInfo Uri in UserInfoEndpoint for Client Registration: " +
userRequest.getClientRegistration().getRegistrationId(),
null
);
throw new OAuth2AuthenticationException(oauth2Error, oauth2Error.toString());
}
String userNameAttributeName = userRequest.getClientRegistration().getProviderDetails()
.getUserInfoEndpoint().getUserNameAttributeName();
if (!StringUtils.hasText(userNameAttributeName)) {
OAuth2Error oauth2Error = new OAuth2Error(
MISSING_USER_NAME_ATTRIBUTE_ERROR_CODE,
"Missing required \"user name\" attribute name in UserInfoEndpoint for Client Registration: " +
userRequest.getClientRegistration().getRegistrationId(),
null
);
throw new OAuth2AuthenticationException(oauth2Error, oauth2Error.toString());
}
RequestEntity<?> request = this.requestEntityConverter.convert(userRequest);
ResponseEntity<String> response;
try {
response = this.restOperations.exchange(request, String.class);
} catch (OAuth2AuthorizationException ex) {
OAuth2Error oauth2Error = ex.getError();
StringBuilder errorDetails = new StringBuilder();
errorDetails.append("Error details: [");
errorDetails.append("UserInfo Uri: ").append(
userRequest.getClientRegistration().getProviderDetails().getUserInfoEndpoint().getUri());
errorDetails.append(", Error Code: ").append(oauth2Error.getErrorCode());
if (oauth2Error.getDescription() != null) {
errorDetails.append(", Error Description: ").append(oauth2Error.getDescription());
}
errorDetails.append("]");
oauth2Error = new OAuth2Error(INVALID_USER_INFO_RESPONSE_ERROR_CODE,
"An error occurred while attempting to retrieve the UserInfo Resource: " + errorDetails.toString(), null);
throw new OAuth2AuthenticationException(oauth2Error, oauth2Error.toString(), ex);
} catch (RestClientException ex) {
OAuth2Error oauth2Error = new OAuth2Error(INVALID_USER_INFO_RESPONSE_ERROR_CODE,
"An error occurred while attempting to retrieve the UserInfo Resource: " + ex.getMessage(), null);
throw new OAuth2AuthenticationException(oauth2Error, oauth2Error.toString(), ex);
}
Jwt jwt = decodeAndValidateJwt(response.getBody());
Map<String, Object> userAttributes = jwt.getClaims();
Set<GrantedAuthority> authorities = new LinkedHashSet<>();
authorities.add(new OAuth2UserAuthority(userAttributes));
OAuth2AccessToken token = userRequest.getAccessToken();
for (String authority : token.getScopes()) {
authorities.add(new SimpleGrantedAuthority("SCOPE_" + authority));
}
return new DefaultOAuth2User(authorities, userAttributes, userNameAttributeName);
}
private Jwt decodeAndValidateJwt(String token) {
return jwtDecoder.decode(token);
}
/**
* Sets the {#link Converter} used for converting the {#link OAuth2UserRequest}
* to a {#link RequestEntity} representation of the UserInfo Request.
*
* #since 5.1
* #param requestEntityConverter the {#link Converter} used for converting to a {#link RequestEntity} representation of the UserInfo Request
*/
public final void setRequestEntityConverter(Converter<OAuth2UserRequest, RequestEntity<?>> requestEntityConverter) {
Assert.notNull(requestEntityConverter, "requestEntityConverter cannot be null");
this.requestEntityConverter = requestEntityConverter;
}
/**
* Sets the {#link RestOperations} used when requesting the UserInfo resource.
*
* <p>
* <b>NOTE:</b> At a minimum, the supplied {#code restOperations} must be configured with the following:
* <ol>
* <li>{#link ResponseErrorHandler} - {#link OAuth2ErrorResponseErrorHandler}</li>
* </ol>
*
* #since 5.1
* #param restOperations the {#link RestOperations} used when requesting the UserInfo resource
*/
public final void setRestOperations(RestOperations restOperations) {
Assert.notNull(restOperations, "restOperations cannot be null");
this.restOperations = restOperations;
}
}
And finally the JwtHttpMessageConverter class:
package demo;
import org.springframework.http.HttpInputMessage;
import org.springframework.http.HttpOutputMessage;
import org.springframework.http.MediaType;
import org.springframework.http.converter.AbstractGenericHttpMessageConverter;
import org.springframework.http.converter.HttpMessageNotReadableException;
import org.springframework.http.converter.HttpMessageNotWritableException;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.lang.reflect.Type;
import java.nio.charset.StandardCharsets;
/**
* Message converter for reading JWTs transmitted with content type {#code application/jwt}.
* <p>
* The JWT is returned as a string and not validated.
* </p>
*/
public class JwtHttpMessageConverter extends AbstractGenericHttpMessageConverter<String> {
public JwtHttpMessageConverter() {
super(MediaType.valueOf("application/jwt"));
}
#Override
protected String readInternal(Class<? extends String> clazz, HttpInputMessage inputMessage) throws IOException, HttpMessageNotReadableException {
return getBodyAsString(inputMessage.getBody());
}
#Override
public String read(Type type, Class<?> contextClass, HttpInputMessage inputMessage) throws IOException, HttpMessageNotReadableException {
return readInternal(null, inputMessage);
}
private String getBodyAsString(InputStream bodyStream) throws IOException {
ByteArrayOutputStream buffer = new ByteArrayOutputStream();
byte[] chunk = new byte[64];
int len;
while ((len = bodyStream.read(chunk)) != -1) {
buffer.write(chunk, 0, len);
}
return buffer.toString(StandardCharsets.US_ASCII);
}
#Override
protected void writeInternal(String stringObjectMap, Type type, HttpOutputMessage outputMessage) throws IOException, HttpMessageNotWritableException {
throw new UnsupportedOperationException();
}
}
Thanks #Codo. You saved my day. I did not have to use JwtHttpMessageConverter class though. I added the below in SecurityConfig class. I had to use JwksUri and SignatureAlgorithm.RS512 in my situation.
#Bean
public SecurityFilterChain filterChain(HttpSecurity http) throws Exception {
http.
authorizeRequests().
anyRequest().authenticated().
and().
oauth2Login().userInfoEndpoint().oidcUserService(oidcUserService());
return http.build();
}
OidcUserService oidcUserService() {
logger.info("OidcUserService bean");
OidcUserService userService = new OidcUserService();
//userService.setOauth2UserService(new ExampleOAuth2UserService(jwtDecoderUsingClientSecret("web-client")));
userService.setOauth2UserService(new ExampleOAuth2UserService(jwtDecoder()));
return userService;
}
public JwtDecoder jwtDecoder() {
String jwksUri = Config.getJwksUri();
System.out.println("jwtDecoder jwksUri="+jwksUri);
return NimbusJwtDecoder.withJwkSetUri(jwksUri).jwsAlgorithm(SignatureAlgorithm.RS512).build();
}
public class ExampleOAuth2UserService implements OAuth2UserService<OAuth2UserRequest, OAuth2User> {
...............
...............
private JwtDecoder jwtDecoder;
public ExampleAuth2UserService(JwtDecoder jwtDecoder) {
this.jwtDecoder = jwtDecoder;
RestTemplate restTemplate = new RestTemplate();
restTemplate.setErrorHandler(new OAuth2ErrorResponseErrorHandler());
this.restOperations = restTemplate;
}
I've been working on Kafka twitter streaming feed data.
I'm following the sample from below link:
http://www.hahaskills.com/tutorials/kafka/Twitter_doc.html
I'm able to use Producer code and it is working fine. Able to get twitter feed and send to Kafka Producer.
I'm not able to use Consumer code, since it has been throwing as deprecated error for many APIs.
Here is the Consumer code:
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Properties;
import kafka.consumer.Consumer;
//import kafka.consumer.ConsumerConfig;
import org.apache.kafka.clients.consumer.ConsumerConfig;
import kafka.consumer.ConsumerIterator;
//import kafka.consumer.KafkaStream;
//import kafka.consumer.KafkaStream;
import kafka.javaapi.consumer.ConsumerConnector;
//import org.apache.kafka.clients.producer.KafkaProducer;
public class KafkaConsumer {
private final ConsumerConnector consumer;
private final String topic;
public KafkaConsumer(String zookeeper, String groupId, String topic) {
Properties props = new Properties();
props.put("zookeeper.connect", zookeeper);
props.put("group.id", groupId);
props.put("zookeeper.session.timeout.ms", "500");
props.put("zookeeper.sync.time.ms", "250");
props.put("auto.commit.interval.ms", "1000");
consumer = Consumer.createJavaConsumerConnector(new ConsumerConfig(props));
this.topic = topic;
}
public void testConsumer() {
System.out.println("Test Con called");
Map<String, Integer> topicCount = new HashMap<>();
topicCount.put(topic, 1);
Map<String, List<KafkaStream<byte[], byte[]>>> consumerStreams = consumer.createMessageStreams(topicCount);
List<KafkaStream<byte[], byte[]>> streams = consumerStreams.get(topic);
System.out.println("For");
for (final KafkaStream stream : streams) {
ConsumerIterator<byte[], byte[]> it = stream.iterator();
System.out.println("Size"+it.length());
while (it.hasNext()) {
System.out.println("Stream");
System.out.println("Message from Single Topic: " + new String(it.next().message()));
}
}
if (consumer != null) {
consumer.shutdown();
}
}
public static void main(String[] args) {
System.out.println("Started");
String topic="twittertopic";
KafkaConsumer simpleTWConsumer = new KafkaConsumer("localhost:XXXX", "testgroup", topic);
simpleTWConsumer.testConsumer();
System.out.println("End");
}
}
It throws error : ConsumerConnector, ConsumerIterator, KafkaStream are deprecated.
ConsumerConfig is not visible.
Is there fixed version of this sample code (Kafka consumer for twitter)?
The tutorial you are following is very old and it's using the old Scala Kafka clients that have been deprecated, see http://kafka.apache.org/documentation/#legacyapis
The classes that have been deprecated are:
kafka.consumer.* and kafka.javaapi.consumer instead use the newer Java Consumer under org.apache.kafka.clients.consumer.*
kafka.producer.* and kafka.javaapi.producer instead use the newer Java Producer under org.apache.kafka.clients.producer.*
Apart from using deprecated classes, your code was mostly correct, I only had to fix a few imports. See below a fixed version. Using it I was able to consume messages I was producing to a topic called twittertopic.
package example;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Properties;
import kafka.consumer.Consumer;
import kafka.consumer.ConsumerConfig;
import kafka.consumer.ConsumerIterator;
import kafka.consumer.KafkaStream;
import kafka.javaapi.consumer.ConsumerConnector;
public class MyConsumer {
private final ConsumerConnector consumer;
private final String topic;
public MyConsumer(String zookeeper, String groupId, String topic) {
Properties props = new Properties();
props.put("zookeeper.connect", zookeeper);
props.put("group.id", groupId);
props.put("zookeeper.session.timeout.ms", "500");
props.put("zookeeper.sync.time.ms", "250");
props.put("auto.commit.interval.ms", "1000");
consumer = Consumer.createJavaConsumerConnector(new ConsumerConfig(props));
this.topic = topic;
}
public void testConsumer() {
Map<String, Integer> topicCount = new HashMap<>();
topicCount.put(topic, 1);
Map<String, List<KafkaStream<byte[], byte[]>>> consumerStreams = consumer.createMessageStreams(topicCount);
List<KafkaStream<byte[], byte[]>> streams = consumerStreams.get(topic);
for (final KafkaStream stream : streams) {
ConsumerIterator<byte[], byte[]> it = stream.iterator();
while (it.hasNext()) {
System.out.println("Message from Single Topic: " + new String(it.next().message()));
}
}
if (consumer != null) {
consumer.shutdown();
}
}
public static void main(String[] args) {
System.out.println("Started");
String topic = "twittertopic";
MyConsumer simpleTWConsumer = new MyConsumer("localhost:2181", "testgroup", topic);
simpleTWConsumer.testConsumer();
System.out.println("End");
}
}
While the code above can be used, the next major Kafka release is likely to remove classes that are currently deprecated, so you should not write new logic using these.
Instead you should get started with the Java clients, you can use the examples provided on Github: https://github.com/apache/kafka/tree/trunk/examples/src/main/java/kafka/examples
Using the new Java Consumer, your logic would look like:
import java.util.Arrays;
import java.util.Properties;
import org.apache.kafka.clients.consumer.ConsumerRecord;
import org.apache.kafka.clients.consumer.ConsumerRecords;
import org.apache.kafka.clients.consumer.KafkaConsumer;
public class MyConsumer {
static final String TOPIC = "twittertopic";
static final String GROUP = "testgroup";
public static void main(String[] args) {
System.out.println("Started");
Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
props.put("group.id", GROUP);
props.put("auto.commit.interval.ms", "1000");
props.put("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
props.put("value.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
try (KafkaConsumer<String, String> consumer = new KafkaConsumer<>(props);) {
consumer.subscribe(Arrays.asList(TOPIC));
for (int i = 0; i < 1000; i++) {
ConsumerRecords<String, String> records = consumer.poll(Duration.ofSeconds(1L));
System.out.println("Size: " + records.count());
for (ConsumerRecord<String, String> record : records) {
System.out.println("Received a message: " + record.key() + " " + record.value());
}
}
}
System.out.println("End");
}
}
Good day,
I was recently testing Oauth2.0 Installed application authentication for bigquery through java api.
I used
this command line sample
as a source since i want to make it for the users eaiser to give access (it would be ugly to ask them to copy the code)
If i use SetAccessType(Offline) in GoogleAuthorizationCodeflow it says, that it automatically refreshes the access token for me.
In related I'd like to ask, if i authorize a com.google.api.services.bigquery.Bigquery instance with
com.google.api.services.bigquery.Bigquery.Builder.Builder(HttpTransport transport, JsonFactory jsonFactory, HttpRequestInitializer httpRequestInitializer)
using
credentials that have a refresh token, do i have to reauthorize it or it will update the credentials inside itself?
If it does not give me a valid connection to bigquery after one hour (token expiration time)
then which is the best method to reauthorize the client?
Shall i use credentials.RefreshToken() and then just Build another bigquery instance whith the new access token, or there is a better way?
(I may also want to change the credentials Refreshlistener, but the problem is that if i instantinate the credential with GoogleAuthorizationCodeflow.createAndStoreCredential(response, Clientid) after that i cannot get the Refreshlistener. How can i get it, so i may use that class to automatically reauthorize my Bigquery Client < maybe this is the better way is it?)
Where does the codeflow store the refreshed access token (if it refreshes it automatically)?
If i declared it like this:
new GoogleAuthorizationCodeFlow.Builder(
transport, jsonFactory, clientSecrets, scopes).setAccessType("offline")
.setApprovalPrompt("auto").setCredentialStore(new MemoryCredentialStore()).build();
Will it store my refreshed access token always in the MemoryCredentialStore?
So if i use .getCredentialStore().load(userId, credential) it will load the refreshed access token from the memorystore?
Is there a way to increase or decrease the time while an access token is valid? Since for testing purposes i really want to do so.
p.s: I was also looking into this source code of google-api-java-client
and google-oauth-java-client but i still couldn't find the sollution.
Most importantly I was looking into: class Credential at code:
public void intercept(HttpRequest request) throws IOException {
lock.lock();...
but I couldnt figure it out actually when this method is called and I eventually got lost in the problem.
Looking forward to your answer:
Attila
There's a lot of questions here. You will want to use the GoogleCredential class to gain a new access token from an existing refresh token, that you well need to store (using whatever method you want, such as MemoryCredentialStore).
The access token only lasts for an hour and there is not way to change this.
Here is an example using the installed flow:
import java.io.BufferedReader;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStreamReader;
import java.util.Collections;
import java.util.List;
import java.util.Properties;
import com.google.api.client.auth.oauth2.Credential;
import com.google.api.client.auth.oauth2.TokenResponse;
import com.google.api.client.googleapis.auth.oauth2.GoogleAuthorizationCodeFlow;
import com.google.api.client.googleapis.auth.oauth2.GoogleAuthorizationCodeRequestUrl;
import com.google.api.client.googleapis.auth.oauth2.GoogleClientSecrets;
import com.google.api.client.googleapis.auth.oauth2.GoogleCredential;
import com.google.api.client.googleapis.auth.oauth2.GoogleTokenResponse;
import com.google.api.client.http.HttpTransport;
import com.google.api.client.http.javanet.NetHttpTransport;
import com.google.api.client.json.JsonFactory;
import com.google.api.client.json.jackson.JacksonFactory;
import com.google.api.services.bigquery.Bigquery;
import com.google.api.services.bigquery.Bigquery.Datasets;
import com.google.api.services.bigquery.BigqueryScopes;
import com.google.api.services.bigquery.model.DatasetList;
class BigQueryInstalledAuthDemo {
// Change this to your current project ID
private static final String PROJECT_NUMBER = "XXXXXXXXXX";
// Load Client ID/secret from client_secrets.json file.
private static final String CLIENTSECRETS_LOCATION = "client_secrets.json";
static GoogleClientSecrets clientSecrets = loadClientSecrets();
private static final String REDIRECT_URI = "urn:ietf:wg:oauth:2.0:oob";
// Objects for handling HTTP transport and JSON formatting of API calls
private static final HttpTransport HTTP_TRANSPORT = new NetHttpTransport();
private static final JsonFactory JSON_FACTORY = new JacksonFactory();
private static GoogleAuthorizationCodeFlow flow = null;
// BigQuery Client
static Bigquery bigquery;
public static void main(String[] args) throws IOException {
// Attempt to Load existing Refresh Token
String storedRefreshToken = loadRefreshToken();
// Check to see if the an existing refresh token was loaded.
// If so, create a credential and call refreshToken() to get a new
// access token.
if (storedRefreshToken != null) {
// Request a new Access token using the refresh token.
GoogleCredential credential = createCredentialWithRefreshToken(
HTTP_TRANSPORT, JSON_FACTORY, new TokenResponse().setRefreshToken(storedRefreshToken));
credential.refreshToken();
bigquery = buildService(credential);
// If there is no refresh token (or token.properties file), start the OAuth
// authorization flow.
} else {
String authorizeUrl = new GoogleAuthorizationCodeRequestUrl(
clientSecrets,
REDIRECT_URI,
Collections.singleton(BigqueryScopes.BIGQUERY)).setState("").build();
System.out.println("Paste this URL into a web browser to authorize BigQuery Access:\n" + authorizeUrl);
System.out.println("... and type the code you received here: ");
BufferedReader in = new BufferedReader(new InputStreamReader(System.in));
String authorizationCode = in.readLine();
// Exchange the auth code for an access token and refesh token
Credential credential = exchangeCode(authorizationCode);
// Store the refresh token for future use.
storeRefreshToken(credential.getRefreshToken());
bigquery = buildService(credential);
}
// Make API calls using your client.
listDatasets(bigquery, PROJECT_NUMBER);
}
/**
* Builds an authorized BigQuery API client.
*/
private static Bigquery buildService(Credential credential) {
return new Bigquery.Builder(HTTP_TRANSPORT, JSON_FACTORY, credential).build();
}
/**
* Build an authorization flow and store it as a static class attribute.
*/
static GoogleAuthorizationCodeFlow getFlow() {
if (flow == null) {
flow = new GoogleAuthorizationCodeFlow.Builder(HTTP_TRANSPORT,
JSON_FACTORY,
clientSecrets,
Collections.singleton(BigqueryScopes.BIGQUERY))
.setAccessType("offline").setApprovalPrompt("force").build();
}
return flow;
}
/**
* Exchange the authorization code for OAuth 2.0 credentials.
*/
static Credential exchangeCode(String authorizationCode) throws IOException {
GoogleAuthorizationCodeFlow flow = getFlow();
GoogleTokenResponse response =
flow.newTokenRequest(authorizationCode).setRedirectUri(REDIRECT_URI).execute();
return flow.createAndStoreCredential(response, null);
}
/**
* No need to go through OAuth dance, get an access token using the
* existing refresh token.
*/
public static GoogleCredential createCredentialWithRefreshToken(HttpTransport transport,
JsonFactory jsonFactory, TokenResponse tokenResponse) {
return new GoogleCredential.Builder().setTransport(transport)
.setJsonFactory(jsonFactory)
.setClientSecrets(clientSecrets)
.build()
.setFromTokenResponse(tokenResponse);
}
/**
* Helper to load client ID/Secret from file.
*/
private static GoogleClientSecrets loadClientSecrets() {
try {
GoogleClientSecrets clientSecrets =
GoogleClientSecrets.load(new JacksonFactory(),
BigQueryInstalledAuthDemo.class.getResourceAsStream(CLIENTSECRETS_LOCATION));
return clientSecrets;
} catch (Exception e) {
System.out.println("Could not load clientsecrets.json");
e.printStackTrace();
}
return clientSecrets;
}
/**
* Helper to store a new refresh token in token.properties file.
*/
private static void storeRefreshToken(String refresh_token) {
Properties properties = new Properties();
properties.setProperty("refreshtoken", refresh_token);
System.out.println(properties.get("refreshtoken"));
try {
properties.store(new FileOutputStream("token.properties"), null);
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
/**
* Helper to load refresh token from the token.properties file.
*/
private static String loadRefreshToken(){
Properties properties = new Properties();
try {
properties.load(new FileInputStream("token.properties"));
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
return (String) properties.get("refreshtoken");
}
/**
*
* List available Datasets.
*/
public static void listDatasets(Bigquery bigquery, String projectId)
throws IOException {
Datasets.List datasetRequest = bigquery.datasets().list(projectId);
DatasetList datasetList = datasetRequest.execute();
if (datasetList.getDatasets() != null) {
List<DatasetList.Datasets> datasets = datasetList.getDatasets();
System.out.println("Available datasets\n----------------");
for (com.google.api.services.bigquery.model.DatasetList.Datasets dataset : datasets) {
System.out.format("%s\n", dataset.getDatasetReference().getDatasetId());
}
}
}
}
An alternative to authorization via storing and using a refresh token to acquire a new access token in your installed application is to use a server to server service account authorization flow. In this case, your application will need to be able to securely store and use a unique private key. Here is an example of this type of flow using the Google Java API Client:
import com.google.api.client.http.HttpTransport;
import com.google.api.client.http.javanet.NetHttpTransport;
import com.google.api.client.json.JsonFactory;
import com.google.api.client.json.jackson.JacksonFactory;
import com.google.api.client.googleapis.auth.oauth2.GoogleCredential;
import com.google.api.services.bigquery.Bigquery;
import com.google.api.services.bigquery.Bigquery.Datasets;
import com.google.api.services.bigquery.model.DatasetList;
import java.io.File;
import java.io.IOException;
import java.security.GeneralSecurityException;
public class JavaCommandLineServiceAccounts {
private static final String SCOPE = "https://www.googleapis.com/auth/bigquery";
private static final HttpTransport TRANSPORT = new NetHttpTransport();
private static final JsonFactory JSON_FACTORY = new JacksonFactory();
private static Bigquery bigquery;
public static void main(String[] args) throws IOException, GeneralSecurityException {
GoogleCredential credential = new GoogleCredential.Builder().setTransport(TRANSPORT)
.setJsonFactory(JSON_FACTORY)
.setServiceAccountId("XXXXXXX#developer.gserviceaccount.com")
.setServiceAccountScopes(SCOPE)
.setServiceAccountPrivateKeyFromP12File(new File("my_file.p12"))
.build();
bigquery = new Bigquery.Builder(TRANSPORT, JSON_FACTORY, credential)
.setApplicationName("BigQuery-Service-Accounts/0.1")
.setHttpRequestInitializer(credential).build();
Datasets.List datasetRequest = bigquery.datasets().list("publicdata");
DatasetList datasetList = datasetRequest.execute();
System.out.format("%s\n", datasetList.toPrettyString());
}
}