I have created a spring batch job and running it locally with following command
java -jar my-indexer-job-1.0-SNAPSHOT.jar jobType=category
Locally, I am able to get the jobType parameter in the step from chunk context as shown below
#Bean
public Step initialStep() {
return steps
.get("initialStep").tasklet(
(stepContribution, chunkContext) -> {
String jobType = chunkContext.getStepContext().getStepExecution().getJobParameters().getString("jobType");
log.info("indexing job of type {}", jobType);
.....
I deployed this job on spring cloud data flow and
try to pass this jobType parameter from Arguments sections of data flow UI as shown in the below image link
data-flow-ui-image
But it is not working.
I am not sure how can I pass this jobType parameter from spring cloud data flow UI. Can anyone please help on this?
Related
I am trying to add a grpc protofile to my swagger-ui. I am consuming a grpc webservice which needs a protofile as input. The input to my spring boot restful webservice needs to have that same grpc structure as its interface. I recevied a jar from the individual that made the protofile and imported it to my webserivce. When I try to add the #ResponseBody tag around the object from the protofile jar, my app hangs on this in the console at startup:
s.d.s.w.s.ApiListingReferenceScanner : Scanning for api listing references
Thanks,
Brian
Never return entity objects in controller method.
in my case. my Controller methods takes this parameter.
"#AuthenticationPrincipal UserSession userSession"
when i exlude UserSession object swagger back to normal.
There were 2 way to do that
first is "#ApiIgnore #AuthenticationPrincipal UserSession userSession"
second is in swaggerConfig class
private Class[] clazz = {UserSession.class};
Docket().ignoredParameterTypes(clazz)
Incase someone needs a solution, what I did was as a work around for now.
in my service's code (response is a String)
return JsonFormat.printer().print(myProtoObject);
in my client's code:
Builder b = ProtoObject.newBuilder();
JsonFormat.parser().merge(result.getBody(), b);
ProtoObject protoObject = b.build();
I have a Spring Boot application deployed and configured as AWS Route 53 > AWS Load Balance -> 2 EC2 instances which hosted the Spring Boot application.
The URL for the Swagger is
https://applicationXYZ.company.net/release/swagger-ui.html
I'm able to see the page without any issue. But we can't use the 'Tryout' feature because the Base URL is wrong.
On top of the page I do see information as
[ Base URL: service/release]
I have no idea where 'service' became my base URL. I also hit api-docs and also see 'server' in 'host' field.
Could you please help on this?
Note: I'm using Spring Boot Starter 2.0.8.RELEASE and Swagger 2.9.2 (without any Spring Security)
Thanks,
Did you ever try to make a redirect,
//Do redirection inside controller
#RequestMapping("/swagger")
public String greeting() {
return "redirect:/swagger-ui.html";
}
you can try to add bean too, inside main method,
#Bean
RouterFunction<ServerResponse> routerFunction() {
return route(GET("/swagger"), req ->
ServerResponse.temporaryRedirect(URI.create("swagger-ui.html")).build());
}
refer: How to change Swagger-ui URL prefix?
Changing the Order of the Spring Security WebFilter
I have an API Gateway implemented using Spring Cloud Gateway that uses Spring Security. Spring Security for WebFlux is implemented as a WebFilter right at the beginning of the filter chain. So after successful authentication the request would be forwarded to Spring Cloud Gateway's RoutePredicateHandlerMapping, which would try to deduce the destination based on the URL pattern, and then it would go to a FilteringWebHandler to execute the other filters of Spring Cloud Gateway.
My problem is the following: I have implemented a customized authentication algorithm which uses query string and header variables as credentials for authentication according to the requirements of the project, an this is working without any problem. The problem occurred when we needed to add a small customization for the authentication algorithm that is path independent. When the request reaches the WebFilter of Spring Security, pattern matching is not yet done so I do not know which application does it point to, for example:
app1:
-Path: /app1/**
app2:
-Path: /app2/**
Which means that instead of having authentication -> route mapping -> filtering web handler I should do route mapping -> authentication -> filtering web handler. Not that these three components are not similar, one of them is a filter another is a mapper and the last one is web handler. Now I know how to customize them but the problem is that I do not know how to intercept the Netty server building process in order to change the order of these operations. I need to wait for the building process to end and alter the content of the server before it starts. How can I do that?
EDIT: here is the final solution:
So here is how I did it:
Goal: removing the WebFilter of Spring Security from the default HttpHandler, and inserting it between RoutePredicateRouteMapping and the FilteringWebHandler of Spring Cloud Gateway
Why: Because I need to know the Application ID while carrying on my customized authentication process. This Application ID is attached to the request by the RoutePredicateRouteMapping by matching the request's URL to a predefined list.
How did I do it:
1- Removing the WebFilter of Spring Security
I created an HttpHandler bean that invokes the default WebHttpHandlerBuilder and then customize the filters. As a bonus, I removed unneeded filters in order to increase the performance of my API Gateway
#Bean
public HttpHandler httpHandler() {
WebHttpHandlerBuilder webHttpHandlerBuilder = WebHttpHandlerBuilder.applicationContext(this.applicationContext);
MyAuthenticationHandlerAdapter myAuthenticationHandlerAdapter = this.applicationContext.getBean(MY_AUTHENTICATED_HANDLER_BEAN_NAME, MyAuthenticationHandlerAdapter.class);
webHttpHandlerBuilder
.filters(filters ->
myAuthenticationHandlerAdapter.setSecurityFilter(
Collections.singletonList(filters.stream().filter(f -> f instanceof WebFilterChainProxy).map(f -> (WebFilterChainProxy) f).findFirst().orElse(null))
)
);
return webHttpHandlerBuilder.filters(filters -> filters
.removeIf(f -> f instanceof WebFilterChainProxy || f instanceof WeightCalculatorWebFilter || f instanceof OrderedHiddenHttpMethodFilter))
.build();
}
2- Wrapping Spring Cloud Gateway's FilteringWebHandler with Spring Web's FilteringWebHandler with the added WebFilter
I created my own HandlerAdapter which would match against Spring Cloud Gateway's FilteringWebHandler and wrap it with Spring Web's FilteringWebHandler plus the security filter I extracted in the first step
#Bean
public MyAuthenticationHandlerAdapter myAuthenticationHandlerAdapter() {
return new MyAuthenticationHandlerAdapter();
}
public class MyAuthenticationHandlerAdapter implements HandlerAdapter {
#Setter
private List<WebFilter> securityFilter = new ArrayList<>();
#Override
public boolean supports(Object handler) {
return handler instanceof FilteringWebHandler;
}
#Override
public Mono<HandlerResult> handle(ServerWebExchange exchange, Object handler) {
org.springframework.web.server.handler.FilteringWebHandler filteringWebHandler = new org.springframework.web.server.handler.FilteringWebHandler((WebHandler) handler, securityFilter);
Mono<Void> mono = filteringWebHandler.handle(exchange);
return mono.then(Mono.empty());
}
}
This way I could achieve better performance with highly customized HttpHandler pipeline that I suppose to be future-proof
END EDIT
Spring Security for WebFlux is implemented as a WebFilter which is executed almost as soon as a request is received. I have implemented custom authentication converter and authentication manager which would extract some variables from the header and URL and use them for authentication. This is working without any problem.
Now I needed to add another variable taken from RoutePredicateRouteMapping before authentication is done. What I want exactly is to remove the WebFilter (called WebFilterChainProxy) from its current position and put it between the RoutePredicateRouteMapping and the FilteringWeHandler.
Here is how the default process goes:
ChannelOperations calls ReactorHttpHandlerAdapter which calls HttpWebHandlerAdapter, ExceptionHandlingWebHandler, and then org.springframework.web.server.handler.FilterWebHandler.
This WebHandler would invoke its filters and then call the DispatchHandler. One of those filters is the WebFilterChainProxy that does the authentication for Spring Security. So first step is removing the filter from here.
Now the DispatchHandler which is called after the filters would invoke RoutePredicateHandlerMapping, which would analyze the routes and give me the route ID that I need, and then it would call the org.springframework.cloud.gateway.handler.FilteringHandler (this is not the same FilteringHandler above), and that in turn would call the other filters of the Spring Cloud Gateway. What I want here is to invoke the filter after RoutePredicatehandlerMapping and before org.springframework.cloud.gateway.handler.FilteringHandler.
What I ended doing was the following:
I created and WebHttpHandlerBuilder that would remove WebFilterChainProxy and pass it as a parameter to a customized DispatcherHandler. Now that the filter is removed the request would pass the first layers without requiring authentication. In my customized DispatcherHandler I would invoke the RoutePredicateHandlerMapping and then pass the exchange variable to the WebFilterChainProxy to do the authentication before passing it to the org.springframework.cloud.gateway.handler.FilteringHandler, which worked perfectly!
I still think that I'm over engineering it and I hope that there is a way to do it using annotations and configuration beans instead of all these customized classes (WebHttpHandlerBuilder and DispatcherHandler).
You should probably implement that security filter as a proper GatewayFilter, since only those are aware of the other GatewayFilter instances and can be ordered accordingly. In your case, you probably want to order it after the routing one.
Also, please don't cross-post, the Spring team is actively monitoring StackOverflow.
I had a similar problem. The accepted solution, while interesting, was a bit drastic for me. I was able to make it work simply by adding my custom filter before SecurityWebFiltersOrder.AUTHENTICATION in the security configuration. This is similar to what I've done with success in a regular Spring mvc application.
Here's an example using oauth authentication. tokenIntrospector is my custom introspector, and requestInitializationFilter is the filter that grabs the tenant id and stashes it in the context.
#AllArgsConstructor
#Configuration
#EnableWebFluxSecurity
public class WebApiGatewaySecurityConfiguration {
private final GatewayTokenIntrospector tokenIntrospector;
private final GatewayRequestInitializationFilter requestInitializationFilter;
#Bean
public SecurityWebFilterChain securityWebFilterChain(ServerHttpSecurity http) {
// #formatter:off
http
.formLogin().disable()
.csrf().disable()
.oauth2ResourceServer(oauth2ResourceServer ->
oauth2ResourceServer.opaqueToken(c -> c.introspector(tokenIntrospector)))
.addFilterBefore(requestInitializationFilter, SecurityWebFiltersOrder.AUTHENTICATION);
return http.build();
// #formatter:on
}
}
I have a streaming job in which I'm listening to message from PubSub and after that reading the data from BigQuery. Data has is queried using the data received from PubSUb. This means I need to form the query dynamically and then pass it to the BigQueryIO.Read.fromQuery() function. Below is the code which is going to read data from the BigQuery and return a TableRow, but it is giving me NullPointerException where my code is executing data to read.
public class RequestDailyUsageTransform extends PTransform<PCollection<DailyUsageJob>, PCollection<TableRow>> {
private String mQuery;
private String mForDate;
private LocalDateTime billingDateTime;
#Override
public PCollection<TableRow> apply(PCollection<DailyUsageJob> input) {
TableReference tableReference = getRequestTableReference();
return input
.apply(ParDo.named("RequestUsageQuery")
.of(new RequestUsageQueryStringDoFn()))
.apply(BigQueryIO.Read.named("RequestUsageReader")
.fromQuery(mQuery)
.from(tableReference).withoutValidation())
.apply(ParDo.named("DailyRequestMapper").of(new DailyRequestMapperDoFn()))
.apply(ParDo.named("BillDailyRequestUsage")
.of(new DailyRequestsBillDoFn(mForDate, billingDateTime)));
}}
I also wanted to know how to pass the string which was generated in a DoFn in BigQueryIO.Read.fromQuery() function.
I think in the case the best thing to do would be to run a daily batch job that queries all the data, and is keyed by the userid. This will pull in more data than you would like, but allow you to locate the per user information. Unfortuately there is not currently a way do perform data dependent reads
We need to be sure that our web application and mobile client are communicating correctly.
There is two-side communications from the server (Rails application with rspec testing) to the mobile client (Ruby application, has mspec testing framework) and from the mobile client to the server.
So to be sure that the synchronization mechanism is working as expected we need to test the following things:
Server prepares the data correctly.
Mobile client requests and gets
correct data.
Mobile client
prepares
the data to be sent to the server
correctly.
Server recieves and
parses the correct data from the
mobile client.
Servers sends
response to mobile client that
everything is ok.
Mobile client
should carry out appropriate actions
on the device.
How to test this in isolation?
As for all tests, don't plan for the unexpected. Start with what you know. The unexpected will rear it's ugly head soon enough to tell you what else you should test.
What you have is actually simple to test if you break it apart. Here is my approach:
public final static String SERVER_DATA = "Prepared data from the server";
#Test
public void testServerPreparesDataCorrectly() throws Exception {
... usual setup ...
String actual = server.handleRequest( CLIENT_REQUEST );
assertEquals( SERVER_DATA, actual );
}
public final static String CLIENT_REQUEST = "...";
#Test
public void testClientRequest() throws Exception {
... usual setup ...
String actual = client.getRequestData(...);
assertEquals( CLIENT_REQUEST, actual );
}
#Test
public void testClientResponseProcessing() throws Exception {
... usual setup ...
client.parseServerResponse( SERVER_DATA );
... verify client state ...
}
and so on. The basic idea is to put the input and output of each process step into a constant, then run the code which implements the process step for each expected input and validate the output. Where most outputs are also inputs for other tests.
If something changes, you update the inputs/outputs accordingly. Run the tests. And the failures will tell you which process steps you have to update.