Get body of error response with Spring Reactive Elasticsearch Client - spring-data-elasticsearch

Question

I'm using Spring Data Elasticsearch to get some aggregations, and I'm trying to use the ReactiveElasticsearchClient. I'm getting a 500 error from Elasticsearch, but can't figure out how to get the body of the response so that I can debug what's wrong with my request. Here's what I have so far:
final Flux<Aggregation> resp = client.aggregate(request -> {
final SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
sourceBuilder.query(QueryBuilders.queryStringQuery(query)).size(0)
.aggregation(AggregationBuilders.composite("report", compositeValueBuilders));
request.indices(index).source(sourceBuilder);
});
resp.doOnError(throwable -> {
throwable.printStackTrace();
}).log().doOnEach(signal -> {
if (signal.hasError()) {
signal.getThrowable().printStackTrace();
} else {
final Aggregation agg = signal.get();
System.out.println(agg.getType());
System.out.println(agg.getClass());
System.out.println(agg.getName());
}
}).blockLast();
With trace logging enabled for org.springframework.data.elasticsearch.client.WIRE, the request body is logged, but the log line for the response is just this:
2020-06-09 14:12:46.625 TRACE 19999 --- [or-http-epoll-1] o.s.data.elasticsearch.client.WIRE : [4ed5a037] Received raw response: 500 INTERNAL_SERVER_ERROR
I'm getting the following stack trace. I can see the ElasticsearchStatusException, but I don't see anything I can do with the Signal object to get the response body.
org.elasticsearch.ElasticsearchStatusException: POST request to /<index name>/_search returned error code 500.
at org.springframework.data.elasticsearch.client.reactive.DefaultReactiveElasticsearchClient.handleServerError(DefaultReactiveElasticsearchClient.java:809) ~[spring-data-elasticsearch-4.0.0.RELEASE.jar:4.0.0.RELEASE]
at org.springframework.data.elasticsearch.client.reactive.DefaultReactiveElasticsearchClient.readResponseBody(DefaultReactiveElasticsearchClient.java:760) ~[spring-data-elasticsearch-4.0.0.RELEASE.jar:4.0.0.RELEASE]
at org.springframework.data.elasticsearch.client.reactive.DefaultReactiveElasticsearchClient.lambda$sendRequest$22(DefaultReactiveElasticsearchClient.java:680) ~[spring-data-elasticsearch-4.0.0.RELEASE.jar:4.0.0.RELEASE]
at reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain.onNext(MonoFlatMapMany.java:156) ~[reactor-core-3.3.5.RELEASE.jar:3.3.5.RELEASE]
at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onNext(FluxOnErrorResume.java:73) ~[reactor-core-3.3.5.RELEASE.jar:3.3.5.RELEASE]
at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1755) ~[reactor-core-3.3.5.RELEASE.jar:3.3.5.RELEASE]
at reactor.core.publisher.MonoFlatMap$FlatMapInner.onNext(MonoFlatMap.java:241) ~[reactor-core-3.3.5.RELEASE.jar:3.3.5.RELEASE]
at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onNext(FluxOnErrorResume.java:73) ~[reactor-core-3.3.5.RELEASE.jar:3.3.5.RELEASE]
at reactor.core.publisher.FluxSwitchIfEmpty$SwitchIfEmptySubscriber.onNext(FluxSwitchIfEmpty.java:67) ~[reactor-core-3.3.5.RELEASE.jar:3.3.5.RELEASE]
at reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:385) ~[reactor-core-3.3.5.RELEASE.jar:3.3.5.RELEASE]
at reactor.core.publisher.MonoPeekTerminal$MonoTerminalPeekSubscriber.onNext(MonoPeekTerminal.java:173) ~[reactor-core-3.3.5.RELEASE.jar:3.3.5.RELEASE]
at reactor.core.publisher.FluxMap$MapConditionalSubscriber.onNext(FluxMap.java:213) ~[reactor-core-3.3.5.RELEASE.jar:3.3.5.RELEASE]
at reactor.core.publisher.FluxPeekFuseable$PeekConditionalSubscriber.onNext(FluxPeekFuseable.java:845) ~[reactor-core-3.3.5.RELEASE.jar:3.3.5.RELEASE]
at reactor.core.publisher.FluxPeekFuseable$PeekConditionalSubscriber.onNext(FluxPeekFuseable.java:845) ~[reactor-core-3.3.5.RELEASE.jar:3.3.5.RELEASE]
at reactor.core.publisher.MonoNext$NextSubscriber.onNext(MonoNext.java:76) ~[reactor-core-3.3.5.RELEASE.jar:3.3.5.RELEASE]
at reactor.core.publisher.Operators$ScalarSubscription.request(Operators.java:2317) ~[reactor-core-3.3.5.RELEASE.jar:3.3.5.RELEASE]
at reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain.onSubscribeInner(MonoFlatMapMany.java:143) ~[reactor-core-3.3.5.RELEASE.jar:3.3.5.RELEASE]
at reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain.onNext(MonoFlatMapMany.java:182) ~[reactor-core-3.3.5.RELEASE.jar:3.3.5.RELEASE]
at reactor.core.publisher.FluxRetryPredicate$RetryPredicateSubscriber.onNext(FluxRetryPredicate.java:82) ~[reactor-core-3.3.5.RELEASE.jar:3.3.5.RELEASE]
at reactor.core.publisher.MonoCreate$DefaultMonoSink.success(MonoCreate.java:156) ~[reactor-core-3.3.5.RELEASE.jar:3.3.5.RELEASE]
at reactor.netty.http.client.HttpClientConnect$HttpIOHandlerObserver.onStateChange(HttpClientConnect.java:428) ~[reactor-netty-0.9.7.RELEASE.jar:0.9.7.RELEASE]
at reactor.netty.ReactorNetty$CompositeConnectionObserver.onStateChange(ReactorNetty.java:514) ~[reactor-netty-0.9.7.RELEASE.jar:0.9.7.RELEASE]
at reactor.netty.resources.PooledConnectionProvider$DisposableAcquire.onStateChange(PooledConnectionProvider.java:536) ~[reactor-netty-0.9.7.RELEASE.jar:0.9.7.RELEASE]
at reactor.netty.resources.PooledConnectionProvider$PooledConnection.onStateChange(PooledConnectionProvider.java:427) ~[reactor-netty-0.9.7.RELEASE.jar:0.9.7.RELEASE]
at reactor.netty.http.client.HttpClientOperations.onInboundNext(HttpClientOperations.java:562) ~[reactor-netty-0.9.7.RELEASE.jar:0.9.7.RELEASE]
at reactor.netty.channel.ChannelOperationsHandler.channelRead(ChannelOperationsHandler.java:96) ~[reactor-netty-0.9.7.RELEASE.jar:0.9.7.RELEASE]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) ~[netty-handler-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436) ~[netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) ~[netty-codec-4.1.49.Final.jar:4.1.49.Final]
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:311) ~[netty-codec-4.1.49.Final.jar:4.1.49.Final]
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:425) ~[netty-codec-4.1.49.Final.jar:4.1.49.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) ~[netty-codec-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251) ~[netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[netty-transport-4.1.49.Final.jar:4.1.49.Final]
at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) ~[netty-transport-native-epoll-4.1.49.Final-linux-x86_64.jar:4.1.49.Final]
at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) ~[netty-transport-native-epoll-4.1.49.Final-linux-x86_64.jar:4.1.49.Final]
at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) ~[netty-transport-native-epoll-4.1.49.Final-linux-x86_64.jar:4.1.49.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[netty-common-4.1.49.Final.jar:4.1.49.Final]
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[netty-common-4.1.49.Final.jar:4.1.49.Final]
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[netty-common-4.1.49.Final.jar:4.1.49.Final]
at java.base/java.lang.Thread.run(Thread.java:834) ~[na:na]
Tl;dr: How do I get the response body of an error response when using the Spring ReactiveElasticsearchClient?

Answer 1

You should activate the logging by setting the logging level of
org.springframework.data.elasticsearch.client.WIRE
to trace level. This logs what is sent and received, especially in the case of errors (excerpt from the implementation):
private <T> Publisher<? extends T> readResponseBody(String logId, Request request, ClientResponse response,
Class<T> responseType) {
// ...
if (response.statusCode().is5xxServerError()) {
ClientLogger.logRawResponse(logId, response.statusCode());
return handleServerError(request, response);
}
if (response.statusCode().is4xxClientError()) {
ClientLogger.logRawResponse(logId, response.statusCode());
return handleClientError(logId, request, response, responseType);
}
// ...
}
Edit:
sorry, my fault, should have seen that the response body is not logged.
What you could do is use an intercepting proxy that monitors the traffic between your application and Elasticsearch. There's the OWASP ZAP proxy and the Burp Suite Community edition. Both applications start as a proxy on port 8080, so you either need to configure your application to listen on a different port (server.port in the application properties, I use for example 9090 for my test app) or change the proxy port. Besides that you need to configure the proxy in your app, I have the following configuration:
#Configuration
public class ReactiveRestClientConfig extends AbstractReactiveElasticsearchConfiguration {
#Override
public ReactiveElasticsearchClient reactiveElasticsearchClient() {
ClientConfiguration clientConfiguration = ClientConfiguration.builder() //
.connectedTo("localhost:9200") //
.withProxy("localhost:8080")
.build();
return ReactiveRestClients.create(clientConfiguration);
}
}
The when accessing you application, the traffic to Elasticsearch goes through the proxy where you can see the requests and responses.