I have a really complicated flux here, something like this:
private Object run(Flux<Pair<String, Flux<Record>>> flux) throws InterruptedException {
BlockingQueue<Object> result = new SynchronousQueue<>();
flux.subscribeOn(Schedulers.boundedElastic())
.parallel(2, 1)
.runOn(Schedulers.boundedElastic())
.flatMap(pair -> pair.getRight().parallel()
.runOn(Schedulers.boundedElastic())
.flatMap(record -> stringToError(record))
.doOnComplete(() -> System.out.println("Complete " + pair.getLeft()))) // #log
.sequential()
.buffer(1000)
.subscribe(new BaseSubscriber<List<Error>>() {
#Override
protected void hookOnComplete() {
result.offer(Boolean.TRUE);
}
#Override
protected void hookOnError(Throwable throwable) {
result.offer(throwable);
}
#Override
protected void hookOnCancel() {
throw new IllegalStateException();
}
});
return result.take();
}
It runs pretty well except that sometimes it blocked forever.
NO hookOnComplete(), NO hookOnError(), NO hookOnCancel().
And when this happened, line #log does already print all data I feed in. That's so strange.
I don't know how to deal with this.
Can anyone tells me what can I do and what could cause this?
By the way, I use reactor-core 3.3.2 here.
Related
To prevent the XY problem, I'll start from the beginning:
I have a non-blocking SOAP client which I wrapped it to make the return type Mono<T> (By default it accepts callback. I can elaborate on this if needed).
Now I want to do (given ID):
1. Get the code by ID
2. Do something with the code
3. After that, get Foo and Bar and create FooBar
What I wrote was:
public class MyService {
private final MySoapClient soapClient;
public Mono<FooBarDto> doSomething(String id) {
return Mono.just(id)
.flatMap(soapClient::getCode) // returns Mono<String>
.flatMap(code ->
soapClient.doSomething(code) // returns Mono<Void>
.then(getFooBar(id, code))); // See this
}
private Mono<FooBarDto> getFooBar(String id, String code) {
return Mono.zip(
soapClient.getFoo(code), // returns Mono<Foo>
soapClient.getBar(code) // returns Mono<Bar>
).map(tuple2 -> toFooBarDto(id, tuple2));
}
private FooBarDto toFooBarDto(String id, Tuple2<Foo, Bar> tuple2) {
return FooBarDto.builder()/* set properties */.build();
}
}
Now the problem is, because methods of the SOAP client are not lazy (the moment you call them they start the process), the semantic of then won't work here. Meaning I want to get Foo and Bar when doSomething is done. They all start together.
I tried to change it fix it by changing then to flatMap, but made it even worse. The getFooBar never got called. (1. Can someone please explain why?).
So what I ended up doing was to wrap SOAP calls again to make them lazy:
public class MySoapClient {
private final AutoGeneratedSoapClient client;
Mono<Foo> getFoo(GetFooRequest request) {
return Mono.just(request).flatMap(this::doGetMsisdnByIccid);
}
private Mono<Foo> doGetFoo(GetFooRequest request) {
val handler = new AsyncHandler<GetFooRequest>();
client.getFoo(request, handler);
return Mono.fromFuture(handler.future);
}
private static class AsyncHandler<T> implements javax.xml.ws.AsyncHandler<T> {
private final CompletableFuture<T> future = new CompletableFuture<>();
#Override
public void handleResponse(Response<T> res) {
try {
future.complete(res.get());
} catch (Exception e) {
future.completeExceptionally(e);
}
}
}
}
Is there any better way to do it? Specifically:
2. Using CompeletableFuture and the callback.
3. Making methods lazy in the SOAP client.
I tried to change it fix it by changing then to flatMap, but made it
even worse. The getFooBar never got called. (1. Can someone please
explain why?)
I think a Mono<Void> always completes empty (or error), so subsequent flatMap is never called.
Using CompeletableFuture and the callback.
Making methods lazy in the SOAP client.
To make the call lazy you can do one of the followings:
1, You can use Mono.fromFuture which accepts a supplier:
private Mono<Foo> doGetFoo(GetFooRequest request) {
return Mono.fromFuture(() -> {
val handler = new AsyncHandler<GetFooRequest>();
client.getFoo(request, handler);
return handler.future;
});
}
2, You can use Mono.defer:
private Mono<Foo> doGetFoo(GetFooRequest request) {
return Mono.defer(() -> {
val handler = new AsyncHandler<GetFooRequest>();
client.getFoo(request, handler);
return Mono.fromFuture(handler.future);
});
}
3, You can get rid of CompletableFuture and use Mono.create instead, something like this:
private Mono<Foo> doGetFoo(GetFooRequest request) {
return Mono.create(sink -> {
AsyncHandler<Foo> handler = response ->
{
try
{
sink.success(response.get());
} catch (Exception e)
{
sink.error(e);
}
};
client.getFoo(request, handler);
});
}
If you do any of these it will be safe to use then method and it will work as expected.
i am not sure my understanding to errorHandler and returnExceptions is right or not.
but here is my goal: i sent a message from App_A, use #RabbitListener to receive message in App_B.
according to the doc
https://docs.spring.io/spring-amqp/docs/2.1.3.BUILD-SNAPSHOT/reference/html/_reference.html#annotation-error-handling
i assume if APP_B has a business exception during process the message,through set errorHandler and returnExceptions in a right way on #RabbitListener can let the exception back to App_A.
do I understood correctly?
if i am rigth, how to use it in a right way?
with my code, i get nothing in APP_A .
here is my code in APP_B
errorHandler:
#Component(value = "errorHandler")
public class ErrorHandler implements RabbitListenerErrorHandler {
#Override
public Object handleError(Message arg0, org.springframework.messaging.Message<?> arg1,
ListenerExecutionFailedException arg2) throws ListenerExecutionFailedException {
throw new ListenerExecutionFailedException("msg", arg2, null);
}
}
RabbitListener:
#RabbitListener(
bindings = #QueueBinding(
value = #Queue(value = "MRO.updateBaseInfo.queue", durable = "true"),
exchange = #Exchange(name = "MRO_Exchange", type = ExchangeTypes.DIRECT, durable = "true"),
key = "baseInfoUpdate"
),
// errorHandler = "errorHandler",
returnExceptions = "true"
)
public void receiveLocationChangeMessage(String message){
BaseUpdateMessage newBaseInfo = JSON.parseObject(message, BaseUpdateMessage.class);
dao.upDateBaseInfo(newBaseInfo);
}
and code in APP_A
#Component
public class MessageSender {
#Autowired
private RabbitTemplate rabbitTemplate;
public void editBaseInfo(BaseUpdateMessage message)throws Exception {
//and i am not sure set RemoteInvocationAwareMessageConverterAdapter in this way is right
rabbitTemplate.setMessageConverter(new RemoteInvocationAwareMessageConverterAdapter());
rabbitTemplate.convertAndSend("MRO_Exchange", "baseInfoUpdate", JSON.toJSONString(message));
}
}
i am very confuse with three points:
1)do i have to use errorHandler and returnExceptions at the same time? i thought errorHandler is something like a postprocessor that let me custom exception.if i don't need a custom exception can i just set returnExceptions with out errorHandler ?
2)should the method annotated with #RabbitListener return something or void is just fine?
3)in the sender side(my situation is APP_A), does have any specific config to catch the exception?
my workspace environment:
Spring boot 2.1.0
rabbitMQ server 3.7.8 on docker
1) No, you don't need en error handler, unless you want to enhance the exception.
2) If the method returns void; the sender will end up waiting for timeout for a reply that will never arrive, just in case an exception might be thrown; that is probably not a good use of resources. It's better to always send a reply, to free up the publisher side.
3) Just the RemoteInvocationAwareMessageConverterAdapter.
Here's an example:
#SpringBootApplication
public class So53846303Application {
public static void main(String[] args) {
SpringApplication.run(So53846303Application.class, args);
}
#RabbitListener(queues = "foo", returnExceptions = "true")
public String listen(String in) {
throw new RuntimeException("foo");
}
#Bean
public ApplicationRunner runner(RabbitTemplate template) {
template.setMessageConverter(new RemoteInvocationAwareMessageConverterAdapter());
return args -> {
try {
template.convertSendAndReceive("foo", "bar");
}
catch (Exception e) {
e.printStackTrace();
}
};
}
}
and
org.springframework.amqp.AmqpRemoteException: java.lang.RuntimeException: foo
at org.springframework.amqp.support.converter.RemoteInvocationAwareMessageConverterAdapter.fromMessage(RemoteInvocationAwareMessageConverterAdapter.java:74)
at org.springframework.amqp.rabbit.core.RabbitTemplate.convertSendAndReceive(RabbitTemplate.java:1500)
at org.springframework.amqp.rabbit.core.RabbitTemplate.convertSendAndReceive(RabbitTemplate.java:1433)
at org.springframework.amqp.rabbit.core.RabbitTemplate.convertSendAndReceive(RabbitTemplate.java:1425)
at com.example.So53846303Application.lambda$0(So53846303Application.java:28)
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:804)
at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:794)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:324)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1260)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1248)
at com.example.So53846303Application.main(So53846303Application.java:15)
Caused by: java.lang.RuntimeException: foo
at com.example.So53846303Application.listen(So53846303Application.java:20)
As you can see, there is a local org.springframework.amqp.AmqpRemoteException with the cause being the actual exception thrown on the remote server.
I have DSL that has comments like
{***** this is comment
When i type comment like {** and hit enter xtext autoedit provide } to close open {, I Can disable it by
commenting configureCompoundBracesBlocks method
#Override
protected void configureCompoundBracesBlocks(IEditStrategyAcceptor acceptor) {
// acceptor.accept(compoundMultiLineTerminals.newInstanceFor("{", "}").and("[", "]").and("(", ")"), IDocument.DEFAULT_CONTENT_TYPE);
}
But I want to auto close for remain all syntax. Is there any way to crack it down ??
here is my screen-cast how it like when i type comment on my DSL
Also here is my code what it like to get configureCompoundBracesBlocks
public class MyAutoEditStrategyProvider extends DefaultAutoEditStrategyProvider {
#Override
protected void configure(IEditStrategyAcceptor acceptor) {
configureCompoundBracesBlocks(acceptor);
}
#Override
protected void configureCompoundBracesBlocks(IEditStrategyAcceptor acceptor) {
acceptor.accept(compoundMultiLineTerminals.newInstanceFor("{", "}").and("[", "]").and("(", ")"), IDocument.DEFAULT_CONTENT_TYPE);
}
}
I was trying reactor library and I'm not able to figure out why below mono never return back with onNext or onComplete call. I think I missing very trivial thing. Here's a sample code.
MyServiceService service = new MyServiceService();
service.save("id")
.map(myUserMono -> new MyUser(myUserMono.getName().toUpperCase(), myUserMono.getId().toUpperCase()))
.subscribe(new Subscriber<MyUser>() {
#Override
public void onSubscribe(Subscription s) {
System.out.println("Subscribed!" + Thread.currentThread().getName());
}
#Override
public void onNext(MyUser myUser) {
System.out.println("OnNext on thread " + Thread.currentThread().getName());
}
#Override
public void onError(Throwable t) {
System.out.println("onError!" + Thread.currentThread().getName());
}
#Override
public void onComplete() {
System.out.println("onCompleted!" + Thread.currentThread().getName());
}
});
}
private static class MyServiceService {
private Repository myRepo = new Repository();
public Mono<MyUser> save(String userId) {
return myRepo.save(userId);
}
}
private static class Repository {
public Mono<MyUser> save(String userId) {
return Mono.create(myUserMonoSink -> {
Future<MyUser> submit = exe.submit(() -> this.blockingMethod(userId));
ListenableFuture<MyUser> myUserListenableFuture = JdkFutureAdapters.listenInPoolThread(submit);
Futures.addCallback(myUserListenableFuture, new FutureCallback<MyUser>() {
#Override
public void onSuccess(MyUser result) {
myUserMonoSink.success(result);
}
#Override
public void onFailure(Throwable t) {
myUserMonoSink.error(t);
}
});
});
}
private MyUser blockingMethod(String userId) throws InterruptedException {
Thread.sleep(5000);
return new MyUser("blocking", userId);
}
}
Above code only prints Subcribed!main. What I'm not able to figure out is why that future callback is not pushing values through myUserMonoSink.success
The important thing to keep in mind is that a Flux or Mono is asynchronous, most of the time.
Once you subscribe, the asynchronous processing of saving the user starts in the executor, but execution continues in your main code after .subscribe(...).
So the main thread exits, terminating your test before anything was pushed to the Mono.
[sidebar]: when is it ever synchronous?
When the source of data is a Flux/Mono synchronous factory method. BUT with the added pre-requisite that the rest of the chain of operators doesn't switch execution context. That could happen either explicitly (you use a publishOn or subscribeOn operator) or implicitly (some operators like time-related ones, eg. delayElements, run on a separate Scheduler).
Simply put, your source is ran in the ExecutorService thread of exe, so the Mono is indeed asynchronous. Your snippet on the other hand is ran on main.
How to fix the issue
To observe the correct behavior of Mono in an experiment (as opposed to fully async code in production), several possibilities are available:
keep subscribe with system.out.printlns, but add a new CountDownLatch(1) that is .countDown() inside onComplete and onError. await on the countdown latch after the subscribe.
use .log().block() instead of .subscribe(...). You lose the customization of what to do on each event, but log() will print those out for you (provided you have a logging framework configured). block() will revert to blocking mode and do pretty much what I suggested with the CountDownLatch above. It returns the value once available or throws an Exception in case of error.
instead of log() you can customize logging or other side effects using .doOnXXX(...) methods (there's one for pretty much every type of event + combinations of events, eg. doOnSubscribe, doOnNext...)
If you're doing a unit test, use StepVerifier from the reactor-tests project. It will subscribe to the flux/mono and wait for events when you call .verify(). See the reference guide chapter on testing (and the rest of the reference guide in general).
Issue is that in created anonymous class onSubscribe method does nothing.
If you look at implementation of LambdaSubscriber, it requests some number of events.
Also it's easier to extend BaseSubscriber as it has some predefined logic.
So your subscriber implementation would be:
MyServiceService service = new MyServiceService();
service.save("id")
.map(myUserMono -> new MyUser(myUserMono.getName().toUpperCase(), myUserMono.getId().toUpperCase()))
.subscribe(new BaseSubscriber<MyUser>() {
#Override
protected void hookOnSubscribe(Subscription subscription) {
System.out.println("Subscribed!" + Thread.currentThread().getName());
request(1); // or requestUnbounded();
}
#Override
protected void hookOnNext(MyUser myUser) {
System.out.println("OnNext on thread " + Thread.currentThread().getName());
// request(1); // if wasn't called requestUnbounded() 2
}
#Override
protected void hookOnComplete() {
System.out.println("onCompleted!" + Thread.currentThread().getName());
}
#Override
protected void hookOnError(Throwable throwable) {
System.out.println("onError!" + Thread.currentThread().getName());
}
});
Maybe it's not the best implementation, I'm new to reactor too.
Simon's answer has pretty good explanation about testing asynchronous code.
We've found the "Fail Fast" principle crucial for improving maintainability of our large Fitnesse-based battery of tests. Slim's StopTestException is our saviour.
However, it's very cumbersome and counterproductive to catch and convert any possible exception to those custom StopExceptions. And this approach doesn't work outside of fixtures. Is there a way to tell fitnesse (preferably using Slim test system) to stop test on any error / exception?
Update: corresponding feature request https://github.com/unclebob/fitnesse/issues/935
Most of the exceptions coming from fixtures are possible to conveniently convert to the StopTestException by implementing the FixtureInteraction interface, e.g.:
public class StopOnException extends DefaultInteraction {
#Override
public Object newInstance(Constructor<?> constructor, Object... initargs) throws InvocationTargetException, InstantiationException, IllegalAccessException {
try {
return super.newInstance(constructor, initargs);
} catch (Throwable e) {
throw new StopTestException("Instantiation failed", e);
}
}
#Override
public Object methodInvoke(Method method, Object instance, Object... convertedArgs) throws InvocationTargetException, IllegalAccessException {
try {
return super.methodInvoke(method, instance, convertedArgs);
} catch (Throwable e) {
throw new StopTestException(e.getMessage(), e);
}
}
public static class StopTestException extends RuntimeException {
public StopTestException(String s, Throwable e) {
super(s, e);
}
}
}