I am wrapping my head around Flux Sinks and cannot understand the higher-level picture. When using Sinks.Many<T> tryEmitNext, the function tells me if there was contention and what should I do in case of failure, (FailFast/Handler).
But is there a simple construct which allows me to safely emit elements from multiple threads. For example, instead of letting the user know that there was contention and I should try again, maybe add elements to a queue(mpmc, mpsc etc), and only notify when the queue is full.
Now I can add a queue myself to alleviate the problem, but it seems a common use case. I guess I am missing a point here.
I hit the same issue, migrating from Processors which support safe emission from multiple threads. I use this custom EmitFailureHandler to do a busy loop as suggested by the EmitFailureHandler docs.
public static EmitFailureHandler etryOnNonSerializedElse(EmitFailureHandler fallback){
return (signalType, emitResult) -> {
if (emitResult == EmitResult.FAIL_NON_SERIALIZED) {
LockSupport.parkNanos(10);
return true;
} else
return fallback.onEmitFailure(signalType, emitResult);
};
}
There are various confusing aspects about the 3.4.0 implementation
There is an implication that unless the Unsafe variant is used, the sink supports serialized emission but actually all the serialized version does is to fail fast in case of concurrent emission.
The Sink provided by Flux.Create does support threadsafe emission.
I hope there will be a solidly engineered alternative to this offered by the library at some point.
Related
It's been hammered into my head that I shouldn't use ThreadLocal with Reactor. But I want to know if I can use ThreadLocal within a single execution of a reactor function.
Specifically, when inside a Spring Webflux Controller method, can the thread ever change if I don't invoke a reactor function?
Please let me know if this is correct
#GetMapping
public Mono<String> someControllerMethod() {
// Thread 1 executing
ThreadLocal<String> USER_ID = new ThreadLocal<>();
USER_ID.set("1");
Thread.sleep(...);
someMethod();
// Thread 1 executing
assertEquals(USER_ID.get(), "1"); // this will ALWAYS be true
return Mono.just("hello ")
// this is the only time a new thread executes and USER_ID is not set
.flatMap(s -> s + USER_ID.get());
}
void someMethod() {
// Thread 1 executing
assertEquals(USER_ID.get(), "1"); // this will ALWAYS be true
}
Is my understanding above correct?
Revised this section for clarity
In a reactor chain of many operators, each operator (e.g. map) could be run under different threads, and even different "instances?" (e.g. map of url N) of the same operator could be on different threads. But once we're in an instance of a operator, will it always be the same thread (ie is it safe to declare ThreadLocal in an instance of an reactor operator)?
// main thread
Flux.fromIterable(urls)
.map(url -> {
// each of these instances runs on a different thread
// but is declaring ThreadLocal here safe to do?
ThreadLocal<String> URL = new ThreadLocal<>();
URL.set(url);
// Will URL always be set deep in the call stack?
someOtherMethod();
// Will URL always be set at the end?
URL.get();
});
.subscribeOn(Schedules.boundedElastic())
.subscribe();
void someOtherMethod() {
URL.get(); // will this will ALWAYS be set?
}
Basically, I'd like to know whether it's safe to use ThreadLocal objects like io.grpc.Context within a single instance of a Reactor operator execution.
It's been hammered into my head that I shouldn't use ThreadLocal with Reactor.
You mustn't use ThreadLocal in a reactive chain with reactor (which is the only sensible way to use that library.) In a reactive chain, the thread might change whenever you invoke an asynchronous operator - so a single reactive chain could have operations executing on many different threads throughout. In this case your ThreadLocal might work sometimes, but it's unreliable - introduce an async operator that switches the thread (say a web request that's executed on the netty worker pool), and you've then introduced a subtle and weird bug that's hard to track down (you're arbitrarily leaking information from one reactive chain to another unintentionally.) In short, it's incredibly bad practice to tie your reactive chains to a single thread - while it might seem to work initially, you're going to eventually run into a lot of problems if you do.
That being said, you don't really have a reactive chain in the above method - it's incredibly weird. If you're returning a Mono<String> to try to make the method reactive, then you need to be executing everything as part of a reactive chain. What you're actually doing is:
Using synchronous & blocking logic, a complete no-no as it ties up an event loop thread which isn't allowed;
Calling another method that's not part of a reactive chain;
Using a JUnit test method in a controller class;
Wrapping up a value to return in Mono.just();
Making one flatMap call at the end (which won't work as it's not even mapping to a publisher to flatten, you'd have to use map instead.)
...so while using your ThreadLocal is technically "safe" in this context, from a wider perspective the implementation makes no sense at all. You realistically have two options - either make the entire method non-blocking and reactive properly, not just wrapping blocking logic in a reactive publisher, or make the whole controller just return a standard object and forget the reactive element entirely.
Follow-up:
once we're in an instance of a operator, will it always be the same thread (ie is it safe to declare ThreadLocal in an instance of an reactor operator)?
No, there's at least two cases I can think of where that wouldn't be safe:
Operators can be nested. Once you're "inside" a certain operator, there's no reason why other operators can't be used that would also switch thread.
Code in other threads can be explicitly started even if there's no operator.
I don't think you can wind up in cases where the thread changes under you other than those two, but I could well be missing something, and it's still a rather delicate scenario (someone could break it quite easily.) If you must use a Threadlocal for some reason then I'd still be seriously considering whether you should be using reactor in this context.
To process HTTP requests, we have to make blocking calls (e.g. JDBC calls) as part of a Mono/Flux-based process. Our current plan looks something like this:
// I renamed getSomething to processJaxrsHttpRequest
CompletionStage<String> processJaxrsHttpRequest(String input) {
return Mono.just(input)
.map(in -> process(in))
.flatMap(str -> Mono.fromCallable(() -> jdbcCall(str)).subscribeOn(fixedScheduler))
.flatMap(str -> asyncHttpCall(str))
.flatMap(str -> Mono.fromCallable(() -> jdbcCall(str)).subscribeOn(fixedScheduler))
.toFuture();
}
where fixedScheduler is used concurrently across HTTP requests.
We were hoping to get some feedback on this strategy for handling block calls within a decent number of fluxes. Of course, we understand that if all our requests were flowing through these blocking calls then we might as well not use reactor (outside of the admittedly nice processing API).
Update: Thanks bsideup for this answer. However, I should have been a little more specific with my questions.
My overall question is how to effectively have a blocking call used across multiple fluxes were these fluxes can be created/subscribed to in large numbers. We tried the suggested approach, but it results in an explosion of threads and quickly OOMs. So we are thinking to use a shared scheduler. So.. here are my questions.
Is using a shared scheduler (fixedScheduler) what you would suggest in the situation I describe? If not, will you point me in any directions?
If using a shared scheduler is good, would this be a good implementation of it: Schedulers.newParallel("blocking-scheduler", maxNumThreads)?
Update 2: Just dug a big on Schedulers#newParallel and realize that won't work since it 'rejects' blocking calls.
Really appreciate any tips!
While subscribeOn is indeed one way of handling blocking calls and your usage is okay, you can as well use publishOn.
It moves processing to the provided Scheduler, unless other publishOn is specified:
CompletionStage<String> getSomething(String input) {
return Mono.just(input)
.map(in -> process(in)) // process must be non-blocking, or go after publishOn
.publishOn(Schedulers.boundedElastic())
.map(::jdbcCall)
.flatMap(str -> asyncHttpCall(str))
.publishOn(Schedulers.boundedElastic())
.map(::jdbcCall)
.toFuture();
}
As you can see, you can continue using async calls too. Just make sure you're not blocking non-blocking schedulers (in that example, I use publishOn again after flatMap because asyncHttpCall may complete from non-blocking scheduler)
Thanks to Telegram open source code, I'm trying to learn a good behaviour for my apps as Telegram guys do.
Like here: http://telegram.org/resources/telegram_iphone.src.zip
Interesting 2 objects I noticed major use in objects ASWatcher & ASHandle.
I'm trying to understand what exactly it's suppose to do.
I tried to put a few logs and follow the methods.
So far what I understood is those object observe items on UIViewController and track changes, and return back the object and the new value from the object.
I don't know how I'm right or wrong but it looks to me a very interesting approach or objects to learn.
I'll glad to help here if someone does know.
I know it's too late answer But for one whom might still interested:
ASWatcher and ASHandle are implementation of Actor Model which is:
A conceptual model to deal with concurrent
computation. It defines some general rules for how the system’s
components should behave and interact with each other
The most famous language that uses this model is probably Erlang
An actor is the primitive unit of computation. It’s the thing that receives a message and do some kind of computation based on it
The idea is very similar to what we have in object-oriented languages: An object receives a message (a method call) and does something depending on which message it receives (which method we are calling).
The main difference is that actors are completely isolated from each other and they will never share memory. It’s also worth noting that an actor can maintain a private state that can never be changed directly by another actor
It’s important to understand that, although multiple actors can run at the same time, an actor will process a given message sequentially. This means that if you send 3 messages to the same actor, it will just execute one at a time. To have these 3 messages being executed concurrently, you need to create 3 actors and send one message to each.
as you can see in telegram code in ASHandle :
- (void)reset
{
TG_SYNCHRONIZED_BEGIN(_delegate);
_delegate = nil;
TG_SYNCHRONIZED_END(_delegate);
}
- (bool)hasDelegate
{
bool result = false;
TG_SYNCHRONIZED_BEGIN(_delegate);
result = _delegate != nil;
TG_SYNCHRONIZED_END(_delegate);
return result;
}
Further reading : Wikipedia and this
Examples:
Asynchronous method with its own dispatching:
// Library
func asyncAPI(callback: Result -> Void) {
dispatch_async(self.queue) {
...
callback(result)
}
}
// Caller
asyncAPI() { result in
...
}
Synchronous method with exposed dispatch queue:
// Library
func syncAPI() -> Result {
assert(isRunningOnCorrectQueue())
...
return result
}
// Caller
dispatch_async(api.queue) {
let result = api.syncAPI()
...
}
These two examples behave the same but I am looking to learn whether one of these ends up complicating a larget codebase more than the other, especially when there is a lot of asynchrony.
I would argue against both of the patterns you propose.
For the first pattern (where the API manages it's own backgrounding) I see little or no benefit to doing it this way, as opposed to leaving it to the caller. If you want to use a private, serial queue to protect data (or any other sort of critical section) internal to your API, that's fine, but that queue should be private, and it should specifically not target any public, non-global-concurrent queue (Note: it should especially not target the main queue). Ideally, the primary implementation of your API would also take a second parameter, so callers can specify on which queue to invoke the callback. (People can work around the lack of such a parameter by passing a callback block that re-dispatches to their desired queue, but I think that's clunkier than having an extra, optional parameter.) This puts the API consumer in complete control of the concurrency, while preserving your freedom to use queues internally to protect state.
As to the second approach, it's my opinion that we all should avoid creating new synchronous, blocking API. When you provide a synchronous, blocking API and don't provide a callback-based version, that means that you have denied consumers of your API any opportunity to avoid blocking. When you only provide synchronous, blocking API, then if someone wants to call your API in the background, at least one thread (in addition to any additional threads that your API consumes behind the scenes) will be consumed from the finite number of threads available to each process. (In the worst case this can lead to starvation conditions that are effectively deadlocks.)
Another red flag with this second example is that it vends a queue; Any time an API vends a queue, something is amiss. As mentioned, if you want to use a private serial queue to protect state or other critical sections internal to your API, go for it, but don't expose that queue to the outside world. If nothing else, it unnecessarily exposes details of your implementation. In looking at the system framework headers, I couldn't find a single case where a dispatch_queue_t was vended where it wasn't immediately obvious that the intent was for the API consumer to push in the queue, and not read it out.
It's also worth mentioning that these patterns are problematic regardless of whether your workload is CPU-bound or IO-bound. If it's CPU-bound, then not managing your own dispatch gives consumers of the API explicit control over how this CPU work is executed. If your workload is IO-bound, then you should use the OS- and libdispatch-provided asynchronous IO mechanisms (dispatch_io, dispatch_sources, kevent, etc) to avoid consuming a thread (or more than one) for the duration of your work.
Another answer here implied that forcing consumers to manage their own concurrency leads to "boilerplate" code. If you feel that the burden of API consumers potentially having to wrap calls to your API with dispatch_async is too great, then feel free to provide a convenience overload that dispatches to the default global concurrent queue, but please always leave the version that allows API consumers the ability to explicitly manage their own concurrency.
If, on the other hand, all this is internal to the implementation, and not part of the public API, then do whatever is most expedient, knowing that you can refactor the implementation behind the public API any time in the future.
As you said, the 2 generally accomplish the same thing but the first is more preferable in most scenarios. There are several benefits to using the first method.
The API is simpler. You simply call the method and provide code for the callback block.
Less boilerplate code, No typing dispatch_async every time you want to call it as it is just included in the method itself.
Less room for bugs/errors. By wrapping the asynchronous logic inside the method itself, you ensure that it is called on the right queue internally without the caller having to worry about any of that.
Touching on the last point, you also have finer control over the queue itself. Let's say you are trying to perform certain tasks on a particular queue. It is way simpler to simply wrap the code in a GCD call on that queue a single time rather than having to remember to reuse that same queue every time you want to call the method.
Apologies if this question is deemed inappropriate for SO, but I was wondering if any of you know if there's a significant performance difference between streaming futures via asStream and consuming futures normally via then. Would you expect a general performance difference between the following two operations?
Operation 1
expensiveOperation().asStream().listen((res) {
doSomething(res);
});
Operation 2
expensiveOperation().then((res) {
doSomething(res);
});
asStream just allocates a wrapper object and forwards the future's outcome to the stream. All in all you probably won't notice the difference.
If you happen to have a case where you actually measure a slow-down, file a bug on http://dartbug.com. There are still ways to make the wrapper cheaper.