I am using Flux.create() to bridge from listener type API (custom HTTP server) to Flux stream. Very simple code, basically:
Flux.create(listener::setSink, FluxSink.OverflowStrategy.BUFFER)
... bunch of operators
.subscribe();
And inside listener:
sink.next(item);
Everything is working fine, but after some troubleshooting I found out that BufferAsyncSink that is created by FluxCreate (for BUFFER strategy) is basically serializing downstream processing.
#Override
public FluxSink<T> next(T t) {
queue.offer(t);
drain();
return this;
}
This adds item to the queue, and then proceeds to drain the queue.
However, this code only allows single thread to drain the queue.
void drain() {
if (WIP.getAndIncrement(this) != 0) {
return;
}
I don't understand why is this limitation is here. Why can't multiple threads drain the queue? In fact, why is the queue even needed?
I know I can offload next item to another thread pool with publishOn/subscribeOn, but we already have a thread pool for our HTTP server and it would be more optimal if thread that received HTTP request would continue processing flux until its down. We have backpressure built into our HTTP server so we don't need Flux to handle it.
I looked all over Reactor documentation and can't find another way to create Flux that would not serialize the stream.
Anybody have any suggestions?
Related
Folks, is it possible to obtain currently used Scheduler within an operator?
The problem that I have is that Mono.fromFuture() is being executed on a native thread (AWS CRT Http Client in my case). As result all subsequent operators are also executed on that thread. And later code wants to obtain class loader context that is obviously null. I realize that I can call .publishOn(originalScheduler) after .fromFuture() but I don't know what scheduler is used to materialize Mono returned by my function.
Is there elegant way to deal with this?
fun myFunction(): Mono<String> {
return Mono.just("example")
.flatMap { value ->
Mono.fromFuture {
// invocation of 3rd party library that executes Future on the thread created in native code.
}
}
.map {
val resource = Thread.currentThread().getContextClassLoader().getResources("META-INF/services/blah_blah");
// NullPointerException because Thread.currentThread().getContextClassLoader() returns NULL
resource.asSequence().first().toString()
}
}
It is not possible, because there's no guarantee that there is a Scheduler at all.
The place where the subscription is made and the data starts flowing could simply be a Thread. There is no mechanism in Java that allows an external actor to submit a task to an arbitrary thread (you have to provide the Runnable at Thread construction).
So no, there's no way of "returning to the previous Scheduler".
Usually, this shouldn't be an issue at all. If your your code is reactive it should also be non-blocking and thus able to "share" whichever thread it currently runs on with other computations.
If your code is blocking, it should off-load the work to a blocking-compatible Scheduler anyway, which you should explicitly chose. Typically: publishOn(Schedulers.boundedElastic()). This is also true for CPU-intensive tasks btw.
In short
We have a mobile app that streams fairly high volumes of data to and from a server through various bidirectional streams. The streams need to be closed on occasion (for example when the app is backgrounded). They are then reopened as needed. Sometimes when this happens, something goes wrong:
From what I can tell, the stream is up and running on the device's side (the status of both the GRPCProtocall and the GRXWriter involved is either started or paused)
The device sends data on the stream fine (the server receives the data)
The server seems to send data back to the device fine (the server's Stream.Send calls return as successful)
On the device, the result handler for data received on the stream is never called
More detail
Our code is heavily simplified below, but this should hopefully provide enough detail to indicate what we're doing. A bidirection stream is managed by a Switch class:
class Switch {
/** The protocall over which we send and receive data */
var protocall: GRPCProtoCall?
/** The writer object that writes data to the protocall. */
var writer: GRXBufferedPipe?
/** A static GRPCProtoService as per the .proto */
static let service = APPDataService(host: Settings.grpcHost)
/** A response handler. APPData is the datatype defined by the .proto. */
func rpcResponse(done: Bool, response: APPData?, error: Error?) {
NSLog("Response received")
// Handle response...
}
func start() {
// Create a (new) instance of the writer
// (A writer cannot be used on multiple protocalls)
self.writer = GRXBufferedPipe()
// Setup the protocall
self.protocall = Switch.service.rpcToStream(withRequestWriter: self.writer!, eventHandler: self.rpcRespose(done:response:error:))
// Start the stream
self.protocall.start()
}
func stop() {
// Stop the writer if it is started.
if self.writer.state == .started || self.writer.state == .paused {
self.writer.finishWithError(nil)
}
// Stop the proto call if it is started
if self.protocall?.state == .started || self.protocall?.state == .paused {
protocall?.cancel()
}
self.protocall = nil
}
private var needsRestart: Bool {
if let protocall = self.protocall {
if protocall.state == .notStarted || protocall.state == .finished {
// protocall exists, but isn't running.
return true
} else if writer.state == .notStarted || writer.state == .finished {
// writer isn't running
return true
} else {
// protocall and writer are running
return false
}
} else {
// protocall doesn't exist.
return true
}
}
func restartIfNeeded() {
guard self.needsRestart else { return }
self.stop()
self.start()
}
func write(data: APPData) {
self.writer.writeValue(data)
}
}
Like I said, heavily simplified, but it shows how we start, stop, and restart streams, and how we check whether a stream is healthy.
When the app is backgrounded, we call stop(). When it is foregrounded and we need the stream again, we call start(). And we periodically call restartIfNeeded(), eg. when screens that use the stream come into view.
As I mentioned above, what happens occasionally is that our response handler (rpcResponse) stops getting called when server writes data to the stream. The stream appears to be healthy (server receives the data we write to it, and protocall.state is neither .notStarted nor .finished). But not even the log on the first line of the response handler is executed.
First question: Are we managing the streams correctly, or is our way of stopping and restarting streams prone to errors? If so, what is the correct way of doing something like this?
Second question: How do we debug this? Everything we could think of that we can query for a status tells us that the stream is up and running, but it feels like the objc gRPC library keeps a lot of its mechanics hidden from us. Is there a way to see whether responses from server may do reach us, but fail to trigger our response handler?
Third question: As per the code above, we use the GRXBufferedPipe provided by the library. Its documentation advises against using it in production because it doesn't have a push-back mechanism. To our understanding, the writer is only used to feed data to the gRPC core in a synchronised, one-at-a-time fashion, and since server receives data from us fine, we don't think this is an issue. Are we wrong though? Is the writer also involved in feeding data received from server to our response handler? I.e. if the writer broke due to overload, could that manifest as a problem reading data from the stream, rather than writing to it?
UPDATE: Over a year after asking this, we have finally found a deadlock bug in our server-side code that was causing this behaviour on client-side. The streams appeared to hang because no communication sent by the client was handled by server, and vice-versa, but the streams were actually alive and well. The accepted answer provides good advice for how to manage these bi-directional streams, which I believe is still valuable (it helped us a lot!). But the issue was actually due to a programming error.
Also, for anyone running into this type of issue, it might be worth investigating whether you're experiencing this known issue where a channel gets silently dropped when iOS changes its network. This readme provides instructions for using Apple's CFStream API rather than TCP sockets as a possible fix for that issue.
First question: Are we managing the streams correctly, or is our way of stopping and restarting streams prone to errors? If so, what is the correct way of doing something like this?
From what I can tell by looking at your code, the start() function seems to be right. In the stop() function, you do not need to call cancel() of self.protocall; the call will be finished with the previous self.writer.finishWithError(nil).
needsrestart() is where it gets a bit messy. First, you are not supposed to poll/set the state of protocall yourself. That state is altered by itself. Second, setting those state does not close your stream. It only pause a writer, and if app is in background, pausing a writer is like a no-op. If you want to close a stream, you should use finishWithError to terminate this call, and maybe start a new call later when needed.
Second question: How do we debug this?
One way is to turn on gRPC log (GRPC_TRACE and GRPC_VERBOSITY). Another way is to set breakpoint at here where gRPC objc library receives a gRPC message from the server.
Third question: Is the writer also involved in feeding data received from server to our response handler?
No. If you create a buffered pipe and feed that as request of your call, it only feed data to be sent to server. The receiving path is handled by another writer (which is in fact your protocall object).
I don't see where the usage of GRXBufferedPipe in production is discouraged. The known drawback about this utility is that if you pause the writer but keep writing data to it with writeWithValue, you end up buffering a lot of data without being able to flush them, which may cause memory issue.
I am working with an external device that I receive data from. I want to handle its data read/write queue asynchronously, in a thread.
I've got it mostly working: There is a class that simply manages the two streams, using the NSStreamDelegate to respond to incoming data, as well as responding to NSStreamEventHasSpaceAvailable for sending out data that's waiting in a buffer after having failed to be sent earlier.
This class, let's call it SerialIOStream, does not know about threads or GCD queues. Instead, its user, let's call it DeviceCommunicator, uses a GCD queue in which it initializes the SerialIOStream class (which in turn creates and opens the streams, scheduling them in the current runloop):
ioQueue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT,0);
dispatch_async(ioQueue, ^{
ioStreams = [[SerialIOStream alloc] initWithPath:[#"/dev/tty.mydevice"]];
[[NSRunLoop currentRunLoop] run];
});
That way, the SerialIOStreams stream:handleEvent: method runs in that GCD queue, apparently.
However, this causes some problems. I believe I run into concurrency issues, up to getting crashes, mainly at the point of feeding pending data to the output stream. There's a critical part in the code where I pass the buffered output data to the stream, then see how much data was actually accepted into the stream, and then removing that part from my buffer:
NSInteger n = self.dataToWrite.length;
if (n > 0 && stream.hasSpaceAvailable) {
NSInteger bytesWritten = [stream write:self.dataToWrite.bytes maxLength:n];
if (bytesWritten > 0) {
[self.dataToWrite replaceBytesInRange:NSMakeRange(0, bytesWritten) withBytes:NULL length:0];
}
}
The above code can get called from two places:
From the user (DeviceCommunicator)
From the local stream:handleEvent: method, after being told that there's space in the output stream.
Those may be (well, surely are) running in separate thread, and therefore I need to make sure they do not run concurrently this code.
I thought I'd solve this by using the following code in DeviceCommunicator when sending new data out:
dispatch_async (ioQueue, ^{
[ioStreams writeData:data];
});
(writeData adds the data to dataToWrite, see above, and then runs the above code that sends it to the stream.)
However, that doesn't work, apparently because ioQueue is a concurrent queue, which may decide to use any available thread, and therefore lead to a race condition when writeData get called by the DeviceCommunicator while there's also a call to it from stream:handleEvent:, on separate threads.
So, I guess I am mixing expectations of threads (which I'm a bit more familiar with) into my apparent misunderstandings with GCD queues.
How do I solve this properly?
I could add an NSLock, protecting the writeData method with it, and I believe that would solve the issue in that place. But I am not so sure that that's how GCD is supposed to be used - I get the impression that'd be a cludge.
Shall I rather make a separate class, using its own serial queue, for accessing and modifying the dataToWrite buffer, perhaps?
I am still trying to grasp the patterns that are involved with this. Somehow, it looks like a classic producer / consumer pattern, but on two levels, and I'm not doing this right.
Long story, short: Don't cross the streams! (haha)
NSStream is a RunLoop-based abstraction (which is to say that it intends to do its work cooperatively on an NSRunLoop, an approach which pre-dates GCD). If you're primarily using GCD to support concurrency in the rest of your code, then NSStream is not an ideal choice for doing I/O. GCD provides its own API for managing I/O. See the section entitled "Managing Dispatch I/O" on this page.
If you want to continue to use NSStream, you can either do so by scheduling your NSStreams on the main thread RunLoop or you can start a dedicated background thread, schedule it on a RunLoop over there, and then marshal your data back and forth between that thread and your GCD queues. (...but don't do that; just bite the bullet and use dispatch_io.)
I'm playing with a tiny web server and I'm implementing one version using the async package, and one synchronous version executing each request in a separate isolate. I would like to simply pipe a file stream to the HttpResponse, but I can't do that synchronously. And I can't find a way to wait for neither the Stream nor a Future synchronously. I'm now using a RandomAccessFile instead which works, but it becomes messier.
One solution would be to execute a periodical timer to check if the future is completed (by setting a boolean or similar), but that is most definitely not something I want to use.
Is there a way to wait synchronously for a Future and a Stream? If not, why?
For future visitors coming here simply wanting to perform some task after a Future or Stream completes, use await and await for inside an async method.
Future
final myInt = await getFutureInt();
Stream
int mySum = 0;
await for (int someInt in myIntStream) {
mySum += someInt;
}
Note
This may be technically different than performing a synchronous task, but it achieves the goal of completing one task before doing another one.
AFAIK there isn't a way to wait synchronously for a Future or a Stream. Why? Because these are asynchronous pretty much definitionally, and as you are discovering, the APIs are designed with asynchronous behavior in mind.
There are a couple of Future constructors, Future.value() and Future.sync(), that execute immediately, but I don't think these are probably what you have in mind.
I've seen 2 flavors of working with asyncronous operations in mvc controllers.
First:
public void GetNewsAsync()
{
AsyncManager.OutstandingOperations.Increment();
using (ManualResetEvent mre = new ManualResetEvent(false))
{
//Perform the actual operation in a worker thread
ThreadPool.QueueUserWorkItem((object _mre) =>
{
//do some work in GetFeed that takes a long time
var feed = GetFeed();
AsyncManager.Parameters["Feed"] = feed;
AsyncManager.OutstandingOperations.Decrement();
mre.Set();
}, mre);
//Wait for the worker thread to finish
mre.WaitOne(TimeSpan.FromSeconds(SomeNumberOfSecondsToWait));
}
}
Second:
public void GetNewsAsync()
{
AsyncManager.OutstandingOperations.Increment();
//Perform the actual operation in a worker thread
ThreadPool.QueueUserWorkItem((object x) =>
{
//do some work in GetFeed that takes a long time
var feed = GetFeed();
AsyncManager.Parameters["Feed"] = feed;
AsyncManager.OutstandingOperations.Decrement();
}, null);
}
The first blocks GetNewsAsync for SomeNumberOfSecondsToWait, the second does not. Both perform the work inside a of a worker thread and the results passed to GetNewsCompleted.
So my question is, which is the correct way to handle an Ajax call to GetNews; Wait, or don't wait?
I don't know where did you see the first example but that's a total anti-pattern that completely defeats the purpose of an asynchronous controller. The whole point of an asynchronous operation is to execute asynchronously and free the main thread as fast as possible.
This being said if GetFeed is a blocking call (which is what its name supposes it is) you get strictly 0 benefit from an asyncrhonous controller so the second example is also wrong for me. You could use a standard synchronous controller action in this case. With the second example you draw a thread from the pool and instead of blocking inside the main thread you block inside the other thread so the net effect is almost the same (in reality it's worse) if you had used a standard synchronous controller action.
So both those examples will bring more overhead than any benefit.
Where asynchronous controllers are useful is when you have some I/O intensive API such as a database or web service call where you could take advantage of IO Completion Ports. The following article provides a good example of this scenario. The newsService used there is providing real asynchronous methods and there is no blocking during the I/O network call. No worker thread being jeopardized.
I would also recommend you reading the following article. Even if it is for classic WebForms it still contains some very useful information.