How can I invoke a method like
select udf(a,b) from Event.win:length_batch(5)
and print the window of size 5 out again. Say i send the following events:
runtime.sendEvent(new Event(1,2));
runtime.sendEvent(new Event(3,4));
runtime.sendEvent(new Event(4,2));
runtime.sendEvent(new Event(6,8));
runtime.sendEvent(new Event(4,6));
and print them out in the same order after batching them. This means that the method invoked udf(a,b) shall only be executed once per batch. I have used the updatelistener, but I have to invoke it as a method event though i can get the desired result using the listener.
Thanks.
I think you are looking for "subscribers".
Subscribers in doc
public void update(Event[] rows) {...}
Related
Suppose I need to send a large amount of data from the client to the server using python gRPC. And I want to continue the rest computation when sending the message out instead of blocking the code. Is there any way can implement this?
I will illustrate the question by an example using the modified code from the greeter_client.py
for i in range(5):
res=computation()
response = stub.SayHello(helloworld_pb2.HelloRequest(data=res))
I want the computation of the next iteration continue while sending the "res" of last iteration. To this end, I have tried the "async/await", which looks like this
async with aio.insecure_channel('localhost:50051') as channel:
stub = helloworld_pb2_grpc.GreeterStub(channel)
for j in range(5):
res=computation()
response = await stub.SayHello(helloworld_pb2.HelloRequest(data=res))
But the running time is actually the same with the version without async/await. The async/await does not work. I am wondering is there anything wrong in my codes or there are other ways?
Concurrency is different than parallelism. AsyncIO allows multiple coroutines to run on the same thread, but they are not actually computed at the same time. If the thread is given a CPU-heavy work like "computation()" in your snippet, it doesn't yield control back to the event loop, hence there won't be any progress on other coroutines.
Besides, in the snippet, the RPC depends on the result of "computation()", this meant the work will be serialized for each RPC. But we can still gain some concurrency from AsyncIO, by handing them over to the event loop with asyncio.gather():
async with aio.insecure_channel('localhost:50051') as channel:
stub = helloworld_pb2_grpc.GreeterStub(channel)
async def one_hello():
res=computation()
response = await stub.SayHello(helloworld_pb2.HelloRequest(data=res))
await asyncio.gather(*(one_hello() for _ in range(5)))
Using a dart stream for the 1st time. In the following code, while stepping through it in VSCODE, jumps from the tranform result to the final bracket. It appears to not execute any of the listen callbacks. How do I resolve this?
Future<void> addListsFileToDb(File file) async {
file.openRead().transform(utf8.decoder).listen(
(String value) => print(value),
onError: (error) => print(error),
onDone: () => print("done"));}
Streams (like futures) are asynchronous. When you call listen, you tell the stream to start working, and set up some callbacks to get called when the stream has a value available ... and then it's done.
Later, potentially much later depending on what the stream is doing, the stream will have generated its first value. At that point it will generate a data event, which will call the data callback/handler (here (String value) => print(value)).
Until then, program execution continues normally, so skipping to the end of the listen call immediately is correct and expected behavior, the listen call completes immediately and doesn't call any of the callbacks yet.
I need to create dependent API calls where the second one needs a value returned by the first one. First thing that comes to mind is using flatMap
ApiManager.shared
.createReport(report: report)
.flatMap { (report) -> Observable<Report> in
return ApiManager.shared.createReportStep(reportID: report.ID)
}
createReport returns Observable<Report> where after successfull call returns updated Report model(with ID), after that I need to call API to create report step, where report.ID is needed.
Everything looks and works fine with that code, but the problem comes when I need to do something after each of these steps(createReport and createReportStep). I placed code in onNext block, but it is called only once, after both of the steps are completed.
Is there a way to receive onNext signal after both steps? I could use something like this:
ApiManager.shared
.createReport(report: report)
.concat(ApiManager.shared.createReportStep(reportID: report.ID))
Which would emmit two signals like I want, but then again where do I get updated report.ID from to pass to createReportStep?
If you don't mind the time component and only need to have access to both report and what is returned by createReportStep(reportID:), you could go with creating a tuple in flatMap's block
ApiManager.shared
.createReport(report: report)
.flatMap { (report) -> Observable<Report> in
return ApiManager.shared.createReportStep(reportID: report.ID)
.map { (report, $0) }
}
The resulting observable would contain both results in a tuple.
If the time component is important, you could do the following
let report = ApiManager.shared
.createReport(report: report)
.share()
let reportStep = report.map { $0.ID }.flatMap(ApiManager.shared.createReportStep)
Observable.concat([report, reportStep])
Here, the important bit is the share call. It will ensure createReport performs its work only once, but you would have two next events as requested.
I need to make a series of database queries that each return a stream of results. Once all the information is collected and sent the 'complete' message needs to be send last. In my code 'sendCompleteMessageToClient' gets sent first.
Future.forEach(centerLdapNames.values, (center) {
db
.collection(center)
.find({'date': {'\$gte': json['from'], '\$lt': json['to']}})
.forEach(sendAppointmentToClient);
}).whenComplete(() => sendCompleteMessageToClient("all"));
How do I wait for all 'sendAppointmentToClient' to finish properly?
I guess you just miss the return of the future
Future.forEach(centerLdapNames.values, (center) {
return db // <== always return the returned future from calls to async functions to keep the chain connected
.collection(center)
.find({'date': {'\$gte': json['from'], '\$lt': json['to']}})
.forEach(sendAppointmentToClient);
}).whenComplete(() => sendCompleteMessageToClient("all"));
If you use wait these calls might be executed in parallel instead of one after the other
Future.wait(centerLdapNames.values.map((center) { ...}, eagerError: false)
.whenComplete((...))
i am trying to understand, how dart event loop works. I read the event loop article from the website The Event Loop and Dart and the author explain pretty good how event loop in dart works.
But what i don't understand is, how event get queue. For example
new Future(() => 21)
.then((v) => v*2)
.then((v) => print(v));
Will here dart gonna create three entries in the event queue or just one? I know, that the class Future is responsible for delay execution and when i create an object from it like
new Future(() => 21)
it will be just one entry in the event loop.
In this article, that i have mentioned above, i read about microtask. This microtask is going to execute before event queue, but i don't see any sense, why dart team implement this microtask? Maybe i need some example!
After some investigation it appears that the right answer is "they will be executed in the next event loop"
To test it you can write something like this:
import "dart:async";
void main() {
new Future(() {
scheduleMicrotask(()=>print("before loop 1"));
print("in event loop");
}).then((_) {
scheduleMicrotask(()=>print("before loop 2"));
print("in event loop");
}).then((_) {
scheduleMicrotask(()=>print("before loop 3"));
print("in event loop");
});
}
it should output:
in event loop
in event loop
in event loop
before loop 1
before loop 2
before loop 3
Although i'm not sure that you can't break this optimization. So only sure fact is that the firstFuture will complete first and the second - second.
EDIT: The strange part(Obsolete):
With code like this:
import "dart:async";
void main() {
new Future(() {
scheduleMicrotask(print("before loop 1"));
print("in event loop");
}).then((_) {
scheduleMicrotask(print("before loop 2"));
print("in event loop");
}).then((_) {
scheduleMicrotask(print("before loop 3"));
print("in event loop");
});
}
output is:
before loop 1
in event loop
before loop 2
in event loop
before loop 3
in event loop
Unhandled exception:
The null object does not have a method 'call'.
NoSuchMethodError: method not found: 'call'
Receiver: null
...
But with this:
import "dart:async";
void main() {
new Future(() {
scheduleMicrotask(()=>print("before loop 1"));
print("in event loop");
}).then((_) {
scheduleMicrotask(()=>print("before loop 2"));
print("in event loop");
}).then((_) {
scheduleMicrotask(()=>print("before loop 3"));
print("in event loop");
});
}
output is:
in event loop
in event loop
in event loop
before loop 1
before loop 2
before loop 3
EDIT2:
I think i got it. In the first(wrong version) scheduleMicrotask actually never got properly scheduled, but since Dart has eager argument execution it executes print() anyway. So what happens is that all Future getting executed in the next event loop and print all text.
That's why output is in call order:
before loop 1
in event loop
before loop 2
in event loop
before loop 3
in event loop
and not in the schedule order.
When you do:
new Future(() => 21)
.then((v) => v*2)
.then(print);
First you call the new Future(...) constructor.
This creates a Future object and schedules a Timer to execute the function you give as argument.
Then you call then. This creates a new future (call it future#2) and adds a listener on the first future. No events are scheduled.
Then you call then again. This creates yet another future (future#3) and adds a listener on the future#2. No events are scheduled.
Then the timer triggers, and the ()=>21 is executed, and the first future is completed with the value 21.
The listener on the first future is then executed immediately. That calls (v)=>v*2 with 21, and then completes future#2 with the value 42.
The listener on future#2 is then executed immediately. That calls print with 42, which prints 42 and returns null. This completes future#3 with the value null.
Future completion is currently done through a "propagate" function that tries to complete as many futures as possible, as long as their listeners are synchronous. That is why completing one future will immediately complete another, without any intermediate microtasks.
The microtask queue is to queue async execution but avoid returning to the main event loop before these microtasks are finished. You can ensure some related activities to be completed entirely even when executed async before other async tasks/events queued in the main queue are executed.
It seems the code executed from then like (v) => v*2 is again executed inside a Future because then always returns a Future.
from https://www.dartlang.org/articles/event-loop/
The microtask queue is necessary because event-handling code sometimes
needs to complete a task later, but before returning control to the
event loop. For example, when an observable object changes, it groups
several mutation changes together and reports them asychronously. The
microtask queue allows the observable object to report these mutation
changes before the DOM can show the inconsistent state.
How I interpret this description doesn't fit with the results in the tests in #Jare 's answer.
Just a little thing to add to previous answers. The 'Event Loop' article explains this behavior pretty well:
The function that you pass into Future’s then() method executes
immediately when the Future completes. (The function isn’t enqueued,
it’s just called.)
(https://www.dartlang.org/articles/event-loop/)
It means that in above examples there's always one event but many microtasks.