I'm looking for clear documentation and/or and example on how to setup a time-based trigger on a global window in Apache beam.
The purpose is to perform a count of the events since the last trigger fired, even when 0 events have been added since.
You can use timers and state, if you need to use global window and emit result even if there was no event since the last firing. I think it is not possible to do it with the built in triggers.
You can keep the count in a state and use a timer to emit the results periodically.
These two blog post explains the usage of timers and state:
Stateful processing with Apache Beam
Timely (and Stateful) Processing with Apache Beam
Related
Are there any guidelines or limitations for using stateful processing and timers with the Beam Dataflow runner (as of v2.1.0)? Things such as limitations on the size of state or frequency of updates etc.? The candidate streaming pipeline would use state and timers extensively for user session state, with Bigtable as durable storage.
Here is some general advice for your use case
Please aggregate multiple elements then set a timer.
Please don't create a timer per element, which would be excessive.
Try and aggregate state, instead of accumulating large amount of state. I.e. aggregate as a sum and count, instead of storing every number when trying to compute a mean.
Please consider session windows for this use case.
In dataflow, state is not supported for merging windows. It is for beam.
Please use state according to your access pattern, i.e. BagState for blind writes.
Here is an informative blog post with some more info on state "Stateful processing with Apache Beam."
I am trying to understand how I shall port my Java chess engine to dart.
So I have understood that I should use Isolates and/or Futures to run my engine in parallell with the GUI but how can I force the engine to terminate the search.
In java I just set some boolean that where shared between the engine thread and the gui thread.
You should send a message to the isolate, telling it to stop. You can simply do something like:
port.send('STOP');
To be clear, isolates and futures are two different things, and you use them differently.
Use an isolate when you want some code to truly run concurrently, in a separate "isolated memory heap". An isolate is like a mini program, running separately from your main program. You send isolates messages, and you can receive messages from isolates.
Use a future when you want to be notified when a value is available later. "Later" is defined as "a future tick in the event loop". Each isolate has its own event loop. It's important to understand that just asking a Future to run a function doesn't make the function run in parallel. It just puts the function onto the event loop to be run "later".
Answering the implied question 'how can I get a long running task in an isolate to cease running?' rather than more explicitly asked 'how can I cause an isolate to terminate, release it's resources and generally cease to be?'
Break the long running task up into smaller, shorter running units.
Execute each unit with a Future. Chain futures as appropriate.
Provide a flag that each unit should check before executing its logic. If the flag is set, bail.
Listen for a 'stop' message and set the flag if/when received.
Splitting the main processing task up into Futures allows processing of the stop message to get onto the event queue ahead of units of processing of the main task.
There is now iso.Isolate.kill()
WARNING: This method is experimental and not handled on every platform yet.
Is there any difference in the order of execution?
Or does the event queue/loop work differently in JavaScript than Dart?
DOM events are handled by the Blink. These events should therefore be handled the same way. In JavaScript there is no other event loop (afaik). For example, a common pattern for yielding execution is window.setTimeout(0, continuation).
In Dart we also have asynchronous events that are handled by dart:async. There we can distinguish between instants and cycles. An instant is composed of one or more cycles. An instant executes all its cycles until there is none left and then moves to the next instant. DOM events are at the same level as instants. That is, a DOM event will never be interleaved with cycles of the same instant. [^1] (This also means that piling up cycles within the same instant can starve the DOM.)
runAsync queues a new cycle.
Timer.run queues a new instant.
Futures and Streams use cycles to queue events that can be executed immediately. In the following example both thens will be scheduled within the same instant and therefore run before any DOM event has the chance to interfere.
var future = new Future.value(499);
future.then(print);
future.then(print);
There are other small differences between Dart and JavaScript: Dart does not have a minimal sleep time for Timer runs. In JavaScript window.setTimeout is not allowed to execute the computation before 5ms. (This is due to unfortunate historic circumstances). Dart doesn't have this restriction. [^2]
The VM's Timer functionality is not based on DOM, and has its own implementation. As of May 2013 the ordering of scheduled timer callbacks is not consistent between JavaScript (and thus dart2js) and the VM. (I'm not sure about Dartium, but I believe it uses the DOM's version and is thus similar to JavaScript).
We are currently discussing changes to the library to guarantee that Timer events are executed in the "correct" order and not just after the waiting-time has elapsed.
[^1] This is not correctly implemented. As of May 2013 ever asynchronous operation is built on top of Timer.
[^2] This, too, is not done yet (May 2013). I'm not sure about Dartium, but dart2js currently still uses window.setTimeout. It will eventually switch to newer primitives (on browsers that support it) that allow for more precise timeouts.
According to Erlang memsup documentation:
system_memory_high_watermark = float()
The threshold, as percentage of system memory, for how much system memory can be allocated before the corresponding alarm is set. Te default is 0.80 (80%)
What is the default behaviour when this alarm is set, in case the corresponding alarm_handler is not present.
Quoting from the alarm_handler documentation:
The alarm handler process is a gen_event event manager process which receives alarms in the system. This process is not intended to be a complete alarm handler. It defines a place to which alarms can be sent. One simple event handler is installed in the alarm handler at start-up, but users are encouraged to write and install their own handlers.
So a default alarm handler is installed, which I think simply does nothing at all. You can write your own and attach it to the alarm handler to react to the alarms sent.
If your question is instead what happens if SASL is not started when you start os_mon, it's not possible: see here
I am developing an application in which I must handle some lists (insertion, deletion). The problem is that the list can suffer alteration from a TTimer component and from a TServerSocket.
How can I protect the lists from being altered by the TTimer and the TServerSocket at the same time? Should I use threads?
Timer events are running in the application's main thread. I am not sure about TServerSocket events (may be a configuration option).
Generally: If both accesses are running in the main thread, you do not need a critical section because the other event can only fire when the first event has already finished (unless you call Application.ProcessMessages which you shouldn't anyway). A critical section wouldn't work in this scenario any way because it will only sync separate threads.
If they are running in different threads, you need some kind of synchronization. A critical section is one option, others include Mutexes, spin locks etc.
Try to use this or a variation thereof:
http://www.swissdelphicenter.ch/en/showcode.php?id=2167