Flux<Integer> shared = Flux.just(1, 2).share();
shared.subscribe(System.out::println);
shared.subscribe(System.out::println);
Since share() turns the flux into a hot one, I expect the first subscriber to get all values and the second one to get none, since the stream has completed at the time of subscription. But the output is the same as without share: 1 2 1 2, but it should be just 1 2.
When I replace share() with publish.autoconnect() it works as expected. Why is that?
The answer is simple, but it took me a while to figure it out.
share() is a shortcut for publish().refCount(). refCount() is like autoConnect() except for one additional feature: It disconnects when all subscribers have cancelled or - and that's the situation here - the stream has completed.
The first shared.subscribe creates a subscription (via share) to the original flux. All values are emitted immediately, the stream completes, the subscription is cancelled.
Since there is no subscription now, the second shared.subscribe again creates a subscription and the stream starts again, from the beginning.
autoConnect, however, does not cancel the subscription. If you use it instead of refCount the subscription to the original flux remains, but because the stream has completed, any additional subscriber won't receive any values.
Related
I have a rule chain in ThingsBoard that does a Create Alarm when temperature is outside threshold and does a Clear Alarm otherwise. I receive a message using a Telegram bot when these events occur. That all works fine.
However when the temperature is hovering around the threshold, I can get many notifications as it goes in and out of the threshold temperature. That is somewhat annoying.
I would like to have the Clear Alarm activity only trigger if it is more than 5 minutes (say) since the last Create Alarm event was triggered.
Any tips on how to achieve this?
I finally worked out how to do this.
I added some server attributes to my device that define the temperatures that trigger alarms. I have a rule chain for controlling these alarms with the following nodes:
Enrichment - originator attributes to add the relevant attributes into the metadata associated with this message
Filter - script to detect if the temperature is outside the expected range
Filter - script to detect if the delay period has expired since the last time the alarm was triggered
Action - create alarm when script detects that temp is out of range
Action - clear alarm when script detects that delay period has expired
Transformation - script to update last alarm time attribute
Action - save attributes to persist updated alarm time attribute
Transformation - script to create a message about alarm set or cleared
Rule chain to handle sending the message to a Telegram bot
As an example, here is the script for checking if the delay period has expired before clearing the alarm:
var alarmTime = Number(metadata.ss_lastWaterTempAlarmTime);
var alarmDelay = Number(metadata.ss_clearAlarmTimeDelay);
return metadata.ts >= alarmDelay + alarmTime;
ss is the prefix added for server side attributes that have been added to metadata.
You can see the complete rule chain json in my Aquamon repo.
I have some questions on EPOLL when using Edge Triggered and with EPOLLONESHOT.
The simplified sequence of statements are listed below. Actually multiple files are monitored by an Epoll Fd and a set is managed through a specific thread. The variable names used speak for themselves and are, of course, set. That part is omitted for the sake of brevity:
1. Create epollFd
epollFd = epoll_create1(EPOLL_CLOEXEC);
2. Create events to monitor
epollEventParam.events = EPOLLIN | EPOLLPRI | EPOLLERR | EPOLLHUP | EPOLLET | EPOLLONESHOT;
3. Add the FD to monitor and the events
epoll_ctl(epollFd, EPOLL_CTL_ADD, socketFd, &epollEventParam);
4. While loop with epoll_wait
while (1) {
noFdsEvented = epoll_wait(epollFd, epollEventsReported, maxEvents, -1);
/***************** REARM Here or after processing the events? ******/
epoll_ctl(epollFd, EPOLL_CTL_MOD, (int)epollEventsReported[i].data.fd, &epollEventParam);
/** if EPOLLIN, read until read returns EAGIN ***/
//Relevant code to read...
//After EAGAIN is returned, REARM Here instead of doing so earlier (above)?
/** if EPOLLOUT, write until write returns EAGIN ***/
//Relevant code to write...
//After EAGAIN is returned, REARM Here instead of doing so earlier (above)?
/*** If other events... process accordingly ***/
}
QUESTIONS:
When EPOLLONESHOT is used, when should EPOLL be REARMED? After the event is received or after the event is processed?
Elementary. When writing or reading, we keep track of the data point written/read until EAGAIN was returned or if partially read/written? Yes/No.
Initially EPOLLOUT is not set. When writing, when write returns EAGAIN, we add EPOLLOUT to the event
to be monitored. Yes/No?
When EPOLLOUT is triggered again for an FD, we continue writing from the point EAGAIN was last received
and do so until we get EAGAIN again. THen we REARM. Yes/No?
If we read partially and do not REARM, new data will continue to arrive but no event will be triggered.
So, if we read partially, we need to keep track of it and not rely only on the event handler to do read processing. Right?
I can't answer all of them, but I can try to answer a few.
Elementary. When writing or reading, we keep track of the data point written/read until EAGAIN was returned or if partially read/written? Yes/No.
For read/recv: you should handle all of the following scenarios
If you read/recv EAGAIN or EWOULDBLOCK. This indicates that there is nothing left to read/recv and you should break from the receive loop.
You receive 0 bytes. This indicates that the other side is no longer connected
You receive 0 > bytes. You successfully read/recv data and should read/recv again.
You dont necessarily need to keep track of the data as opposed to just ensure you handle the above 3 scenarios.
Initially EPOLLOUT is not set. When writing, when write returns EAGAIN, we add EPOLLOUT to the event to be monitored. Yes/No?
Yes...I believe that in the instance where EPOLLOUT is initially set, the same behaviour on the send() FD should occur as you described in the question...given that you will simply write to the FD regardless of the event notification and only stop once you get an EAGAIN/EWOULDBOCK error.
I'm not sure why, but for some reason when using the observable that is created via concat I will always get all values that are pushed from my list (works as intended). Where as with the normal subscribe it seems that some values never make it to those who have subscribed to the observable (only in certain conditions).
These are the two cases that I am using. Could anyone attempt to explain why in certain cases when subscribing to the second version not all values are received? Are they not equivalent? The intent here is to rewind the stream. What are some reasons that could explain why Case 2 fails while Case 1 does not.
Replay here is just a list of the ongoing stream.
Case 1.
let observable =
Observable.Create(fun (o:IObserver<'a>) ->
let next b =
for v in replay do
o.OnNext(v.Head)
o.OnNext(b)
o.OnCompleted()
someOtherObs.Subscribe(next, o.OnError, o.OnCompleted))
let toReturn = observable.Concat(someOtherObs).Publish().RefCount()
Case 2.
let toReturn =
Observable.Create(fun (o:IObserver<'a>) ->
for v in replay do
o.OnNext(v.Head)
someOtherObs.Subscribe(o)
).Publish().RefCount()
Caveat! I don't use F# regularly enough to be 100% comfortable with the syntax, but I think I see what's going on.
That said, both of these cases look odd to me and it greatly depends on how someOtherObs is implemented, and where (in terms of threads) things are running.
Case 1 Analysis
You apply concat to a source stream which appears to work like this:
It subscribes to someOtherObs, and in response to the first event (a) it pushes the elements of replay to the observer.
Then it sends event (a) to the observer.
Then it completes. At this point the stream is finished and no further events are sent.
In the event that someOtherObs is empty or just has a single error, this will be propagated to the observer instead.
Now, when this stream completes, someOtherObs is concatenated on to it. What happens now is a little unpreditcable - if someOtherObs is cold, then the first event would be sent a second time, if someOtherObs is hot, then the first event is not resent, but there's a potential race condition around which event of the remainder will go next which depends on how someOtherObs is implemented. You could easily miss events if it's hot.
Case 2 Analysis
You replay all the replay events, and then send all the events of someOtherObs - but again there's a race condition if someOtherObs is hot because you only subscribe after pushing replay, and so might miss some events.
Comments
In either case, it seems messy to me.
This looks like an attempt to do a merge of a state of the world (sotw) and a live stream. In this case, you need to subscribe to the live stream first, and cache any events while you then acquire and push the sotw events. Once sotw is pushed, you push the cached events - being careful to de-dupe events that may been read in the sotw - until you are caught up with live at which point you can just pass live events though.
You can often get away with naive implementations that flush the live cache in an OnNext handler of the live stream subscription, effectively blocking the source while you flush - but you run the risk of applying too much back pressure to the live source if you have a large history and/or a fast moving live stream.
Some considerations for you to think on that will hopefully set you on the right path.
For reference, here is an extremely naïve and simplistic C# implementation I knocked up that compiles in LINQPad with rx-main nuget package. Production ready implementations I have done in the past can get quite complex:
void Main()
{
// asynchronously produce a list from 1 to 10
Func<Task<List<int>>> sotw =
() => Task<List<int>>.Run(() => Enumerable.Range(1, 10).ToList());
// a stream of 5 to 15
var live = Observable.Range(5, 10);
// outputs 1 to 15
live.MergeSotwWithLive(sotw).Subscribe(Console.WriteLine);
}
// Define other methods and classes here
public static class ObservableExtensions
{
public static IObservable<TSource> MergeSotwWithLive<TSource>(
this IObservable<TSource> live,
Func<Task<List<TSource>>> sotwFactory)
{
return Observable.Create<TSource>(async o =>
{
// Naïve indefinite caching, no error checking anywhere
var liveReplay = new ReplaySubject<TSource>();
live.Subscribe(liveReplay);
// No error checking, no timeout, no cancellation support
var sotw = await sotwFactory();
foreach(var evt in sotw)
{
o.OnNext(evt);
}
// note naive disposal
// and extremely naive de-duping (it really needs to compare
// on some unique id)
// we are only supporting disposal once the sotw is sent
return liveReplay.Where(evt => !sotw.Any(s => s.Equals(evt)))
.Subscribe(o);
});
}
}
Is it possible to get the last added value or previous values from a StreamController.broadcast / Stream.
Basically I have some subscriptions to streams like on('data') and on('ready') and I want for example that if someone listens to on('ready') after the event was triggered they can get the last value from the broadcast stream, eg. 'I\'m ready'.
Something like:
StreamController x = StreamController.broadcast();
first = x.first.then(doSomething);
x.add('oneOffEvent');
// first ~ 'oneOffEvent'
x.close()
other = x.last.then(doSomethingElse);
// other ~ 'oneOffEvent'
Is this even possible without caching the value somewhere else?
Broadcast Streams, by design, do not wait for subscriptions to fire events.
You can, however, do both a first and last subscription before adding events.
Add many subscriptions. That's the upside of broadcast subscriptions.
Suppose I use QTPs recovery scenario manager to set the playback synchronization timeout to 0. The handler would return with "continue with next statement".
I'd do that to make sure that any following playback statements don't waste their time waiting for the next non-existing/non-matching step before failing:
I have a lot of GUI tests that kind of get stuck because let's say if 10 controls are missing, their (consecutive) playback steps produce 10 timeout waits before failing. If the playback timeout is 30 seconds, I loose 10x30 seconds=5 minutes execution time while it really would be sufficient to wait for 30 seconds ONCE (because the app does not change anymore -- we waited a full timeout period already).
Now if I have 100 test cases (=action iterations), this possibly happens 100 times, wasting 500 minutes of my test exec time window.
That's why I come up with the idea of a recovery scenario function setting the timeout to 0 after/upon the first failed playback step. This would accelerate the speed while skipping the rightly-FAILED step, yet would not compromise the precision/reliability of identifying the next matching GUI context (which creates a PASSED step).
Then of course upon the next passed playback step, I would want to restore the original timeout value. How could I do that? This is my question.
One cannot define a recovery scenario function that is called for PASSED steps.
I am currently thinking about setting a method function for Reporter.ReportEvent, and "sniffing" for PASSED log entries there. I'd install that method function in the scenario recovery function which sets timeout to 0. Then, when the "sniffer" function senses a ReportEvent call with PASSED status during one of the following playback steps, I'd reset everything (i.e. restore the original timeout, and uninstall the method function). (I am 99% sure, however, that .Click and .Set methods do not call ReportEvent to write their result status...so this option might probably not work.)
Better ideas? This really bugs me.
It sounds to me like you tests aren't designed correctly, if you fail to find an object why do you continue?
One possible (non recovery scenario) solution would be to use RegisterUserFunc to override the methods you are using in order to do an obj.Exist(0) before running the required method.
Function MyClick(obj)
If obj.Exist(1) Then
obj.Click
Else
Reporter.ReportEvent micFail, "Click failed, no object", "Object does not exist"
End If
End Function
RegisterUserFunc "Link", "Click", "MyClick"
RegisterUserFunc "WebButton", "Click", "MyClick"
''# etc
If you have many controls of which some may be missing and you know that after 10 seconds you mentioned (when the first timeout occurs), nothing more will show up, then you can use the exists method with a timeout parameter.
Something like this:
timeout = 10
For Each control in controls
If control.exists(timeout) Then
do something with the control
Else
timeout = 0
End If
Next
Now only the first timeout will be 10 seconds. Each and every subsequent timeout in your collection of controls will have the timeout set to 0 which will save your time.