I want to try blockOptional in Project Reactor. According to the description, it is used to
Subscribe to this Mono and block indefinitely until a next signal is
received or the Mono completes empty
I tried the following
.
Mono<Void> triggerSignal() {
Signal.next("signal");
}
triggerSignal()
.delayElement(Duration.ofSeconds(30))
.blockOptional();
System.out.println("Outside chain");
I expect to see "Outside chain" printed before the 30s delay. However I have to wait 30s.
Isn't a signal received and the block should stop when a signal received? Is it the correct way to send a signal?
triggerSignal() does indeed produce a signal immediately.
However, the next item in your chain is a delayElement() call which will always delay the element emitted by 30s (before it reaches the next operator in the reactive chain); hence blockOptional() will never "see" the signal before 30s has elapsed.
It sounds like you want to, instead, block for up to 30s while you wait for the signal. If that's the case, then you can pass a duration to blockOptional() instead of delaying the element, such as:
triggerSignal()
.blockOptional(Duration.ofSeconds(30));
Related
Can I wake up task before vTaskDelay expires?
I have code like this:
In the task (hnd_uart_task) code:
transmit_frame();
vTaskDelay(100); // task should wait 100 ticks or be woken up by uart ISR
parse_response();
UART Interrupt:
// if byte was received
BaseType_t xYieldRequired = xTaskResumeFromISR(hnd_uart_task);
portYIELD_FROM_ISR(xYieldRequired);
Instead of using vTaskDelay(), you can use task notifications with timeout.
USART Interrupt:
// if byte was received
BaseType_t xHigherPriorityTaskWoken = pdFALSE;
vTaskNotifyGiveFromISR(hnd_uart_task, &xHigherPriorityTaskWoken);
portYIELD_FROM_ISR(xHigherPriorityTaskWoken);
Task Code:
transmit_frame();
ulTaskNotifyTake(pdTRUE, 100);
parse_response();
ulTaskNotifyTake(pdTRUE, 100) returns when a task notification is received from the ISR, or when 100 tick timeout period elapses.
But as #Jacek Ślimok pointed out, a byte-by-byte parsing may not be good idea. The exact method depends on the protocol used. But in general, you set up your DMA or interrupts to fill a reception buffer with incoming bytes. For example, when parsing Modbus frames, you can use idle line detection hardware interrupt and give notification to a task which parses the RX buffer.
No, because that's not what vTaskDelay was meant to be used for.
The closest solution to yours would be to create a semaphore that you attempt to take inside the task with a 100ms delay and that you give from ISR. This way the task will block for a maximum of 100ms waiting for semaphore to be given, after which it'll unblock and resume execution anyway. If it's given earlier, it'll unblock earlier, which is what I assume you want.
However, from what you've written I assume you want to achieve something like the following:
Send some data over UART
Wait for response
As soon as response is received, do something with the response
In this case, doing both blocking and parsing in the same task is going to be hard (you reaaaalllly don't want to do any sort of parsing inside of your ISR). I'd therefore recommend the following "layout" of two tasks, an interrupt and one shared smepahore between the two tasks:
"High level" task (let's call it ApplicationTask) that can do the following:
Construct whole frames and request them to be sent over UART (add them to some kind of queue). This "construction of whole frames" and sending them over to the other tasks would usually be wrapped into some functions.
Will block waiting for response
Will receive already parsed data (full frame or object/structure holding that parsed data)
"Byte level" task (let's call it ByteTask) that can do the following:
Has a queue for transmitted data (queue of frames or queue of raw bytes)
Has a queue for received data
"Pushes" data from "data to be transmitted" queue into UART
Parses UART data that appears in "received data" queue and gives semaphore to unblock the ApplicationTask
UART Interrupt:
Only transmits data that it's told to transmit by ByteTask
Pushes received data into ByteTask receive queue
Shared semaphore between ApplicationTask and ByteTask:
Whenever ApplicationTask wants to "wait to receive response", it attempts to take this semaphore. Maximum blocking time can be used as a "response receiving timeout".
Whenever ByteTask receives and parses enough data to decide that "yes, this is a complete response", it gives this semaphore.
This above is a super simple example of something that should be easy enough to scale to more tasks as you develop your application. You can get a lot more fancy than that (e.g. ByteTask handling multiple UARTs at the same time, have a pool of semaphores used for blocking for multiple tasks, do some more sophisticated message dispatching etc.), but the example above should hopefully give you a better idea of how something like this can be approached.
Our pipeline buffers events and does an external fetch (for enrichment) every 500 events. When a timer is fired, these events are then processed when a timer fires. Of course, when you have e.g. 503 events, there will be 3 events that were not enriched.
From experiments we learned that #FinishBundle is always called before the timer. It even seems the result of the bundle in committed before the timer executed (checkpointing?). If we could access the state from #FinishBundle and perform an enrichment on these last events, they would be part of the committed bundle.
I believe this would solve our exactly-once problem: currently the timer also needs to fetch and will do this again upon re-execution. When we could adjust the state in the #FinishBundle the fetch is no longer needed and when the timer re-executes it will start from the state.
Apparently, it is not possible to access the state from the #FinishBundle function, as the following gives errors:
#FinishBundle
public void finishBundle(FinishBundleContext c,
#StateId("buffer") BagState<AwesomeEvent> bufferState) {
LOG.info("--- FINISHBUNDLE CALLED ---");
// TODO: ENRICHMENT
LOG.info("--- FINISHBUNDLE DONE ---");
}
Am I doing something wrong or is there another way of accessing the state from this function?
Also, am I making the correct assessment about the timer behavior?
I am using the Serverless Framework to consume messages from SQS. Some of the messages sent to the queue do not get consumed. They go straight to the in-flight SQS status and from there to my dead letter queue. When I look at my log of the consumer, I can see that it consumed and successfully processed 9/10 messages. One is always not consumed and ends up in the dead letter queue. I am setting reservedConcurrency to 1 so that only one consumer can run at a time. The function consumer timeout is set to 30 seconds. This is the consumer code:
module.exports.mySQSConsumer = async (event, context) => {
context.callbackWaitsForEmptyEventLoop = false;
console.log(event.Records);
await new Promise((res, rej) => {
setTimeout(() => {
res();
}, 100);
});
console.log('DONE');
return true;
}
Consumer function configuration follow:
functions:
mySQSConsumer:
handler: handler.mySQSConsumer
timeout: 30 # seconds
reservedConcurrency: 1
events:
- sqs:
arn: arn:aws:sqs:us-east-1:xyz:my-test-queue
batchSize: 1
enabled: true
If I remove the await function, it will process all messages. If I increase the timeout to 200ms, even more messages will go to straight to the in-flight status and from there to the dead letter queue. This code is very simple. Any ideas why it's skipping some messages? The messages that don't get consumed don't even show up in the log using the first console.log() statement. They seem entirely ignored.
I figured out the problem. The SQS queue Lambda function event triggering works differently than I thought. The messages get pushed into the Lambda function, not pulled by it. I think this could be engineered better by AWS, but it's what it is.
The issue was the Default Visibility Timeout set to 30 seconds together with Reserved Concurrency set to 1. When the SQS queue gets filled up quickly with thousands of records, AWS starts pushing the messages to the Lambda function at a rate that is faster than the rate at which the single function instance can process them. AWS "assumes" that it can simply spin up more instances of the Lambda to keep up with the backpressure. However, the concurrency limit doesn't let it spin up more instances - the Lambda function is throttled. As a result, the function starts returning failure to the AWS backend for some messages, which will, consequently, hide the failed messages for 30 seconds (the default setting) and put them back into the queue after this period for reprocessing. Since there are so many records to process by the single instance, 30 seconds later, the Lambda function is still busy and can't process those messages again. So the situation repeats itself and the messages go back to invisibility for 30 seconds. This repeats total 3 times. After the third attempt, the messages go to the dead letter queue (we configured our SQS queue that way).
To resolve this issue, we increased the Default Visibility Timeout to 5 minutes. That's enough time for the Lambda function to process through most of the messages in the queue while the failed ones wait in invisibility. After 5 minutes, they get pushed back into the queue and since the Lambda function is no longer busy, it will process most of them. Some of them have to go to invisibility twice before being successfully processed.
So the remedy to this problem is either increasing the Default Invisibility Timeout like we did or increasing the number of failures necessary before a message goes to the dead letter queue.
I hope this helps someone.
let (signal, sink) = Signal<[CLBeacon], BeaconManagerError>.pipe()
When I call this because the user disabled the Bluetooth:
sendError(self.sink, error)
the Signal is interrupted and I don't receive more next nor interrupted events after enabling the Bluetooth back again. The Signal is broken.
How can I send error types to the observer without interrupting / breaking the Signal? I can't find in the RAC 4 documentation. Thanks!
By design, an error causes the signal to finish. The documentation says:
failures should only be used to represent “abnormal” termination. If
it is important to let operators (or consumers) finish their work, a
Next event describing the result might be more appropriate.
If you want to turn errors into Next events, you can use flatMapError operator as described here or use retry if you want to allow only several occurances of the error.
If you want different semantics than Next* (Error|Completed) I recommend encoding that in the type. You can use a Signal that can't fail, but which values can be either success or failure, by using Result:
Signal<Result<[CLBeacon], BeaconManagerError>, NoError>
That signal will emit no errors, but its Next events will be Result.Success<[CLBeacon]> or Result.Failure<BeaconManagerError>, **and the signal won't terminate when receiving a Result.Failure.
Suppose I use QTPs recovery scenario manager to set the playback synchronization timeout to 0. The handler would return with "continue with next statement".
I'd do that to make sure that any following playback statements don't waste their time waiting for the next non-existing/non-matching step before failing:
I have a lot of GUI tests that kind of get stuck because let's say if 10 controls are missing, their (consecutive) playback steps produce 10 timeout waits before failing. If the playback timeout is 30 seconds, I loose 10x30 seconds=5 minutes execution time while it really would be sufficient to wait for 30 seconds ONCE (because the app does not change anymore -- we waited a full timeout period already).
Now if I have 100 test cases (=action iterations), this possibly happens 100 times, wasting 500 minutes of my test exec time window.
That's why I come up with the idea of a recovery scenario function setting the timeout to 0 after/upon the first failed playback step. This would accelerate the speed while skipping the rightly-FAILED step, yet would not compromise the precision/reliability of identifying the next matching GUI context (which creates a PASSED step).
Then of course upon the next passed playback step, I would want to restore the original timeout value. How could I do that? This is my question.
One cannot define a recovery scenario function that is called for PASSED steps.
I am currently thinking about setting a method function for Reporter.ReportEvent, and "sniffing" for PASSED log entries there. I'd install that method function in the scenario recovery function which sets timeout to 0. Then, when the "sniffer" function senses a ReportEvent call with PASSED status during one of the following playback steps, I'd reset everything (i.e. restore the original timeout, and uninstall the method function). (I am 99% sure, however, that .Click and .Set methods do not call ReportEvent to write their result status...so this option might probably not work.)
Better ideas? This really bugs me.
It sounds to me like you tests aren't designed correctly, if you fail to find an object why do you continue?
One possible (non recovery scenario) solution would be to use RegisterUserFunc to override the methods you are using in order to do an obj.Exist(0) before running the required method.
Function MyClick(obj)
If obj.Exist(1) Then
obj.Click
Else
Reporter.ReportEvent micFail, "Click failed, no object", "Object does not exist"
End If
End Function
RegisterUserFunc "Link", "Click", "MyClick"
RegisterUserFunc "WebButton", "Click", "MyClick"
''# etc
If you have many controls of which some may be missing and you know that after 10 seconds you mentioned (when the first timeout occurs), nothing more will show up, then you can use the exists method with a timeout parameter.
Something like this:
timeout = 10
For Each control in controls
If control.exists(timeout) Then
do something with the control
Else
timeout = 0
End If
Next
Now only the first timeout will be 10 seconds. Each and every subsequent timeout in your collection of controls will have the timeout set to 0 which will save your time.