What source code in MINIX 3 handles timing and message passing. How code I modify this code to collect timing data about messages sent by whom to whom using soft and real-time timer.
Related
I'm writing a data pipeline using Reactor and Reactor Kafka and use spring's Message<> to save
the ReceiverOffset of ReceiverRecord in the headers, to be able to use ReciverOffset.acknowledge() when finish processing. I'm also using the out-of-order commit feature enabled.
When an event process fails I want to be able to log the error, write to another topic that represents all the failure events, and commit to the source topic. I'm currently solving that by returning Either<Message<Error>,Message<myPojo>> from each processing stage, that way the stream will not be stopped by exceptions and I'm able to save the original event headers and eventually commit the failed messages at the button of the pipeline.
The problem is that each step of the pipline gets Either<> as input and needs to filter the previous errors, apply the logic only on the Either.right and that could be cumbersome, especially when working with buffers and the operator get 'List<Either<>>' as input. So I would want to keep my business pipeline clean and get only Message<MyPojo> as input but also not missing errors that need to be handled.
I read that sending those message erros to other channel or stream is a soulution for that.
Spring Integration uses that pattern for error handling and I also read an article (link to article) that solves this problem in Akka Streams using 'divertTo()':
I couldn't find documentation or code examples of how to implement that in Reactor,
is there any way to use Spring Integration error channel with Reactor? or any other ideas to implement that?
Not familiar with reactor per se, but you can keep the stream linear. The trick, since Vavr's Either is right-biased is to use flatMap, which would take a function from Message<MyPojo> to Either<Message<Error>, Message<MyPojo>>. If the Either coming in is a right (i.e. a Message<MyPojo>, the function gets invoked and otherwise it just gets passed through.
// Apologies if the Java is atrocious... haven't written Java since pre-Java 8
incomingEither.flatMap(
myPojoMessage -> ... // compute a new Either
)
Presumably at some point you want to do something (publish to a dead-letter topic, tickle metrics, whatever) with the Message<Error> case, so for that, orElseRun will come in handy.
I have been trying to implement a wrapper library for the Linux interface to SCTP sockets, and I am not sure how to integrate the asynchronous style of errors (where they are delivered via events). All example code I have seen, if it deals with the errors at all, simply prints out the information related to the error when it is received, but inserting error-handling code there seems like it would be ineffective, because by that point all of the context related to the original message which was sent has been lost and only a 32-bit integer sinfo_context remains. It also seems that there is no way to directly tell when a given message has been acknowledged successfully by the remote peer, which would make it impossible to implement an approach which listens for errors after sending a message, because the context information for successfully-delivered messages could never be freed.
Is there a way to handle the errors related to a given sending operation as part of the call to a send function, or is there a different way to approach error handling for SCTP which does not lose the context of the error?
One solution which I have considered is using the SCTP_SENDER_DRY notification to tell when packets have been sent, however this requires sending only one packet at a time. Another idea is to use the peer's receiver window size together with the sinfo_cumtsn field of sctp_sndrcvinfo to calculate how much data has been acknowledged as fully received using the cumulative TSN, however there are a couple of disadvantages to this: first, it requires bookkeeping overhead to calculate a number of bytes received by the peer based on the cumulative TSN (especially if the peer's window size may change); second, it requires waiting until all earlier packets were received before reporting success, which seems to defeat the purpose of SCTP's multistreaming; and third, it seems like it would not work for unordered packets.
We are working on a billing invoice system. As a part of processing our request, we need to make an asynchronous call by placing a message in a queue. We work at 20TPS and have SLA for entire transaction of 12 sec. Occasionally, we have observed that when MQ server becomes very slow but still operational it's taking a lot of time just to write the message in the queue. We want to handle this scenario and have a system that throws an exception when it exceeds a predefined limit for writing the message in the queue.
In simple words, we want to implement a write timeout when there is a delay in writing a message in the queue. Any help is appreciated.
We are aware of mentioning timeout for receiving the response but we are unable to find any fix for mentioning timeout while writing the message in the queue.
We have found some suggestions on revalidating the destination. But in our case, we already know the destination is operational and our system becomes slow only during the response.
Most server framework/examples using sockets and I/O completion ports makes notifications in a way I couldn't completely figure out the purpose.
Upon read packets are processed, usually they are reordered to circumvent thread scheduling issues processing packets out of order no matter IOCP ensure a FIFO queue.
The problem is when a socket is closed gracefully or by an error. I saw in both situation, and again by the o.s. thread scheduler, the close notification may be sent to the application (i.e. http server using the framework) "before" the notification of data previously readed.
I think that the close notification should be queued in such way so the application receives it after previous reads.
Is there any intended use in most code I saw or my behavior may be correct depending on the situation?
What you suggest makes sense and I would imagine that any code that handles graceful close (a read returning 0 bytes) would do so by processing it after any proceeding successful read. Errors coming out of GetQueuedCompletionStatus(), such as connection reset errors, etc, are harder to integrate into the receive flow as they occur out of band as far as the receive data is concerned. Your question's a bit vague and depends very much on the code you're using and how you (or the people who wrote that code) want to handle these things. There is no single correct way, IMHO.
I'm designing a .net interface for sending and receiving a HL7 message and noticed on this forum theres a few people with this experience.
My question is.... Would anyone be able to share their experience on how long it could take to get a message response back from a hospital HL7 server. (Particularly when requesting patient demographics) - seconds / minutes / Hours?
My dilemma is do I design my application to make the user wait for the message to come back.
(Sorry if this is a little off topic, it’s still kinda programming related? – I searched the web for HL7 forums but got stuck so again if anyone knows of any please let me know )
cheers,
Jason
In my experience, you should receive an ACK or NAK back within a few seconds. The receiving application shouldn't do something like making you wait while it performs operations on the message. We have timeouts set to 30 seconds, and we almost never wait that long for a response.
This is quite dependent on the kind of HL7 message sent, typically messages like ADT's are sent as essentially updates to the server, and are acknowledged almost immediately if the hospital system is behaving well. This will result in a protocol level acknowledgement, which indicates that the peer has received the message but not necessarily processed it yet.
Typically, most systems will employ a broker or message queue in their integration engines so you get your ack almost immediately.
Other messages like lab request messages may actually send another non-ack message back which contains the information requested. These requests can take longer.
You can check with the peer you're communicating with to see what integration engine they are using, and if a queue sits on that end which would help ensure the response times are short.
In the HL7 integration tool I work on, we use queues for inbound data so we can responde immediately. And for our outbound connections, 10s timeouts are default, and seem to work fine for most of our customers.
When sending a Query type event in HL7, it could take a number of seconds to get the proper response back. You also need to code for the possibility that you will never get a response back, and the possibility that connected systems "don't do" queries.
Most HL7 nets that I have worked on, assume that all interested systems are listening for demographic updates at all times. Usually, receiving systems process these updates into a patient database that documents both the Person and Encounter (Stay) information on the fly.
In my location, my system usually gets about 10-20 thousand messages a day, most of which are patient demographic updates.
It depends if the response is generated automatically by a system or if the response is generated after an user does something on the system. For an automatic response it might take less than a second, depending of course on the processing that is done by the system and the current work load of that system. If the system is not too busy and processing is just a couple of queries and verification of some conditions, considering network delays, response time should be a few seconds or less.