Suppose I have two processor:
The first one P0
Call MPI_Send to send message A to p1;
Call MPI_Recv to receive B from p1;
The second one P1
Call MPI_Send to send message B to p0;
Call MPI_Recv to receive A from p0;
What will happen if the sizes of both message A and B exceed the system buffer?
One should never ever assume that such a thing as buffering of the standard send exists. The MPI standard explicitly warns against it in Section 3.5 Semantics of Point-to-Point Communication:
A program is "safe" if no message buffering is required for the program to complete. One can replace all sends in such program with synchronous sends, and the program will still run correctly. This conservative programming style provides the best portability, since program completion does not depend on the amount of buffer space available or on the communication protocol used.
MPI specifically addresses the use case in your question and provides the two send-receive calls MPI_Sendrecv and MPI_Sendrecv_replace. The former uses separate send and receive buffers that must not overlap, while the latter uses a single buffer. Both guarantee that no deadlock will occur if the send and receive parts are matched with a corresponding receive/send operation.
The code is wrong in any case.
It may work by the mercy of the MPI implementation / configuration / state. But generally, this is a deadlock. You shouldn't ponder about the buffering of standard blocking send calls for correctness. They are allowed to buffer exclusively for performance reasons, which can be surprising for beginners. Code that seemed to be working for small messages sizes suddenly deadlocked for larger message sizes, but actually the code was wrong all along, it just din't show.
Related
I am working on CANopen architecture and had three questions:
1- When the 'synchronous window' is closed until the next SYNC message, should we send the SDO message? Can we not send a message during this period?
2- Is it possible not to send the PDO message during the simultaneous window?
3- What is the answer that the slaves give in the SYNC message?
Disclaimer: I don't have exact answers but I just wanted to share my assumptions & thoughts.
CiA 301 doesn't mention the relation between synchronous window and SDOs. In normal operation after the initial configuration, one may assume that SDOs aren't present on the system, or at least they are rare compared to PDOs. Although not strictly necessary, SDOs are generally initiated by a device which has the master role, and that device also produces the SYNC messages (again, it's not strictly necessary but it's the usual/common implementation). So, the master device may adjust the timing of SDO requests according to the synchronous window.
Here is a quote from CiA 301:
If the synchronous window length expires all synchronous TPDOs may be
discarded and an EMCY message may be transmitted; all synchronous
RPDOs may be discarded until the next SYNC message is received.
Synchronous RPDO processing is resumed with the next SYNC message.
CiA 301 uses the word "may" (see the quote above). So I'm not sure if it's mandatory or not. In my opinion, it makes sense to follow the advice and abort synchronous TPDO transmissions after the synchronous window and send an EMCY message. Event-driven (non-synchronous) TPDOs can be sent within or after the synchronous window.
There is no direct response to SYNC messages. On SYNC reception, SYNC consumers (slaves) sample their inputs, drive their outputs according to the previous RPDOs, and start transmitting their TPDOs containing the previous samples (or the current ones? I'm not sure about this).
Synchronous windows are for specific PDO synchronization only. For hard real-time systems, data might be required to arrive within certain fixed time intervals - not too early, not too late. That is, it acts as a real-time deadline. If such features are enabled, you need to take that in consideration when doing the specific CANopen bus implementation.
For example if some SDO communication would occupy the bus so that the PDO can't meet its time window, that would be a problem. But this is easily solved by giving the PDO a lower COBID than the SDO, which should already be the case in most default device profile setups like "DS401 GPIO module". Other than that, you would have to make sure there is no ridiculous bus loads or that nodes hang up or get busy doing other things.
In systems with hard real-time requirements you probably don't want to allow any SDO communication during operational mode to begin with.
What is the answer that the slaves give in the SYNC message?
That question doesn't make any sense. You need to study what the SYNC message does and what it is for.
I have been trying to implement a wrapper library for the Linux interface to SCTP sockets, and I am not sure how to integrate the asynchronous style of errors (where they are delivered via events). All example code I have seen, if it deals with the errors at all, simply prints out the information related to the error when it is received, but inserting error-handling code there seems like it would be ineffective, because by that point all of the context related to the original message which was sent has been lost and only a 32-bit integer sinfo_context remains. It also seems that there is no way to directly tell when a given message has been acknowledged successfully by the remote peer, which would make it impossible to implement an approach which listens for errors after sending a message, because the context information for successfully-delivered messages could never be freed.
Is there a way to handle the errors related to a given sending operation as part of the call to a send function, or is there a different way to approach error handling for SCTP which does not lose the context of the error?
One solution which I have considered is using the SCTP_SENDER_DRY notification to tell when packets have been sent, however this requires sending only one packet at a time. Another idea is to use the peer's receiver window size together with the sinfo_cumtsn field of sctp_sndrcvinfo to calculate how much data has been acknowledged as fully received using the cumulative TSN, however there are a couple of disadvantages to this: first, it requires bookkeeping overhead to calculate a number of bytes received by the peer based on the cumulative TSN (especially if the peer's window size may change); second, it requires waiting until all earlier packets were received before reporting success, which seems to defeat the purpose of SCTP's multistreaming; and third, it seems like it would not work for unordered packets.
I'm wondering if this is possible to setup a timeout for receiving data over USB interface in STM32 microcontrollers. Such approach is possible for example in UART connection (please refer to AN3109, section 2. Receive DMA timeout).
I can't find anything similar related to USB interface. What's more, it is said that DMA for USB should be enabled only if really necessary because data transfer shall be aligned to 32-bit word.
You have a receive call back function (if you use the HAL) in your ...._if.c file. Copy reived chars to the buffer. Implement timeout there.
What you refer to in case of UART is either DMA receive timeout as you've said or (when not using DMA) an IDLE interrupt. I'm not aware of such thing coming "out of the box" for USB CDC - you'd have to implement this timeout yourself, which shouldn't be too hard. Have a timer (hardware of software) that you re-trigger every time you receive data. Set its period to the timeout value of your choice and do protocol parsing after timeout elapses.
If I had to add anything - these kind of problems (not knowing how many bytes to receive) are typically solved at the protocol level. Assuming binary protocol, one way of achieving this is having frame start and end bytes which never occur in data (and if they do - you escape them) in which case you receive everything starting after "start byte" until you reveive "end byte". Yet another way is having a "start byte" and a field indicating how many bytes there are to receive. All of it should of course be checksumed in some way.
Having said that, if you have an option to change the protocol, you really should do so. Relying on timings in your communication, especially on such low level only invites problems and headaches in the long run. You introduce tight coupling between your protocol layer and interface layer. This is going to backfire on you every time you decide to use a different interface, as you'll have to re-invent the same thing again. Not to mention how painful it's going to be when you decide to move to TCP/IP with all its greatness - network jitter, dropped packets etc.
I am learning server development with IO Completion Ports. My book, "Network Programming for Microsoft Windows - Second Edition", states the following:
With every overlapped send or receive operation, it is probable that
the data buffers submitted will be locked. When memory is locked, it
cannot be paged out of physical memory. The operating system imposes a
limit on the amount of memory that may be locked. When this limit is
reached, overlapped operations will fail with the WSAENOBUFS error. If
a server posts many overlapped receives on each connection, this limit
will be reached as the number of connections grow. If a server
anticipates handling a very high number of concurrent clients, the
server can post a single zero byte receive on each connection. Because
there is no buffer associated with the receive operation, no memory
needs to be locked. With this approach, the per-socket receive buffer
should be left intact because once the zero-byte receive operation
completes, the server can simply perform a non-blocking receive to
retrieve all the data buffered in the socket's receive buffer. There
is no more data pending when the non-blocking receive fails with
WSAEWOULDBLOCK.
Now, I'm trying to understand this paragraph; I think I've got it but want to make sure please.
I understand about memory being locked if I post make multiple WSARecv() calls with large buffers. But I am not entirely sure how a zero byte buffer prevents this.
I am thinking it is this (and would like confirmation please):
If I have n connections, and I post 50 WSARecv() calls with a 1KB buffer on each connection, that is n * 50KB total memory locked. All of that memory is locked, regardless of whether or not it is actually being used (i.e. whether or not anything is being copied into it from the TCP buffers). Hence if I keep adding more connections, I will keep locking more memory that may or may ever be used. Thus I can run out, with WSAENOBUFS error.
If I however post a zero byte receive on each connection, a completion packet will be generated on that connection only when there is data available for reading. (That is my first assumption, is that correct?)
Now, when I know there is some data, I can then post a WSARecv() with a buffer of 1KB (or however much) - or indeed loop repeatedly reading it all as suggested in my book - knowing that it will be filled immediately hence not remain unused and locked (second assumption, is that correct?)
Question 1
Thus, if my two assumptions are correct, then I have understood my book :) This means then that my server could, in theory, post a zero byte receive when a new connection is established, then when a completion packet is generated, read all of the data until there is no more, then post another zero byte receive - is that correct?
Question 2
However, isn't there still a risk that if I receive completion packets for lots of my zero byte receive posts at once, and I then go onto make multiple WSARecv() calls, that I will still end up with some failing with WSAENOBUFS?
Hopefully someone can clarify these two assumptions and two questions for me.
OK I've done research into this along with experimentation and have found the following:
Assumptions:
1) Assumption 1 is correct.
2) I believe assumption 2 is correct.
Questions
1) I have tested this and this seems to work.
2) This I guess remains a possibility but much less likely than if I posted receives with a none-zero buffer.
Note that we can still raise the WSAENOBUF error when sending too fast; more details here.
As the title suggests, I would like to know whether writes to CBPeripherals using CBCharacteristicWriteWithoutResponse return instantly (and do the writing on another thread) or return only when the write is complete.
I would expect the latter, but a colleague of mine has given me reason to believe the former might be true (he's seeing packets not being sent by the iOS device when repeatedly writing).
While Michal is right about the asynchronous nature of write request, I would like to elaborate a bit more on why packets might be not sent during repeatable writes without response.
I have recently experimented with BLE throughput under different conditions and found the following. When you repeatably write data for characteristic supporting write without response, it first gathers in the internal buffer. If you write too fast, this buffer overflows and all the data that doesn't make it into the buffer gets silently discarded. This has nothing to do with synchronous/asynchronous structure of BLE writes, it's just an iOS implementation of the internal buffer. As I figured out, on iPhone 6 running iOS 9 and connected to a single peripheral device, the available buffer size is around 3kb for uni-directional transfer, for bi-directional transfer its size decreases. So to prevent data loss you should ensure the buffer doesn't get overflown. You cannot control it at iOS side, but you can add some integrity checks to the data transfer protocol you use and orchestrate flow control from the peripheral side to e.g. retransmit lost packets.
It's funny, by the way, that in case of writes with responses you can write whatever large data to your characteristics (well, I didn't try too big values, but tried ones exceeding the MTU) and these data will be split into chunks and transmitted to the peer transparently for your code by iOS means without data loss. However, if the data sent to the characteristic will exceed MTU, you won't receive delivery callback.
Bluetooth Low Energy supports writing to the characteristic's value with or without response. In the latter case, the central won't receive any response from the peripheral, so it won't know whether write succeeded. In the former case, the central will be notified and Core Bluetooth will call peripheral:didWriteValueForCharacteristic:error: method from CBPeripheralDelegate; in the latter case, this method won't be called.
The documentation says:
When you call this method to write the value of a characteristic, the
peripheral calls the peripheral:didWriteValueForCharacteristic:error:
method of its delegate object only if you specified the write type as
CBCharacteristicWriteWithResponse.
I understand your colleague may have thought that instead of getting the write result in delegate method, you will get it in the return value. But actually you won't get any write result when using CBCharacteristicWriteWithoutResponse.
So writing to the characteristic's value will always be asynchrounous and the method will return immediately. If it didn't, it could block the main thread. And Core Bluetooth is designed to be used safely in from the main thread if you want to. For example, if you don't specify a dispatch queue for handling CBCentralManager's events, it will handle them in the main queue by default.