I'm having a mental block for the words describing data flow in a communications protocol + google isn't helping, due to information glut.
In the following scenarios A and B are communicating to each other.
command or request: a packet of data going from A to B indicating that B should take some kind of action
response: a packet of data going from B to A in response to a particular packet that A has previously sent to B.
acknowledge or ACK: a specific kind of response that just indicates Yes I got that packet of data. (negative acknowledge or NAK indicates No there was some problem receiving data)
{X}: unsolicited information either from A to B, or B to A, which is neither a response, nor a request for the recipient to take action. Examples: datalogging packets, notification packets, etc.
I can't think of what to call {X}, I'm having a brain cramp.
Also are there other common words in this category? Where would you look them up?
Why not call your "X" just "Message" or "Signal" ?
For some inspiration regarding this subject, may I suggest the IETF or ITU-T standards bodies. These two bodies are dominating the communications scene.
I would following the terminology in Steven's TCP/IP Illusrated
So you have a request, response, acknowledement, push and poll are used if I remember correctly
Related
Read a lot of specifications and still can't get a simple thing.
All UDS requests encapsulated in ISO-TP packets, which are encapsulated in simple CAN frames, so ECU constantly receives a stream of frames from CAN bus.
How does ECU decide that this CAN frame is a part of any high-level protocol?
For example, I've sent Security request to ECU, CAN frame data will look like this
02 27 01
How does ECU determine that this is not just a chunk of data but a part of the protocol?
I wasn't able to find any relation to ISO/OSI stack when high-level protocols "talk to each other" using headers, so we know how to decode data packets.
The CAN message IDs that are used for specific protocols are defined per system.
In most cases the OBD-II will be sent over CAN ID 7DFh for the query and higher IDs for responses from different modules, but even that might be different on specific car models.
One way of figuring out the CAN IDs that are used for UDS-based communication is to send simple tester-present (SID 3Eh) messages and watching for CAN IDs which seems to have an appropriate response.
UDS via CAN is specified in the DoCAN ISO-15765-2 part and describe the network and transport layer for a functional (broadcast) and physical (p2p) communication between ECUs or better control functions.
Normal CAN id's don't implement any network functionalities like a addressing. For that purpose the SAE J1939 network layer is used. In a J1939 network each CAN client has a address and each functionality a parameter group (PGN). All of this is encoded in a 29bit CAN ID. For example the CAN ID 0x18EF8081. This will transport a message from the CAN client 0x81 to 0x80 via the PGN 0xEF, 0x18 is the priority.
In UDS over CAN the PGNs 0xDA (physical) and 0xDB (functional) are used for all communications. With that information's you can implement a CAN ID filter which match only the PGN part of the CAN ID.
After that I send different runicast messages with the function runicast_send, how can I understand which message was acknowledged when the callback sent_runicast is triggered?
The runicast.h file states:
The runicast primitive adds two packet attributes: the single-hop
packet type and the single-hop packet ID. The runicast primitive
uses the packet ID attribute as a sequence number for matching
acknowledgement packets to the corresponding data packets.
but I didn't understand how to do it in practice. Can somebody provide an example?
One way would be to look at the field sndnxt of struct runicast_conn *c before you send the packet, and then compare that value of the packetbuf_attr(PACKETBUF_ATTR_PACKET_ID) in the "sent" callback of your code.
However note that by default the runicast packet ID is just 2 bits long. Enough to demultiplex the ACK in most cases, but may be insufficient for your purposes. (The packet ID size in bits can be changed by redefining RUNICAST_PACKET_ID_BITS.)
Also Rime is obsolete. Don't use it in your code, especially production code unless you know what you're doing. runicast was never one of the highlights of Rime, I doubt there are no better alternatives (e.g. the uIPv6 stack) for what you want to do.
How a CAN Bus controller decides based on message identifier that this particular message belongs to it?Is it like the receiver already know that if identifier has suppose value 5 then its for me . And we program receiver to tell it that you should be interested in value 5 ?
The software in the CAN node must decide what message IDs it is interested in, based on the network specification which is usually some kind of document or other electronic representation of which messages contain what sorts of information. If a message arrives that is of no interest, it simply does not process it and the software returns to what it was doing just before the message arrived (assuming interrupt driven CAN handling).
Some CAN controllers (ie the part of the chip which does the CAN protocol transmission and reception) have message filtering which means that uninteresting messages can be dropped before they reach the software. Other controllers have message filtering which can be set to accept only a single message ID in a particular "message box", and these can be configured to accept the messages you are interested in. Again, other messages are dropped. Some controllers have both filters and message boxes.
At the CAN protocol level all nodes in a CAN network are equal and make a decision about whether to process a message or not. A "CAN controller" is a higher-level concept; it still needs to examine the message identifier like any other node.
Note that "processing" a message is different to the CAN protocol message check and acknowledgement. All nodes take part in that processing unless they're in "listen only" mode.
Update:
How you decide which message to process depends on what you are trying to do and the higher level protocol in use over CAN. In principle you mask out the ID bits that are relevant and then test them to see whether the message should be processed.
For example if you want to process all messages with 5 (binary 0101) in the low order four bits, your mask is 15 (binary 1111), you binary-and this with the received message ID, and then you compare the result with five.
For example:
(msg_id & 15) == 5
is a way of coding that test. Which bits you care about, and your implementation details depend on many other factors.
Specifically for PDU1 (Protocol Data Unit) messages, a destination address is specified (byte 3). If a device receives a message not addressed to it, it can simply ignore it. Addresses are assigned by various standards, or a manufacturer may assign them ad-hoc.
In the general case the CAN-ID (bytes 0-4) contains all the details about what kind of message it is, and devices can inspect particular fields to decide whether they care about the message. For example the transmission controller probably doesn't care about battery status messages, nor the fuel gauge about which doors are locked.
While reading the ERTS user's guide, I found this section:
The only signal ordering guarantee given is the following. If an entity sends multiple signals to the same destination
entity, the order will be preserved. That is, if A sends a signal S1 to B, and later sends the signal S2 to B, S1 is
guaranteed not to arrive after S2.
I've also happened across this while doing further research googling:
Erlang Reference Manual, 13.5:
Message sending is asynchronous and safe, the message is guaranteed to eventually reach the recipient, provided that the recipient exists.
That seems very vague and I'd like to know what guarantees I can rely on in the following scenario:
A,B are processes on two different nodes.
Assume A does not crash and B was a valid node at some point.
A and B monitor each other.
A sends messages M1,M2,M3 to B
In the above scenario, is it possible that B receives M1,M3 (M2 is dropped),
without any sort of 'DOWN'/'EXIT'/heartbeat timeout being received at A?
There are no other guarantees other than the ordering guarantee. Note that by default you don't even know who the sender is, unless the sender encodes this in the message.
Your example could happen:
A sends M1 and M2
B receives M1
The node on which B resides gets disconnected
The node on which B resides comes up again
A sends M3 to B
B receives M3
M2 can be lost on the network link in this scenario. It is highly unlikely this happens, but it can happen. The usual trick is to have some kind of notion of such errors. Either by having a timeout trigger, or by monitoring the node or Pid which is the recipient of the message.
Updated scenario:
In the updated scenario, provided I read it correctly, then A would get a 'DOWN' style message at some point, and likewise, it would get a message telling you that the node is up again, if you monitor the node.
Though often, such things are better modeled using an idempotent protocol if at all possible.
Reading through the erlang mailing-list and the academic faq, it seems like there are a few guarantees provided by the ERTS implementation, however I was not able to determine whether or not they are guaranteed at a language/specification level, too.
If you assume TCP is "reliable", then the current implementation guarantees that
given A,B are processes on different nodes (&hosts) and A monitors B, A sends to B, assuming A doesn't crash, any message delivery failures* between the two nodes or host/node/process failures on B will lead to A getting a 'DOWN' message (or 'EXIT' in the case of links). [ See 1 and 2 ]
*From what I have read on the mailing-list thread , this property is almost entirely based on the fact that TCP is used, so "message delivery failure" means any situation where TCP decides that a failure has occurred/the connection needs to be closed.
The academic faq talks about this like it's also a language/specification level guarantee, however I couldn't yet find anything to back that up.
I am about to write a message protocol going over a TCP stream. The receiver needs to know where the message boundaries are.
I can either send 1) fixed length messages, 2) size fields so the receiver knows how big the message is, or 3) a unique message terminator (I guess this can't be used anywhere else in the message).
I won't use #1 for efficiency reasons.
I like #2 but is it possible for the stream to get out of sync?
I don't like idea #3 because it means receiver can't know the size of the message ahead of time and also requires that the terminator doesn't appear elsewhere in the message.
With #2, if it's possible to get out of sync, can I add a terminator or am I guaranteed to never get out of sync as long as the sender program is correct in what it sends? Is it necessary to do #2 AND #3?
Please let me know.
Thanks,
jbu
You are using TCP, the packet delivery is reliable. So the connection either drops, timeouts or you will read the whole message.
So option #2 is ok.
I agree with sigjuice.
If you have a size field, it's not necessary to add and end-of-message delimiter --
however, it's a good idea.
Having both makes things much more robust and easier to debug.
Consider using the standard netstring format, which includes both a size field and also a end-of-string character.
Because it has a size field, it's OK for the end-of-string character to be used inside the message.
If you are developing both the transmit and receive code from scratch, it wouldn't hurt to use both length headers and delimiters. This would provide robustness and error detection. Consider the case where you just use #2. If you write a length field of N to the TCP stream, but end up sending a message which is of a size different from N, the receiving end wouldn't know any better and end up confused.
If you use both #2 and #3, while not foolproof, the receiver can have a greater degree of confidence that it received the message correctly if it encounters the delimiter after consuming N bytes from the TCP stream. You can also safely use the delimiter inside your message.
Take a look at HTTP Chunked Transfer Coding for a real world example of using both #2 and #3.
Depending on the level at which you're working, #2 may actually not have an issues with going out of sync (TCP has sequence numbering in the packets, and does reassemble the stream in correct order for you if it arrives out of order).
Thus, #2 is probably your best bet. In addition, knowing the message size early on in the transmission will make it easier to allocate memory on the receiving end.
Interesting there is no clear answer here. #2 is generally safe over TCP, and is done "in the real world" quite often. This is because TCP guarantees that all data arrives both uncorrupted* and in the order that it was sent.
*Unless corrupted in such a way that the TCP checksum still passes.
Answering to old message since there is stuff to correnct:
Unlike many answers here claim, TCP does not guarantee data to arrive uncorrupted. Not even practically.
TCP protocol has a 2-byte crc-checksum that obviously has a 1:65536 chance of collision if more than one bit flips. This is such a small chance it will never be encountered in tests, but if you are developing something that either transmits large amounts of data and/or is used by very many end users, that dice gets thrown trillions of times (not kidding, youtube throws it about 30 times a second per user.)
Option 2: size field is the only practical option for the reasons you yourself listed. Fixed length messages would be wasteful, and delimiter marks necessitate running the entire payload through some sort of encoding-decoding stage to replace at least three different symbols: start-symbol, end-symbol, and the replacement-symbol that signals replacement has occurred.
In addition to this one will most likely want to use some sort of error checking with a serious checksum. Probably implemented in tandem with the encryption protocol as a message validity check.
As to the possibility of getting out of sync:
This is possible per message, but has a remedy.
A useful scheme is to start each message with a header. This header can be quite short (<30 bytes) and contain the message payload length, eventual correct checksum of the payload, and a checksum for that first portion of the header itself. Messages will also have a maximum length. Such a short header can also be delimited with known symbols.
Now the receiving end will always be in one of two states:
Waiting for new message header to arrive
Receiving more data to an ongoing message, whose length and checksum are known.
This way the receiver will in any situation get out of sync for at most the maximum length of one message. (Assuming there was a corrupted header with corruption in message length field)
With this scheme all messages arrive as discrete payloads, the receiver cannot get stuck forever even with maliciously corrupted data in between, the length of arriving payloads is know in advance, and a successfully transmitted payload has been verified by an additional longer checksum, and that checksum itself has been verified. The overhead for all this can be a mere 26 byte header containing three 64-bit fields, and two delimiting symbols.
(The header does not require replacement-encoding since it is expected only in a state whout ongoing message, and the entire 26 bytes can be processed at once)
There is a fourth alternative: a self-describing protocol such as XML.