I surely must have missed something from my reading of the LoRaWAN specifications, because this seems too bad to be true. Please tell me I'm delirious :)
The following seems to happen in my testbed when I have many OTAA nodes and I can't figure out what would prevent it:
multiple nodes in my network issues JOIN REQUEST at the same time (this can happen by chance or if they are powered on simultaneously)
gateway receives (at least) one of them successfully and responds with a JOIN ACCEPT assigning a DevAddr, thinking one node did a join req
all the nodes that did the JOIN REQUEST will receive the ACCEPT and think the JOIN ACCEPT was directed at them, and gladly sets the same received DevAddr
From here on, we have several nodes that all think they joined successfully and all think they are unique but have the same DevAddr. Needless to say, the system gets severely messed up.
Reading the LoRaWAN specification, the JOIN REQUEST has a node unique DevEUI, a network unique AppEUI, and a random DevNonce (to prevent replay attacks). The MIC is calculated from these and the secret network unique AppKey stored in the node.
The JOIN ACCEPT has, as far as I can see, no data in it which is derived from the JOIN REQUEST, and therefore it can't be directed to a specific node in the case that many nodes are currently listening to an ACCEPT.
It has: AppNonce NetID DevAddr DLSettings RxDelay CFList, and is encrypted with the AppKey which is network unique and not node unique. The MIC only involves these values and so doesn't help either.
I would have expected that the JOIN ACCEPT at the minimum includes the DevEUI requesting the join as a part of the MIC, and also that it would include the DevNonce. It seems it includes neither.
What gives? Is OTAA broken or not? :)
The MIC will be different for each device because it's base on the secret (and supposedly unique) master key (AppKey) shared between the device and the network.
The first thing a device do is checking the MIC, if it's not what's expected it will drop the message.
So what you said below is not exactly right :
The JOIN ACCEPT has, as far as I can see, no data in it which is derived from the JOIN > REQUEST, and therefore it can't be directed to a specific node in the case that many > > nodes are currently listening to an ACCEPT.
It has: AppNonce NetID DevAddr DLSettings RxDelay CFList, and is
encrypted with the AppKey which is network unique and not node unique.
The MIC only involves these values and so doesn't help either
Of course if you set the same AppKey on every of your device, you will
get what you described :)
Apart from the different AppKey as mentioned in Pierre's answer (strongly recommended), the node also includes a DevNonce in its Join Request. This DevNonce is used to derive the NwkSKey and AppSKey session keys from the Join Accept response.
In LoRaWAN 1.0.x, this DevNonce should be random. So even when using the same AppKey for all devices, chances should be low that they would also have generated the same DevNonce. So even if the MIC somehow validated, then the derived keys will not match the keys known to the server, basically rendering the device useless without it knowing it.
In LoRaWAN 1.1 I think that the DevNonce is an increasing number, but in 1.1 OTAA has changed so I don't know how that affects the results.
See https://runkit.com/avbentem/deciphering-a-lorawan-otaa-join-accept.
The question also states:
this can happen by chance or if they are powered on simultaneously
As for switching on simultaneously, the 1.0.x specifications state:
The transmission slot scheduled by the end-device is based on its own communication needs with a small variation based on a random time basis
Still then, such small variation probably won't avoid nodes hearing each others Join Accept messages in this scenario, as the downlink receive window will need to be slightly lenient too.
One qualifier is the timing requirements for Join Request (JR) and Join Accept (JA). The specification is that a device can only use a JA received "precisely" 5 or 6 (2nd window) seconds after it sent the JR.
I'd hope there are better fail-safes then this timing but the intention might be to prevent the wrong tags from taking a JA.
Related
I know that you're not supposed to rely on IDs as identifier for nodes over the long term because when you delete nodes, the IDs may be re-assigned to new nodes (ref).
Neo4j reuses its internal ids when nodes and relationships are deleted. This means that applications using, and relying on internal Neo4j ids, are brittle or at risk of making mistakes. It is therefore recommended to rather use application-generated ids.
If I'm understanding this correctly, then only looking up a node/relationship by its id when you can't guarantee if it may have been deleted puts you at risk.
If through my application design I can guarantee that the node with a certain ID hasn't been deleted since the time ID was queried, am I alright to use the IDs? Or is there still some problem that I might run into?
My use case is that I wish to perform a complex operation which spans multiple transactions. And I need to know if the ID I obtained for a node during the first transaction of that operation is a valid way of identifying the node during the last transaction of the operation.
As long as you are certain that a node/relationship with a given ID won't be deleted, you can use its native ID indefinitely.
However, over time you may get want to add support for other use cases that will need to delete that entity. Once that happens, your existing query could start producing intermittent errors (that may not be obvious).
So, it is still generally advisable to use your own identification properties.
I have several new Ford vehicles with SYNC at a workplace and a driver could be assigned a different vehicle day to day. It is easy enough to pair an iPhone to a single vehicle. However, when pairing to a dozen or so vehicles I noticed a couple problems. The most obvious is there are a fairly low number of devices that the head unit will accept to pair to. I have to delete a pairing to be able to add another. Not terrible and likely a manufacturer limitation. The most annoying and time consuming problem is the Bluetooth name is the same for every vehicle. The name is always “SYNC”. There is no way I see to edit the name or add a note to the Bluetooth connection in IOS to signify a vehicle fleet number. So the user has no idea which “SYNC” to connect to. Much worse, when you can’t connect for some reason and need to blow away the old connection because it doesn’t appear as a new device to connect to, then you have to blow away every SYNC connection because you have no label signifying which connection belongs to which vehicle you’re using at the time. Ford sells fleet vehicles and I’d think they would test such a scenario. Any work around? Any way to edit the SYNC name coming from the head unit to something like “SYNC 234” so that it contains a vehicle number?
Has anyone posted a response to this problem? There have been other posts with no answers. Our situation is that we are pushing messages onto a topic that is backing a KTable in the first step of our stream process. We are then pulling a small amount of data from those messages and passing them along. We are doing multiple computations on that smaller amount of data for grouping and aggregation. At the end of the streaming process, we simply want to join back to that original topic via a KTable to pick up the full message content again. The results of the join are only a subset of the data because it can not find the entries in the KTable.
This is just the beginning of the problem. In another case, we are using KTables as indexes for lookups meant to enrich the data coming in. Think of these lookups as identifying whether we have seen a specific pattern in the streaming message before. If we have seen the pattern we want to tag it with an ID (used for grouping) pulled from an existing KTable. If we have not seen the pattern before we would assign it an ID and place it back into the KTable to be used to tag future messages. What we have found is that there is no guaranty that the information will be present in the KTable for future messages. This lack of guaranty seems to make KTables useless. We can not figure out why there is a very little discussion of this on the forums.
Finally, none of this seemed to be a problem when running with a single instance of the streams application. However, as soon as our data got large and we were forced to have 10 instances of the app, everything broke. As well, there is no way that we could use things like GlobalKTables because there is too much data to be loaded into a single machine's memory.
What can we do? We are currently planning to abandon KTables all together and use something like Hazelcast to store the lookup data. Should we just move to Hazelcast Jet and drop Kafka streams all together?
Adding flow:
Kafka data flow
I'm sorry for this non-answer answer, but I don't have enough points to comment...
The behavior you describe is definitely inconsistent with my understanding and experience with streams. If you can share the topology (or a simplified one) that is causing the problem, there might be a simple mistake we can point out.
Once we get more info, I can edit this into a "real" answer...
Thanks!
-John
So IMAP has a feature where, once I've looked at a mailbox, I can efficiently look for any new messages by asking it for any new UIDs which I haven't seen yet.
But I how do I efficiently find expunged messages? I haven't found any commands for doing this yet; my only option is to retrieve the full UID list of the mailbox and looking for missing ones. This is rather suboptimal.
I have mailboxes with 25000+ messages. Scanning one of these mailboxes takes megabytes of traffic just to do the UID SEARCH command, so I'd like to avoid this. Is there anything hidden in the depths of the IMAP protocol that I'm missing?
Okay, for offline use this can work:
Since big mailboxes typically grow big by having many messages added and few deleted, you can optimise for there being large unchanged parts of the messages. If your clientside store contains 10000 messages, then you can send 'x UID SEARCH 2000,4000,6000,8000,10000` which will return five UIDs in one SEARCH response. Depending on which ones have changed, you know whether there have been any expunges in each fifth of the mailbox, so if the mailbox hasn't changed you've verified that very cheaply. If a particular fifth has changed, you can retrieve all 2000 UIDs in that.
QRESYNC/CONDSTORE is much nicer though, and also let you resync flag state.
The only effective answer I know to this is to change the problem.
Instead of learning which of the 25000 messages have been deleted, the client can learn which of the messages in its cache have been deleted and which ones still exist, and that can be done fairly efficiently. One approach is to keep a per-message flag in the client, "this message has been observed to exist in this IMAP connection" and when you access a cached message for which the flag isn't set, you send "x UID SEARCH UID y", which will return the message's UID if the message exists and an empty result if not. QRESYNC's SELECT arguments provide a natural improvement to this technique.
The problems, of course, are that with a non-QRESYNC server the client will cache messages for longer than necessary, and that depending on how you implement it the client might flicker or have unpleasant delays.
If two nodes with same name claiming same address in j1939 what will happen in this situation? will any one of node will claim address or error will occour ?
My copy of the specification is dated, but I'm sure this rule hasn't changed since 2003 (SAE J1939-81):
"Manufacturers of ECUs and integrators of networks must assure that the NAMEs of all CAs intended to
transmit on a particular network are unique."
Of course, that being said, it is of course possible to put devices with the same NAME on the
same set of wires, either through ignorance or malicious intent.
I personally haven't played with it, but in theory, if your device has the exact same NAME as another,
your address claim will exactly overlap the other, neither would be aware of the other's presence,
the message would go through successfully, and each device will assume it is the one that sent it.
I may be wrong, but I think the only thing odd an CA might see is message coming in from an address
it thought it had claimed, a problem which it may not even be checking for.
From the network standpoint, there is no way to distinguish that the nodes are different since they are identifying themselves as the same entity. What would happen is that the first requests will be addressed, and the second request will be ignored. In other words, this is race condition because only one message is processed at a time in the datalink. By the time the second node tries to claim the same address, the address table is already occupied and the late request node won't be able to get the notification that the address was assigned to it. Remember that each node has its own internal states/configuration.
J1939-81 says
"Repeated Collisions Occur, devices go BUS OFF CAs should retry using
a Pseudo-random delay before attempting to reclaim and then revert to
Figures 2 and 3."