We have a critical need to lower the latency of our UDP listener on iOS.
We're implementing an alternative to RTP-MIDI that runs on iOS but relies on a simple UDP server to receive MIDI data. The problem we're having is that RTP-MIDI is able receive and process messages around 20ms faster than our simple UDP server on iOS.
We wrote 3 different code bases in order to try and eliminate the possibility that something else in the code was causing the unacceptable delays. In the end we concluded that there is a lag between time when the iPAD actually receives a packet and when that packet is actually presented to our application for reading.
We measured this with with a scope. We put a pulse on one of the probes from the sending device every time it sent a Note-On command. We put another probe attached to the audio output of the ipad. We triggered on the pulse and measured the amount of time it took to hear the audio. The resulting timing was a reliable average of 45ms with a minimum of 38 and maximum around 53 in rare situations.
We did the exact same test with RTP-MIDI (a far more verbose protocol) and it was 20ms faster. The best hunch I have is that, being part of CoreMIDI, RTPMIDI could possibly be getting higher priority than our app, but simply acknowledging this doesn't help us. We really need to figure out how fix this. We want our app to be just as fast, if not faster, than RTPMIDI and I think this should be theoretically possible since our protocol will not be as messy. We've declared RTPMIDI to be unacceptable for our application due to the poor design of its journal system.
The 3 code bases that were tested were:
Objective-C implementation derived from the PGMidi example which would forward data received on UDP verbatim via virtual midi ports to GarageBand etc.
Objective-C source base written by an experienced audio engine developer with a built-in low-latency sine wave generator for output.
Unity3D application with Mono-based UDP listener and built-in sound-font synthesizer plugns.
All 3 implementations showed identical measurements on the scope test.
Any insights on how we can get our messages faster would be greatly appreciated.
NEWER INFORMATION in the search for answers:
I was digging around for answers, and I found this question which seems to suggest that iOS might respond more quickly if the communication were TCP instead of UDP. This would take some effort to test on our part because our embedded system lacks TCP capabilities, only UDP. I am curious as to whether maybe I could hold open a TCP connection for the sole purpose of keeping the Wifi responsive. Crazy Idea? I dunno. Has anyone tried this? I need this to be as real-time as possible.
Answering my own question here:
In order to keep the UDP latency down, it turns out, all I had to do was to make sure the Wifi doesn't go silent for more than 150ms (or so). The exact timing requirements are unknown to me at this time, however the initial tests I was running were with packets 500ms apart and that was too long. When I increased the packet rate to 1 every 150ms, the UDP latency was on par with RTPMIDI giving us total lag time of around 18ms average (vs. 45ms) using the same techniques I described in the original question. This was on par with our RTPMIDI measurements.
Related
I need to realize a source-synchronous receiver in a Virtex 6 that receives data and a clock from a high speed ADC.
For the SERDES Module I need two clocks, that are basically the incoming clock, buffered by BUFIO and BUFR (recommended). I hope my picture makes the situation clear.
Clock distribution
My problem is, that I have some IOBs, that cannot be reached by the BUFIO because they are in a different, not adjacent clock region.
A friend recommended using the MMCM and connecting the output to a BUFG, which can reach all IOBs.
Is this a good idea? Can't I connect my LVDS clock buffer directly to a BUFG, without using the MMCM before?
My knowledge about FPGA Architecture and clocking regions is still very limited, so it would be nice if anybody has some good ideas, wise words or has maybe worked out a solution to a similar problem in the past.
It is quite common to use a MMCM for external inputs, if only to cleanup the signal and realize some other nice features (like 90/180/270 degree phase shift for quad-data rate sampling).
With the 7-series they introduced the multi-region clock buffer (BUFMR) that might help you here. Xilinx has published a nice answer record on which clock buffer to use when: 7 Series FPGA Design Assistant - Details on using different clocking buffers
I think your friends suggestion is correct.
Also check this application note for some suggestions: LVDS Source Synchronous 7:1
Serialization and Deserialization Using
Clock Multiplication
I'm writing an IOS/iPhone app that communicates with an sp-10c imu via bluetooth low energy. The packets I am receiving were originally 61 bytes long, but I have shortened them to 38 bytes so that the packet is sent in the minimum number of notifications.
I have no control over the actual programming of the sp-10c, but they do give me control of some of the settings. I can change what all info I receive (accelerations, gyros quaternions, etc) and thereby change the packet size, and I can also change the transmission interval (by 10 millisecond intervals). I have read a lot of the questions and answers on here related to this subject, but the only solutions I have seen require having programming control over the peripheral device, which I do not have.
I have also read Apple's handbook on this, which states that the minimum interval should be 20 ms. If I could reliably get entire packets at 50 hz, that would work for my needs. Anything less is pretty much worthless. My problem is that I'm getting intervals of massive packet loss at the 20ms interval (there are times when 40 or more packets are lost, and this happens regularly). I really don't see reliable results until I slow it down to an interval of 60 ms or more, which, as I said, is worthless. (I am trying to sense a sharp change in acceleration that lasts about 20 - 40ms)
Most of the questions and answers on here are somewhat dated, but I saw where someone said that things may have gotten worse as far as the BLE connection + Apple devices go. It has been suggested that classic bluetooth is the only way to go when large amounts of throughput are needed, but the sp-10c does not offer classic bluetooth as an option.
My questions are:
Is this the normal behavior for a ble connection with apple devices?
Has anyone had any success getting reliable ble notifications at 20ms with longer packets?
Should I give up now and try to find an imu with classic bluetooth?
I guess what I'm really getting at is this: Am I doing something wrong, or is this just par for the ble/iPhone course?
Would it help to try and limit the packet to less than 20 bytes so it is received in one notification?
So i'm curious, is there any info on a router's config page or manual, or is there any way to measure it's buffer? (I'm talking about the "memory" it has to keep packets until the router can transmit them)
You should be able to measure your outgoing buffer by sending lots of (UDP) data out, as quickly as possible, and see how much goes out before data starts getting dropped. You will need to send it really fast, and have something at the other end to capture it as it comes in; your send speed has to be a lot faster than your receive speed.
Your buffer will be a little smaller than that, since in the time it takes you to send the data, at least the first packet will have left the router.
Note that you will be measuring the smallest buffer between you and the remote end; if you have, for example, a firewall and a router in two separate devices, you don't really know which buffer you are testing.
Your incoming buffer is a lot more difficult to test, since you can't fill it fast enough (from the internet side) not to be deliverable quickly enough.
I'm using Wireshark to monitor network traffinc to test a new software installed on a router. The router itself lets other networks (4g, mobile devices through usb etc) connect to it and enhance the speed on that router.
What I'm trying to do is to disconnect the connected devices and discover if there are any packet losses while doing this. I know I can simply use a filter stating "tcp.analysis.lost_segment" to track down lost packets, but how can I eventually isolate the specific device that causes the packet loss? Or even know if the reason was because of a disconnected device when there is a loss?
Also, what is the most stable method to test this with? To download a big file? To stream a video? Etc etc
All input is greatly appreciated
You can't detect lost packets solely with Wireshark or any other packet capture*.
Wireshark basically "records" what was seen on the line.
If a packet is lost, then by definition you will not see it on the line.
The * means I lied. Sort of. You can't detect them as such, but you can extremely strongly indicate them by taking simultaneous captures at/near both devices in the data exchange... then compare the two captures.
COMPUTER1<-->CAPTURE-MACHINE<-->NETWORK<-->CAPTURE-MACHINE<-->COMPUTER2
If you see the data leaving COMPUTER1, but never see it in the capture at COMPUTER2, there's your loss. (You could then move the capture machines one device closer on the network until you find the exact box/line losing your packets... or just analyze the devices in the network for eg configs, errors, etc.)
Alternately if you know exactly when the packet was sent, you could not prove but INDICATE its absence with a capture covering a minute or two before and after the packet was sent which does NOT have that packet. Such an indicator may even stand on its own as sufficient to find the problem.
I'm sending messages upon detection of both discrete and continuous gestures. For continuous, UDP should be fine because even if a couple packets are lost, there's so many change events that it shouldn't matter.
I'm wondering about discrete events though, for example tap or swipe. Since there would only be a single packet sent, what is the risk that it doesn't arrive, and the application at the other end isn't notified of the gesture?
I understand that TCP guarantees delivery, but I'm thinking it might be too much overhead for the high frequency of messages generated from continuous gestures.
If your only concern with TCP is the extra overhead, then I wouldn't worry too much. Certainly, TCP has more overhead than UDP. However, the overhead isn't that much, especially for the modest amount of data you are likely to be sending. Some quick back-of the envelope calculations:
Assume you want to send status information every millisecond. (Likely more often than you really need to.)
Assume your individual messages can easily fit within 50 bytes / each. (Likely larger than you really need.)
Total bandwidth 50 bytes / ms = 400 bits / ms = 400 kbps
Even with these larger-than-necessary messages, and faster-than-needed updates, your total bandwidth is only around 5% of a slowish 802.11b wireless network. The extra overhead of TCP isn't likely to make a big difference here.
Personally, I tend to stick with TCP unless I have a strong reason not to. Sure, you could save some extra bits by using UDP, but to me, having the reliable delivery (including correctly ordered data, non duplicated data) is worth the extra overhead. One less thing to worry about.
EDIT: TCP does have some other drawbacks. In particular, it can take a bit more programming effort to create the initial connection, and to parse individual messages from the byte stream. UDP can certainly make these tasks easier. However, you didn't list programming complexity as one of your criteria, so I focused on your overhead questions instead.
LATENCY: As noted in comments below, latency is a critical factor. In this case, UDP has some significant advantages: If a packet is dropped, TCP will wait for that packet to be retransmitted before sending others. The benefit of this approach is that data are guaranteed to arrive in the original order. (A big plus when messages must be processed sequentially.) The drawback, of course, is that there could be a significant delay for new data.
UDP, on the other hand, will continue sending subsequent packets, even if one is dropped. The good news is that UDP reduces the latency of the remaining packets. The bad news, though, is that you must add some sort of "retry" in case discrete events are lost-- if you do so, you now have cases of gestures arriving out of order, perhaps significantly so. Can your app handle a case where a "move then click" gets changed to a "click then move"? If you chose to go the UDP route, you'll need to carefully think through all these cases to make sure it won't cause (perhaps subtle) problems in your app.