I have a written a program using gamekit/bluetooth to transfer low quality video using compressed jpegs from one iOS device to another. I do already realize that gamekit/bluetooth should not be used for this purpose (for small chunks of data) but it does indeed work well streaming 15 low quality compressed jpegs/second with little to no latency.
The question I have is once I increase either the quality or frame rate from the iOS device sender to the iOS receiver, a lag or delay will occur and will no longer be real time. If there is a delay, I'd like somehow for the sending iOS device to discard frames so that the receiver can catchup or for the receiver to ignore the backlog queue.
In GameKit I have set the session mode to use GKSendDataUnreliable to see if it could help, but to no avail.
If delays occur, what is the best solution and correct approach to discard the frames (jpegs) so that the iOS receiver can then catch up back to real time? Would the sender need to stop transmission for a period of time or is there something that receiving client can do to discard the accumulating queue.
I've used NSStream before as well, and while using wifi allows for greater bandwidth, the same problem will still occur in terms of delays if too much data is being transmitted.
Thank you in advance for your help.
Could you not attach a timestamp to each jpg (time since epoch perhaps) so the receiving client will ignore all images that are not within a given timeframe.
Also you could have the receiving client respond back with simple acknowledgement packets indicating that a jpg has been received. If the sending client hasn't received an acknowledge packet within a given timeframe it discards all images it was going to send and starts from scratch.
With this solution if the receiving client falls X seconds behind the sender it will stop sending acknowledgement packets and discard all incoming data until the sender throws out everything in its queue and starts sending "live" frames again.
Related
Is it possible to use writeValue with type CBCharacteristicWriteWithoutResponse and still have some flow control to avoid sending data faster than BLE stack manages to actually send it out? Currently it works on Android but not on iOS.
The long story.
I have previously implemented duplex communication channel over BLE on Android. It is basically using two ATT characteristics - one is write/writeWithoutResponse and the other is notifiable.
On Android, even when I use writeWithoutResponse, Android sends me onCharacteristicWrite callback to signal that the data packet has at least reached BLE stack, and in this callback I send out the next data packet with size of current ATT_MTU-3 bytes.
This works fine, the data reaches the target intact and I can achieve transfer speeds about 10 KB/s.
But on iOS there is a problem. When using writeValue with type CBCharacteristicWriteWithoutResponse, iOS (at least iOS 8) does not call didWriteValueForCharacteristic and this is intended and documented behavior. Thus I have no way of knowing if the data packet has reached BLE stack. The best I can is to call writeValue in a loop. Also, writeValue seems to be non blocking (async). As the result, not all of my data reaches the peripheral device. In the logs I see that the incoming data stream is stopped too soon. My guess is that if I call writeValue too fast, iOS is just carelessly overwriting previous cached writable characteristic value and thus misses some data bytes in between.
If I use writeValue with CBCharacteristicWriteWithResponse, it works fine, and what's strange - it works fine even if I ignore didWriteValueForCharacteristic and just call writeValue in a loop. It seems, with CBCharacteristicWriteWithResponse iOS is doing some internal housekeeping and uses BLE acknowledgments to avoid overwriting current value of the characteristic, therefore the data is being sent in order and without any losses.
Of course, I don't expect to get reliable writes using CBCharacteristicWriteWithResponse, but at least make it work for most cases. If it works on Android, then why shouldn't it work on iOS?
Apple's implementation kind of sucks. All other implementations I've seen have proper flow control. What you could do if you don't want to implement some advanced TCP-like layer on top of BLE is to simply stick with Write Without Response packets but send each 10th packet or so as a Write With Response. Then you won't (with high probability) not get any packet drops. This will probably only be a small performance decrease. You should also increase the MTU to increase the throughput even further.
Able to connect to multiple iOS devices via Bluetooth, working as 1 master and 4 slave devices.
Data transfers from central to peripheral through the following code
[peripheral.peripheral writeValue:currentData forCharacteristic:peripheral.characteristic type:CBCharacteristicWriteWithoutResponse];
But this made hugs data loss, but was faster.
then tried with the following code for not losing data
[peripheral.peripheral writeValue:currentData forCharacteristic:peripheral.characteristic type:CBCharacteristicWriteWithResponse];
Trying to transfer data to multiple peripheral at a same time (concurrently)
for (Peripheral * peripheral in self.connectedPeripherals) {
[peripheral.peripheral writeValue:currentData forCharacteristic:peripheral.characteristic type:CBCharacteristicWriteWithResponse];
}
Data transfers one by one it seems like a delay once 1st peripheral is received the data then 2nd peripheral gets the data and go on.
Want to transfer data simultaneously and reflect at same time to all the peripherals.
When you transfer data with response, you have to wait for the acknowledgement of its receipt each time you send a packet. When you transfer data without response, the acknowledgement is not being sent back, so the throughput is higher. However, as you correctly point out, when transferring data without response there can be data loss. This data loss happens because of the overflow of internal iOS buffer that holds the data between your call to - writeValue:forCharacteristic:type: and its actual departure. If you want to prevent data loss, you can do either of the following things.
Don't write too much data to the buffer, because it gets silently discarded if the buffer overflows. My experiments indicate that the size of this buffer in normal conditions is around 3kb (iPhone 6, iOS9, 1 peripheral). For other devices, several connected peripherals and/or bidirectional transfer this size can be smaller. So, if you have e.g. 1 kb of data you want to send to your 4 peripherals and you do it by iteratively calling - writeValue:forCharacteristic:type:, you'll definitely face data loss.
Implement a protocol to request re-sending missed packets in case of data loss on top of the characteristic used for writes without response.
Write with response, but split your data into as large chunks as possible. As I said earlier, the acknowledgement is sent back after every packet of data, but these packets can be of different sizes. With iOS8/iOS9 you can expect to send up to 155 bytes of payload in a single packet. So if you need to send e.g. 300 bytes, it's better to split them into 2 150-bytes chunks than 15 20-bytes chunks. By the way, when you want to write with response and submit a value longer than 155 bytes, iOS will split it for you, but in this case you won't receive a callback `
peripheral:didWriteValueForCharacteristic:error:` after the data is delivered.
So I've been dumped with a very broken and outdated code base. I'm being asked however to fix only one bug with the latency that occurs when a message is sent between two devices.
The app is for streaming audio to several devices and playing them all at once. The issue is obviously caused when the host sends a "play" message and starts playing itself. This play message is delayed by up to 3 seconds and therefore the clients all end up out of sync.
I've attempted sending a CFAbsoluteTimeGetCurrent(); value to the clients where they then work out the latency but device clocks are very unreliable and I often get negative differences in time despite obviously being positive.
Any idea on how I can combat this? And before suggestions of changing the method entirely, there's isn't much time
Have you considered sending a SYN message along with a local timestamp and then have the per return the timestamp as a ACK message? You can take the difference between the current time and the return time and half it to get a latency.
Source: I did this.
My iOS app requires socket communication. I'm following this Ray Wenderlich tutorial for setting up the input and output streams. The server I'm using is Twisted. My app requires sending and receiving fast bursts of data generated by external events like gyroscope data. It is sending/receiving data in form of JSON string. So largely, it's very much like a real-time messaging chat app but sending and receiving is very fast and in bursts.
So my app layout is that I have 1 view controller: DViewContorller and a tabbarcontroller with 3 tableviewcontrollers. I need to send and receive data in all these 4 view controllers, hence I implemented the socket stream initialization in App Delegate. For all the 3 tabs, my App Delegate sets the [self.inputstream setDelegate:self] but when it is in the DViewController it sets the delegate of input stream to a reference of DViewController. In (void)viewWillDisappear of DViewController, I reset the input stream delegate to a reference of AppDelegate to let it regain the control over the inputstream.
For outputstream, the delegate is always set to AppDelegate and never changed.
Both my AppDelegate and DViewController are <NSStreamDelegate> and both implement:
- (void)stream:(NSStream *)theStream handleEvent:(NSStreamEvent)streamEvent}
{
}
with all the stream event cases implemented.
So basically, my entire setup works well, but only if data is not fast(or for that matter, I cannot seem to pinpoint the exact problem).
So here are a few observations which I have made while testing (with a Simulator-iPhone and iPhone-iPhone setup):
A. SIMULATOR - iPHONE:
Now in this setup, I am able to send data fast and exactly the way I want it to send from Simulator to iPhone, but not from iPhone to Simulator. The iPhone receives all the strings well and acts as per required.
Sending from iPhone to Simulator, simulator can read data only one JSON string at a time, and doesn't work when data is sent fast. If sent fast, then all strings received by the simulator are half(only one half is received). NOTE: The server receives and sends full strings, and all the strings even if it is fast, there is no problem with the server.
If I send data simultaneously, at the same time, from both the Simulator and iPhone, even if fast, both receive and process all the strings well.
B. iPHONE - iPHONE:
Either iPhones (any one of them sending, not both together) can receive data only one JSON string at a time, and both don't work when data is sent fast. If sent fast, then all strings received by the any of the iPhones are half(only one half is received). NOTE: The server receives and sends full strings, and all the strings even if it is fast, there is no problem with the server.
If I send data simultaneously, at the same time, from both the iPhones to each other, even if fast, both receive and process all the strings well.
These observations led me to believe that the iPhone is receiving all fast strings only if it is simultaneously sending something to the server. OR i could be totally wrong, because when the Simulator sends to iPhone, the iPhone is able to receive everything no matter what. I want to know what the simulator is doing differently that the string received from it is taken in as full by the iPhone but not the other way round. Is it that the iPhone sends way too fast than a Simulator, hence all its sent strings don't get registered by the receiver? Somebody help me crack this please!
NOTE: In all cases, the server works perfectly and it sends and receives data in full length, no matter whatever speed. And I'm using iOS 7.
UPDATE 1:
Okay, so been experimenting with it the entire day, I finally made it to work. The thing is, it is exactly what my question statement is, output stream cannot stay idle if you want to receive continuously and fast from the input stream. I don't know why that happens, if anyone could enlighten me please. So the quick-fix I'm using is that whenever I get bytes on input stream, I immediately send blank data to server to keep the output stream active. So now the input stream can read complete data and fast. But I feel it is a wastage of server resources. Plus it's not a reliable solution. I'm looking for a concrete solution. I want to know how the Simulator does it without being bothered about the utilization of output stream. Can anyone help please?
UPDATE 2:
Learning from the previous update, it's not about sending blank data to the server, but i need to send dummy data to the sender if i want to receive the next string from him complete. I need to keep the end-to-end communication alive with dummy/blank data if i want to end/receive data fast and complete. Anyone has had this issue and found a better reliable/concrete way to do it?
I made a simple game which connects to other peers using GKSession from GameKit. It was easy to set up but I discovered some problems:
Latency varies a lot. Sometimes message arrives instantly on other devices. Sometimes latency is > 1 second for sending data to other peers. Data is only a string with 10 chars.
My game depends on precise clock synchronisation. The game has music playing and it sounds odd when it doesn't start playing the same time on all connected devices.
I couldn't find anything in the documentation about how to snychroize timing. The problem is that the "master" pier which starts the game immediately begins playing the music and then all other piers receive the message a little bit later and thus start playing after delay.
Then I tried to delay playing the game music after sending the start message to all piers, but sometimes latency is lower or higher and I just can't get a good reliable sync.
Is there a open source framework which makes peer to peer and clock sync more reliable and easier to use on iOS?
Unfortunately, you're in for a world of hurt here. It is very difficult to do precise clock synchronization over a network interface. You would need your devices to sync to within 20ms here.
I would recommend doing the following: send a bunch of ping packets at 50ms intervals, make the client respond immediately. Take the average of this to give you average round trip time (RTT) and then halve that to estimate the one-way latency. Then send your 'start clock' message and spin in a loop until you've waited for your one-way latency and begin playing the music on your end. The client should play its music immediately upon receiving 'start clock'.
There is a more robust long-term sync solution outlined here: http://en.wikipedia.org/wiki/Precision_Time_Protocol