I have an iOS application where I POST transactions to an API each time a transaction is completed. Once I get a 200 response code from the server I update an attribute on the transaction:
newTransaction.Synced = true
Incase the network connection ever drops I also POST every transaction where Synced = false when Reachability detects a network connection.
In perfect network conditions this works wells. However when I enable the Network Link Conditioner on my iPad and set packet loss to say 40% I start to see duplicated transactions on my server. What I assumed was happening is that it was taking longer than 30 seconds (the client side timeout on the request) to send my request and get the response from the server due to the high packet loss.
To confirm this, I made my API Sleep for 40 seconds for each web request and disabled Network Link Conditioner. As expected, the iOS app never set the Synced attribute to true as it was timing out before it got the response. However the server still created the entity for each POST request that was generated each time the iOS app launched or got network connectivity.
What's the best way to handle this situation so that duplicates never occur? I did think of adding a GUID to the transaction and then coding the API not to re-add the transaction if the GUID already exists. However the flip side is the iOS app would still never know the transaction has successfully synced. Is there a better way to handle this? Perhaps a timeout on the request which the server also adheres to?
Your Idea of assigning the GUID to transaction is good, but you might need to maintain a table on client side (browser memory) which will hold a record of all the calls you made to server and never heard back.
Related
Is there any configuration like android user-permission in iOS to control access to internet?
I think all new projects access to internet by default, is that correct?
When I send a request to the internet it returns 0 http-error code, it means I can't access to the internet.
yes, it is correct all the new ios project have access to the internet by default.
A status code of 0 in an NSHTTPURLResponse object generally means there was no response and can occur for various reasons. The server will never return a status of 0 as this is not a valid HTTP status code.
Any http request will first be processed by the operating system, and during that phase you can get an error. Getting an error means that your request never got a response from the server (and with the exception of https requests where certificates were not accepted, most likely didn't reach the server).
If this phase succeeds, then you get eventually a reply from the server. This may take time, you may have to wait 60 seconds. Which is why you do all your internet requests on a background thread. That reply will have a status code (status, not error). The status code is NEVER 0.
By default, iOS doesn't allow http requests, and doesn't allow https requests to unsave servers, so you better use only https unless you have a very good reason. You will need a very good reason to convince Apple to let your app on the app store if you want http requests to succeed. But if you get this wrong, you get an error quite early on.
A status of zero most likely means that a background request didn't finish by the time you read the status, a basic programming mistake. You need to learn how background threads and callbacks work. Without that, you won't be able to use http successfully.
Also google for "Reachability" which can tell you if your app currently has internet access (like when WiFi and Mobile Data are turned off, or in Airplane mode).
I am using the Twilio Network Traversal Service as part of a native application I am working on to perform peer-to-peer remote desktop connections. We implement a subset of the WebRTC protocol stack that is equivalent to the WebRTC data channels (not the WebRTC video and audio protocols). When using a TURN relay, the TURN allocation seems to be invalidated randomly somewhere between a few minutes and a maximum of 12 minutes from the session start. This issue looks very similar to this one, but the proposed workaround (sending silent audio) is not acceptable in my case, since I do not implement the WebRTC audio/video protocols.
I have been pulling my hair on this problem for the last two weeks, and isolated the issue as being the Twilio service itself. To compare, I have used a web based WebRTC data channel demo using firefox and the Xirsys TURN server cloud. I have wireshark captures showing firefox getting disconnected with Twilio just like my native application, while the exact same firefox demo doesn't get disconnected when using the Xirsys servers.
I was using Xirsys originally, but I experienced some instability with their service that made me switch to Twilio, which is why I would rather have Twilio fix this issue instead of going back with Xirsys. At the bare minimum, I would rather have two WebRTC hosting providers I can choose from that I know should work fine. This is why I am taking the time to explain the issue in detail so it can get fixed.
Here are two wireshark captures (with the peer-to-peer data messages filtered out) showing firefox using WebRTC data channels and the Twilio TURN relay servers:
The traffic stops being relayed after 4 minutes in the first capture, and after about 11 minutes in the second capture. In both captures, firefox detects that traffic stops being relayed (at the data channel level) and attempts a graceful disconnection by sending a Refresh request packet with a lifetime of zero. Both graceful disconnections result in a 437 Allocation Mismatch error, indicating that the server doesn't even know about the allocation firefox is trying to close gracefully.
With my native application, this would often take the form of a CreatePermission Request message that fails with a 438 "Wrong nonce" error, which is basically what should happen if a client tries to update the permission on an allocation that no longer exists. The error code 438 usually means "Stale nonce", which is not really an error, but an indication that the nonce has expired and the client should try again using the new nonce contained in the "error" message. It took me a while to figure out, but even if the error code is 438, the error string is not the same. I have observed a true stale nonce error with Xirsys and successfully updated my permission with the new nonce from the error response, so I know I can properly handle this case in my implementation.
Here is the source code for the WebRTC data channel demo I have used:
https://github.com/devolutions/webrtc-demo
For comparison, here is the same firefox data channel demo using the Xirsys TURN server cloud:
In this capture, I have let the demo run for about 16 minutes (it works for much longer than that, the longest I have tried is two hours). We can see that the traffic keeps getting relayed for the entire duration of the session, and CreatePermission requests keep getting sent by firefox with success. At the end, the graceful disconnection is triggered by firefox closing the WebRTC data channel (instead of being closed due to traffic no longer being relayed). As opposed to the Twilio captures, the Refresh request with a lifetime of zero is successful: the Xirsys TURN server still knows about the allocation and sends back a success response, as expected.
It should be noted that the ICMP unreachable errors are normal because I think in this case firefox is no longer listening on the given port when the response comes back. In other words, it sends the Refresh request with a lifetime of zero and doesn't wait for the answer to come back.
For the time being, I have no other choice but to go back with Xirsys, but I would really like if the Twilio Network Traversal Service could be fixed. Let me know if you have more questions regarding the issue.
I have uploaded the wireshark captures here for reference.
EDIT: I have modified the webrtc demo page such that it doesn't close the connection when the ice connection state is set to 'disconnected'. Now I get the real disconnection when the ice connection state goes to 'failed'. However, it effectively didn't change anything, since in this case it takes just a few seconds more for the state to go from 'disconnected' to 'failed'.
Since I have new relevant screenshots and captures, I am updating the original question to clarify certain problems pointed out by Philipp Hancke:
First, here is a new capture with the ice connection state fix (the browser closes the connection only when the state goes to 'failed'):
It's interesting to see that this time, the session stayed up for a whole 18 minutes. This was taken on a saturday morning, so I'm guessing that the issue could be related to the current workload on the twilio servers. However, it failed in the exact same way as it always does so far for me. As a bonus, we even have a valid stale nonce response that is correctly handled by firefox.
However, if we take a different view of the same capture, we can see that the traffic stops being relayed for a solid 30 seconds before firefox considers the connection as being dropped and sends the Refresh request with a lifetime of zero. As in previous captures, the server responds with an Allocation Mismatch error, indicating it doesn't know which allocation firefox is talking about.
The last eight packets being sent are of the same size, so my guess is that they are retransmissions. After 30 seconds of retransmissions, it is likely that SCTP considers the transport as being dropped.
With regards to the refresh request with a lifetime of zero, I did a test where I close the connection early on, from the browser. In this case, the server recognizes the allocation and returns a success response:
The allocation mismatch is the easiest symptom to observe, but in my testing with my native application, I have seen similar errors with Refresh requests for non-zero lifetimes, and with CreatePermission requests (438 "Wrong nonce" error). However, since the browser closes the connection after 30 seconds of data not being relayed, it is hard to observe these errors with the current webrtc demo. If we could change that timeout to 10 minutes, we would see those errors as well.
Excellent problem description!
Without the server log this is hard to determine what goes wrong. I tried with the appear.in TURN servers which run an up-to-date version of coturn and show the same behaviour as the Twilio servers. Xirsys seems to be running a custom version of coturn (Coturn-0.5 'Xirsys Turn Services' from the software field but coturn never had such a version).
In both captures, firefox detects that traffic stops being relayed (at the data channel level) and attempts a graceful disconnection by sending a Refresh request packet with a lifetime of zero.
Not quite. A refresh request with a lifetime of 0 is used to discard an allocation. At that point it does not matter what the server returns as the connection is beyond repair anyway.
This is caused by peerjs closing the peerconnection if the iceconnectionstate changes to disconnected, here in your bundled library version.
This is overly aggressive (and does not even fix things) and we've had a discussion about what the specification should do wrt to trying to fix things with an ice restart here which also links to a great explanation of the disconnected state.
The disconnected state probably happens because a few packets get lost. But this is something that can happen when there is minor congestion. I'd recommend removing the pc.close() in the disconnected case.
If you are looking for other TURN providers, Tokbox provides the same service. For datachannels the latency of a properly run distributed TURN network does not matter as much as for VoIP so you might run your own servers in a single location instead.
I'm experiencing slow response times for my first http POST request to my server.
This happens both in Android and iOS networking libraries. (Volley on Android, and Alamofire in iOS).
First response is roughly 0.7s-0.9s, whereas subsequent requests are 0.2s.
I'm guessing this is due to the session being kept-alive by the server, therefore eliminating the need for establishing a new session on each request.
I figure I can make a dummy request when the app starts to start the session, but it doesn't seem very elegant.
I also control the server side (Node.js) so if any configuration needs to be done there I can also try it.
Investigating a little further, I tried sending an https CONNECT request before issuing the first "real" POST request, and the behavior replicates.
After 30 seconds or so, the connection is dropped (probably at the iOS URLSession level, the load balancer is configured to keep connections as 60 seconds).
In theory this makes sense because setting up an https connection takes up several (12 total) packets and I'm on an inter continental connection.
So my solution is to send a CONNECT request when I expect the user to send a regular request.
I have an app that lets the user send messages with images. A user might hit send, then immediately close their phone or switch to another app.
We were running into an issue that if there's temporarily a bad network connection the message would fail to send. We switched to using NSURLSession backgroundConfigurationWithIdentifier so that backgrounding the app doesn't immediately time out the running request. We switched to using this for all our api requests, thinking that it wouldn't hurt for every request to able to continue in the background if the app were closed at the wrong time.
Fast forward a couple weeks we're noticing all requests seem slow. Using wireshark I just discovered that this background session seems to use a new http connection per request, meaning it requires setting up a TCP connection and new TLS handshake for every request, which was adding a ~500ms latency on every request in our app. This is a pretty big deal but I can't find this behavior documented anywhere, including the link above or Apple's background transfer considerations.
So my question is, is this behavior expected, or am I doing something wrong somewhere? Is there an easy way with NSURLSession to make an HTTP request that will use an existing keep-alive connection if there is one, but can fall back to the backgroundConfiguration if the app gets moved to the background?
NSURLSession is the recommended way to fulfill your use case. Have you tried setting backgroundSessionConfig.discretionary = true
iOS Reference
A Boolean value that determines whether background tasks can be
scheduled at the discretion of the system for optimal performance.
If that doesn't help, I recommend filing a bug with iOS.
In the iOS Developers Guide for Parse, it states "By default, all connections have a timeout of 10 seconds." I am looking to change this for all requests made from the app, but am not finding any information on how to do so.
The reason we'd like to modify this is that it's taking a long time for requests to fail when the user doesn't have Wi-Fi or Cellular enabled. We want to reduce the amount of time it takes to receive said error message, just a little. We don't want to implement our own reachability tests, as it will result in duplicate popup error messages and we have many requests in various view controllers throughout the app.
Can the timeout be modified, or is there some other way to obtain a better user experience than waiting 10 seconds for an error message?
There is no information on this but certainly the request timeout limits are set by Parse and a developer will not be able to change them. I think they kept the timeouts to be long to avoid a user request being rejected if their connection becomes suddenly intermittent or they go in a tunnel, etc.
You can try to warp Parse queries around a timer which uses let say 5 seconds timeout, if the response does not come in that time you cancel your your query using PFQuery cancel function and show them a message.
If you want to avoid timing out, consider checking Reachability before making the call. You may want to show the user an alert if they're not connected and you need to do something.
A lot of people say you should just assume a connection and make the attempt without checking reachability; basically just let the connection fail and handle the error that way. I think as long as the failure isn't invisible to the user, so they don't blame your app vs their network you're good though.