I'm looking at the Apple networking guidelines that suggest that the user should try to work with the highest level of abstraction possible when dealing with networking.
I'm working on a client-server app, where the server is master, and an iOS device is slave. These communicate over HTTP, establishing a connection that lives for the lifetime of the app's usage session. The app and the server synchronize assets over this connection.
My question is - what level of abstraction is appropriate for implementing bi-directional sync over HTTP? Is it sockets, NSURLConnection, some AFNetworking subclass, input/output streams?
There are a lot of possible good answers to this. I think all I can do is offer one pattern which has worked well for me but it may not apply to your needs and use cases. To restate my comment above "whatever you do will be a tradeoff between responsiveness, power consumption, data consistency, and implementation cost."
The level of abstraction I aim for is a set of service objects which expose an interface in terms of the application's domain models. The rest of the app, primarily objects in the controller layer, should be able to communicate with these services by passing models to methods (e.g. "fetchUserWithId:userId" or "createUser:user") and without any awareness of the urls, paths, or HTTP verbs involved at the network layer.
Those service objects can map domain model operations into paths, HTTP verbs, and possibly request bodies or headers. In most cases I find that the services themselves can then share a lower level service which accepts those values and constructs the actual HTTP request. This provides a single location to configure host names, set global headers, and manage a request queue via NSURLRequest, NSURLSession, AFNetworking, or whatever library you prefer.
I'll include completion blocks on my service object methods so that controllers can be notified of success or failure but try not to use those blocks to pass models back up to the controller layer. Instead I prefer to have controllers monitor Core Data or some other persistence layer and react to changes. That way controllers remain flexible and respond to any update in the models they are concerned with and do not assume that they are aware of all possible sources of changes to those models.
So far none of this addresses how you should check for remote changes to your models. The best option may be to design a system which does not need to do so. What if your client obtained a set of recent changes only when posting data to the server, could it still provide a good user experience? Could the server use push notifications to occasionally notify clients of updates?
If you must check for changes sockets or long polling are usually more responsive than short polling but it may be hard for roaming mobile clients to keep those connections open. All of these approaches also tend to keep the client's radios active and consume lots of power in the process.
Without knowing more about the problem I'd default to short polling but try to design interactions which allow this to be as infrequent as possible (e.g. one check when the app resumes). I also use HTTP features (etags, if-modified-since, or custom content ranges) to limit the size of responses when there are no changes. If you have a good service layer managing network requests that also gives you a good place to introduce rate limiting. Allowing controllers to express interest to fetching up to date information but deferring to the services to throttle or batch requests based on what the rest of the app is doing (e.g. don't repeat the same request if those models were updated recently unless the user deliberately triggered the action).
Related
What are the core differences between service worker and AppCache. What are the pros and cons of each and when to prefer one over another .
The primary difference is that AppCache is a high-level, declarative API, with which you specify the set of resources you'd like the browser to cache; whereas Service Worker is a low-level, imperative, event-driven API with which you write a script that can intercept fetch events and cache their responses along with doing other things (like displaying push notifications).
The pros and cons are largely a function of API design: theoretically, AppCache is easier to use, while having more limited use cases; whereas Service Worker is harder to use, but is more flexible.
Nevertheless, AppCache is considered hard to use in practice due to poor design (see Application Cache Is A Douchebag for a list of design issues). And it has been deprecated, so it is being removed from browsers (per Using the application cache).
Thus the only reason to prefer AppCache is to offline an app on browsers that don't yet support Service Worker, as Kenneth Ormandy recommends in Don’t Wait for ServiceWorker: Adding Offline Support with One-Line.
Compare Can I use Service Workers? to Can I use Offline web applications? to see the differences in browser support. But note that browsers that support Service Worker, like Chrome and Firefox, are removing support for AppCache, so you'll need to implement both to offline your app across all browsers that support either standard.
In addition of what Myk Melez said, One of the main benefits of Service Workers against Application Cache is that Application Cache only works when user is disconnected from the network, so you can not manage situations of:
1- "slow network" - Your connection signal is strong, however some external entities (server, routes, etc) are delaying the transmission to your specific application.
2- "Lie-fi" (your phone shows is connected to a wi-fi or a cell network with low signal) so it seems to be connected when actually is not.
Service Workers is like a middle ware giving you control over the requests the browser is making, you can actually intercept the request and respond wherever you want, no matter you are connected or not. So you can implement "offline first" principle.
I have a software architecture problem.
I have to design an IOS application which will communicate with a Linux application to get the state of a sensor, and to publish an actuator command. The two applications run in a Local network with an Ad-Hoc WiFi connection between the IOS device and the Linux computer.
So I have to synchronize two values between two applications (as described in figure 1). In a Linux/Linux system, I resolve this kind of problem thanks to any publisher / subscriber middleware. But how can I solve this problem in an IOS / Linux world ?
Actually the Linux application embed an asynchronous TCP Server, and the IOS application is an asynchronous TCP client. Both applications communicate through the TCP Socket. I think that this method is a low level method, and I would like to migrate the communication layer to a much higher level Service based communication framework.
After some bibliographic research I found three ways to resolve my problem :
The REST Way :
I can create a RESTful Web Service which modelize the sensor state, and which is able to send command to the actuator. An implementation of a RESTful web service client exists for IOS, that is "RESTKit", and I think I can use Apache/Axis2 on the server side.
The RPC Way :
I can create on my Linux computer a RPC service provider thanks to the libmaia. On the IOS side, I can use xmlrpc (https://github.com/eczarny/xmlrpc). My two programs will communicate thanks to the service described in the figure below.
The ZeroConf way :
I didn't get into detail of this methods, but I suppose I can use Bonjour on the IOS side, and AVAHI on the linux side. And then create custom service like in RPC on both side.
Discussion about these methods :
The REST way doesn't seem to be the good way because : "The REST interface is designed to be efficient for large-grain hypermedia data transfer" (from the Chapter 5 of the Fielding dissertation). My data are very fined grain data, because my command is just a float, and my sensor state too.
I think there is no big difference between the ZeroConf way and the RPC Way. ZeroConf provide "only" the service discovering mechanism, and I don't need this kind of mechanism because my application is a rigid application. Both sides knows which services exists.
So my question are :
Does XML RPC based method are the good choice to solve my problem of variable synchronization between an iPhone and a Computer ?
Does it exist other methods ?
I actually recommend you use "tcp socket + protobuf" for your application.
Socket is very efficient in pushing messages to your ios app and protobuf can save your time to deliver a message instead of character bytes. Your other high level proposal actually introduces more complications...
I can provide no answers; just some things to consider in no particular order.
I am also assuming that your model is that the iOS device polls the server to synchronize state.
It is probably best to stay away from directly using Berkeley sockets on the iOS device. iOS used to have issues with low level sockets not connecting after a period of inactivity. At the very least I would use NSStream or CFStream objects for transport or, if possible, I'd use NSURL, NSURLConnection, NSURLRequest. NSURLConnection's asynchronous data loading capability fits well with iOS' gui update loop.
I think you will have to implement some form of data definition language independent of your implementation method (RES, XML RPC, CORBA, roll your own, etc.)
The data you send and receive over the wire would probably be XML or JSON. If you use XML you would have to write your own XML document handler as iOS implements the NSXMLParser class but not the NSXMLDocument class. I would refer JSON as the JSON parser will return an NSArray or NSDictionary hierarchy of NSObjects containing the unserialized data.
I have worked on a GSOAP implementation that used CFStreams for transport. Each request and response was handled by a request specific class to create request specific objects. Each new request required a new class definition for the returned data. Interactivity was maintained by firing the requests through an NSOperationQueue. Lots of shim here. The primary advantage of this method was that the interface was defined in a wsdl schema (all requests, responses, and data structures were defined in one place.
I have not looked at CORBA on iOS - you would have to tie in C++ libraries to your code and change the transport to use CFStreams Again, lots of shim but the advantage of having the protocol defined in the idl file. Also you would have a single connection to the server instead of making and breaking TCP connections for each request.
My $.02
XML RPC and what you refer to as "RESTful Web Service" will both get the job done. If you can use JSON instead of XML as the payload format, that would simplify things somewhat on the iOS side.
Zeroconf (aka bonjour) can be used in combination with either approach. In your case it would allow the client to locate the server dynamically, as an alternative to hard-coding an URL or other address in the client. Zeroconf doesn't play any role in actual application-level data transfer.
You probably want to avoid having the linux app call the iOS app, since that will complicate the iOS app a lot, plus it will be hard on the battery.
You seem to have cherry picked some existing technologies and seem to be trying to make them fit the problem.
I would like to migrate the communication layer to a much higher level Service based communication framework
Why?
You should be seeking the method which meets your requirements in terms of available resources (should you assume that the client can maintain a consistent connection? how secure does it need to be?) However besides functionality, availability and security, the biggest concern should be how to implement this with the least amount of effort.
I'd be leaning towards the REST aproach because:
I do a lot of web development so that's where my skills lie
it has minimal dependencies
there is well supported code implementing the protocol stack at both ends
it's trivial to replace either end of the connection to test out the implementation
it's trivial to monitor the communications (if they're not encrypted) to test the implementaiton
adding encryption / authentication does not change the data exchange
Regards your citation, no HTTP is probably not the most sensible for SCADA - but then neither is iOS.
we are currently trying to determine a application architecture for an application that will need to accept a number of SOAP calls and also make SOAP calls. One of the design goals is simplicity and robustness which we need to take into account.
In the Grails space we could all tie this into one big Grails application but this gives headaches in the robustness aspect as and update of the Grails application will disable all incoming SOAP request.
I was wondering if splitting up the Grails app and combining this with something like ActiveMQ/ServiceMix/Mule etc is recommend? Any advice or comments are appreciated! And what kind of solution woud be a good candidate?
You can achieve some robustness with your monolithic Grails app by running it behind a network load balancer. This would allow you to perform no-downtime rolling upgrades.
Now this doesn't address other concerns like the need to deal with possibly unreachable remote SOAP services, etc... This is when a tool/framework, like Mule, can become helpful as it will provide you exception handling, retries and whatnot.
This is conditioned by the intended behavior of your SOAP bridge: is it asynchronous (ie. fire and forget, send the message to the bridge, get an immediate ACK and let the bridge do the remote dispatch whenever possible) or is it synchronous (ie. the caller of the bridge is held until a remote response is received and forwarded back to it).
If your bridge is fundamentally synchronous, I'd say you can stick with your single Grails app and use a load balancer. It will be up to the caller to deal with retries.
Otherwise, if it's async, consider a messaging middleware to help with the temporary message persistence and redelivery in case of failure.
If you have a complex requirement set with many users(&servers) how will your websocket infrastructure (server[s]) will scale, especially with broadcasting?
Of course, broadcasting is not part of the any websocket spec but it's there even in basic chat examples (a.k.a. hello world for websocket).
Client side (asking for new data) solution still seems more scalable than server side (broadcasting) solution with websockets' low latency and relatively cheap (http headerless) nature.
Edit:
OK, just think that you want to replace all your ajax code with websocket implementations which may mean that so many connections within so many different contexts. This adds enormous complexity to your system if you want to keep track of every possible scenario for broadcasting.
Low (network/thread etc) level implementation suggestions are also part of the problem not the solution, because this means you have to code a special server unlike general http servers.
Moreover, broadcasting brings some sort of stateful nature to the table which can't easily scale. Think about adding more servers and load balancing.
Scaling realtime web solutions can be a complex problem but one that services like Pusher (who I work for) have solved, and one that there are most definitely solutions defined for self hosted realtime web solutions - the PubSub paradigm is well understood and has been solved many times and in order to solve the problem there needs to be some state (who is subscribing to what). This paradigm is used in broadcasting the the types of scenarios that you are talking about.
Realtime web technologies have been built with large amounts of simultaneous connections in mind - many from the ground up. If you wanted to create a scalable solution you would most likely use an existing realtime web server that supports WebSockets, in the same way that it's highly unlikely that you would decide to implement your own HTTP Server you are unlikely to want to implement your own server which supports WebSockets from scratch.
Dedicated Realtime web servers also let you separate your application logic from your realtime communication mechanism (separation of concerns). Your application might need to maintain some state but the realtime technology deals with managing subscriptions and connections. How communication between the application and the realtime web technology is achieved is up to you but frequently messages queues are used and specifically redis is very popular in this space.
HTTP polling may conceptually be easier to understand - you can maintain statelessness and with each HTTP poll request you specify exactly what you are looking for. But it most definitely means that you will need to start scaling much sooner (adding more resource to handle the load).
WebSocket polling is something I've not considered before and I don't think I've seen it suggested anywhere before either; the idea that the client should say "I'm ready for my next set of data and here's what I want" is an interesting one. WebSockets have generally taken a leap away from the request/response paradigm but there may be scenarios where the increased efficiency of WebSockets and request/response using them may have some benefits. The SocketStream application framework might be worth a look as it might be relevant; after the initial application load all communication is performed over WebSockets which means that event basic request/response functionality uses WebSockets.
However, since we are talking about broadcasting data we need to go back to the PubSub paradigm where it makes much more sense to have active subscriptions and when new data is available that new data is distributed to those active subscriptions (pushed). All your application needs to know is if there are any active subscriptions or not in order to decide whether to publish the data or not. That problem has been solved.
The idea of websockets is that you keep a persistent connection with each client. When there is new data that you want to send to every client, you already know who all the clients are so you should just send it.
It sound like you want each client to constantly be sending requests to the server for new data. Why? It seems like that would waste everyone's bandwidth and I don't know why you think it will be more scalable. Maybe you could add more detail to your question like what kind of information you are broadcasting, how often, how many bytes, how many clients, etc.
Why not just consider an open websocket connection to be like a standing request from the client for more data?
I'm in the planning phase of our new site - it's an extension of some mobile apps we've built. We want to provide our users with a central point for communication and also provide features for users who don't want to/can't use the mobile apps. One of the features we're looking at adding is a reputation system similar in nature to the SO badge system. We're designing the system to use SOA.
I don't want to have to code all of this logic into the main app as discreet chunks. I'm thinking of creating a means to accomplish this which will allow us to define new thresholds and rules for gaining reputation and have them injected into some service. The two ways I've thought of doing this so far are:
To look for certain traits in a users actions and respond, this would mean having a service running that can run through the 'plugged in' award definitions and check for thresholds that have been met and respond appropriately.
To fire events when the user performs actions - listen out for those events and respond appropriately. Because the services which will be carrying out these actions are running in separate app domains potentially on separate servers the only way I can see having a central message bus to listen and respond to these events is by using something like MassTransit, nServiceBus or Rhino.Esb.
I know that using a service bus can very easily be inappropriately designed into an application that simply doesn't need it and most times - unless you're integrating disparate, heterogenous systems - you most likely won't need one when designing a new system but I'm a bit lost for options as to the best way to do this. I don't like the idea of having a service hammer the Db all the time in the background. But it does sound like it might be a lot simpler early on - later on - I dread to think!
Has anyone here designed a system like this? How did you accomplish this? We're designing for high throughput as we expect there will be times when the system will need to be able to cope with bursts of users.
I've designed a system that had similar requirements. To achieve this the key elements were:
Plugins
Event messaging - using Emesary
The basic concept is that the core is not aware of exactly which module will perform any given task.
The messages are defined and at points within the system they are dispatched. The sender is not aware if the message is required. This effectively decouples vast chunks of the system.
So to perform a job some code is plugged in, that registers with the event messaging bus and will receive messages. When it receives a message that it needs to process it will process it.
The Emesary code is extremely small and efficient in the first instance I've called it (Emesary and you're free to use it; or from Emesary CodePlex
As the system becomes more complex it is possible that there are lots of events flying about, if you get more than 20k a second it was always in my design to add filtering and routing (implemented by the recipient interface being extended to allow a recipient to specify messages it wants to receive during registration). I've never needed to add this filtering because Emesary is sufficiently efficient that it is the processing of the messages that takes the time.
I've build a version of Emesary which bridges two Notifiers across disparate systems using WCF, Corba and TCP/IP. I investigated using RabbitMQ and decided it was possible to use this underneath Emesary if needed.
Base Class Diagram
Scalable server.
This is a fairly complex example however it shows where Emesary fits in. In this diagram anything with a drop shadow can have multiple instances and this is managed outside of what I'm trying to explain here.