I’m working on an iOS app that aggregates contents (i.e. events) from different services (Google Calendar, Basecamp…) using their REST APIs.
Instead of querying these services in the client, I wonder if it wouldn’t make more sense to do it server side. This would allow me to unify the different models beforehand, reduce the client side logic significantly, and IMO make it both more scalable and easier to maintain.
But I have no experience of this kind of backend and no idea where to start, or even if it’s a good idea at all? Do you have any idea? I was thinking of using Node.js and Express/LoopBack…
Thank you in advance for your help!
David
I am afraid this question will be soon marked as opinionated, and myself most likely be down voted... but maybe I will be on time to pass a little of my experience working on a back-end for my employer.
If you do not need to impose any control over who and when and what data is being pulled from those multiple external services it makes more sense to free yourself from the hurdle of developing and maintaining your own back end. Keep in mind the costs associated with having to support your own servers...
Additionally, having this logic on the client scales proportionate to the external services being used while with your own back end you will have to make sure that it is programmed to scale well, which could be a considerable undertaking if you never did it (not saying that it is impossible to learn in a reasonable timeframe).
Basically, what I am saying is it is always better to delegate security/control agnostic logic to the client side. This way by the way, you will be following along the lines of currently buzzing micro-services trend :)
Related
I'm looking for some resources on taking an existing monolithic Rails 3.0 application (35K LOC) and breaking it apart into an SOA design. Any books, blogs, screencasts, or example applications would be awesome.
The main questions I'm looking to answer are:
Is SOA even the right design?
Where do I start?
What are some common pitfalls I can avoid?
What should I be thinking about now vs what can I do later? (ie performance)
Some resources I've seen but not totally sure if they are the right places to start:
Service Oriented Design with Rails
RESTful Web Services
Enterprise Rails (not sure if this book is still relevant)
SOA Design Patterns (maybe too "enterprisey"?)
Is SOA even the right design?
It depends. Don't you hate these kinds of answers?
Breaking up your app to loosely coupled services using messaging or API calls would, by definition, be implementing SOA.
The beauty of it is that you can interchange service implementation without changing their interfaces and allow for independent deployments without having to bring down the whole app. Also, I'd implement the SOA via specialised API controllers that are versioned and expose custom state rather than their whole state which you'd reserve for authenticated users or role-based sessions.
The dilemma, from my experience, is whether to implement synchronous or asynchronous calls. Synchronous calls are obviously easier to program, but may leave your user hanging while they are being executed, and you'd have to handle timeouts for long-running queries. Watch out for database and web server timeouts.
If you implement asynchronous calls, let's say via ActiveMessaging or alike, youd have to handle callbacks or some kind of notifications to bubble up to your user. It also entails setting up primary and secondary message brokers and maybe some JavaScript or pollers to check for status. It's all fun though!
Where do I start?
I'd first see if it's "worth it": after all, SOA is cool, but does introduce multiple points of failure that you do not have currently.
If you think your broken up app will result in discrete services that are HA and will serve other projects, I'd start with "the druby book" and "service oriented" as you mentioned.
What are some common pitfalls I can avoid?
I think the biggest concern would be transactions across multiple services and the ability to roll-back the whole operation if a distant service fails. The problems begins if you are in some operation where you call A and it calls B and B calls C and C fails.
Who knows that C failed? How will you tell B and A to roll back? Can they? Do they save state? Tough questions for upfront design.
One other issue is that life gets complicated when you throw a workflow on top of your SOA: who's the keeper of the business process? Centralised or distributed? It's all absolutely cool stuff again, but heaviness creeps in, no? But that's life if you must move to SOA.
What should I be thinking about now vs what can I do later? (ie performance)
I'd factor out the obvious generic services that can be used in other apps right now. I would not over-SOA your environment to avoid adding points of failure and keep the ones SOA introduces to a minimum.
This is an excellent resource from my friend who is the CTO at Crowdtap. They did the same thing and it has really helped them massively improve the velocity of product development and give them better test coverage. Hope it helps https://www.youtube.com/watch?v=KsiQXAXsQDQ
I'm programming a mobile application in Sencha Touch with the backend being Rails. I've found that I've been separating the two more and more as I get deeper into Sencha: where Im basically at the point where Rails only functions as my model storage (database) and Sencha pulls in everything it needs via JSON - reproducing much of the logic already present in rails.
My question, is what do you advise when it comes to delegating functions to each application? I've implemented REST in my Sencha app so it can communicate User, and associated data and store it in the same format.
Is this the right way to go for User Session management? Should I give more power back to rails? IE : where do I store the session? Can I do it on the server? Should I do it as session storage management? Local storage? I just dont know.
I'd appreciate any advice. Thanks.
This is not exactly a specific answer to your question, but I would just like to add that I do think you're on the right lines, and I wouldn't be worried that you've overstepped the architectural line, so to speak.
The web is going from one of rendered documents (where the server did absolutely everything and the browser was essentially dumb) to one where the browser and server are more symmetric peers - and your challenges become more around keeping two fully-fledged MVC apps in sync!
(Arguably, we might see a world where servers become fairly dumb, relative to the richness of the applications on the client side. I guess this is just the next cycle of the thick-client/thin-client pendulum that's been swinging for decades ;-) )
But for mobile, this is not just some arbitrary computer science problem - the mobile device might easily have partial or sporadic network coverage, and so the ultimate test of your application design is to work out whether the user can continue to work on the app when the device is offline - driven into a tunnel, for example - and then re-sync back once the network is available again. A rich, responsive client is really the only way to go.
In that scenario, storing the session richly in the browser seems like a reasonable step. In fact, it's easier to keep session state in sync between a single client and a server than it might for some other types of data record (that might be being manipulated by multiple clients simultaneously).
How can i write a cloud-aware application? e.g. an application that takes benefit of being deployed on cloud. Is it same as an application that runs or a vps/dedicated server? if not then what are the differences? are there any design changes? What are the procedures that i need to take if i am to migrate an application to cloud-aware?
Also i am about to implement a web application idea which would need features like security, performance, caching, and more importantly free. I have been comparing some frameworks and found that django has least RAM/CPU usage and works great in prefork+threaded mode, but i have also read that django based sites stop to respond with huge load of connections. Other frameworks that i have seen/know are Zend, CakePHP, Lithium/Cake3, CodeIgnitor, Symfony, Ruby on Rails....
So i would leave this to your opinion as well, suggest me a good free framework based on my needs.
Finally thanks for reading the essay ;)
I feel a matrix moment coming on... "what is the cloud? The cloud is all around us, a prison for your program..." (what? the FAQ said bring your sense of humour...)
Ok so seriously, what is the cloud? It depends on the implementation but usual features include scalable computing resource and a charge per cpu-hour, storage area etc. So yes, it is a bit like developing on your VPS/a normal server.
As I understand it, Google App Engine allows you to consume as much as you want. The back-end resource management is done by Google and billed to you and you pay for what you use. I believe there's even a free threshold.
Amazon EC2 exposes an API that actually allows you to add virtual machine instances (someone correct me please if I'm wrong) having pre-configured them, deploy another instance of your web app, talk between private IP ranges if you wish (slicehost definitely allow this). As such, EC2 can allow you to act like a giant load balancer on the front-end passing work off to a whole number of VMs on the back end, or expose all that publicly, take your pick. I'm not sure on the exact detail because I didn't build the system but that's how I understand it.
I have a feeling (but I know least about Azure) that on Azure, resource management is done automatically, for you, by Microsoft, based on what your app uses.
So, in summary, the cloud is different things depending on which particular cloud you choose. EC2 seems to expose an API for managing resource, GAE and Azure appear to be environments which grow and shrink in the background based on your use.
Note: I am aware there are certain constraints developing in GAE, particularly with Java. In a minute, I'll edit in another thread where someone made an excellent comment on one of my posts to this effect.
Edit as promised, see this thread: Cloud Agnostic Architecture?
As for a choice of framework, it really doesn't matter as far as I'm concerned. If you are planning on deploying to one of these platforms you might want to check framework/language availability. I personally have just started Django and love it, having learnt python a while ago, so, in my totally unbiased opinion, use Django. Other developers will probably recommend other things, based on their preferences. What do you know? What are you most comfortable with? What do you like the most? I'd go with that. I chose Django purely because I'm not such a big fan of PHP, I like Python and I was comfortable with the framework when I initially played around with it.
Edit: So how do you write cloud-aware code? You design your software in such a way it fits on one of these architectures. Again, see the cloud-agnostic thread for some really good discussion on ways of doing this. For example, you might talk to some services on GAE which scale. That they are on GAE (example) doesn't really matter, you use loose coupling ideas. In essence, this is just a step up from the web service idea.
Also, another feature of the cloud I forgot to mention is the idea of CDN's being provided for you - some cloud implementations might move your data around the globe to make it more efficient to serve, or just because that's where they've got space. If that's an issue, don't use the cloud.
I cannot answer your question - I'm not experienced in such projects - but I can tell you one thing... both CakePHP and CodeIgniter are designed for PHP4 - in other words: for really old technology. And it seems nothing is going to change in their case. Symfony (especially 2.0 version which is still in heavy beta) is worth considering, but as I said on the very beginning - I can not support this with my own experience.
For designing applications for deployment for the cloud, the main thing to consider if recoverability. If your server is terminated, you may lose all of your data. If you're deploying on Amazon, I'd recommend putting all data that you need persisted onto an Elastic Block Storage (EBS) device. This would be data like user generated content/files, the database files and logs. I also use the EBS snapshot on a 5 day rotation so that's backed up itself. That said, I've had a cloud server up on AWS for over a year without any issues.
As for frameworks, I'm giving Grails a try at the minute and I'm quite enjoying it. Built to be syntactically similar to Rails but runs on the JVM. It means you can take advantage of all the Java goodness, like threading, concurrency and all the great libraries out there to build your web application.
This question contains a lot of background information, to make sure you fully understand why we are looking at these technologies.
The question is basically this:
For a large, spreadsheet-type, module that we need to develop for our webmodule for our application, are there any pitfalls we should know about if we decide to use Silverlight for it?
Issues we already know, and don't need any discussion/reminders about:
We're aware of the problems around using a plugin-type solution, which may or may not be installed on the users machine (and in some cases, probably can't be installed). These risks needs to be mitigated, but we're aware of them. Please don't get hung up on this.
We're a .NET company, so while ruby on rails and lots of other different platforms and architectures are good for this solution, they are not in the scope of the decision here. We have lots of code already written in .NET that we need to take advantage of, otherwise the project will never be finished regardless of platform.
Background
We have a web module for our application with employee-related information and some input forms. Our Windows desktop application is mostly a department leader type of application, to manage employees, but the web module contains mostly employee-centric functions. The web module contains mostly report-type webpages, to list information from the system, or input-forms.
The module we need to add now is more of a heavy spreadsheet type application. You change something one place, and something changes somewhere else, like sums, what is enabled/disabled, etc.
We know we can manage all of that with AJAX, but another issue here is that the application will potentially load a lot of database data in order to put the data in front of the user, and with a AJAXy solution, we're afraid that the request/response method here will have to reload quite a lot of information on every request, even to respond to seemingly easy questions.
A way to mitigate that would basically be to load information into a Session-object or similar, but that's a big no-no, so we'd rather not do that. This is a multi-user module, and some of the data is rather static, but some of the data is also going to have to be refreshed from time to time, so if 10 users loads a lot of data into the session, that's going to be a pretty big memory-hit.
We will be using ASP.NET (MVC) for this if we choose to go this route, that is, developing the module in pure HTML and similar technologies.
Then we looked at Silverlight, and would then load all the information down into the Silverlight application on the client. It would hold the current state, and would only need to touch the database to refresh some of the information, some of the time, instead, as we think the request/response model with ASP.NET (MVC) would work, on every little request.
But, since we have only done minor things with Silverlight, we're not that experienced with it, and we're afraid that some assumptions we might have, stated or unconcious, turns out to be wrong or flawed, which will make this project impossible or very hard to manage at some point.
For instance, just to take an example, is there a limit to how much memory the Silverlight application is allowed to load (I know, if I have to ask I can probably not afford it), for instance if there is a limit on 10MB, then that would be nice to know about before we're midway and start to load the really heavy data.
To make it simpler to give examples, let's just assume we're building a spreadsheet, that has so much data, that for the simple "changed a number here, what else changed", too much data from the database has to be loaded for a proper request/response model to be used, and if we move the entire thing to Silverlight, what will make that project hard or impossible?
Knowing about such things would at least give us the ability to consider if the price is acceptable.
In short, why should we not use Silverlight for this and instead go for ASP.NET (MVC)?
And again, "use Ruby on Rails instead", is not really an answer here. The options are ASP.NET (MVC) which we have experience with, or Silverlight which we don't but can gain.
Of course, if Ruby on rails, given that we'd have to start pretty much from scratch infrastructure-wise, and have to learn a new programming language, and framework, and download and learn a new IDE/tool, if it would still allow us to cut the development time in half, then please give us some information about how that might work, but I daresay that won't really happen here.
You should know that Silverlight (version 3.0) does not support any printing whatsoever, which to me sounds like a whopper of a showstopper for you (sorry, I couldn't resist). The good news is that full printing support has been added in version 4, but that is still in beta. Rumours say it should be out before the summer if everything works out according to plan, so if that fits with your roadmap I would use SL4 right from the start.
There are no memory limitations in Silverlight, but for the local storage (IsolatedStorage) mechanism there is a default limit of 1MB. But you can easily get around that by asking the users permission to increase the local storage space when he/she starts up the application. More on that here: Silverlight Tip of the Day #20 – How to Increase your Isolated Storage Quota.
(Edit)
Aside from the missing printing functionality that will be fixed in SL4 I cannot see any problems with your scenario. I would easily take the Silverlight route if I were you, especially since you already have extensive knowledge of .NET/C#.
For a rich interface as you've described, I would definately go with Silverlight or Flash rather than a html/javascript/ajax solution.
These technologies make for much better and consistent interfaces across platforms, you can buy in various components to speed things up and support things like copy-n-paste and code in a more structured way.
Another element is skills, if you have the skills to achieve it in a particular technology, then go with that.
To the answer you question the best way I can; you should not use silverlight if you decide to use flash.
HTH
I need to deploy a Delphi app in an environment that needs centralized data and file storage system (for document imaging) but has multiple branch offices with relatively poor inter connectivity. I believe a 3 tier database application is the best way to go so I can provide a rich desktop experience with relatively light-weight data transfer needs. So far I have looked briefly at Delphi Datasnap, kbmMW and Remobjects SDK. It seems that kbmMW and Remobjects SDK use the least bandwidth. Does anyone have any experience in deploying any of these technologies in a challenging environments with a significant number of users (I need to support 700+)? Thanks!
Depends if you are tied to remote datasets. If you aren't dataset bound then SOAP would likely be a good choice. Or, what I've done is write my own protocol that is similar to SOAP in nature. This was done before SOAP was standard and I'm glad I did - this gives you the ability to control more of the flow of data. It's given that if you have poor connectivity then you will be spending time supporting it. It's very nice if it's your own code you are supporting versus having to wait on a vendor. (Although KBM and REM are known to be pretty good vendors.)
Personal note: 700 users in a document imaging application over poor connectivity sounds like a mess. Spend the money on upgrading connectivity as it'll be cheaper in the long run.
Both kbmMW and RO SDK offer binary format, which is more compact than SOAP format,specially you are working with documents.
RO sdk seems to offer more GUI tools to help you doing your services.
Also give a RealThinClient SDK a look, it's a lightweight remoting framework.
But what ever framework you go with, your design of work will make it fast or slow, I have some applications working on slow 128kb lines, and it's working perfect without any user complain, but I don't do a large transfer for files.
One thing to remember...its not the number of users, but the number of them using the resources at the same time that will be the issue. Attempt to develop your application "server stateless" if at all possible, this will allow greater flexibility in the long term if you find you have to add more servers to the pool to support your customer base. The hardest thing about n-tier is scaling beyond the first server...plan on that from the start. Each request should not know anything about a prior request...or at the very least the request should have a way of passing the context so the server can look it up in a session table or something.
Personally, I would recommend RemObjects. I have used it with good results.
I don't know if it's the very best / most efficient (glad you asked this question!), but I've had good results w/RemObjects SDK + DataAbstract. The latter made much of the plumbing details less involved, which was helpful. Still implementing, but so far so good.
If you really wanna go "low-bandwidth" use BSD Sockets API - that'll give you full control over what's being sent and there you can send as little information as you want. Of course then you'll have to implement all the tiers yourself, but hey - that's still an option :D