How can i write a cloud-aware application? e.g. an application that takes benefit of being deployed on cloud. Is it same as an application that runs or a vps/dedicated server? if not then what are the differences? are there any design changes? What are the procedures that i need to take if i am to migrate an application to cloud-aware?
Also i am about to implement a web application idea which would need features like security, performance, caching, and more importantly free. I have been comparing some frameworks and found that django has least RAM/CPU usage and works great in prefork+threaded mode, but i have also read that django based sites stop to respond with huge load of connections. Other frameworks that i have seen/know are Zend, CakePHP, Lithium/Cake3, CodeIgnitor, Symfony, Ruby on Rails....
So i would leave this to your opinion as well, suggest me a good free framework based on my needs.
Finally thanks for reading the essay ;)
I feel a matrix moment coming on... "what is the cloud? The cloud is all around us, a prison for your program..." (what? the FAQ said bring your sense of humour...)
Ok so seriously, what is the cloud? It depends on the implementation but usual features include scalable computing resource and a charge per cpu-hour, storage area etc. So yes, it is a bit like developing on your VPS/a normal server.
As I understand it, Google App Engine allows you to consume as much as you want. The back-end resource management is done by Google and billed to you and you pay for what you use. I believe there's even a free threshold.
Amazon EC2 exposes an API that actually allows you to add virtual machine instances (someone correct me please if I'm wrong) having pre-configured them, deploy another instance of your web app, talk between private IP ranges if you wish (slicehost definitely allow this). As such, EC2 can allow you to act like a giant load balancer on the front-end passing work off to a whole number of VMs on the back end, or expose all that publicly, take your pick. I'm not sure on the exact detail because I didn't build the system but that's how I understand it.
I have a feeling (but I know least about Azure) that on Azure, resource management is done automatically, for you, by Microsoft, based on what your app uses.
So, in summary, the cloud is different things depending on which particular cloud you choose. EC2 seems to expose an API for managing resource, GAE and Azure appear to be environments which grow and shrink in the background based on your use.
Note: I am aware there are certain constraints developing in GAE, particularly with Java. In a minute, I'll edit in another thread where someone made an excellent comment on one of my posts to this effect.
Edit as promised, see this thread: Cloud Agnostic Architecture?
As for a choice of framework, it really doesn't matter as far as I'm concerned. If you are planning on deploying to one of these platforms you might want to check framework/language availability. I personally have just started Django and love it, having learnt python a while ago, so, in my totally unbiased opinion, use Django. Other developers will probably recommend other things, based on their preferences. What do you know? What are you most comfortable with? What do you like the most? I'd go with that. I chose Django purely because I'm not such a big fan of PHP, I like Python and I was comfortable with the framework when I initially played around with it.
Edit: So how do you write cloud-aware code? You design your software in such a way it fits on one of these architectures. Again, see the cloud-agnostic thread for some really good discussion on ways of doing this. For example, you might talk to some services on GAE which scale. That they are on GAE (example) doesn't really matter, you use loose coupling ideas. In essence, this is just a step up from the web service idea.
Also, another feature of the cloud I forgot to mention is the idea of CDN's being provided for you - some cloud implementations might move your data around the globe to make it more efficient to serve, or just because that's where they've got space. If that's an issue, don't use the cloud.
I cannot answer your question - I'm not experienced in such projects - but I can tell you one thing... both CakePHP and CodeIgniter are designed for PHP4 - in other words: for really old technology. And it seems nothing is going to change in their case. Symfony (especially 2.0 version which is still in heavy beta) is worth considering, but as I said on the very beginning - I can not support this with my own experience.
For designing applications for deployment for the cloud, the main thing to consider if recoverability. If your server is terminated, you may lose all of your data. If you're deploying on Amazon, I'd recommend putting all data that you need persisted onto an Elastic Block Storage (EBS) device. This would be data like user generated content/files, the database files and logs. I also use the EBS snapshot on a 5 day rotation so that's backed up itself. That said, I've had a cloud server up on AWS for over a year without any issues.
As for frameworks, I'm giving Grails a try at the minute and I'm quite enjoying it. Built to be syntactically similar to Rails but runs on the JVM. It means you can take advantage of all the Java goodness, like threading, concurrency and all the great libraries out there to build your web application.
Related
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 years ago.
Improve this question
I'm planning the basic architecture for a new software that will need to be modular.
I'm trying to define a multitenancy application to have a single instance running for all the users.
What I need is the possibility to scale when and where needed, so I don't like the idea to spawn multiple applications (monolitical architectured) behind a load balancer when it could be a single part of the computation that needs more resources.
So I'm thinking about a Service Oriented Architecture, it would have the rails application as web client and other services, that could be virtually written in any language and accessed via APIs by the rails application.
I'd also like to have this APIs open to users to integrate with their existing software and easily extend these services.
I've some specific question:
would it be a good idea to have this kind of architecture for a new startup (1-5 employees)?
using APIs i don't need to use any RPC since the API request itself is an RPC, am I right on this concept?
what would be a nice standard for the APIs (REST only defines HOW to access resources)?
what could be, pratically, the best (= a good) way to expose those APIs to customers? Via the Web Rails application? Directly via a proxy that makes them all available under the same domain? APIs would be accessible in a RESTful way so via HTTP requests.
with this kind of architecture would it be less expensive to have VPS's, Cloud, or dedicated servers? I like clouds because of their failure-tollerant nature, it would free us from worry about data persistence and backups (including the fact that we want to build an architecture almost 100% available).
Any other suggestion or point of view, and any simply start point to think about this would be very appreciated.
I know very well Python, C/C++, JS, Perl, other pl and I started recently with Ruby/Rails. I'm choosing this last one because it seems to me that this community is strongly oriented in building services and what I mind (before that extreme performances) is the ability to learn asap and have someone to share experience with and to learn from, also with pratical examples (I know it's about an architecture, not PL that implements it, but I think it would be more easy to get it wrong in an immature environment that is still working with web1 or web2.0 style in mind).
P.S. I also need to write the basic architecture design, do you have any template where I can start from? I do need to share it with my team and other very-expert pros, I'd like to have it complete and easy to understand.
Hope to read some good suggestions here guys!
Thanks,
Alex.
Architecture
Here is an example stack that would I think mostly do what you're looking to accomplish:
Cluster
One or more app servers
One or more database servers
Zero or more job servers
Instances
Chef for configuration
Unicorn or Passenger
Nginx
Application
Ruby on Rails
Check out Grape for simple APIs
More Specific Answers
would it be a good idea to have this kind of architecture for a new startup (1-5 employees)?
If done correctly, this approach can be very stable and robust. What you don't want to do it get into a situation where you are spending all your time managing your servers. You want to get it up, be able to deal with problem instances quickly, and work on making your application do stuff. If you do it right, creating instances can be simple and totally automated.
using APIs i don't need to use any RPC since the API request itself is an RPC, am I right on this concept?
Yes.
what would be a nice standard for the APIs (REST only defines HOW to access resources)?
Here we'd need a bit more clarification on how you need to use RESTful design to accomplish specific goals.
what could be, pratically, the best (= a good) way to expose those APIs to customers? Via the Web Rails application? Directly via a proxy that makes them all available under the same domain? APIs would be accessible in a RESTful way so via HTTP requests.
A domain (or subdomain) should be accessible via HTTP and RESTful software design. It might return JSON or something else. It's all up to you.
with this kind of architecture would it be less expensive to have VPS's, Cloud, or dedicated servers? I like clouds because of their failure-tollerant nature, it would free us from worry about data persistence and backups (including the fact that we want to build an architecture almost 100% available).
You get what you pay for. I'd recommend cloud servers. Check out Heroku to get started, or Rackspace if you are prepared to "roll your own." Or Engine Yard.
Any other suggestion or point of view, and any simply start point to think about this would be very appreciated.
I would try creating a test API using something like a free Heroku account.
I am beginning work on an individual project to bring an existing product out of the dark ages of classic ASP and into the light. My biggest decision to make before embarking on this lengthy journey is determining what frameworks and methodology I will implement for the new design.
Right now I am looking at MVC or MVVM (from what I gather this is just Silverlight?) for the web interface, Entity Framework or something I write myself as the model and MSSQL as the data.
Unfortunately I am just a fledgling programmer and I am not particularly aware of trends in the world of programming in general. I don't know what is just a passing fad and what technologies actually have lasting potential. I would really like to use something that is likely to remain relevant for some time. So I am looking to the professionals here for input on ideas that worked for you, pitfalls to watch out for and things to keep an eye on.
I appreciate any and all suggestions, keeping in mind that using the Microsoft and .Net is something of a prerequisite. I really want to make sure I am headed in the right direction before I start as this will probably take several months.
As for frameworks I personally suggest:
ASP.NET MVC 3 of MVC 4, depending on the question if beta software is allowed.
Entity Framework 4.3 or 5.0. 5.0 is a lot faster (is has auto compilation) but it's still a Release Candidate.
AutoMapper to map between Entities and ViewModels.
Ninject for dependency injection (useful if you want to write unit tests).
JQuery for stuff like clientside validation (integrates perfectly with ASP.NET MVC).
Possible some CSS framework like Bootstrap.
Maybe RestSharp so you can easily perform requests.
In case it's a cloud service (most SaaS are) and you'd like to host it on Azure (brilliant integration with the .NET stack) you'll need the Azure SDK.
As for software achitecture:
Use service layers
Use the repository patterns
Use ViewModels to pass to your view instead of entities
Set up a dependency injection container
That's my advice, I personally find this a golden combination for building enterprise applications (while not wasting too much time configuring lots of things).
Pitfalls:
I don't know if unit testing is really necessary. I should definately keep it in mind while setting up the architecture, but I personally choose to do that later because I don't even know if my product will succeed, so I can better put my time in building a fast Minimal Viable Product.
Don't assume anything. You can waste months of your precious time working on a cool feature that you think everyone will like, but often this is not the case. Do just the absolutely required minimum, and improve it later if your users like it.
I will add more to #Leon suggestions as I see those suggestion are great from application framework perspective, while I wanted to write here from cloud methodology perspective.
As you have chosen SaaS, definitely you are moving completely in Cloud while bring your application and data to cloud all together, that's great!!
There are several layers to any cloud application and to understand lets see what a cloud service stack look like. If we take an example of Windows Azure:
You have Compute, where your application runs with a web server (or not).
You have Azure table store which you can use to store key value pair in a row and then access them very fast.
You have Azure Queue allows decoupling of different parts of a cloud application, enabling cloud applications to be easily built with different technologies and easily scale with traffic needs.
You have Access Control Services to authenticate users through OpenID or AD
You have service bus to connect other services in cloud or on-premise at 3rd party.
You have Azure Blob storage to use as web based flat file server
You have Azure Cache (an in-memory cache build to scale in cloud)
You have SQL Azure as you cloud database
There are many more services which you can explorer and use
So when you decide to move your application from traditional web hosting to cloud you really have to look about how to take advantage of these different cloud services to scale your application when needed and save you lots of money.
With you application in Cloud you try something as below:
Keep you application logic as small as possible
Keep your static content outside the compute
Use cloud based cache for fast access as application scale out
Move data out of traditional RDBMS databases to NoSQL Framework (key-value pair, document etc to save money and flexibility), if possible and applicable
Take advantage of other available services to reduce application complexity
If you consider above aspect in your mind you will create a true cloud based application which will be fast and will save you money.
I am trying to develop an iOS application that stores and loads data to and from a server. The data needs of the client can be pretty much narrowed down to REST. MY question is, is REST something widely used for data/server driven iOS applications? is there a paradigm proven better or more suitable for iOS apps?
if REST is the way to go, what server environment would you choose? what server side Technology? PHP? Java? something else? We'd set up a test/dev environment at first, but eventually we are going to deploy on services like Amazon cloud or any other hosting/cloud service.
Any insight will be most welcome.
So are you writing the backend too?
Most projects that I've worked with in the past few years use REST. It's made little difference in the implementation of the iOS app (I think...). I'm more concerned about the type of data I'm consuming, which is usually JSON - and ensuring that it is as lean as possible. People writing web services should be concerned about REST.
I've also worked with projects this year that use different technologies on the backend. Java on one, and Ruby on Rails on the other.
I know another guy who uses PHP on one project, and Ruby on another.
If I was to do a backend, I'd probably use Java - solely because I know it. Not the best excuse to pick a technology - but as a full time iOS developer, I don't have time to learn something new that I won't use very often.
If you are going to deploy on a cloud based service, see what technologies they support. Maybe picking the common denominator would be a wise choice to keep your options open. Some languages tend to have more expensive server hosting costs. Java is usually more expensive than PHP to host, I guess due to the complexities of running shared java VM's instead of PHP interpreter.
We are building an image and file hosting website and we will save these files on our servers, so I want to know if there are any best practices or standards I need to read and follow to make our website scalable and easy to extend in the future.
Is there a book or articles or videos talking about this subject, please share.
As per my experience to deal with large data.
its always best to opt for Cloud, check for "Amazon S3" (Amazon AWS) or Windows Azure.
features like "CDN" (cloud front) is a big plus.
I believe this is not a simple question that can be answered without knowing
how many files are expected ?
how many users/files accesses per hour/day/minute ?
your usage scenarios with this files (downloading? streaming? how many concurrent files downloaded at once?
are you stuck in one particular OS (windows) and filesystem (NTFS), or is there freedom in this ?
My personal note : Building own image/file hosting is not a trivial task, i strongly recommend you to hire somebody with experience from this area.
I would recommend that if possible, you look at a 3rd party solution that provides an api. you'll then get the benefits of lower cost of ownership, no maintenance costs for the hardware and continual updates thrown in for free when the 3rd party adds new features to the core offering. I know this from 1st hand experience as we scoped out the options for doing this in a recent project and came to the conclusion that we'd spend 100 times more on our own solution and even then, may not get it right. We opted for a company called Razuna who offer both a hosted and open source version of their platform. Their api is very straightfwd and can be consumed inside your mvc app with potentially only a few days effort (depending on your use case). The beauty of this approach is that the hosted elements are actually on the nirvanix backbone and are served via their CDN - so win win.
You can get the details at:
http://www.razuna.com
and can view the api docs at:
http://wiki.razuna.com/display/ecp/Developer+Guides
Good luck and if you need any further real-life guidence on this, feel free to come back. Oh and btw, we were also able to ask for 'paid for' features to be added to the core offering at pretty much standard market day rates.
What language, framework, and hosting considerations should one make before starting development of a scalable web application?
The most important consideration is not to over-engineer to the point that it gets in the way of building and launching something. Analysis paralysis is the single biggest inhibitor to productivity, progress and results.
Yes, do some planning. Pick a framework. Perfection in a framework will be impossible to find because it doesn't exist, partially because you don't know what you need until you build it anyways. Chances are, if you pick something, it will be better than picking nothing.
Yes, try to pick flexible, inter-operable tools for where you see yourself going.
Yes, look for a good built-in feature set where you see yourself going in the next 6-18 Months. Trying to look beyond that is not really realistic anyways as most projects change so much anyways going towards the first release.
So, pick what you're comfortable with or what is familiar. Don't follow the crowd, do what gets you the best results, quickest, and often. Understand that you might have to change in the future. So, whatever you build now, try to use unit testing so you can re-factor if ever needed.
If what you're building is going to be super successful, it will be a great problem to have, and an easy one to work on once it's making money as you'll be able to get other talent to help you.
Share what you end up picking and why for your situation -- it helps the us learn from you too!
Don't necessarily marry yourself to one language or framework. It may be that some parts of your site work better with different languages and frameworks than others. For example, all of 37signals' sites are based on Ruby on Rails, but they recently wrote a blog post about how the underlying technology of one is actually written in Erlang now because it's much easier to do concurrency that way.
Obviously there's a level of complexity where things turn into a mishmash, but using the right tool for the job — even if that means different tools for different jobs — can simplify things.
Firstly on language, it largely doesn't matter. PHP, Java and .Net being probably the biggest three are all proven in the sense that they run some of the largest sites on the Web so don't listen to anyone who tells you one is more suited than any of the others.
Some might also put Ruby and Django/Python in this list. I have nothing against them but I'm not aware of any big (say top 50) sites using either.
Hosting considerations depend on how low you want to start but basically the order is:
Shared;
Virtual Private Server;
Dedicated.
Scalability will largely be about your application's design than any language, framework or provider. Efficient database schema, efficient delivery and use of Javascript/CSS and in-memory caching are all issues common to any language or framework.
Language - I'd recommend something with good frameworks and good testing libraries like Perl or Java.
Framework - it depends on what do you plan to do. If you start with a hosting that does not allow FastCGI, it is best to avoid such frameworks like Catalyst or Rails. That's why I love CGI::Application (primarily Perl, but ported to other languages too) - it can run as CGI, FastCGI or mod_perl. For development it can be run from it's own web server.
Hosting - nothing is better than you own server. It can be your own server, leased server or virtual server. But you can start with cheapest hosting and when you need more, you should be able to afford it.
It depends.
Start by looking at your requirements (Functional or user defined) (Non Functional - aspects that describe your desired system link text)
Next I would clarify what it means to have a scalable web application. Define it as test cases that can be clearly tested (must support X page views / second with response time < Y seconds).
Once I had those pieces in place I would look at what type of skills my development team can support (for the intial project and on going maintenance). Then find some case studies of applications out in the wild that use similar language or framework. If someone else has made a specific language / framework scale then chances are good that you can too.
Finally go out and look for some hosting providers that support your chosen language, framework and requirements.