I have been using Heroku up to now, but need to offer something more "enterprise-y" to an organisation that would otherwise insist on local server deployment.
AWS VPC sounds like the ideal infrastructure to use. I'll have Rails + Postgres + AWS S3.
Maybe a silly questions, but is there an automated PaaS that runs on top of AWS? Such that we own the AWS account but can outsource the server admin side? Or is this just a case of having to hire a consultant and deal with upgrades etc. as they arise?
is there an automated PaaS that runs on top of AWS?
Yes, there are LOTS of them. They range in spectrum from "a thin layer on Elastic BeanStalk" to wanna-be Heroku clones. (From my experience, most don't think about HA enough, so it works great, but has plenty of Single Points of Failure that can bite you down the road. But maybe you don't need so many nines of availability at first. Anyone remember Twitter in the early days?)
There are plenty of points in between, such as EC2 Containers, OpsWorks, Flynn, CoreOS, etc. Even Empire which I haven't looked at yet.
Every PaaS is a "Straitjacket" that reduces your "freedom to do anything" in exchange for "simplifying some things". That trade-off is not a constant, nor universal.
A PaaS that works for one company may not work for another, or even the same company six months later. I frequently see companies use Heroku or Google AppEngine for months/years, then suddenly hit a wall and need move off of it. After, they wonder why they put up with the limitations for so long.
we own the AWS account but can outsource the server admin side?
Yes, you can easily pay someone to setup a "PaaS" for you. The problem is they have to have a lot of knowledge to know which one will work best. (Few people have tried them all and talk intelligently about the trade-offs between them all. And new ones get written practically every day.)
The one universal thing: Companies rarely outgrow AWS. (Remember Zynga? They thought they were "spending too much money" on AWS, so they built their own datacenter. It supposedly "saved them money" until their fortunes turned around. Then that datacenter became a boat anchor that pulled them under. Ooops.)
Or is this just a case of having to hire a consultant and deal with upgrades etc. as they arise?
You need to think deeply about what it is that you want.
You could stay high level and run on a 3rd party PaaS. There are no shortage of them (Engine Yard, Heroku, etc). You trade a bit of extra money now (and it's really a few bucks per month) for speed. But you pay a cost later when you have to move off. That's OK, as long as it bought you some time to prove your business model, get customers, etc.
You could hire a consultant to build you a PaaS on AWS. (Why? Is it merely to save money? To gain flexibility? How will you know they are competent? How do you know they aren't a one-trick pony building the same solution they built for everybody else, without knowing where it will work and not work?)
You could bite the bullet and hire (or grow) a cloud guy. They learn AWS and iterate. This will give you the most flexibility in the future. You will be prepared to take advantage of cost savings offered by tuning your application to AWS. (A generic PaaS doesn't encourage you do to this, which means you will loose out of performance or cost savings.)
Long term, I think only the last one makes sense. The 3rd party PaaS is also a fine way to gain speed in the short-term. I'm not so keen on a consultant, unless it's to help accelerate knowledge transfer to your company.
Related
We are writing our first micro services using Docker containers using Amazon fargate. We have many doubts on the implementation level using Spring Boot
We will have multiple micro services in the project, is it a good practice we are writing all the micro services in a single container or I have to create separate Docker container for separate micro services. In a cost effective way we use single container but is that make any problems for our project structure in future?
We are planning to deploy the application in AWS fargate and our application will have large option to extend in future and expecting around 100 to 150 different micro services. In this case is it cost effective if we are uploading all these microservices in different containers too?
The most important thing to remember with microservices is that they're not primarily about solving technical problems but organisational problems. So when we look at whether an organisation should be using microservices, and how those services are deployed, we need to look at whether the org has the problems that the microservices style solves.
The answer to your question about your architecture, then, will mostly depend on the size of your technology team, the organisational structure, the age of your product, your current deployment practices, and how those are likely to change over the medium term.
As an example, if your organisation:
has less than 25 tech staff,
organised into 1 or 2 teams,
each of which works on any part of the product,
which is less than 12 months old,
and is deployed all at once on a regular basis (e.g. daily, weekly, monthly),
and the org isn't about to grow rapidly,
then you almost definitely want to forget about microservices for now. In a situation like this, the team is still new in learning about the domain, so likely don't know everything they'd need to know to really understand what would be a great way to split the system up into a distributed architecture. That means if they split it up now, they'll probably want to change the boundaries later, and that becomes very expensive when you already have a distributed system, while being far simpler in a monolith. What's more, with only a small team who can all work on (and support) any part of the system, there's little reason to invest in building a platform where individual teams can deploy and maintain individual services. An organisation at this stage will typically be far more concerned with finding customers and iterating the product quickly, perhaps even pivoting the product, as opposed to making teams autonomous and building a high-scaling, resilient architecture. A monolithic architecture makes sense at this point, but a well-designed monolith, with clear component boundaries enforced by APIs, and encapsulated data access, making it easy to pull out services into separate processes later.
Let's look a little further on and consider an organisation that is...
over 50 tech staff,
organised into 7 teams,
each of which works only on specific areas of the product,
which is 3 years old,
and has teams wanting to deploy their work independently of what other teams are doing.
Such an organisation should definitely be building a distributed architecture. If they don't, and have all these teams working in a monolith instead, they will run into all kinds of organisational problems, with teams needing to coordinate their work, releases being delayed while the one team finishes QA on their new feature, patch deploys being a big hassle for staff and customers. What's more, with a mature product, the organisation should know enough about the domain to be able to sensibly split both the domain and the teams (in that order; see Conway's Law) into sensible, autonomous units that can make progress while minimising coordination.
You seem to have chosen microservices already. Depending on where you sit on the scales above, maybe you want to revisit that decision.
If you want to keep developing with microservices but deploying them all in one container, know that there's nothing wrong with that if it suits the way your organisation works at the moment. Will it make problems for your project structure in future? Well, if you're successful and your organisation grows, there will probably come a time where this single-container deployment is no longer the best fit, in particular when teams start owning services and want to deploy just their service without deploying the whole application. But that autonomy will come at the cost of extra work and complexity, and it may give you no benefit at this point in time. Just because it won't be the right approach for your system in the future doesn't mean that it isn't the right approach for today. The trick is in keeping an eye on it and knowing when to make the extra investment.
No problem if you are using single container for your microservices but the main goal of microservices is to maintain each services separately each service should be loosely coupled and each service should have separate database (if you want to achieve database per service architecture).
So try to achieve this thing run your services in separate container and orchestrate those services with docker swarm or Kubernetes .
I know cost matters but if you do it in right way you will then see the power of microservices architecture then.
So I have a lot of smaller clients that want more simple websites. This is very common and even more common is the mess of code that I get from their last developer who used wordpress. Since I am trying to hone my rails skills, I would prefer to just redo the site in rails. The biggest problem I've encountered though is server hosting costs.
It is really hard to convince someone to go from their $5-10 a month cost for their servers to $34 with something like heroku or amazon EC2. So my question is how can I effectively use rails to rebuild these wordpress sites without incurring more costs on the client?
Some options I have thought of are:
Find a shared web hoster and figure out how to get rails on it (doesn't seem like a good idea due to lack of rails support by these host sites like godaddy, host monster, etc).
Use a scalable server like EC2 and put multiple apps on it, and since most of the sites are low traffic it should balance out to a reasonable cost.
Submit to the popularity of wordpress
Some of the example sites are a store, blog/store, and a marketing website.
There's nothing wrong with honing your skills on the job but you also have to be considerate of the clients needs.
Just because you'd prefer to redo the site in rails doesn't mean that's the best thing for the client. If their site is running smoothly on their cheap hosting do they really want to increase their monthly outgoings to support a platform that they didn't want in the first place.
There's very little functionality the client might need that they can't get from WordPress and the examples you gave certainly don't suggest rails is necessary.
I'd suggest keeping them on WordPress and seeking clients who need a rails developer.
Here's my 2 cents. Stick with WordPress. If you're trying to hone your Rails skills, start a side project. But if you try to convert these sites to Rails, you're going to be incurring a lot of cost to the client (your time), and hosting. As you mentioned, hosting a Rails app costs much more.
It's hard to justify that cost just to hone your skills unless you could convince the client that, in the long term, they will save a great deal of money, either by the reduced overhead on code maintenance, or they're trying to do something that Rails can do that WordPress genuinely can't. With the ubiquity of high quality WordPress plugins, you'd really have to dig for a functionality that you absolutely cannot do with WordPress that you can with Rails, though there are some.
Sure, Rails is much more pleasurable to work in than WordPress, but if these sites are for clients, you can't just factor in developer happiness. You need to be adding value for your customer, and building a site in Rails because it's more fun to work in adds little value for your customer.
Honestly, if they're a few static sites, then keep them on WordPress. You could start new clients out on a Rails-based CMS (or a custom solution, if you can sell them on it and you're sure you can make it worth their money), but my advice would be that it's not worth the cost to you or to them to build the sites in Rails just for the sake of building the sites in Rails.
I´m trying to estimate the monthly cost involveld in a SAAS app Rails based. Is it calculated per bandwitch, database, space, or what? If so how much per user/month we will be charged? And what is the best SAAS solutions avaliable for Rails/Ajax like apps? We are an brazilian startup, based at Sao Paulo.
Sorry but i have no idea about how to calculate the monthly costs and i need that to our business plan.
The software will be an web app that helps teachers and students to share information in real time.
Thanks and sorry by my poor english!
The amount you pay for hosting generally depends on the needs of your application. I would recommend starting with a cloud hosting provider to give you the flexibility you might need early on. Cloud hosting is great because you can easily scale up and down to fit your needs. There may come a time where it makes sense to move to a dedicated server, but it won't happen over night.
As far as cloud hosting providers go, there are tons out there but these seem to be a few of the more popular ones. I personally use EngineYard and am happy with them, but each offers different pricing and options, so you will want to determine what your specific needs are first.
Heroku
EngineYard AppCloud
Rackspace Cloud
As far as including pricing in your business plan, I would recommend having a range, rather than a fixed monthly cost. Starting off with a low amount of traffic will cost less than $100/mo. As your needs change, this can go up into the thousands or higher.
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
I'm trying to decide between Heroku and Engineyard.
Heroku seems so much better but they charge for everything and their prices are crazy!
Why one should use Heroku over EY or vice versa?
Heroku makes setup and launching of an app super-simple. You will be dependent to some degree on versions that Heroku supports (for instance, I heard today of a bundler versioning issue).
One thing to take into account with any "managed full-stack" solution like Heroku or EY is cost. You don't have to hire an ops person or have ops expertise, but you're still paying. Storage is where things get really expensive. Crank up your DB to a more than a few GB and watch the price go up.
We have in-house ops (was me doing it while coding, now a dedicated person) and run on Joyent. A big cost savings was having a few master-slave DBs and sharing them among a few dozen applications. We essentially have 100 Facebook apps running on Joyent at the same cost as 10 apps on Heroku. But this doesn't take into account the ops salary/time.
Everyone's needs are different, but the great thing is that its easy to experiment with these cloud deployment tools in quick fashion, and you will find that they each have their own strengths that you can leverage as you need.
What is most valuable to me, and my smaller clients, is to be able to experiment and get end-user feedback quickly. I have startup clients that want to be able to push out new ideas and test them quickly, deploy different combinations of ideas to different markets, get customer feedback, and keep moving forward. Launch a facebook app, a test server for an API integration client, a lightweight 'freemium' version of a product, etc. As traffic picks up, we make changes to scale up, and the increase in cost is never out of bounds (eg. our hosting costs are still well under the increase in value/revenue/marketing juice, etc).
EngineYard lets you play around with 500hrs for free, and you can easily turn it off when you are not using it, to stretch the 500hrs out. You can deploy your app quickly, deploy a CI server (that updates the app on every successful build), create a backup of your app or 'staging' server and see how it goes.
Amazon will give you 750hrs per month for free, for a year, if you are a new AWS customer. You can use this for a super fast CI server, hard-core image processing, batch reporting, whatever.
Personally I happen to use Heroku the most, as it just seems to work the best for my needs. I can put together a new application with full monitoring, backup, analytics, email, etc really fast, and feel confident in how to manage my setup (and confident that I can bring another person on board, and their learning curve will be pretty easy). As a freelancer, my use of Heroku has brought my setup time down to almost nothing, so I'm able to focus my time on understanding the business, and developing a great product. I'm not saying that can't be done on other platforms, I'm just saying heroku is working great for me in that way.
I do have one app that processes Voip data over UDP, so I'll need to figure out if I prefer amazon or engineyard for that (heroku won't let you open a UDP port, as far as I know).
I recently put together a presentation on these tools, and how I use them. (it was for newer developers, so it may be too basic for this audience, but there is a list of pros/cons that others may find useful)
Also, I think this conversation does belong here, and not necessarily on a webmasters forum, because the choice of hosting platform will influence your development capabilities and architecture, and the people making the decision are developers, not 'webmasters' or systems people.
I'd vote to use EngineYard over Heroku. Although you can probably deploy a large scale application on Heroku, there's a lot of lock-in you'll have to endure and the pricing can become crippling at higher levels of use.
EngineYard does provide application-level support, too, which is a fair bit better than what Heroku does.
If you're making a quick hobby application or simple demo site, Heroku is great for launching small, simple instances. If you're building a real application where it will need to scale, use EngineYard.
We have been running our platform on Heroku for about 9 months, and I am very satisfied.
I think the biggest complaint that most people have is that it gets "expensive" when your site gets large or high traffic. Personally, I think it is much more effective to focus on growing your business or improving your value proposition than maintaining servers or figuring how to get Rails working. (It is no easy task unless you want to spend a lot of time figuring it out). I would much rather pay Heroku to manage the servers for me than hire someone.
Here's what's great about Heroku:
Pretty easy to use. I didn't know anything about Rails when I got started, and Heroku was simple to get working.
Good documentation for most things.
OK tech support.
Extremely cost effective when you are small.
Heroku is pretty smart, and I am sure they are going to read this, so here's what can be improved:
Tech support: Typically you ask a question and they respond, and that begs a new obvious question. The tech support person should answer the next question I am going to ask. For example, I might ask how to do something, and then they tell me a certain way of doing it. Now I need information about it. Supply all the information in the first response, so I don't have to ask, "How do I use it?"
Documentation: Everyone has the same questions. The documentation could be greatly improved by adding all the questions and answers that I have asked, let alone the tens of thousands of other customers.
Logs: The free logging options are useless, and $100/month for real logs is silly. Our solution has been http://papertrailapp.com which has been outstanding. Use it.
I might as well throw my opinion in here since I have "tried" to use EngineYard and "successfully" use Heroku. While I think both are potentially good choices, I found deploying to Heroku much easier. The ala-carte pricing for Heroku add-ons may add expense, but it also gives you the opportunity to add functionality immediately to your app. The largest expense for our app is the actual web dynos, followed by the database. Heroku has a great selection of add-ons, many of which are free or low cost.
EngineYard also seems like a great company but I think they "hold your hand" a little less than Heroku. For my company, the benefits of Heroku outweighed the cost issue. The read-only filesystem which is a platform feature of Heroku also forces you to learn some new tricks.
I now have several apps (small to medium) on Heroku and happily have my assets served up from s3.
In the end, I would encourage you to try them both. EngineYard offers a 500 hour trial (though that is computing hours, not necessarily real-time hours) and Heroku let's you get started right away for pretty much free.
PS: When selecting add-ons consider your choice carefully, just like when you choose gems for your project. I have experienced an add-on that I was using, simply flaming out and had to scramble to replace that functionality. What was it? Progstr-Filer, which I was using for file uploads. That was a lesson learned.
It depends of the condition. On some case, it's highly expensive
Here we can get a 24 GB o RAM dedicated server for 99 euros.
I can have it set up an running my rail app in less than half an hour, with a mongodb database, as many runner that I want, etc...
Additionally, I can add "small" other project (the ones that costs between 15$ and 35$ monthly at Heroku)
If your business require huge amount of data and processing power, my advice is to use a dedicated hosting and spend the time in managing and monitoring your app.
How can i write a cloud-aware application? e.g. an application that takes benefit of being deployed on cloud. Is it same as an application that runs or a vps/dedicated server? if not then what are the differences? are there any design changes? What are the procedures that i need to take if i am to migrate an application to cloud-aware?
Also i am about to implement a web application idea which would need features like security, performance, caching, and more importantly free. I have been comparing some frameworks and found that django has least RAM/CPU usage and works great in prefork+threaded mode, but i have also read that django based sites stop to respond with huge load of connections. Other frameworks that i have seen/know are Zend, CakePHP, Lithium/Cake3, CodeIgnitor, Symfony, Ruby on Rails....
So i would leave this to your opinion as well, suggest me a good free framework based on my needs.
Finally thanks for reading the essay ;)
I feel a matrix moment coming on... "what is the cloud? The cloud is all around us, a prison for your program..." (what? the FAQ said bring your sense of humour...)
Ok so seriously, what is the cloud? It depends on the implementation but usual features include scalable computing resource and a charge per cpu-hour, storage area etc. So yes, it is a bit like developing on your VPS/a normal server.
As I understand it, Google App Engine allows you to consume as much as you want. The back-end resource management is done by Google and billed to you and you pay for what you use. I believe there's even a free threshold.
Amazon EC2 exposes an API that actually allows you to add virtual machine instances (someone correct me please if I'm wrong) having pre-configured them, deploy another instance of your web app, talk between private IP ranges if you wish (slicehost definitely allow this). As such, EC2 can allow you to act like a giant load balancer on the front-end passing work off to a whole number of VMs on the back end, or expose all that publicly, take your pick. I'm not sure on the exact detail because I didn't build the system but that's how I understand it.
I have a feeling (but I know least about Azure) that on Azure, resource management is done automatically, for you, by Microsoft, based on what your app uses.
So, in summary, the cloud is different things depending on which particular cloud you choose. EC2 seems to expose an API for managing resource, GAE and Azure appear to be environments which grow and shrink in the background based on your use.
Note: I am aware there are certain constraints developing in GAE, particularly with Java. In a minute, I'll edit in another thread where someone made an excellent comment on one of my posts to this effect.
Edit as promised, see this thread: Cloud Agnostic Architecture?
As for a choice of framework, it really doesn't matter as far as I'm concerned. If you are planning on deploying to one of these platforms you might want to check framework/language availability. I personally have just started Django and love it, having learnt python a while ago, so, in my totally unbiased opinion, use Django. Other developers will probably recommend other things, based on their preferences. What do you know? What are you most comfortable with? What do you like the most? I'd go with that. I chose Django purely because I'm not such a big fan of PHP, I like Python and I was comfortable with the framework when I initially played around with it.
Edit: So how do you write cloud-aware code? You design your software in such a way it fits on one of these architectures. Again, see the cloud-agnostic thread for some really good discussion on ways of doing this. For example, you might talk to some services on GAE which scale. That they are on GAE (example) doesn't really matter, you use loose coupling ideas. In essence, this is just a step up from the web service idea.
Also, another feature of the cloud I forgot to mention is the idea of CDN's being provided for you - some cloud implementations might move your data around the globe to make it more efficient to serve, or just because that's where they've got space. If that's an issue, don't use the cloud.
I cannot answer your question - I'm not experienced in such projects - but I can tell you one thing... both CakePHP and CodeIgniter are designed for PHP4 - in other words: for really old technology. And it seems nothing is going to change in their case. Symfony (especially 2.0 version which is still in heavy beta) is worth considering, but as I said on the very beginning - I can not support this with my own experience.
For designing applications for deployment for the cloud, the main thing to consider if recoverability. If your server is terminated, you may lose all of your data. If you're deploying on Amazon, I'd recommend putting all data that you need persisted onto an Elastic Block Storage (EBS) device. This would be data like user generated content/files, the database files and logs. I also use the EBS snapshot on a 5 day rotation so that's backed up itself. That said, I've had a cloud server up on AWS for over a year without any issues.
As for frameworks, I'm giving Grails a try at the minute and I'm quite enjoying it. Built to be syntactically similar to Rails but runs on the JVM. It means you can take advantage of all the Java goodness, like threading, concurrency and all the great libraries out there to build your web application.