Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 8 years ago.
Improve this question
Sorry for the broad question, but I'd really like to know the combination of services people are using to get great monitoring for their apps. Right now, I'm thinking of going with Pingdom + Hoptoad (+ PagerDuty), but I want to know what others are using as pretty much everything I'm deciding between has several competitors.
I want to know when my app is unreachable, unstable, or down
I want to do basic sanity check that my app is good (renders something), but doesn't have to be deep (no need to simulate a log in for a 'ping')
I want to be able to cheaply and easily add new things to monitor
I want to be able to generate my own notifications (Seems pretty standard now, but worth calling out)
I want to know when I get a lot of exceptions (the odd one here and there I'm willing to check in the morning, but 1 every few min is worth getting up for)
I want reporting
It needs to scale with multiple apps, apps running across multiple nodes, and with multiple users on the same projects.
I've seen the Hoptoad vs. Exceptional discussion. I'm more interested in what people have put together to form their entire monitoring suite. What are you using? What do you like about it? What do you not like about it?
Would really love to get suggestions!
These are the tools we use for monitoring:
Monit: http://mmonit.com/monit/
For both internal & external monitoring: apache, nginx, mongrel, mysql, sphinx, delayed_job, postfix, CPU, etc
Whenever a process goes down, its will try to bring it up. If it is not able to start a process it will notify us.
Cloudkick: https://www.cloudkick.com/
For both internal & external. We are in the process of evaluating this tool. It supports rackspace, ec2, slicehost, etc. If you use more than one cloud services, this tool might help you.
Alert Grid: http://alert-grid.com/
For monitoring our cron tasks.
Exception Notification:
Error App: http://errorapp.com/ - Its free
Airbrake(hoptoad) http://airbrakeapp.com/pages/home
Logwatch:
http://linuxcommand.org/man_pages/logwatch8.html - Sends daily report about whats happening in a server. Like new packages installed, list of cron tasks running, authentication failures, etc.
Intrusion Detection:
http://www.ossec.net/
The rails error management landscape has changed a lot since this question was first posted. But this thread came up when I was doing google searches for airbrake alternatives.
Several new companies have started offering error monitoring services for rails. The one we like at work is Honeybadger.io. Paper trail is also a nice service in this space.
Another option for error tracking and monitoring in Rails is Raygun, at http://raygun.io - it's got the usual complement of features like intelligent email notifications and error grouping, but adds nice organisation support for multiple devs and integration in GitHub, Fogbugz, YouTrack etc. It's also got Hipchat notifications so when an exception occurs your team sees it in your chat room.
Installing and setting up the ruby gem (https://rubygems.org/gems/raygun4ruby) into your project's pretty quick, too.
One wonderful way to get updates about your application is to use Exception Notification.
This allows you to get emails based on exceptions that take place in your app. It won't be a solution to all of your information desires, but if someone tries to access a non-existent page, you could get information about what they tried to access, how they got there, etc.
Hoptaud and Exception Notification are good, Heroku has another addons to use like StillAlive.
FogBugz and LightHouseApp may also help you, Lighthouse is free for open-source.
If you want to get involved deeply in Rails errors and notifications read The Rails 3 Way book's first chapter and /environment folder's content.
The best way is still BDD and TDD. Work out all redirections, add nice 404 page, use all stages for development (devel, test, hopefully staging and QA too, and only finally production), then get notifications and always check logfiles before going for production. HTH
Related
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 8 years ago.
Improve this question
I'm a rails developer (with about a year of experience) and I am getting pretty comfortable with it, but I find that I lack even a cursory understanding of how deploys or servers work. I am familiar with terms like Unicorn or Apache, but I don't know much beyond the notion that they manage rails instances (?).
I have heard of chef, and I know it is used for deploys, but I don't know where I'd begin.
Is there a book or a screencast series that would be useful in learning this sort of thing?
(Side note: I have a project I want to make that will run sort of like a template builder and a web host, so understanding how to add domains/subdomains/manage that sort of thing is one of the primary drives for learning, other than the natural want to learn).
When dealing with remote (or even local) deploys, I find that the best solution is using capistrano. You can find all the information you need at their website.
You can start reading the long README from Github and then switch to more detailed information starting from here.
When deploying a Rails application, usually servers like Apache or nginx acts as a router to your application's local server. Eg. you start a thin server for your deployed app that answers on the 1234 port and configure Apache to redirect all the calls to you remote address to the local server at localhost:1234.
Capistrano will start/restart/stop the local Rails server through the configuration you'll put in the config files, so you'll have complete control over it.
Hopes this helps. You can find lots of information online about capistrano and the integration with various http servers and rails servers.
Eg. a good starting point can be this screencast made by Ryan Bates, but beware! This is for the older 2.x version of capistrano.
Don't go anywhere but straight to latest awesome chef fundamental series, check opsode user on youtube (http://www.youtube.com/user/Opscode).
First episode:
http://www.youtube.com/watch?v=yh9osPQA_-k
after it you can go to irc, docs, watch other advanced things (including berkshelf). But first things first.
I have lots of things to tell regarding this. I'm on chef for around a month, and I've spent tons of hours understanding how it works. And I'd say the official docs are just a disaster. They give everything in one place, although you don't need to know it in the begging. I can only offer you some resources like: http://learnchef.getharvest.com/ which is best ever introduction I've found around. All those official learnchef links didn't work out for me.
I'd really recommend to stay away from Vagrant for a while, just buy a 5$ vps on DigitalOcean cloud and try this manual: http://adamcod.es/2013/06/04/deploy-a-basic-lamp-stack-digital-ocean-chef-solo.html
Start with a chef-solo instead of chef-server, and try knife-solo. Use berkshelf although it's rarely mentioned in official docs, because chef can't handle cookbook dependencies although it can download cookbooks. then slowly start looking how I've automatized a chef-server installation: http://github.com/holms/chef-starter This will give you an idea, which steps to take for chef-server setup, and how to deploy stuff with knife.
After all that, try vagrant, as it offers to run everything from your desktop machine. I've started from vagrant and I've wasted too much time. When you put chef-server and vagrant together, you get tons of information which you just unable to handle. Lots of cave heats appears, lots of things to write.. you ending up wasting your 40 hours non-stop learning, without achieving your desired results.
You can contact me on freenode irc if you need any help. I'd never wish this chef experience even to my enemy.
Update:
This is quite old comment :) So just ignore it. Ended up using vagrant+chef, then vagrant+ansible, now using docker. To be exact docker+traefik
I am developing an app that allows users to post to rooms/pages, much like a forum or here (stackoverflow). I have already built out the entire app, however, I have one last obstacle to climb: How to handle updating posts for a user.
I have done alot of research into the topic, and I believe Long Polling is the way to go, at least until I get an immense amount of traffic (if that happens, I'll figure out how to implement websockets). This is because Long Polling will be easier to implement, it is more widely supported, and given that I am deploying to heroku it will be easy to upgrade my server resources quickly. Anyway, I wanted to know what the best practices are of long polling, both in terms of user interaction, and server load.
Should I automatically place posts on the page, or create an alert system such as the one here on stackoverflow?
Any and all input would be appreciated. If you think my logic is wrong in anyway, please tell me!
For a system like this we suggest using something like pusher. Check out the documentation for more details as to how to do it. https://devcenter.heroku.com/articles/pusher
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
I'm trying to decide between Heroku and Engineyard.
Heroku seems so much better but they charge for everything and their prices are crazy!
Why one should use Heroku over EY or vice versa?
Heroku makes setup and launching of an app super-simple. You will be dependent to some degree on versions that Heroku supports (for instance, I heard today of a bundler versioning issue).
One thing to take into account with any "managed full-stack" solution like Heroku or EY is cost. You don't have to hire an ops person or have ops expertise, but you're still paying. Storage is where things get really expensive. Crank up your DB to a more than a few GB and watch the price go up.
We have in-house ops (was me doing it while coding, now a dedicated person) and run on Joyent. A big cost savings was having a few master-slave DBs and sharing them among a few dozen applications. We essentially have 100 Facebook apps running on Joyent at the same cost as 10 apps on Heroku. But this doesn't take into account the ops salary/time.
Everyone's needs are different, but the great thing is that its easy to experiment with these cloud deployment tools in quick fashion, and you will find that they each have their own strengths that you can leverage as you need.
What is most valuable to me, and my smaller clients, is to be able to experiment and get end-user feedback quickly. I have startup clients that want to be able to push out new ideas and test them quickly, deploy different combinations of ideas to different markets, get customer feedback, and keep moving forward. Launch a facebook app, a test server for an API integration client, a lightweight 'freemium' version of a product, etc. As traffic picks up, we make changes to scale up, and the increase in cost is never out of bounds (eg. our hosting costs are still well under the increase in value/revenue/marketing juice, etc).
EngineYard lets you play around with 500hrs for free, and you can easily turn it off when you are not using it, to stretch the 500hrs out. You can deploy your app quickly, deploy a CI server (that updates the app on every successful build), create a backup of your app or 'staging' server and see how it goes.
Amazon will give you 750hrs per month for free, for a year, if you are a new AWS customer. You can use this for a super fast CI server, hard-core image processing, batch reporting, whatever.
Personally I happen to use Heroku the most, as it just seems to work the best for my needs. I can put together a new application with full monitoring, backup, analytics, email, etc really fast, and feel confident in how to manage my setup (and confident that I can bring another person on board, and their learning curve will be pretty easy). As a freelancer, my use of Heroku has brought my setup time down to almost nothing, so I'm able to focus my time on understanding the business, and developing a great product. I'm not saying that can't be done on other platforms, I'm just saying heroku is working great for me in that way.
I do have one app that processes Voip data over UDP, so I'll need to figure out if I prefer amazon or engineyard for that (heroku won't let you open a UDP port, as far as I know).
I recently put together a presentation on these tools, and how I use them. (it was for newer developers, so it may be too basic for this audience, but there is a list of pros/cons that others may find useful)
Also, I think this conversation does belong here, and not necessarily on a webmasters forum, because the choice of hosting platform will influence your development capabilities and architecture, and the people making the decision are developers, not 'webmasters' or systems people.
I'd vote to use EngineYard over Heroku. Although you can probably deploy a large scale application on Heroku, there's a lot of lock-in you'll have to endure and the pricing can become crippling at higher levels of use.
EngineYard does provide application-level support, too, which is a fair bit better than what Heroku does.
If you're making a quick hobby application or simple demo site, Heroku is great for launching small, simple instances. If you're building a real application where it will need to scale, use EngineYard.
We have been running our platform on Heroku for about 9 months, and I am very satisfied.
I think the biggest complaint that most people have is that it gets "expensive" when your site gets large or high traffic. Personally, I think it is much more effective to focus on growing your business or improving your value proposition than maintaining servers or figuring how to get Rails working. (It is no easy task unless you want to spend a lot of time figuring it out). I would much rather pay Heroku to manage the servers for me than hire someone.
Here's what's great about Heroku:
Pretty easy to use. I didn't know anything about Rails when I got started, and Heroku was simple to get working.
Good documentation for most things.
OK tech support.
Extremely cost effective when you are small.
Heroku is pretty smart, and I am sure they are going to read this, so here's what can be improved:
Tech support: Typically you ask a question and they respond, and that begs a new obvious question. The tech support person should answer the next question I am going to ask. For example, I might ask how to do something, and then they tell me a certain way of doing it. Now I need information about it. Supply all the information in the first response, so I don't have to ask, "How do I use it?"
Documentation: Everyone has the same questions. The documentation could be greatly improved by adding all the questions and answers that I have asked, let alone the tens of thousands of other customers.
Logs: The free logging options are useless, and $100/month for real logs is silly. Our solution has been http://papertrailapp.com which has been outstanding. Use it.
I might as well throw my opinion in here since I have "tried" to use EngineYard and "successfully" use Heroku. While I think both are potentially good choices, I found deploying to Heroku much easier. The ala-carte pricing for Heroku add-ons may add expense, but it also gives you the opportunity to add functionality immediately to your app. The largest expense for our app is the actual web dynos, followed by the database. Heroku has a great selection of add-ons, many of which are free or low cost.
EngineYard also seems like a great company but I think they "hold your hand" a little less than Heroku. For my company, the benefits of Heroku outweighed the cost issue. The read-only filesystem which is a platform feature of Heroku also forces you to learn some new tricks.
I now have several apps (small to medium) on Heroku and happily have my assets served up from s3.
In the end, I would encourage you to try them both. EngineYard offers a 500 hour trial (though that is computing hours, not necessarily real-time hours) and Heroku let's you get started right away for pretty much free.
PS: When selecting add-ons consider your choice carefully, just like when you choose gems for your project. I have experienced an add-on that I was using, simply flaming out and had to scramble to replace that functionality. What was it? Progstr-Filer, which I was using for file uploads. That was a lesson learned.
It depends of the condition. On some case, it's highly expensive
Here we can get a 24 GB o RAM dedicated server for 99 euros.
I can have it set up an running my rail app in less than half an hour, with a mongodb database, as many runner that I want, etc...
Additionally, I can add "small" other project (the ones that costs between 15$ and 35$ monthly at Heroku)
If your business require huge amount of data and processing power, my advice is to use a dedicated hosting and spend the time in managing and monitoring your app.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 9 years ago.
Improve this question
Are Ruby on Rails site usually slower than java or .net sites?
(This is assuming developers are not abusing the technology.)
A lot of Ruby sites I have seen have performance issues.
Yes, Ruby on Rails sites do have performance issues, just like any other site. And just like any other site, those performance almost always are rooted in architectural decisions, not the language or the framework.
There was a nice presentation a couple of years back by Joyent (might have been RailsConf 2007?), which showed on one slide all the servers that are running on a single instance of their Rails platform. About 40 processes. Only one of those processes was the Ruby interpreter, everything else was stuff like the DNS resolver, web server, database server, MTA, memcached, message queue, reverse proxy, load balancer, etc. Every single one of those can screw up your performance. That's a 97.5% chance that your performance problems have nothing whatsoever to do with Rails or Ruby.
There's some really nice E-Mails on the JRuby mailinglists, and also some Tweets and blog posts by people who rewrote their Java web applications in Ruby and got a 10% performance improvement.
A really good example is Twitter. Twitter is one of the biggest Ruby on Rails sites in the world. It is also one with a very unusual usage pattern that will give any framework that is designed for "normal" web applications a tough nut to crack.
Now, you might think, why did I choose Twitter of all things as an example of performance and scalabilit, when clearly it is the exact opposite that they are known for? Well, that's exactly the point: they have had a ton of scaling, performance and reliability problems. And not a single one of those had anything to do with Rails or Ruby. In fact, Rails and Ruby were pretty much the only pieces in their stack that they did not have problems with.
They had problems with unexpected growth. It doesn't matter which language or framework you use: if users are signing on faster than you can buy new servers, there is nothing you can do.
They had problems with the performance and scalability of MySQL. Again, MySQL has nothig to do with Rails or Ruby. In fact, MySQL is written in C, so if you really absolutely must make any ridiculous conclusion about a programming language, based solely on a single incident, then it would be this: C is slow. If you want performance, stay away from C.
(In this particular case, in one interview, they actually did blame Rails: they said that because Rails only supported a single connection to a single database, their MySQL instance simply got overloaded. Within hours of that interview being posted, two Rails developers independently of one another both released Rails plugins to implement multiple connections. The lesson is: only the 80% solutions are in the core. Twitter clearly isn't in the 80%. The plugin API is there for a reason.)
They had problems with the performance and scalability of the overall system. It turned out that, in order to get the product out quickly, they implemented absolutely no caching whatsoever. Even the "static" Twitter homepage was completely dynamically generated for every single request. Again, this had nothing to do with Rails or Ruby. You can bring any site down by turning off their caches, I guarantee you that.
They hit some very bad scalability problems (and the MySQL problems mentioned above are related to that) which were simply caused by the fact that people used the site in a way not anticipated by the developers. Everybody knows that Twitter is a micro-messaging platform. Well, except for the Twitter founders. They had this brilliant idea for a micro content management system.
And so, they did build a micro-CMS. And of course, the central piece of a content management system, is a content repository, IOW a database. Users however used Twitter as a micro-messaging platform. And the central piece of a messaging platform is a message queue.
As a result, MySQL was being used as a message queue. No two things could be further apart than a database (especially a SQL database) and a message queue. The two have almost exactly opposite requirements and constraints.
And of course, the entire architecture was built around that content repository which was now being abused as a message queue.
In response to that, the Twitter developers wrote their own message queue in Ruby, which helped performance and scalability a lot. But not enough. So, they wrote another message queue, this time in Scala.
It is this single rewrite that is wholly responsible for, I would estimate, at leat 70% of all the Rails FUD out there. But, I don't know about you: when I write something that I have absolutely no idea how to write it, and then I write the exact same thing a second time, when I actually do know what the heck I'm doing … the second one is always better than the first. And I suspect that this is the case here, too.
In several interviews, the Twitter developers have pointed out that Ruby on Rails was not responsible for their scaling problems. On the contrary, only the maintainability of Ruby made it possible to do such large-scale architectural changes to fix their scaling problems.
To cut a long story short: today, Twitter is actually using Ruby on Rails for what it was intended to be used: for a web application. And they use a database for storing data and not as a message queue. And they use an actual proper message queue. The message queue and some other backend stuff is written in Scala. The frontend is written in Ruby on Rails. Some stuff is written in C.
And all is peachy.
The real moral of the story here is that you can substitute pretty much any large web app, any language, any framework into the above story and it would still be true. MySpace is one of the slowest, most unreliable websites I know, and that is a .NET site. GitHub is one of the fastest websites I know, while being the biggest hosting platform (it has over a million repositories after just over 2 years, that's more than SourceForge and Google Code combined) and it is written in Ruby on Rails for the frontend, Sinatra for the web service, Ruby for the Git interfacing and Git infrastructure, Erlang for the federation and cloud infrastructure and Node.JS for the download server.
Here's a start
Scaling ROR
Why Rails can run slowly
Framework Preformance Comparison
SO related question
Related interest article Twitter abandons ROR its old I know but I didnt actually know that lol
Short answer
Could potentially sure.
Might be more likely than some other languages.
Depends on the application, the programmer and the architecture.
As I see it - RoR is only a bit slower. At least slower than .Net, cause ruby is interpreted language.
But in general - it depends on quality of developers. RoR app that uses nice caching will work n times faster than .net/java app that loads half database into memory and sends plenty database requests cause of select n+1.
yes it is slower, but in production it will probably only hurt you once your load "exceeds a certain point" ("x requests/sec") and most sites never see more load than that.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 4 years ago.
Improve this question
Hi I'm looking to write a multiplatorm tasks application for technical people. I want to handle as many platforms as I can (web, shell, desktop) and therefore I have decided to begin with a server/API.
I want to write it in Ruby, however i think that Rails is a bit too heavy for this, even though it would do the job. Sinatra also doesn't seem quite suited for the task.
All the server/API would do would be to translate simple requests to Database queries, and at a later stage some authentication and authorization.
So basically I want to know:
1) Should I use a REST api or a SOAP api?
2) Is there a framework for this? Or what is the closest framework avalable?
For the adventurous, there is also a less known project called grape. It is a rack based application, similar to Sinatra, but is only purposed to write API. I don't think it is mature enough to be used in serious projects yet, but it is still interesting to know.
1) REST, SOAP is a terrible system and its support in Ruby is quite lacking. REST, on the other hand, is basically the ruby default and takes very little effort to use, especially if you are using REST/JSON.
2) Sinatra and Rails are basically your options. It comes down to how complex this application will be. Sinatra can probably handle the task just fine, but Rails does much of the work for you at the expense of bloat. You will already be taking on some of the rails bloat if you use ActiveRecord for the database. When authentication and/or roles come into play, Rails has mature solutions for both. Without any additional information, I'd lean towards Rails as it does much of the work for you and, when written properly, can still be fairly fast.
Actually SOAP is very VERY easy to implement with AWS.
At the same time, REST API is also very easy to implement.
I have written a couple of different and parallel (JSON, XML and custom format) APIs with rails. Im sure the framework stack performance will not be your bottleneck, so don't bother with worrying about performance just yet. Your first bottleneck will anyhow be database and then perhaps requests per second.
All in all i would suggest going with Rails, it has a lot of work cut out for you.
Since this old thread still comes up high on related Google searches, I should chip in my highly biased (as co-author and user) recommendation for Hoodoo. Unlike other offerings, Hoodoo includes an API specification that says how API calls must be made and how they must respond; it enforces a consistency across your design that calling clients will appreciate. If you can call one API, you can call them all. Hoodoo implements a lot of the boilerplate so you can focus on meaningful service code.
We've been using Hoodoo services for over two years very successfully at Loyalty New Zealand, who run the country's largest loyalty programme. Our Hoodoo-based microservice platform handles 100% of our client transactions.
http://hoodoo.cloud/
https://github.com/LoyaltyNZ/hoodoo
https://github.com/LoyaltyNZ/hoodoo/tree/master/docs/api_specification
https://github.com/LoyaltyNZ/service_shell
Hoodoo has 100% non-trivial rspec test coverage and 100% rdoc documentation coverage. As you'll see from the above links, there's quite a lot there!
Hoodoo is a Rack application, so works with any Rack compatible web server. Our preferred deployment mechanism though is an indefinitely horizontally scalable arrangement based on an HTTP-over-AMQP bridge and an AMQP cluster of nodes each running the same collection of services, managed inside Docker containers and deployed with Fleet. The system self-load balances across the service nodes via the queue and the decoupling of front end HTTP->AMQP processor versus the AMQP->HTTP input into the Rack stack dramatically reduces the system's attack surface. We wrote the front-end component in Node and for more about this along with Node implementations of other parts of the framework concept, see the Alchemy Framework. Alchemy Node services and Hoodoo Ruby services can coexist on the same grid happily.