i have a rails app which makes heavy use of activeresource and httparty to make api calls.
Is there any library/extension to log the requests and parse them, so that log analysis becomes easier and automated.
RailsLogAnalyser is good but what about extra calls, what are the conventions?
Something like a opensource/self-hosted alternative to newrelic, but with extensions to plug in your own logging.
EDIT: let me clarify:
1) if we use activereresource it logs the calls in a certain format but any plain-bones http calls u make will not follow this convention.
2) a analytics/logging software that will make sense of the logs and have some metric of the number of calls made and count the number of calls etc.
3) will it support syslog, syslog-ng. Any other distributed logging framework.
I have started playing around with rails-analyzer but I am not sure if that will meet your needs or not. It seems to be aimed more at looking for optimization points but I have not delved enough to see what automated portions there are yet.
I'm using this, check this out maybe it will be helpfull for you this https://github.com/wvanbergen/request-log-analyzer
Related
I am trying to figure out whether I can build a web application completely on Ruby stack.
I am looking for following architecture:
Frontend <--- Websockets with Pub/Sub ---> Backend
On top of that, I really like Rails approach with "convention over configuration". As result, I am looking for some "opinionated" framework which will handle this scenario.
I know that it's easier to achieve with Javascript stack. As example using Sails.js.
However, it's experiment. So, I am not looking for the most efficient solution.
Has anybody build anything like that? What would be a reasonable set of libraries/frameworks to build something like it?
Since you said it's an experiment, I would recommend you to try out Volt. It uses Opal, which transcribes Ruby to JS and has reactive synchronization which Meteor.js is famous for. Also there are a lot of Railisms and you will find yourself in familiar waters very quickly.
Both technologies are not quite there yet, but there is a lot of hype around them and they will for sure play a big role in Ruby's near future.
As #ndn mentioned, Volt is interesting... But, it seems to me that the part I would adopt would be the Opal core for a front end.
Volt is too opinionated for me.
I would like to add my own pet project to the table, Plezi, which can be both an independent framework and can also run together within a Rails app by using the Iodine server.
I feel that Plezi has a much better balance between 'convention over configuration' and code-freedom. It manages the backend by automatically mapping websocket broadcasts/unicasts to their respective controller and method (broadcast :method, arg1, arg2...) and automatically maps Redis channels to controllers (or global multicasting) for easy scaling.
At the same time, Plezi does nothing for the front end and assume nothing regarding the front end. This is by design, both for better security (doesn't auto-trust incoming network data) and to allow for for the variety of possible clients such as native apps (iOS, android), browsers, scripts etc'... This would be where Opal can come in (or RubyMotion).
Curious how people are integrating sending of metrics to graphite. It appears most are using a client (many available on statsd github) that sends to statsd which sends on to carbon.
My question is do you think it is ok to include this "cross cutting concern" as such directly into your code. What I am seeing is my application code has went from been nice and clean to now not so nice and clean with code to gather metrics intertwined with my business code.
Any thoughts?
I do think that it's fine to include code to send business metrics in your application code. As #tomer-peled says in his comment, there is a close analogy with logging.
However I understand your reluctance to scatter this stuff around and possibly obfuscate the code. My approach in situations like this is to accept a certain amount of mess to begin with, then as more examples emerge I try to identify emergent structures which point to abstractions that would clean things up a bit.
I'm going to develop a collaborative site, and one of the features will be collaborative editing with realtime changes. i.e. when two or more users are editing the same doc, they can see each other changes as soon as they happen.
I have some experience with Ruby on Rails, so I was thinking about using EventMachine, but with all this hype around Node.js, I am know considering using it instead. So, what would be the main benefits of using Node.js instead of EventMachine?
tl;dr
What are the main differences between EventMachine and Node.js (besides the language)?
EventMachine has nothing to do with Rails apart from them both being written in the same language. You can get EventMachine as bare as Node.js; all you have to do is not add libraries to your project. In my experience the EventMachine libraries (like em-http) are much nicer than anything for Node. And you can use fibers instead of callbacks to avoid callback hell. Complete exception handling is pretty much impossible in Node because of all the callbacks. Plus Ruby is a nicer, more complete language than Javascript.
I tend towards the "use what you know" (even if it's a heavier architecture). Because of that, I don't see it being quite as simple as "EventMachine vs NodeJS." Mainly, the difference can be summarized as this:
NodeJS is a framework/language that was written to handle event based programming in JavaScript. That is its driving force. It's not an after thought, or a third party mechanism. It's baked right in to the language. You create callbacks/events because that's how the language is built. It's not a third party plug in, and doesn't alter your workflow.
EventMachine is a gem in Ruby that gives developers access to some of the goodness of the event based programming model. It's heavily used and well tested, but not baked directly in to the language. Both are locked to one CPU, but with event programming at Nodes core, it still has a leg up. Ruby wasn't written with concurrency in mind.
That said, technical problems can be overcome. The more important questions (from my view) that should guide your decision are these:
What will your production environment look like? Do you have complete control over the server? Can you host it however you want? Or will it be on a shared system to start with, and then you have to expand on that?
Do all the developers on your team have the ability to learn a new language very fast? How fast will they be able to understand an event-based language like JavaScript for the middle tier?
Do you need all of the architecture that Rails gives you (full Testing framework, scaffolding, models, controllers, etc)? Or is that overkill?
There are quite a few technical differences between the two. One is a language, one is a framework. Really, how heavy of a stack you want to run? How much learning will your developers have to do? Do you want a full stack the gives you a lot of niceties, that you may not use, or do you want a bare bones set up that runs extremely fast and concurrent, even though you may have to write extra boiler plate code and learn a new lanugage?
While Rails is not as heavy as some web application architectures, you're still going to need more processor power than you would to handle a similar amount of throughput in NodeJS. Assuming quality code for both systems. Bad code written on either stack is going to prevent the stack from shining. It really comes down to- Do you really want to learn a whole new way of doing things, or utilize your current understanding of Ruby to get things off the ground fast?
I know it's not really a definitive answer, but I hope this helps guide you to a decision!
One thing worth mentioning is the production story. EM, like most Rack stuff, has plenty of testing and monitoring tools available that are well tested, whereas Node.js falls well short in this respect.
At the time of writing, it seems almost impossible to get clear metrics from Node to answer questions like 'Do I need to scale'. There are options starting to form out there from the likes of Joyent, and always the roll-your-own argument, but nothing anywhere near tools such as NewRelic.
Node.js is very good from a performance / configurability point of view, but personally I wouldn't host it in production just yet.
Node.js
You get far better control low level control over what's going in. You can include general libraries to build on top of node.js to tweak your level of abstraction to your own liking. For example you can use connect or express depending on whether you want a view engine written for you.
You can use socket.io or now depending on how much you want your client-server connection abstracted. You can opt to include any of numerous MVC libraries or write your own.
Event-Machine
An asynchronous IO library just like node.js
It comes down to a Ruby vs JavaScript preference, how much flexibility you want with abstractions or lack of abstractions and whether you want to use node as your actual web server.
a detailed view at confusion has already been proposed... just a personal view
[] node.js will be better, if you are ready to learn and experiment more than you think because:
it's thread mechanism is awesome (inspired from that of 'erlang')
you can build a purpose specific server (easily) which will be real productive
I want to let users write small custom apps (think themes or plugins on Wordpress) and upload/run them on my site.
I'm thinking about using Sinatra apps for this since it would give the users a lot of flexibility, and then running them as middleware inside my rails app.
But I can't figure out the security implications of this. I tried creating a simple sinatra app as middleware, and it has access to all the rails models and everything - so that is bad. Is there a way for rack to keep these separate so that the sinatra apps are effectively sandboxed and can't do any bad things (outside of an API or some specific way I setup for them to communicate)?
There may be an easier way to accomplish this that I haven't thought of too, so ideas welcome. Thanks!
Maybe a look at TryRuby / The Freaky Sandbox or one of the other Ruby sandboxes could take you somewhere.
How well do you know/trust your users? I hope the answer contains words such as "extremely" and "intimately". How easily could you reach them with something sharp and painful if they mess up?
I can't think of a simple way (heck, I can't think of a complicated one) to allow upload and installation of such things. Even with Wordpress, I don't think they allow any random plugin to be installed/run on the WP-hosted site - the risks of even well-meaning code causing problems just seem immense to my paranoid mind.
Sinatra would certainly give flexibility, but that may not be a Good Thing, as you've already identified.
Of course, if your users are Ruby/Sinatra literate (or able to become acceptably so) then you could give them a repository (git, svn or whatever) for their apps and deploy them into the appropriate place in Rack either ad hoc or on some schedule.
Thinking on the hoof, and without any notion of feasibility, if the functions that can safely be made available are relatively limited, you might consider looking at writing some sort of constrained DSL for them to utilise. If the scripts thus produced were short/efficient enough, perhaps they could be stored within the app's database and executed via eval?
What is the current state of the art in rails for preventing spam accounts? Captcha? Any good plugins, tutorials or suggestions?
Use a library: You're (almost) always better off appropriating code from people who are better at the subdomain than you are. The Wordpress guys behind Akismet have forgotten more about blog spam than I know, and I was an email anti-spam researcher for a while. You might be interested in a Rails integration plugin for Akismet.
Defense in Diversity: Spam is a quirky problem, in that the more popular a countermeasure gets the worse it becomes. As such, particularly for low-profile sites, you can get disgustingly good results by coding simple one-off tripwires. I won't give you any code to copy/paste because it defeats the purpose of the excercize: having a countermeasure which is globally unique.
One simple example is having a hidden form element which starts as some randomized string, and which is set to a known good value by Javascript code. You then bounce anything which doesn't have the good value supplied. This blocks clients which don't implement Javascript, which includes the overwhelming majority of spam scripts. There are issues, of course, as some legitimate clients also block Javascript -- but realistically, if you're using Rails, I'm guessing you're sort of assuming cookies are on and Javascript works.
I also recommend ReCAPTCHA, both because it's a highly-reliable service you don't have to manage, and because it serves two common goods - the OCR tasks described by the ReCAPTCHA team, and the progress towards teaching people how captchas work, reducing abandonment rates.
Edit:
It appears BranBuster is dead (this was years ago). But I really like:
https://github.com/matthutchinson/acts_as_textcaptcha
I'm a big fan of the rails plugin called "BrainBuster". It's a logic-based CAPTCHA which I find preferable over the "type these words" things, because it is annoying to decipher the words sometimes... It's simple to look at "What is 10 minus 3?" and come up with the answer. YMMV:
https://github.com/rsanheim/brain_buster
There is a re-captcha plugin if you want to use captch to verifye that only human can register, or add content: http://ambethia.com/recaptcha/files/README_rdoc.html
Spam is fair. It doesn't care what you're running behind the scenes.
So by extension, the things that work well on Rails are the same things that work for PHP, ASPNET, etc. Take a look at Akismet and the various "karma" anti-bot tools there are about.
For some there are existing ruby ports but you might have to rewrite a few to task.
For account creation, you may want to use Captchas. I personally am not terribly fond of them and I don't think they are that effective. But if you use them, I strongly suggest you use a service instead of trying to whip up your own. Re-captcha comes to mind. Not sure if there are wrappers for Ruby or Rails, though.
To prevent spam content, however, I strongly suggest Defensio (disclaimer: I've worked there in the past). It uses state of the art spam filtering techniques like what's used for email, such as bayesian filtering. There are plugins for many blogging platforms, including Mephisto (made with Rails). The API is simple and you can look in a few places to get working examples of how to use it with Ruby.