I'm working on a cms and wanted the ability to offer custom extentions for certain accounts. Like having a plugin with custom code that is only available or only used by that account. These custom extentions would be specific to the business needs of an account and perhaps unlikely that any other accounts would need it, but maybe. Is there a way that this could be done and to be loaded without having to restart the whole app, thereby creating downtime for the other accounts?
In terms of per-client plugin code, you could store the code in a data model and then eval() the code to dynamically execute it. (Of course you would want to do some serious sanity-checking / scrubbing on update to ensure the code does not contain malicious calls). Another approach could be to develop a custom tag library, much along the lines of what the Radiant CMS developers have built ... and then let your users "program" their behaviour using the tags provided. This gives you more control and better security at the expense of less flexibility.
In terms of the downtime question, if you are using a modern rails deployment approach I don't see how this should be an issue. The eval() approach above doesn't require a restart (unless your custom code calls "include ..." on libraries that are not installed at the time of the last boot - but getting these libraries installed is also an "out of band" problem that you would need to solve.
Passenger gives you the restart.txt file that you can touch to force a refresh. Similarly there are recipes for mongrel (like see saw) that allow you to progressively restart your mongrel stack to avoid downtime. I would pull these two issues apart mentally if I were you, as the dependencies between the two are not that great. Hope this helps.
I built a cms and added plugin support for it. Best thing you can do is have it be all database driven, the plugin exists for everyone, technically, but you can only make use of it if you've "purchased" it, or some other way of turning it on.. Which is really just a db record.
That'd be 0 downtime. :) Then again, I have no idea what the rest of your setup looks like. I'd think your solution is going to be pretty specifically tailored to your cms system design.
how long would this downtime be really? i mean running migrations and stuff would be a pain for a system that allows any tom dick or harry to upload a plugin. You'd have to verify that the migrations were set up correctly etc. if you aren't getting that 'fancy' and just allow them to do something 'neat' in js, then i guess it's a question of restarting passenger, which is what 5 secs?
I'd check out other 'famous' CMS like radiant or something to see if/how they do it, personally. good luck.
Im not if this is exactly what you are trying to achieve, but have you checked out pancake?
Pancake is a tool & framework to let you stack and loosely couple Rack-based webapps.
http://www.rubyinside.com/pancake-rack-webapps-stacking-2863.html
Related
soliciting advice about upgrading (or re-writing?) a legacy app. It's a single page webapp with lots of dynamically generated windows and forms, roughly comprised of
13,000 lines in .rb files
11,000 lines in .erb files
25,000 lines of javascript (not including large 3rd party libraries that bring this to nearer 60000 lines)
This acts as a UI for end users of our system, which also has a number of core business services (mostly written in Java, with a small amount of Node.js) and a fairly sizeable MySQL database (>200GB). Some of these services push AJAX to the client browser for real-time updates.
Reasons for upgrade
It's ruby 1.8.7, rails 2.3.15. Most of the core code dates from 2009. This makes it both insecure and hard to maintain (think "predates the existence of gemfiles".)
The app has been maintained by Java devs for most of its life, as most of the company's devs have been hired as Java devs to work on all the other services that perform business logic. It's probably safe to assume that this has lead to lots of hacks from people who didn't want to break anything, and certainly lots will not be done in a "rails way".
The javascript is also a bit of a mess. It's got a knot of frameworks (the original Angular is used sporadically; jquery and prototype are both fighting over the $ symbol in different places.) There are files that are 7000 lines long.
the css styling has been upgraded since 2009(!) but is starting to look a little tired. We'd like to implement a bootstrap theme that will look sharp without too much front-end skill, but right now the code that renders all of our pop-up windows, sidebars etc breaks badly if we try and add bootstrap.
It would be nice to modernise our push servers, replacing them with websockets.
Context
There are 3 of us on the dev team- this is my first job, and I've only been here since January. Of the other two, one has only been here about 4 months longer than me, and it's his first dev job too. The other guy is the only one who has ever spoken to someone who spoke to someone from the original team.
Oh, and of course we have little or no test coverage.
Options
When I was hired (as a Java developer), I was told that we were looking to replace the website with one based on Spring MVC. This effort is partially underway, having been attacked in drips and drabs over a couple of years. Because different devs who never met have attacked it as if it's their own brand new project, the same problems are solved in different ways in different places. They've tried some flashy techniques such as custom annotations that I find hard to follow, but as far as I can tell don't fully work. The most senior of us estimates it would take our team a year's dedicated full-time work to finish it (which is not a realistic business proposition, based on how many requests for new features we get from customers).
I'm inclined to upgrade the website rather than spin out a new one. This is partly because I can see the sense in that post. Another reason is that we're all employed as jack-of-all trade full stack developers (doubling as DBAs, sysadmins, etc...). We've got no particular expertise in UI design, and the UI for our present interface, although dated, is pretty user-friendly; it feels like a blank canvas would throw that structure out, and play to our weaknesses. Upgrading ruby/rails would also make any features we add during the upgrade much easier to add to the new site.
Apparently some experienced ruby devs who are friends with my boss have advised him informally it'd be so much work to bring the website up to date that it would be comparable to a complete re-write, which was the motivation for the spring project. This would have the advantage of only having to think about Java + javascript, and not trying to hire people who know both Java and ruby well.
Conventional wisdom seems to be upgrade rails in stages. I'm not sure how well this would work for us, for 2 reasons. For one thing, there 3 major versions for us to upgrade, which might have significant changes between them. More importantly, the code needs some TLC anyway with refactoring and the creation of a test suite.
I'm inclined to follow the following strategy:
Invest some developer time in training, to get a sense of the relevant best practises and the "rails ways" of doing things, rather than the "good enough to hack" knowledge most of us have now.
fire up a new rails 5.1 project on ruby 2.4.0
Configure active record to use our old database
Copy across the javascript from the public folder of the existing project in to the relevant parts of the asset pipeline and save that headache for "phase 2".
Sort out a gemfile with updated versions of our dependencies (for example, mysql gem has been replaced by a mysql2 gem.) Installing rubocop seems like a good idea at this point.
Copy files across from old to new project one at a time. Read the code, figure out what it's doing, write the relevant tests, fix where they break. Use the ruby API and rails upgrade guides to update the syntax. Refactor until rubocop is appeased.
Once we've reproduced the functionality of the existing site, write a new stackoverflow post on how to sort out the javascript ;)
This certainly sounds like a lot of work, but seems less likely to produce a buggy mess than trying to reproduce our existing functionality from scratch in a different language. So...
Questions
Does this strategy seem sensible? Is this a case where the re-write is really a better option? Is tackling the JS separately the best call, or is it better to restructure it as we're examining the calls to it from the views? Or should we really upgrade -> 3.0, 3.1, ... 5.0, 5.1?
We've altered the database manually, adding new tables, new fields and whatnot directly rather than using .rails generate. 'Rails magic' seems to make this work at present, but should we anticipate problems in step 3?
Is there any logical order in which to approach the migration of ruby? As there's major changes to the routing, which is the central entry point of the application, it seems sensible to start there, followed by authentication, then the main page, and then add functions one at a time.
Part of the problem is "not knowing what we don't know" about the rails way of doing things. Apart from the canonical Ruby/Rails tutorials (Hartl, Ruby Monk, Ruby Koans, Kehoe's rails book), is there any essential reading we should be aware of before trying to take on such a large job? I'm especially thinking about things that may not be immediately obvious, like proper use of helper functions, module structure, etc.
Any other advice, comments, prayers, ... welcome!
For the first time I'm creating a quite complex Rails app.
I'd like to know what's the best way to organize that app by folders. Until now, I'd do everything under one app (all the models, controllers, etC) but reading some open source code I realize that they put everything under different apps.
Like for example Spree Commerce. They have a general folder and inside that they have different apps (API, core, admin, etc). How is that done and is that the best way to do it?
I'd like to get pointed to the best way to do it (a book, blog, anything) so I can understand how I can architect my app for future maintenance.
thank you
As an aside I think the title of your question is a little confusing. Rails, by using convention over configuration, defines 'how to organise a Rails app'. I think your question is rather about how to architect your application as opposed to anything Rails-specific. Maybe tweak the title?
That aside, without knowing any more detail about your project it's a tricky question to answer, but I'll give it a go.
All applications should start off simple, if you believe (like I do) that you should start by building the simplest thing that could possibly work. Given this, since you're using Rails, then in all likelihood the simplest thing would be to structure your app as a vanilla Rails 3 application. This will probably (I say 'probably' because I don't know any specifics about the app) allow you to get a beta version of your app up and running pretty quickly without worrying about complexities which at this stage in the development of your project are not a problem.
If you need to create an XML or JSON-based API then Rails makes this really easy using the standard framework, which will allow you to spend more time thinking about the API design than how to code it, and it's the API design which is the most important thing to get right in the first instance.
Similarly, your Admin site can be part of the same app just in a different namespace. If you find later down the line that you want it as a separate app, you can do this (maybe you could use the awesome API you designed to facilitate this), but why bother designing it with this added complexity (and hence extended development time) in the first place if you don't have a good reason for doing so?
Once you have your app up and running and people are starting to use it, you start to get a picture of where the bottlenecks are and where the design could be improved. At this stage, if there's a need, you can start to move parts of the app to scalable solutions, such as running your API as a standalone service, introducing caching, changing data stores and other improvements and optimisations.
Even if your app is as wildly successful (and I hope it is!) then re-architecting your application whist continuing to run the existing service is still entirely possible, as Twitter have proved. Just stick to Knuth's statement and you'll be alright.
Regarding reading material, that's a tricky one. For me a lot of the XP and agile development classics taught me a huge amount about how to approach program and app design. I'd also check this StackOverflow topic for book inspiration.
Good luck!
Spree uses Rails' Railties (Rails::Engines). Railties are introduced in Rails 3 to make it more modular and easy to extend. Rails 3 itself is a collection of Railties (ActiveSupport, ActiveModel, ActiveRecord, etc.).
If you are developing a complex app I would suggest spending some time planing its' architecture. Designing a complex app without any initial planning would definitely end with a maintenance nightmare down the road. It also introduces a huge learning curve for the new team members, slow down your new feature introduction and of course, frustration.
Anyway, don't over optimize, but don't forget to design your architecture for your needs.
IMHO, I will create very complex projects as one app. I have reason to believe that Spree and Radiant build under seperate apps so that under the pretense of their open source communities, contributors can contribute code easily without tampering with the core data, and the core workings of the application.
Otherwise, you should be alright just building it as one app. Just keep it neat.
Here is what have kept me sane for several years of RoR development:
I use Rails Engines, but keep them in same codebase as the main app. Here is good starter for modular Rails app:
https://github.com/shageman/the_next_big_thing
Wherever I can I try to reduce coupling and use composition to make things easily testable, reusable and maintainable. This helps to eventually extract module or engine as separate gem. Composition is done by routes (mounting), directory overlaying (assets), dependency injection or configuration.
If I don't need to re-use an engine I put it in the same code base as the main app which is single deployment unit. Thanks to that I don't need to switch between projects in my IDE. While in development environment any changes to the engine code are instantly picked up by Rails reload mechanism.
I want to let users write small custom apps (think themes or plugins on Wordpress) and upload/run them on my site.
I'm thinking about using Sinatra apps for this since it would give the users a lot of flexibility, and then running them as middleware inside my rails app.
But I can't figure out the security implications of this. I tried creating a simple sinatra app as middleware, and it has access to all the rails models and everything - so that is bad. Is there a way for rack to keep these separate so that the sinatra apps are effectively sandboxed and can't do any bad things (outside of an API or some specific way I setup for them to communicate)?
There may be an easier way to accomplish this that I haven't thought of too, so ideas welcome. Thanks!
Maybe a look at TryRuby / The Freaky Sandbox or one of the other Ruby sandboxes could take you somewhere.
How well do you know/trust your users? I hope the answer contains words such as "extremely" and "intimately". How easily could you reach them with something sharp and painful if they mess up?
I can't think of a simple way (heck, I can't think of a complicated one) to allow upload and installation of such things. Even with Wordpress, I don't think they allow any random plugin to be installed/run on the WP-hosted site - the risks of even well-meaning code causing problems just seem immense to my paranoid mind.
Sinatra would certainly give flexibility, but that may not be a Good Thing, as you've already identified.
Of course, if your users are Ruby/Sinatra literate (or able to become acceptably so) then you could give them a repository (git, svn or whatever) for their apps and deploy them into the appropriate place in Rack either ad hoc or on some schedule.
Thinking on the hoof, and without any notion of feasibility, if the functions that can safely be made available are relatively limited, you might consider looking at writing some sort of constrained DSL for them to utilise. If the scripts thus produced were short/efficient enough, perhaps they could be stored within the app's database and executed via eval?
What behavior have you had to implement in your Rails applications that you feel could exist nicely as a plugin?
What plugin functionality have you searched for in the past but couldn't find?
What existing Rails plugins could be improved or extended, and how?
I'd like to see an engines plugin that dropped in an admin interface that provided a dashboard summary of all the models in the app, with configurable activity charts.
I once wrote a component for an application that did uploaded a zip of photos to an application so they didn't have to be done one by one, I think I may make that into a plugin when I have time, well a new version of it, the code's a little ugly.
Not easy to answer. Everyone is biased towards his own current projects. Additionally a lot of great plugins already exists.
Personally I would like to see some menu plugin for a typical data base application. E.g. where there are few user roles, and every user role has different rights. And a main menu with sub menu which depends on user's rights to actions.
A (CSS) menu is reimplemented in almost every data base application. It is rather given application specific thing and I don't know how hard it would be to implement some general solution or template.
A Credit Card Payment System
Right now, no one bothers implementing credit card payments until a site has become "production quality". Single-person projects or small startups don't bother making this at first b/c they would much rather spend their time prototyping new, hard, or interesting features.
This is bad in the long-run for everyone, including end-users, b/c the default is to offer (often innovative and really great) services for free, which means only already-funded teams or people with lots of extra time can even get to the point of making something.
If this existed, were packaged up neatly, and were as dead-simple as Rails scaffolding, small projects could default to whatever payment scheme that actually made sense -- like pay-per-use, donations, trial periods, first 5 free, etc. -- making it possible to fund the really great products out there, instead of forcing them to rely on outside funding or ads, which ruins the product.
In a real physical store, people expect to pay. If the person on the other side of the counter gives me something for free (w/o buying anything at all), I say "Are you serious??" and walk away bewildered. But online, I almost expect it. This is bad! Because now everyone expects it, and people trying to make great things can't focus on actually doing that b/c they are too busy trying to figure out how to make it great and free.
Oftentimes, there are free things out there online that I like so much that I would like to pay for, but there's simply no easy way to do it. And I'm not alone. I use tipjoy and have donated to things that have a PayPal donation setup like Auditorium, but I don't see these as adequate.
The best thing out there that I've seen is RailsKits. But as far as I understand, they aren't packaged properly so that I can add them to my site whenever I please like a plugin or gem.
I'd like to see some sort of in-built video playback/streaming. I need it for my current project.
I agree with Greg... there are lots of great plugins, and lots of great ideas for more.
I think there's always a need for more web APIs to be supported. As we go forward with API driven services and semantic web, there's tons of work to be done on that front.
http://www.programmableweb.com/ is a great starting point for thinking about what really useful APIs don't have a rails plugin or ruby gem yet.
Cheers!
Walt
I'm wondering if it's possible to distribute a RoR app for production use without source code? I've seen this post on SO, but my situation is a little different. This would be an app administered by people with some clue, so I'm cool with still requiring an Apache/Mongrel/MySQL setup on the customer end. All I really want is for the source to be protected. Encoding seems a popular way to go for distributing PHP apps (eg: Helpspot).
I've found these potential solutions:
Zenobfuscate - not all types of Ruby code is supported however, so that counts that out
Ruby Encoder - may be the best option, as their PHP encoder looks alright (I haven't tried it however) but it's not available yet. I've used IONcube for PHP before and it worked well, but it doesn't seem that IONcube is interested yet.
Slingshot - it was mentioned in the other SO post, but it solves a different problem to mine and the source is still visible.
RubyScript2Exe - from the doco, it's not production ready, so that counts that out.
I've heard that potentially using JRuby and distributing bytecode might be a way to achieve this, but I've never used JRuby so I'm not sure what's involved.
Can anyone offer any ideas and/or known examples? Ideally I'd love to have some kind of automated build scenario as well.
Your best option right now is to use JRuby. A little bit of background: My company (BitRock) works with many proprietary and commercial open source vendors. We help them package their server software, which is typically based on PHP, Java or Ruby together with a web server or application server (Apache, Tomcat), the language runtime and a database (typically Postgres, MySQL) into a self-contained, easy to use installer. We have a large number of PHP-based customers (including HelpSpot, which you mention) but also several Rails-based ones. In the case of the RoR customers the norm is to use JRuby together with Tomcat or Glassfish although in some cases we also bundle a native Ruby interpreter to run specific scripts that rely on libraries not yet ported to JRuby (usually not core to the application). JRuby has matured quickly and in many cases it actually runs their code faster than regular Ruby. You will need to also consider that although porting your code to JRuby is fairly straightforward, you will need to invest some time on that. You may want to check JRuby Stack which is a free installer of everything you need to get started. Good luck!
If you release the source, obfuscated or otherwise, your app will be pirated. See, for example, Mint. It depends on what you're building, but you may find that you're better off releasing the app as a hybrid of sorts: A hosted app with a well-defined API, and a component that runs on the customer's server. As long as the true value of your product lives on the server side, you don't need to obfuscate your code, and you can just release the source code unmodified. Additionally, this may also give you the opportunity to reach clients running, say, PHP rather than Ruby. See, for example, Google Analytics, HopToad, Scout, etc, etc.
You can, but it wouldn't do anything to prevent somebody from reverse-engineering or modifying it. I remember there was an article about similar attempts to obfusticate Perl and how they could be effectively bypassed by a debugger and 5 minutes of effort.
If you can't wait for the delivery of RubyEncoder, then I think ZenObfuscate is the most promising. Though it may require some modifications to your source code, they do say this on their site:
ZenObfuscate costs $2500 for a site license or is individually negotiable for other licensing schemes. Yes, that is expensive. That was on purpose. But don't let that thwart you too much. If your product is really cool and we want to see it succeed, we'll make it work. "Really cool" is not freecell.
Of course, for $2500 (or more), you'd hope to get a few tweaks to the compiler that'd make your codebase fully supported. It might be worth engaging them in the conversation.
You can also take a look at Mingle from ThoughtWorks studios as an example of using JRuby for this.
It's a Ruby on Rails app, they run it using JRuby. They've customized jruby to load encrypted .rb files.
Take a look at JumpBox.
I've had conversations with them on the topic, and they seem to have a solution that will work soon for Rails apps.
I'm wondering if you could just "compile" the ruby code into an executable using something like RubyScript2Exe ?
To be honest I haven't used it but it seems like it could be what you want, even if it just packages up the scripts with the interpreter into a single executable.