I'm working on application which can be logically grouped into a core engine, and business domain modules. The business domain modules essentially encapsulate code which is specific to our customers' businesses. We initially separated this out using the root rails structure for our core engine, and having all customer code in separate plugins.
But we've run into various problems with this approach, most of which can probably be put down to Rails class reloading in the development environment. While we have managed to get reloading largely working, we've run into weird Rails bugs with partially unloaded classes combined with the Rails.cache.
What I would like to know is, are we abusing the intended usage pattern for Rails plugins? Was packaging up aspects of our application as plugins the right move? And is there a better way to do it? Or should we rather soldier on and try sort out these remaining issues?
We're currently moving toward rewriting the plugins as modules within the root rails structure, but I must confess I rather like the elegance of plugin mini-application directory structure.
Brendon McLean.
My large app includes several plugins that are private to the app. I agree that plugins can nicely isolate sets of functionality.
I haven't run into the loading problem that you describe since I turned off the dynamic class reloading in dev mode.
Why not have dynamic class reloading? It seemed to slow things down too much. Easier to cntrl-c and restart the test mongrel when needed. After all, I usually run a given iteration of the code for more than one html request/reply cycle before making further changes.
Your original plugin architecture sounds like it was solving several problems for you. I'd first try to change the tool (turn off dynamic reloads) before changing the software architecture.
Related
Currently there are multiple (about 15-30) independent web applications written in another language. Each one is completely independent with files, images, headers, users, databases etc. etc. The whole 9yards, except that they all exist under the same domain and should have the same style (but they don't). They will soon be converted to C# ASP.NET MVC 2. They do share the same LDAP authentication.
The question has come up in my mind as to whether these should be setup as multiple MVC solutions or be done within a single MVC application. They will all have the same styles, mostly the same images, and it would be nice for them to share basic functions.
The reason this isn't a simple cut and dry solution to me, is that some of these applications are quite large by themselves and throwing them all together might be hard to manage. Not to mention the development of new applications will continue as well as new features added to the existing ones. Making this possibly an extremely large solution.
I am fairly new to MVC and even though I have a good understanding of it now, I'm still trying to rewire my brain here and there to work with the methodology and design.
I guess what I'm asking for, is those of you who have more experience with MVC than I do to share some incite and wisdom about MVC in practical use to give me a direction to start thinking.
Please, make yourself a favor and do not combine them in a single solution. I worked once in a project where we had one huge solution to work and that was the root of all evil. If you place everything in a single solution, you are increasing the complexity of all projects, you might be thinking, I am actually going to save a few lines of code by reusing something, but the truth is that you are creating a deadly solution which will become a bottleneck eventually
Consider the following:
The performance of Visual Studio is affected when you have more than 30-40 projects, which means that your build is going to take more and more time.
If you implement a build server (and you should) if you have one huge solution, the script to build only the projects related to each application would be really complex
Now I think you already did the most difficult part of the design when you say:
Currently there are multiple (about 15-30) independent web applications written in another language
If your applications are independent that means they have an independent domain, so there is no reason to place them in a single solution, not even treat them as modules.
Managing independent solutions does not mean that you cannot have shared components among them, (BTW when I say shared components I mean infrastructure components, please do not try to reuse domain objects).
So now the question is how should I reference the shared components?
In these days, I have found that the best way to reuse infrastructure components among solutions-projects, is by using Nugets. Using Nugets makes it easy to distribute new version of the components, so my suggestion is: create a private Nuget server in your organization (a simple IIS application) and add to this server your own private packages and just reference them from your solutions
You can place in your Nuget packages practically anything you need including:
Assemblies
XML config files (including common XML logger configuration files)
Common JavaScript files
Common Style Sheets files
etc...
This is a good article to create a private Nuget repository
http://docs.nuget.org/docs/creating-packages/hosting-your-own-nuget-feeds
To create a Nuget:
http://docs.nuget.org/docs/creating-packages/creating-and-publishing-a-package
And finally to integrate the creation of a Nuget in your CI server:
http://www.hanselman.com/blog/NuGetForTheEnterpriseNuGetInAContinuousIntegrationAutomatedBuildSystem.aspx
http://docs.nuget.org/docs/reference/command-line-reference
When I go for combining multiple web applications into single.. I'll consider the below points.
If all the applications shares a common business model.
If they shares a common infrastructure (security, validation, logging and others..)
If they shares same a common user base.
If combining multiple projects into one helps me to reduce the cost of maintenance and enhancement.
In your case you said each one of them is completely independent then why you need to combine?
My recomendation is DI and create each proyect like a plug-in ,so each proyect can be developed or manage in separate without affect others
I have a few proyects with MEF and it's so easy create new or manage existents plug-ins
Here is a getting started MVC and MEF… http://blog.maartenballiauw.be/post/2009/04/21/ASPNET-MVC-and-the-Managed-Extensibility-Framework-%28MEF%29.aspx
and a downloadable example http://www.hanselman.com/blog/ExtendingNerdDinnerAddingMEFAndPluginsToASPNETMVC.aspx
The same style could be accomplished with a unified stylesheet referenced by all, as long as you use similar mark-up in your pages within the apps. Common functionality could be provided through a unified class library. To me, it really depends on exactly how close the style and functionality are between apps... do you want the exact same markup on every page, etc.
It is common to have an instance of the Controller per application, however if you implement this using a data driven Front Controller then there only needs to be a single Class within your new WeB Application framework. So each Application might have a configuration file that maps URL to Command Class files. These can be constructed on demand or requested from a Resource Pool. A big advantage of this approach is that many of these commands would start as a very thin wrappers (ServiceToWorker) over the existing application and/or ASP views.
I totally agree with Marks answer, ask yourself "why" do you need to combine them. Do they really need to be independent?
My additional comments though are....
What you should definitely think of....
Create a unified CSS files which use the same images to be used by your applications
Write some universal JQuery (Mobile version if these are public facing) using JQuery templates/partial views as well to give all these seperate applications a unified experience
If you are not going to unify your server side code in terms of the DAL etc, then just concentrate on the client side.
Breaking a large rails app into smaller apps?
Modularizing Rails applications
Best practice for structuring a 'large' Rails app
I have a quick question on modularization in a large Ruby on Rails App.
Setup:
I am building a core app that stores information about people. I also have several 'modules' that use that information in very different ways. (e.g. one could display information about the people, another could figure out connections and commonalities between them, etc).
The question:
How do I modularize this app efficiently?
Potential Answers:
Since the modules share models and views with the core app (and eachother) it makes sense for me to combine them into one app. However, as the app grows this will obviously lead to problems. This to me suggests either Namespacing the controllers and models "How to organize controller in moderately large Rails application?" or using engines "Modularizing Rails applications".
Since the modules are in active development it is very helpful to use rails generators on them, which seems to make using Engines a pain in the butt. It also seems that while engines are fully supported from a Rails point of view, they still seem rather hacky with regard to lack of generator support and database migrations. Does anyone have experience with developing engines successfully? It seems like engines would be a great solution if you had a working app and wanted to port it into a plugin (i.e. copy paste code) but if you are actively developing it (changing models etc) it would be difficult.
The last thing I have seen around is using multiple apps and one database. This way seems like a royal pain with migrations and keeping models straight etc etc. But I thought I would get thoughts on that as well.
Rails engines seem to be the correct solution for you. I have used engines when I worked with Refinery CMS, which is built on top of Rails Engines.
The following are some nice links which talks about Rails Engines:
http://www.themodestrubyist.com/2010/03/01/rails-3-plugins---part-1---the-big-picture/
http://www.themodestrubyist.com/2010/03/05/rails-3-plugins---part-2---writing-an-engine/
http://www.themodestrubyist.com/2010/03/16/rails-3-plugins---part-3---rake-tasks-generators-initializers-oh-my/
http://www.themodestrubyist.com/2010/03/22/rails-3-plugins---part-4---more-on-generators/
I wouldn't use engines. Engines are meant to be used to share functionality between apps. What you want to do is not to share functionality but to organise it.
In order to organise your code you can do a lot of things.
Use namespaces.
Keep your controllers thin.
Keep your models thin (Yes, your models; see next bullet point)
Use the Data Context Interaction (DCI) pattern.
Use a widget framework like Apotomo, or Cells.
Write tests, so you can refactor.
Consider a Service Oriented Architecture(Consider Hypermedia API design) if your app's responsability grows too much.
Andrzej has very good articles about DCI.
http://andrzejonsoftware.blogspot.com/2011/08/dci-patterns-how-to-write-dci-contexts.html
One application, one database.
Share the models between the modules.
Namespace your controllers and views.
Test.
You can also use Rack Middleware to handle specific tasks.
For the bulk of your application, Engines seem like the best solution – it's something I'm looking at doing too. Looks like you can define generators in the Engine easily enough.
Depending on the structure of the data, and access patterns, it might be useful to separate the app into multiple apps, and possibly, additionally, provide data access by (RESTful) APIs.
In my experience, this allows for the best structures when your application(s) grow from middle to large size, and forces you to think about structures and separation of concern. Scaling up is also usually easier.
For the first time I'm creating a quite complex Rails app.
I'd like to know what's the best way to organize that app by folders. Until now, I'd do everything under one app (all the models, controllers, etC) but reading some open source code I realize that they put everything under different apps.
Like for example Spree Commerce. They have a general folder and inside that they have different apps (API, core, admin, etc). How is that done and is that the best way to do it?
I'd like to get pointed to the best way to do it (a book, blog, anything) so I can understand how I can architect my app for future maintenance.
thank you
As an aside I think the title of your question is a little confusing. Rails, by using convention over configuration, defines 'how to organise a Rails app'. I think your question is rather about how to architect your application as opposed to anything Rails-specific. Maybe tweak the title?
That aside, without knowing any more detail about your project it's a tricky question to answer, but I'll give it a go.
All applications should start off simple, if you believe (like I do) that you should start by building the simplest thing that could possibly work. Given this, since you're using Rails, then in all likelihood the simplest thing would be to structure your app as a vanilla Rails 3 application. This will probably (I say 'probably' because I don't know any specifics about the app) allow you to get a beta version of your app up and running pretty quickly without worrying about complexities which at this stage in the development of your project are not a problem.
If you need to create an XML or JSON-based API then Rails makes this really easy using the standard framework, which will allow you to spend more time thinking about the API design than how to code it, and it's the API design which is the most important thing to get right in the first instance.
Similarly, your Admin site can be part of the same app just in a different namespace. If you find later down the line that you want it as a separate app, you can do this (maybe you could use the awesome API you designed to facilitate this), but why bother designing it with this added complexity (and hence extended development time) in the first place if you don't have a good reason for doing so?
Once you have your app up and running and people are starting to use it, you start to get a picture of where the bottlenecks are and where the design could be improved. At this stage, if there's a need, you can start to move parts of the app to scalable solutions, such as running your API as a standalone service, introducing caching, changing data stores and other improvements and optimisations.
Even if your app is as wildly successful (and I hope it is!) then re-architecting your application whist continuing to run the existing service is still entirely possible, as Twitter have proved. Just stick to Knuth's statement and you'll be alright.
Regarding reading material, that's a tricky one. For me a lot of the XP and agile development classics taught me a huge amount about how to approach program and app design. I'd also check this StackOverflow topic for book inspiration.
Good luck!
Spree uses Rails' Railties (Rails::Engines). Railties are introduced in Rails 3 to make it more modular and easy to extend. Rails 3 itself is a collection of Railties (ActiveSupport, ActiveModel, ActiveRecord, etc.).
If you are developing a complex app I would suggest spending some time planing its' architecture. Designing a complex app without any initial planning would definitely end with a maintenance nightmare down the road. It also introduces a huge learning curve for the new team members, slow down your new feature introduction and of course, frustration.
Anyway, don't over optimize, but don't forget to design your architecture for your needs.
IMHO, I will create very complex projects as one app. I have reason to believe that Spree and Radiant build under seperate apps so that under the pretense of their open source communities, contributors can contribute code easily without tampering with the core data, and the core workings of the application.
Otherwise, you should be alright just building it as one app. Just keep it neat.
Here is what have kept me sane for several years of RoR development:
I use Rails Engines, but keep them in same codebase as the main app. Here is good starter for modular Rails app:
https://github.com/shageman/the_next_big_thing
Wherever I can I try to reduce coupling and use composition to make things easily testable, reusable and maintainable. This helps to eventually extract module or engine as separate gem. Composition is done by routes (mounting), directory overlaying (assets), dependency injection or configuration.
If I don't need to re-use an engine I put it in the same code base as the main app which is single deployment unit. Thanks to that I don't need to switch between projects in my IDE. While in development environment any changes to the engine code are instantly picked up by Rails reload mechanism.
I'm working on a cms and wanted the ability to offer custom extentions for certain accounts. Like having a plugin with custom code that is only available or only used by that account. These custom extentions would be specific to the business needs of an account and perhaps unlikely that any other accounts would need it, but maybe. Is there a way that this could be done and to be loaded without having to restart the whole app, thereby creating downtime for the other accounts?
In terms of per-client plugin code, you could store the code in a data model and then eval() the code to dynamically execute it. (Of course you would want to do some serious sanity-checking / scrubbing on update to ensure the code does not contain malicious calls). Another approach could be to develop a custom tag library, much along the lines of what the Radiant CMS developers have built ... and then let your users "program" their behaviour using the tags provided. This gives you more control and better security at the expense of less flexibility.
In terms of the downtime question, if you are using a modern rails deployment approach I don't see how this should be an issue. The eval() approach above doesn't require a restart (unless your custom code calls "include ..." on libraries that are not installed at the time of the last boot - but getting these libraries installed is also an "out of band" problem that you would need to solve.
Passenger gives you the restart.txt file that you can touch to force a refresh. Similarly there are recipes for mongrel (like see saw) that allow you to progressively restart your mongrel stack to avoid downtime. I would pull these two issues apart mentally if I were you, as the dependencies between the two are not that great. Hope this helps.
I built a cms and added plugin support for it. Best thing you can do is have it be all database driven, the plugin exists for everyone, technically, but you can only make use of it if you've "purchased" it, or some other way of turning it on.. Which is really just a db record.
That'd be 0 downtime. :) Then again, I have no idea what the rest of your setup looks like. I'd think your solution is going to be pretty specifically tailored to your cms system design.
how long would this downtime be really? i mean running migrations and stuff would be a pain for a system that allows any tom dick or harry to upload a plugin. You'd have to verify that the migrations were set up correctly etc. if you aren't getting that 'fancy' and just allow them to do something 'neat' in js, then i guess it's a question of restarting passenger, which is what 5 secs?
I'd check out other 'famous' CMS like radiant or something to see if/how they do it, personally. good luck.
Im not if this is exactly what you are trying to achieve, but have you checked out pancake?
Pancake is a tool & framework to let you stack and loosely couple Rack-based webapps.
http://www.rubyinside.com/pancake-rack-webapps-stacking-2863.html
I have a client that wants to take their Rails app that has been successful in one niche and apply it to another similar niche. This new instance of the app is going to start out very similar: all the same functionality, different logo and colors. However, if the new site is successful it will inevitably need significant customizations that shouldn't be applied to the original site. At the same time, if bugs are fixed and improvements are made to one app, then both apps should be able to share those improvements.
Can anyone suggest strategies or resources that address this issue? How do I keep changes that apply to both apps from taking significantly longer to test and implement?
Yes, I know the answer involves SCM, plugins, gems, and Rails engines. These tools will and are being used. but I want to know when and how to use these tools towards solving this problem.
Links are also welcome.
This question is not the same as:
Multiple websites running on same codebase?
In my question, I'm not running the exact same app with different settings.
How do you sync changes between multiple codebases? I'm asking a similar question, but I'm specifically asking about Rails apps.
We currently work with a setup quite similar with what you are describing.
We started developing a somewhat big Rails app (sales, stock management, product catalogue, etc) for a client. After finishing it, there came several new requests for almost identical functionality.
The original app, however, had to keep being maintained, adding new features, correcting bugs and whatnot.
The extended ones needed to maintain most functionality, but change appearance and looks.
What we did was follow a series of steps:
First we started cleaning up the code, pulling hardcode references to tables, reducing and optimizing queries, looking up missing indexes and ways to improve our ActiveRecord use
After being somewhat satisfied, we started developing missing tests. I can't stress hard enough why it's useful, since we'll be maintaining a same codebase for several apps, and need the core functionality to be as protected as it can be from new changes.
That was also the magic word: core functionality. We started selecting base functionality that could be reused, and extrating all generic code. That gave us a mix of controllers, models and views, which we started to change into modules, plugins and gems.
What goes where? Depends greatly on your code. As a rule of thumb, functionality that doesn't deal with the domain language goes to plugins (or gems if it doesn't depends too much on Rails)
This approach led us to a several of plugins, gems which we then pulled together reassembling the original project, and then it got to it's own GIT repository. That way, we had a main "template" repository which glued all the components and several other GIT repositories for each of them.
Finally, we develop an easy theme system (basically loading /stylesheets/themes/:theme_name/ and getting theme_name from the DB). Since it's an intranet project, we could almost do anything with proper CSS styling. I'd guess for working with IE you'd need a more complex approach.
Then, we just used that main repository developing the new functionality on top of it.
Now, how do we deal with changes to the core base. We start with our template repository. We fix or define where the fix or change should be and either change it there or on it's corresponding gem/plugin. After properly testing it, we deploy it to our GitHub account.
Finally, we merge/rebase the other projects from that template repository, getting the new updates.
Sounds a bit complicated, but it was only for the setup. The current workflow is quite simple and easy, with the given advantage of working with several developers without bigger issues.
With minimal touching of the main site, it might be possible to use the Ruby code from it while extending the templates and changing the styles. I have worked on that extensively in Django and the layout can look like:
project/
sites/
site_one/
templates/
models.py
settings.py
urls.py
views.py
site_two/
templates/
models.py
settings.py
urls.py
views.py
base_app/
settings.py
You could try do something similar in Rails:
main_webapp/
app/
config/
...
sites/
site_one/
controllers/
models/
views/
site_two/
controllers/
models/
views/
Assuming the functionalities are identical across sites but they just have different layout and styles, there will be none or very little model and controller code. Should you wish to add more functionality to specific sites, just stick the code under the desired site folder.
Django also have the concept of Sites and the ability to look for templates in one specific project folder and an app folder. You could try to copy those features and bring them over to Rails to achieve running multiple site from one codebase.
I recognize that you are looking for a Rails solution but you can still checkout how it's done in Django and copy some of the useful features to the other side. If I like a Rails specific feature, I'll port it other to Django/Python.
We're doing something similar at my company. Except its currently involving multiple environments (production, test, development). We're using SVN as our SCM to keep our code straight and lets us duplicate the current stable environment and create separate versions of an application (and potentially changing things like the logos or certain functionality). I highly highly recommend running the environment with Apache/Nginx and Phusion's Passenger. This is letting us run all of these applications separately, on the same/similar codebase(s). And that's it. We have to DBs, one Production and one Development to keep our live data separate, but you can easily connect two app instances to the same db this way. Its worked out really well for us so far in being able to develop, test and deploy multiple web applcations without taking down the primary production server.
I know this could be possible using Git Submodules