I'm dealing with a rails application which is growing too big. It takes long time to start and lots of memory. We are having performance issues. Tests are very slow. Managing the codebase, debugging and introducing new futures becoming harder everyday. We are thinking to split the application into smaller components (Rails engines or different Rails apps) where all components will share a single database.
Considering such scenario we will need to share the models, maybe some libs, tests, and some gems, etc... It doesn't sound right for me!
Is there any pattern can be applied?
There are some great suggestions here about to to split up a Rails code base however I think that before you apply any of then you need to stop and seriously consider how you got into this position. All of these solutions introduce new complexities and challenges. They might be worthwhile trade offs but only if you make sure they also solve a problem you have today.
Take each of the pain points you listed (startup time is slow, memory use if high, performance is poor, rest performance is poor, development speed is slow) and run a "5 whys" exercise on them. Why are these things happening. Why did the app get into this state.
Most importantly before you commit to any plan for splitting up a large app consider if the app should be large in the first place. If your app is more complex than your product demands then switching to an equally complex cluster of services is not an improvement.
More concretely I would recommend against shared database access between apps/services/whatever. A shared database has a shared schema which becomes fragile. It also leads to tightly coupled services which lack the separation of concerns you need to see any improvement in your pace of development. Just as splitting one massive class into several tightly coupled files does not improve it nor does splitting one app into coupled services.
If you must maintain a large app you need to isolate separate concerns. In order to do so you need to break the dependencies between them. Based on the efforts I have seen to repair Rails monoliths you'll have better success creating clean interface within your existing app an then splitting out components than if you split the app apart and then hope the resulting pieces can be improved independently.
Yes, there are several ways to do it:
Micro services Erin Swenson-Healey has a nice post for rails.
Hexagonal Architecture GoRuCo 2012 Hexagonal Rails by Matt Wynne and Refactoring with Hexagonal Rails
Rails Engines Approach. Dealing with Rails Application Complexity - A Report from MWRC
Use other framework more suitable for high complexity and more PORO oriented. http://lotusrb.org/
Also see Ruby Midwest 2011 - Keynote: Architecture the Lost Years by Robert Martin
Read "Practical Object Oriented Design in Ruby" by Sandi Metz. Also, watch every presentation and talk by her on the internet. Youtube is a great place to start. She's a great speaker. Rails is written in Ruby and Ruby is an object oriented language which brings with it massive benefits. But grokking the object oriented part takes a little work. Sandi will get you there.
Related
I have a large rails application with 3 separate 'components'.
One is a mostly static WWW site, one is a shopping cart based on Spree, and one is a reward-program based on Instagram's API.
Currently they are all one giant Rails 4.0 application. As this app has grown over time, I have it harder to make distinctions between components. I want to modularize the app to keep these 3 components separate.
Is there a preferred way to do this "SOA" sort of architecture? or would it be better to turn each 'component' of the app into their own mountable engine? Or is there an even better strategy?
I have been looking at Spree's core, and how they have each component as an engine, and load them in the top level, and I'm thinking this may be the best route.
I don't have any experience with Ruby or Rails, but based on my experience you will need to ask/answer the below question and then decide how you want to proceed forward.
Who's going to be developing the code base and who'll be maintaining it?
If it's just you who's wearing all the hats, you may not want to have the overhead of implementing SOA (web-services to be specific). That said, you should definitely have a 'Contract' between each of these components or modules (however you refer to them). That way your modules/components can evolve independently and changes made to one to make the logic better doesn't necessarily mandate changes to the other components.
If it's you and a couple of other developers, I'd still say that you may not want to take the WS route yet.
If it's different teams that are developing and maintaining these components, then you are taking about an application at enterprise level and then you will start seeing the benefits of SOA (based on WS).
Cheers,
K
Is it possible to merge two apps, one e-commerce application based on PrestaShop with second one "ordinary" build on Rails?
Is that even possible? Which issues I would encountered during building that application?
Or maybe I just saying "stupid things" and that is a very bad idea?
-
Second one approach is to use Spree with Rails. However I heard that Spree is much slower than PrestaShop and doesn't have much modules.
Advantages on Spree are that my skills at Ruby are much more superior than in PHP, so I would mastered Spree much faster than PrestaShop.
I have practically zero experience with both sollutions(Spree, PrestaShop).
Priority in choosing option is "how much time it would take" and quality of final application(this order).
If I understand correctly, you are weighing the advantages and disadvantages of several solutions and can't decide which one to pick, so you wonder whether it makes sense to use more than one. The answer is usually no. Even though it may be possible, multiple technologies are much more difficult to set up and maintain than choosing one single technology. I strongly advise against it.
Even if one particular solution doesn't have all the features you need (e.g. specific modules), it's usually much easier to simply write those features yourself than to add a completely different technology to your stack. I say usually because as always, there are exceptions, but only in rare cases where some highly specific feature is needed.
It's true that Rails apps are sometimes (not always) slower than PHP apps, but the speed difference only becomes relevant if you need to scale to a very large number of users (millions). And even at those high scales, Rails will perform well if you're smart about setting up your server infrastructure, make use of caching, etc.
Finally, I would personally recommend Spree over Prestashop, but that's a matter of opinion. If you already have experience with Ruby, I definitely recommend Spree.
I'm looking for tools to monitor/test performance in rails, and I'm not having much luck finding anything particularly effective. I've read the rails 'performance' guide, but I use RSpec instead of Rake:Test, so I'm not particularly keen to use the rake:test framework.
So, what do folks use for performance testing in rails apart from the rake:test benchmarker? Any suggestions appreciated
Performance benchmarking is one of those things that you'll get different opinions about depending on who you ask. One thing I hear over and over is that you shouldn't obsess over performance early on. I'm not sure where you're at with your application, but this could be something to consider. After developing a rather large application, I can honestly say I agree with them. It's better to use good practice when developing and wait to do performance tuning at a later time. Best practices include things like indexing database columns.
For performance monitoring of live Rails applications, New Relic is one of the best tools out there*. The free plan is a little limited as it only provides 30 minutes of historical data, but the information it collects is priceless. Some of the cloud hosts like Heroku and Engine Yard are offering free bronze plan upgrades, which stores a week of data. Once you have information about your application, you can make educated decisions about where to focus your time.
* My opinion
When your app needs some performance testing, the default TestUnit based performance benchmarking tests are a great start. However, you shouldn't stop there, and should consider using a variety of tools based on the nature of your application.
For example, analyzing production logs using a tool like the request-log-analyzer is a great way to identify the real performance bottlenecks. Bullet is another great tool you can run in your development environment to identify performance inefficiencies in your database calls. For low level benchmarking, rails also gives you the benchmark helper methods in models, controllers and views. This can be handy if you are focusing on tuning some specific part of your application.
It is also worth noting that rspec is not the best tool for benchmarking performance (to date). In my opinion, trying to assert things like it should_take_less_than 50 is stretching the idea of performance testing and trying to force it into the concept of BDD. Performance is less often about absolute expectations and more about identifying the slowest parts of your app and making them faster.
There are many online resources on the topic. I've found these railscasts to be a great starting point:
http://railscasts.com/episodes/368-miniprofiler (free)
http://railscasts.com/episodes/411-performance-testing (pro, requires subscription)
For the first time I'm creating a quite complex Rails app.
I'd like to know what's the best way to organize that app by folders. Until now, I'd do everything under one app (all the models, controllers, etC) but reading some open source code I realize that they put everything under different apps.
Like for example Spree Commerce. They have a general folder and inside that they have different apps (API, core, admin, etc). How is that done and is that the best way to do it?
I'd like to get pointed to the best way to do it (a book, blog, anything) so I can understand how I can architect my app for future maintenance.
thank you
As an aside I think the title of your question is a little confusing. Rails, by using convention over configuration, defines 'how to organise a Rails app'. I think your question is rather about how to architect your application as opposed to anything Rails-specific. Maybe tweak the title?
That aside, without knowing any more detail about your project it's a tricky question to answer, but I'll give it a go.
All applications should start off simple, if you believe (like I do) that you should start by building the simplest thing that could possibly work. Given this, since you're using Rails, then in all likelihood the simplest thing would be to structure your app as a vanilla Rails 3 application. This will probably (I say 'probably' because I don't know any specifics about the app) allow you to get a beta version of your app up and running pretty quickly without worrying about complexities which at this stage in the development of your project are not a problem.
If you need to create an XML or JSON-based API then Rails makes this really easy using the standard framework, which will allow you to spend more time thinking about the API design than how to code it, and it's the API design which is the most important thing to get right in the first instance.
Similarly, your Admin site can be part of the same app just in a different namespace. If you find later down the line that you want it as a separate app, you can do this (maybe you could use the awesome API you designed to facilitate this), but why bother designing it with this added complexity (and hence extended development time) in the first place if you don't have a good reason for doing so?
Once you have your app up and running and people are starting to use it, you start to get a picture of where the bottlenecks are and where the design could be improved. At this stage, if there's a need, you can start to move parts of the app to scalable solutions, such as running your API as a standalone service, introducing caching, changing data stores and other improvements and optimisations.
Even if your app is as wildly successful (and I hope it is!) then re-architecting your application whist continuing to run the existing service is still entirely possible, as Twitter have proved. Just stick to Knuth's statement and you'll be alright.
Regarding reading material, that's a tricky one. For me a lot of the XP and agile development classics taught me a huge amount about how to approach program and app design. I'd also check this StackOverflow topic for book inspiration.
Good luck!
Spree uses Rails' Railties (Rails::Engines). Railties are introduced in Rails 3 to make it more modular and easy to extend. Rails 3 itself is a collection of Railties (ActiveSupport, ActiveModel, ActiveRecord, etc.).
If you are developing a complex app I would suggest spending some time planing its' architecture. Designing a complex app without any initial planning would definitely end with a maintenance nightmare down the road. It also introduces a huge learning curve for the new team members, slow down your new feature introduction and of course, frustration.
Anyway, don't over optimize, but don't forget to design your architecture for your needs.
IMHO, I will create very complex projects as one app. I have reason to believe that Spree and Radiant build under seperate apps so that under the pretense of their open source communities, contributors can contribute code easily without tampering with the core data, and the core workings of the application.
Otherwise, you should be alright just building it as one app. Just keep it neat.
Here is what have kept me sane for several years of RoR development:
I use Rails Engines, but keep them in same codebase as the main app. Here is good starter for modular Rails app:
https://github.com/shageman/the_next_big_thing
Wherever I can I try to reduce coupling and use composition to make things easily testable, reusable and maintainable. This helps to eventually extract module or engine as separate gem. Composition is done by routes (mounting), directory overlaying (assets), dependency injection or configuration.
If I don't need to re-use an engine I put it in the same code base as the main app which is single deployment unit. Thanks to that I don't need to switch between projects in my IDE. While in development environment any changes to the engine code are instantly picked up by Rails reload mechanism.
I have a memory of talking to people who have got so far in using Ruby on Rails and then had to abandon it when they have hit limits, or found it was ultimately too rigid. I forget the details but it may have had to do with using more than one database.
So what I'd like is to know is what features/requirements fall outside of Ruby on Rails, or at least requires such contortions that it is better to use another more flexible framework, even though you may have to lose some elegance or write extra boilerplate code.
Rails (not ruby itself) is proud to be "Opinionated Software".
What this means in practice is that the authors of rails have a certain target audience in mind (themselves basically) and aim rails specifically at that. If X feature isn't needed for that target audience, it doesn't get added.
Off the top of my head, things that rails explicitly doesn't support that people may care about:
Foreign keys in databases
Connections to multiple DB's at once
SOAP web services (since rails 2.0)
Connections to multiple database servers at once
That said, it is very easy to extend rails with plugins, and there are plugins which add all of the above functionality to rails, and a lot more, so I wouldn't really count these as limits.
The only other caveat is that rails is built around the idea of creating CRUD web applications using MVC. If you're trying to do something which is NOT a CRUD web app (like twitter, which is actually a messaging system, or if you are insane and want to use a model like ASP.NET webforms) then you will also encounter problems. In this case you're better off not using rails, as you're essentially trying to build a boat out of bicycle parts.
In all likelihood, the problems you will run into that can't just be fixed with a quick plugin or a day or 2 of coding are all inherent problems with the underlying C Ruby runtime (memory leaks, green threads, crap performance, etc).
Ruby on Rails does not support two-phase commits out of the box, which maybe required if your database-backed application needs to guarantee immediate consistency AND you need to use two or more database schemas.
For many web applications, I would venture that this is not a common use-case. One can perfectly well support eventual consistency with two or more databases. Or one could support immediate consistency with one database schema. The former case is a great problem to have if your app has to support a mondo amount of transactions (note the technical term :). The latter case is more typical, and Rails does just fine.
Frankly, I wouldn't worry about limits to using Ruby on Rails (or any framework) until you hit real scalability problems. Build a killer app first, and then worry about scalability.
CLARIFICATION: I'm thinking of things that Rails would have a hard-time supporting because it might require a fundamental shift in its architecture. I'll be generous and include some things that are part of the gem/plugin ecosystem such as foreign key enforcement or SOAP services.
By two-phase commits, I mean attempting to make two commits to physically distinct servers within one transactional context.
Use case #1 for a two-phase commit: you've clustered your database, so that you have 2 or more database servers and your schema is spread across both servers. You may want to commit to both servers, because you want to allow ActiveRecord think do a "foreign key map" that traverses across the different servers.
Use case #2 for a two-phase commit: you're attempting to implement a messaging solution (sorry, I'm J2EE developer by day). The message producer commits to the messaging broker (one server) and to the database (a different server).
Also found some good discussion about the limits of ActiveRecord.
I think there is a greater “meta-question” here, that could be answered and that is “when is it OK to lean on external libraries to speed up development time?”
Third party libraries are often great and can drastically reduce development time, however there is a major problem, Joel Spolsky calls this “the law of leaky abstractions.” If you look that up on Google his post will come up first. Essentially this means that the trade off in development time means that you have no idea what is going on under the covers. So when something breaks you are completely stuck and have very limited methods of debugging. This also means that if you hit one of the features that are simply unsupported in RAILS, that you really need, you’ll have no next step except to write the feature yourself, if you’re lucky. Many libraries can make this difficult to do.
We’ve been burned badly in my dev shop by this issue. Our solutions worked fine under normal load, but we found that the third party subscription libraries that we were using simply could not stand up to the kind of load that we experienced once our site started to get a large number of concurrent users. This puts us in a very difficult place; essentially we have to rewrite the entire subscription service ourselves, with performance in mind. Doing this means that we’ve wasted all the time that we spent using the library.
Third party libraries can be great for small to medium sized applications; they can drastically reduce development time and hide complexities that aren’t necessary to deal with in the early stages of development. However eventually they will catch up with you and you’ll likely have to rewrite or re-engineer your solution to get past the “law of leaky absctractions”
Ruby don't have a functionality like IsPostBack in ASP.Net
Orion's answer is right on. There are few hard limits to AR/Rails: deploying to Windows, AR connectors that aren't frequently used, e.g. Firebird, ), but even the things he mentioned, multiple databases and DB servers, there are gems and plugins that address those for legacy, sharding, and other reasons.
The real limitation is how time-consuming it is to keep on top of all the things that rails devs are working on, and researching specific issues, given how many blogs, and how much mailing list volume there are.