There is a production system that has been running for many years, first as a PHP application, then as a hybrid with Rails, and now completely in Rails. It's unclear how long it has been around. The oldest git commit is from 5 years ago.
The goal is to keep the system running at all costs. It doesn't matter what code we use as long as nothing breaks. Currently it's at Rails version 3.2.33.
If we don't upgrade any gems we run the chance of becoming obsolete and undeployable. If we upgrade we will need to make changes to the code causing potential bugs to creep in. Not only do we face code rot, but also downtime due to AWS outages.
What would be the first step to make sure nothing breaks? I've spent months writing cucumber (integration) tests but it's hard to cover every edge case. The app has been running so long that most bugs have been fixed and there are few new exceptions. Testing was not a priority from the beginning so most of the code is undocumented.
Honestly, I find that Ruby on Rails is not ideal for this sort of application. Both Ruby and Rails have a very aggressive release schedule, and Rails especially isn't afraid to ditch backwards compatibility. Rails is great for agile development where things are always changing, but at the cost of long-term stability.
I'm assuming your application is big enough that you don't want to switch to anything else. Sinatra, for instance, doesn't change much and would be a much more stable option.
If you're sticking with Rails, I'd recommend getting rid of as many gem dependencies as possible. There is always a danger that they will no longer be developed or that they could introduce bugs or vulnerabilities.
Also, as much as possible, it's a good idea to favor POROs (plain old Ruby objects) over Rails dependent code. It generally takes more work but you wind up with more stable and reusable code.
I realize that may be more work than you want to put into an application like this, but that's my best advice.
The first step would be to put specific gem version for all gems used in the gem file.
For exmaple
gem 'rspec-rails'
might become
gem 'rspec-rails', '2.14.1'
You can figure out which versions are currently being used by looking at your Gemfile.lock, for example, this line in Gemfile.lock shows the version selected for rspec:
rspec (2.14.1)
and even if the Gemfile has no version, e.g.
gem chronic
the Gemfile will have the version used, e.g.
chronic (0.10.2)
If we don't upgrade any gems we run the chance of becoming obsolete and undeployable. If we upgrade we will need to make changes to the code causing potential bugs to creep in.
Yes, that's your dilemma. There isn't any magic, you have to pick which of these two priorities you want to address. As aNoble indicates, RoR is not a framework that can 'stay in place'. constant change of the gems that combine to make up most applications means that RoR applications do not age well.
You should explain and repeat, repeat, repeat this to the project owner. Frequently this is the kind of principle that is "accepted" - but not really - as the same questions continue to be asked "despite that, how can I upgrade it, how can I make sure nothing changes or breaks, etc."
If the application will be retired in a couple of months, no problem. If there is no real plan to sunset the application and it will continue to be an important part of the business then you just have to use resources to maintain it. no free lunch in RoR land.
Writing many tests is a key first step, glad to see that. Check your statement coverage to see if you're missing tests of important areas of code, and be sure that you have integration tests that cover the key sequences. The idea is to modify your tests to reduce the risk that a change will cause a user-visible failure.
Now setup continuous testing on some system.
Set your Gemfile and .ruby-version so that you have specific control over exactly what versions of everything gets loaded. That doesn't automatically update - but it ensures that you have control over what you update. Check in both Gemfile and Gemfile.lock.
At this point you can slowly increase version numbers. Don't jump lots of version numbers - it's typically better to upgrade slowly so that you can see deprecation warnings. Fix those, rinse, repeat.
Modify your (input) validators to be picky whitelists ("it must be of this form or I won't accept it"). If you can prevent bad data from entering your system, it's more likely to work correctly and will typically be harder to attack.
For security, consider adding secureheaders, and set CSP as strong as you can stand.
Start adding some static analyzers. Rubocop and Brakeman are very useful. You'll probably have to configure Rubocop to only complain about a few things, and then slowly increase what they report. Add all your checks to the default "rake" command, so that you can just type "rake" to run static analyzers and the test suite.
There's no magic, regardless of what framework you use. People make mistakes, and pretending otherwise isn't helpful.
You might the CII Best Practices badge project a useful example. It uses RoR, and I lead that. In particular, see:
* CONTRIBUTING
* Security information (assurance case)
From CONTRIBUTING:
"In general we try to be proactive to detect and eliminate mistakes and vulnerabilities as soon as possible, and to reduce their impact when they do happen. We use a defensive design and coding style to reduce the likelihood of mistakes, a variety of tools that try to detect mistakes early, and an automatic test suite with significant coverage."
Related
whenever some major rails version bump happens, sometimes its real pain to upgrade existing apps for rails (4.x, 5.x etc), given there are plenty of gems which might depend on older versions of rails (ActiveRecord, ActionController, ActiveModel, etc.). And those dependent gems sometimes takes too much time to be upgrade (either not maintained, less active or unavailability of core maintainer team, even for accepting Pull Requests). What is the solution which people around follows for the same?
One workflow which people suggest is fork all the dependent gems and then change whatever you like and keep updating them from their respective master/main branches. I would love to hear what people follow in their workflow.
I would argue that you should remove/replace gems that do not have a compatible version available shortly after a major Rails update.
Beta versions of Rails 5 were available almost 6 months before Rails 5 was released. If a gem doesn't update within that time it will probably not be updated later on or it will slow do down on every other Rails update as well.
I see several options when you run into that situation:
Revisit the reason why you are using that gem. Sometimes you use a gem for a few simple methods. Rewrite those methods in your application (or your own gem) and remove this dependency.
Are there other projects available that solve the same problem? Sometimes these projects even have a better structure or other benefits. You might consider switching to that gems. Hopefully, you encapsulated the old gem and didn't customize that gem too much, because that makes it harder to replace it.
You might consider forking that gem and fixing the problem yourself (bonus points for sending a pull request to the original gem). That is an okay solution in the short-term. But you will very like have a similar issue with this gem with the next Ruby version. Furthermore, when you work with a forked version of a gem, you are on your own. Do you really want to maintain that private gem version for the following years?
With every gem that you add to your stack, there is a cost and a risk coming into your application. The cost of watching that gem, how does it change, are there security fixes? And the risk that the gem will be abandoned at some time.
There might be some things you can do upfront to decrease the pain when you run into situations like this:
Avoid adding gems to your application, when you only need a few simple helper methods from that gem.
Encapsulate that gem with your own wrapper. And have good test coverage for your wrappers interface. That makes it much easier to change the underlying gem.
Always be prepared to remove/replace gems as soon as you face a problem with them. Waiting for an external gem maintainer is probably not a good strategy when you have problems in your application to solve.
I have a large ruby on rails 2.3 which was now a disaster because of the slowness and many bugs. I'm the only programmer and every day I've done debugging and tearing my hair off because of this. The users are already using the product but so many bugs and data are scattered.
I was employed without prior knowledge of project development and management. Now I'm suffering of having more overtime and a crisis on my codes to be fixed.
And also I've created this app while learning rails so there are codes there that became stranger to me.
What should I do?
What are your suggestions?
What books do I need to read about more?
Please I need some help.
Thanks.
I would love to recommend you upgrade to Rails 3. Especially since there are many newer features and some things are simplified, and it would ease future maintainability.
However, unfortunately, I am hesitant to (or rather simply cannot) actually recommend that given that you already have much more on your hands.
In this case, the best thing you can do is to start writing tests. If there are so many bugs, I have to assume that either you have no tests or your have an incomplete test suite. Tests will help to give you confidence that you do not break anything when you try to fix something else.
The default rails test framework can be found at the Ruby on Rails Guides. Having said that, many people prefer the RSpec testing framework. There are indeed shortcomings of the default Rails testing framework (notably the fragility of fixtures - try get a factory gem, and other features such as mocks and expectations, and nested contexts).
You should read up on the testing frameworks, and maybe try it a bit. Pick one testing framework early on however, and start testing everything!
Perhaps when you become more confident in your test suite and have fixed the most important bugs, you should think more about a path to upgrading Rails - because all the gems will march on, and gradually drop support for Rails 2.3, which means you will be using increasingly old gems which may no be well supported anymore.
From what I understood, you are asking for project managment tips and tools how to get a rails project under control.
I believe first thing you need to is stabilize the project. To do this, you will need to minimize the bugs and chart the required work.
I see two complementary approaches for this:
use a task/bug tracking tool
start using cucumber for testing
Task/bug tracking
This is very important, because you will need some kind of list that itemizes all bugs.
Sometimes users discover a bug, and suddenly you have to drop everything, because at that moment, that single bug is the most important bug ever, and needs to be solved immediately.
However, if you would ask them outright if this means the bugs you are fixing are more or less important, the answer could be different.
So it is in your advantage if there is a clear way to let the user participate in that decision process. If there is a shared bug-list, users can also follow the current state (what you are working on), they can indicate/choose which bugs are more important for them.
Secondly: having a list of items(work/tasks/outstanding bugs/...) will also help you planning the work.
There are a lot of options to some kind of bug-tracking, but some easy/pragmatic/free suggestions are
checkout trello
use the issues from github
Tracking the bugs/tasks will give you the feeling you gain control of your project, and furthermore: it will make this also more visible to your client.
Cucumber
When fixing bugs there is always the danger to introduce new bugs, definitely in a project that is originally not your own.
In a project where there are next to no tests, I always propose to start with cucumber. Cucumber has a few advantages:
it tests your application/website from the outside in: no need to understand the code fully, you just need to know what the application should do. If I click this link, it should take me to that page.
it is really easy to write tests in cucumber, and you get test-coverage really quickly
as a bonus, your test-code is readable, which you could show your clients/users, and they would actually understand what is covered by the tests (and could correct/improve it).
Upgrading or not?
I personally believe your first step should be stabilizing the project and minimize/remove all bugs. Whilst upgrading to rails 3 would be a huge improvement, it is not a straightforward process. There are good guidelines, but if you do it now, you will have no idea if a bug was introduced during the upgrade, or existed before. First get your code quality in order, and then do the upgrade.
Hope this helps.
Actually the thing you are asking is completely depending on how much refactoring you need to clean up the whole project. If you have enough time in your hand to clean it up completely. I would suggest following steps:
Getting Ready
Get the visualization of the whole project. What is required and what is not required.
Define your resources and relation between them properly.
Use proper RESTful routing.
Decide the test tools and frameworks like cucumber, rspec, factory girl etc.
Plan of Action
Decide (if possible, as a team) that what are the minimum or necessary changes required.
Isolate all the components in different groups so that each group can be refactored individually. Smaller groups are preferred.
Decide test cases.
Break down tasks into as much as small size possible.
Make sure to keep your test coverage more than 90%.
This process will take around 4-5 months for a medium scale project for a team of 4 members.
Let us know if you have any specific confusion.
What to read: http://guides.rubyonrails.org/v2.3.11/ Start with chapters about debugging and performance testing.
Forget about upgrading to Rails 3 for now, at this moment it would only introduce many more bugs and probably lots of problems with legacy gems and plugins. And don't forget that you need ruby 1.9 in order to upgrade to rails 3 - yet another batch of problems.
Adding tests is a good idea, as ronalchn suggested. I'd recommend to start with unit tests. You might have to rewrite code a lot in order for it to be testable. (In other words, instead of trying tests to fit current legacy code, it's usually better to refactor the code to make it testable.)
Unfortunately there is no quick fix in your current solution. If you seriously consider an upgrade, it will take time to fill in the missing tests if you haven't been writing them, pick up the testing frameworks on top of Rails 3, and work out the necessary data migration once you are ready to flip the switch.
The other option is to continue with Rails 2.x, which isn't completely unfathomable although the support avenues will be much more limited. You still need to work in the tests as a first priority and understand the various nuances that are present in the existing application.
For your scenario (one-man racket with little to no prior experience), I feel that sticking with what you have and improving the testability up to the point where you would feel an upgrade is worthwhile, would be the prudent action. No matter which course of action you would take, be prepared to put in quite a bit of work in the short-to-middle term (but such is the case of accumulating technical debt).
I am using Ruby on Rails 3.2.2. and I would like to know if the following thought (born from my previous question) is correct:
I tend not to use third-party gems (at least when it is possible and reasonable) because they might be abandoned at any time. Also, if the Ruby on Rails framework changes, I might have to wait for those gems to be updated before updating my application to the latest RoR version.
In a perfect world where you're immortal, have infinite time, are the world's best programmer, and it makes no difference whether you launch your product in 1 hour or 10 years so you can code everything yourself from scratch, then maybe it makes more sense to avoid the code of other people.
But in the real world, people have solved problems for you already. And gems allow you to plug in those solutions.
A good rule of thumb is to prefer the community's favorite/popular gems for a given solution when you can. The more people using a gem, the more people have an interest in keeping it updated and the more eyeballs are scrutinizing it and sending pull requests. Chances are that a gem that's battle-tested by a bunch of people in production environments is better than what you'd come up with on your first attempt, right?
Gems that depend on one hobbyist maintainer that's not even using it in a self-itch-scratching production application are where you tend to find some risk. But even in those situations, if you end up having to fork his gem one day, you're still ahead of where you'd be if you started from scratch.
A better tendency is to avoid writing everything from scratch and instead leverage the brains of other people with similar needs.
Well, it's the risk you're taking in exchange for saved development time/budget and shorter time-to-market.
If you find yourself in a situation when one of your gems is abandoned, then you could look at similar gems or fork and improve that one. Either way, it's still better than developing everything by yourself.
Abandon this philosophy! A lot of websites/companies tend to do this -- it's better known at the "Not invented here" syndrome. Twitter, in particular, is a big offender. Open source is great -- use it.
A work is perfectly finished only when nothing can be added to it and nothing taken away.
– Joseph Joubert
That's the same philosophy we follow with our Rails projects at work. If you want to add a gem to the Gemfile, you basically have to argue it in. If you can't find a good reason why it should be in there, it will get kicked out. We are definitely not trying to reinvent the wheel, but keeping the number of potentially badly maintained moveable parts down is IMHO a good choice. So say yes to well-maintained and established Rails plugins, but be wary about stuff that looks like a quickly thrown together solution. More often that not you end up maintaining your own patches of it or forking the repo, so you might have as well reinvented that particular wheel...
Try, for example,devise gem, and then implement its functionality yourself. So you can define your own vision of the problem.
I prefer to use all gems, which I found useful. If after update any gem doesn't work, you can always fork and upgrade it.
There are lots of questions about the dreaded Could not find a JavaScript runtime error when running rails or rake. While the solution is clear, why is it standard to leave the JavaScript framework out of the Gemfile?
The only reasons I could think of are:
Fuzzy crowd-sourcing of the testing of the app with different JS frameworks.
Force newbies to learn about Gemfiles before doing anything useful with Rails.
Both of these seem to do more harm than good to the Rails reputation. Isn't the point of Rails to be up and running as soon as possible? Seems to me that the principle of least astonishment and convention over configuration would dictate that developers (and especially newbies)
shouldn't have to care about (and much less compare) JS frameworks before starting work any more than they do for SQLite vs. PostgreSQL,
shouldn't have to do more configuration before running an every app, and in summary
should just download, install and run, like with pretty much every mainstream app in every other language these days.
I have always found that a lot of rails developer will usually defer a lot of tasks to gems rather writing code themselves, and sometimes I find myself not using them at all due to the fact that the gems doesn't behave exactly like how I wanted to in some minor way, is this weird to any of you?
Btw, I do use gems that perform cryptographic tasks like bcrypt and mysql2 for database connections but I never find myself using gems like Devise/Authlogic for authentication.
You can forget about Rails and ask yourself - "whether I need standard libraries or do I need to create my own?"
Reuse of existence code is a good practice in the software development.
It is in the Rails philosophy to use gems to not reinvent the wheel and above all to do more with less.
Good gems should be configurable as you could use it for your needs.
You asked whether it's weird?
No, it is not. I often have the same problem to solve. When you use a gem, you often need to adapt it to your needs. When you update this gem, you may need to apply your changes again, or you have a problem when the gem starts to act in a different way you want it.
When you create your own solution, you know it well (so, you don't need to deal with other persons' bugs, just with your own ones), the solution always exactly matches your needs and you change it whenever you need.
The examples of gems you use (bcrypt and mysql2) are the good examples of some very general tasks, which do not need any modifications in the lifetime of your application. You will not need to change the crypto protocols, neither the database connections need application-specific changes.
But the authentication system may require adaptations, so it's better to depend on your own, independent solution. However, you may want to copy 'their' solutions, without making your project dependent on 'foreign' code. The project gnulib takes a similar approach.
UPDATE:
See the article http://www.aidanf.net/rails_user_authentication_tutorial. There you will find a few reasons for writing your own authentication system, explained much better than I tried ;)
I think I agree with the position to not use plugins/gems every time you want to add a functionality in your rails apps.
I feel like some gems or plugins may behave oddly when there are associated with others gems or plugins. I think about the act_as_*, authentification systems..
For example, to build your own tag system, which is quite easy, you can study act_as_taggable and try to write your own. I think the time spent will be the same than having to learn the plugin.