When developing a Rails app with a legacy schema that is in-use by an existing application, if tables have NOT NULL constraints on foreign id columns throughout the schema, in order to create/save models for tests, models need to exist for those associations, and their associations, etc. So, it isn't an easy as just creating one model at a time as you need it and testing with it.
As far as testing goes, this seems to be a problem if you are using FactoryGirl and want to create and save model instances to return from controllers, etc. when all of the association dependencies are involved. Another option is to mock, but mocking can be a little more time consuming and it doesn't allow you to do integration testing as easily. Another is to use fixtures, but those are time consuming and brittle. Another is to pre-populate the test database with production data, but that doesn't solve the need for factories, etc./known data to expect in tests, and Rails usually expects to start with a clean DB for the test environment.
What strategies for developing models, tests, etc. do you use when you have an existing complex schema that you are bolting a Rails app onto- not just for reading data, but also writing to the existing schema that is in use by an existing production application? (i.e. the "rebuild the ship while at sea" problem)
When we first started out on this problem, I was looking for something that could autogenerate models from the existing schema. I found Dr. Nic's magic model generator, but it only stated that worked in Rails 2.x. I got a Rails 2.x environment running (forget the specific version it worked in) but it didn't seem to help much, so I wrote a script to generate our models. However, when we started development we then had a load of models, so we started to try to move out the ones we didn't need, then needed to comment out associations to models that were no longer there, but some of those were required/NOT NULL foreign keys, so we're now having to move them back in and uncomment those more than we thought we would.
We did write and uncover some things that were helpful during the process, not all of which we've used, but they might be helpful to others:
https://github.com/influitive/apartment
https://github.com/karledurante/secondbase
https://github.com/drnic/composite_primary_keys
https://github.com/garysweaver/activerecord-define_nils
https://github.com/garysweaver/activerecord-native_db_types_override
https://github.com/garysweaver/modelist
https://github.com/garysweaver/stepford
https://github.com/garysweaver/structure_sql_to_migration
As for test data setup, I started writing a tool to automate the development of FactoryGirl factories called Stepford and wrote Modelist to help us test models, resolve circular dependencies, and identify model dependencies, so we don't have to include all automatically generated models.
So far really, the only answer I've come up with and have heard other say also is that rebuilding an existing application, even piecemeal, with different technologies is difficult, slow, and error-prone.
Related
I am new to ruby on rails and I am working on a web application written by ruby on rails. It has more than 10 models and I need to add some new attributes to some of the models as well as new methods and views. I also will need to remove or enhance some of the functionality. I know that I would need to generate new migrations and from there add/remove new columns. Then in controllers, add/modify methods, and update the views.
I wanted to know what would be the best steps (and in which order) for doing the above tasks. Also, do I need to change other files in folders like test or any other folder? What things should I consider to minimize the troubles later?
Thanks in Advance.
Since you are new to rails, the first thing you should do is to read through the getting started guide. This will help you understand the fundamentals of the rails framework and the application you inherited. After that, there are several other guides worth reading (from the same site) that may be directly applicable to the work you are doing.
Another incredibly helpful resource is railscasts. Some of these are outdated, but they are still a great starting place and can help introduce you to both new, powerful gems and rails techniques to get the work done faster and better.
As to your specific question, rails is built on an MVC architecture (meaning Model Views Controllers). It is in your best interest to try and follow this practice whenever possible. Reading up on this will clarify some of your questions as well.
When you run a migration, you will be modifying your database. The changes are viewable in the database schema (which should never be modified by hand, always modify it through migrations). This will add attributes to your models whose table you modified. In the controllers, you will add logic to deal with all of these things and the views will present the data to your users (or allow users to enter data). Asking which order is best is probably rather opinion based, but I would say you should modify the tables (run needed migrations) first. This way you can then generate the logic to deal with the new attributes. I would then create the controller logic and finally the views.
You also ask what other files need to be changed. This is heavily dependent on your system. At a base level, you should definitely be writing tests to support the logic you are generating (and many developers will advocate that you should do this before you write the other logic, a process called Test Driven Development).
TL;DR: Read the guides, work through a basic tutorial, and watch some Railscasts. This should get you up to speed on the process and best practices that go along with rails development.
Perhaps a stupid question, and without code, so I'm not sure I'm on the right StackExchange site. If so, sorry and give me a comment please.
I begin programming in Sinatra (only intranet until now) and in the samples they almost always use migration with activerecord, the same with RoR.
I have experience enough with activerecord itself and it is very helpfull but I'm not sure why migration is always used ? If I create a project I just create a SQL script or a Ruby activerecord script that does the creation of the DB for me and that's it..
If I need the site or script somewhere else I just need to execute that script and ready.
Obviously I'm missing here a lot, so who can me explain the other benefits or point me to a good explanation ?
From Rails docs:
Migrations are a convenient way to alter your database schema over
time in a consistent and easy way. They use a Ruby DSL so that you
don't have to write SQL by hand, allowing your schema and changes to
be database independent.
So, the main two benefits are:
It's like Git for db schema, you won't know how that's useful until you are in a mid-size or big project with many contributors and someone makes a bobo :)
You write ruby code to build your db schema, this way if you decide to move from mysql to pg for example, you don't need to open up pg manual and check code compatibility
Update
In the api docs of migrations, you will find many nice use cases (to be honest i didn't know about half of them) ... check it out (http://api.rubyonrails.org/classes/ActiveRecord/Migration.html)
Building a creation script for your database is great, provided two things:
All your database deployments are on new machines
You get your database schema exactly right the first time
In our modern agile environment we not only don't believe it is possible for a project larger than a few lines of code, we also don't encourage aspiring to it.
Active Record's Migrations offer a way to describe your database incrementally. Each "migration" defines how to add its features to the database ("upgrade"), and how to remove them if necessary ("downgrade").
When running migrations, on top of actually running the scripts, Active Record also maintains its own table (schema_migrations), to remember which migrations have been applied to the database, and which are pending.
This enables you to build your database alongside the features as you develop them. It also facilitates working in teams, since each team member develops her own migrations, and Active Record "stitches" everything together, so you don't have to maintain a monolithic creation script.
I have been picking up on Rails best practices and learning to write good tests for my rails application.
I am using rspec + FactoryGirl to test at the moment.
For a while I was writing basic model and controller specs which usually required to set up only a couple of models and associations.
Now I have a Model with a complex algorithm which does a lot of calculations based on different models.
To test this method, I almost need all the models of my app to exists in test database with all associated relationships built between them.
Now things are getting overwhelming. I need to keep track if all factories are correctly referencing each other. Also I need to verify if all factories have been created properly.
Also a lot of FactoryGirl.create(:model) calls in my before(:each) method.
I have a feeling that I am not doing things right. What is the best way to go about this situation? Most books and examples only cover only the very basic cases.
Any pointers ideas and approches are welcomed.(Hope this question is not too wide in scope)
Example:
My application has multiple projects. Each project has_many indexes , companies and mastertags. Each company has_many channels. Every channel has_many channel_tag_weights across different mastertags. Also every channel has_many Rawdatas.
This could be an example of a minimum model setup required.
I wrote a gem for this called stepford.
In your gemfile:
group :test do
gem 'stepford'
#...
end
Then bundle install.
Back up your factories.rb or existing multiple files containing factories. In fact, you need to move them out of the way because if it loads other factories, that can interfere.
Then: bundle exec stepford factories --path spec (if want to put factories.rb in the spec dir- there are other options to create a factory per model, etc. though).
We had problems with dependencies that needed to be created where there were foreign key constraints. In the process, I wrote the modelist gem that can help identify circular dependencies, which you might want to check out and run if you are working with legacy data. And Stepford's deep creation methods get around these issues. But, we've started to steer away from using the deep_* methods of Stepford, even though we still use it a little. It is just better to be specific about what you are doing and use factory girl directly, even though it can make test code more brittle and overly-complicated looking.
You could also look at fixtures, or machinist. It really just depends on what you need.
If you need to actually create all of the code for the models, that is a different problem. I wrote a script to generate models for the local legacy schema (which was quite large), and it did help, but it took months of iterative work. The thing is that the schema itself is not (necessarily) the definitive description of the associations or setup in the models. To give you an idea of what I'm talking about, I wrote a number of gems to assist with some of the things that legacy apps, etc. were doing/had done that we had to deal with like undeletable, activerecord-define_nils, mark_only, activerecord-custom_timestamps, activerecord-native_db_types_override, and some others in earlier attempts. And we use some great gems that already existed like composite_primary_keys and foreigner.
Some other things we grappled with was using of MTI (multiple table inheritance), which despite a few gems out there that try to help, we ended up handling it ourselves in our models via hooks, etc. And we merged some schemas (not something to take lightly, but the divisions weren't necessary, and apartment, second_base and similar gems would be adding to the complexity, not even considering inability to join, use normal queries).
Please note: in our case we had a lot of tables. If you don't have quite as many, you still might use Stepford to maybe make the first creation of factories a little easier, but then I think you will be ok.
Whenever I get into this type of situation, I try to make sure that all of my FactoryGirl models take advantage of FactoryGirl's built in association methods:
https://github.com/thoughtbot/factory_girl/blob/master/GETTING_STARTED.md#associations
That way the associated models that are required in order for a single model to validate properly are present. FactoryGirl will go through the process of populating your database with the extra test data.
Because you are using has_many associations, you will want to study the after(:create) method of the factory definition in order to set up your associations properly. This will also allow you to create definitions specific to these deeply integrated specs, and keep simpler factories independent for the simpler specs.
If you are testing only a method, that is unit testing.
I don't think you need to setup too many things.
A method has input and output, you just feed it with examples of inputs created by you manually, and expect it to generate right output.
If a method has too much things to concern, you can split it into several methods, and unit testing each one.
When times for integration testing, the whole environment should be setup. And yes, you need to prepare all the factories needed.
I am using Ruby on Rails 3.2.2, cucumber-rails-1.3.0, rspec-rails-2.8.1 and capybara-1.1.2. I have this problem but I started thinking that maybe I'm doing something wrong... mostly about seeding data in the test database for testing purposes. Specifically, my issue is related to how to properly manage data in the test database when I have to test my application.
My doubt is: By seeding data (For Your Information: I use the ROOT_PATH/db/seed.rb file to inject that data) in the test database I'm doing things as they should be done? That is, how should I populate the test database since the data in that database* is required in order to make to properly work my application for testing purposes? Should I populate the test database at all?
In other words, what are best practices to handle database data in test mode (in my case)? And, generally speaking, how the situation should be handled?
***** For example, in order to work my application requires at least data related to an "anonymous" user, to "basic" articles, to "basic" article categories, etc.
You should use one of the following:
Fixtures. See corresponding Rails documentation
Factories. The most popular tool to created/manage factories is FactoryGirl. IMHO that's the best solution.
Make sure data is seeded into test database. See this StackOverflow question.
I had a similar problem, association made it necessary to have a bit of seed data:
Factories will make your tests really slow, they are perfectly OK for single objects, but not for a lot of seed data that has to be created for each test
Fixture Builder - http://github.com/rdy/fixture_builder
I created a bunch of fixtures and just load them from the DB for each test, cut my test time down by 40%. You can also load the seeds file instead.
But be careful, deleting or updating records will create unwanted side effects. Use factories for those specs.
Mock and Stub everything so that your tests rarely touch the DB.
This has become very unpopular, you will eventually get passing specs that don't pick up on your actual errors.
I'm about to begin a Ruby on Rails project and I'd love to hear how others go through the process of starting an application design. I have quite a bit of experience with RoR, but don't have that many starting from scratch with only a vision experiences and would appreciate the wisdom of others who've been there.
I'm looking for an order of events, reasons for the order, and maybe why each part is important. I can think of a few starting points, but I'm not sure where it's best to begin
Model design and relationships (entities, how they relate, and their attributes)
Think of user use-cases (or story-boards) and implement the minimum to get these done
Create Model unit-tests then create the necessary migrations and AR models to get the tests to pass
Hack out the most basic version of the simplest part of your application and go from there
Start with a template for a rails app (like http://github.com/thoughtbot/suspenders)
Do the boring gruntwork first (User auth, session management, ...)
...
I found myself looping the following tasks for most projects:
Gather User requirements
Design database Models
Build Views
Streamline Layouts
Find and learn Plug-ins/Gems
Testing
User Review/Acceptance
Deploy the app
Documentation
After these years of working as a freelancer, I think step 1 and 2 are the most important (at least for small projects). Before writing any code, I urge users to finalize all UI first. HTML skeleton is better than written document. Users do not and will not understand software specifications. They can only give feedback after they see can click something. So being fluent in building a HTML site is a helpful skill. Sometimes I delegate the task to a partner, as a SA role.
Rails is very good for building an ever-evolving schema. Try to use migrations and data seeding instead of writing SQL statements directly. I find myself rely on ActiveRecord more and more overtime. The script/console is a nice tool to test out those many-to-many relationships and building :conditions =>.
I worked on a few legacy database lately, the establish_connection and set_table_name feature in ActiveRecord glue old and new database elegantly.
I'd also like to use this chance to thank Ryan Bates, I learn a lot of Rails from his railscasts.