I have always scripted all DB changes. But I'm wondering if I really need to do that with Grails applications, since GORM will make the changes automatically.
So what's the best practice for managing database changes in a Grails application?
This is the pattern that I've used on several large Grails projects (and some smaller ones):
In GORM we trust™ for the first
stages of development
(pre-production / without data)
Just before releasing to the
production environment start using a
tool like Autobase, Liquibase, Database Migration Tasks (similar to RoR's rake), or another schema versioning utility.
Maintain all database changes
through the tool in an automated
fashion.
Test your migrations, by writing tests that exercise corner cases and data integrity to the level that you are comfortable running them on production data.
I wouldn't use straight GORM in production unless it is a smaller project that can handle a few possible speed bumps and manual interventions.
Once you start managing several environments (local development, QA/UAT, Staging, Production) you be glad that you spent the time to manage your DB changes.
Liquibase and Autobase both give you some good tools for writing many of the common refactorings, but you can always drop down into raw SQL if you want/need to.
GORM won't handle all schema changes automatically; for example, if you delete a field from one of your domain objects, GORM won't delete the corresponding database column.
There are a couple of Grails plugins to manage database migration - see Autobase and Liquibase. I'd say that using whichever of these meets your needs more closely is the best practice; here's a good blog post that explains a few gotcha's with both plugins.
Related
I have an idea for a multi-tenant app, and I'm trying to decide if I should use one large database or use separate databases for each tenant.
I don't even know if the latter is possible in Rails, or with rails on Heroku.
I also don't know if this is a good idea, or cost prohibitive.
But I guess to start I just want to know if it's possible.
There are many approaches to multi-tenancy, each with its own pros and cons. Postgres has this nice feature called schemas, which means you can have one database but multiple namespaces inside. This can be a convenient solution for Rails, as Rails was designed for connecting with only one database. It is easy to integrate with apartment gem. It takes care of migrations and tenant switching based on specified rules, usually subdomain. But this solution has downsides. While Postgres does not have any limitation on number of schemas, when you have a lot, migrations will take forever. And there are problems with backups. Heroku recommends using less than 50 schemas.
If you want to have multiple physical databases then it is a little bit tricky with Rails. There are some gems that allow connecting to multiple databases. Recently I heard about octoshark gem, but I haven't use it.
In summary, Postgres schemas are nice if you want to have good isolation without too much work. It will be also cost efficient on Heroku, as you will use only one database. But it won't scale for a lot of tenants. Multiple databases provide the best isolation, but support for this solution in Rails is not that great I think. And it will be costly as you you will need to provision separate database for each tenant. And the last resort is to just use one database and scope all your tenant data with tenant_id. In this solution you need to guarantee isolation which requires additional work and it is easy to miss some parts of the application.
With Orchard CMS 1.6, I have four environments set up: Development, Testing, Staging, and Production.
I have made a lot of changes from the Orchard UI Dashboard in Development and I want to migrate those changes into other environments. A related question shows this can be done manually through the Orchard Dashboard using Import/Export modules, but I'm looking for a solution for data migrations that I can automate. Really, I'm looking to nail step three for SQL Server 2005/2008 database in the accepted answer for this related question: that states, "Migrate your db to production environment."
There is not much documentation out there when it comes to setting up and maintaining Orchard CMS in multiple environments outside of Azure and I have 88 tables in my current Orchard database. Obviously, anything I can do to automate data deployments would be a huge help.
I ran a Schema and Data compare between Development and Testing(which currently reflects Production). After backing up databases and replicating schema, I noticed there are a lot of data differences in almost every table. Before migrating data, I want to be sure I've isolated the tables I do not want changed. Environmental variables such as connection strings are going to have to stay unchanged.
What tables should persist in their environments?
Right now I am thinking they are:
Orchard_Users_UserPartRecord - I have users in Production I do not want to have anywhere else.
Environmental Data - I have connection strings I put in a table for a custom module that are different for each environment.
Am I missing anything? What other data should be retained in tables on destination environments?
I really wouldn't recommend a database migration: you are going to have huge problems with ids, relationships, etc. Content items are distributed onto many tables, sometimes in ways that are hard to predict, and there's no guarantee that the same id hasn't been attributed on different environments to different items or records. This is why a higher-level procedure like import/export is the recommended approach.
Automating it is absolutely something we want, and there is a feature branch doing exactly that for a future version of Orchard (feature/deployment IIRC).
In the meantime, I'm pretty sure import and export also exist as commands for the CLI, which should enable you to automate it to a degree.
I'm quite new to Rails but in my current assignment I have no other choice but use RoR. My problem is that in my app I would like to create, connect and destroy databases automatically on user demand but as far as I understand it is quite hard to accomplish this with ActiveRecord. It would be nice to hear some advice from more experienced RoR developers on this issue.
The problem in details:
I have a main database (which I access with activerecord). In this database I store a list of my active programs (and some template data for creating new programs). I would like to create a separate database for each of this programs (when a user creates a new program in my app).
In the programs' databases I would like to store the state and basic info of the particular program and a huge amount of program related data (which is used to calculate the state and is necessary to have for audit reasons).
My problem is that for example I want a dashboard listing all the active programs and their state data. So first I have to get the list from my main db and after that I have to connect to all the required program databases and get the state data.
My question is what is the best practice to accomplish this? What should I use (ActiveRecord, a particular gem, etc.)?
Hi, thanks for your answers so far, I would like to add a couple of details to make my problem more clear for you:
First of all, I'm not confusing database and table. In my case there is a tool which is processing log files. Its a legacy tool (written in ruby 1.8.6) and before running it, I have to run an SQL script which creates a database with prefilled- and also with empty tables for this tool. The tool then processes the logs and inserts the calculated data into different tables in this database. The catch is that the new system should support running programs parallel which means I have to create different databases for different programs.(this was not an issue so far while the tool was configured by hand before each run, but now the configuration must be automatic by my tool) There is no way of changing the legacy tool while it would be too complicated in the given time frame, also it's a validated tool. So this is the reason I cannot use different tables for different programs, because my solution should be based on an other tool.
Summing my task up:
I have to crate a complex tool using RoR and Ruby 2.0.0 which:
- creates a specific database for a legacy tool every time a user want to start a new program
- configures this old tool on a daily basis to process the required logs and insert the calculated data into the appropriate database
- access these databases and show dashboards based on their data
The database I'm using is MySQL.
I cannot use other framework, because the future owner of my tool won't be able to manage/change/update it. So I have to go with RoR, which is quite painful for me right now and I really hope some of you guys can give me a little guidance.
Ok, this is certainly outside of the typical use case scenario, BUT it is very doable within Rails and ActiveRecord.
First of all, you're going to want to execute some SQL directly, which is fine, but you'll also have to take extra care if you're using user input to determine the name of the new database for instance, and do your own escaping. (Or use one of ActiveRecord's lower-level escaping methods that we normally don't worry about.) The basic idea though is something like:
create_sql = <<SQL
CREATE TABLE foo ...
SQL
ActiveRecord::Base.connection.execute(create_sql)
Although now that I look at ActiveRecord::ConnectionAdapters::Mysql2Adapter, there's a #create method that might help you.
The next step is actually doing different things in the context of different databases. The key there is ActiveRecord::Base.establish_connection. Using that, and passing in the params for the database you just created, you should be able to do what you need to for that particular db. If the db's weren't being created dynamically, I'd put that line at the top of a standard ActiveRecord model so that that model would always connect to that db instead of the main one. If you want to use the same class, and connect it to different db's (one at a time of course), you would probably remove_connection before calling establish_connection to the next one.
I hope this points you in the right direction. Good luck!
I'm extending an existing Rails app, and I have to add multi-tenant support to it. I've done some reading, and seeing how this app is going to be hosted on Heroku, I thought I could take advantage of Postgres' multi-schema functionality.
I've read that there seems to be some performance issues with backups when multiple schemas are in use. This information I felt was a bit outdated. Does anyone know if this is still the case?
Also, are there any other performance issues, or caveats I should take into consideration?
I've already thought about adding a field to every table so I can use a single schema, and have that field reference to the tenants table, but given the time windows multiple schemas seem the best solution.
I use postgres schemas for a multi-tenancy site based on some work by Ryan Bigg and the Apartment gem.
https://leanpub.com/multi-tenancy-rails
https://github.com/influitive/apartment
I find that having seperate schemas for each client an elegant solution which provides a higher degree of data segregation. Personally I find the performance improves because Postgres can simply return all results from a table without have to filter to an 'owner_id'.
I also think it makes for simpler migrations and allows you to adjust individual customer data without making global changes. For example you can add columns to specific customers schemas and use feature flags to enable custom features.
My main argument relating to performance would be that backup is a periodic process, whereas customer table scoping would be on every access. On that basis, I would take any performance hit on backup over slowing down the customer experience.
I wonder how people deal with gradually rolling out features and versions in a production evironment. the scenario is where you have two versions of tested code one already in production and one to be rolled out, these are the common issues..
different versions of code within same rails app.
different versions of rails app during rollout to users.
different database structures between version
moving data across new databases and servers.
here are some ideas for the above for discussion
if statements with constant, version numbers in M,V,C names
load balance to different app servers (how to make sticky?) , RVM
have old and new fields in tables as temporary, or migrate records to new tables or
databases.
no easy way to move data between
servers.
It sounds like you need a good branching and merging strategy. If you're using something like Git or SVN, then anything on master or trunk, respectively, should be production-ready quality. If you're running into situations where the AbcController is good and ready to go, but XyzController is flaky, then the XyzController probably needs more testing and shouldn't be in master yet.
Migrations in rails also follow this policy, which lead to your data structure. If you think that you're ready for production, then there should't be significant changes to your database. Maybe you need to add a column or feature, but you should be well past wholesale database refactorings.
Finally, uploading/updating data is a pain in any migration situation. In my experience, it involves writing SQL scripts to perform the moves, or update the database for some new feature. Those SQL scripts should also be under your source control. Rails can make this easier, by writing your migration scripts in the migration file itself. Depending on your exact situation, this can work.