UPDATE: I wrote a Sequence property type for DataMapper in the end. Take and use at your own risk ;) https://gist.github.com/959059
We're moving a large, already in production PHP web application to Ruby on Rails. Our schema is far from compatible with ActiveRecord's defaults, and it's too large to simply migrate the schema, so I've ditched ActiveRecord and started using DataMapper, which allows us to hide the schema differences more easily. This is working well with some read-only tests I've done.
Now, one of the biggest incompatibilities with our schema is that we use ADODB and generate primary keys prior to the insert, using a sequences table (this is a common pattern), instead of with auto_increment.
Is there a way to tell DataMapper to generate IDs in the same way? I don't see mention of it in the docs.
We can't really switch the tables to use auto_increment because the size of the application means we're actually running a hybrid Rails/PHP setup with some proxying and session sharing so we can progressively migrate across, therefore the PHP application needs to keep working with the schema as-is (or with only minor changes).
I should really have posted that edit as an answer:
I wrote a Sequence property type for DataMapper in the end. Take and use at your own risk ;) https://gist.github.com/959059
Related
I have a Rails app and a Sinatra app, sharing the same database. The Sinatra app uses ActiveRecord.
Can I run migrations from within each app, as if they were in the same app? Will this cause any problems?
The schema.rb file in the Rails app tracks the current migration via
ActiveRecord::Schema.define(:version => 20121108154656) do
but, how does the Sinatra app know the current version the database?
Rails 3.2.2, Ruby 1.9.3.
The version column in the schema_migrations table equate to the time stamp on the front of the ruby migration file example: 20130322151805_create_customers.rb So if two ore more applications are contributing to the schema_migrations table roll backs will not be possible if rails can't find the down() method (because it will not find a migration file contained in another app ie db/migrate/...)
I have a current situation that is exactly this and I have opted to have a master ActiveRecord app that manages migration and data conversions as our database evolves. Keep in mind that part of the deal is to keep models up to date as well. This has been time consuming so we are considering breaking apart the DB in to business domains and providing APIs (JSON) to query support data for another application. This way each application manages it domain and is responsible for exposing data via API.
regards.
If you connect both applications to the same database you should be able to run migrations on it but I strongly suggest you use another option since you will almost surely hit a wall at one time or another:
split the database in two if possible with each application responsible for its own database /migrations.
have one application considered the "master" database and use another database for the data specific to the second application but make it connects to both database (each application still only apply migrations to one database)
If you need to share data between multiple applications another option is to implement a REST service in one and use it on the other, you can have a look at the grape gem for a simple way of doing so.
Edit: I realize I forgot to speak about the activerecord migration, there is no longer any "version" of the schema, what activerecord does is that it read all your migration filename, extract their identifier (the starting part) and check if they have already been applied so in theory you can run migrations from two applications on the same database provided they don't interfere.
But if both migrations act on the same tables you will almost certainly run into big troubles at one point.
I disagree with Schmurfy, even if his presented options are valid, its a bit of an overkill to share data through REST (granted, its pretty easy to implement with ruby / rails).
If your case is simple you could just use one database from both apps, and since you use AR in both of them you have no problems with versioning, AR takes care of that.
Also i dont know what happens if you run db:migrate from both apps simultaniously if you use a inferior dbms like mysql which does not allow DDL in a transaction, certainly nothing good..
Also it would bother me to look which app needs what column and not have the migrations in one place. You could use a shared repository to manage the migrations from both apps.
Rails migrations store current database version in schema_migrations table in the database. So both of your apps will be able to check the current version.
The version numbers are timestamps, so there shouldn't be any problem with duplicate values, as it'll be almost impossible to generate two migrations at the exact same millisecond. So you should be fine here.
The only problem I see is that when you rollback a migration in one app, it'll set the db to the previous known version and I'm not sure if it will pick the previous one from the db (which could be from the other app), or the number from the previous migration file. You may want to test that scenario to make sure.
I decided to put all migrations in the Rails app because:
Since there is only one database
Rails manages migrations
This has worked well.
This simplifies the system because all migrations are stored in one place. And, the Sinatra app doesn't need to know about them anyway.
Can I use MongoDB and a PostgreSQL in one rails app? Specifically I will eventually want to use something like MongoHQ. So far I have failed to get this working in experimentation. And it concerns me that the MongoDB documentation specifically says I have to disable ActiveRecord. Any advice would be appreciated.
You don't need to disable ActiveRecord to use MongoDB. Check out Mongoid and just add the gem plus any models along side any of your existing ActiveRecord models. You should note that MongoHQ is just a hosting service for MongoDB and can be used alongside any Object Document Mapper (ODM).
For further details check http://mongoid.org/en/mongoid/docs/installation.html. Just skip the optional 'Getting Rid of Active Record' step.
On a recent client site I worked with a production system that merged MySQL and MongoDB data with a single Java app. To be honest, it was a nightmare. To join data between the two databases required complex Java data structures and lots of code, which is actually databases do best.
One use-case for a two database system is to have the pure transactional data in the SQL database, and the aggregate the data into MongoDB for reporting etc. In fact this had been the original plan at the client, but along the way the databases became interrelated for transactional data.
The system has become so difficult to maintain that is is planned to be scrapped and replaced with a MongoDB-only solution (using Meteor.js).
Postgres has excellent support for JSON documents via it's jsonb datatype, and it is fully supported under Rails 4.2, out of the box. I have also worked with this and I find it a breeze, and I would recommend this approach.
This allows an easy mix of SQL and NoSQL transactions, eg
select id, blast_results::json#>'{"BlastOutput2","report","results","search","hits"}'
from blast_caches
where id in
(select primer_left_blast_cache_id
from primer3_output_pairs where id in (185423,185422,185421,185420,185419) )
It doesn't offer the full MongoDB data manipulation features, but probably is enough for most needs.
Some useful links here:
http://nandovieira.com/using-postgresql-and-jsonb-with-ruby-on-rails
https://dockyard.com/blog/2014/05/27/avoid-rails-when-generating-json-responses-with-postgresql
There are also reports that it can outperform MongoDB on json:
http://www.slideshare.net/EnterpriseDB/the-nosql-way-in-postgres
Another option would be to move your Rails app entirely to MongoDB, and Rails has very good support for MongoDB.
I would not recommend running two databases, based on personal observations on how it can go bad.
I'm a PHP programmer for over a decade and making the move to RoR. Here is what I'm used to from the PHP world:
Create DB schema in a tool like MySQL WorkBench -- and make fields precisely the size I want without wasting space (e.g. varchar(15) if it's ip_address).
Write models using Datamapper and place those exact field lengths and specifications in there so my app doesn't try to put in any larger values.
In the RoR world from what I've seen over the past two days, this seems to be the flow suggested:
Add fields / schema using the command line which creates a migration script and apparently created large ass fields (e.g. "ip_address string" is probably making the field varchar(255) in the db when I run the migration).
Put in validations during model creation.
Am I missing something here? What's the process in the RoR world for enterprise level applications where you actually want to create a highly customized schema? Do I manually write out migration scripts?
The scaffolding is what you use to get started quickly. But before running the migration, you can edit it and add constraints and specific column lengths.
Validations specified in the model (in the ruby code) does not carry the same level of security as validations /constraints specified on the database. So you still need to define those on the database.
While it is possible to work with Rails without migrations, I would strongly advice against it. In some cases it cannot be avoided (when working with legacy databases for instance).
The biggest advantage of using the migrations is that your database schema, accross different platforms, can be held in sync through different stages. E.g. your development and your production database. When deploying your code, the migrations will take care that the database is migrated correctly.
You can edit the migration scripts before you run the migration in order to customize the fields.
Yes, if you need to tweak the defaults, you edit the migration scripts.
Also note that you don't need to use migrations, they're a "convenience" while iterating through DB development. There's nothing that says you must use them. The active record pattern doesn't rely on how the DB tables/fields/etc. are created or defined.
For example, migrations are useless when dealing with legacy DBs, but you can still write a Rails app around them.
I'm wondering if working with rails (3) is a good idea when a huge/ugly legacy database is already there (Oracle, SQLServer).
I only have experience with ActiveRecord, is there another ORM more suitable for that kind of job?
Cheers
ActiveRecord can still do the job - for example there are directives that can be applied within your model that make non-conventional table names, primary key names (multi-column PKs, if you have them, used to require some additional work, not sure how true that is in AR3).
For both Oracle and SQL Server you're going to need to get the relevant DB adapters; I don't think either is bundled with AR.
A lot of legacy DB Rails work only needs read-only access - if that's the case - and you can get access to do so - then you may find that defining views that are more "AR-friendly" and referencing those through your models may make life easier. If update is going to be necessary then either a useable primary key will be needed or you'll have to consider dropping down to building and executing custom SQL, something that's fully supported in AR for occasions when the abstractions can't cope.
I have a 'legacy' DB2 database that has many other applications and users. Trying to experiment with a rails app. Got everything working great with the ibm_db driver.
Problem is that I have some tables like schema1.products, schema1.sales and other tables like schema2.employees and schema2.payroll.
In the ibm_db adapter connection, I specify a schema, like schema1 or schema2, and I can work within that one schema, but I need to be able to easily (and transparently) reference both schemas basically interchangeably. I don't want to break the other apps, and the SQL I would normally write against DB2 doesn't have any of these restrictions (schemas can be mixed in SQL against DB2 without any trouble at all).
I would like to just specify table names as "schema1.products" for example and be done with it, but that doesn't seem to jive with the "rails way" of going about it.
Suggestions?
Schemas in DB2 are a very handy way to provide separate namespace to different applications. For example, you can separate all database objects for an application called "recruiting" from say application called "payroll". You can have objects (tables, views, procedures etc.) with the same name reside in multiple schemas and not colide with one another. Having your application set a schema is a handy way for it to say "hey, I am a payroll and I only want to work with my objects". So, what happens when you want to work with objects owned by another application? Well, in traditional procedural programming languages your application code would explicitly specify the schema when referencing an object in another schema or you would just do a SET CURRENT SCHEMA to switch to another schema. The problem with ORMs like ActiveRecord in Ruby on Rails is that you are not supposed to use SQL i.e. you don't have a good "Rails way" to specify schema when referencing an object. You can use find_by_sql and qualify your objects in the SQL statement but this is not what RoR people will consider to be good Rails.
You can fix things on the DB2 side. You can define a view per table in the "foreign" schema but you will have to take care to name the view so that it does not colide with what you already have in your primary schema. And, when you do that, you will undoubtedly create names that are not true Rails names.
Rails people are very proud of the "Rails way". It makes it very easy to create new applications. Rails is really awesome in this space. However, when it comes to integration with what is already out there Rails ... how do we say it ... sucks. I suggest you will have to accept to do things that are not the best examples of the Rails Way if you want to work with existing database structures.
How do i work with two different databases in rails with active records?