Query Remote Database from Rails 3.2 Application - ruby-on-rails

I need to query a 3rd party database which is entirely separate from the Rails 3.2 application I'm building (belongs to a different application which my company uses internally).
Ultimately, I'll be setting up a cron to load new rows from the "other" database which my Rails application will be processing.
I have the access to otherdb set up, and I'm wondering where to go from hereā€”do I create a new entry in config/database.yml? If so, how do I then specify when a query is to be directed to otherdb, instead of my default Rails development or production db?

There's a few ways to implement this requirement, the easiest of which would be to use config/database.yml and custom namespaced model(s).
Setting up something similar to the below, using a suffix of Rails.env to follow the naming convention will provide the functionality you outlined.
First, create new entries for the external database each of your existing environments. It'll help you to be able to test the functionality.
# database.yml
development:
# add configuration as required
otherdb_development:
# add configuration as required
Second, add a model for each of the specific tables you need to access in the otherdb database. I'd recommend adding a namespace directory for these models (otherdb in the example below) to avoid confusion and potential clobbering:
# /app/models/otherdb
class Otherdb::Foo < ActiveRecord::Base
establish_connection "otherdb_#{Rails.env}"
set_table_name "foo" # customize this if the table name will be different from the classname and is required
end
You can then use (as an example) methods on Otherdb::Foo and use the resulting data as required.

I had the same problem yesterday. Since you are using Rails 3.2, all your models that connect to the external database will have to be subclasses of a single, abstract class that establishes the connection. In earlier versions of Rails, #Sasha's answer would have worked. But in 3.2, that answer will lead you to getting various confusing database errors. (What errors you get depends on what DB you use.)
In Rails 3.2, this is the only way I have found that will work:
Make a common base class for all models that need to talk to a
non-default database.
Tell ActiveRecord that this base class is abstract by calling self.abstract_class = true.
Call establish_connection in the base class.
Here is an example with students and courses from an external table:
# database.yml:
development:
# default configuration goes here
other_development:
# external db configuration goes here
class OtherTable < ActiveRecord::Base
self.abstract_class = true
establish_connection "other_#{Rails.env}"
end
class Student < OtherTable
end
class Course < OtherTable
end
If you'd like more detail, see the blog post I wrote titled Establishing a Connection to a Non-Default Database in Rails 3.2.

Related

How to generate a model, whose table in stored in a different database?

I work with an app, which uses three different databases. I want to create a model and store its table in one particular database, which is not the default appname_development database. How do I tell the rails generator where to create the DB table for this model?
In short: you can't. You can't tell the model generator which database to use to create the table (out of the box).
You'd need to change the generated migration yourself to target the database you want using establish_connection. Something like:
conn = ActiveRecord::Base.establish_connection(:"which_db_#{Rails.env}")
conn.create_table :my_table do |t|
Instead of the usual call to create_table that gets generated for you.
You'll also need to add the establish_connection call to any models.
However, you could provide your own templates for the the generation of the your migration and model though in order to add those lines for any generated model. There's more details in the generators guide but you should be able to provide versions of the files to generate a model and a migration. You'd then have to react to a variable passed in on the rails generate to specify which db to use. Might be a bit fiddly but probably worth it if you're going to get generating a lot of models in different places.
You can use the establish_connection method.
class SomeClassInDb1 < ActiveRecord::Base
establish_connection "db_1_#{Rails.env}"
end
class SomeClassInDb2 < ActiveRecord::Base
establish_connection "db_2_#{Rails.env}"
end
and in your config/database.yml
development:
# your normal db settings
db_1_development:
# config for db 1 in development env
db_2_development:
# config for db 2 in development env
db_1_test:
# config for db 1 in test env
db_2_test:
# config for db 2 in test env
db_1_production:
# config for db 1 in production env
db_2_production:
# config for db 2 in production env

Reverse Engineering (Generating) Tables or Database Schema from Models and Views in Ruby on Rails

Update: The Question is Still Open, any reviews, comments are always welcome
I am having an existing rails project in which some important files and directories has been missed.
project rails version (2.3.8) i found it in environment.rb
currently what i am having is
app
controllers (already fully coded)
helpers (already fully coded)
models (already fully coded)
reports (already fully coded)
views (already fully coded)
config ---> default configurations (already fully coded)
lib ---> contains nothing
public --> contains images and scripts (already fully coded)
script ---> contains server,runner,plugin,dbconsole....
app directory fully contains working state of codes, app/model contains more than 100 .rb files , so i assume it will be more than 100 tables
the mainly missing things are db directory, .gem file, rake file, doc, test, vendor, database,schema.rb and migrations
Note:
i don't have the table schema and database for that project
i am in Need to generate tables or complete database from models and views and
i am looking for reverse engineering kind of stuff for generating db schema from models or views
I am newbie to rails and i am from java background , in java by using hibernate there is an pojo(model in rails) to database option available, i am looking for similar kind of stuffs for rails , and my main aim to run that project , so guys please help me.
To recreate the database schema, it will take quite a bit of time.
You can get a lot of information about the database in the app/models, app/controllers app/views directory.
You should know that ActiveRecord does not require you to explicitly list all the attributes of a model. This has important implications - you can only infer what attributes you still have to add to the database, based on whether an attribute is referred to! This means doing this will be a bit of an ART. And there are no CLEAR steps to complete this work. But below are some rules which you can use to HELP you.
This is a BIG project, below are guidelines, rules and tips to help you. But be aware that this could take a long time, and be frustrating at times to get this done.
What Tables you need:
Each table will normally have a matching ActiveRecord::Base model. So in the app/models directory, check each file, and if the class inherits from ActiveRecord::Base, it is an extra table.
The table name is by default a pluralized snake case version of the name of the class.
class UserGroup < ActiveRecord::Base # for this class
the name of the table is user_groups. Notice it is plural, and instead of camel case, it is lowercase, with underscores to separate the words.
All these tables will have an "id" integer column. By default, the tables also have a "created_at", and "updated_at" column of type datetime.
Associations and foreign keys:
You can infer what foreign keys exist by the associations in the Models. All associations are explicitly listed, so this is not too hard.
For example:
class UserGroup < ActiveRecord::Base # for this class
belongs_to :category
This means that the user_groups table has a column named "category_id", which is a foreign key for the categories table.
This means that the Category model likely has an inverse relationship (but no extra column):
class Category < ActiveRecord::Base
has_many :user_groups
The main other association is the has_many_and_belongs_to association. Eg.
class A < ActiveRecord::Base
has_and_belongs_to_many :bs
end
class B < ActiveRecord::Base
has_and_belongs_to_many :as
end
This means that there is a join table to add called "as_bs" (as and bs are sorted alphabetically), with the foreign keys "a_id" and "b_id".
All foreign keys are integers.
Attributes
Ok, so that's the table associations. Now for the normal attributes...
You should check the app/views/user_groups/ or other similar app/views directories.
Inside you will find the view templates. You should look at the _form.html.erb templates (assuming it is .erb templates, otherwise it could be .haml etc templates).
The _form.html.erb template, if it exists, will normally have many of the attributes listed as form fields.
In the form_for block, check if it says something like f.text_field :name, it means there is an attribute/(column in the table) called "name". You can infer what type the column should be by what type of field it is. Eg. in this case, it is a string, so maybe a VARCHAR(255) is appropriate (referred to as string in Rails).
You might also need to infer what type is appropriate based on the name of the attribute (eg. if it mentions something like :time, then it is probably either of type Time or DateTime).
This may give you all the other attributes in the table. But in some cases, you might miss the attributes. If you find a reference to other attributes in the controller, eg. app/controllers/user_groups_controller.rb, then you should add that as a column in your table. You can leave this until the end when you test it though, because when you test it, if an attribute is missing, then it will throw a NoMethodError for the object of the relevant model. Eg. if it says that #user_group variable, of class UserGroup, is missing a method named title, then it probably is missing a column named "title" of type string.
Recreate your migration/database
Ok, so now you know what the database tables and column names and types should be.
You should generate/recreate a migration for your database.
To do this, just use the command rails generate migration RecreateTables.
Then you should find a file in db/migrate/???_recreate_tables.rb.
Inside, start writing ruby code to create your tables. Reference for this can be found at http://guides.rubyonrails.org/migrations.html.
But essentially, you will have something like:
class RecreateTables < ActiveRecord::Migration
def up
create_table :user_groups do |t|
t.string :name # adds a string (VARCHAR) column called "name"
t.text :description # adds a textarea type column called "description
t.timestamps # adds both "created_at" and "updated_at" columns for you
end
end
def down
drop_table :products # this is the reverse commands to undo stuff in "up"
end
end
To recreate your Gemfile:
Start by adding a default Gemfile. This can be done by using rails new testapplication somewhere to create an empty rails application. Then copy the Gemfile to your actual application. It will get you started by including rails and other common gems.
It is VERY hard to work out exactly what gems are needed. The best you can do is try adding them one by one as you look through the code.
Again, here, MethodNotFound errors are your FRIEND. When you test the application, based on the gems you have added, it might detect some missing methods which might be supplied by gems. Some missing methods on models might indicate missing gems (or they might indicate missing fields/columns in the database). However, missing methods on Controller or ActiveRelation classes are VERY likely because of missing gems.
You will have to look through the code and try to infer what gems to add.
If it uses methods like can, can?, and has a file app/models/ability.rb, then you need gem 'cancan'. If it calls devise in a model, it needs gem 'devise'. Many common gems can be seen at http://ruby-toolbox.com.
After adding gems to your Gemfile, you should run bundle on your command line to install the new gems before testing again. When you test it again, you should restart your test server. Rerun bundle exec rails server to start a local test server on localhost:3000 or something like that.
You can simply copy the Rakefile from rails new testapp, and it will probably include everything you need.
Missing Tests
The missing test/ directory is not relevant to your actual application. It is not required to run the application. However, it does hold automatic scripts to test your application. You will have to re-write new tests if you want to automatically test your application. However for the purpose of getting your application back up, you can ignore it for now.
Missing vendor directory
Some extra code is not installed as a gem, but as a plugin. Anything installed as a plugin is lost if you don't have the vendor directory. As with gems, the best you can do is try to infer what might be missing, and re-download the missing plugin, either re-installing the plugin, or using a gem replacement.
Additional tips:
Try reading some of the comments which might name some of the gems used.
If a method or set of methods are missing, that you think are not database fields/columns, it might be due to a missing gem. The best thing to do is to search google for those method names. Eg. if it is missing "paginate", you can search "rails paginate gem", and see what likely gems you might need. This example will probably come up with "will_paginate", and "kaminari". Then you have to try and infer which of the gems are required. Maybe do a grep will_paginate app -r on the command line to see if it is using will paginate. The grep command searches for the string "will_paginate", in the directory called "app", -r makes it do this recursively for all files
Even though rails is a full stack web framework it would work with out some parts as well, if you wish to,
Ex: in your case
db - directory is there for keep the migrations to create you DB/tables, but if you are using a legacy DB or the database stuff is handled by DB administrators, you might not want it. (you can simply connect to the DB via database.yml file)
Gem file is helping you to keep all the gems (libraries) in one place as you do with Maven (in java)
test, again if you done write test cases (which is absolutely a bad idea), you done need this
vendor, is for 3rd party plugins and doc is for documentation, so same rule applies, if you done need them you can skip them
Hibernate in rails called "Activerecord", same concept, a model is bind with a database table (technically model represents a raw in the table)
So if you really want them add them but if not just leave them
BUT, I think having a proper Gem file and test cases is a must
welcome come to Rails
HTH
In the following, I assume you already know how to:
dump your database schema into an SQL file
start a Rails console (rails c)
generate a Rails migration
Here's what I think you should do.
Identify which of your classes correspond to physical tables (you mention some views in your question, which leads me to believe a subset of your models are bound to database views instead of actual tables). To do this you need to match the definitions of your models (classes which extend ActiveRecord::Base) to CREATE TABLE statements in your schema dump. For instance, class Person in your Ruby code matches to CREATE TABLE people in your DB schema dump.
Once you identified those models (class names), you start up a Rails console and you type those model names, one at a time, and press Enter. The console output for a model called Person would presumably look like this:
>> Person
=> Person(id: integer, first_name: string, last_name: string)
You then take what's inside the parentheses, strip the leading id: integer,, get rid of commas, get rid of those blanks after the colons, thus obtaining something like this:
first_name:string last_name:string
Having done this, the command to generate the migration would look like this:
rails g migration Person first_name:string last_name:string
You then start a new Rails project somewhere else, perform all of these migrations and inspect the contents of db/migrate. Your migrations are most likely 90% done, what you still need to do is replace some instances of t.integer with t.references, and other minor stuff that's completely domain-specific and impossible to capture in a generic answer.
HTH.

Ruby on rails with multiple production settings

How can we run a ruby on rails application with different database configuration?
in detail: I want to run multiple instance of a rails app with different database config for each on production. How is it possible?
I think you can duplicate the config in database.yml into different environments, like prod1, prod2 ... and then set RAILS_ENV environment variable to match before starting up each respective server...
You can duplicate your database.yml file as DGM mentioned. However, the right way to do this would be to use a configuration management solution like Chef.
If you look at the guide for setting up Rails stack, it includes a 2 front-end Web server + 1 back-end DB server. Which would include your case of duplicating database.yml file.
If you are able to control and configure each Rails instance, and you can afford wasting resources because of them being on standby, save yourself some trouble and just change the database.yml to modify the database connection used on every instance. If you are concerned about performance this approach won't cut it.
For models bound to a single unique table on only one database you can call establish_connection inside the model:
establish_connection "database_name_#{RAILS_ENV}"
As described here: http://apidock.com/rails/ActiveRecord/Base/establish_connection/class
You will have some models using tables from one database and other different models using tables from other databases.
If you have identical tables, common on different databases, and shared by a single model, ActiveRecord won't help you. Back in 2009 I required this on a project I was working on, using Rails 2.3.8. I had a database for each customer, and I named the databases with their IDs. So I created a method to change the connection inside ApplicationController:
def change_database database_id = params[:company_id]
return if database_id.blank?
configuration = ActiveRecord::Base.connection.instance_eval { #config }.clone
configuration[:database] = "database_name_#{database_id}_#{RAILS_ENV}"
MultipleDatabaseModel.establish_connection configuration
end
And added that method as a *before_filter* to all controllers:
before_filter :change_database
So for each action of each controller, when params[:company_id] is defined and set, it will change the database to the correct one.
To handle migrations I extended ActiveRecord::Migration, with a method that looks for all the customers and iterates a block with each ID:
class ActiveRecord::Migration
def self.using_databases *args
configuration = ActiveRecord::Base.connection.instance_eval { #config }
former_database = configuration[:database]
companies = args.blank? ? Company.all : Company.find(args)
companies.each do |company|
configuration[:database] = "database_name_#{company[:id]}_#{RAILS_ENV}"
ActiveRecord::Base.establish_connection configuration
yield self
end
configuration[:database] = former_database
ActiveRecord::Base.establish_connection configuration
end
end
Note that by doing this, it would be impossible for you to make queries within the same action from two different databases. You can call *change_database* again but it will get nasty when you try using methods that execute queries, from the objects no longer linked to the correct database. Also, it is obvious you won't be able to join tables that belong to different databases.
To handle this properly, ActiveRecord should be considerably extended. There should be a plugin by now to help you with this issue. A quick research gave me this one:
DB-Charmer: http://kovyrin.github.com/db-charmer/
I'm willing to try it. Let me know what works for you.
Well. We have to create multiple environments in you application
create config/environmenmts/production1.rb which will be same as of config/environmenmts/production.rb
then edit database.yml for production1 settings and you are done.
start server using rails s -e production1

Rails RSpec with Multiple Databases

I run a Rails app, and we're in the process of splitting out our signup process to a separate app. The signup app has its own separate database (for CMS and collecting prospects), but it also needs to have access to the main database. This works really well using ActiveRecord::Base.establish_connection.
However, I'd like to be able to write some specs. The trouble is, how can I write specs/tests without clearing out my development database every time my tests run? If I go into the console in test mode, it's obvious the the test mode is hooked into the development database from my main app.
Here's what my database.yml file looks like:
development:
database: signup_dev
test:
database: signup_test
main_app_dev:
database: main_app_dev
main_app_test:
database: main_app_test
Based on this file, I'd like establish_connection to connect to connect to the database my_app_dev in development mode, and my_app_test in test mode. Any ideas?
We have a gem that is basically a collection of ActiveRecord models that connect to our legacy system. In our case we have all those models contained in a module from which all models related to the legacy database connects.
module Legacy
class Base < ActiveRecord::Base
establish_connection :legacy
end
class User < Base
end
end
With this setup it makes it really easy to switch out the database connection. If you really go for that automated detection you can put logic in your base class to determine which database to use:
module Legacy
class Base < ActiveRecord::Base
if Rails.env == 'test'
establish_connection :legacy_test
else
establish_connection :legacy
end
end
Or simply tell your module which connection to use in your spec helper:
# spec/spec_helper.rb
Legacy::Base.establish_connection(ActiveRecord::Base.configurations['legacy_test'])
Personally I would recommend the second option. Of course both solutions depend on namespaced models.
Peer
Ryan, we were also in the process of migrating from one datastore to another. We needed to develop against two databases and maintain separate migrations and fixtures for each.
I created a gem called Secondbase to help with this. Essentially, it allows you to manage two databases seamlessly in a single Rails app. Perhaps it will solve your issue as well: https://github.com/karledurante/secondbase
Here's what I came up with as a mixin:
# lib/establish_connection_to_master_database.rb
module EstablishConnectionToMasterDatabase
def establish_connection_to_master_database
case RAILS_ENV
when "development"
establish_connection :master_dev
when "test"
establish_connection :master_test
when "production"
establish_connection :master
end
end
end
ActiveRecord::Base.send(:extend, EstablishConnectionToMasterDatabase)
# models/subscription.rb
class Subscription < ActiveRecord::Base
establish_connection_to_master_database
end
# config/initializers/config.rb
require 'establish_connection_to_master_database'
In order for this to work with RSpec, this needs to be loaded in an initializer - apparently loading it in the environment file causes it to be loaded it too late, and it won't work.
We just used interpolation for this:
class ServiceModel < ActiveRecord::Base
establish_connection :"main_app_#{Rails.env}"
end
The funny :"main_app_" syntax makes a symbol from a string. This could be also written "main_app_#{Rails.env}".to_sym. In either case with Rails 4.1 this must be a symbol (under 3.2 we had just used a string).

Rails Console Doesn't Automatically Load Models For 2nd DB

I have a Rails project which has a Postgres database for the actual application but which needs to pull a heck of a lot of data out of an Oracle database.
database.yml looks like
development:
adapter: postgresql
database: blah blah
...
oracle_db:
adapter: oracle
database: blah blah
My models which descend from data on the Oracle DB look something like
class LegacyDataClass < ActiveRecord::Base
establish_connection "oracle_db"
set_primary_key :legacy_data_class_id
has_one :other_legacy_class, :foreign key => :other_legacy_class_id_with_funny_column_name
...
end
Now, by habit I often do a lot of my early development (and this is early development) by coding for a bit and then playing in the Rails console. For example, after defining all the associations for LegacyDataClass I'll start trying things like a = LegacyDataClass.find(:first); puts a.some_association.name. Unexpectedly, this dies with LegacyDataClass not being already loaded.
I can then require 'LegacyDataClass' which fixes the problem until I either need to reload!, which won't actually reload it, or until I open a new instance of the console.
Thus the questions:
Why does this happen? Clearly there is some Rails magic I am not understanding.
What is the convenient Rails workaround?
I believe this might have to do with your model name, rather than your connection. The Rails convention is that model class names are CamelCase, while the files they reside in are lowercase+underscore.
The "LegacyModel" class should therefore be in models/legacy_model.rb. Your statement about "require 'LegacyDataClass'" indicates that this is not the case, and therefore Rails doesn't know how to automagically load that model.
I wrote something for an app at work that handles connections to other databases' at runtime, it might be able to help.
http://github.com/cherring/connection_ninja

Resources