I am writing Symfony project (using symfony 1.4 ver. with Propel as its ORM) where some data is stored in MySQL database and some other data is stored on another server in PostgreSQL database.
To be more precise I want to store some models in MySQL database and other models in PostgreSQL database at the same time and do it seamlessly without explicit database switching (I mean Propel will use proper database connection and SQL dialect to retrieve/store data). Models from MySQL part will not have relations with PostgreSQL.
Is it possible? If yes I also would like to know how to setup development environment (I want to access different MySQL/PostgreSQL DBs in developement and production environments).
UPD: I've found question on SO reagrding this problem: Multiple databases support in Symfony But i have to check if it works with recent versions of Symfony.
i work with symfony every day and in fact you can have 2 databases in order to store unrelated parts of the model. You need to set up both connection in you database.yml (i'm unfamiliar with posgress so you will have to figure out how to set it up correclty):
mysql_connection:
class: sfPropelDatabase
param:
phptype: mysql
classname: MysqlPropelPDO
dsn: ''
username: user
password: pass
encoding: UTF-8
pooling: true
postgress_connection:
class: sfPropelDatabase
param:
phptype: postgres
classname: PostgresPropelPDO
dsn: ''
username: user
password: pass
encoding: UTF-8
pooling: true
Once you have done that, we should get started with the schema.yml file or files (as you will be using 2 databases i would suggest to have 2 files, one for the mysql and another for the postgres database):
mysql_schema.yml file:
//This is how you tell witch connection you are using for this tables
connection: mysql_connection
classes:
CLassName:
tableName: table_name
columns:
id:
name:
type: varchar(255)
required: true
[...]
postgres_schema.yml file:
connection: postgress_connection
classes:
ClassName:
tableName: table_name
columns:
id:
name:
type: varchar(255)
required: true
[...]
Once you have finished setting up your schema files, you should be good to go, create all classes and start to have fun. Hope this helps
I believe you can!
Google has quite a few results for this, try
http://snippets.symfony-project.org/snippet/194
Thats based on an older version of propel/symfony, but from a quick look, I believe it's still valid. Plus there are recent comments suggesting it works.
Related
I am using spring boot and trying to import spring session.
All I have done is add one single line to the application.yaml:
spring:
datasource:
password: xx
url: jdbc:postgresql://c:5432/dbname?schema=public
username: pg
session:
store-type: jdbc
Then I found that two tables spring_session and spring_session_attributes auto generated in my database. This is expected except one thing: these two tables are generated in a different table schema. However, the tables generated by jpa(hibernate) are being put in another schema.
I tired to dig into the source codes, however, I cannot find the codes which are calling the org/springframework/session/jdbc/xx-h2.sql to create the table.
What's the magic?
Getting this error:
PG::InsufficientPrivilege: ERROR: permission denied to set parameter
"client_min_messages" to "warning" : SET client_min_messages TO
'warning' when connecting with redshift database.
enter image description here
My database.yml file setting looks like this
development:
adapter: postgresql
encoding: utf8
host: nacfhrcluster123.ctvpledrvuobs5.us-east-1.redshift.amazonaws.com
port: 5439
username: nacfhr123
password: NACFDChr12345!
database: devnacfhrdc
pool: 5
schema_search_path: 'beta'
timeout: 5000
min_messages: warning
Thank you guys for your responses. I resolved the issue by adding a gem with the name 'activerecord5-redshift-adapter' in my gemfile. So you have to add below line in your gem file.
gem 'activerecord5-redshift-adapter'
And then run bundle install again.
It is important to realise that Redshift IS NOT Postgres. There are many differences.
One difference is that the available parameters are much different, the only parameters that can be set on Redshift are:
analyze_threshold_percent
datestyle
extra_float_digits
query_group
search_path
statement_timeout
wlm_query_slot_count
You will need to alter your rails connection so that is does not request this parameter to be set.
I guess that means removing min_messages: warning
I hope this will help. As far as I know there is no active records available for Redshift and it don't it make sense as well as Redshift is not a typical RDBMS.
We have similar scenario where we use ROR for front end for one of analytics and planing application.
Here is our app scenario:
Front-end : ROR
Database : mysql(OLTP)
Datawarehouse : Redshift(OLAP)
This is how data flows back and forth between ROR and Redshift.
Insert/updates from ROR to Redshift(happens very rarely).
For any operations from ROR that impacts something on data warehouse, ROR executes PSQL commands with specific query, not via active records.
SELECT from Redshift(happens frequently)
Similarly, for getting data from DW for various reason done via select query and redirection to file using PSQL command, then it import it into OLTP using myssqlimport.
Link for psql, mysqlimport tools that we use.
Comment if you have further specific followup questions.
I'm on the early stages on learning Ruby.I really don't have an idea on how to use an existing database filled with tables and data on ruby. Every guide, every article that I have or find on the internet is always creating a new one using the migration functions.
But which are the steps for using an existing database in SQL server on RoR?
You're in luck, friend. My first Rails project (7 years ago) was against a horribly set up SQL Server database.
Per the above, you need to set up your database.yml appropriately. But for an existing database, obviously it is unlikely that the table and column names conform to the Rails conventions. The good news is that you can override all of those defaults. Here is a non-exhaustive list of those directives:
In a model descended from AR::Base,
set_table_name 'actual_table_name'
set_primary_key 'actual_primary_key_name'
On the various association directives (has_one, has_many, belongs_to), there are :foreign_key keys that let you specify the name of the foreign keys.
Now, one of the things that MS SQL Server allows you to do which is TERRIBLE is that you can embed spaces into your column names. Fear not, you can still refer to these columns by name using write_attribute("badly named column") and read_attribute("badly named column"). You may also refer to them in various directives like so:
validates_length_of "Fax Number", :maximum => 17, :allow_nil => true
Finally, you may refer to the implied methods these devilishly named columns generate like so:
self.send('Fax Number=', new_fax_number)
Obviously, you can't refer to them as symbols, since spaces are disallowed in Ruby symbols.
Good luck, and next time I hope that you get to work with a real RDBMS, like Informix :).
First you have to setup your application to user sql server for databases connectivity.
you have to use gem for sql server in your gemfile and have to setup database.yml file accordingly.
In database.yml, in config folder put the name of same database In the Development part of this file.
development:
adapter:
database: db_name_dev
username:
password:
host: localhost
socket:
To use a existing server. In your database.yml you have to specify the host, port and the database name.
`database: <host>:<port>/<database_name>`
For eg
development:
adapter: mysql2
database: your.mysqlserver.com:1521/project_database
username: project_user
password: project_password
I'm refactoring some features of a legacy php application which uses multiple databases with the same structure, one per language. When an user login choose his language and then all the following connections are made with the db for that app with some code like this:
$db_name = 'db_appname_' . $_SESSION['language'];
mysql_connect(...);
mysql_select_db($db_name);
I'd like to refactor also the database, but currently it's not an option because other pieces of software should remain in production with the old structure while the new app is developed, and for some time after it's been developed.
I saw this question, but both the question and the suggested gems are pretty old and it seems that they are not working with Rails 3.
Which is the best method to achieve this behavior in my new rails 3 app? Is there any other choice that avoid me to alter the db structure and fits my needs?
Last detail: in the php app even the login information are kept in separate tables, i.e. every db has its own users table and when the user logs in it also passes a language param in the login form. I'd like to use devise for auth in the new app which likely won't work with this approach, so I'm thinking to duplicate (I know, this is not DRY) login information in a separate User model with a language attribute and a separate db shared among languages to use devise features with my app. Will this cause any issue?
EDIT:
For completeness, I ended with this yml configuration file
production: &production
adapter: mysql
host: localhost
username: user
password: secret
timeout: 5000
production_italian:
<<: *production
database: db_app_ita
production_english:
<<: *production
database: db_app_eng
and with this config in base model (actually not exactly this but this is for keeping things clear)
MyModel < AR::Base
establish_connection "production_#{session[:language]}"
...
end
use establish_connection in your models:
MyModel < AR::Base
establish_connection "db_appname_#{session[:language]}"
...
end
Use MultiConfig gem I created to make this easy.
You could specify the configs in separate file like database_italian.yml etc and then call:
ActiveRecord::Base.config_file = 'database_italian'
This way it will be much easier to maintain and cleaner looking. Just add more language db config files as you wish
Can this be done? In a single application, that manages many projects with SQLite.
What I want is to have a different database for each project my app is managing.. so multiple copies of an identically structured database, but with different data in them. I'll be choosing which copy to use base on params on the URI.
This is done for 1. security.. I'm a newbe in this kind of programming and I don't want it to happen that for some reason while working on a Project another one gets corrupted.. 2. easy backup and archive of old projects
Rails by default is not designed for a multi-database architecture and, in most cases, it doesn't make sense at all.
But yes, you can use different databases and connections.
Here's some references:
ActiveRecord: Connection to multiple databases in different models
Multiple Database Connections in Ruby on Rails
Magic Multi-Connections
If you are able to control and configure each Rails instance, and you can afford wasting resources because of them being on standby, save yourself some trouble and just change the database.yml to modify the database connection used on every instance. If you are concerned about performance this approach won't cut it.
For models bound to a single unique table on only one database you can call establish_connection inside the model:
establish_connection "database_name_#{RAILS_ENV}"
As described here: http://apidock.com/rails/ActiveRecord/Base/establish_connection/class
You will have some models using tables from one database and other different models using tables from other databases.
If you have identical tables, common on different databases, and shared by a single model, ActiveRecord won't help you. Back in 2009 I required this on a project I was working on, using Rails 2.3.8. I had a database for each customer, and I named the databases with their IDs. So I created a method to change the connection inside ApplicationController:
def change_database database_id = params[:company_id]
return if database_id.blank?
configuration = ActiveRecord::Base.connection.instance_eval { #config }.clone
configuration[:database] = "database_name_#{database_id}_#{RAILS_ENV}"
MultipleDatabaseModel.establish_connection configuration
end
And added that method as a before_filter to all controllers:
before_filter :change_database
So for each action of each controller, when params[:company_id] is defined and set, it will change the database to the correct one.
To handle migrations I extended ActiveRecord::Migration, with a method that looks for all the customers and iterates a block with each ID:
class ActiveRecord::Migration
def self.using_databases *args
configuration = ActiveRecord::Base.connection.instance_eval { #config }
former_database = configuration[:database]
companies = args.blank? ? Company.all : Company.find(args)
companies.each do |company|
configuration[:database] = "database_name_#{company[:id]}_#{RAILS_ENV}"
ActiveRecord::Base.establish_connection configuration
yield self
end
configuration[:database] = former_database
ActiveRecord::Base.establish_connection configuration
end
end
Note that by doing this, it would be impossible for you to make queries within the same action from two different databases. You can call change_database again but it will get nasty when you try using methods that execute queries, from the objects no longer linked to the correct database. Also, it is obvious you won't be able to join tables that belong to different databases.
To handle this properly, ActiveRecord should be considerably extended. There should be a plugin by now to help you with this issue. A quick research gave me this one:
DB-Charmer: http://kovyrin.github.com/db-charmer/
I'm willing to try it. Let me know what works for you.
I got past this by adding this to the top of my models using the other database
class Customer < ActiveRecord::Base
ENV["RAILS_ENV"] == "development" ? host = 'devhost' : host = 'prodhost'
self.establish_connection(
:adapter => "mysql",
:host => "localhost",
:username => "myuser",
:password => "mypass",
:database => "somedatabase"
)
You should also check out this project called DB Charmer:
http://kovyrin.net/2009/11/03/db-charmer-activerecord-connection-magic-plugin/
DbCharmer is a simple yet powerful plugin for ActiveRecord that does a few things:
Allows you to easily manage AR models’ connections (switch_connection_to method)
Allows you to switch AR models’ default connections to a separate servers/databases
Allows you to easily choose where your query should go (on_* methods family)
Allows you to automatically send read queries to your slaves while masters would handle all the updates.
Adds multiple databases migrations to ActiveRecord
It's worth noting, in all these solutions you need to remember to close custom database connections. You will run out of connections and see weird request timeout issues otherwise.
An easy solution is to clear_active_connections! in an after_filter in your controller.
after_filter :close_custom_db_connection
def close_custom_db_connection
MyModelWithACustomDBConnection.clear_active_connections!
end
in your config/database.yml do something like this
default: &default
adapter: postgresql
encoding: unicode
pool: 5
development:
<<: *default
database: mysite_development
test:
<<: *default
database: mysite_test
production:
<<: *default
host: 10.0.1.55
database: mysite_production
username: postgres_user
password: <%= ENV['DATABASE_PASSWORD'] %>
db2_development:
<<: *default
database: db2_development
db2_test:
<<: *default
database: db2_test
db2_production:
<<: *default
host: 10.0.1.55
database: db2_production
username: postgres_user
password: <%= ENV['DATABASE_PASSWORD'] %>
then in your models you can reference db2 with
class Customers < ActiveRecord::Base
establish_connection "db2_#{Rails.env}".to_sym
end
What you've described in the question is multitenancy (identically structured databases with different data in each). The Apartment gem is great for this.
For the general question of multiple databases in Rails: ActiveRecord supports multiple databases, but Rails doesn’t provide a way to manage them. I recently created the Multiverse gem to address this.
The best solution I have found so far is this:
There are 3 database architectures that we can approach.
Single Database for Single Tenant
Separate Schema for Each Tenant
Shared Schema for Tenants
Note: they have certain pros and cons depends on your use case.
I got this from this Blog! Stands very helpful for me.
You can use the gem Apartment for rails
Video reference you may follow at Gorails for apartment
As of Rails 6, multiple databases are supported: https://guides.rubyonrails.org/active_record_multiple_databases.html#generators-and-migrations
Sorry for the late and obvious answer, but figured it's viable since it's supported now.