Rails 3 - rake tasks to work with multiple database connections - ruby-on-rails

Let's say I have s different database connections...
Right now for the purposes of this example let's only be concerned with development. When I run 'rake db:setup' it only creates development because from the rake task perspective the other connection is another environment. I could pass in the other connection as the RAILS_ENV to create the database. However, the problem comes in with how we've defined our database connections. I'd rather not undo this as it makes management and deployment much easier. What we've done is this...
In our database.yml we've added the following code.
databases_file = File.join(Rails.root.to_s, "config", "databases", "#{Rails.env.to_s}.yml")
IO.read(databases_file) if File.exist?(databases_file)
Then in a databases sub-folder under config we've created the different environment yml files.
So in development.yml we have...
development:
... details ...
logging:
... details ...
Now if I set RAILS_ENV to 'logging' it will never load the file since the file is called development.yml. Thus it will never find the logging connection.
If I consolidate all these settings back into the database.yml file then we're back to not checking in the database.yml into the git repo and having to manually create it on the development machines for new hires and such. We'd also have to change our deploy process to start putting the file in place on the lower environments. (Our production process already supports this)

You could use environment variables to manage these. You could have a single database.yml with things defined such as:
development:
database: ENV['DEVELOPMENT_DATABASE']
And then set the environment variables in your .bashrc/.zshrc.

We have multiple databases and we use branches heavily... so I came up with this solution to create multiple databases and tie their names to the branch being worked on...
# config/database.yml
<%=
databases_file = File.join(Rails.root.to_s, "config", "databases", "#{Rails.env.to_s}.yml")
if Rails.env.development? || Rails.env.test?
branch = ENV['BRANCH'] || `git branch --no-color 2> /dev/null`.chomp.split("\n").grep(/^[*]/).first[/(\S+)$/,1].sub("-", "_")
puts "Using databases for #{branch}"
IO.read(databases_file).gsub!("<branch>", branch) if File.exist?(databases_file)
else
IO.read(databases_file) if File.exist?(databases_file)
end
%>
And then you create a database file for each environment. In the development.yml and test.yml you use the token to let this script look for what it needs to in order to do replacement. So an example file would look like this.
# config/databases/development.yml
development:
adapter: mysql2
host: 127.0.0.1
database: <branch>_dev
username: user
password: pass
versions:
adapter: mysql2
host: 127.0.0.1
database: <branch>_versions
username: user
password: pass
reporting:
adapter: mysql2
host: 127.0.0.1
database: <branch>_reporting
username: user
password: pass
ods:
adapter: mysql2
host: 127.0.0.1
database: <branch>_ods
username: user
password: pass
I then extended the rake tasks to handle multiple databases
This is where most of the work went in. Therefore, I'm going to leave it out of the answer and make you work for it! No actually, it's just a big large mess that I'm not proud of and would love time to fix it, but haven't found any. I don't want to lead anyone down the wrong path but if you have questions just get me a message and I'll be happy to help.

Related

How to configure database.yml for production

I would like to deploy my application via Dokku to a VPS.
Putting together the dokku-postgres documentation and the relative, scarce internet documentation on the matter (one at GitHub), it seems necessary to configure database.yml to use the url environment variable url: <%= ENV['DATABASE_URL'] %>
Since I could not find any other sources of information, I wonder how database.yml should be configured, and how Rails will connect to the postgres service created with Dokku.
For instance, taken for granted that linking url to the DATABASE_URL variable is necessary, will this be enough to establish a connection between my Rails application and the postgres service or would it be still necessary to use a username and a password? In the latter case, what username and password am I expected to use?
Below is how at present my database.yml looks like.
default: &default
adapter: postgresql
encoding: unicode
pool: 5
username: asarluhi
development:
<<: *default
database: fireworks_app_development
test:
<<: *default
database: fireworks_app_test
production:
<<: *default
database: fireworks_app_production
pool: 25
username: fireworks_app
password: <%= ENV['FIREWORKS_APP_DATABASE_PASSWORD'] %>
This file was created as it is (apart from a higher pool size for production) when I created the application. How would you suggest to edit the production section?
The dokku-postgres documentation states that the following (and nothing else) will be set on the linked application by default:
DATABASE_URL=postgres://postgres:SOME_PASSWORD#dokku-postgres-lolipop:5432/lolipop
In place of the lollipop postgres service example, I would use fireworks_app_production to match the name of the database in database.yml
Are username and password still necessary after pointing url to the DATABASE_URL variable? Am I expected to add or remove anything else?
You don't need to worry about the database.yml with dokku, just upload your app to the server, let's use "fireworks" as the name for example on this.
when you upload the first time the app, this is created automatically so you don't need to create it.
then you install the postgres plugin and run
# that will create the container for the database
$ dokku postgres:create fireworks
# and then you link both, the app with the database
$ dokku postgres:link fireworks fireworks
you don't have to worry about anything else, with that dokku will connect this
then you just have to run db:migrate and everything is ready to work!

How can I keep AWS-SDK gem from trying to run .load_yaml_config?

I am working on an old-ish codebase that supports connections to many different databases. I am trying to create a new environment so I can open a rails console with a connection to the particular database that I need to use for the part of the application I'm working on. The name of the environment is nhl_development.
So far, I've created a new config file and added relevant database config:
# database.yml
generic: &generic
adapter: postgresql
encoding: utf8
pool: 30
dev_default: &dev_default
<<: *generic
host: <%= ENV['FOO']%>
username: <%= ENV['BAR']%>
password: <%= ENV['BAZ']%>
pool: 30
nhl_development:
<<: *dev_default
database: nhl
# config/environments/nhl_development.rb
Stats::Application.configure do
# copied entirely from config/environments/development.rb
end
When I run rails c nhl_development from my command line, I get the following error from the AWS-SDK gem:
/usr/local/rvm/gems/ruby-2.2.1/gems/aws-sdk-v1-1.60.2/lib/aws/rails.rb:110:in `load_yaml_config': config/aws.yml is missing a section for `nhl_development` (RuntimeError)
...
...
from /home/me/app/config/environment.rb:5:in `<top (required)>'
I understand the basic point of this error message. When I open config/aws.yml there is no set of config for nhl_development. However, what I don't understand is:
Where is AWS::Rails.load_yaml_config being called? I don't think I even want/need AWS to load configuration in this environment. The stack trace leads back to config/environment.rb:5 but that's just the line that initializes the Rails app. There is no initializer for AWS in config/initializers and I've done a find-in-project for .load_yaml_config with no results. I can get everything working by adding a section of config for this environment in config/aws.yml but I don't to load any AWS config at this time. How can I keep AWS::Rails.load_yaml_config from being called in this environment so that I don't have to mess with config/aws.yml right now??
It's the gem itself that does this. When it is required (which bundles will do if it is in your Gemfile) it loads this file which registers an initializer that (among other things) calls load_yaml_config.
You could conceivably solve this by tuning off autorequire for that one gem (require: false) in your Gemfile and ensuring that the gem isn't loaded by any other means until after initialisers have run but that sounds a bit hacky. I think the easiest way is just to add an entry to that yaml file

Rake aborted during postgresql migration

Rails/PostgreSQL newbie here, having real problems dealing with creating PostgreSQL projects in Rails.
Briefly, I am following Ryans excellent Railscast episodes, in particular the episode on deployment. I create a new project like this:
rails new blog -d postgresql
This generates a database.yml file similar to the following (comments extracted):
development:
adapter: postgresql
encoding: unicode
database: blog_development
pool: 5
username: blog
password:
test:
adapter: postgresql
encoding: unicode
database: blog_test
pool: 5
username: blog
password:
production:
adapter: postgresql
encoding: unicode
database: blog_production
pool: 5
username: blog
password:
Looks good so far.
However, on my development machine, whenever I try to run rake db:create:all, I get a message similar to the following:
rake aborted!
FATAL: role "blog" does not exist
My understanding is that this is because I haven't (or, rather, Rails hasn't) created a user called "blog" when the application was created. Therefore, I need to either:
Change the username for all environments to my system's "super
user" I chose when I installed Homebrew (which works); OR
Create a new super user for each individual project I set up (e.g. a
super user "blog" in this instance).
Now, the problem is this - should I be creating a super user for each individual project I create? If so, how? The reason I ask is that Ryan's Railscasts episodes never actually mention this, so I'm not sure if this is the correct approach to be taking (it seems a bit long-winded), or if there is a problem with my PostgreSQL database installation (as I would have thought that the super user would be automatically set up along with the database when the application is created).
Additionally, so as to make deployment easier, I'd like to keep the overall database.yml file the same in both the development and production environments (although I admit to knowing even less about deployment, so maybe this isn't ideal).
Thanks!
Creating database users is up to you. Rails does not create database users.
Depending on your platform, you can use some kind of database management GUI to make it easier to create users and manage their rights. I use PGAdmin, which came with the Mac OS X executable for PostgreSQL. It is available for other platforms as well.
In development, many people use the same "superuser" for all projects. In production, I would advice you to have application-specific users with minimal privileges.
I don't exactly understand the last paragraph about the database.yml being the same for dev and prod envs, because that is precisely the point - one file for all environments. You should probably not use the same credentials for development and production.

Deploy database.yml file with capistrano with multiple database definitions

I have a rails application which needs to connect to two different databases. These database definitions are both in the database.yml file in my repository, with the format:
production:
adapter: ...
database: ...
username: ...
password: ...
secondary-production:
adapter: ...
database: ...
username: ...
password: ...
I have that format for production, staging, and development databases. When running cap deploy or cap production deploy, I need to make both database definitions come through.
I wasn't the person who set up capistrano on this app, and I've never used it before, so I'm finding it hard to work out how to do this. Could anyone point me to a relevant piece of documentation or let me know what file and syntax I need to use for it?
I found the answer.
There was a database.yml.erb file in the /config/recipes/templates folder that I had to update.

Database not being selected in rails project when attempting to rake db:migrate

Working with a rails app, having some manner of weird database / rake issues.
When I execute:
rake db:migrate
I am getting the following error:
Mysql2::Error: No database selected: SHOW TABLES
(See full trace by running task with --trace)
The trace isn't revealing much useful information. Can be seen here: http://pastebin.com/WdsguudC
The config file looks right, and the user is getting logged in, or I would have gotten some kind of access error. The database exists, the user has correct permission, and I can access and manipulate it manually. I have done a bunch of googling, and haven't found anything helpful. Not sure if there is any other code that needs provided, because this seems like fairly low level problem.
after all that it was a spacing issue in the yaml.
Note that ruby has exchanged its YAML parser in a recent 1.9.2 version.
This might also cause this problem.
In order to switch back to the old YAML parser syck, use this in boot.rb:
require 'yaml'
YAML::ENGINE.yamler= 'syck'
Well, it is a common issue for us beginners. This issue comes from the moment when you create your new project in rails. Let’s say to have an example
$ rails new toy –d mysql
After you do the bundle and start your server, most likely you will have an error. To correct it you need to go to your database.yml and modify the following:
Add a password in the password field as shown below, this is the password you use to secure mysql.
default: &default
adapter: mysql2
encoding: utf8
pool: 5
username: root
password: mypassword
socket: /tmp/mysql.sock
Also, comment out the database adding a hash tag (#)before the name as shown below
development:
: *default
database: #toy_development
Then restart your command line and go to the root of your application and type:
$ rails s
You have to see the Ruby on Rails welcome page..
After, you need to create a database.
Create a DATABASE.
The issue message is saying that not DATABASE is selected. It is because I didn’t create one. When you work with MySQL you have to create one, so:
Go to the root of my application and type:
$ mysql –u root –p
$ Passwor: mypassword (Enter your password, this is the one you entered to secure MySQL)
Note: This example works wit a project called toy and the user I wanted to grant privileges is mark and the password I’ll give is 45mark. Below you will see where I apply these elements. Remember to apply your own elements on each part of the statement.
Create and user for this project
Once you are in, you will see the pointer (mysql> ), so type after it:
mysql> GRANT ALL PRIVILEGES ON toy_development.* TO 'mark'#'localhost' IDENTIFIED BY '45mark';
Then type:
mysql> exit;
Check that it is working by typing:
$ mysql –u mark –p toy_development
Enter password: 45mark (You enter the one you gave)
Open database.yml file and configure what is needed and fix as required. In my case I will chance the username to mark and the password to 45mark
default: &default
adapter: mysql2
encoding: utf8
pool: 5
username: mark
password: 45mark
socket: /tmp/mysql.sock
- Also, REMOVE the hash tag (#) added before
development:
: *default
database: toy_development
Save it.
Go to the root of the application and type
$ rake db:schema:dump
Done!!
I hope this helps. Happy coding!!
Thanks
I had the same issue with ruby 1.9.2-p180 , upgraded to p290 and it works
Just restart the server; in the command line:
Press Ctrl + C
execute:
rails s
I had a similar error when i typed rake db:schema:dump and it turns out that I just have to comment out all the databases on my yaml file except my development one.
Give a try to this.
rake db:test:prepare
Install this to see if you have actually created a table or not. Open the "development.sqlite3" in db folder
http://sqlitebrowser.org/
Its a simple error checkout the entire database.yml file and see that where is default decription is given database name is given or not if not then look below it there will another development name is also given where configuration of database is use check that give your database name in it
default: &default
adapter: mysql2
encoding: utf8
pool: 5
username: root
password: 12345
host: localhost
development:
<<: *default
database: db_name
One potential cause is that there is a DATABASE_URL environment variable defined.
$ echo $DATABASE_URL
=> mysql2://root#localhost:3306
If you get a similar output to the above url (i.e., the database name is not present), then you might want to add the database name to the string or unset the env var.
$ export DATABASE_URL=mysql2://root#localhost:3306/my_rails_app_development
$ unset DATABASE_URL
If you unset the var, you probably want to specify the database details in database.yml instead.

Resources