Rake unconsistent behaviour between dev and test? - ruby-on-rails

I develop a RoR application using a PostgreSQL database, based on this database.yml definition:
# PostGre databases
default: &default
host : localhost
adapter: postgresql
encoding: unicode
pool: 5
username: keyman
password: keymanApp
schema_search_path: "keyman"
development:
<<: *default
database: keyman_dev
test:
<<: *default
database: keyman_test
I created a small Rake routine, so I can easily drop and create my postgreSQL database, including the schema I work with:
namespace :db do
desc 'Create database schemas before going for the first migration'
task init: ['db:drop','db:create'] do
ActiveRecord::Base.connection.execute("CREATE SCHEMA keyman AUTHORIZATION keyman")
puts 'Database initialised'
end
end
When I run rake db:init, it is executed both on dev and test environments:
$ rake db:init
Dropped database 'keyman_dev'
Dropped database 'keyman_test'
Created database 'keyman_dev'
Created database 'keyman_test'
Database initialised
But the result is not the same: the schema 'keyman' is created for the keyman_dev database, but not for the keyman_test database.
I need to run explicitly rake db:init RAILS_ENV=test to get the schema created on the test database.
It sounds strange to me! Do you have an explanation?
Thanks

when running bin/rake -T db we can see the following descriptions for db:create and db:drop
rake db:create # Creates the database from DATABASE_URL or config/database.yml for the current RAILS_ENV (use db:create:all to create all databases in the config). Without RAILS_ENV or when RAILS_ENV is development, it defaults to creating the development and test databases
rake db:drop # Drops the database from DATABASE_URL or config/database.yml for the current RAILS_ENV (use db:drop:all to drop all databases in the config). Without RAILS_ENV or when RAILS_ENV is development, it defaults to dropping the development and test databases
So, when you run those tasks in the development environment, they both do the development and test databases, but that doesn't mean that other tasks (such as your custom task) automatically run in both environments, you're task is still only running in the development environment.
Something like this in your rake task should get that query to run in both tables, though this is untested.
environments = Rails.env.development? ? [:development, :test] : [Rails.env.to_sym]
environments.each do |env|
ActiveRecord::Base.establish_connection(env)
# do something
end

Related

Postgres db not working on Heroku

I've spend several hours figuring out how to get my database up and running. I created a new rails app and wanted to deploy it to heroku. I followed the instructions from heroku (to switch from sqlite3 -> postgresql) but it just doesn't work.
This is in my database.yml file:
default: &default
adapter: postgresql
pool: 5
timeout: 5000
development:
<<: *default
database: myapp_development
test:
<<: *default
database: myapp_test
production:
<<: *default
database: myapp_production
url: <%= ENV['DATABASE_URL'] %>
I can't create or seed any data in the database. Sometimes it executes the db:migrate, but even then it doesn't create anything. This is what I get when running:
heroku run rake db:create
FATAL: permission denied for database "postgres"
DETAIL: User does not have CONNECT privilege.
Does anyone has an idea on how to solve this? I don't have a clue anymore ...
Thanks!
By default you don't need to create a db on heroku.
Just run heroku run rails db:migrate and rest of the stuff will be handled by heroku itself.
Also your database.yml should be changed to following for Production env.
production:
<<: *default
database: myapp_production
username: myapp
password: <%= ENV['MYAPP_DATABASE_PASSWORD'] %>
Also your rails app is by default assigned with a Postgres addon.
You can check this by running command heroku addons in the console.
Source -
Heroku Getting Started Guide
Heroku Postgres Addon Guide
You cannot create a database on Heroku using db:create (you cannot drop it neither). Your database is created when you add an add-on (such as Heroku Postgres). You can only migrate and seed. And if you want to start over, you can use pg:reset (instead of drop and create)
So the correct sequence should be:
add the Heroku add-on (such as Heroku Postgres). Add-ons are located here: https://elements.heroku.com/addons.
rake db:migrate
rake db:seed
if you want to start over
rake pg:reset
rake db:migrate
rake db:seed
From Heroku documentation: https://devcenter.heroku.com/articles/heroku-postgresql
The PostgreSQL user your database is assigned doesn’t have permission to create or drop databases. To drop and recreate your database use pg:reset.
As per the given stacktrace, it seems like you are trying to create a database on heroku which in turn is giving you Permission Denied Error.
Firstly, you do not need to run
heroku run rake db:create
Instead run
heroku run rake db:migrate
and it should migrate the migrations which are down.
For checking the current status of migrations run the following command:
heroku run rake db:migrate:status
Other Point you mentioned:
-> I can't create or seed any data in the database
As already mentioned above you can't create a database as heroku does it for you .
For seeding data in the database run the following command:
heroku run rake db:seed

Rake db:reset execute in development and test environment

I got a Ruby on Rails v5 application and I'm trying to run the following Rake task in development environment:
rake db:reset
Sadly, Rake runs the task in both development and test environment. Since I don't use the test environment, I don't wanna double my database. I have read somewhere that Rake run task in both environment when there is no RAILS_ENV defined. So I try to add the following line to my .bash_profile without any success:
export RAILS_ENV="development"
I also tried to add RAILS_ENV=development at the end of my task but it also did not worked.
Is there a way to use Rake in development only ?
Update #1
Thanks to Taryn East for the quick comment. I'm trying to update my post as quick as possible to make it easy for you to answer efficiently.
What do you actually observe when you run that command?
The command are simply executed twice. Since I have different database for my development and test environment, it does not show me any error at the moment. On the other hand, it does force me to use two database for no reason at all. I also tried to set the same database for the two environments, but then I was getting error since Rake tried to run the task twice on the database.
eg of output for rake db:drop
Dropped database 'db'
Database 'db' does not exist
It did drop my database, but as you can see Rake tried to run the command another time for the other environment.
Why aren't you using the test environment?
Because I don't see any advantage of it for my current project. I don't have time to create and update tests. I'm also not used to use both development and test environment at the same time.
If you want to run a rake task in just a single environment, you can also add RAILS_ENV to the command line.
As I said in my post, it does not work. Here's what I did:
RAILS_ENV=development rake db:drop
And here's the output:
Dropped database 'db'
Database 'db' does not exist
Update #2
Here's my database.yml configuration:
default: &default
adapter:mysql2
pool: 5
timeout: 5000
development:
<<: *default
database: db
username: something
password: something
host: localhost
port: 3306
test:
<<: *default
database: db_test
username: something
password: something
host: localhost
port: 3306
I don't know if it can help, but my environment is development according to the rake about task.
I have read somewhere that Rake run task in both environment when
there is no RAILS_ENV defined.
This is not correct. If the environment is development, test is also added to current environments. See: https://github.com/rails/rails/blob/f507085fa2a7625324e6f4e3ecef9b27dabefb4f/activerecord/lib/active_record/tasks/database_tasks.rb#L339
So this is expected behaviour.
On the other hand what you're saying will be true for any other env. E.g. RAILS_ENV=test rails db:drop will only execute for test env.

Errors when trying to prepare testing with Ruby on Rails that uses a SQL Server 2008 R2 database

What do I need to do to be able to run tests with a SQL Server 2008 database server using the gem activerecord-sqlserver-adapter?
I am able to connect to the development database just fine using a 32bit ODBC connection through dev settings in database.yml:
#SQL Server
development:
adapter: sqlserver
mode: odbc
dsn: <odbc_name>
username: <db_user>
password: <db_password>
host: <sql host>
I have started to setup some testing but run into an error when running:
rake db:test:prepare
Here is the error:
rake aborted!
ODBC::Error: 37000 (3708) [Microsoft][SQL Native Client][SQL Server]Cannot drop
the database 'master' because it is a system database.: DROP DATABASE [master]
Tasks: TOP => db:test:load => db:test:purge
(See full trace by running task with --trace)
This error doesn't make sense to me. Why is it trying to drop the master db?
My test setup in database.yml is exactly the same as my dev above except it points to a different odbc which points to a different database.
EDIT
I have noticed that when I run rake db:test:prepare the first step must be to delete the test database. So it knows what database it is supposed to use at first.
Why does it then try to delete the master?
Is this because when sql doesn't find the default database it gives the user master by default and rake db:test:prepare cannot create the users default db so it loops back to the beginning and tries to drop again?
I got this to work by specifying the name of the database in the test configuration:
# config/database.yml
...
test:
adapter: sqlserver
mode: odbc
dsn: <odbc_name>
database: <database_name>
username: <db_user>
password: <db_password>
...

Rake db:test:prepare task deleting data in development database

Using a simple Rails sqlite3 configuration example in my config/database.yml for a Rails 3.2.6 app, I used to reset my development database, re-seed it, and prepare my test database simply by performing:
$ rake db:reset
$ rake db:test:prepare
After looking at this blog entry about testing a Rails application with Travis CI on different database engines, I thought I'd give it a try, so I installed mysql and postgresql using Homebrew (I'm on OSX Snow Leopard), set them up as per the brew info instructions. I installed the relevant gems, and configured the database and Travis files as follows:
Gemfile
# ...
group :development, :test do
# ...
gem 'sqlite3', '1.3.6'
end
group :test do
# ...
# Test mysql on Travis CI
gem 'mysql2', '0.3.11'
end
group :test, :production do
# ...
# Test postgres on Travis CI and deploy on Heroku
gem 'pg', '0.13.2'
end
config/database.yml
sqlite: &sqlite
adapter: sqlite3
database: db/<%= Rails.env %>.sqlite3
mysql: &mysql
adapter: mysql2
username: root
password:
database: my_app_<%= Rails.env %>
postgresql: &postgresql
adapter: postgresql
username: postgres
password:
database: my_app_<%= Rails.env %>
min_messages: ERROR
defaults: &defaults
pool: 5
timeout: 5000
host: localhost
<<: *<%= ENV['DB'] || "sqlite" %>
development:
<<: *defaults
test: &test
<<: *defaults
production:
<<: *defaults
cucumber:
<<: *test
.travis.yml
language: ruby
rvm:
- 1.9.2
- 1.9.3
env:
- DB=sqlite
- DB=mysql
- DB=postgresql
script:
- RAILS_ENV=test bundle exec rake --trace db:migrate
- bundle exec rake db:test:prepare
- bundle exec rspec spec/
before_script:
- mysql -e 'create database my_app_test'
- psql -c 'create database my_app_test' -U postgres
bundler_args: --binstubs=./bundler_stubs
Now, though, when I run rake db:reset, I get a Couldn't drop db/development.sqlite3 error message before the development database is successfully created. So, it seems that there are now multiple calls being made to drop the same database(?). The traced output looks like:
$ rake db:reset --trace
** Invoke db:reset (first_time)
** Invoke environment (first_time)
** Execute environment
** Execute db:reset
** Invoke db:drop (first_time)
** Invoke db:load_config (first_time)
** Invoke rails_env (first_time)
** Execute rails_env
** Execute db:load_config
** Execute db:drop
Couldn't drop db/development.sqlite3 : #<Errno::ENOENT: No such file or directory - my_app/db/development.sqlite3>
** Invoke db:setup (first_time)
** Invoke db:schema:load_if_ruby (first_time)
** Invoke db:create (first_time)
** Invoke db:load_config
** Execute db:create
db/development.sqlite3 already exists
# ...
This is odd, but at least the development database gets created and seeded. The real issue comes when I run rake db:test:prepare: although there are no error messages, as well as the test database not being created, the data in the development database gets blown away (schema is still in tact, though). I tried directly specifying the Rails environment for the command and got:
$ rake db:test:prepare RAILS_ENV=test
You have 7 pending migrations:
20120503193649 CreateUsers
# ...
Run `rake db:migrate` to update your database then try again.
After running rake db:migrate RAILS_ENV=test, I could run my rspec tests again. So, my rake commands to get the same results have now changed to:
$ rake db:reset # (with an error)
$ rake db:migrate RAILS_ENV=test
If I change my config/database.yml file back to a simple sqlite3 only configuration, db:reset and db:test:prepare work as I expect.
So, does this mean that my mysql and/or postgres settings are causing rake tasks to repeat and/or they're messing with the Rails environment settings? Where should I be looking to confirm if my environment is really set up to work properly with these 3 database engines?
Edit
Looking at the release notes for Rails 3.2.8.rc2, I found a change to ActiveRecord potentially related to this question:
Do not set RAILS_ENV to development when using db:test:prepare and related rake tasks. This was causing the truncation of the development database data when using RSpec. In RC2 was fixed again when using config.active_record.schema_format = :sql
config/application.rb has the following explanation:
# Use SQL instead of Active Record's schema dumper when creating the database.
# This is necessary if your schema can't be completely dumped by the schema dumper,
# like if you have constraints or database-specific column types
# config.active_record.schema_format = :sql
My schema doesn't have constraints or database-specific column types, so I didn't uncomment this line, however, given the content of the release note, I wagered that RAILS_ENV defaulting to development could be responsible for the deleted data in the development environment. So, I tried out a few things and got expected results by doing what I did before (after upgrading Rails to 3.2.8.rc2):
$ rake db:reset # (with an error)
$ rake db:test:prepare RAILS_ENV=test # (with no "pending migrations" issue)
This is a bit better, but still seems wrong to me since there is still an error with rake db:reset, and it doesn't make sense to me to have to set RAILS_ENV=test when running a rake command specifically tailored for the test database.
Update
It would seem that upgrading to Rails 3.2.9 solves this issue due to the following fix:
Fix bug where rake db:test:prepare tries to load the structure.sql into development database. Fixes #8032.
Grace Liu + Rafael Mendonça França
I can now again reset my development database, re-seed it, and prepare my test database simply by performing:
$ rake db:reset
$ rake db:test:prepare
Your development database is being purged because ActiveRecord::Base.configurations has the test database set to "development.sqlite3". When the rake task is run, the yaml configuration is eval'ed into the ActiveRecord::Base.configurations hash and at that time Rails.env is set to development.
If RAILS_ENV=development, the database value for test will be set to
database: db/development.sqlite3
or for a different adapter:
database: my_app_development
You can reproduce this with a simple sqlite only configuration buy changing the test block inside database.yml to the following:
test:
adapter: sqlite3
database: db/<%= Rails.env %>.sqlite3
pool: 5
timeout: 5000
If you inspect the full ActiveRecord::Base.configurations hash you'll see that test is set to use the development db if no RAILS_ENV is specified. And if you were to specify 'production' or 'staging' it would be set to that. From the console:
# rails c
> ActiveRecord::Base.configurations['test']['database']
=> "db/development.sqlite3"
compared with:
# RAILS_ENV=test rails c
> ActiveRecord::Base.configurations['test']['database']
=> "db/test.sqlite3"
Update
The issue you are seeing with db:reset is also because your yaml file is interpreted once and then the config is set.
db:reset will invoke db:drop and db:setup for the given environment. However, if the environment is development, it also does those tasks for the test environment. So it succeeds in dropping for the development environment and then when it executes for test, the database key of the configuration is identical to the development section, hence it can't drop something that no longer exists. Here is what the ActiveRecord::Base.configurations hash looks like when Rails.env == 'development'
"development" => {
"adapter" => "sqlite3",
"database" => "db/development.sqlite3",
"pool" => 5,
"timeout" => 5000
},
"test" => {
"adapter" => "sqlite3",
"database" => "db/development.sqlite3",
"pool" =>5,
"timeout"=>5000
},
"production" => {
"adapter" => "sqlite3",
"database" => "db/development.sqlite3",
"pool"=>5,
"timeout"=>5000
}
And once it's in that hash, it doesn't go back and re-read the database.yml file. That hash is what is generated given this database.yml
development:
adapter: sqlite3
database: db/<%= Rails.env %>.sqlite3
pool: 5
timeout: 5000
test:
adapter: sqlite3
database: db/<%= Rails.env %>.sqlite3
pool: 5
timeout: 5000
production:
adapter: sqlite3
database: db/<%= Rails.env %>.sqlite3
pool: 5
timeout: 5000
Same problem i.e. development database destroyed after a "rake".
My way out "rake RAILS_ENV=test".
Using ruby 1.9.3p194 Rails 3.2.7 sqlite3.

why 'rake test' is trying to connect to my development DB?

I have my config/database.yml like this:
development:
adapter: postgresql
database: psql_dev
username: postgres
min_messages: WARNING
test:
adapter: sqlite3
database: db/test.sqlite3
min_messages: WARNING
When I run rake test:units, it reports an error:
rake aborted!
could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/tmp/.s.PGSQL.5432"?
Why didn't it connect to my test DB(db/test.sqlite3).
and, If I run the test like this rake test RAILS_ENV=test, it works well.
Isn't RAILS_ENV=test the default setting for rake test?
I'm running rails 2.3.5 with ruby 1.8.7, and my $RAILS_ENV is not defined in my shell.
What's happening is that rake test depends on rake db:test:prepare which will attempt to load the current schema from the development database. That's how the test database gets updated when a migration is run on the development database
do you have a test:units rake task? Run:
rake test
does that work? Also can you paste the output of:
rake -T | grep tests

Resources