I have some weird issues going on (for a very weird use case as I'll explain). I'm setting up a multi-tenant application using postgres schemas for data multi-tenancy.
Each company in my system will get its own schema. The way I accomplish this is with an after_commit on the model, on create, that then goes and creates a new postgres schema, and loads schema.rb into it. (copied from rake db:schema:load code) using ruby load.
You can see the gem here
Anyway, all this works (in the console). Creating a company creates the new schema and i can switch to it etc... my problem lies in my integration tests. I have an rspec test that creates to companies like so:
before do
#c1 = Factory :company
#c2 = Factory :company
end
What's odd is that I start to get the logs about the db schema loading, but they're truncated. Almost as if they're happening in parallel. Here's a sample output:
>> create: database: unique_name1
-- create_table("first_table_in_schema_rb", {:force=>true})
>> create: database: unique_name2
create: database is my log line, the -- create_table is from schema.rb itself.
As you can see, the second create: database seems to happen while I'm loading schema.rb from the first company creation.
Does anyone know if load is somehow asynchronous? I know ruby doesn't have real threads, but could it be using fibres or something? This is really messing me up because when my test comes around, the postgres schema that was meant to be created doesn't seem to exist.
Rails 3.0.8
Ruby 1.9.2
Im not 100% sure this is your problem because im sure of what happens with require but not with load, the things this happen to me once with require because require is not atomic, so loading code from a file with require will cause a race condition. Maybe that is what is happening with load but i was not able to find any info about load been atomic or not.
nevermind... issue had nothing to do with load it was the fact that i was already connected to the wrong schema when importing the schema.rb
There was in fact an exception being thrown that was silently caught somewhere
Related
I'm trying to load some seed data into an app with heroku. I'm more looking just to insert a bunch of seed (sample) data into my app so my client can see it with a lot of objects. (Imagine a demo e-commerce app - I want to be able to show a couple dozen sample products without manually entering them in)
My seed data works fine in development but in production, the one HMBT association causes the below error:
WARNING: Rails was not able to disable referential integrity.
This is most likely caused due to missing permissions.
Rails needs superuser privileges to disable referential integrity.
Heroku documentation said this this so I tried a conditional to remove the referential integrity on production in my seeds file(see below), but now seeds won't run. It just does this:
Running rake db:seed on ⬢ app... up, run.1225 (Free)
ActiveRecord::SchemaMigration Load (1.7ms) SELECT "schema_migrations".* FROM "schema_migrations"
Here's my seeds file below:
if Rails.env.development?
ActiveRecord::Base.connection.disable_referential_integrity do
end
1.upto(14) do |n|
pic_one = File.open(File.join(Rails.root,'app/assets/images/file1.jpg'))
pic_two = File.open(File.join(Rails.root,'app/assets/images/carpet_2/file2.jpg'))
image = PicAttachment.create!([
{picture: pic_one, w_dream_id: "#{n}"},
{picture: pic_two, w_dream_id: "#{n}"},
])
rug = Product.create!(
pic_attachments: image
end
if Rails.env.development?
end
end
Does anyone where I'm going wrong?
The link you posted states that referential integrity cannot be removed in Heroku. It suggests considering using another test data scaffolding tool (like FactoryGirl or Fabrication Gem)
Anyway, your code does nothing if the environment is not development. All code is inside the if Rails.env.development?. The first end corresponds to the do. The indentation is wrong. Your code is in fact:
if Rails.env.development?
ActiveRecord::Base.connection.disable_referential_integrity do
end
1.upto(14) do |n|
pic_one = File.open(File.join(Rails.root,'app/assets/images/file1.jpg'))
pic_two = File.open(File.join(Rails.root,'app/assets/images/carpet_2/file2.jpg'))
image = PicAttachment.create!([
{picture: pic_one, w_dream_id: "#{n}"},
{picture: pic_two, w_dream_id: "#{n}"},
])
rug = Product.create!(
pic_attachments: image
if Rails.env.development?
end
end
Ultimately I took this answer, which was already in my code to ensure associations work in development.
For development, this is needed:
ActiveRecord::Base.connection.disable_referential_integrity do
For production:
the disable_referential_integrity isn't allowed in heroku and the problem is the associated model (Pic_Attachment) gets created before the model object it belongs to, hence an error gets thrown because it needs an object to belong. What worked for me was to delete the disable_referential_integrity from the seeds file AND comment out the belongs_to line in the associated model (PicAttachment), then commit/push your changes and it works. (Add these lines back in afterwards so your development works)
I hope this helps someone. Took me a few days of work here.
Working on a 3.0 rails project for a client and I have a sensitive migration that I need to run off a live production server. Its essentially suppose to down case all the State abbreviations in the DB, FL -> fl, PA -> pa etc...
I can't test locally due to restrictions:
does calling the wording of the migration effect anything? I know it does with add and create etc but not sure when updating info like this.
rails g migration UpdateStateAbbreviation
def self.up
say_with_time "Updating states abbreviation..." do
State.find(:all).each do |s|
tmp = s.abbreviation.downcase
s.update_attribute :abbreviation, tmp
end
end end
Rake db:migrate
One very important rule with migrations is to never reference models in your migrations. This might seem like an academic concern, but at some point in the future you may not have a State model at all, and when you remove app/models/state.rb then this migration will not work.
A properly constructed migration will execute properly regardless of changes in the future. Whatever it does may be later un-done, there's nothing wrong with that, but setting it up for failure is never a good idea.
You can do this downcasing operation in your database using a string function and something like:
execute "UPDATE states SET abbreviation=LOWER(abbreviation)"
Using models in the migration causes all sorts of problems. This goes for using your model to pre-populate certain key records as well. Use seeds.rb if you must, or even better, a rake task to do it for you.
As a note, if you can't test locally you have a very flawed development process. You should always run and test your migrations, both up and down where applicable, to ensure that they work correctly. Where you can't get actual production data for reasons of security or privacy, work with your DBA to get a scrubbed, non-sensitive version for testing purposes. State names should not be confidential, for instance.
I have a lot of spots in my code that actually call activerecord finders. For example, in a Blog engine, I might have a table of tags that correspond to an activerecord model Tag. Suppose, for some reason, that I want special logic to happen if a post is created with a tag where tag.description == 'humor'. Then I might have a method in the model:
class Tag < ActiveRecord::Base
def self.humor_tag
find_by_description('humor')
end
end
Whether or not this is poor design, it causes insane amounts of problems for me when using rake commands to build a database. Say that later on, I've finished my development and I want to deploy to production. So I take the dumped schema.rb file, and then I want to load a new database structure from that schema.rb, or alternatively, just run my migrations to create a production database.
RAILS_ENV=production rake db:schema:load
The problem is that, in the production environment, the rake command seems to load every model. When it tries to load the Tag#humor_tag method, it throws an error that stops the process:
rake aborted!
Table 'production_database.tags' doesn't exist
Well of course it doesn't exist, it hasn't been created yet! I've googled around and people seem to solve this problem by either cloning the database in SQL or moving around their code just so they can run the rake task.
What are you supposed to do? It seems like there might be some configuration somewhere to let you tell rake to freaking ignore calls to database tables before any tables are created.
I would suggest replacing queries by class methods with scopes: http://guides.rubyonrails.org/active_record_querying.html#scopes
and if you have an initializer that is causing the models to load, use a proc in the scope definition, such as
class Post < ActiveRecord::Base
scope :published, Proc.new { where(:published => true) }
end
to prevent the scope from running at initialization time.
I'm not completely satisfied with this answer, but if anyone gets to this question and has a similar problem, this may be helpful. In order to move over a database in a situation where you would usually rake db:schema:load or just create it and run the migrations, you can alternatively load the database from SQL (or presumably other database technologies).
rake db:structure:dump
That command will dump the structure of the database into a file that will then be able to recreate it. For me, it created a file db/development_dump.sql, that contained calls to create all of the tables and indices, but didn't copy any of the data like on a normal sql dump. Then, I moved that file to my production database, and ran it:
mysql prod_database < development_dump.sql
This doesn't answer the question at hand, but it may be relevant for someone facing a similar problem.
I have a Rails project which has a Postgres database for the actual application but which needs to pull a heck of a lot of data out of an Oracle database.
database.yml looks like
development:
adapter: postgresql
database: blah blah
...
oracle_db:
adapter: oracle
database: blah blah
My models which descend from data on the Oracle DB look something like
class LegacyDataClass < ActiveRecord::Base
establish_connection "oracle_db"
set_primary_key :legacy_data_class_id
has_one :other_legacy_class, :foreign key => :other_legacy_class_id_with_funny_column_name
...
end
Now, by habit I often do a lot of my early development (and this is early development) by coding for a bit and then playing in the Rails console. For example, after defining all the associations for LegacyDataClass I'll start trying things like a = LegacyDataClass.find(:first); puts a.some_association.name. Unexpectedly, this dies with LegacyDataClass not being already loaded.
I can then require 'LegacyDataClass' which fixes the problem until I either need to reload!, which won't actually reload it, or until I open a new instance of the console.
Thus the questions:
Why does this happen? Clearly there is some Rails magic I am not understanding.
What is the convenient Rails workaround?
I believe this might have to do with your model name, rather than your connection. The Rails convention is that model class names are CamelCase, while the files they reside in are lowercase+underscore.
The "LegacyModel" class should therefore be in models/legacy_model.rb. Your statement about "require 'LegacyDataClass'" indicates that this is not the case, and therefore Rails doesn't know how to automagically load that model.
I wrote something for an app at work that handles connections to other databases' at runtime, it might be able to help.
http://github.com/cherring/connection_ninja
I have a migration that runs an SQL script to create a new Postgres schema. When creating a new database in Postgres by default it creates a schema called 'public', which is the main schema we use. The migration to create the new database schema seems to be working fine, however the problem occurs after the migration has run, when rails tries to update the 'schema_info' table that it relies on it says that it does not exist, as if it is looking for it in the new database schema and not the default 'public' schema where the table actually is.
Does anybody know how I can tell rails to look at the 'public' schema for this table?
Example of SQL being executed: ~
CREATE SCHEMA new_schema;
COMMENT ON SCHEMA new_schema IS 'this is the new Postgres database schema to sit along side the "public" schema';
-- various tables, triggers and functions created in new_schema
Error being thrown: ~
RuntimeError: ERROR C42P01 Mrelation "schema_info" does not exist
L221 RRangeVarGetRelid: UPDATE schema_info SET version = ??
Thanks for your help
Chris Knight
Well that depends what your migration looks like, what your database.yml looks like and what exactly you are trying to attempt. Anyway more information is needed change the names if you have to and post an example database.yml and the migration. does the migration change the search_path for the adapter for example ?
But know that in general rails and postgresql schemas don't work well together (yet?).
There are a few places which have problems. Try and build and app that uses only one pg database with 2 non-default schemas one for dev and one for test and tell me about it. (from thefollowing I can already tell you that you will get burned)
Maybe it was fixed since the last time I played with it but when I see http://rails.lighthouseapp.com/projects/8994/tickets/390-postgres-adapter-quotes-table-name-breaks-when-non-default-schema-is-used or this http://rails.lighthouseapp.com/projects/8994/tickets/918-postgresql-tables-not-generating-correct-schema-list or this in postgresql_adapter.rb
# Drops a PostgreSQL database
#
# Example:
# drop_database 'matt_development'
def drop_database(name) #:nodoc:
execute "DROP DATABASE IF EXISTS #{name}"
end
(yes this is wrong if you use the same database with different schemas for both dev and test, this would drop both databases each time you run the unit tests !)
I actually started writing patches. the first one was for the indexes methods in the adapter which didn't care about the search_path ending up with duplicated indexes in some conditions, then I started getting hurt by the rest and ended up abandonning the idea of using schemas: I wanted to get my app done and I didn't have the extra time needed to fix the problems I had using schemas.
I'm not sure I understand what you're asking exactly, but, rake will be expecting to update the version of the Rails schema into the schema_info table. Check your database.yml config file, this is where rake will be looking to find the table to update.
Is it a possibility that you are migrating to a new Postgres schema and rake is still pointing to the old one? I'm not sure then that a standard Rails migration is what you need. It might be best to create your own rake task instead.
Edit: If you're referencing two different databases or Postgres schemas, Rails doesn't support this in standard migrations. Rails assumes one database, so migrations from one database to another is usually not possible. When you run "rake db:migrate" it actually looks at the RAILS_ENV environment variable to find the correct entry in database.yml. If rake starts the migration looking at the "development" environment and database config from database.yml, it will expect to update to this environment at the end of the migration.
So, you'll probably need to do this from outside the Rails stack as you can't reference two databases at the same time within Rails. There are attempts at plugins to allow this, but they're majorly hacky and don't work properly.
You can use pg_power. It provides additional DSL for migration to create PostgreSQL schemas and not only.