After migrating from RDS Postgresql to RDS Aurora PostgreSQL, I am having an issue to where new inserts are trying to start from key ID 2 instead of the last record.
In my rails app, here's what I'm seeing when I try to create a new record:
ActiveRecord::RecordNotUnique (PG::UniqueViolation: ERROR: duplicate key value violates unique constraint "global_options_pkey"
DETAIL: Key (id)=(3) already exists.
):
app/controllers/schedules_controller.rb:45:in `create'
Not sure why this would be the case. To set up Aurora PSQL in Rails, I followed this tutorial: https://guides.rubyonrails.org/active_record_multiple_databases.html. It seems like everything is working fine (auto switching between reader/writer instances, etc.).
With the migration, I specifically used the AWS Schema Conversion Tool (SCT), followed by a migration with the Database Migration Service.
Is this error caused by something that was done incorrectly in the migration process or do I need to some post-migration processes to have this fixed?
Definitely a case of when you migrated old data, but not with INSERT and your sequences are still on the initial values.
Use sequence manipulation functions to check what is the current value with lastval(), and set it to max(id) with setval()
You can try
ActiveRecord::Base.connection.tables.each do |table_name|
ActiveRecord::Base.connection.reset_pk_sequence!(table_name)
end
Related
I'm pretty new to rails (3.2.13) development and have not been able to find a way to do what I'm trying to do.
Background: I created a migration that, because there appears no other way to do this using sqlite, creates a new table in the database with a foreign key. Specifically, I ran:
rails g model period_type_specs period_type_id:integer start_date:datetime end_date:datetime user:references
I then wanted to ensure that the period_type_specs table had a foreign key constraint (period_type_id -> period_types(period_type_id)). After researching and trying a few things, the only thing I found that worked with sqlite was, in the CreatePeriodTypeSpecs migration file, replace what was there with pure SQL to create the table with the desired foreign key, using 'execute'. This worked - that is, after rake db:migrate, the new table, with the needed foreign key, was created in the development database.
Problem: I've so far found no "kosher" way to get the same table/schema into the test database. After the db:migrate, rake db:test:prepare creates the table, but with no foreign key. I believe this is because db:test:prepare simply uses db/schema.rb to create the missing table and schema.rb uses the standard "create_table..." operations and (I believe) raw SQL is not supported in schema.rb.
The only way I can think of at this point to create the table as I want in the test database is to simply run the sql commands, from the migration file, directly on the test database. But, of course, this would interfere with the goal of using rake to reset or remove and then automatically recreate the database schema.
So - is there a way to do what I want without cheating?
Update: I ended up putting the SQL statement to create the table with the foreign key into db/seeds.rb, which is also where the table it refers to (a static reference table) is populated.
I've been debugging this strange problem of Rails giving me "Unknown primary key for table...", even when the table's ID is there.
I've copied the database from one heroku app to another, on the original databse there is no problem and the new one gives me a db error.
This is the error:
ProductsController# (ActionView::Template::Error) "Unknown primary key for table collections in model Collection."
/app/vendor/bundle/ruby/2.0.0/gems/activerecord-3.2.13/lib/active_record/reflection.rb:366:in `primary_key'
/app/vendor/bundle/ruby/2.0.0/gems/activerecord-3.2.13/lib/active_record/reflection.rb:480:in `association_primary_key'
/app/vendor/bundle/ruby/2.0.0/gems/activerecord-3.2.13/lib/active_record/associations/association_scope.rb:58:in `block in add_constraints'
/app/vendor/bundle/ruby/2.0.0/gems/activerecord-3.2.13/lib/active_record/associations/association_scope.rb:39:in `each'
/app/vendor/bundle/ruby/2.0.0/gems/activerecord-3.2.13/lib/active_record/associations/association_scope.rb:39:in `each_with_index'
/app/vendor/bundle/ruby/2.0.0/gems/activerecord-3.2.13/lib/active_record/associations/association_scope.rb:39:in `add_constraints'
/app/vendor/bundle/ruby/2.0.0/gems/activerecord-3.2.13/lib/active_record/associations/association_scope.rb:31:in `scope'
/app/vendor/bundle/ruby/2.0.0/gems/activerecord-3.2.13/lib/active_record/associations/association.rb:98:in `association_scope'
/app/vendor/bundle/ruby/2.0.0/gems/activerecord-3.2.13/lib/active_record/associations/association.rb:87:in `scoped'
/app/vendor/bundle/ruby/2.0.0/gems/activerecord-3.2.13/lib/active_record/associations/collection_association.rb:573:in `first_or_last'
/app/vendor/bundle/ruby/2.0.0/gems/activerecord-3.2.13/lib/active_record/associations/collection_association.rb:105:in `last'
/app/vendor/bundle/ruby/2.0.0/gems/activerecord-3.2.13/lib/active_record/associations/collection_proxy.rb:46:in `last'
/app/app/helpers/likes_helper.rb:62:in `significant_liker'
The line that causes it:
product.collections.last.try :user
and the table:
d8apjspa441pad=> \d collections
Table "public.collections"
Column | Type | Modifiers
----------------+------------------------+----------------------------------------------------------
id | integer | not null default nextval('collections_id_seq'::regclass)
name | character varying(255) |
user_id | integer |
permalink | character varying(255) |
category_id | integer |
products_count | integer |
is_featured | boolean |
Indexes:
"index_lists_on_user_id_and_permalink" UNIQUE, btree (user_id, permalink)
Any idea why this might happen?
Thanks!
Seems primary key is missing for the table collections.
Prior to Rails 3.2, set the primary key in model like
class Collection < ActiveRecord::Base
set_primary_key "my_existing_column"
end
In Rails 3.2+ and Rails 4, set the primary key in model like
class Collection < ActiveRecord::Base
self.primary_key = "my_existing_column"
end
OR
We can alter the table and set the primary key for id like
Create a migration file to set the primary key
class AddPrimaryKeyToCollections < ActiveRecord::Migration
def change
execute "ALTER TABLE collections ADD PRIMARY KEY (id);"
end
end
I was having a similar problem and this was the only page I could find. So just in case it will be of help to anyone else...
I started suddenly getting missing primary key messages on a couple tables. I'm guessing, but not sure, that this started happening after pushing data (pg_dump local, heroku pg:restore)
The primary keys in question were both on tables that had been renamed so that the pkey name did not match the table name--but on the other hand lots of other renamed tables were in the same boat and did not have problems.
Anyway, at one point in my flailing around I tried uploading another dump file and I noticed some complaints on the offending indices. First it would try to delete them and complain that it couldn't because they did not exist. Later it would try to create them and complain that it couldn't because they already existed.
Very annoying considering that pkey info doesn't show up in schema.rb and is supposed to 'just work', right?
Anyway, what worked for me (and thus the reason I'm posting) is to do a heroku pg:reset and then load the dump again. Side note, I got 'internal server error' the first two times I tried heroku pg:reset. But later I tried again and it worked.
TL;DR: try restarting the app.
heroku restart
I recently encountered this error: "Unknown primary key for table", and like the question asker, it appeared after copying a database to a Heroku app.
The copied database had no errors, so I was confident the table and primary key were fine.
I tried a few suggestions on this page, including starting from scratch with a heroku pg:reset, new pg_dump of the old database, and pgbackups:restore into the new database, then running migrations and seeding... nothing worked.
What finally solved my problem was restarting the app. The new app had many database migrations, and restarting the app reloaded the schema and picked up the changes. This page from Heroku's documentation explains:
Running Rake Commands
After running a migration you’ll want to restart your app with heroku restart to reload the schema and pickup any schema changes.
I was restoring database dump from heroku to my local system and was getting this error..
ActiveRecord::UnknownPrimaryKey: ActiveRecord::UnknownPrimaryKey
I was restoring on existing database, so I dropped the database, created new database and then restore the dump and it worked for me
What helped for me (happened on heroku after a db restore) is reindexing the primary key index:
reindex index $primary_key_index_name
I was having this problem and the issue turned out to be that my table somehow actually didn't have a primary key index. The solution was to create a migration that added a primary key:
execute "ALTER TABLE appointment_reminder_text ADD PRIMARY KEY (id)"
The same thing happens to me also while importing the data dump from heroku to local. I have the Rails 5.1.6 for the application.
After adding self.primary_key = "id" to all of the models which is showing issue, everything works fine for me.
If you're trying to troubleshoot this issue make sure you check your logs carefully. I noticed an earlier error related to a js asset that wasn't being precompiled. This got lost in the stack of rendering messages.
Once I fixed the asset precompilation issue, the 'Unknown primary key for table' error was no longer thrown.
This was 100% definitely the only thing I changed.
Thanks changing the index above worked for me. Just another quick note about how this error will manifest itself if there's a more complex relation involved:
ActiveRecord::StatementInvalid - PG::SyntaxError: ERROR: zero-length delimited identifier at or near """"
LINE 1: ...CT "users".* FROM "users" WHERE "benefits"."" IN ('1'...
restarting the heroku server worked for me. Maybe it was the spring-preloader which was recognizing the empty db-schema, during the db-restorement
I encountered this error doing tests in RSPEC.
In my case, i solved it by adding/setting primary_key in my Rails model
class Tablename < ApplicationRecord
self.primary_key ="id" if Rails.env.test? #optional condition
......
end
You should reload the schema on the target app. Then load the database.
I am in the process of learning Rails and I've ran into an interesting problem tonight.
I was creating a migration that would require an index on the foreign key:
Whenever I would run 'bundle exec rake db:migrate', I would receive this console error:
It looks as if it was trying to create the index before it was creating the reference.
The reason I believe this is because when I change the "subject" reference to a symbol:
The migration then suddenly works as expected!
This may just be the fact that I'm a total newby, but are symbols actually processed faster by Ruby than strings are?
Just curious - thanks!
This isn't a "faster" problem, or a problem of speed. The migrations are executed one line at a time, in order. The way you had it specified before simply didn't create the column correctly, hence when it got to the line where you create the index, the names didn't match up.
My guess is, with the string version it created the column name exactly as you spelled it, "subject" as opposed to subject_id when you use a symbol. Either way, you definitely had a name mismatch between when the column was created, and when the index was being built.
Always use symbols for this in your migrations, and you should be fine. Always check your schema.rb file, or browse the database using a GUI tool, after a migration to make sure the columns got created the way you expect, and with the data types you think they are, and you should be good.
I have a rails project that uses an old versions of the FlexImage plug-in to handle images.
In the old version that image data was stored in a column called "data", in the new version that column must be named "image_file_data".
I wrote a migration to rename the column, but when I try to run the migration, my guess is that rails tries to load the models, which then automatically check to see if the valid column is there (which it isn't) and that throws an error which halts the migration.
I would guess that my problems would be solved if I never loaded the model classes in question and just wrote some sql to rename the columns. But the following line doesn't work, since rails still tries to load the model.
Apartment.connection.execute "ALTER TABLE logos CHANGE DATA image_file_data MEDIUMBLOB;"
Oops, I figured it out. I was calling
model = (table_name.to_s).classify.constantize
Earlier, and this was causing the model to load
I have a migration that runs an SQL script to create a new Postgres schema. When creating a new database in Postgres by default it creates a schema called 'public', which is the main schema we use. The migration to create the new database schema seems to be working fine, however the problem occurs after the migration has run, when rails tries to update the 'schema_info' table that it relies on it says that it does not exist, as if it is looking for it in the new database schema and not the default 'public' schema where the table actually is.
Does anybody know how I can tell rails to look at the 'public' schema for this table?
Example of SQL being executed: ~
CREATE SCHEMA new_schema;
COMMENT ON SCHEMA new_schema IS 'this is the new Postgres database schema to sit along side the "public" schema';
-- various tables, triggers and functions created in new_schema
Error being thrown: ~
RuntimeError: ERROR C42P01 Mrelation "schema_info" does not exist
L221 RRangeVarGetRelid: UPDATE schema_info SET version = ??
Thanks for your help
Chris Knight
Well that depends what your migration looks like, what your database.yml looks like and what exactly you are trying to attempt. Anyway more information is needed change the names if you have to and post an example database.yml and the migration. does the migration change the search_path for the adapter for example ?
But know that in general rails and postgresql schemas don't work well together (yet?).
There are a few places which have problems. Try and build and app that uses only one pg database with 2 non-default schemas one for dev and one for test and tell me about it. (from thefollowing I can already tell you that you will get burned)
Maybe it was fixed since the last time I played with it but when I see http://rails.lighthouseapp.com/projects/8994/tickets/390-postgres-adapter-quotes-table-name-breaks-when-non-default-schema-is-used or this http://rails.lighthouseapp.com/projects/8994/tickets/918-postgresql-tables-not-generating-correct-schema-list or this in postgresql_adapter.rb
# Drops a PostgreSQL database
#
# Example:
# drop_database 'matt_development'
def drop_database(name) #:nodoc:
execute "DROP DATABASE IF EXISTS #{name}"
end
(yes this is wrong if you use the same database with different schemas for both dev and test, this would drop both databases each time you run the unit tests !)
I actually started writing patches. the first one was for the indexes methods in the adapter which didn't care about the search_path ending up with duplicated indexes in some conditions, then I started getting hurt by the rest and ended up abandonning the idea of using schemas: I wanted to get my app done and I didn't have the extra time needed to fix the problems I had using schemas.
I'm not sure I understand what you're asking exactly, but, rake will be expecting to update the version of the Rails schema into the schema_info table. Check your database.yml config file, this is where rake will be looking to find the table to update.
Is it a possibility that you are migrating to a new Postgres schema and rake is still pointing to the old one? I'm not sure then that a standard Rails migration is what you need. It might be best to create your own rake task instead.
Edit: If you're referencing two different databases or Postgres schemas, Rails doesn't support this in standard migrations. Rails assumes one database, so migrations from one database to another is usually not possible. When you run "rake db:migrate" it actually looks at the RAILS_ENV environment variable to find the correct entry in database.yml. If rake starts the migration looking at the "development" environment and database config from database.yml, it will expect to update to this environment at the end of the migration.
So, you'll probably need to do this from outside the Rails stack as you can't reference two databases at the same time within Rails. There are attempts at plugins to allow this, but they're majorly hacky and don't work properly.
You can use pg_power. It provides additional DSL for migration to create PostgreSQL schemas and not only.