How can I merge column from one table into another table and save it as a variable, what would be the most efficient way of accomplishing this. Here is an example:
Table 1:
ID First Name LastName contact_id
Table 2:
Id Phone Address email
Id from table 2 is a foreign key of contact_id
I need my third table to look like this:
Id Phone Address Email person_id
person_id corresponds to ID from table 1.
What would be the most efficient way using rails commands to accomplish this result so that it gives me the third table?
Why not just add person_id to table 2...
Table1Model.all.each do |model|
x=Table2Model.find(model.contact_id)
x.person_id = model.id
x.save
end
Related
Platform: Ruby on Rails with PostgreSQL database.
Problem:
We are doing some backfilling to migrate our data to a new structure. It's created a rather convoluted situation, and we'd like to handle it as efficiently as possible. It's partially addressed with SQL similar to this:
with rows as (
insert into responses (prompt_id, answer, received_at, user_id, category_id)
select prompt_id, null as answer, received_at, user_id, category_id
from prompts
where user_status = 0 and skipped is not true
returning id, category_id
)
insert into category_responses (category_id, response_id)
select category_id, id as response_id
from rows;
The tables and columns have been obfuscated/simplified so the reasoning behind it may not be as clear, but category_responses is a many-to-many join table. What we're doing is grabbing existing prompts, and creating a set of empty responses (answer is NULL) for each.
The piece that's missing is to then associate the records in prompts with the newly created responses. Is there a way to do this within the query? I would like to avoid adding a prompt_id column to answers if possible, but I am guessing this would be one way to handle that, including it in the returning clause, then issuing a second query to update the prompts table - and anyway I'm not even sure you can run more than one query with the results of a single with clause.
What's the best way to accomplish this?
I have settled on adding the needed column, and updated the query as follows:
with tab1 as (
insert into responses (prompt_id, answer, received_at, user_id, category_id, prompt_id)
select prompt_id, null as answer, received_at, user_id, category_id
from prompts
where user_status = 0 and skipped is not true
returning id, category_id, prompt_id
),
tab2 as (
update prompts
set response_id = tab1.response_id,
category_id = tab1.category_id
from tab1
where prompts.id = tab1.prompt_id
returning prompts.response_id as response_id, prompts.category_id as category_id
)
insert into category_responses (category_id, response_id)
select category_id, id as response_id
from tab2;
I think I'm pretty close on this one, but can't get it to click.
I've got two simple tables set up.
Table A:
CREATE TABLE customer(
id INTEGER PRIMARY KEY,
first_name TEXT,
last_name TEXT,
email TEXT UNIQUE NOT NULL,
password TEXT NOT NULL,
create_time TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP
);
I've got two rows of data populating correctly in Table A.
Table B:
CREATE TABLE address(
...> id INTEGER PRIMARY KEY,
...> street_address_1 TEXT NOT NULL,
...> street_address_2 TEXT,
...> street_address_3 TEXT,
...> city TEXT NOT NULL,
...> state TEXT NOT NULL,
...> zip TEXT NOT NULL);
And I've successfully imported a CSV file into that table.
I'm trying to create a 3rd table that joins Table A to Table B with the use of Foreign Keys.
I can create the table with the code below, but when I try to select the table, I'm getting a blank, which means I'm obviously doing something wrong. I'm expecting to see data where the two tables overlap on mutual Id numbers, i.e. where the ID from customer = Id from address I'd like to see the data from both tables for those rows appear in Table C.
Table C (the join table):
CREATE TABLE customer_address(
id INTEGER PRIMARY KEY,
customer_id INTEGER,
address_id INTEGER,
first_name TEXT NOT NULL,
last_name TEXT NOT NULL,
email TEXT NOT NULL,
password TEXT NOT NULL,
street_address_1 TEXT NOT NULL,
street_address_2 TEXT,
street_address_3 TEXT,
city TEXT NOT NULL,
state TEXT NOT NULL,
zip TEXT NOT NULL,
FOREIGN KEY (customer_id) REFERENCES customer(id),
FOREIGN KEY (address_id) REFERENCES address(id)
);
Thanks!
I imported the data to the address table using this:
sqlite> .mode csv
sqlite> .import address.csv address
I manually typed in data to the first table using this:
insert into customer(first_name, last_name, email, password)
values('Ad','Mac','a.Mac#gmail.com','Mab'),('Brian','Obrien','bob#example.com','123456');
Don't duplicate the data in your join table (often called a bridge table). This should do for Table C:
CREATE TABLE customer_address(
id INTEGER PRIMARY KEY,
customer_id INTEGER,
address_id INTEGER,
FOREIGN KEY (customer_id) REFERENCES customer(id),
FOREIGN KEY (address_id) REFERENCES address(id));
Duplicating columns is bad practice because it 1)defeats the purpose of using a relational model; 2)can lead to conflicting records if information is updated or deleted in one table, but not another.
Furthermore, you shouldn't have street_address_1, street_address_2, street_address_3 all in the same table. That's a violation of First Normal Form. Think of it this way, can a person have more than three addresses? Can they have two addresses in different cities? Do all three of those addresses have the same zip?
I want to create unique id for each Employee of company. For example currently I am getting like this.
Company has many employees and employee belongs_to company
company1= employee1
company1 = employee2
company1 = employee3
company2 = employee4
company2 = employee5
So, here employee id's are not getting changed while company_id is changed.
So, I want something like this:
For each company employee_id should start with 1.
like
Company1 = employee1
company1 = employee2
company1 = employee3
company2 = employee1
company2 = employee2
Could anyone please help me how to do that in Rails5?
employee table
id name position
1 aaa eg_position
2 vvv qqq
company table
id name
1 xxx
1 yyy
so present its storing like
xxx - 1, yyy-2
I want like this xxx-1.yyy-1
Use uniqueness validation based on scope of composite key. employee_id and company_id
validates :employee_id, uniqueness: {scope: [:employee_id, :company_id]}
We have to be very clear.
I am assuming that you have an employees table where all employees of all different companies are stored.
If that is the case, then you cannot have two primary keys which are the same. You cannot do this.
But what you can do is have another field, perhaps employee number (which is different to id) which can be the same across different employees in different companies.
Personally i don't like the idea of a composite primary key.
What I would do is create another column in your table called: employee number. And use that to store what you call: "id"
I hope this helps.
edit:
Further explanations.
Every row in a table has a column called "id". this is a special name. this is the name of the "primary key". Don't change that name. Two rows in the same table cannot have the same "id" or primary key. You can't do it. It's fundamental to a database. you can't have the same primary key in two rows of the same table - for mysql, psql, Access etc.
but you want to identify two employees with the same "identification number". That's fine. You can do it. Just don't call the employee identification number: "id" because that won't work. You can call it "employee_no" and that would work fine.
show my your database schema and we can further clarify.
hope this helps.
I have a table called shoppers and another table called Users. I have a shopper_id which is the foreign key in the Shoppers table and refers to the primary key id in the Users table.
I ran a query called #shoppers = shoppers.where("some condition")
This allowed me to get a set of shoppers who satisfy the condition. Next I would like to select those Users who have the same id as the shopper_id as the individual objects in #shoppers.
I know I could do this by writing a loop, but I am wondering if ruby on rails allows me to write a Users.where condition that can help me obtain the subset of user objects with the same id as shopper_id arranged in ascending order by the name field in the Users table.
Any ideas?
Try this.
#shoppers = Shopper.where("some condition")
#users = User.where(id: #shoppers.collect(&:shopper_id)).order('name asc')
I have a database I would like to convert to use UUID's as the primary key in postgresql.
I have roughly 30 tables with deep multi-level associations. Is there an 'easy' way to convert all current ID's to UUID?
From this: https://coderwall.com/p/n_0awq, I can see that I could alter the table in migration. I was thinking something like this:
for client in Client.all
# Retrieve children
underwritings = client.underwritings
# Change primary key
execute 'ALTER TABLE clients ALTER COLUMN id TYPE uuid;'
execute 'ALTER TABLE clients ALTER COLUMN id SET DEFAULT uuid_generate_v1();'
# Get new id - is this already generated?
client_id = client.id
for underwriting in underwritings
locations = underwriting.locations
other_record = underwriting.other_records...
execute 'ALTER TABLE underwritings ALTER COLUMN id TYPE uuid;'
execute 'ALTER TABLE underwritings ALTER COLUMN id SET DEFAULT uuid_generate_v1();'
underwriting.client_id = client_id
underwriting.saved
underwriting_id = underwriting.id
for location in locations
buildings = location.buildings
execute 'ALTER TABLE locations ALTER COLUMN id TYPE uuid;'
execute 'ALTER TABLE locations ALTER COLUMN id SET DEFAULT uuid_generate_v1();'
location.undewriting_id = underwriting_id
location.save
location_id = location.id
for building in buildings
...
end
end
for other_record in other_records
...
end
...
...
end
end
Questions:
Will this work?
Is there an easier way to do this?
Will child records be retrieved properly as long as they are retrieved before the primary key is changed?
Will the new primary key be already generated as soon as the alter table is called?
Thanks very much for any help or tips in doing this.
I found these to be quite tedious. It is possible to use direct queries to PostgreSQL to convert table with existing data.
For primary key:
ALTER TABLE students
ALTER COLUMN id DROP DEFAULT,
ALTER COLUMN id SET DATA TYPE UUID USING (uuid(lpad(replace(text(id),'-',''), 32, '0'))),
ALTER COLUMN id SET DEFAULT uuid_generate_v4()
For other references:
ALTER TABLE students
ALTER COLUMN city_id SET DATA TYPE UUID USING (uuid(lpad(replace(text(city_id),'-',''), 32, '0')))
The above left pads the integer value with zeros and converts to a UUID. This approach does not require id mapping and if needed old id could be retrieved.
As there is no data copying, this approach works quite fast.
To handle these and more complicated case of polymorphic associations please use https://github.com/kreatio-sw/webdack-uuid_migration. This gem adds additional helpers to ActiveRecord::Migration to ease these migrations.
I think trying to do something like this through Rails would just complicate matters. I'd ignore the Rails side of things completely and just do it in SQL.
Your first step is grab a complete backup of your database. Then restore that backup into another database to:
Make sure that your backup works.
Give you a realistic playpen where you can make mistakes without consequence.
First you'd want to clean up your data by adding real foreign keys to match all your Rails associations. There's a good chance that some of your FKs will fail, if they do you'll have to clean up your broken references.
Now that you have clean data, rename all your tables to make room for the new UUID versions. For a table t, we'll refer to the renamed table as t_tmp. For each t_tmp, create another table to hold the mapping from the old integer ids to the new UUID ids, something like this:
create table t_id_map (
old_id integer not null,
new_id uuid not null default uuid_generate_v1()
)
and then populate it:
insert into t_id_map (old_id)
select id from t_tmp
And you'll probably want to index t_id_map.old_id while you're here.
This gives us the old tables with integer ids and a lookup table for each t_tmp that maps the old id to the new one.
Now create the new tables with UUIDs replacing all the old integer and serial columns that held ids; I'd add real foreign keys at this point as well; you should be paranoid about your data: broken code is temporary, broken data is usually forever.
Populating the new tables is pretty easy at this point: simply use insert into ... select ... from constructs and JOIN to the appropriate t_id_map tables to map the old ids to the new ones. Once the data has been mapped and copied, you'll want to do some sanity checking to make sure everything still makes sense. Then you can drop your t_tmp and t_id_map tables and get on with your life.
Practice that process on a copy of your database, script it up, and away you go.
You would of course want to shut down any applications that access your database while you're doing this work.
Didn't want to add foreign keys, and wanted to to use a rails migration. Anyways, here is what I did if others are looking to do this (example for 2 tables, I did 32 total):
def change
execute 'CREATE EXTENSION "uuid-ossp";'
execute <<-SQL
ALTER TABLE buildings ADD COLUMN guid uuid DEFAULT uuid_generate_v1() NOT NULL;
ALTER TABLE buildings ALTER COLUMN guid SET DEFAULT uuid_generate_v1();
ALTER TABLE buildings ADD COLUMN location_guid uuid;
ALTER TABLE clients ADD COLUMN guid uuid DEFAULT uuid_generate_v1() NOT NULL;
ALTER TABLE clients ALTER COLUMN guid SET DEFAULT uuid_generate_v1();
ALTER TABLE clients ADD COLUMN agency_guid uuid;
ALTER TABLE clients ADD COLUMN account_executive_guid uuid;
ALTER TABLE clients ADD COLUMN account_representative_guid uuid;
SQL
for record in Building.all
location = record.location
record.location_guid = location.guid
record.save
end
for record in Client.all
agency = record.agency
record.agency_guid = agency.guid
account_executive = record.account_executive
record.account_executive_guid = account_executive.guid unless account_executive.blank?
account_representative = record.account_representative
record.account_representative_guid = account_representative.guid unless account_representative.blank?
record.save
end
execute <<-SQL
ALTER TABLE buildings DROP CONSTRAINT buildings_pkey;
ALTER TABLE buildings DROP COLUMN id;
ALTER TABLE buildings RENAME COLUMN guid TO id;
ALTER TABLE buildings ADD PRIMARY KEY (id);
ALTER TABLE buildings DROP COLUMN location_id;
ALTER TABLE buildings RENAME COLUMN location_guid TO location_id;
ALTER TABLE clients DROP CONSTRAINT clients_pkey;
ALTER TABLE clients DROP COLUMN id;
ALTER TABLE clients RENAME COLUMN guid TO id;
ALTER TABLE clients ADD PRIMARY KEY (id);
ALTER TABLE clients DROP COLUMN agency_id;
ALTER TABLE clients RENAME COLUMN agency_guid TO agency_id;
ALTER TABLE clients DROP COLUMN account_executive_id;
ALTER TABLE clients RENAME COLUMN account_executive_guid TO account_executive_id;
ALTER TABLE clients DROP COLUMN account_representative_id;
ALTER TABLE clients RENAME COLUMN account_representative_guid TO account_representative_id;
SQL
end