What is the fastest way to compare two values from different hashes? - ruby-on-rails

I am trying to compare two hashes to see if the values match so i can reconcile some data.
I have a hash of existing accounts and a hash of accounts i need to check if they exist based on the existing accounts to see if it needs to be inserted.
# Reconcile Adaccounts
def self.reconcileAdaccounts(ad_accounts, user_id)
# These are the accounts that exist
existing_accounts = FbAdaccount.active
.select("fb_id")
.where("user_id = ?", user_id)
.as_json
ad_accounts.each do |ad_account|
if #here i would need to check if ad_account["account_id"] matches one of the existing records
p "This one already exists"
p ad_account["account_id"]
else
p "I need to create this one"
p ad_account["account_id"]
end
end
end
I could have nested loops but that does not seem like it would be the best solution, i also tried looking for any rails API functions that could do this but i have not found one.
What is the most efficient way of doing this?

What you're looking for is find_or_create_by. I've additionally wrapped the call in a transaction so all of this hits the database just once β€”Β in a single query.
def self.reconcileAdaccounts(ad_accounts, user_id)
FbAdaccount.transaction do
ad_accounts.each do |ad_account|
FbAdaccount.active
.select('fb_id')
.where('user_id = ?', user_id)
.find_or_create_by(ad_account['account_id'])
end
end
end
Transactions
A transaction doesn't magically alter the query, but it wraps it together so that it either all happens together, or doesn't happen at all.
With the following code:
numbers = (3..7).to_a
Ear.transaction do
numbers.each do |number|
Ear.find_or_create_by(bla: number)
end
end
Note the BEGIN and COMMIT statements. They signify where the call starts and ends.
(0.1ms) BEGIN
Ear Load (0.5ms) SELECT "ears".* FROM "ears" WHERE "ears"."bla" = $1 LIMIT $2 [["bla", 3], ["LIMIT", 1]]
Ear Create (0.8ms) INSERT INTO "ears" ("bla", "created_at", "updated_at") VALUES ($1, $2, $3) RETURNING "id" [["bla", 3], ["created_at", "2019-02-05 23:17:28.157989"], ["updated_at", "2019-02-05 23:17:28.157989"]]
Ear Load (0.2ms) SELECT "ears".* FROM "ears" WHERE "ears"."bla" = $1 LIMIT $2 [["bla", 4], ["LIMIT", 1]]
Ear Create (0.2ms) INSERT INTO "ears" ("bla", "created_at", "updated_at") VALUES ($1, $2, $3) RETURNING "id" [["bla", 4], ["created_at", "2019-02-05 23:17:28.160838"], ["updated_at", "2019-02-05 23:17:28.160838"]]
Ear Load (0.1ms) SELECT "ears".* FROM "ears" WHERE "ears"."bla" = $1 LIMIT $2 [["bla", 5], ["LIMIT", 1]]
Ear Create (0.2ms) INSERT INTO "ears" ("bla", "created_at", "updated_at") VALUES ($1, $2, $3) RETURNING "id" [["bla", 5], ["created_at", "2019-02-05 23:17:28.162241"], ["updated_at", "2019-02-05 23:17:28.162241"]]
Ear Load (0.2ms) SELECT "ears".* FROM "ears" WHERE "ears"."bla" = $1 LIMIT $2 [["bla", 6], ["LIMIT", 1]]
Ear Create (0.2ms) INSERT INTO "ears" ("bla", "created_at", "updated_at") VALUES ($1, $2, $3) RETURNING "id" [["bla", 6], ["created_at", "2019-02-05 23:17:28.163794"], ["updated_at", "2019-02-05 23:17:28.163794"]]
Ear Load (0.2ms) SELECT "ears".* FROM "ears" WHERE "ears"."bla" = $1 LIMIT $2 [["bla", 7], ["LIMIT", 1]]
Ear Create (0.2ms) INSERT INTO "ears" ("bla", "created_at", "updated_at") VALUES ($1, $2, $3) RETURNING "id" [["bla", 7], ["created_at", "2019-02-05 23:17:28.165589"], ["updated_at", "2019-02-05 23:17:28.165589"]]
(40.9ms) COMMIT
Now here's the same code, just without a transaction to wrap it in:
numbers = (20..25).to_a
numbers.each do |number|
Ear.find_or_create_by(bla: number)
end
Note the total number BEGIN and COMMIT messages:
Ear Load (249.3ms) SELECT "ears".* FROM "ears" WHERE "ears"."bla" = $1 LIMIT $2 [["bla", 20], ["LIMIT", 1]]
(73.4ms) BEGIN
Ear Create (48.7ms) INSERT INTO "ears" ("bla", "created_at", "updated_at") VALUES ($1, $2, $3) RETURNING "id" [["bla", 20], ["created_at", "2019-02-06 00:12:35.754374"], ["updated_at", "2019-02-06 00:12:35.754374"]]
(79.8ms) COMMIT
Ear Load (0.4ms) SELECT "ears".* FROM "ears" WHERE "ears"."bla" = $1 LIMIT $2 [["bla", 21], ["LIMIT", 1]]
(0.2ms) BEGIN
Ear Create (77.7ms) INSERT INTO "ears" ("bla", "created_at", "updated_at") VALUES ($1, $2, $3) RETURNING "id" [["bla", 21], ["created_at", "2019-02-06 00:12:35.918461"], ["updated_at", "2019-02-06 00:12:35.918461"]]
(0.3ms) COMMIT
Ear Load (0.2ms) SELECT "ears".* FROM "ears" WHERE "ears"."bla" = $1 LIMIT $2 [["bla", 22], ["LIMIT", 1]]
(0.2ms) BEGIN
Ear Create (2.7ms) INSERT INTO "ears" ("bla", "created_at", "updated_at") VALUES ($1, $2, $3) RETURNING "id" [["bla", 22], ["created_at", "2019-02-06 00:12:36.003822"], ["updated_at", "2019-02-06 00:12:36.003822"]]
(110.1ms) COMMIT
Ear Load (0.3ms) SELECT "ears".* FROM "ears" WHERE "ears"."bla" = $1 LIMIT $2 [["bla", 23], ["LIMIT", 1]]
(0.2ms) BEGIN
Ear Create (0.3ms) INSERT INTO "ears" ("bla", "created_at", "updated_at") VALUES ($1, $2, $3) RETURNING "id" [["bla", 23], ["created_at", "2019-02-06 00:12:36.125187"], ["updated_at", "2019-02-06 00:12:36.125187"]]
(37.0ms) COMMIT
Ear Load (0.2ms) SELECT "ears".* FROM "ears" WHERE "ears"."bla" = $1 LIMIT $2 [["bla", 24], ["LIMIT", 1]]
(0.2ms) BEGIN
Ear Create (0.4ms) INSERT INTO "ears" ("bla", "created_at", "updated_at") VALUES ($1, $2, $3) RETURNING "id" [["bla", 24], ["created_at", "2019-02-06 00:12:36.168414"], ["updated_at", "2019-02-06 00:12:36.168414"]]
(72.4ms) COMMIT
Ear Load (0.3ms) SELECT "ears".* FROM "ears" WHERE "ears"."bla" = $1 LIMIT $2 [["bla", 25], ["LIMIT", 1]]
(1.7ms) BEGIN
Ear Create (0.4ms) INSERT INTO "ears" ("bla", "created_at", "updated_at") VALUES ($1, $2, $3) RETURNING "id" [["bla", 25], ["created_at", "2019-02-06 00:12:36.249285"], ["updated_at", "2019-02-06 00:12:36.249285"]]
(121.2ms) COMMIT

Related

Rails 6 pessimistic lock with jsonb

I have a dedicated service used to push on an jsonb column timestamps about some events.
Configuration:
pg: 1.2.2
PostgreSQL: 11.4
Rails: 6.0.2.1
Ruby: 2.6.5
Here is the schema of my table:
create_table "invitations", force: :cascade do |t|
t.uuid "uuid", default: -> { "gen_random_uuid()" }, null: false
t.datetime "created_at", precision: 6, null: false
t.datetime "updated_at", precision: 6, null: false
t.jsonb "notification_statuses", default: {}, null: false
end
Here is the typical jsonb I have for the column notification_statuses:
{emails: [{created_at: "some date", sent_at: "some date2"},
{created_at: "some other date", sent_at: "some other date2"}]}
Here is the code:
class MyNotificationManager
def initialize(invitation_id)
#invitation = Invitation.find(invitation_id)
end
def start_new_notification_status(type)
#invitation.reload.with_lock do
#invitation.notification_statuses[type] = [] unless #invitation.notification_statuses.key?(type)
hsh = { created_at: Time.zone.now, sent_at: nil }
#invitation.notification_statuses[type].push(hsh)
#invitation.save!
puts "push #{#invitation.notification_statuses[type].count}"
end
end
end
Here is the test:
describe '#start_new_notification_status' do
subject { MyNotificationManager }
describe "with concurrency" do
let(:concurrency_level) { 4 }
let!(:invitation) { create(:invitation) }
it "updates invitation one at the time" do
expect(ActiveRecord::Base.connection.pool.size).to eq(5)
fail_occurred = false
wait_for_it = true
threads = concurrency_level.times.map do |i|
Thread.new do
true while wait_for_it
begin
serv = subject.new(invitation.id)
serv.start_new_notification_status('emails')
rescue StandardError => err
fail_occurred = true
end
end
end
wait_for_it = false
threads.each(&:join)
expect(fail_occurred).not_to be_truthy
expect(invitation.reload.notification_statuses['emails'].count).to eq(concurrency_level)
end
end
end
Finally, my RSpec logs:
Event::NotificationManager
#start_new_notification_status
with concurrency
push 1
push 2
push 2
push 2
Failure/Error: expect(invitation.reload.notification_statuses['emails'].count).to eq(concurrency_level)
expected: 4
got: 2
What am I missing? Logs should obviously be:
Event::NotificationManager
#start_new_notification_status
with concurrency
push 1
push 2
push 3
push 4
Here are my logs.
(0.5ms) SELECT "ar_internal_metadata"."value" FROM "ar_internal_metadata" WHERE "ar_internal_metadata"."key" = $1 [["key", "schema_sha1"]]
(0.7ms) SELECT "schema_migrations"."version" FROM "schema_migrations" ORDER BY "schema_migrations"."version" ASC
(0.9ms) BEGIN
(0.3ms) SAVEPOINT active_record_1
Address Create (19.8ms) INSERT INTO "addresses" ("uuid", "line1", "line2", "locality", "zip_postcode", "country_province", "country", "latitude", "longitude", "created_at", "updated_at") VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11) RETURNING "id" [["uuid", "6ca512f7-aa9d-4deb-9fce-7ce3e878f5ee"], ["line1", "27862 Kautzer Dam"], ["line2", "Apt. 831"], ["locality", "West Gladys"], ["zip_postcode", "27812-2854"], ["country_province", "North Carolina"], ["country", "Saint Kitts and Nevis"], ["latitude", 6.990987893090221], ["longitude", -79.50793749705787], ["created_at", "2020-04-20 07:49:21.760116"], ["updated_at", "2020-04-20 07:49:21.760116"]]
(0.3ms) RELEASE SAVEPOINT active_record_1
(0.3ms) SAVEPOINT active_record_1
Establishment Create (14.0ms) INSERT INTO "establishments" ("name", "contact_name", "description", "note", "insta_user_name", "address_id", "created_at", "updated_at") VALUES ($1, $2, $3, $4, $5, $6, $7, $8) RETURNING "id" [["name", "Stehr-Donnelly"], ["contact_name", "Narcisa"], ["description", "Asperiores voluptates consequuntur."], ["note", "Dolore qui."], ["insta_user_name", "berta.king"], ["address_id", 1], ["created_at", "2020-04-20 07:49:21.792061"], ["updated_at", "2020-04-20 07:49:21.792061"]]
ActsAsTaggableOn::Tagging Load (0.6ms) SELECT "taggings".* FROM "taggings" WHERE "taggings"."taggable_id" = $1 AND "taggings"."taggable_type" = $2 [["taggable_id", 1], ["taggable_type", "Establishment"]]
(0.2ms) RELEASE SAVEPOINT active_record_1
(0.3ms) SAVEPOINT active_record_1
(1.0ms) SELECT COUNT(*) FROM "events" WHERE "events"."establishment_id" = $1 AND ((start_at <= '2020-04-22 16:05:59' AND end_at >= '2020-04-22 16:05:59' AND end_at <= '2020-04-22 18:05:59') OR
(end_at >= '2020-04-22 18:05:59' AND start_at <= '2020-04-22 18:05:59' AND start_at >= '2020-04-22 16:05:59') OR
(start_at >= '2020-04-22 16:05:59' AND end_at <= '2020-04-22 18:05:59') OR
(start_at <= '2020-04-22 16:05:59' AND end_at >= '2020-04-22 18:05:59')) [["establishment_id", 1]]
Event Create (17.1ms) INSERT INTO "events" ("uuid", "establishment_id", "name", "start_at", "end_at", "max_female_guest", "max_male_guest", "created_at", "updated_at", "published") VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10) RETURNING "id" [["uuid", "f110cba0-1f97-45ed-bc84-a8bebc73b8f8"], ["establishment_id", 1], ["name", "Clifford Kulas Conroy"], ["start_at", "2020-04-22 16:05:59"], ["end_at", "2020-04-22 18:05:59"], ["max_female_guest", 5], ["max_male_guest", 5], ["created_at", "2020-04-20 07:49:21.830331"], ["updated_at", "2020-04-20 07:49:21.830331"], ["published", true]]
Establishment Update All (0.6ms) UPDATE "establishments" SET "events_count" = COALESCE("events_count", 0) + $1 WHERE "establishments"."id" = $2 [["events_count", 1], ["id", 1]]
(0.2ms) RELEASE SAVEPOINT active_record_1
(0.4ms) SAVEPOINT active_record_1
Area Create (8.1ms) INSERT INTO "areas" ("uuid", "name", "slug", "created_at", "updated_at") VALUES ($1, $2, $3, $4, $5) RETURNING "id" [["uuid", "e13cec8d-4767-490f-a909-f07970006743"], ["name", "Derick Robel"], ["slug", "derick-robel"], ["created_at", "2020-04-20 07:49:21.970034"], ["updated_at", "2020-04-20 07:49:21.970034"]]
(0.3ms) RELEASE SAVEPOINT active_record_1
(0.3ms) SAVEPOINT active_record_1
User Exists? (0.9ms) SELECT 1 AS one FROM "users" WHERE "users"."email" = $1 LIMIT $2 [["email", "vernon#bergstrom.biz"], ["LIMIT", 1]]
User Create (18.8ms) INSERT INTO "users" ("uuid", "email", "encrypted_password", "confirmation_token", "confirmed_at", "confirmation_sent_at", "created_at", "updated_at") VALUES ($1, $2, $3, $4, $5, $6, $7, $8) RETURNING "id" [["uuid", "3bb5012a-ff5b-4dc8-b474-98aa06298c5c"], ["email", "vernon#bergstrom.biz"], ["encrypted_password", "$2a$04$dnmdv.wKESDVpFTn9UyqE.uFQwCexFGcmYBdg18HfGrvv0/4xrrQm"], ["confirmation_token", "bfa804e9-cd89-49ba-acb5-3afcfd5425f6"], ["confirmed_at", "2020-04-20 07:49:21.916321"], ["confirmation_sent_at", "2020-04-20 06:49:21.916422"], ["created_at", "2020-04-20 07:49:22.008360"], ["updated_at", "2020-04-20 07:49:22.008360"]]
AreasUser Create (4.8ms) INSERT INTO "areas_users" ("area_id", "user_id") VALUES ($1, $2) [["area_id", 1], ["user_id", 1]]
(0.2ms) RELEASE SAVEPOINT active_record_1
(0.2ms) SAVEPOINT active_record_1
Consumer Create (8.4ms) INSERT INTO "consumers" ("user_id", "first_name", "last_name", "whatsapp_user_name", "phone_number", "created_at", "updated_at") VALUES ($1, $2, $3, $4, $5, $6, $7) RETURNING "id" [["user_id", 1], ["first_name", "Kermit"], ["last_name", "Cremin"], ["whatsapp_user_name", "7a206657-8365-4075-b4e8-b58d421ac2e2"], ["phone_number", "+855 618.798.2641"], ["created_at", "2020-04-20 07:49:22.036784"], ["updated_at", "2020-04-20 07:49:22.036784"]]
(0.3ms) RELEASE SAVEPOINT active_record_1
(0.2ms) SAVEPOINT active_record_1
Guest Create (18.1ms) INSERT INTO "guests" ("uuid", "gender", "birthdate", "profession", "insta_user_name", "consumer_id", "created_at", "updated_at") VALUES ($1, $2, $3, $4, $5, $6, $7, $8) RETURNING "id" [["uuid", "47d43976-e9a9-49d4-a722-22c62821d3b4"], ["gender", 1], ["birthdate", "1998-09-11 00:00:00"], ["profession", "Global Sales Engineer"], ["insta_user_name", "jeannetta.harber"], ["consumer_id", 1], ["created_at", "2020-04-20 07:49:22.049803"], ["updated_at", "2020-04-20 07:49:22.049803"]]
(0.2ms) RELEASE SAVEPOINT active_record_1
(0.2ms) SAVEPOINT active_record_1
Invitation Create (12.8ms) INSERT INTO "invitations" ("guest_id", "event_id", "created_at", "updated_at") VALUES ($1, $2, $3, $4) RETURNING "id" [["guest_id", 1], ["event_id", 1], ["created_at", "2020-04-20 07:49:22.071668"], ["updated_at", "2020-04-20 07:49:22.071668"]]
Event Update All (0.5ms) UPDATE "events" SET "invitations_count" = COALESCE("invitations_count", 0) + $1 WHERE "events"."id" = $2 [["invitations_count", 1], ["id", 1]]
(0.2ms) RELEASE SAVEPOINT active_record_1
Invitation Load (0.5ms) SELECT "invitations".* FROM "invitations" WHERE "invitations"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Invitation Load (1.2ms) SELECT "invitations".* FROM "invitations" WHERE "invitations"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Invitation Load (0.3ms) SELECT "invitations".* FROM "invitations" WHERE "invitations"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
(3.3ms) SAVEPOINT active_record_1
Invitation Load (0.9ms) SELECT "invitations".* FROM "invitations" WHERE "invitations"."id" = $1 LIMIT $2 FOR UPDATE [["id", 1], ["LIMIT", 1]]
Guest Load (0.4ms) SELECT "guests".* FROM "guests" WHERE "guests"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Event Load (0.4ms) SELECT "events".* FROM "events" WHERE "events"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Invitation Update (0.7ms) UPDATE "invitations" SET "updated_at" = $1, "notification_statuses" = $2 WHERE "invitations"."id" = $3 [["updated_at", "2020-04-20 07:49:22.111406"], ["notification_statuses", "{\"emails\":[{\"created_at\":\"2020-04-20T07:49:22.105Z\"}]}"], ["id", 1]]
(0.2ms) RELEASE SAVEPOINT active_record_1
Invitation Load (0.4ms) SELECT "invitations".* FROM "invitations" WHERE "invitations"."id" = $1 LIMIT $2 FOR UPDATE [["id", 1], ["LIMIT", 1]]
Invitation Load (0.3ms) SELECT "invitations".* FROM "invitations" WHERE "invitations"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
(0.2ms) SAVEPOINT active_record_1
Invitation Load (1.1ms) SELECT "invitations".* FROM "invitations" WHERE "invitations"."id" = $1 LIMIT $2 FOR UPDATE [["id", 1], ["LIMIT", 1]]
Guest Load (0.9ms) SELECT "guests".* FROM "guests" WHERE "guests"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Event Load (0.3ms) SELECT "events".* FROM "events" WHERE "events"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Invitation Update (0.7ms) UPDATE "invitations" SET "updated_at" = $1, "notification_statuses" = $2 WHERE "invitations"."id" = $3 [["updated_at", "2020-04-20 07:49:22.129222"], ["notification_statuses", "{\"emails\":[{\"created_at\":\"2020-04-20T07:49:22.105Z\"},{\"created_at\":\"2020-04-20T07:49:22.124Z\"}]}"], ["id", 1]]
(0.2ms) RELEASE SAVEPOINT active_record_1
Guest Load (0.3ms) SELECT "guests".* FROM "guests" WHERE "guests"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Invitation Load (1.1ms) SELECT "invitations".* FROM "invitations" WHERE "invitations"."id" = $1 LIMIT $2 FOR UPDATE [["id", 1], ["LIMIT", 1]]
(2.1ms) SAVEPOINT active_record_1
Invitation Update (2.7ms) UPDATE "invitations" SET "updated_at" = $1, "notification_statuses" = $2 WHERE "invitations"."id" = $3 [["updated_at", "2020-04-20 07:49:22.136369"], ["notification_statuses", "{\"emails\":[{\"created_at\":\"2020-04-20T07:49:22.105Z\"},{\"created_at\":\"2020-04-20T07:49:22.124Z\"},{\"created_at\":\"2020-04-20T07:49:22.135Z\"}]}"], ["id", 1]]
(0.5ms) RELEASE SAVEPOINT active_record_1
Invitation Load (4.1ms) SELECT "invitations".* FROM "invitations" WHERE "invitations"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
(1.0ms) ROLLBACK
(0.3ms) BEGIN
(3.4ms) ALTER TABLE "tags" DISABLE TRIGGER ALL;ALTER TABLE "schema_migrations" DISABLE TRIGGER ALL;ALTER TABLE "ar_internal_metadata" DISABLE TRIGGER ALL;ALTER TABLE "areas_establishments" DISABLE TRIGGER ALL;ALTER TABLE "consumer_ratings" DISABLE TRIGGER ALL;ALTER TABLE "addresses" DISABLE TRIGGER ALL;ALTER TABLE "oauth_access_tokens" DISABLE TRIGGER ALL;ALTER TABLE "guests" DISABLE TRIGGER ALL;ALTER TABLE "areas_users" DISABLE TRIGGER ALL;ALTER TABLE "active_storage_blobs" DISABLE TRIGGER ALL;ALTER TABLE "areas" DISABLE TRIGGER ALL;ALTER TABLE "invitations" DISABLE TRIGGER ALL;ALTER TABLE "establishments_managers" DISABLE TRIGGER ALL;ALTER TABLE "consumers" DISABLE TRIGGER ALL;ALTER TABLE "events" DISABLE TRIGGER ALL;ALTER TABLE "taggings" DISABLE TRIGGER ALL;ALTER TABLE "users" DISABLE TRIGGER ALL;ALTER TABLE "oauth_applications" DISABLE TRIGGER ALL;ALTER TABLE "establishments" DISABLE TRIGGER ALL;ALTER TABLE "active_storage_attachments" DISABLE TRIGGER ALL;ALTER TABLE "oauth_access_grants" DISABLE TRIGGER ALL;ALTER TABLE "managers" DISABLE TRIGGER ALL;ALTER TABLE "devices" DISABLE TRIGGER ALL;ALTER TABLE "admins" DISABLE TRIGGER ALL
(5.0ms) COMMIT
(2.3ms) SELECT schemaname || '.' || tablename
FROM pg_tables
WHERE
tablename !~ '_prt_' AND
tablename <> 'schema_migrations' AND tablename <> 'ar_internal_metadata' AND
schemaname = ANY (current_schemas(false))
(1.7ms) select table_name from information_schema.views where table_schema = 'secret_core_test'
(1425.7ms) TRUNCATE TABLE "public"."tags", "public"."areas_establishments", "public"."consumer_ratings", "public"."addresses", "public"."oauth_access_tokens", "public"."guests", "public"."areas_users", "public"."active_storage_blobs", "public"."areas", "public"."invitations", "public"."establishments_managers", "public"."consumers", "public"."events", "public"."taggings", "public"."users", "public"."oauth_applications", "public"."establishments", "public"."active_storage_attachments", "public"."oauth_access_grants", "public"."managers", "public"."devices", "public"."admins" RESTART IDENTITY CASCADE;
(0.5ms) BEGIN
(2.4ms) ALTER TABLE "tags" ENABLE TRIGGER ALL;ALTER TABLE "schema_migrations" ENABLE TRIGGER ALL;ALTER TABLE "ar_internal_metadata" ENABLE TRIGGER ALL;ALTER TABLE "areas_establishments" ENABLE TRIGGER ALL;ALTER TABLE "consumer_ratings" ENABLE TRIGGER ALL;ALTER TABLE "addresses" ENABLE TRIGGER ALL;ALTER TABLE "oauth_access_tokens" ENABLE TRIGGER ALL;ALTER TABLE "guests" ENABLE TRIGGER ALL;ALTER TABLE "areas_users" ENABLE TRIGGER ALL;ALTER TABLE "active_storage_blobs" ENABLE TRIGGER ALL;ALTER TABLE "areas" ENABLE TRIGGER ALL;ALTER TABLE "invitations" ENABLE TRIGGER ALL;ALTER TABLE "establishments_managers" ENABLE TRIGGER ALL;ALTER TABLE "consumers" ENABLE TRIGGER ALL;ALTER TABLE "events" ENABLE TRIGGER ALL;ALTER TABLE "taggings" ENABLE TRIGGER ALL;ALTER TABLE "users" ENABLE TRIGGER ALL;ALTER TABLE "oauth_applications" ENABLE TRIGGER ALL;ALTER TABLE "establishments" ENABLE TRIGGER ALL;ALTER TABLE "active_storage_attachments" ENABLE TRIGGER ALL;ALTER TABLE "oauth_access_grants" ENABLE TRIGGER ALL;ALTER TABLE "managers" ENABLE TRIGGER ALL;ALTER TABLE "devices" ENABLE TRIGGER ALL;ALTER TABLE "admins" ENABLE TRIGGER ALL
(1.8ms) COMMIT
EDIT: It seems that not all SQL updates are generated. I've the exact same number of not stored data on my array than missing SQL updates. Even if #invitation.save! always returns true (I've checked that right now).
**EDIT2: ** Sometime I have the error <RuntimeError: Locking a record with unpersisted changes is not supported. Use saveto persist the changes, orreload to discard them explicitly. But I have a .reload right before .with_lock!. How is this error possible? πŸ€”
I have finally fixed it. Problem came from my manager. Here is the fix:
class MyNotificationManager
def initialize(invitation_id)
#invitation = Invitation.find(invitation_id)
end
def start_new_notification_status(type)
Invitation.transaction do
inv = Invitation.lock.find(#invitation.id)
inv.notification_statuses =
init_type_key(#invitation.notification_statuses, type)
inv.notification_statuses[type] =
add_creation_hash(#invitation.notification_statuses[type], details: details)
inv.save!
end
end
end
If I don't lock it before load (pure SQL lock), it does not really lock it. I'm not 100% sure this is a normal behavior. It sound like a bug to me.
P.S: I don't want #invitation to be locked on init so I had to reload it manually.
P.S2: .reload does not work.

generate consecutive unique number in range

I have a field in my database to save a unique number adding the last two digits of the created_at year. At the moment it all works well, except instead of using the SecureRandom method i'm looking for an option to save consecutive numbers. ie. 1901, 1902, 1903...
Model
before_create :create_number
def create_number
loop do
self. number = ((created_at + 1.year).strftime("%y")).concat(sprintf '%02d', SecureRandom.random_number(200))
break unless self.class.exists?(:number => number)
end
end
The key is to use a count query fetch the count of records for that specific year:
class Thing < ApplicationRecord
before_commit :create_number!, if: -> { number.nil? }
private
def count_records_from_same_year
self.class.where(
created_at: (created_at.beginning_of_year..created_at.end_of_year)
).count
end
def create_number!
loop do
year = (created_at + 1.year).strftime("%y")
self.number = year.concat(sprintf '%02d', count_records_from_same_year)
break unless self.class.where(number: self.number).exists?
end
end
end
As you can see it generates consecutive numbers:
irb(main):001:0> Thing.create
(0.3ms) BEGIN
Thing Create (1.3ms) INSERT INTO "things" ("created_at", "updated_at") VALUES ($1, $2) RETURNING "id" [["created_at", "2018-10-12 09:57:24.527527"], ["updated_at", "2018-10-12 09:57:24.527527"]]
(1.2ms) SELECT COUNT(*) FROM "things" WHERE "things"."created_at" BETWEEN $1 AND $2 [["created_at", "2018-01-01 00:00:00"], ["created_at", "2018-12-31 23:59:59.999999"]]
Thing Exists (1.4ms) SELECT 1 AS one FROM "things" WHERE "things"."number" = $1 LIMIT $2 [["number", "1901"], ["LIMIT", 1]]
(0.7ms) COMMIT
=> #<Thing id: 1, number: "1901", created_at: "2018-10-12 09:57:24", updated_at: "2018-10-12 09:57:24">
irb(main):002:0> Thing.create
(0.3ms) BEGIN
Thing Create (0.8ms) INSERT INTO "things" ("created_at", "updated_at") VALUES ($1, $2) RETURNING "id" [["created_at", "2018-10-12 09:57:26.402797"], ["updated_at", "2018-10-12 09:57:26.402797"]]
(0.7ms) SELECT COUNT(*) FROM "things" WHERE "things"."created_at" BETWEEN $1 AND $2 [["created_at", "2018-01-01 00:00:00"], ["created_at", "2018-12-31 23:59:59.999999"]]
Thing Exists (0.8ms) SELECT 1 AS one FROM "things" WHERE "things"."number" = $1 LIMIT $2 [["number", "1902"], ["LIMIT", 1]]
(0.7ms) COMMIT
=> #<Thing id: 2, number: "1902", created_at: "2018-10-12 09:57:26", updated_at: "2018-10-12 09:57:26">
irb(main):003:0> Thing.create
(0.5ms) BEGIN
Thing Create (0.7ms) INSERT INTO "things" ("created_at", "updated_at") VALUES ($1, $2) RETURNING "id" [["created_at", "2018-10-12 09:57:27.537635"], ["updated_at", "2018-10-12 09:57:27.537635"]]
(1.6ms) SELECT COUNT(*) FROM "things" WHERE "things"."created_at" BETWEEN $1 AND $2 [["created_at", "2018-01-01 00:00:00"], ["created_at", "2018-12-31 23:59:59.999999"]]
Thing Exists (0.6ms) SELECT 1 AS one FROM "things" WHERE "things"."number" = $1 LIMIT $2 [["number", "1903"], ["LIMIT", 1]]
(0.7ms) COMMIT
=> #<Thing id: 3, number: "1903", created_at: "2018-10-12 09:57:27", updated_at: "2018-10-12 09:57:27">

Upgraded from SQLite3 to PG - db is 100 times slower

Just streamlined my dev environment and switched from SQLite3 to PG so it's the same on my production environment.
The same working code that I had before is now taking much, MUCH more time than it did before.
Haven't changed anything except installing pg.
Before PG:
Processing by CollectionsController#show as HTML
Parameters: {"keyword"=>"cat and dog towel", "id"=>"37"}
Collection Load (0.2ms) SELECT "collections".* FROM "collections" WHERE "collections"."id" = ? LIMIT ? [["id", 37], ["LIMIT", 1]]
Seller Load (0.2ms) SELECT "sellers".* FROM "sellers" WHERE "sellers"."id" = ? LIMIT ? [["id", 13], ["LIMIT", 1]]
User Load (0.4ms) SELECT "users".* FROM "users" WHERE "users"."id" = ? ORDER BY "users"."id" ASC LIMIT ? [["id", 1], ["LIMIT", 1]]
(0.1ms) begin transaction
(0.1ms) commit transaction
Search Load (0.2ms) SELECT "searches".* FROM "searches" WHERE "searches"."term" = ? LIMIT ? [["term", "cat and dog towel"], ["LIMIT", 1]]
Listing Load (1.4ms) SELECT DISTINCT "listings".* FROM "listings" INNER JOIN "listings_searches" ON "listings"."id" = "listings_searches"."listing_id" WHERE "listings_searches"."search_id" = ? ORDER BY "listings"."id" ASC LIMIT ? [["search_id", 775], ["LIMIT", 1]]
Search Load (0.3ms) SELECT "searches".* FROM "searches" WHERE "searches"."term" = ? ORDER BY "searches"."id" ASC LIMIT ? [["term", "Beach Towel"], ["LIMIT", 1]]
(0.1ms) begin transaction
Search Exists (0.2ms) SELECT 1 AS one FROM "searches" WHERE "searches"."term" = ? LIMIT ? [["term", "Beach Towel"], ["LIMIT", 1]]
SQL (0.5ms) INSERT INTO "searches" ("term", "created_at", "updated_at") VALUES (?, ?, ?) [["term", "Beach Towel"], ["created_at", "2018-06-18 12:47:58.519223"], ["updated_at", "2018-06-18 12:47:58.519223"]]
(7.0ms) commit transaction
Item Exists (0.3ms) SELECT 1 AS one FROM "items" WHERE "items"."search_id" = ? AND "items"."collection_id" = 37 LIMIT ? [["search_id", 776], ["LIMIT", 1]]
(0.1ms) begin transaction
Collection Load (0.1ms) SELECT "collections".* FROM "collections" WHERE "collections"."id" = ? LIMIT ? [["id", 37], ["LIMIT", 1]]
Item Exists (0.2ms) SELECT 1 AS one FROM "items" WHERE "items"."collection_id" = ? AND "items"."search_id" = 776 LIMIT ? [["collection_id", 37], ["LIMIT", 1]]
SQL (2.3ms) INSERT INTO "items" ("collection_id", "search_id", "created_at", "updated_at") VALUES (?, ?, ?, ?) [["collection_id", 37], ["search_id", 776], ["created_at", "2018-06-18 12:47:58.540021"], ["updated_at", "2018-06-18 12:47:58.540021"]]
(6.2ms) commit transaction
Started GET "/sellers/16" for 127.0.0.1 at 2018-06-18 15:48:00 +0300
Processing by SellersController#show as HTML
Parameters: {"id"=>"16"}
Search Load (5.5ms) SELECT "searches".* FROM "searches" WHERE "searches"."term" = ? ORDER BY "searches"."id" ASC LIMIT ? [["term", "Bath Towel"], ["LIMIT", 1]]
(0.1ms) begin transaction
Search Exists (0.2ms) SELECT 1 AS one FROM "searches" WHERE "searches"."term" = ? LIMIT ? [["term", "Bath Towel"], ["LIMIT", 1]]
SQL (1.4ms) INSERT INTO "searches" ("term", "created_at", "updated_at") VALUES (?, ?, ?) [["term", "Bath Towel"], ["created_at", "2018-06-18 12:48:01.250699"], ["updated_at", "2018-06-18 12:48:01.250699"]]
Seller Load (0.3ms) SELECT "sellers".* FROM "sellers" WHERE "sellers"."id" = ? LIMIT ? [["id", 16], ["LIMIT", 1]]
(9.6ms) commit transaction
User Load (0.4ms) SELECT "users".* FROM "users" WHERE "users"."id" = ? ORDER BY "users"."id" ASC LIMIT ? [["id", 1], ["LIMIT", 1]]
Item Exists (0.3ms) SELECT 1 AS one FROM "items" WHERE "items"."search_id" = ? AND "items"."collection_id" = 37 LIMIT ? [["search_id", 777], ["LIMIT", 1]]
(0.1ms) begin transaction
Collection Load (0.3ms) SELECT "collections".* FROM "collections" WHERE "collections"."id" = ? LIMIT ? [["id", 37], ["LIMIT", 1]]
Item Exists (0.2ms) SELECT 1 AS one FROM "items" WHERE "items"."collection_id" = ? AND "items"."search_id" = 777 LIMIT ? [["collection_id", 37], ["LIMIT", 1]]
SQL (1.0ms) INSERT INTO "items" ("collection_id", "search_id", "created_at", "updated_at") VALUES (?, ?, ?, ?) [["collection_id", 37], ["search_id", 777], ["created_at", "2018-06-18 12:48:01.285942"], ["updated_at", "2018-06-18 12:48:01.285942"]]
Shop Load (8.8ms) SELECT "shops".* FROM "shops" WHERE "shops"."seller_id" = ? ORDER BY "shops"."id" DESC LIMIT ? [["seller_id", 16], ["LIMIT", 1]]
(40.2ms) commit transaction
Rendering sellers/show.html.erb within layouts/application
(0.4ms) SELECT COUNT(*) FROM "listings" WHERE "listings"."shop_id" = 12
Listing Load (0.4ms) SELECT "listings".* FROM "listings" WHERE "listings"."shop_id" = 12 LIMIT ? OFFSET ? [["LIMIT", 24], ["OFFSET", 0]]
Rendered sellers/show.html.erb within layouts/application (4.8ms)
Rendered layouts/_shim.html.erb (0.4ms)
Rendered layouts/_rails_default.html.erb (93.9ms)
Rendered layouts/_meta.html.erb (0.3ms)
Seller Exists (0.3ms) SELECT 1 AS one FROM "sellers" WHERE "sellers"."user_id" = ? LIMIT ? [["user_id", 1], ["LIMIT", 1]]
Seller Load (0.3ms) SELECT "sellers".* FROM "sellers" WHERE "sellers"."user_id" = ? ORDER BY "sellers"."id" DESC LIMIT ? [["user_id", 1], ["LIMIT", 1]]
Rendered layouts/_header.html.erb (5.5ms)
Seller Load (0.3ms) SELECT "sellers".* FROM "sellers" WHERE "sellers"."user_id" = ? [["user_id", 1]]
Shop Load (0.3ms) SELECT "shops".* FROM "shops" WHERE "shops"."seller_id" = ? ORDER BY "shops"."id" DESC LIMIT ? [["seller_id", 13], ["LIMIT", 1]]
CACHE Shop Load (0.0ms) SELECT "shops".* FROM "shops" WHERE "shops"."seller_id" = ? ORDER BY "shops"."id" DESC LIMIT ? [["seller_id", 16], ["LIMIT", 1]]
Rendered layouts/_sidebar.html.erb (7.3ms)
Rendered layouts/_end_sidebar.html.erb (0.4ms)
Rendered layouts/_footer.html.erb (0.4ms)
Completed 200 OK in 212ms (Views: 120.5ms | ActiveRecord: 12.3ms)
After PG:
Started POST "/collections/1/item/choose.25" for 127.0.0.1 at 2018-06-18 16:56:12 +0300
Processing by ItemsController#choose as
Parameters: {"authenticity_token"=>"11+Rpf2qXEKhQ5vBJLWie2EBg1b2Dtrw8iIgoLwXyvake7+myDrVErcqTwNcPYAZ5xs+zFKXaJjwM7fTakXaoA==", "collection_id"=>"1"}
Collection Load (0.4ms) SELECT "collections".* FROM "collections" WHERE "collections"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Seller Load (0.6ms) SELECT "sellers".* FROM "sellers" WHERE "sellers"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Item Load (0.3ms) SELECT "items".* FROM "items" WHERE "items"."id" = $1 LIMIT $2 [["id", 25], ["LIMIT", 1]]
(0.1ms) BEGIN
Collection Load (0.5ms) SELECT "collections".* FROM "collections" WHERE "collections"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Search Load (0.9ms) SELECT "searches".* FROM "searches" WHERE "searches"."id" = $1 LIMIT $2 [["id", 31], ["LIMIT", 1]]
Item Exists (0.7ms) SELECT 1 AS one FROM "items" WHERE "items"."collection_id" = $1 AND ("items"."id" != $2) AND "items"."search_id" = 31 LIMIT $3 [["collection_id", 1], ["id", 25], ["LIMIT", 1]]
SQL (0.8ms) UPDATE "items" SET "chosen" = $1, "updated_at" = $2 WHERE "items"."id" = $3 [["chosen", "t"], ["updated_at", "2018-06-18 13:56:12.465841"], ["id", 25]]
(2.7ms) COMMIT
(0.4ms) BEGIN
SQL (0.6ms) UPDATE "collections" SET "keyword" = $1, "updated_at" = $2 WHERE "collections"."id" = $3 [["keyword", "Bookish Candles"], ["updated_at", "2018-06-18 13:56:12.473173"], ["id", 1]]
(1.5ms) COMMIT
Search Load (0.6ms) SELECT "searches".* FROM "searches" WHERE "searches"."term" = $1 LIMIT $2 [["term", "Bookish Candles"], ["LIMIT", 1]]
Listing Load (4.1ms) SELECT DISTINCT "listings".* FROM "listings" INNER JOIN "listings_searches" ON "listings"."id" = "listings_searches"."listing_id" WHERE "listings_searches"."search_id" = $1 ORDER BY "listings"."id" ASC LIMIT $2 [["search_id", 31], ["LIMIT", 3]]
Search Load (0.7ms) SELECT "searches".* FROM "searches" WHERE "searches"."term" = $1 ORDER BY "searches"."id" ASC LIMIT $2 [["term", "soy candles"], ["LIMIT", 1]]
(0.2ms) BEGIN
Search Exists (0.8ms) SELECT 1 AS one FROM "searches" WHERE "searches"."term" = $1 LIMIT $2 [["term", "soy candles"], ["LIMIT", 1]]
SQL (0.6ms) INSERT INTO "searches" ("term", "created_at", "updated_at") VALUES ($1, $2, $3) RETURNING "id" [["term", "soy candles"], ["created_at", "2018-06-18 13:56:12.491262"], ["updated_at", "2018-06-18 13:56:12.491262"]]
(2.3ms) COMMIT
Item Exists (0.6ms) SELECT 1 AS one FROM "items" WHERE "items"."search_id" = $1 AND "items"."collection_id" = 1 LIMIT $2 [["search_id", 46], ["LIMIT", 1]]
(0.2ms) BEGIN
Collection Load (1.3ms) SELECT "collections".* FROM "collections" WHERE "collections"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Item Exists (0.7ms) SELECT 1 AS one FROM "items" WHERE "items"."collection_id" = $1 AND "items"."search_id" = 46 LIMIT $2 [["collection_id", 1], ["LIMIT", 1]]
SQL (1.0ms) INSERT INTO "items" ("collection_id", "search_id", "created_at", "updated_at") VALUES ($1, $2, $3, $4) RETURNING "id" [["collection_id", 1], ["search_id", 46], ["created_at", "2018-06-18 13:56:12.504381"], ["updated_at", "2018-06-18 13:56:12.504381"]]
(1.1ms) COMMIT
Search Load (0.8ms) SELECT "searches".* FROM "searches" WHERE "searches"."term" = $1 ORDER BY "searches"."id" ASC LIMIT $2 [["term", "book candle"], ["LIMIT", 1]]
(0.2ms) BEGIN
Search Exists (0.5ms) SELECT 1 AS one FROM "searches" WHERE "searches"."term" = $1 LIMIT $2 [["term", "book candle"], ["LIMIT", 1]]
SQL (0.5ms) INSERT INTO "searches" ("term", "created_at", "updated_at") VALUES ($1, $2, $3) RETURNING "id" [["term", "book candle"], ["created_at", "2018-06-18 13:56:14.812727"], ["updated_at", "2018-06-18 13:56:14.812727"]]
(2.8ms) COMMIT
Item Exists (0.7ms) SELECT 1 AS one FROM "items" WHERE "items"."search_id" = $1 AND "items"."collection_id" = 1 LIMIT $2 [["search_id", 47], ["LIMIT", 1]]
(0.1ms) BEGIN
Collection Load (1.0ms) SELECT "collections".* FROM "collections" WHERE "collections"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Item Exists (0.6ms) SELECT 1 AS one FROM "items" WHERE "items"."collection_id" = $1 AND "items"."search_id" = 47 LIMIT $2 [["collection_id", 1], ["LIMIT", 1]]
SQL (0.5ms) INSERT INTO "items" ("collection_id", "search_id", "created_at", "updated_at") VALUES ($1, $2, $3, $4) RETURNING "id" [["collection_id", 1], ["search_id", 47], ["created_at", "2018-06-18 13:56:14.825865"], ["updated_at", "2018-06-18 13:56:14.825865"]]
(1.4ms) COMMIT
Search Load (1.0ms) SELECT "searches".* FROM "searches" WHERE "searches"."term" = $1 ORDER BY "searches"."id" ASC LIMIT $2 [["term", "literary gifts"], ["LIMIT", 1]]
(0.4ms) BEGIN
Search Exists (0.7ms) SELECT 1 AS one FROM "searches" WHERE "searches"."term" = $1 LIMIT $2 [["term", "literary gifts"], ["LIMIT", 1]]
SQL (0.7ms) INSERT INTO "searches" ("term", "created_at", "updated_at") VALUES ($1, $2, $3) RETURNING "id" [["term", "literary gifts"], ["created_at", "2018-06-18 13:56:17.776368"], ["updated_at", "2018-06-18 13:56:17.776368"]]
(2.4ms) COMMIT
Item Exists (0.8ms) SELECT 1 AS one FROM "items" WHERE "items"."search_id" = $1 AND "items"."collection_id" = 1 LIMIT $2 [["search_id", 48], ["LIMIT", 1]]
(0.2ms) BEGIN
Collection Load (1.6ms) SELECT "collections".* FROM "collections" WHERE "collections"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Item Exists (0.8ms) SELECT 1 AS one FROM "items" WHERE "items"."collection_id" = $1 AND "items"."search_id" = 48 LIMIT $2 [["collection_id", 1], ["LIMIT", 1]]
SQL (1.1ms) INSERT INTO "items" ("collection_id", "search_id", "created_at", "updated_at") VALUES ($1, $2, $3, $4) RETURNING "id" [["collection_id", 1], ["search_id", 48], ["created_at", "2018-06-18 13:56:17.790424"], ["updated_at", "2018-06-18 13:56:17.790424"]]
(1.2ms) COMMIT
Search Load (0.7ms) SELECT "searches".* FROM "searches" WHERE "searches"."term" = $1 ORDER BY "searches"."id" ASC LIMIT $2 [["term", "book candles"], ["LIMIT", 1]]
(0.2ms) BEGIN
Search Exists (0.7ms) SELECT 1 AS one FROM "searches" WHERE "searches"."term" = $1 LIMIT $2 [["term", "book candles"], ["LIMIT", 1]]
SQL (0.6ms) INSERT INTO "searches" ("term", "created_at", "updated_at") VALUES ($1, $2, $3) RETURNING "id" [["term", "book candles"], ["created_at", "2018-06-18 13:56:20.872076"], ["updated_at", "2018-06-18 13:56:20.872076"]]
(1.4ms) COMMIT
Item Exists (0.6ms) SELECT 1 AS one FROM "items" WHERE "items"."search_id" = $1 AND "items"."collection_id" = 1 LIMIT $2 [["search_id", 49], ["LIMIT", 1]]
(0.1ms) BEGIN
Collection Load (0.6ms) SELECT "collections".* FROM "collections" WHERE "collections"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Item Exists (0.7ms) SELECT 1 AS one FROM "items" WHERE "items"."collection_id" = $1 AND "items"."search_id" = 49 LIMIT $2 [["collection_id", 1], ["LIMIT", 1]]
SQL (0.8ms) INSERT INTO "items" ("collection_id", "search_id", "created_at", "updated_at") VALUES ($1, $2, $3, $4) RETURNING "id" [["collection_id", 1], ["search_id", 49], ["created_at", "2018-06-18 13:56:20.883450"], ["updated_at", "2018-06-18 13:56:20.883450"]]
(2.3ms) COMMIT
Search Load (0.9ms) SELECT "searches".* FROM "searches" WHERE "searches"."term" = $1 ORDER BY "searches"."id" ASC LIMIT $2 [["term", "literary candles"], ["LIMIT", 1]]
(0.2ms) BEGIN
Search Exists (0.5ms) SELECT 1 AS one FROM "searches" WHERE "searches"."term" = $1 LIMIT $2 [["term", "literary candles"], ["LIMIT", 1]]
SQL (0.8ms) INSERT INTO "searches" ("term", "created_at", "updated_at") VALUES ($1, $2, $3) RETURNING "id" [["term", "literary candles"], ["created_at", "2018-06-18 13:56:22.523896"], ["updated_at", "2018-06-18 13:56:22.523896"]]
(1.4ms) COMMIT
Item Exists (0.7ms) SELECT 1 AS one FROM "items" WHERE "items"."search_id" = $1 AND "items"."collection_id" = 1 LIMIT $2 [["search_id", 50], ["LIMIT", 1]]
(0.2ms) BEGIN
Collection Load (0.7ms) SELECT "collections".* FROM "collections" WHERE "collections"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Item Exists (0.9ms) SELECT 1 AS one FROM "items" WHERE "items"."collection_id" = $1 AND "items"."search_id" = 50 LIMIT $2 [["collection_id", 1], ["LIMIT", 1]]
SQL (0.9ms) INSERT INTO "items" ("collection_id", "search_id", "created_at", "updated_at") VALUES ($1, $2, $3, $4) RETURNING "id" [["collection_id", 1], ["search_id", 50], ["created_at", "2018-06-18 13:56:22.536553"], ["updated_at", "2018-06-18 13:56:22.536553"]]
(2.2ms) COMMIT
Search Load (0.9ms) SELECT "searches".* FROM "searches" WHERE "searches"."term" = $1 ORDER BY "searches"."id" ASC LIMIT $2 [["term", "soy candle"], ["LIMIT", 1]]
Item Exists (0.7ms) SELECT 1 AS one FROM "items" WHERE "items"."search_id" = $1 AND "items"."collection_id" = 1 LIMIT $2 [["search_id", 18], ["LIMIT", 1]]
Search Load (0.6ms) SELECT "searches".* FROM "searches" WHERE "searches"."term" = $1 ORDER BY "searches"."id" ASC LIMIT $2 [["term", "8oz candles"], ["LIMIT", 1]]
(0.3ms) BEGIN
Search Exists (0.6ms) SELECT 1 AS one FROM "searches" WHERE "searches"."term" = $1 LIMIT $2 [["term", "8oz candles"], ["LIMIT", 1]]
SQL (0.5ms) INSERT INTO "searches" ("term", "created_at", "updated_at") VALUES ($1, $2, $3) RETURNING "id" [["term", "8oz candles"], ["created_at", "2018-06-18 13:56:25.455733"], ["updated_at", "2018-06-18 13:56:25.455733"]]
(2.4ms) COMMIT
Item Exists (0.8ms) SELECT 1 AS one FROM "items" WHERE "items"."search_id" = $1 AND "items"."collection_id" = 1 LIMIT $2 [["search_id", 51], ["LIMIT", 1]]
(0.2ms) BEGIN
Collection Load (0.7ms) SELECT "collections".* FROM "collections" WHERE "collections"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Item Exists (0.6ms) SELECT 1 AS one FROM "items" WHERE "items"."collection_id" = $1 AND "items"."search_id" = 51 LIMIT $2 [["collection_id", 1], ["LIMIT", 1]]
SQL (1.0ms) INSERT INTO "items" ("collection_id", "search_id", "created_at", "updated_at") VALUES ($1, $2, $3, $4) RETURNING "id" [["collection_id", 1], ["search_id", 51], ["created_at", "2018-06-18 13:56:25.468089"], ["updated_at", "2018-06-18 13:56:25.468089"]]
(1.2ms) COMMIT
Search Load (0.7ms) SELECT "searches".* FROM "searches" WHERE "searches"."term" = $1 ORDER BY "searches"."id" ASC LIMIT $2 [["term", "handmade soy candles"], ["LIMIT", 1]]
(0.2ms) BEGIN
Search Exists (0.5ms) SELECT 1 AS one FROM "searches" WHERE "searches"."term" = $1 LIMIT $2 [["term", "handmade soy candles"], ["LIMIT", 1]]
SQL (0.5ms) INSERT INTO "searches" ("term", "created_at", "updated_at") VALUES ($1, $2, $3) RETURNING "id" [["term", "handmade soy candles"], ["created_at", "2018-06-18 13:56:27.445609"], ["updated_at", "2018-06-18 13:56:27.445609"]]
(1.3ms) COMMIT
Item Exists (1.1ms) SELECT 1 AS one FROM "items" WHERE "items"."search_id" = $1 AND "items"."collection_id" = 1 LIMIT $2 [["search_id", 52], ["LIMIT", 1]]
(0.2ms) BEGIN
Collection Load (0.5ms) SELECT "collections".* FROM "collections" WHERE "collections"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Item Exists (0.6ms) SELECT 1 AS one FROM "items" WHERE "items"."collection_id" = $1 AND "items"."search_id" = 52 LIMIT $2 [["collection_id", 1], ["LIMIT", 1]]
SQL (0.6ms) INSERT INTO "items" ("collection_id", "search_id", "created_at", "updated_at") VALUES ($1, $2, $3, $4) RETURNING "id" [["collection_id", 1], ["search_id", 52], ["created_at", "2018-06-18 13:56:27.457050"], ["updated_at", "2018-06-18 13:56:27.457050"]]
(2.5ms) COMMIT
Search Load (0.7ms) SELECT "searches".* FROM "searches" WHERE "searches"."term" = $1 ORDER BY "searches"."id" ASC LIMIT $2 [["term", "bookish candle"], ["LIMIT", 1]]
Item Exists (0.6ms) SELECT 1 AS one FROM "items" WHERE "items"."search_id" = $1 AND "items"."collection_id" = 1 LIMIT $2 [["search_id", 13], ["LIMIT", 1]]
Search Load (0.5ms) SELECT "searches".* FROM "searches" WHERE "searches"."term" = $1 ORDER BY "searches"."id" ASC LIMIT $2 [["term", "bookish candles"], ["LIMIT", 1]]
(0.2ms) BEGIN
Search Exists (0.5ms) SELECT 1 AS one FROM "searches" WHERE "searches"."term" = $1 LIMIT $2 [["term", "bookish candles"], ["LIMIT", 1]]
SQL (1.0ms) INSERT INTO "searches" ("term", "created_at", "updated_at") VALUES ($1, $2, $3) RETURNING "id" [["term", "bookish candles"], ["created_at", "2018-06-18 13:56:29.102682"], ["updated_at", "2018-06-18 13:56:29.102682"]]
(1.6ms) COMMIT
Item Exists (0.8ms) SELECT 1 AS one FROM "items" WHERE "items"."search_id" = $1 AND "items"."collection_id" = 1 LIMIT $2 [["search_id", 53], ["LIMIT", 1]]
(0.3ms) BEGIN
Collection Load (0.5ms) SELECT "collections".* FROM "collections" WHERE "collections"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Item Exists (0.5ms) SELECT 1 AS one FROM "items" WHERE "items"."collection_id" = $1 AND "items"."search_id" = 53 LIMIT $2 [["collection_id", 1], ["LIMIT", 1]]
SQL (0.6ms) INSERT INTO "items" ("collection_id", "search_id", "created_at", "updated_at") VALUES ($1, $2, $3, $4) RETURNING "id" [["collection_id", 1], ["search_id", 53], ["created_at", "2018-06-18 13:56:29.114736"], ["updated_at", "2018-06-18 13:56:29.114736"]]
(2.1ms) COMMIT
Search Load (1.0ms) SELECT "searches".* FROM "searches" WHERE "searches"."term" = $1 ORDER BY "searches"."id" ASC LIMIT $2 [["term", "personalized gifts"], ["LIMIT", 1]]
(0.2ms) BEGIN
Search Exists (0.8ms) SELECT 1 AS one FROM "searches" WHERE "searches"."term" = $1 LIMIT $2 [["term", "personalized gifts"], ["LIMIT", 1]]
SQL (0.8ms) INSERT INTO "searches" ("term", "created_at", "updated_at") VALUES ($1, $2, $3) RETURNING "id" [["term", "personalized gifts"], ["created_at", "2018-06-18 13:56:30.446268"], ["updated_at", "2018-06-18 13:56:30.446268"]]
(1.5ms) COMMIT
Item Exists (1.0ms) SELECT 1 AS one FROM "items" WHERE "items"."search_id" = $1 AND "items"."collection_id" = 1 LIMIT $2 [["search_id", 54], ["LIMIT", 1]]
(0.2ms) BEGIN
Collection Load (0.9ms) SELECT "collections".* FROM "collections" WHERE "collections"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Item Exists (0.7ms) SELECT 1 AS one FROM "items" WHERE "items"."collection_id" = $1 AND "items"."search_id" = 54 LIMIT $2 [["collection_id", 1], ["LIMIT", 1]]
SQL (0.8ms) INSERT INTO "items" ("collection_id", "search_id", "created_at", "updated_at") VALUES ($1, $2, $3, $4) RETURNING "id" [["collection_id", 1], ["search_id", 54], ["created_at", "2018-06-18 13:56:30.459078"], ["updated_at", "2018-06-18 13:56:30.459078"]]
(2.2ms) COMMIT
Search Load (0.7ms) SELECT "searches".* FROM "searches" WHERE "searches"."term" = $1 ORDER BY "searches"."id" ASC LIMIT $2 [["term", "personalized gift"], ["LIMIT", 1]]
(0.3ms) BEGIN
Search Exists (0.5ms) SELECT 1 AS one FROM "searches" WHERE "searches"."term" = $1 LIMIT $2 [["term", "personalized gift"], ["LIMIT", 1]]
SQL (0.6ms) INSERT INTO "searches" ("term", "created_at", "updated_at") VALUES ($1, $2, $3) RETURNING "id" [["term", "personalized gift"], ["created_at", "2018-06-18 13:56:32.460438"], ["updated_at", "2018-06-18 13:56:32.460438"]]
(2.5ms) COMMIT
Item Exists (0.6ms) SELECT 1 AS one FROM "items" WHERE "items"."search_id" = $1 AND "items"."collection_id" = 1 LIMIT $2 [["search_id", 55], ["LIMIT", 1]]
(0.2ms) BEGIN
Collection Load (0.7ms) SELECT "collections".* FROM "collections" WHERE "collections"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Item Exists (0.7ms) SELECT 1 AS one FROM "items" WHERE "items"."collection_id" = $1 AND "items"."search_id" = 55 LIMIT $2 [["collection_id", 1], ["LIMIT", 1]]
SQL (0.7ms) INSERT INTO "items" ("collection_id", "search_id", "created_at", "updated_at") VALUES ($1, $2, $3, $4) RETURNING "id" [["collection_id", 1], ["search_id", 55], ["created_at", "2018-06-18 13:56:32.474614"], ["updated_at", "2018-06-18 13:56:32.474614"]]
(1.1ms) COMMIT
Search Load (0.8ms) SELECT "searches".* FROM "searches" WHERE "searches"."term" = $1 ORDER BY "searches"."id" ASC LIMIT $2 [["term", "book smell"], ["LIMIT", 1]]
(0.5ms) BEGIN
Search Exists (0.8ms) SELECT 1 AS one FROM "searches" WHERE "searches"."term" = $1 LIMIT $2 [["term", "book smell"], ["LIMIT", 1]]
SQL (0.7ms) INSERT INTO "searches" ("term", "created_at", "updated_at") VALUES ($1, $2, $3) RETURNING "id" [["term", "book smell"], ["created_at", "2018-06-18 13:56:34.573441"], ["updated_at", "2018-06-18 13:56:34.573441"]]
(1.3ms) COMMIT
Item Exists (0.6ms) SELECT 1 AS one FROM "items" WHERE "items"."search_id" = $1 AND "items"."collection_id" = 1 LIMIT $2 [["search_id", 56], ["LIMIT", 1]]
(0.1ms) BEGIN
Collection Load (1.4ms) SELECT "collections".* FROM "collections" WHERE "collections"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Item Exists (0.6ms) SELECT 1 AS one FROM "items" WHERE "items"."collection_id" = $1 AND "items"."search_id" = 56 LIMIT $2 [["collection_id", 1], ["LIMIT", 1]]
SQL (0.6ms) INSERT INTO "items" ("collection_id", "search_id", "created_at", "updated_at") VALUES ($1, $2, $3, $4) RETURNING "id" [["collection_id", 1], ["search_id", 56], ["created_at", "2018-06-18 13:56:34.585728"], ["updated_at", "2018-06-18 13:56:34.585728"]]
(2.5ms) COMMIT
Redirected to http://localhost:3000/collections/1
Completed 302 Found in 24422ms (ActiveRecord: 153.5ms)
The amount of items I'm inserting varies but even between many different examples, PG always takes longer, even though when looking at those two logs I posted, it seems like SQLite3 is taking longer per transaction.
Is this normal or is there something I can do to optimize it?
Thanks!
Looks like ActiveRecord is responding pretty quickly for both databases:
Sqlite: ActiveRecord: 12.3ms
Postgres: ActiveRecord: 153.5ms
While Postgres is certainly higher, I would think both of those response times would be acceptable, and that they would vary slightly each time you hit that action.
You do, however, have some sort of redirect taking place 302 Redirected to http://localhost:3000/collections/1 that seems to be adding a bunch of time to the request.
You might try making sure that there are no scripts on that page that are gumming up the works. Also, you might make sure that the indexes have been applied to your Postgres database, and also make sure your collections page doesn't have any n+1 queries.
Hard to say more with the limited information given in the question,

Rails not including attribute in SQL Query even though specified

Despite providing the name, the SQL query clearly shows that it's not being passed properly. Rails console doesn't require any whitelist parameters as far as I'm aware, but I've included my controller as well.
Query in rails console: Profession.first.skills.create(name: 'rails')
Profession Load (0.5ms) SELECT "professions".* FROM "professions" ORDER BY "professions"."id" ASC LIMIT $1 [["LIMIT", 1]]
(0.2ms) BEGIN
Skill Exists (0.4ms) SELECT 1 AS one FROM "skills" WHERE "skills"."name" = $1 LIMIT $2 [["name", "twitter"], ["LIMIT", 1]]
SQL (1.0ms) INSERT INTO "skills" ("profession_id", "created_at", "updated_at") VALUES ($1, $2, $3) RETURNING "id" [["profession_id", 1], ["created_at", "2017-12-18 12:22:11.154775"], ["updated_at", "2017-12-18 12:22:11.154775"]]
(0.1ms) ROLLBACK
Not sure what's going on here.
Validation errors are still working though...
Valid object:
:027 > Skill.new(name: "rails", profession: Profession.first).valid?
Profession Load (0.4ms) SELECT "professions".* FROM "professions" ORDER BY "professions"."id" ASC LIMIT $1 [["LIMIT", 1]]
Skill Exists (1.9ms) SELECT 1 AS one FROM "skills" WHERE "skills"."name" = $1 LIMIT $2 [["name", "twitter"], ["LIMIT", 1]]
=> true
Name being detected for validations:
:020 > Skill.create!(name: String.new, profession: Profession.first)
Profession Load (0.4ms) SELECT "professions".* FROM "professions" ORDER BY "professions"."id" ASC LIMIT $1 [["LIMIT", 1]]
(0.1ms) BEGIN
Skill Exists (0.4ms) SELECT 1 AS one FROM "skills" WHERE "skills"."name" = $1 LIMIT $2 [["name", ""], ["LIMIT", 1]]
(0.1ms) ROLLBACK
ActiveRecord::RecordInvalid: Validation failed: Name can't be blank
from (irb):20
Max Length:
:023 > Skill.create!(name: "sjadfkahskdfkjsahdfkjaskjdfkjhsdjkfhksajhfjksahasdljflasjdlfkjaskldjflkasjdklfjklasjdklfjlasjdflkjasklfjsdfhkjsahkjdfhjkasdhfkjhkj", profession: Profession.first)
Profession Load (0.4ms) SELECT "professions".* FROM "professions" ORDER BY "professions"."id" ASC LIMIT $1 [["LIMIT", 1]]
(0.2ms) BEGIN
Skill Exists (0.3ms) SELECT 1 AS one FROM "skills" WHERE "skills"."name" = $1 LIMIT $2 [["name", "sjadfkahskdfkjsahdfkjaskjdfkjhsdjkfhksajhfjksahasdljflasjdlfkjaskldjflkasjdklfjklasjdklfjlasjdflkjasklfjsdfhkjsahkjdfhjkasdhfkjhkj"], ["LIMIT", 1]]
(0.3ms) ROLLBACK
ActiveRecord::RecordInvalid: Validation failed: Name is too long (maximum is 50 characters)
from (irb):23
ProfessionsController
class ProfessionsController < ApplicationController
def new
#profession = Profession.new
end
def create
#profession = Profession.find_by(name: profession_params[:name])
skill_params = get_nested_params(profession_params, :skills_attributes)
if #profession
# Save skill under existing profession
#skill = Skill.create(name: skill_params[:name], profession_id: #profession.id)
else
#profession = Profession.new(name: profession_params[:name])
if #profession.save {
saved_profession = Profession.find_by(name: profession_params[:name])
saved_profession.skills.create(name: "twitter")
# Skill.create(name: skill_params[:name], profession_id: Profession.find_by(name: profession_params[:name])).save!
}
end
end
respond_to do |format|
if #profession.save || #skill.save
format.js { render layout: false }
format.html { redirect_back fallback_location: root_path, notice: 'Profession was successfully created.' }
else
format.html { redirect_back fallback_location: root_path, notice: 'Skill was not created.' }
end
end
end
private
def profession_params
params.require(:profession).permit(:name,
skills_attributes: [:id,
:name,
:starting_date,
:profession_id,
:_destroy])
end
def get_nested_params parent_params, nested_params
nested_attrs = parent_params[nested_params]
nested_attrs[nested_attrs.keys[0]]
end
end
Updated:
Error from controller when using saved_profession.skills.create(name: "twitter")
Started POST "/professions" for 127.0.0.1 at 2017-12-19 01:50:50 +1300
Processing by ProfessionsController#create as JS
Parameters: {"utf8"=>"βœ“", "profession"=>{"name"=>"software", "skills_attributes"=>{"1513601431887"=>{"name"=>"rails", "_destroy"=>"false"}}}, "commit"=>"Create Profession"}
Profession Load (0.3ms) SELECT "professions".* FROM "professions" WHERE "professions"."name" = $1 LIMIT $2 [["name", "software"], ["LIMIT", 1]]
(0.1ms) BEGIN
Profession Exists (0.3ms) SELECT 1 AS one FROM "professions" WHERE "professions"."name" = $1 LIMIT $2 [["name", "software"], ["LIMIT", 1]]
SQL (0.3ms) INSERT INTO "professions" ("name", "created_at", "updated_at") VALUES ($1, $2, $3) RETURNING "id" [["name", "software"], ["created_at", "2017-12-18 12:50:50.202270"], ["updated_at", "2017-12-18 12:50:50.202270"]]
Profession Load (0.2ms) SELECT "professions".* FROM "professions" WHERE "professions"."name" = $1 LIMIT $2 [["name", "software"], ["LIMIT", 1]]
Skill Exists (0.3ms) SELECT 1 AS one FROM "skills" WHERE "skills"."name" = $1 LIMIT $2 [["name", "twitter"], ["LIMIT", 1]]
SQL (0.7ms) INSERT INTO "skills" ("profession_id", "created_at", "updated_at") VALUES ($1, $2, $3) RETURNING "id" [["profession_id", 4], ["created_at", "2017-12-18 12:50:50.205484"], ["updated_at", "2017-12-18 12:50:50.205484"]]
(0.1ms) ROLLBACK
Completed 500 Internal Server Error in 8ms (ActiveRecord: 2.2ms)
ActiveRecord::NotNullViolation (PG::NotNullViolation: ERROR: null value in column "name" violates not-null constraint
DETAIL: Failing row contains (11, null, 2017-12-14, 4, 2017-12-18 12:50:50.205484, 2017-12-18 12:50:50.205484, null).
: INSERT INTO "skills" ("profession_id", "created_at", "updated_at") VALUES ($1, $2, $3) RETURNING "id"):
app/controllers/professions_controller.rb:21:in `block in create'
app/controllers/professions_controller.rb:18:in `create'
Try modifying your get_nested_params as:
def get_nested_params(parent_params, nested_params)
parent_params # For logging purpose only
# => {"name"=>"software", "skills_attributes"=>{"1513601431887"=>{"name"=>"rails", "_destroy"=>"false"}}}
nested_params # For logging purpose only
# => :skills_attributes
nested_attrs = parent_params[nested_params]
# => {"1513601431887"=>{"name"=>"rails", "_destroy"=>"false"}}
nested_attrs.values[0] # Return this
# => {"name"=>"rails", "_destroy"=>"false"}
end

Setting value as params on creation

Attempting to make different orders where depending on the order there is different revisions_left which is an integer in the migration file.
I pass the params[:orderref] through a button like so <%= link_to 'Package 1', new_order_path(orderref: 1)%> and then am trying to call it in the create method like so
if params[:orderref] == "1"
#order.revisions_left = 1
elsif params[:orderref] == "2"
#order.revisions_left = 2
elsif params[:orderref] == "3"
#order.revisions_left = 3
end
However the revisions_left fails to save and am unsure why
Edit: Logs
Processing by OrderController#create as HTML
Parameters: {"utf8"=>"βœ“", "authenticity_token"=>"VGt/2PCPMmVnzXGm+UgHcmSvX7FEHxOoF/TfJoKf8gn3NJ0grD9JkydfswwV50GXIneFuTSClLK7cqvpsRCwRg==", "order"=>{"name"=>"sdada", "email"=>"as#tt.com", "company"=>"", "event_type"=>"Birthday", "country"=>"Brazil", "description"=>"1233 1 123 11", "order_type"=>"3", "price"=>"80"}, "commit"=>"Create Order"}
Order Load (3.0ms) SELECT "orders".* FROM "orders" WHERE "orders"."name" = $1 AND "orders"."company" = $2 AND "orders"."email" = $3 AND "orders"."event_type" = $4 AND "orders"."country" = $5 AND "orders"."description" = $6 AND "orders"."order_type" = $7 AND "orders"."price" = $8 ORDER BY "orders"."id" ASC LIMIT $9 [["name", "sdada"], ["company", ""], ["email", "as#tt.com"], ["event_type", "Birthday"], ["country", "Brazil"], ["description", "1233 1 123 11"], ["order_type", 3], ["price", 80], ["LIMIT", 1]]
(0.4ms) BEGIN
Order Exists (1.0ms) SELECT 1 AS one FROM "orders" WHERE LOWER("orders"."email") = LOWER($1) LIMIT $2 [["email", "as#tt.com"], ["LIMIT", 1]]
SQL (6.2ms) INSERT INTO "orders" ("name", "company", "email", "event_type", "country", "description", "price", "order_type", "created_at", "updated_at") VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10) RETURNING "id" [["name", "sdada"], ["company", ""], ["email", "as#tt.com"], ["event_type", "Birthday"], ["country", "Brazil"], ["description", "1233 1 123 11"], ["price", 80], ["order_type", 3], ["created_at", 2017-08-03 05:26:21 UTC], ["updated_at", 2017-08-03 05:26:21 UTC]]
(71.2ms) COMMIT
(0.1ms) BEGIN
Order Exists (0.5ms) SELECT 1 AS one FROM "orders" WHERE LOWER("orders"."email") = LOWER($1) AND ("orders"."id" != $2) LIMIT $3 [["email", "as#tt.com"], ["id", 3], ["LIMIT", 1]]
SQL (0.7ms) UPDATE "orders" SET "status" = $1, "updated_at" = $2 WHERE "orders"."id" = $3 [["status", 1], ["updated_at", 2017-08-03 05:26:22 UTC], ["id", 3]]
(0.3ms) COMMIT

Resources