I have 3 models: Project, MonthlySubscription (sti of Subscription), and MonthlyTransactionQueue (sti of TransactionQueue). Subscription and TransactionQueue both belong_to Project.
I want to create a copy of MonthlySubscription and place it into MonthlyTransactionQueue, for Projects that have a Release.released = false. How would I do this using AR?
My sql looks like this:
insert into transaction_queues
select a.*, b.id as release_id
from subscriptions a
left join releases b
on a.project_id = b.project_id
where b.released = false
and a.type = 'ReleaseSubscription'
For AR I have started with this ReleaseSubscription.joins(project: :releases) but it doesn't keep the Release.released field
You have a few options
Execute sql
ReleaseSubscription.connection.execute("insert into transaction_queues...")
Use AR inside of a transaction.
MonthlyTransactionQueue.transaction do
# I'm unsure what Release.released is and how it relates but this should work other than that.
MonthlySubscription.where(released: false).each do |sub|
MonthlyTransactionQueue.create(sub.attributes)
end
end
This creates multiple insert statements but runs them all in the same transaction.
Another good option would be to dump everything that matches your query into a sql file and use load data in file to add everything at once in sql.
Related
I am supporting a legacy application with Ruby on Rails (Rails version 4.2).
I have a search functionality, where I have on the user interface one search box.
There is a functionality that already exist.
The user can enter a search term and submit the search, and in the back end the code will search for this "search term" in 7 columns, and all of them belong in one table (the table "Tickets" with the model Ticket)
The 7 column names are stored in an array Ticket::SEARCH_FIELDS
and the search will be all SQL "like".
So the sql statement WHERE clause will be
Select .... FROM Tickets WHERE (column1 like '%term%' or column2 like '%term%' or ....)
so the code that build this where clause is as follows
query = Ticket.where(
Ticket::SEARCH_FIELDS.map { |field|
ticket.arel_table[field].matches("%#{search_term}%")
}.inject(:or)
)
The code is working fine.
But there is a new requirement to add one more column to those 7 columns, but that column is not from the same table but from a lookup table called Employee.
There is already an association between the two model in the Model definition for Ticket, as follows:
class Ticket < ActiveRecord::Base
belongs_to :employee
So, I want to add to the previous where clause another OR clause with that column as follows:
WHERE column1 like '%term%' .... OR employees.name like '%term%'
I tried few attempts but without any success.
I really appreciate any help
This works in Rails 6. Should work in version 4.2 as well.
query1 = Ticket.joins(:employee).where(
Ticket::SEARCH_FIELDS.map { |field|
ticket.arel_table[field].matches("%#{search_term}%")
}.inject(:or)
)
query2 = Ticket.joins(:employee).where(Employee.arel_table[name].matches("%#{search_term}%"))
query1.or(query2)
edit:
While Rails 4.2 doesn't support or, you can use a backport library such as https://github.com/Eric-Guo/where-or to get the same functionality.
I know this is a very special case, and very few people will encounter it. But I found a way around this:
table_columns = Ticket::SEARCH_FIELDS.map { |field|
ticket.arel_table[field].matches("%#{search_term}%")
employee_column = Employee.areal_table[:name].matches("%#{search_term}%")
table_columns.append(employee_column)
query = Ticket.joins("LEFT OUTER JOIN employees on tickets.employee_id = employees.id").where (table_columns.inject(:or))
Consider a scenario where I have an ActiveRecord model Document that has a has_many relation with subdependent model File. I can eager load the files for a given parent Document using a query such as:
document_with_files = Document.includes(:files).find_by(document_id: document_id)
Is there a way in ActiveRecord to delete the parent document if there is only one child file as part of the same SQL statement? I've tried something like the following and it's quite close:
Document.select("document.*, COUNT(files.id) file_count")
.joins(:files)
.group("document.id")
.where(document_id: document_id_to_be_found)
.where("file_count = ?", 1)
It seems that my assignment of file_count isn't being recognized in the final .where clause.
For your scenario, the following code could find the document which only has one child file.
Document.joins(:files).group('documents.id').having('count(files.id) = 1')
And if you want to delete the parent document if there is only one child file as part of the same SQL statement.
Document.joins(:files)
.group('documents.id')
.having('count(files.id) = 1')
.find_by(document_id: document_id_to_be_found)
.destroy
I am trying to use distinct on in rails with a scope, I've created a method in my model like this:
def self.fetch_most_recent_by_user(scope)
scope.where(guid: scope.except(:select).select("DISTINCT ON (eld_logs.user_id) user_id, eld_logs.guid").order("user_id, eld_logs.created_at desc").map(&:guid))
end
When I execute this I get and error like:
TestModel.fetch_most_recent_by_user(TestModel.includes(:user))
ERROR: syntax error at or near "DISTINCT"
LINE 1: SELECT guid, DISTINCT ON (user_id) user_id...
On searching on DISTINCT ON I found out that it should be the first element in a select statement for postgres to make it work.
I want to prepend the DISTINCT ON in the select statement. I have tried clearing the old select statements using except(:select) which I got from here, but it doesn't work because the includes(:user) prepends users attributes first while doing a left join.
I am using Rails 4.0.13 and Postgres 9.4.12. Any help is appreciated.
I found that if the includes was meddling with the distinct my sub query, because which DISTINCT ON failed. I modified my method to this and it works:
def self.fetch_most_recent_eld_log_by_user(scope, include_associations = { })
scope.where(guid: scope.except(:select).select("DISTINCT ON (eld_logs.user_id) eld_logs.user_id, eld_logs.guid").order("eld_logs.user_id, eld_logs.created_at desc").map(&:guid))
.includes(include_associations)
end
Still it'll be good if someone can provide a way to prepend something in the select statement of active record scope.
I am working on an app that allows Members to take a survey (Member has a one to many relationship with Response). Response holds the member_id, question_id, and their answer.
The survey is submitted all or nothing, so if there are any records in the Response table for that Member they have completed the survey.
My question is, how do I re-write the query below so that it actually works? In SQL this would be a prime candidate for the EXISTS keyword.
def surveys_completed
members.where(responses: !nil ).count
end
You can use includes and then test if the related response(s) exists like this:
def surveys_completed
members.includes(:responses).where('responses.id IS NOT NULL')
end
Here is an alternative, with joins:
def surveys_completed
members.joins(:responses)
end
The solution using Rails 4:
def surveys_completed
members.includes(:responses).where.not(responses: { id: nil })
end
Alternative solution using activerecord_where_assoc:
This gem does exactly what is asked here: use EXISTS to to do a condition.
It works with Rails 4.1 to the most recent.
members.where_assoc_exists(:responses)
It can also do much more!
Similar questions:
How to query a model based on attribute of another model which belongs to the first model?
association named not found perhaps misspelled issue in rails association
Rails 3, has_one / has_many with lambda condition
Rails 4 scope to find parents with no children
Join multiple tables with active records
You can use SQL EXISTS keyword in elegant Rails-ish manner using Where Exists gem:
members.where_exists(:responses).count
Of course you can use raw SQL as well:
members.where("EXISTS" \
"(SELECT 1 FROM responses WHERE responses.member_id = members.id)").
count
You can also use a subquery:
members.where(id: Response.select(:member_id))
In comparison to something with includes it will not load the associated models (which is a performance benefit if you do not need them).
If you are on Rails 5 and above you should use left_joins. Otherwise a manual "LEFT OUTER JOINS" will also work. This is more performant than using includes mentioned in https://stackoverflow.com/a/18234998/3788753. includes will attempt to load the related objects into memory, whereas left_joins will build a "LEFT OUTER JOINS" query.
def surveys_completed
members.left_joins(:responses).where.not(responses: { id: nil })
end
Even if there are no related records (like the query above where you are finding by nil) includes still uses more memory. In my testing I found includes uses ~33x more memory on Rails 5.2.1. On Rails 4.2.x it was ~44x more memory compared to doing the joins manually.
See this gist for the test:
https://gist.github.com/johnathanludwig/96fc33fc135ee558e0f09fb23a8cf3f1
where.missing (Rails 6.1+)
Rails 6.1 introduces a new way to check for the absence of an association - where.missing.
Please, have a look at the following code snippet:
# Before:
Post.left_joins(:author).where(authors: { id: nil })
# After:
Post.where.missing(:author)
And this is an example of SQL query that is used under the hood:
Post.where.missing(:author)
# SELECT "posts".* FROM "posts"
# LEFT OUTER JOIN "authors" ON "authors"."id" = "posts"."author_id"
# WHERE "authors"."id" IS NULL
As a result, your particular case can be rewritten as follows:
def surveys_completed
members.where.missing(:response).count
end
Thanks.
Sources:
where.missing official docs.
Pull request.
Article from the Saeloun blog.
Notes:
where.associated - a counterpart for checking for the presence of an association is also available starting from Rails 7.
See offical docs and this answer.
Goal: Using a CRON task (or other scheduled event) to update database with nightly export of data from an existing system.
All data is created/updated/deleted in an existing system. The website does no directly integrate with this system, so the rails app simply needs to reflect the updates that appear in the data export.
I have a .txt file of ~5,000 products that looks like this:
"1234":"product name":"attr 1":"attr 2":"ABC Manufacturing":"2222"
"A134":"another product":"attr 1":"attr 2":"Foobar World":"2447"
...
All values are strings enclosed in double quotes (") that are separated by colons (:)
Fields are:
id: unique id; alphanumeric
name: product name; any character
attribute columns: strings; any character (e.g., size, weight, color, dimension)
vendor_name: string; any character
vendor_id: unique vendor id; numeric
Vendor information is not normalized in the current system.
What are best practices here? Is it okay to delete the products and vendors tables and rewrite with the new data on every cycle? Or is it better to only add new rows and update existing ones?
Notes:
This data will be used to generate Orders that will persist through nightly database imports. OrderItems will need to be connected to the product ids that are specified in the data file, so we can't rely on an auto-incrementing primary key to be the same for each import; the unique alphanumeric id will need to be used to join products to order_items.
Ideally, I'd like the importer to normalize the Vendor data
I cannot use vanilla SQL statements, so I imagine I'll need to write a rake task in order to use Product.create(...) and Vendor.create(...) style syntax.
This will be implemented on EngineYard
I wouldn't delete the products and vendors tables on every cycle. Is this a rails app? If so there are some really nice ActiveRecord helpers that would come in handy for you.
If you have a Product active record model, you can do:
p = Product.find_or_initialize_by_identifier(<id you get from file>)
p.name = <name from file>
p.size = <size from file>
etc...
p.save!
The find_or_initialize will lookup the product in the database by the id you specify, and if it can't find it, it will create a new one. The really handy thing about doing it this way, is that ActiveRecord will only save to the database if any of the data has changed, and it will automatically update any timestamp fields you have in the table (updated_at) accordingly. One more thing, since you would be looking up records by the identifier (id from the file), I would make sure to add an index on that field in the database.
To make a rake task to accomplish this, I would add a rake file to the lib/tasks directory of your rails app. We'll call it data.rake.
Inside data.rake, it would look something like this:
namespace :data do
desc "import data from files to database"
task :import => :environment do
file = File.open(<file to import>)
file.each do |line|
attrs = line.split(":")
p = Product.find_or_initialize_by_identifier(attrs[0])
p.name = attrs[1]
etc...
p.save!
end
end
end
Than to call the rake task, use "rake data:import" from the command line.
Since Products don't really change that often, the best way I would see is to update only the records that change.
Get all the deltas
Mass update using a single SQL statement
If you are having your normalization code in the models, you could use Product.create and Vendor.create or else it would be just a overkill. Also, Look into inserting multiple records in a single SQL transaction, its much faster.
Create an importer rake task that is cronned
Parse the file line by line using Faster CSV or via vanilla ruby like:
file.each do |line|
products_array = line.split(":")
end
Split each line on the ":" and push in into a hash
Use a find_or_initialize to populate your db such as:
Product.find_or_initialize_by_name_and_vendor_id("foo", 111)