Produce a report by getting data from multiple tables - ruby-on-rails

I have 4 tables:
key: id, name
project: id, name
project_key: id, key_id, project_id
project_report: id, status, project_id, key_id
project_c_report: id, status, project_id, key_id, c_id
I want to produce a report using those tables:
The output should be:
Key.name, project_report.status, project_c_report.status
I was able to do this by getting all the keys from a project, and loop over them
array = []
project.keys.each do |k|
p = ProjectReport.where(keyword_id: k, project_id: p.id).map(&:status)
c = ProjectCReport.where(keyword_id: k, project_id: p.id, c_id:1).map(&:status)
array << {name: k.name, pr: p, pcr: c}
end
array
The problem is that I am doing a lot of selects and everything is slow, can someone help me please with a better way of doing this.
Thank you

First, create a function in your DataBase. This is just a brief example, and also its done in PostgreSQL but shouldnt difer much
from MySQL, SQLServer, etc
Function get_myreport(key_id integer, project_id integer [As many params as you'd like the function to get))
pProject ALIAS FOR $1;
pKey ALIAS FOR $2;
BEGIN
CREATE TEMP TABLE IF NOT EXISTS tmp_project_report(id integer, project_name character varying, *All the values you want to see in the report);
TRUNCATE tmp_project_report;
INSERT INTO tmp_project_report(all the columns)
SELECT a.table1_fields, b.table2_fields, c.table3_fields, d.table4_fields, e.table5_fields
FROM table1 a, table2 b, table3 c, table4 d, table5 e
WHERE
a.key = pKey
AND b.project_key = pProject
END;
Then, in your controller's method you call the up the function like this
myFunction = ActiveRecord:Base.connection.execute = "Select get_myreport("param1, param2, etc...")
You will have to make a model where you put all the fields that are on the temp_table you've made, and also you will set the temp_table as the self.table_name
Then, in your view, you'd only have to iterate on your collection and display the values accordingly
#report = TempTable.all
<% #report.each_do |report| %>
<% report.value1 %>
<% etc... %>
<% end %>

Figure out the database query, then query the database directly from your model:
def records
connection = ActiveRecord::Base.connection
records = connection.select %Q {
SELECT key.name, project_report.status, project_c_report.status
FROM ...
JOIN ...
;
}
records
end

Here is something you can try if you choose to keep this within Rails (note that the following query is untested and is shown for concept only):
report_data = Project.joins(project_key: :key)
.joins('left join project_reports on project_keys.project_id = project_reports.project_id and project_keys.key_id = project_reports.key_id
left join project_c_reports on project_keys.project_id = project_c_reports.project_id and project_keys.key_id = project_c_reports.key_id')
.where('project_c_reports.c_id = ?', 1)
.select('projects.name, project_reports.status as report_status, project_c_reports.status as c_report_status')
This should give you an array of Project objects each including the selected three attributes name, report_status, c_report_status. To get these values in an array of these three elements you could do:
report_data.map { |p| [ p.name, p.report_status, p.c_report_status ] }
The type of join for the query depends on your requirement. Given the index are in place the query should be better compared to how it looks in code!

Related

How do I group axlsx file to sort alphabetically by org

Org is a column in attendee that's nested in this includes statement. I want it so that it groups attendee.org alphabetically
#meetings = NwMeeting.includes(meeting_attendees: [:attendee]).where(show_id: #show.id)
#meetings.each do |meeting|
#nw_attendee_meetings = meeting&.meeting_attendees(&:attendee)
#nw_attendee_meetings.each do |nw_attendee_meeting|
attendee = nw_attendee_meeting&.attendee
data = []
data << attendee&.org
data << attendee&.name
data << meeting&.status&.capitalize
AxlsxTools.add_row(data) (edited)
Because you want to list Attendee records, it may be a little bit easier if you switch the "main" model you query to Attendee. Joining the NwMeeting and SELECT the status column, then you can just do ORDER BY attendees.org ASC, the records with NULL value will be placed in front of others by default.
#attendees = Attendee.joins(meeting_attendees: :nw_meeting)
.select("attendees.*, nw_meetings.status AS meeting_status")
.where(nw_meetings: { show_id: #show.id })
.order("attendees.org ASC")
# .order("attendees.org ASC NULL FIRST") if you use PosgreSQL
#attendees.each do |attendee|
data = []
data << attendee.org
data << attendee.name
data << attendee.meeting_status&.capitalize
AxlsxTools.add_row(data)
end
btw, #engineersmnky 's Arel method is more advanced, and that should work, too :)

Create a WHERE (columns) IN (values) clause with Arel?

Is there a way to programatically create a where clause in Arel where the columns and values are specified separately?
SELECT users.*
WHERE (country, occupation) IN (('dk', 'nurse'), ('ch', 'doctor'), ...
Say the input is a really long list of pairs that we want to match.
I'm am NOT asking how to generate a WHERE AND OR clause which is really simple to do with ActiveRecord.
So far I just have basic string manipulation:
columns = [:country, :occupation]
pairs = [['dk', 'nurse'], ['ch', 'doctor']]
User.where(
"(#{columns.join(', ')}) IN (#{ pairs.map { '(?, ?)' }.join(', ')})",
*pairs
)
Its not just about the length of the query WHERE (columns) IN (values) will also perform much better on Postgres (and others as well) as it can use an index only scan where OR will cause a bitmap scan.
I'm only looking for answers that can demonstrate generating a WHERE (columns) IN (values) query with Arel. Not anything else.
All the articles I have read about Arel start building of a single column:
arel_table[:foo].eq...
And I have not been able to find any documentation or articles that cover this case.
The trick to this is to build the groupings correctly and then pass them through to the Arel In Node, for example:
columns = [:country, :occupation]
pairs = [['dk', 'nurse'], ['ch', 'doctor']]
User.where(
Arel::Nodes::In.new(
Arel::Nodes::Grouping.new( columns.map { |column| User.arel_table[column] } ),
pairs.map { |pair| Arel::Nodes::Grouping.new(
pair.map { |value| Arel::Nodes.build_quoted(value) }
)}
)
)
The above will generate the following SQL statement (for MySQL):
"SELECT users.* FROM users WHERE (users.country,
users.occupation) IN (('dk', 'nurse'), ('ch', 'doctor'))"
This will still generate long query with 'OR' in between. But I felt this is lil elegant/different approach to achieve what you want.
ut = User.arel_table
columns = [:country, :occupation]
pairs = [['dk', 'nurse'], ['ch', 'doctor']]
where_condition = pairs.map do |pair|
"(#{ut[columns[0]].eq(pair[0]).and(ut[columns[1]].eq(pair[1])).to_sql})"
end.join(' OR ')
User.where(where_condition)
I have tried this different approach at my end. Hope it will work for you.
class User < ActiveRecord::Base
COLUMNS = %i(
country
occupation
)
PAIRS = [['dk', 'nurse'], ['ch', 'doctor']]
scope :with_country_occupation, -> (pairs = PAIRS, columns = COLUMNS) { where(filter_country_occupation(pairs, columns)) }
def self.filter_country_occupation(pairs, columns)
pairs.each_with_index.reduce(nil) do |query, (pair, index)|
column_check = arel_table[columns[0]].eq(pair[0]).and(arel_table[columns[1]].eq(pair[1]))
if query.nil?
column_check
else
query.or(column_check)
end
end.to_sql
end
end
Call this scope User.with_country_occupation let me know if it works for you.
Thanks!
I think we can do this with Array Conditions as mentioned here
# notice the lack of an array as the last argument
Model.where("attribute = ? OR attribute2 = ?", value, value)
Also, as mentioned here we can use an SQL in statement:
Model.where('id IN (?)', [array of values])
Or simply, as kdeisz pointed out (Using Arel to create the SQL query):
Model.where(id: [array of values])
I have not tried myself, but you can try exploring with these examples.
Always happy to help!

How to get a unique set of parent models after querying on child

Order has_many Items is the relationship.
So let's say I have something like the following 2 orders with items in the database:
Order1 {email: alpha#example.com, items_attributes:
[{name: "apple"},
{name: "peach"}]
}
Order2 {email: beta#example.com, items_attributes:
[{name: "apple"},
{name: "apple"}]
}
I'm running queries for Order based on child attributes. So let's say I want the emails of all the orders where they have an Item that's an apple. If I set up the query as so:
orders = Order.joins(:items).where(items: {name:"apple"})
Then the result, because it's pulling at the Item level, will be such that:
orders.count = 3
orders.pluck(:email) = ["alpha#exmaple.com", "beta#example.com", "beta#example.com"]
But my desired outcome is actually to know what unique orders there are (I don't care that beta#example.com has 2 apples, only that they have at least 1), so something like:
orders.count = 2
orders.pluck(:email) = ["alpha#exmaple.com", "beta#example.com"]
How do I do this?
If I do orders.select(:id).distinct, this will fix the problem such that orders.count == 2, BUT this distorts the result (no longer creates AR objects), so that I can't iterate over it. So the below is fine
deduped_orders = orders.select(:id).distinct
deduped_orders.count = 2
deduped_orders.pluck(:email) = ["alpha#exmaple.com", "beta#example.com"]
But then the below does NOT work:
deduped_orders.each do |o|
puts o.email # ActiveModel::MissingAttributeError: missing attribute: email
end
Like I basically want the output of orders, but in a unique way.
I find using subqueries instead of joins a bit cleaner for this sort of thing:
Order.where(id: Item.select(:order_id).where(name: 'apple'))
that ends up with this (more or less) SQL:
select *
from orders
where id in (
select order_id
from items
where name = 'apple'
)
and the in (...) will clear up duplicates for you. Using a subquery also clearly expresses what you want to do–you want the orders that have an item named 'apple'–and the query says exactly that.
use .uniq instead of .distinct
deduped_orders = orders.select(:id).uniq
deduped_orders.count = 2
deduped_orders.pluck(:email) = ["alpha#exmaple.com", "beta#example.com"]
If you want to keep all the attributes of orders use group
deduped_orders = orders.group(:id).distinct
deduped_orders.each do |o|
puts o.email
end
#=> output: "alpha#exmaple.com", "beta#example.com"
I think you just need to remove select(:id)
orders = Order.joins(:items).where(items: {name:"apple"}).distinct
orders.pluck(:email)
# => ["alpha#exmaple.com", "beta#example.com"]
orders = deduped_orders
deduped_orders.each do |o|
puts o.email # loop twice
end

Rails join and HABTM relationship

In my application I have model Car which:
has_and_belongs_to_many :locations
Now I'm buidling searching and I want search Car which has given locations.
In my view I have:
.row
= horizontal_simple_form_for :cars, {url: cars_path, method: :get} do |f|
= f.input :handover_location, label: I18n.t('.handover'), collection: Location.all.map{|hl| [hl.location_address, hl.id]}
= f.input :return_location, label: I18n.t('.return') ,collection: Location.all.map{|rl| [rl.location_address, rl.id]}
= f.submit class: 'btn btn-success'
and in my controller I filter results based on params:
#cars = Car.joins(:locations).where("locations.id= ? AND locations.id= ?", params[:cars][:handover_location], params[:cars][:return_location])
But this code does not work properly. Maybe I shouldn't use "locations.id" twice?
I'm going to assume your join table is called cars_locations. If you wanted to do this just in sql, you could join this table to itself
... cars_locations cl1 join cars_locations cl2 on e1.car_id = e2.car_id ...
... which would make a pseudo-table for the duration of the query with this structure:
cl1.id | cl1.car_id | cl1.location_id | cl2.id | cl2.car_id | cl2.location_id
then query this for the required location_id - this will give you entries that have the same car at both locations - let's say the ids of the pickup and return locations are 123 and 456:
select distinct(cl1.car_id) from cars_locations cl1 join cars_locations cl2 on cl1.car_id = cl2.car_id where (c11.location_id = 123 and cl2.location_id = 456) or (cl1.location_id = 123 and cl2.location_id = 456);
Now we know the sql, you can wrap it into a method of the Car class
#in the Car class
def self.cars_at_both_locations(location1, location2)
self.find_by_sql("select * from cars where id in (select distinct(cl1.car_id) from cars_locations cl1 join cars_locations cl2 on cl1.car_id = cl2.car_id where (c11.location_id = #{location1.id} and cl2.location_id = #{location2.id}) or (cl1.location_id = #{location2.id} and cl2.location_id = #{location1.id}))")
end
This isn't the most efficient method, as joins on big tables start to get very slow.
A quicker method would be
def self.cars_at_both_locations(location1, location2)
self.find(location1.car_ids & location2.car_ids)
end
in this case we use & which is the "set intersection" operator (not to be confused with &&): ie it will return only values that are in both the arrays on either side of it.
You definitely shouldn't be using the locations.id twice in the where clause, as that is physically impossible. The resulting query from that will essentially try and find the location where it's id is both the handover location, AND the return location. So in essence, what you're asking for is something like
where 1 == 1 AND 1 == 2
Which needless to say, will always return nothing.
In theory if you just change the AND for an OR you'll get what you're after. This way, you'll be asking the database for any location that has an ID or either start_location OR handover_location
UPDATE
Re-read the question. It's a little tricker than I'd thought initially, so you'll probably need to do some processing on the results. As I've said, using the AND query the way you are is asking the database for something impossible, but using the OR as I originally said, will result in cars that have EITHER or the locations, not both. This could be done in raw SQL, but using Rails this is both awkward, and frowned upon, so here's another solution.
Query the data using the OR selector I originally proposed as this will reduce the data set considerably. Then manually go through it, and reject anything that doesn't have both locations:
locations = [params[:cars][:handover_location], params[:cars][:return_location]]
#cars = Car.joins(:locations).where("locations.id IN [?]")
#cars = #cars.reject { |c| !(locations - c.location_ids).empty? }
So what this does, is query all cars that have either of the requested locations. Then it loops through those cars, and rejects any whose list of location id's does not contain both of the supplied IDS. Now the remaining cars are available at both locations :)

Removing "duplicate objects"

Let's say I have an array of objects from the same class, with two attributes of concern here: name and created_at.
How do I find objects with the same name (considered dups) in the array, and then delete the duplicate record in the database. The object with the most-recent created_at date, however, is the one that must be deleted.
seen = []
#sort by created date and iterate
collection.sort({|a,b| a.created_at <=> b.created_at}).each do |obj|
if seen.map(&:name).include? obj.name #check if the name has been seen already
obj.destroy!
else
seen << obj #if not, add it to the seen array
end
end
Should do the job hopefully.
If this is just a one-time bugfix before introducing UNIQUE INDEX on the table, you might as well do it in SQL:
DELETE FROM t WHERE id IN (
SELECT t1.id
FROM t t1
LEFT JOIN t t2 ON t1.name = t2.name AND t2.created_at < t1.created_at
WHERE t2.id IS NOT NULL
)

Resources