I want to update a column in a Supply model that updates with its stock based on a sum of a subset. The thing is that actually hits two queries to make it works.
total_stock = History.sum(:quantity) # first query
Supply.update_columns(stock: total_stock) # second query
Then, I tried to write a subquery so:
total_stock_subquery = mysubset.select(Arel.sql('SUM(quantity)')).to_sql
Supply.update_columns(stock: total_stock_subquery) # it should be only one query
But, ActiveRecord will replace the total_stock_subquery variable as a string inside the resulting sql statement, then query fails.
I tried to wrap the total_stock_subquery parameter inside an Arel.sql thinking that ActiveRecord will detect that the parameter is not a string, but Arel.sql, and would escape this as raw sql, but it didnt work.
Supply.update_columns(stock: Arel.sql(total_stock_subquery)) # dont work too
Then, do you know how can I pass an sql subquery as parameter for a column inside .update_column, o maybe how to tell ActiveRecord that some parameter should be treated as pure sql not as string?
I know I could write this update as raw sql but I think it would be off Rails style.
EDIT: Table definitions
History
id: integer
supply_id: reference
quantity: decimal(9,2)
Supply
id: integer
product: string
stock: decimal(9,2)
class History
belongs_to: supply
end
class Supply
has_many: histories
end
We use the History model to track in and out (positive for in, negative for out) of supplies, then the sum of the histories will be the current stock.
Of course I can implement an after_save on History model to update the stock when some quantity changes it, but for this example, Im implementing a "rebase" or "recalculate" method for all supplies stock, then I need to make the sum for each Supply.
Related
In order to learn Ruby on Rails I am writing a web app that will be used to sort teams within a tournament given their performance to date.
The complication is that I want each tournament organiser (system user) to be able to use a variety of metrics in an arbitrary order.
Expressed as SQL (my background) I want User 1 to be able to choose:
ORDER BY
METRIC1
,METRIC2
,METRIC3
Whilst User 2 could choose:
ORDER BY
METRIC2
,METRIC3
,METRIC1
How would I accept this user input and use it to create a query on the Team table?
Edit 1 Neglected to mention (sorry) that the metrics themselves are calculated on the fly. Currently they are instance methods (e.g #team.metric1 etc). The abortive attempts I have made so far all involve trying to convert user strings to method names which just seems wrong (and I haven't been able to get it to work).
Edit 2 some example code in teams_controller.rb:
class Team < ApplicationRecord
belongs_to :tournament
has_many :matches
def score_for
matches.sum(:score_for)
end
def score_diff
matches.sum(:score_for) - matches.sum(:score_against)
end
end
ActiveRecord allows multiple arguments to be passed to the order method. So you could do something like:
Team.order(:metric2, :metric3, metric1: :desc)
Another options is you can also use ActiveRecord to dynamically construct a query. ActiveRecord queries are lazily evaluated, so the SQL won't be executed until you call an operation that requires loading the records.
For example you could construct a scope on Team like this:
class Team < ApplicationRecord
scope :custom_order, lambda { |sorting_order|
sorting_order.each do |metric|
order(metric)
end
}
end
You would then just need to input a collection of attributes in the order you wanted the order by clauses to be executed. For example:
Team.custom_order([:metric2, :metric3, :metric1])
A working but probably awful solution:
class Tournament < ApplicationRecord
has_many :teams
serialize :tiebreaker, Array
TIEBREAKER_WHITELIST = %w[score opponent_score possession].freeze
def sorted_teams
list = teams.shuffle
(TIEBREAKER_WHITELIST & tiebreaker).reverse.each do |metric|
list = list.sort_by { |team| [team.send(metric), list.find_index(team)] }
end
list.reverse
end
end
Each tournament has many teams. A tournament instance has a serialized field called tiebreaker. This contains an array of strings something like ["score", "possession"] where each string matches the name of a public instance method on team. Each of these methods returns a number.
The tiebreaker field is in descending order of precedence, so for the above example I would only expect possession to affect sorting for teams with an equal score.
list = teams.shuffle - this randomises the list to start with, in case teams are tied for all of the following tiebreakers.
(TIEBREAKER_WHITELIST & tiebreaker) - this returns only strings that appear in both the tiebreaker field and the whitelist constant to protect against end users running arbitrary methods.
.reverse.each do |metric| - this reverses the array of metrics so that the list is sorted by the lowest precedence metric first.
[team.send(metric), list.find_index(team)] - this is the sort for each metric. send turns the string into a method call. I found find_indexwas necessary to preserver sort order from previous sorts. i.e. if I had first sorted for possession this would preserve the order for teams with the same score.
list.reverse - reverse the list then return it. This was because I wanted higher scoring/possession teams first on my list and sort_by sorts ascending.
I wanted some metrics sorted ascending (opponent_score) and others descending (score) so I handled this in the respective methods, returning negative values for opponent_score for example.
I'm not entirely happy with the solution as is but it does seem to work!
for a data analysis i need both results into one set.
a.follower_trackings.pluck(:date, :new_followers, :deleted_followers)
a.data_trackings.pluck(:date, :followed_by_count)
instead of ugly-merging an array (they can have different starting dates and i obv. need only those values where the date exists in both arrays) i thought about mysql
SELECT
followers.new_followers,
followers.deleted_followers,
trackings.date,
trackings.followed_by_count
FROM
instagram_user_follower_trackings AS followers,
instagram_data_trackings AS trackings
WHERE
followers.date = trackings.date
AND
followers.user_id=5
AND
trackings.user_id=5
ORDER
BY trackings.date DESC
This is Working fine, but i wonder if i can write the same with ActiveRecord?
You can do the following which should render the same query as your raw SQL, but it's also quite ugly...:
a.follower_trackings.
merge(a.data_trackings).
from("instagram_user_follower_trackings, instagram_data_trackings").
where("instagram_user_follower_trackings.date = instagram_data_trackings.date").
order(:date => :desc).
pluck("instagram_data_trackings.date",
:new_followers, :deleted_followers, :followed_by_count)
There are a few tricks turned out useful while playing with the scopes: the merge trick adds the data_trackings.user_id = a.id condition but it does not join in the data_trackings, that's why the from clause has to be added, which essentially performs the INNER JOIN. The rest is pretty straightforward and leverages the fact that order and pluck clauses do not need the table name to be specified if the columns are either unique among the tables, or are specified in the SELECT (pluck).
Well, when looking again, I would probably rather define a scope for retrieving the data for a given user (a record) that would essentially use the raw SQL you have in your question. I might also define a helper instance method that would call the scope with self, something like:
def Model
scope :tracking_info, ->(user) { ... }
def tracking_info
Model.tracking_info(self)
end
end
Then one can use simply:
a = Model.find(1)
a.tracking_info
# => [[...], [...]]
I am developing a rails 4 app using ActiveRecord models for my db tables.
The main issue is that my model is quite complicated, and I would like to display a lot of information when I do an index of the main object.
For example, let's assume I have the following tables and columns:
Person: name(string)
Address: address(string), person_id(int)
EmailAddress: email(string), person_id(int)
Email: spam (boolean), email_address_id(int)
and the relations:
person has_many: :email_addresses
person has_one: :address
email_address has_many: :emails
Now I would like to display the following information
person.name
person.address.name
person.email_addresses.count
person.email_addresses.map do |email_address|
email_address.email.where(spam: false).count
end
The main issue is that I have a big amount of records, and I don't want to instantiate all of them (I have some memory issues because of that). Therefore, I was wondering how to do this kind of thing directly to get either an array of hashes or of arrays.
I managed to get the beginning using pluck:
Person.joins(:address).pluck('persons.name, addresses.address')
The problem begins with the count part.
Has someone already encountered such a situation? And is there a way to do this without writing the complete SQL query?
You can't use pluck for complex queries, but you can always use select to fetch the columns you want. First you join all the tables you need. Note I joined emails table twice, the second one with the spam: false condition. Then you define your columns, directly from the table or COUNT'ed, in your select statement:
persons = Person.joins(:address, email_addresses: :emails).
joins('INNER JOIN emails not_spammy_email_addresses ON emails.email_address_id = email_addresses.id AND emails.spam = 0').
select('persons.name, addresses.address AS address_address,
COUNT(email_addresses.id) AS email_addresses_count,
COUNT(not_spammy_email_addresses.id) AS not_spammy_email_addresses_count')
And then call your result's columns like this:
person = persons.first
person.name
person.address_address # note I'm not using *address* to prevent conflict with the model Adress
person.email_addresses_count
person.not_spammy_email_addresses_count
I believe this is as far as you can get with active_record and a single query, but I'd love to see other approaches. For instance, if you use Arel this query would feel less SQLish.
I have a massive table Foo from which I need to pluck all values in a certain field, Foo.who.
The array has millions of rows, but only a few thousand different values in the who column.
If the table was smaller of course I'd simply use Foo.pluck(:who)
If I use Foo.find_in_batches do |a_batch| each set is an Array of Foo records, rather than an activerecord collection of Foo records, so I cannot use .pluck() and AFAIK the only way to extract the who column is via something like .map(&:who) that iterates over the array.
Is there a way to pluck the who column from Foo in batches that does not require then iterating over each element of each batch to extract the who column?
In Rails 5 you can use:
Foo.in_batches do |relation|
values = relation.pluck(:id, :name, description)
...
end
Upd: for prevent memory leaks use:
Foo.uncached do
Foo.in_batches do |relation|
values = relation.pluck(:id, :name, description)
...
end
end
Here's a method to get the ids that were retrieved by the in_batches method itself, without need to run another query yourself.
in_batches already runs pluck(:id) under the hood (when load param is false which is the default) and yield the relation object. This relation object is created with where(id: ids_from_pluck).
Is it possible to get the list of ids directly from the relation object via where_values_hash method, without the need to run another query in DB. For example:
Foo.in_batches do |relation|
ids = relation.where_values_hash['id']
end
This should work on both Rails 5.x and 6.x, but it relies on implementation detail of in_batches so it is not guaranteed to work in future.
Try this:
Foo.select(:id, :who).find_in_batches do |a_batch|
...
end
I have three rails objects: User, DemoUser and Stats. Both the User and the DemoUser have many stats associated with them. The User and Stats tables are stored on Postgresql (using ActiveRecord). The DemoUser is stored in redis. The id for the DemoUser is a (random) string. The id for the User is a (standard-rails) incrementing integer.
The stats table has a user_id column that can contain either the User id or the DemoUser id. For that reason, the user_id column is a string, rather than an integer.
There isn't an easy way to translate from the random string to an integer, but there's a very easy way to translate the integer id to a string (42 -> "42"). The ids are guaranteed not to overlap (there won't be a User instance with the same id as a DemoUser, ever).
I have some code that manages those stats. I'd like to be able to pass over a some_user instance (which can either be a DemoUser or a User) and then be able to use the id to fetch Stats, update them etc. Also would be nice to be able to define a has_many for the User model, so I can do things like user.stats
However, operations like user.stats would create a query like
SELECT "stats".* FROM "stats" WHERE "stats"."user_id" = 42
which then breaks with PG::UndefinedFunction: ERROR: operator does not exist: character varying = integer
Is there a way to either let the database (Postgresql), or Rails do auto-translation of the ids on JOIN? (the translation from integer to string should be simple, e.g. 42 -> "42")
EDIT: updated the question to try to make things as clear as possible. Happy to accept edits or answer questions to clarify anything.
You can't define a foreign key between two types that don't have built-in equality operators.
The correct solution is to change the string column to be an integer.
In your case you could create a user-defined = operator for varchar = string, but that would have messy side effects elsewhere in the database; for example, it would allow bogus code like:
SELECT 2014-01-02 = '2014-01-02'
to run without an error. So I'm not going to give you the code to do that. If you truly feel it's the only solution (which I don't think is likely to be correct) then see CREATE OPERATOR and CREATE FUNCTION.
One option would be to have separate user_id and demo_user_id columns in your stats table. The user_id would be an integer that you could use as a foreign key to the users table in PostgreSQL and the demo_user_id would be a string that would link to your Redis database. If you wanted to treat the database properly, you'd use a real FK to link stats.user_id to users.id to ensure referential integrity and you'd include a CHECK constraint to ensure that exactly one of stats.user_id and stats.demo_user_id was NULL:
check (user_id is null <> demo_user_id is null)
You'll have to fight ActiveRecord a bit to properly constrain your database of course, AR doesn't believe in fancy things like FKs and CHECKs even though they are necessary for data integrity. You'd have to keep demo_user_id under control by hand though, some sort of periodic scan to make sure they link up with values in Redis would be a good idea.
Now your User can look up stats using a standard association to the stats.user_id column and your DemoUser can use stats.demo_user_id.
For the time being, my 'solution' is not to use a has_many in Rails, but I can define some helper functions in the models if necessary. e.g.
class User < ActiveRecord::Base
# ...
def stats
Stats.where(user_id: self.id.to_s)
end
# ...
end
also, I would define some helper scopes to help enforce the to_s translation
class Stats < ActiveRecord::Base
scope :for_user_id, -> (id) { where(user_id: id.to_s) }
# ...
end
This should allow calls like
user.stats and Stats.for_user_id(user.id)
I think I misunderstood a detail of your issue before because it was buried in the comments.
(I strongly suggest editing your question to clarify points when comments show that there's something confusing/incomplete in the question).
You seem to want a foreign key from an integer column to a string column because the string column might be an integer, or might be some unrelated string. That's why you can't make it an integer column - it's not necessarily a valid number value, it might be a textual key from a different system.
The typical solution in this case would be to have a synthetic primary key and two UNIQUE constraints instead, one for keys from each system, plus a CHECK constraint preventing both from being set. E.g.
CREATE TABLE my_referenced_table (
id serial,
system1_key integer,
system2_key varchar,
CONSTRAINT exactly_one_key_must_be_set
CHECK (system1_key IS NULL != system2_key IS NULL),
UNIQUE(system1_key),
UNIQUE(system2_key),
PRIMARY KEY (id),
... other values ...
);
You can then have a foreign key referencing system1_key from your integer-keyed table.
It's not perfect, as it doesn't prevent the same value appearing in two different rows, one for system1_key and one for system2_key.
So an alternative might be:
CREATE TABLE my_referenced_table (
the_key varchar primary key,
the_key_ifinteger integer,
CONSTRAINT integerkey_must_equal_key_if_set
CHECK (the_key_ifinteger IS NULL OR (the_key_ifinteger::varchar = the_key)),
UNIQUE(the_key_ifinteger),
... other values ...
);
CREATE OR REPLACE FUNCTION my_referenced_table_copy_int_key()
RETURNS trigger LANGUAGE plpgsql STRICT
AS $$
BEGIN
IF NEW.the_key ~ '^[\d]+$' THEN
NEW.the_key_ifinteger := CAST(NEW.the_key AS integer);
END IF;
RETURN NEW;
END;
$$;
CREATE TRIGGER copy_int_key
BEFORE INSERT OR UPDATE ON my_referenced_table
FOR EACH ROW EXECUTE PROCEDURE my_referenced_table_copy_int_key();
which copies the integer value if it's an integer, so you can reference it.
All in all though I think the whole idea is a bit iffy.
I think I may have a solution for your problem, but maybe not a massively better one:
class User < ActiveRecord::Base
has_many :stats, primary_key: "id_s"
def id_s
read_attribute(:id).to_s
end
end
Still uses a second virtual column, but maybe more handy to use with Rails associations and is database agnostic.