In my company, we are trying to cache some data that we are querying from an API. We are using Rails. Two of my models are 'Query' and 'Response'. I want to create a one-to-many relationship between Query and Response, wherein, one query can have many responses.
I thought this is the right way to do it.
Query = [query]
Response = [query_id, response_detail_1, response_detail_2]
Then, in the Models, I did the following Data Associations:
class Query < ActiveRecord::Base
has_many :response
end
class Response < ActiveRecord::Base
belongs_to :query
end
So, canonically, whenever I want to find all the responses for a given query, I would do -
"_id" = Query.where(:query => "given query").id
Response.where(:query_id => "_id")
But my boss made me use an Array column in the Query model, remove the Data Associations between the models and put the id of each response record in that array column in the Query model. So, now the Query model looks like
Query = [query_id, [response_id_1, response_id_2, response_id_3,...]]
I just want to know what are the merits and demerits of doing it both ways and which is the right way to do it.
If the relationship is really a one-to-many relationship, the "standard" approach is what you originally suggested, or using a junction table. You're losing out on referential integrity that you could get with a FK by using the array. Postgres almost had FK constraints on array columns, but from what I researched it looks like it's not currently in the roadmap:
http://blog.2ndquadrant.com/postgresql-9-3-development-array-element-foreign-keys/
You might get some performance advantages out of the array approach if you consider it like a denormalization/caching assist. See this answer for some info on that, but it still recommends using a junction table:
https://stackoverflow.com/a/17012344/4280232. This answer and the comments also offer some thoughts on the array performance vs the join performance:
https://stackoverflow.com/a/13840557/4280232
Another advantage of using the array is that arrays will preserve order, so if order is important you could get some benefits there:
https://stackoverflow.com/a/2489805/4280232
But even then, you could put the order directly on the responses table (assuming they're unique to each query) or you could put it on a join table.
So, in sum, you might get some performance advantages out of the array foreign keys, and they might help with ordering, but you won't be able to enforce FK constraints on them (as of the time of this writing). Unless there's a special situation going on here, it's probably better to stick with the "FK column on the child table" approach, as that is considerably more common.
Granted, that all applies mainly to SQL databases, which I notice now you didn't specify in your question. If you're using NoSQL there may be other conventions for this.
Related
I have a Rails application that includes tables for surveys and (survey) questions.
A survey has_many questions
A question belongs_to a survey
The surveys are inherently teacher surveys. Now we are introducing student surveys, which are meaningfully different from teacher surveys, with different types of information that we need to store about them, such that they seem to each warrant their own table/model, so I'm thinking we want separate tables for teacher_surveys and student_surveys.
However, the questions are really pretty much the same. Includes things like the question text, type of question (text, checkbox, dropdown), etc. So it seems like questions should remain a single table.
How do I best model this data?
Should the questions table have a teacher_survey_id and a student_survey_id where each is optional but one of the two of them is required?
Should I have join tables for questions_teacher_surveys and questions_student_surveys?
Something else?
There is no easy answer to this question. Separating questions into student_question and teach_question tables does mean you have a slight bit of duplication and you can't query them as a homogenized collection if that happens to be important.
The code duplication can be very simply addressed by using inheritance or composition but there is an increased maintainence burdon / complexity cost here.
But it does come with the advantage that you get a guarentee of referential integrity from the non-nullable foreign key column without resorting to stuff like creating a database trigger or the lack of real foreign key constraints if you choose to use a polymorphic association.
An additional potential advantage is that you're querying smaller tables that can remain dense instead of sparse if the requirements diverge.
sounds like you need to make your surveys table polymorphic and add a type column as an enum with :student and :teacher as options, you can use the type column as a flag to control the different business logic. Once the survey table becomes polymorphic you probably wont need to do anything with your questions table. If you decide to go with this solution its also recommend to add a concern class named Surveyable to hold the needed associations and shared logic between your surveyed models (in your case students and teachers).
Say I have the model Item which has one Foo and many Bars.
Foo and Bar can be used as parameters when searching for Items and so Items can be searched like so:
www.example.com/search?foo=foovalue&bar[]=barvalue1&bar[]=barvalue2
I need to generate a Query object that is able to save these search parameters. I need the following relationships:
Query needs to access one Foo and many Bars.
One Foo can be accessed by many different Queries.
One Bar can be accessed by many different Queries.
Neither Bar nor Foo need to know anything about Query.
I have this relationship set up currently like so:
class Query < ActiveRecord::Base
belongs_to :foo
has_and_belongs_to_many :bars
...
end
Query also has a method which returns a hash like this: { foo: 'foovalue', bars: [ 'barvalue1', 'barvalue2' } which easily allows me to pass these values into a url helper and generate the search query.
This all works fine.
My question is whether this is the best way to set up this relationship. I haven't seen any other examples of one-way HABTM relationships so I think I may be doing something wrong here.
Is this an acceptable use of HABTM?
Functionally yes, but semantically no. Using HABTM in a "one-sided" fashion will achieve exactly what you want. The name HABTM does unfortunately insinuate a reciprocal relationship that isn't always the case. Similarly, belongs_to :foo makes little intuitive sense here.
Don't get caught up in the semantics of HABTM and the other association, instead just consider where your IDs need to sit in order to query the data appropriately and efficiently. Remember, efficiency considerations should above all account for your productivity.
I'll take the liberty to create a more concrete example than your foos and bars... say we have an engine that allows us to query whether certain ducks are present in a given pond, and we want to keep track of these queries.
Possibilities
You have three choices for storing the ducks in your Query records:
Join table
Native array of duck ids
Serialized array of duck ids
You've answered the join table use case yourself, and if it's true that "neither [Duck] nor [Pond] need to know anything about Query", using one-sided associations should cause you no problems. All you need to do is create a ducks_queries table and ActiveRecord will provide the rest. You could even opt to use has_many :through relationship if you need to do anything fancy.
At times arrays are more convenient than using join tables. You could store the data as a serialized integer array and add handlers for accessing the data similar to the following:
class Query
serialize :duck_ids
def ducks
transaction do
Duck.where(id: duck_ids)
end
end
end
If you have native array support in your database, you can do the same from within your DB. similar.
With Postgres' native array support, you could make a query as follows:
SELECT * FROM ducks WHERE id=ANY(
(SELECT duck_ids FROM queries WHERE id=1 LIMIT 1)::int[]
)
You can play with the above example on SQL Fiddle
Trade Offs
Join table:
Pros: Convention over configuration; You get all the Rails goodies (e.g. query.bars, query.bars=, query.bars.where()) out of the box
Cons: You've added complexity to your data layer (i.e. another table, more dense queries); makes little intuitive sense
Native array:
Pros: Semantically nice; you get all the DB's array-related goodies out of the box; potentially more performant
Cons: You'll have to roll your own Ruby/SQL or use an ActiveRecord extension such as postgres_ext; not DB agnostic; goodbye Rails goodies
Serialized array:
Pros: Semantically nice; DB agnostic
Cons: You'll have to roll your own Ruby; you'll loose the ability to make certain queries directly through your DB; serialization is icky; goodbye Rails goodies
At the end of the day, your use case makes all the difference. That aside, I'd say you should stick with your "one-sided" HABTM implementation: you'll lose a lot of Rails-given gifts otherwise.
I have two ActiveRecord models: Post and Vote. I want a make a simple query:
SELECT *,
(SELECT COUNT(*)
FROM votes
WHERE votes.id = posts.id) AS vote_count
FROM posts
I am wondering what's the best way to do it in activerecord DSL. My goal is to minimize the amount of SQL I have to write.
I can do Post.select("COUNT(*) from votes where votes.id = posts.id as vote_count")
Two problems with this:
Raw SQL. Anyway to write this in DSL?
This returns only attribute vote_count and not "*" + vote_count. I can append .select("*") but I will be repeating this every time. Is there an much better/DRY way to do this?
Thanks
Well, if you want to reduce amount of SQL, you can split that query into smaller two end execute them separately. For instance, the votes counting part could be extracted to query:
SELECT votes.id, COUNT(*) FROM votes GROUP BY votes.id;
which you may write with ActiveRecord methods as:
Vote.group(:id).count
You can store the result for later use and access it directly from Post model, for example you may define #votes_count as a method:
class Post
def votes_count
##votes_count_cache ||= Vote.group(:id).count
##votes_count_cache[id] || 0
end
end
(Of course every use of cache raises a question about invalidating or updating it, but this is out of the scope of this topic.)
But I strongly encourage you to consider yet another approach.
I believe writing complicated queries like yours with ActiveRecord methods — even if would be possible — or splitting queries into two as I proposed earlier are both bad ideas. They result in extremely cluttered code, far less readable than raw SQL. Instead, I suggest introducing query objects. IMO there is nothing wrong in using raw, complicated SQL when it's hidden behind nice interface. See: M. Fowler's P of EAA and Brynary's post on Code Climate Blog.
How about doing this with no additional SQL at all - consider using the Rails counter_cache feature.
If you add an integer votes_count column to the posts table, you can get Rails to automatically increment and decrement that counter by changing the belongs_to declaration in Vote to:
belongs_to :post, counter_cache: true
Rails will then keep each Post updated with the number of votes it has. That way the count is already calculated and no sub-query is needed.
Maybe you can create mysql view and just map it to new AR model. It works similar way to table, you just need to specify with set_table_name "your_view_name"....maybe on DB level it will work faster and will be automatically re-calculating.
Just stumbled upon postgres_ext gem which adds support for Common Table Expressions in Arel and ActiveRecord which is exactly what you asked. Gem is not for SQLite, but perhaps some portions could be extracted or serve as examples.
I watched this rails cast http://railscasts.com/episodes/22-eager-loading but still I have some confusions about what is the best way of writing an efficient GET REST service for a scenario like this:
Let's say we have an Organization table and there are like twenty other tables that there is a belongs_to and has_many relations between them. (so all those tables have a organization_id field).
Now I want to write a GET and INDEX request in form of a Rails REST service that based on the organization id being passed to the request in URL, it can go and read those tables and fill the JSON BUT NOT for ALL of those table, only for a few of them, for example let's say for a Patients, Orders and Visits table, not all of those twenty tables.
So still I have trouble with getting my head around how to write such a
.find( :all )
sort of query ?
Can someone show some example so I can understand how to do this sort of queries?
You can include all of those tables in one SQL query:
#organization = Organization.includes(:patients, :orders, :visits).find(1)
Now when you do something like:
#organization.patients
It will load the patients in-memory, since it already fetched them in the original query. Without includes, #organization.patients would trigger another database query. This is why it's called "eager loading", because you are loading the patients of the organization before you actually reference them (eagerly), because you know you will need that data later.
You can use includes anytime, whether using all or not. Personally I find it to be more explicit and clear when I chain the includes method onto the model, instead of including it as some sort of hash option (as in the Railscast episode).
Good morning,
I have a Rails model in which I’m currently serializing an array of information. Two things are important to me:
I want to be able to ensure that this is unique (i.e. can’t have two models with the same array)
I want to be able to search existing models for this hash (in a type of find_or_create_by method).
This model describes a “portfolio” – i.e. a group of stock or bonds. The array is the description of what securities are inside the portfolio, and in what weights. I also have a second model, which is a group of portfolios (lets call it a “Portcollection” to keep things simple). A collection has many portfolios, and a portfolio can be in many collections. In other words:
class Portfolio
serialize :weights
has_and_belongs_to_many :portcollections
class Portcollection
has_and_belongs_to_many :portfolios
When I am generating a “portcollection” I need to build a bunch of portfolios, which I do programmatically (implementation not important). Building a portfolio is an expensive operation, so I’m trying to check for the existence of one first. I thought I could do this via find_or_create_by, but wasn’t having much luck. This is my current solution:
Class Portcollection
before_save :build_portfolios
def build_portfolios
……
proposed_weights = ……
yml =proposed_weights.to_yaml
if port = Portfolio.find_by_weights(yml)
self.portfolios << port
else
self.portfolios << Portfolio.create!(:weights => proposed_weights)
end
……..
end
This does work, but it is quite slow. I have a feeling this is because I’m converting stuff to YAML each time it runs when I try to check for an existing portfolio (this is running probably millions of times), and I’m searching for a string, as opposed to an integer. I do have an index on this column though.
Is there a better way to do this? A few thoughts had crossed my mind:
Calculate an MD5 hash of the “weights” array, and save to a database column. I’ll still have to calculate this hash each time I want to search for an array, but I have a gut feeling this would be easier for the database to index & search?
Work on moving from has_and_belongs_to_many to a has_many => through, and store the array information as database columns. That way I could try to sort out a database query that could check for the uniqueness, without any YAML or serialization…
i.e. something like :
class Portfolio
has_many :portcollections, :through => security_weights
class Portcollections
has_many :portfolios, :through => security_weights
SECURITY_WEIGHTS
id portfolio_id portcollection_id weight_of_GOOG weight_of_APPLE ……
1 14 15 0.4 0.3
In case it is important, the “weights” array would look like this:
[ [‘GOOG’, 0.4] , [‘AAPL’, 0.3] , [‘GE’, 0.3] ]
Any help would be appreciated. Please keep in mind I'm quite an amateur - programming is just a hobby for me! Please excuse me if I'm doing anything really hacky or missing something obvious....
Thanks!
UPDATE 1
I've done some research into the Rails 3.2 "store" method, but that doesn't seem to be the answer either... It just stores objects as JSON, which gives me the same lack of searchability I have now.
I think storing a separate hash in it's own column is the only way to do this efficiently. You are using serialization or a key/value store that is designed to not be easily searchable.
Just make sure you consider sorting on your values before hashing them, other wise you could have the same content but differing hashes.