Update:
Originally, this post was using Books as the example entity, with
Books1, Books2, etc. being the
separated table. I think this was a
bit confusing, so I've changed the
example entity to be "private
todo_items created by a particular
user."
This kind of makes Horace and Ryan's original comments seem a bit off, and
I apologize for that. Please know that
their points were valid when it looked
like I was dealing with books.
Hello,
I've decided to use multiple tables for an entity (e.g. todo_items1, todo_items2, todo_items3, etc.), instead of just one main table which could end up having a lot of rows (e.g. just todo_items). I'm doing this to try and to avoid a potential future performance drop that could come with having too many rows in one table.
With that, I'm looking for a good way to handle this in Rails, mainly by trying to avoid loading a bunch of unused associations for each User object. I'm guessing that other have done something similar, so there's probably a few good tips/recommendations out there.
(I know that I could use a partition for this, but, for now, I've decided to go the 'multiple tables' route.)
Each user has their todo_items placed into a specific table. The actual "todo items" table is chosen when the user is created, and all of their todo_items go into the same table. The data in their todo items collection is private, so when it comes time to process a users todo_items, I'll only have to look at one table.
One thing I don't particularly want to have is a bunch of unused associations in the User class. Right now, it looks like I'd have to do the following:
class User < ActiveRecord::Base
has_many :todo_items1, :todo_items2, :todo_items3, :todo_items4, :todo_items5
end
class todo_items1 < ActiveRecord::Base
belongs_to :user
end
class todo_items2 < ActiveRecord::Base
belongs_to :user
end
class todo_items3 < ActiveRecord::Base
belongs_to :user
end
The thing is, for each individual user, only one of the "todo items" tables would be usable/applicable/accessible since all of a user's todo_items are stored in the same table. This means only one of the associations would be in use at any time and all of the other has_many :todo_itemsX associations that were loaded would be a waste.
For example, with a user.id of 2, I'd only need todo_items3.find_by_text('search_word'), but the way I'm thinking of setting this up, I'd still have access to todo_items1, todo_items2, todo_items4 and todo_items5.
I'm thinking that these "extra associations" adds extra overhead and makes each User object's size in memory much bigger than it has to be. Also, there's a bunch of stuff that Ruby/Rails is doing in the background which may cause other performance problems.
I'm also guessing that there could be some additional method call/lookup overhead for each User object, since it has to load all of those associations, which in turn creates all of those nice, dynamic model accessor methods like User.find_by_something.
I don't really know Ruby/Rails does internally with all of those has_many associations though, so maybe it's not so bad. But right now I'm thinking that it's really wasteful, and that there may just be a better, more efficient way of doing this.
So, a few questions:
1) Is there's some sort of special Ruby/Rails methodology that could be applied to this 'multiple tables to represent one entity' scheme? Are there any 'best practices' for this?
2) Is it really bad to have so many unused has_many associations for each object? Is there a better way to do this?
3) Does anyone have any advice on how to abstract the fact that there's multiple "todo items" tables behind a single todo_items model/class? For example, so I can call todo_items.find_by_text('search_phrase') instead of todo_items3.find_by_text('search_phrase').
Thank you!
This is not the way to scale.
It would probably be better going with master-slave replication and proper indexing (besides primary key) on fields such as "title" and/or "author" if that's what you're going to be looking up books based on. Having it in n-tables, how are you going to know the best place to go looking for the book the user is after? Are you going to go looking through 4 tables?
I agree with Horace: " don't try to solve a performance issue before you have figures to prove it." I suggest, however, that you should really look into adding indexes to your table if you want lookups to be fast. If they aren't fast, then tell us how they aren't fast and we will tell you how to make it go ZOOOOOM.
Related
I have a Notification model. Any time certain actions like Comment, Like, Mention, Follow happens this table gets a single record added to it. This table is now many millions of records long. The only index I have on it is on the user_id - which has been important as I show notifications to the current_user.
On each of these related models, I included the following line
has_one :notification, dependent: :destroy
The issue is that all these actions are reversible, so whenever someone, say, unlikes, I need to destroy the related Notification.
As such Notification#destroy action is very slow now, often taking >10secs! I'm certain this is because each time a destroy happens, it has to look up foreign keys like like_id, comment_id, or mention_id
I can add indexes on all these records, but I'm concerned given how big this table is and how often it gets INSERTS and DELETES. Should I be concerned? Is there a better way to structure this?
A possibility is considering flipping the relationship around, use belongs_to rather than has_one so you rely on only the primary key on that table.
Otherwise, you should really have indices on those; it shouldn't be too much of a burden. You'll get hammered on performance if you don't; it may even be locking the database on those deletes.
Use SHOW TABLE STATUS to see index sizes to get an idea of what you might be looking at.
I'm looking to write some sort of Rails app to help users book time slots at a restaurant. How can this be modeled in such a way so that it can be reservations can be displayed and booked through the browser? I have two models:
class Restaurant < ActiveRecord::Base
has_many :reservations
class Reservation < ActiveRecord::Base
belongs_to :restaurant
attr_accessible :name, :date, :time
At first I toyed with a using a hash within the Restaurant model to store availability, using dates as keys. But then I realized that Rails databases must serialize hashes, and I'd like to make sure there isn't a better way to go about this before I dive into that stuff.
I'm using Postgres (if that's relevant). Any help would be much appreciated!
Your basic model structure is fine. Note that attr_accessible is not current best practices, as of Rails 4. (It has been replaced by strong parameters)
Doesn't matter too much about what database you are using (even though PG is a solid choice) but the general engineering that is truly important.
I won't give you a copy paste answer but hopefully give you some direction.
So a Restaurant can have many Reservations throughout the day. I assume each restaurant can only hold so many people and thus have some type of "reservation limit" and Reservations cannot overlap.
Because of the constraints I imagine your 2 model method will work. You just need to figure out exactly how they must interact to work as you plan.
Restaurants should be able to keep track of open times/vacancies (or whatever important details). While Reservations will keep track of the number of people in the party, time, etc.
Your initial relationship looks to be well defined. But the other answer does correctly point out the new Rails 4 preferred method.
I am not sure exactly what I should name this question. I just started server-side programming and I need some help.
All the tutorials I have read so far on RoR deal with creating a pre-defined table and with pre-defined fields (id, name, email, etc etc). They use ActiveRecord as base class and saving to db is handled automatically by superclass.
What I am trying to program is something that allows user-defined table with fields. So think of this way. The web UI will have an empty table, the user will name the table, and add columns (field), and after that, add rows, and then later save it. How would I implement this? I am not asking for details, just an overview of it. As I said, all the tutorials I have read so far deal with pre-defined tables with fields where the ActiveRecord subclass is predefined.
So in a nutshell, I am asking, how to create tables in db on runtime, and add fields to the tables.
Hope I was clear, if not, please let me know and i will try to elaborate a bit more.
Thanks.
Unless you're building a DB administration tool (and even maybe then), allowing the user direct access to the database layer in the way you're suggesting is probably a bad idea. Apart from issues of stability and security, it'll get really slow if your users are creating lots of tables.
For instance, if you wanted to search for a certain value across 100 of your users' tables, you'd have to run 100 separate queries. The site would get exponentially slower the more user tables that were created.
A saner way to do it might be to have a Table model like this
class Table < ActiveRecord::Base
has_many :fields
has_many :rows
end
Every table would have fields attached to it, and rows to store the corresponding data (which would be encoded somehow).
However, as #Aditya rightly points out, this is not really beginner stuff!
I agree with previous answers generally speaking. It's not clear from your question why you want to create a table at runtime. It's not really obvious what the advantage of doing this would be. If you are just trying to store data that seems to fit into a table with rows and columns, why not just store it as an array in a field of your user table. If your user is allowed to create many tables, then you could have something like
class User < ActiveRecord::Base
has_many :tables
end
and then each table might have a field to store a serialized array. Or you could go with Alex's suggestion - the best choice really depends on what you are going to do with the data, how often it changes, whether you need to search it and so on ...
You can create a database as shown in tutorials which stores name of tables and their columns name those your user want. Then you can have worker (which can be build using Redis and Resque, here is simple Tut on Resque and Redis) and have those worker run migration (write migration with variables and use params to replace them) for you for new table in DB as soon as new entry is made in database. Tell me if you have questions on this.
I would like to create an application that would use the same system as backpack (http://www.backpackit.com) to create different types of pages.
Basically, you can add different elements to a page, and reorder them. Some elements can contains other elements (like an image gallery which contains... images, or lists etc).
I'm not sure how to model that.
I'd like to be able to do something like:
page.elements
without having to retrieve all elements myself
class Page < ActiveRecord::Base
has_many :texts, :dependent => :destroy
has_many :titles, :dependent => :destroy
def elements
#elts = texts + titles + ...
#order elts...
end
end
So I was thinking about single table inheritance.
I could have a Containers table, and Notes, Galleries, Lists etc could inherit from Containers.
And then, I would have Elements that could be linked to various Containers using polymorphism.
How would you do that? Do you see any fundamental flaws in my approach?
Thanks!
First off, the design is not as efficient as it could be, but whether or not it is fundamentally flawed actually depends on your level of experience:
Case 1: You are relatively new to programming and trying to get started by reverse-engineering and implementing something you can see and understand (backpackit). If this is true then you cannot go wrong by diving in and using the ORM philosophy that database tables can be designed as if they were persisting classes. It will be inefficient, but you'll learn plenty by not having to worry about the database -- yet.
Case 2: You are a veteran programmer (at least one decent app actually being used by people who paid for it) and for some reason are still expressing database design questions in object-oriented terminology. Then you have a fundamental flaw only because there is a good chance you will experience success that will stress the system, at which point the fundamental inefficiency of "table inheritance" will bite you.
I was wondering what the best way to model a relationship where an object is associated with exactly n objects of another class. I want to extend the has_one relationship to a specific value of n.
For example, a TopFiveMoviesList would belong to user and have exactly five movies. I would imagine that the underlying sql table would have fields like movie_id_1, movie_id_2, ... movie_id_5.
I know I could do a has_many relationship and limit the number of children at the model level, but I'd rather not have an intermediary table.
I think implementing this model through a join model is going to be you're best bet here. It allows the List model to worry about List logic and the Movie model to worry about Movie logic. You can create a Nomination (name isn't the greatest, but you know what I mean) model to handle the relationship between movies and lists, and when there's a limit of 5, you could just limit the number of nominations you pull back.
There are a few reasons I think this approach is better.
First, assuming you want to be able to traverse the relationships both ways (movie.lists and list.movies), the 5 column approach is going to be much messier.
While it'd be so much better for ActiveRecord to support has n relationships, it doesn't, and so you'll be fighting the framework on that one. Also, the has n relationship seems a bit brittle to me in this situation. I haven't seen that kind of implementation pulled off in ActiveRecord, though I'd be really interested in seeing it happen. :)
My first instinct would be to use a join table, but if that's not desirable User.movie[1-5]_id columns would fit the bill. (I think movie1_id fits better with Rails convention than movie_id_1.)
Since you tagged this Rails and ActiveRecord, I'll add some completely untested and probably somewhat wrong model code to my answer. :)
class User < ActiveRecord::Base
TOP_N_MOVIES = 5
(1..TOP_N_MOVIES).each { |n| belongs_to "movie#{n}".to_sym, :class_name => Movie }
end
You could wrap that line in a macro-style method, but unless if that's a common pattern for your application, doing that will probably just make your code that harder to read with little DRY benefit.
You might also want to add validations to ensure that there are no duplicate movies on a user's list.
Associating your movie class back to your users is similar.
class Movie < ActiveRecord::Base
(1..User::TOP_N_MOVIES).each do |n|
has_many "users_list_as_top_#{n}".to_sym, :class_name => User, :foreign_key => "movie#{n}_id"
end
def users_list_as_top_anything
ary = []
(1..User::TOP_N_MOVIES).each {|n| ary += self.send("users_list_as_top_#{n}") }
return ary
end
end
(Of course that users_list_as_top_anything would probably be better written out as explicit SQL. I'm lazy today.)
I assume you mean "implement" rather than "model"? The modeling's pretty easy in UML, say, where you have a Person entity that is made up of 5 Movie entities.
But the difficulty comes when you say has_one, going to has_5. If it's a simple scalar value, has_one is perhaps a property on the parent entity. Has_5 is probably 2 entities related to one another through an "is made up of" relationship in UML.
The main question to answer is probably, "Can you guarantee that it will always be 'Top 5'?" If yes, model it with columns, as you mentioned. If no, model it with another entity.
Another question is perhaps, "How easy will it be to refactor?" If it's simple, heck, start with 5 columns and refactor to separate entities if it ever changes.
As usual, "best" is dependent on the business and technical environment.