I have run into a problem and can't figure out how to fix it.
I basically have a ModelA that has a has_many relationship with another ModelB. Let's say that ModelB has two attributes. A created_at and a filename. Now let's say the created_at is not actually accurate due to a race condition. However the filename contains a timestamp that is accurate. Unfortunately the filename is also inconsistent and extracting and adding the timestamp beforehand is not an option.
Is there any way to do something like this?
class ModelA
has_many: model_bs, order: extracted_timestamp_from_filename.asc
def extracted_timestamp_from_filename
#self would ideally be a single model_bs
extract_timestamp_from_filename(self.filename)
end
end
At least with order I don't think this could ever work without having an actual proper timestamp attribute. Is there a way to get this functionality? If there is a way to do this with hooks somehow I haven't found it.
This is not really a fix but turns out Rails with a MongoDB is actually able to order by strings and it so happens that the files in my example can be ordered like this. However if the filenames have different formats this will still not work. In my example I think I'll get away with it for now.
Thanks.
Related
I have a model which has over 40,000 entires in it. I want to be able to have this table permanently sorted by one of its attributes. The tricky part of this is that some of the elements have a nil value for the attribute I want to sort by.
Some poking around has led me to default_scope, but it appears this is being deprecated and everyone warns against it. It seems like putting default_scope order('director_id DESC') or something like this would fix things, but this doesn't take into account nil values. What is the better alternative?
Thanks!
EDIT
I'm also using Tire with ElasticSearch for managing searches.
Yes, it's best to be explicit with model scopes. You can just do:
class MyModel < ActiveRecord::Base
def self.default_order
order('director_id DESC NULLS LAST')
end
end
Your database will have a syntax as part of ORDER BY for the placement of NULL values. If you don't want NULL values in the output at all then you can add a where call and the method should be renamed.
Can I order my users in the database, so I don't have to say order_by("created_at desc") each time I query?
Sounds for me like a logical thing to do, but I don't know if it's possible and if it's best practice?
SOLUTION
I'm already using the default_scope and as I understand it from you, it is the best way to do it? Thanks a lot for the answers though.
If you are after results sorted by create date desc, the reverse natural order will be close to this (but not guaranteed to be identical).
If you want a specific ordering, adding order_by() to an indexed query is the best way to assure this.
If you are using the default generated ObjectIds the first 4-bytes are actually a unix timestamp (seconds since the epoch) .. and the _id field is indexed by default aside from a few exceptions noted in the documentation.
So a query like last 50 users created (based on ObjectId) in the mongo shell would be:
db.users.find().sort({_id:-1}).limit(50)
There are mixed views about default scopes, but to achieve what you're asking:
http://apidock.com/rails/ActiveRecord/Base/default_scope/class
class User < ActiveRecord::Base
default_scope order('created_at DESC')
### other model code here ###
end
you should be able to add an index or indexes to your db table. Be careful with running this on a live system as the overhead for creating an index on a large table can be disabling.
EDIT: should have expanded.
By creating an index, you will still have to order, but your ordering/sorting will be more efficient.
ref: Is it okay to add database indexes to a database that already has data?
Good morning,
I have a Rails model in which I’m currently serializing an array of information. Two things are important to me:
I want to be able to ensure that this is unique (i.e. can’t have two models with the same array)
I want to be able to search existing models for this hash (in a type of find_or_create_by method).
This model describes a “portfolio” – i.e. a group of stock or bonds. The array is the description of what securities are inside the portfolio, and in what weights. I also have a second model, which is a group of portfolios (lets call it a “Portcollection” to keep things simple). A collection has many portfolios, and a portfolio can be in many collections. In other words:
class Portfolio
serialize :weights
has_and_belongs_to_many :portcollections
class Portcollection
has_and_belongs_to_many :portfolios
When I am generating a “portcollection” I need to build a bunch of portfolios, which I do programmatically (implementation not important). Building a portfolio is an expensive operation, so I’m trying to check for the existence of one first. I thought I could do this via find_or_create_by, but wasn’t having much luck. This is my current solution:
Class Portcollection
before_save :build_portfolios
def build_portfolios
……
proposed_weights = ……
yml =proposed_weights.to_yaml
if port = Portfolio.find_by_weights(yml)
self.portfolios << port
else
self.portfolios << Portfolio.create!(:weights => proposed_weights)
end
……..
end
This does work, but it is quite slow. I have a feeling this is because I’m converting stuff to YAML each time it runs when I try to check for an existing portfolio (this is running probably millions of times), and I’m searching for a string, as opposed to an integer. I do have an index on this column though.
Is there a better way to do this? A few thoughts had crossed my mind:
Calculate an MD5 hash of the “weights” array, and save to a database column. I’ll still have to calculate this hash each time I want to search for an array, but I have a gut feeling this would be easier for the database to index & search?
Work on moving from has_and_belongs_to_many to a has_many => through, and store the array information as database columns. That way I could try to sort out a database query that could check for the uniqueness, without any YAML or serialization…
i.e. something like :
class Portfolio
has_many :portcollections, :through => security_weights
class Portcollections
has_many :portfolios, :through => security_weights
SECURITY_WEIGHTS
id portfolio_id portcollection_id weight_of_GOOG weight_of_APPLE ……
1 14 15 0.4 0.3
In case it is important, the “weights” array would look like this:
[ [‘GOOG’, 0.4] , [‘AAPL’, 0.3] , [‘GE’, 0.3] ]
Any help would be appreciated. Please keep in mind I'm quite an amateur - programming is just a hobby for me! Please excuse me if I'm doing anything really hacky or missing something obvious....
Thanks!
UPDATE 1
I've done some research into the Rails 3.2 "store" method, but that doesn't seem to be the answer either... It just stores objects as JSON, which gives me the same lack of searchability I have now.
I think storing a separate hash in it's own column is the only way to do this efficiently. You are using serialization or a key/value store that is designed to not be easily searchable.
Just make sure you consider sorting on your values before hashing them, other wise you could have the same content but differing hashes.
Update:
Originally, this post was using Books as the example entity, with
Books1, Books2, etc. being the
separated table. I think this was a
bit confusing, so I've changed the
example entity to be "private
todo_items created by a particular
user."
This kind of makes Horace and Ryan's original comments seem a bit off, and
I apologize for that. Please know that
their points were valid when it looked
like I was dealing with books.
Hello,
I've decided to use multiple tables for an entity (e.g. todo_items1, todo_items2, todo_items3, etc.), instead of just one main table which could end up having a lot of rows (e.g. just todo_items). I'm doing this to try and to avoid a potential future performance drop that could come with having too many rows in one table.
With that, I'm looking for a good way to handle this in Rails, mainly by trying to avoid loading a bunch of unused associations for each User object. I'm guessing that other have done something similar, so there's probably a few good tips/recommendations out there.
(I know that I could use a partition for this, but, for now, I've decided to go the 'multiple tables' route.)
Each user has their todo_items placed into a specific table. The actual "todo items" table is chosen when the user is created, and all of their todo_items go into the same table. The data in their todo items collection is private, so when it comes time to process a users todo_items, I'll only have to look at one table.
One thing I don't particularly want to have is a bunch of unused associations in the User class. Right now, it looks like I'd have to do the following:
class User < ActiveRecord::Base
has_many :todo_items1, :todo_items2, :todo_items3, :todo_items4, :todo_items5
end
class todo_items1 < ActiveRecord::Base
belongs_to :user
end
class todo_items2 < ActiveRecord::Base
belongs_to :user
end
class todo_items3 < ActiveRecord::Base
belongs_to :user
end
The thing is, for each individual user, only one of the "todo items" tables would be usable/applicable/accessible since all of a user's todo_items are stored in the same table. This means only one of the associations would be in use at any time and all of the other has_many :todo_itemsX associations that were loaded would be a waste.
For example, with a user.id of 2, I'd only need todo_items3.find_by_text('search_word'), but the way I'm thinking of setting this up, I'd still have access to todo_items1, todo_items2, todo_items4 and todo_items5.
I'm thinking that these "extra associations" adds extra overhead and makes each User object's size in memory much bigger than it has to be. Also, there's a bunch of stuff that Ruby/Rails is doing in the background which may cause other performance problems.
I'm also guessing that there could be some additional method call/lookup overhead for each User object, since it has to load all of those associations, which in turn creates all of those nice, dynamic model accessor methods like User.find_by_something.
I don't really know Ruby/Rails does internally with all of those has_many associations though, so maybe it's not so bad. But right now I'm thinking that it's really wasteful, and that there may just be a better, more efficient way of doing this.
So, a few questions:
1) Is there's some sort of special Ruby/Rails methodology that could be applied to this 'multiple tables to represent one entity' scheme? Are there any 'best practices' for this?
2) Is it really bad to have so many unused has_many associations for each object? Is there a better way to do this?
3) Does anyone have any advice on how to abstract the fact that there's multiple "todo items" tables behind a single todo_items model/class? For example, so I can call todo_items.find_by_text('search_phrase') instead of todo_items3.find_by_text('search_phrase').
Thank you!
This is not the way to scale.
It would probably be better going with master-slave replication and proper indexing (besides primary key) on fields such as "title" and/or "author" if that's what you're going to be looking up books based on. Having it in n-tables, how are you going to know the best place to go looking for the book the user is after? Are you going to go looking through 4 tables?
I agree with Horace: " don't try to solve a performance issue before you have figures to prove it." I suggest, however, that you should really look into adding indexes to your table if you want lookups to be fast. If they aren't fast, then tell us how they aren't fast and we will tell you how to make it go ZOOOOOM.
Let's assume this model:
Movie
- Title: String
- Has many:
- Alternative Title: String
My questions is, how should I store the alt. title attribute? I am deciding between three approaches:
Separate AR model: probably an overkill
CSV in a signle DB column
Serialized array in single DB column
The latter two seems logically equivilent. I am leaning towards the CSV approach. Can anyone give some advise on this? What would be the implications on speed and searchability?
If a movie can have many titles, it makes most sense to have a Title model and give the Movie model a has_many :titles relation, especially if you later on decide to add more metadata about titles. It may seem like overkill, but I think it will be the least hassle in the long run. Furthermore, I think that a movie's "main" title should be a Title object as well, perhaps with an is_main_title or similar attribute to distinguish it from the others.
If most of the time you only use the primary title, I'll go with your CSV option.
If most of the time you use all the titles, I'll put all the titles (primary and secondary) inside a single CSV column (named "titles") and just get the first when the primary is needed (with a helper function).
Why?
Because it makes things simple- and if the time has come, like Jordan said, that you need another attribute you can always migrate to a separate model.
Until then, YAGNI.
I would also vote for a separate model even though it seems like overkill it will allow you to basically follow the Rails way the easiest. However, if you choose not to reap the benefits of all the baked in magic associated with associations, then I would recommend YAML or JSON over CSV. CSV is quite simple, but Rails has baked in support for YAML serialization and would probably be the easiest solution. Check out RDoc on #serialize. For the given example this would basically amount to:
class Movie < ActiveRecord::Base
serialize :alternate_titles
end
With that, Rails would handle a lot of the drudgery for you and you'll have a nice array of alternate titles always available.