I'm geting towards the end of my application and have 25 lookup tables. Some of my domain classes have 15 table references as properties (one-to-one). Now I have just remembered/learned I can use enums instead. I'm sure I can refactor a bunch of those lookup tables to be enum classes. However, what are the best practices for using an enum vs. a lookup table? A few things about my application scenario that may assist in the answer:
There is only one developer (me)
This is a web based application
It won't be a heavily used application
When the view for the domain class with currently 15 lookups is rendered, all of those lookups will need to be loaded to display the data in the view. (15 joins)
The lookups will be cached in memory.
The lookup relationships to the domain class are all one-to-one
The data in the lookups will rarely change
Some of the values in the lookup tables are wordy, example: "Taking care of animals"
Some of the lookups have many records. Example: one lookup is U.S. states. Another is a person's occupation.
I think the most obvious answer is to use Enums for things that don't change - ever. Like Days of the week,
or Planets in the Solar System - ok so sometimes thnigs you thought wouldnt change actually changed, but that OK - a quick update and you are good to go for a few more years.
But when you are working with data that will certainly change over time and often enough, then by all means store that in the database. A change to a table will not require a code change and redeployment. Additionally it would be nice to add an admin interface to these - and scaffolded screens should be quick and easy enough.
This is just my opinion. I don't think there is a "right" answer here.
If it gets stored in a database I use a lookup table. It is more flexible and based on a few things, should more performant. While you do have to do join queries, you can lessen that impact by enabling 2nd level cache. cache: true In your mappings.
One thing to remember when using an Enum instead of a lookup is that there is no longer a foreign key to a separate table so you can't guarantee the values are in the list you desire from a database level. In addition, if you need to query based on the Enum, it is now a string comparison instead of a number comparison typically.
Related
I have setup a time-series / events database using the AWS Firehose -> S3/Glue -> Athena stack. It is being used to track various user actions - session started, action performed etc. across a number of our products. My question is about how best to store different types of IDs in this system.
The existing schema is one big 'fact table' with a bunch of different columns. Two of the most important columns are event_type_id and object_id. To use StackOverflow as an example, two events might be:
question_asked - in this case I would be storing the question id in the object_id column.
tag_created - in this case I would be storing the tag id in the object_id column.
My question is - is storing multiple different types of IDs in the same column bad practice? It's working OK for us at the moment, but it does require the person/system performing queries to know what type of object the object_id column refers to, based on the event they are querying.
If bad practice, what other approaches might be better? Multiple columns where they are NULL if not relevant for the event in that row? Or is this where dimension tables would be a better fit?
This isn't necessarily bad practice, depending on how you use it.
It sounds like you're aware of the potential pitfalls of such an approach (i.e. users of the data have to be aware of the context - in this case "event type" - to use the values correctly), so as you're using Athena you could mitigate that by creating views over source table for different event types, inserting a WHERE clause filter on event type and possibly renaming object_id to something more context specific e.g. question_id.
This makes it easier for users to work with the data and understand exactly what the values are they're working with.
In a big data environment I wouldn't recommend creating dimension tables if it can be avoided as JOINs between tables start to get expensive. Having multiple columns for different ids is possible but then you create new problems for users such as having to account for NULL values in an Id column, and this also potentially makes it harder to add new event types and ids as you have to change the schema to accommodate them.
We have to create a request system which will have roughly 10 different types of requests. All of these requests will belong to the 'accounting' aspect of our application. Therefore we've called them "Accounting requests".
All requests share maybe only a few columns and each has up to 20 columns individually.
We started to wonder if having separate tables for each request type would be practical in terms of speed when we start to have to do very complicated joins or queries, for example, fetching ALL requests types into a single table and then sorting it.
Maybe it would be easier to just use Single Table Inheritance since it will have a type column and we'd be using one table to store all 10 accounting request types.
What do you think regarding using STI for this many polymorphic associations and requirements?
Essentially, it would have models like so:
AccountingRequest
BillingRequest < AccountingRequest
CheckRequest < AccountingRequest
CancellationRequest < AccountingRequest
Each subclass has roughly 10+ fields.
Currently reading about Multiple Table Inheritance here. This seems like the solution that fits my requirements in this case. Not sure yet though.
STI is a good fit if your models all share the same attributes.
However if your sub classes start having attributes specific to them and not applicable to others, then STI can result in a lot of null columns. In that case, I usually prefer to go with polymorphic association.
This railscast episode is a great example of the difference between the 2
You can use STI in that situation. But making STI will require all the columns into one single table and that's not the good think. The table will go very large in the number of fields.
I think you should divide into two tables like as below...
Request: A request table will be the polymorphic table which saved the information for the type of requests.
RequestItem: The request item table will save all the 20 fields records into the table and will have a foreign key of request table. The request item table will have two fields into the database that's called key and value.
It sounds do-able.
When I've looked into this, I found that making extensive use of value objects helped to control the non-applicability of some attributes to some of the types.
In my case I had types of products, some of which would not have particular measurements for example. In those cases I used a Null Object to indicate "Not applicable" where appropriate.
Edit: I also found the composed_of syntax very convenient: https://apidock.com/rails/ActiveRecord/Aggregations/ClassMethods/composed_of
For now I'm using a bit of NoSQL for such cases. Postgresql's JSONB type allows to store multilevel ruby hash. It also provides rich functionality: DB level constraints, indexes and query operators.
So common attributes are stored in standard way and child specific - in jsonb. Then you can use whatever you need on top of this: STI, Value Objects pattern, serialization or just create scopes for each child. I prefer the last one - my models are thin, most of constraints are DB level and all business logic is in service classes.
Pros:
Avoiding alter table on big tables when need to add one more child type
Keeping my queries efficient
Preventing storing and selecting unnecessary columns
Serialization out of the box for JSON APIs
Cons:
A bit of schemaless
Vendor lock
I have an app that consists mainly of restaurant model instances. One of the essential attributes for these restaurants is labeling the cuisine it falls under. I'm currently at odds with myself in regards to designing this. On one hand I thought of creating a Cuisine model and creating either a HMT or HABTM association between Restaurants and Cuisines.
More recently I came across this post which shows how to create a pre-defined set of attributes. To take the answer one step further I'm assuming (in my case) I'd add a string-based cuisine column to my restaurant model and setup a select box in my restaurant form that would save the selected value.
What I was wondering was what would be the most efficient way of doing this? The goal is to eventually be able to query restaurants based what cuisine(s) they fall under. I wasn't sure if a model would be the best choice due to it only serving as a join table in a sense with a name attribute. Wasn't sure if having this extra table for something so minute would be optimal.
On the other hand I didn't know if using YAML for this would be conducive since the values are essentially dummy strings with no tangible records on file like I'd have with a model instance. Can someone help me sort out this confusion?
There are many benefits of normalizing many-to-many relationships in the db. Here are some:
Searching, sorting, and creating indexes is faster, since tables are narrower, and more rows fit on a data page.
You can have more clustered indexes (one per table), so you get more flexibility in tuning queries.
Index searching is often faster, since indexes tend to be narrower and shorter.
More tables allow better use of segments to control physical placement of data.
You usually have fewer indexes per table, so data modification commands are faster.
Fewer null values and less redundant data, making your database more compact.
Triggers execute more quickly if you are not maintaining redundant data.
Data modification anomalies are reduced.
Normalization is conceptually cleaner and easier to maintain and change as your needs change.
Also, by normalizing you get the cleaner syntax and other infrastructure benefits from ActiveRecord, e.g.
cuisine.restaurants.where(city: 'Toledo')
I currently have a model Feature that is used by various other models; in this example, it will be used by customer.
To keep things flexible, Feature is used to store things such as First Name, Last Name, Name, Date of Birth, Company Registration Number, etc.
You will have noticed a problem with this - while most of these are strings, features such as Date of Birth would ideally be stored in a column of type Date (and would be a datepicker rather than a text input in the view).
How would this best be handled? At the present time I simply have a string column "value"; I have considered using multiple value columns (e.g. string_value, date_value) but this doesn't seem particularly efficient as there will always be a null column in every record.
Would appreciate any advice on how to handle this - thanks!
There are a couple of ways I could see you going with this, depending on your needs. I'm not completely satisfied with any of these, but perhaps they can point you in the right direction:
Serialize Everything
Rails can store any object as a byte stream, and in Ruby everything is an object. So in theory you could store string representations of any object, including Strings, DateTimes, or even your own models in a database column. The Marshal module handles this for you most of the time, and allows you to write your own serialization methods if your objects have special needs.
Pros: Really store anything in a single database column.
Cons: Ability to work with data in the database is minimal - It's basically impossible to use this column as anything other than storage - you (probably) wouldn't be able to sort or filter your data based on it, since the format won't be anything the database will recognize.
Columns for every datatype
This is basically the solution you suggested in the question - figure out exactly which datatypes you might need to store - you mention strings and datestamps. If there aren't too many of those, it's feasible to simply have a column of each type and only store data in one of them. You can override the attribute accessor functions to use the proper column, and from the outside, Feature will act as though .value is whatever you need it to be.
Pros: Only need one table.
Cons: At least one null value in every record.
Multiple Models/Tables
You could make a model for each of the sorts of Feature you might need - TextFeature, DateFeature, etc. This guide on Multiple Table Inheritance conveys the idea and methodology.
Pros: No null values - every record contains only the columns it needs.
Cons: Complexity. In addition to needing multiple models, you may find yourself doing complex joins and unions if you need to work directly with features of different kinds in the database.
So this is probably a fairly easy question to answer but here goes anyway.
I want to have this view, say media_objects/ that shows a list of media objects. Easy enough, right? However, I want the list of media objects to be a collection of things that are subtypes of MediaObject, CDMediaObject, DVDMediaObject, for example. Each of these subtypes needs to be represented with a db table for specific set of metadata that is not entirely common across the subtypes.
My first pass at this was to create a model for each of the subtypes, alter the MediaObject to be smart enough to join into those tables on it's conceptual 'all' behavior. This seems straightforward enough but I end up doing a lot of little things that feel not so rails-O-rific so I wanted to ask for advice here.
I don't have any concrete code for this example yet, obviously, but if you have questions I'll gladly edit this question to provide that information...
thanks!
Creating a model for each sub-type is the way to go, but what you're talking about is multiple-table inheritance. Rails assumes single-table inheritance and provides really easy support for setting it up. Add a type column to your media_objects table, and add all the columns for each of the specific types of MediaObject to the table. Then make each of your models a sub-class of MediaObject:
class MediaObject < ActiveRecord::Base
end
class CDMediaObject < MediaObject
end
Rails will handle pulling the records out and instantiating the correct subclass, so that when you MediaObject.find(:all) the results will contain a mixture of instances of the various subclasses of MediaObject.
Note this doesn't meet your requirement:
Each of these subtypes needs to be represented with a db table for specific set of metadata that is not entirely common across the subtypes.
Rails is all about convention-over-configuration, and it will make your life very easy if you write your application to it's strengths rather than expecting Rails to adapt to your requirements. Yes, STI will waste space leaving some columns unpopulated for every record. Should you care? Probably not; database storage is cheap, and extra columns won't affect lookup performance if your important columns have indexes on them.
That said, you can setup something very close to multiple-table inheritance, but you probably shouldn't.
I know this question is pretty old but just putting down my thoughts, if somebody lands up here.
In case the DB is postgres, I would suggest use STI along hstore column for storing attributes not common across different objects. This will avoid wasting space in DB yet the attributes can be accessed for different operations.
I would say, it depends on your data: For example, if the differences between the specific media objects do not have to be searchable, you could use a single db table with a TEXT column, say "additional_attributes". With rails, you could then serialize arbitrary data into that column.
If you can't go with that, you could have a general table "media_objects" which "has one :dataset". Within the dataset, you could then store the specifics between CDMediaObject, DVDMediaObject, etc.
A completely different approach would be to go with MongoDB (instead of MySQL) which is a document store. Each document can have a completely different form. The entire document tree is also searchable.