I have a "catalog" that I am trying to display information on. This information will be pulled from a few different tables that a user will be able to set a preference to hide a record from the respective table on their "catalog". I am running a Postgres database
So, my question is:
Would it be better (performance wise) to create a new table (table_a_to_catalog) where it would store the table_a_id and the catalog_id for the record from table_a that the user wants to hide for that catalog. Then have another table (table_b_to_catalog) to hold that connection...and so on...
OR
Would it be better to store the hide preference as a json value in the record of the catalog? So it would be something like {"table_a" => [id1, id2, id3], "table_b" => [id1, id2, id3]}
It really depends on the usecase of this catalog... If the information is readonly and you are running a job once a day to update the said catalog then json would be better. However, if you want to update information on the catelog live and and allow it to be editable then having a separate table would be best.
As for personal preference, I think keeping data in table allows more flexibility when you want to use the data for other features
Having very large tables negatively impacts for performance. Keeping "hide" view data in a postgres table means having a DB entry for each hidden entry in each catalog. Each client application will need to filter that table for information relevant to their user, and with many users this could take considerable time.
If one simply adds a field to the user table, containing an hstore, JSON or CSV of view data (e.g. hide preferences), that will reduce the initial load time marginally. JSON would make more sense if "hiding" means simply not displaying it client-side, wheras hstore makes more sense if you wish to not send the data to the client to begin with.
I say marginally because many other factors (caching) will impact performance more than this. You may want to look into using Redis for the application runtime and Postgres for data warehousing.
Related
We haave Accounts, Deals, Contacts, Tasks and some other objects in the database. When a new organisation we want to set up some of these objects as "Demo Data" which they can view/edit and delete as they wish.
We also want to give the user the option to delete all demo data so we need to be able to quickly identify it.
Here are two possible ways of doing this:
Have a "IsDemoData" field on all the above objects : This would mean that the field would need to be added if new types of demo data become required. Also, it would increase database size as IsDemoData would be redundant for any record that is not demo data.
Have a DemoDataLookup table with TableName and ID. The ID here would not be a strong foreign key but a theoretical foreign key to a record in the table stated by table name.
Which of these is better and is there a better normalised solution.
As a DBA, I think I'd rather see demo data isolated in a schema named "demo".
This is simple with some SQL database management systems, not so simple with others. In PostgreSQL, for example, you can write all your SQL with unqualified names, and put the "demo" schema first in the schema search path. When your clients no longer want the demo data, just drop the demo schema.
I'm geting towards the end of my application and have 25 lookup tables. Some of my domain classes have 15 table references as properties (one-to-one). Now I have just remembered/learned I can use enums instead. I'm sure I can refactor a bunch of those lookup tables to be enum classes. However, what are the best practices for using an enum vs. a lookup table? A few things about my application scenario that may assist in the answer:
There is only one developer (me)
This is a web based application
It won't be a heavily used application
When the view for the domain class with currently 15 lookups is rendered, all of those lookups will need to be loaded to display the data in the view. (15 joins)
The lookups will be cached in memory.
The lookup relationships to the domain class are all one-to-one
The data in the lookups will rarely change
Some of the values in the lookup tables are wordy, example: "Taking care of animals"
Some of the lookups have many records. Example: one lookup is U.S. states. Another is a person's occupation.
I think the most obvious answer is to use Enums for things that don't change - ever. Like Days of the week,
or Planets in the Solar System - ok so sometimes thnigs you thought wouldnt change actually changed, but that OK - a quick update and you are good to go for a few more years.
But when you are working with data that will certainly change over time and often enough, then by all means store that in the database. A change to a table will not require a code change and redeployment. Additionally it would be nice to add an admin interface to these - and scaffolded screens should be quick and easy enough.
This is just my opinion. I don't think there is a "right" answer here.
If it gets stored in a database I use a lookup table. It is more flexible and based on a few things, should more performant. While you do have to do join queries, you can lessen that impact by enabling 2nd level cache. cache: true In your mappings.
One thing to remember when using an Enum instead of a lookup is that there is no longer a foreign key to a separate table so you can't guarantee the values are in the list you desire from a database level. In addition, if you need to query based on the Enum, it is now a string comparison instead of a number comparison typically.
Normally to create/alter a table in database I use migrations (manually run rake db:migrate) and then in my code I use ActiveRecord. This is very cool as I don't have to worry about representation of the data in db and about a specific kind of db (sqlserver, pg or other).
But now a customer wants to be able to create "things" on-fly himself like, say, he starts selling computers, so he wants to an interface where he can dynamically create an object "computer" with properties like "Name, RAM, HD, ...". It seems to be quite natural to create a separate table in db with all these fields. But how can I do that in RoR and keep all these nice things about ActiveRecord?
Please suggest.
The usual way is to do exactly the opposite:
Have a table for object types
Have a table for field names for each object type
Have a very big table with all the custom attributes for each object of any type
This is called EAV (Entity-attribute-value model, see http://en.wikipedia.org/wiki/Entity-attribute-value_model). And it scales pretty bad.
Alternatively, you can use a store text column instead of the big EAV table (see http://api.rubyonrails.org/classes/ActiveRecord/Store.html) so you don't have to make those difficult attribute retrievals, typical of EAV. You still need to store somewhere the "object types" definitions, so the expected fields etc are available when building forms and tables.
The problem with this approach is that you are not able to query (where/join/select) on those attributes because they are not columns. There are a number of solutions to that:
Don't do filtering on those attributes (meh...)
Have an external search server that's able to do faceted search
(as #Amar correctly says) Use a document database
Use postgreSQL and use hstore instead of a simple serialized column.
NoSQL database(Document Database Mongodb,CouchDB) can be best fit for this or use redis. As per my thoughts you can use Vertical Table concept Try to run Rails 2.x Demo of application for MySQL.
You can try with Mongodb, check if this is needed.
Hi iam new in dynamo db and, with my knowledge its a non relational db ie we cant join the tables. My doubt is how we design the table structure. Please clarify with following example.
I have a following tables
1) users - user_id, username, password, email, phone number, role
2) roles - id, name [ie admin, supervisor, ect..]
a) My first doubt is we have any provision to set auto increment for user_id fields ?
b) Is this correct way of setting primary key as user_id?
c) Is this is the correct method to store user role in dynamo db? ie a roles table contains id and title and store role id in user table?
e) Is this possible to retrieve two tables data along with each user? Am using rails 3 and aws-sdk gem
If anybody reply it will be very helpful for me like a new dynamodb user
Typically with nosql style databases you would provide the unique identifier, rather than having an auto increment PK field do that for you. This usually would mean that you would have a GUID be the key for each User record.
As far as the user roles, there are many ways to accomplish this and each has benefits and problems:
One simple way would be to add a "Role" attribute to the Users table and have one entry per role for that user. Then you could grab the User and you would have all the roles in one query. DynamoDB allows attributes to have multiple values, so one attribute can have one value per role.
If you need to be able to query users in a particular role (ie. "Give me all the Users who are Supervisors") then you will be doing a table scan in DynamoDB, which can be an expensive operation. But, if your number of users is reasonably small, and if the need to do this kind of lookup is infrequent, this still may be acceptable for your application.
If you really need to do this expensive type of lookup often, then you will need to create a new table something like "RolesWithUsers" having one record per Role, with the userIds of the users in the role record. For most applications I'd advise against doing something like this, because now you have two tables representing one fact: what role does a particular user have. So, delete or update needs to be done in two places each time. Not impossible to do, but it takes more vigilance and testing to be sure your application doesn't get wrong data. The other disadvantage of this approach is that you need two queries to get the information, which may be more expensive than the table scan, again, depending on the quantity of records.
Another option that makes sense for this specific use case would be to use SimpleDb. It has better querying capability (all attributes are indexed by default) and the single table with roles as multi-valued attribute is going to be a much better solution than DynamoDB in this case.
Hope this helps!
We have a similar situation and we simply use two DBs, a relational and a NoSQL (Dynamo). For a "User" object, everything that is tied to other things, such as roles, projects, skills, etc, that goes in relational, and everything about the user (attributes, etc) goes in Dynamo. If we need to add new attributes to the user, that is fine, since NoSQL doesn't care about those attributes. The rule of thumb is if we only need something on that object page (that is, we don't need to associate with other objects), then we put in Dynamo. Otherwise, it goes in relational.
Using a table scan on the NoSQL DB is not really an option after you cross even a small threshold (up to that point, you can just use an in memory DB anyway).
I am not sure exactly what I should name this question. I just started server-side programming and I need some help.
All the tutorials I have read so far on RoR deal with creating a pre-defined table and with pre-defined fields (id, name, email, etc etc). They use ActiveRecord as base class and saving to db is handled automatically by superclass.
What I am trying to program is something that allows user-defined table with fields. So think of this way. The web UI will have an empty table, the user will name the table, and add columns (field), and after that, add rows, and then later save it. How would I implement this? I am not asking for details, just an overview of it. As I said, all the tutorials I have read so far deal with pre-defined tables with fields where the ActiveRecord subclass is predefined.
So in a nutshell, I am asking, how to create tables in db on runtime, and add fields to the tables.
Hope I was clear, if not, please let me know and i will try to elaborate a bit more.
Thanks.
Unless you're building a DB administration tool (and even maybe then), allowing the user direct access to the database layer in the way you're suggesting is probably a bad idea. Apart from issues of stability and security, it'll get really slow if your users are creating lots of tables.
For instance, if you wanted to search for a certain value across 100 of your users' tables, you'd have to run 100 separate queries. The site would get exponentially slower the more user tables that were created.
A saner way to do it might be to have a Table model like this
class Table < ActiveRecord::Base
has_many :fields
has_many :rows
end
Every table would have fields attached to it, and rows to store the corresponding data (which would be encoded somehow).
However, as #Aditya rightly points out, this is not really beginner stuff!
I agree with previous answers generally speaking. It's not clear from your question why you want to create a table at runtime. It's not really obvious what the advantage of doing this would be. If you are just trying to store data that seems to fit into a table with rows and columns, why not just store it as an array in a field of your user table. If your user is allowed to create many tables, then you could have something like
class User < ActiveRecord::Base
has_many :tables
end
and then each table might have a field to store a serialized array. Or you could go with Alex's suggestion - the best choice really depends on what you are going to do with the data, how often it changes, whether you need to search it and so on ...
You can create a database as shown in tutorials which stores name of tables and their columns name those your user want. Then you can have worker (which can be build using Redis and Resque, here is simple Tut on Resque and Redis) and have those worker run migration (write migration with variables and use params to replace them) for you for new table in DB as soon as new entry is made in database. Tell me if you have questions on this.