I'll try to be short and clear with this question.
We have an asp.net mvc app that uses entity framework 4.
Our business model is relatively straightforward:
We have an object (which corresponds to a table) called Photo(s).
That photos table has a handful of columns that match up to properties on the object.
Description,Title,Date etc.
It also has a number columns that reference foreign keys for other tables:
AuthorId,LicenseId etc...
The author and license tables are complex in their own right, with multiple fields (Title,Summary,Date etc.)
I have multiple clients using this application to view their photos. I would like each client to dictate what fields they see when viewing the photos, as well as what fields they see when editing those fields.
My thought is to have tables setup saying client-a should see Field1,Field2 and Field3 when viewing their photos - and client-b should see Field1,Field4 and Field5. But some of these fields are not simply columns in the main photos table, they may be fields in a child table. so Field1 might be: Table.Photos.Title -> which corresponds to an object as: Objects.Photo.title...
but Field3 might be: Table.Licenses.LicenseSummary -> which corresponds to an object as: Objects.Photo.License.LicenseSummary
I'm trying to figure out the methodology that we would use to have a very data driven environment so in the DB I can say, display this object/property (for viewing or editing) and then it would know how to map to whatever table it needs to pull that information. also, during editing... give it some way to pull a list of available values if it is that type of property, and not just a text field.
I'm looking for an example of what this might involve, our model is actually more complex than this, but this is just an idea of what we are trying to accomplish. I don't know if what I'm trying to do is normal, perhaps it involves reflection? This is a new area for me.
If the clients are defining their own custom fields, I would simply give them a Key/Value pairs table.
PhotoID FK
Key string
Value string
Display bool
Note that this essentially amounts to EAV, which comes with its own set of difficulties.
If it's just about permissions on existing fields, you need to capture that information:
PhotoID FK
ClientID FK
FieldName string
Display Bool
You can use this information to inhibit the display of fields in the View. The easiest way to do that would be to use a loop in the View itself, writing the field to the output only if Display is set to true.
Related
I have a scaffolded Grails application with two domains, Person and Course. Person belongs to Course, and Course hasMany Persons. I have modified show.gsp for Course to list all of the Persons associated with the selected Course.
To achieve this, Course.groovy contains the following line:
List persons = new ArrayList()
And, as a result, the "person" database table contains a persons_idx field. I frequently will be adding new data to the "person" table outside of my Grails application, from an external website.
When INSERTing new data, how to I figure out what to set persons_idx as?
I had originally used a SortedSet instead of an ArrayList for persons, since I care about sorting. But since I am sorting on Person.lastName, and there will always be multiple people with the same last name, then the list will exclude those persons who have the same last names as others. I wish there was another way...
Thanks.
Having two applications manipulate the same Database is a thing to avoid, when possible. Can your 2nd application instead call an action on the controlling app to add a Person to the Course with parameters passed to specify each? That way, only one app is writing to the DB, reducing caching, index, and sequence headaches.
You also state that Person belongsTo Course... so you create a new Person for "Bob Jenkins" for each course that he's in? This seems excessive. You should probably look into a ManyToMany for this.
Without moving to a service, unfortunately, you'd want to change the indices on some if not many of the rows for the children of the Course you're trying to add a Person to, as that index is the sorted index for all the Persons in the Course.
I would suggest going back to a "Set", and do your sorting in the app. Your other question about sorting already told you not to override compareTo to just check the last name. If I were you, I'd forget about overriding compareTo at all (except to check IDs, if you want), and just use the sort() method, passing in a closure that correctly sorts the objects.
We've got a data warehouse design with four dimension tables and one fact table:
dimUser id, email, firstName, lastName
dimAddress id, city
dimLanguage id, language
dimDate id, startDate, endDate
factStatistic id, dimUserId, dimAddressId, dimLanguageId, dimDate, loginCount, pageCalledCount
Our problem is: We want to build the fact table which includes calculating the statistics (depending on userId, date range) and filling the foreign keys.
But we don't know how, because we don't understand how to use natural keys (which seems to be the solution to our problem according to the literature we read).
I believe a natural key would be the userId, which is needed in all ETL jobs which calculate the dimension data.
But there are many difficulties:
in the ETL jobs load(), we do bulk inserts with INSERT IGNORE INTO to remove duplicates => we don't know the surrogate keys which were generated
if we create meta data (including a set of dimension_name, surrogate_key, natural_key) this will not work because of the duplicate elimination
The problem seems to be the duplicate elimination strategy. Is there a better approach?
We are using MySQL 5.1, if it makes any difference.
If your fact table is tracking logins and page calls per user, then you should have set of source tables which track these things, which is where you'll load your fact table data from. I would probably build the fact table at the grain of one row per user / login date - or even lower to persist atomic data if at all possible.
Here you would then have a fact table with two dimensions - User and Date. You can persist address and language as dimensions on the fact as well, but these are really just attributes of user.
Your dimensions should have surrogate keys, but also should have the source "business" or "natural" key available - either as an attribute on the dimension itself, or through a mapping table as your colleague suggested. It's not "wrong" to use a mapping table - it does make things easier when there are multiple sources.
If you store the business keys on a mapping table, or in the dimension as an attribue, then for each row to load in the fact, it's a simple lookup (usually via a join) against the dim or mapping table to get the surrogate key for the user (and then from the user to get the user's "current" address / language to persist on the fact). The date dimension usually hase a surrogate key stored in a YYYYMMDD or other "natural" format - you can just generate this from the date information on your source record that you're loading into the fact.
do not force for single query, try to load the data in separated queries and mix the data in some provider...
I am developing an ASP.NET MVC4 web application. It uses the entity framework for data access. Many of the pages contain grids. These need to support paging, sorting, filtering and grouping. For performance the grid filtering, sorting, paging etc needs to occur on the database (i.e. the entity framework needs to generate a suitable SQL query). One complication is that the view model to represent the grid rows is built by combining the data from multiple business entities (tables). This could be simply getting the data from an entity a couple of levels down or by calculating it based on the values of related business entities. What approach is recommended to handle this scenario? Does anyone know of a good example on the web? Most have a simple mapping between the view model and business domain model.
Update 28/11 - To further clarify the initial display of the grid and paging works performs well. (See comment below) The problem is how do you handle sorting/ordering (and filtering) when the column that the user clicked on does not map directly to a column on the underlying business table. I am looking for a general solution to achieving this as the system will have approx 100 grids with a number of columns each and trying to handle this on a per column basis will not be maintainable.
If you want to be able to order a calculated field that isn't pre calculated in the database or do any Database Operation against it, then you are going to have to precalulate the value and store it in the database. I don't know anyway around that.
The only other solution is to move the paging and sorting etc to the web server, I am sure you don't really want to do that as you will have to calculate ALL the values to find what order they go in.
So if you want to achieve what you want - I think you will have to do the following, I would love to hear alternate solutions though:
Database Level Changes:
Add a Nullable Column for each calculated field you have in your View Model.
Write a SQL Script the calculates these values.
Set the column to Not Null if necessary
App Level Changes:
In your Add and Edit Pages you will have to calculate these values and Commit them with the rest of the data
You can now query against these at a Database level and use Queryable as you wanted.
I currently have a model Feature that is used by various other models; in this example, it will be used by customer.
To keep things flexible, Feature is used to store things such as First Name, Last Name, Name, Date of Birth, Company Registration Number, etc.
You will have noticed a problem with this - while most of these are strings, features such as Date of Birth would ideally be stored in a column of type Date (and would be a datepicker rather than a text input in the view).
How would this best be handled? At the present time I simply have a string column "value"; I have considered using multiple value columns (e.g. string_value, date_value) but this doesn't seem particularly efficient as there will always be a null column in every record.
Would appreciate any advice on how to handle this - thanks!
There are a couple of ways I could see you going with this, depending on your needs. I'm not completely satisfied with any of these, but perhaps they can point you in the right direction:
Serialize Everything
Rails can store any object as a byte stream, and in Ruby everything is an object. So in theory you could store string representations of any object, including Strings, DateTimes, or even your own models in a database column. The Marshal module handles this for you most of the time, and allows you to write your own serialization methods if your objects have special needs.
Pros: Really store anything in a single database column.
Cons: Ability to work with data in the database is minimal - It's basically impossible to use this column as anything other than storage - you (probably) wouldn't be able to sort or filter your data based on it, since the format won't be anything the database will recognize.
Columns for every datatype
This is basically the solution you suggested in the question - figure out exactly which datatypes you might need to store - you mention strings and datestamps. If there aren't too many of those, it's feasible to simply have a column of each type and only store data in one of them. You can override the attribute accessor functions to use the proper column, and from the outside, Feature will act as though .value is whatever you need it to be.
Pros: Only need one table.
Cons: At least one null value in every record.
Multiple Models/Tables
You could make a model for each of the sorts of Feature you might need - TextFeature, DateFeature, etc. This guide on Multiple Table Inheritance conveys the idea and methodology.
Pros: No null values - every record contains only the columns it needs.
Cons: Complexity. In addition to needing multiple models, you may find yourself doing complex joins and unions if you need to work directly with features of different kinds in the database.
I am trying to do the following in my iPad app. I have a structure that allows people to create grouped lists which we call "Templates". So The top level CoreOffer(has Title) which can have many groups(has grouptitle/displayorder) which can have many items(has ItemTitle, DisplayOrder). As shown below. This works great, I can create Templates perfectly.
Image link
http://img405.imageshack.us/img405/9145/screenshot20110610at132.png
But once Templates are created people than can use them to map against the Template which I will call an Evaluation. A Template can be used many times. The Evaluation will contain a date(system generated) and which items from this particular Template have been selected.
Example below, people will be able to check particular rows in the screen below, this is then an Evaluation.
Image link
http://img41.imageshack.us/img41/8049/screenshot20110610at133.png
I am struggling to figure out how to create and store this information in the core data model without duplicating the Template. (struggling coming from a SQL background!) In SQL this would involve something like an evaluation table recording each itemid and its selection status.
I expect its quite simple but I just cant get my head around it!
Thanks
The first thing you want to do is clean up the naming in your data model. Remember, you are dealing with unique objects here and not the names of tables, columns, rows, joins etc in SQL. So, you don't need to prefix everything with "Core" (unless you have multiple kinds of Offer, Group and Item entities.)
Names of entities start with uppercase letters, names of attributes and relationships with lower case. All entity names are singular because the modeling of the entity does not depend on how many instances of the entity there will be or what kind of relationships it will have. To-one relationship names should be singular and to-many plural. These conventions make the code easy to read and convey information about the data model without having to see the actual graphic.
So, we could clean up your existing model like:
Offer{
id:string
title:string
groups<-->>Group.offer
}
Group{
title:string
displayOrder:number
offer<<-->Offer.groups
items<-->>Item.group
}
Item{
title:string
displayOrder:number
isSelected:Bool
group<<-->Group.items
}
Now if you read a keypath in code that goes AnOfferObj.groups.items you can instantly tell you are traversing two to-many relationships without knowing anything else about the data model.
I am unclear exactly what you want your "Evaluations" to "copy". You appear to either want them to "copy" the entire graph of any Offer or you want them "copy" a set of Item objects.
In either case, the solution is to create an Evaluation entity that can form a relationship with either Offer or Item.
In the first case it would look like:
Evaluation{
title:string
offer<<-->Offer.evaluations
}
Offer{
id:string
title:string
groups<-->>Group.offer
evaluations<-->>Evaluation.offer
}
... and in the second case:
Evaluation{
title:string
items<<-->>Item.evaluations
}
Item{
title:string
displayOrder:number
isSelected:Bool
group<<-->Group.items
evaluations<<-->>Evaluation.items
}
Note that in neither case are you duplicating or copying anything. You are just creating a reference to an existing group of objects. In the first case, you would find all the related Item objects for a particular Evaluation object by walking a keypath of offer.groups.items. In the second case, you would walk just the keypath of the items relationship of the Evaluation object with items.
Note that how you ultimately display all this in the UI is independent of the data model. Once you have the objects in hand, you can sort or otherwise arrange them as you need to based on the needs of view currently in use.
Some parting advice: Core Data is not SQL. Entities are not tables. Objects are not rows. Attributes are not columns. Relationships are not joins. Core Data is an object graph management system that may or may not persist the object graph and may or may not use SQL far behind the scenes to do so. Trying to think of Core Data in SQL terms will cause you to completely misunderstand Core Data and result in much grief and wasted time.
Basically, forget everything you know about SQL. That knowledge won't help you understand Core Data and will actively impede your understanding of it.