We've an MVC web app which has a Claim management wizard (similar to a typical Order entry stuff). Claim can have multiple Items and each Item can have multiple files related to it.
Claim --> Items --> Files
While adding a new Claim - we want to allow the user to add one or more items to it and also allow file upload for those items. But we want to keep everything in memory until the Claim is actually saved - so that if the user doesn't complete the Claim entry or discards it, no database interaction is done.
We're able to handle data level in-memory management via session. We serialize the Claim object (which also includes a Claim.Items property) in session. But how to manage files?
We store files in < ClaimID >\< ItemID > folder but while creating a new
claim in memory we don't have any IDs until the record is being
saved in the database (both are auto-increment int).
For now, we've to restrict the user from uploading files until a Claim is saved.
Why not interact with the database? It sounds like you're intending to persist data between multiple requests to the application, and databases are good for that.
You don't have to persist it in the same tables or even in the same database instance as the more long-term persisted data. Maybe create tables or a database for "transient" data, or data that isn't intended to persist long-term until it reaches a certain state. Or perhaps store it in the same tables as the long-term data but otherwise track state to mark it as "incomplete" or in some other way transient.
You could have an off-line process which cleans up the old data from time to time. If deletes are costly on the long-term data tables then that would be a good reason to move the transient data to their own tables, optimized for lots of writes/deletes vs. lots of reads over time.
Alternatively, you could use a temporary ID for the in-memory objects to associate them with the files on the disk prior to be persisted to the database. Perhaps even separate the file-associating ID from the record's primary ID. (I'm assuming that you're using an auto-incrementing integer for the primary key and that's the ID you need to get from the database.)
For example, you could have another identifier on the record which is a Guid (uniqueidentifier in SQL) for the purpose of linking the record to a file on the disk. That way you can create the identifier in-memory and persist it to the database without needing to initially interact with the database to generate the ID. This also has the added benefit of being able to re-associate with different files or otherwise change that identifier as needed without messing with keys.
A Guid shouldn't be used as a primary key in many cases, so you probably don't want to go that route. But having a separate ID could do the trick.
Related
I need to keep my data separately in different stores(user profiles). What is the best way to achieve this? I'm going to play with Persistent Object Stores in runtime. Should I simply remove() the current one and addPersistentStore() to make a new or to use the early created instance.
I would have one core data stack (using NSPersistentContainer) for user management. This stack would have the basic account details and the name of the sql file. (Store just the sql filename NOT the full url path, as the path can change in rare circumstance such as an iTunes restore). This would be used for the login, or select account page.
Then I would setup a second core data stack using the sql file name that was stored in the user account object. This would be the main stack used by the application. If you need to logout, then tear down the second stack and start over. Removing and adding store is a bad idea, as it won't deal with the row cache or other managedObjects that are floating around.
Or you could simply have one core data stack and manage the relationship so that every object belongs to a user object. Then you would manage your fetches to only look at objects belonging to the correct user.
I have a dilemma regarding Core Data and syncing data with server.
I wrote an app which uses Core Data, don't use id attributes, everything is set with relationships. Most of data is being generated on device and should be sent to server as backup. On the other hand, there is some data that can be reused among users and I want to have control over it, i.e. modifying, deleting, adding.
Question
When sending data to server, what's prefered way of dealing with relationships? In my opinion, it would be very inefficient to think in terms of Core Data, sending all relation objects to server and then deal with them if they already exist on server. So, using uniqueId is obligatory? Generating ones on server which will be shared and others on devices? Is there any other approach?
Thank you.
Assuming that the server database works with foreign keys, one common solution is to introduce id attributes and set them to some invalid state for new objects. For example, for new relationships you could generate an arbitrary number of unique "invalid" ids by using negative integers. The server would have to then assign new (server-unique) ids and send them back to the client. Of course, when importing data from the server, you replace foreign keys with relationships.
So if you have potentially more than one device trying to modify data also used by other users or devices, the server will have to be part of the solution. Otherwise, you could just generate unique IDs so the server can store the relationships.
First, Thank you for any help provided.
I have an iOS leveraging CoreData to retain various presentations, this data comes from a sqlite file and there is no server connection.
I will have to be able to provide App updates (via appstore), this update may add more data to the database.
The tricky part is that it can not simply overwrite the current database, there are a few user tables that I will not like touched.
Please provide any information I should consider when accomplishing this or any links are greatly appreciated.
Thank you.
Given your app has no server connection, you will have to rely on shipping data within the updated application itself. I would recommend using a plist file or define your own xml or json structure. You can then read this data to create/update core data nsmanagedobjects.
It looks like someone in the past was using plist->coredata on SO
Would you have relationships between user created data and shipped data?
If not, you might go the route of connecting two stored to the persistent store coordinator. The shipped store would be read-only. The store with user created data would be read-write. You can use this approach, too, if you have relationships between shipped and user-created objects, but it's a lot more complicated, since CoreData doesn't manage cross-store relationships for you, and you'll need to write your own logic (doable, but not straight forward).
If you need to have relationships between shipped and user-created objects, you can still ship a CoreData store. When the app launches for the first time (no user-created objects), you copy the store to the Documents folder and user this store to create your CoreData stack. User created objects will be added to this store. Once you have new 'shipped' objects (i.e. a new store in the app-bundle), you'll have to manually migrate that stores data into the store that the user has changed. You'll have to be able to find
(1) objects that need to be deleted
(2) objects that need to be updated (changed)
(3) objects that need to be added
If you mark your shipped objects with a special flag such that you can tell if it's a user created object or a shipped one, that would be doable. You also have to have some sort of ID to be able to tell which objects in the new store correspond to which ones in the existing (old) store.
You do not need to go the route of using plists. In fact, I'd recommend against it. You can easily open two stores at the same time. Either to use both stored, or just to migrate objects from one store to the other store.
I'm just starting to learn ASP.NET MVC and I'd like to know how I can retain model objects between subsequent requests to controller action methods?
For example say I'm creating a contact list web app. Users can create, update, rename, and delete contacts in their list. However, I also want users to be able to upload a contact list exported from other programs. Yet I don't want to just automatically add all the contacts in the uploaded file I want to give the user a secondary form where they can pick which uploaded contacts should be actualy added to their list.
So first I have a ContactController.Upload() method which shows an upload form. This submits to ContactController.Upload(HttpPostedFileBase file) which reads the file that was posted into a set of Contact model objects. Then I want to display a list of all the names of the contacts in the list and allow the user to select those that should be added to their contact list. This might be a long list that needs to be split up into multiple pages, and I might also want to allow the user to edit the details of the contacts before they are actually added to their contact list.
Where should I save the model objects between when a user uploads a file and when they finally submit the specific contacts they want? I'd rather not immediately load all the uploaded contacts into the back end database, as the user may end up only selecting a handful to actually add. Then the rest would need to be deleted. Also I would have to account for the case when a user uploads a file, but never actually completes the upload.
From what I understand an instance of a controller only lasts for one request. So should I create a static property on my Contact controller that contains all the latest uploaded contact model object collections? And then have some process that periodically checks the age of these collections and clears out any that are older then some specified expiration time?
A static property on the controller is trouble. First off, it won't work in a web farm and second it you'd have to deal with multiple requests from different users. If you really don't want to use your database you could use the ASP.NET Session.
No, you don't want a static property, as that would be static to all instances of the controller, even for other users.
Instead, you should create a table used to upload the data to. This table would be used as an intermediary between when the user uploads the data, and completes the process. Upon completion, you copy the contacts you want to keep into your permanent table, then delete the temporary data. You can then run a process every so often that purges incomplete data that is older than a specified time limit.
You could also use the HttpContext.Cache, which supports expiration (and sliding expiration) out-of-the box.
Alternatively, and perhaps even better (but more work) you could use cookies and have the user modify the data using javascript in her browser before finally posting it to you.
However, I'd strongly recommend to store the uploaded information in the database instead.
As you pointed out, it might be a lot of data and the user might want to edit it before clicking 'confirm'. What happens if the user's machine (or browser) crashes or she has to leave urgently?
Depending on the way you store the data the data in this scenario will probably be lost. Even if you used the user id as a cache key, a server restart, cache expiration or cache overflow would cause data loss.
The best solution is probably a combination of database and cookie storage where the DB keeps the information in a temporary collection. Every n minutes, or upon pagination, the modified data is sent to the server and updated in the DB.
The problem with storing the data in session or memory is what happens if the user uploads 50k contacts or more. You then have a very large data set in memory to deal with which depending on your platform may effect application performance.
If this is never going to be an issue and the size of the imported contacts list is manageable you can use either the session or cache to store the dataset for further modifications. Just remember to clear it when the user has committed the changes, you don't want a few heavy datasets hanging around in session.
If you store the dataset in session using your application controller then it will be available to all controllers while it is needed.
I'm refactoring a Rails-based event registration application that has a checkout process that hits several ActiveRecord models. Ideally, the objects should not be saved unless checkout is complete (payment is successfully processed). I'm not entirely certain why it would be a bad thing to serialize these objects into the session temporarily, but I've read over and over that its bad mojo. At least there's no risk of getting out of sync with existing records, as there won't be any existing records.
My questions are:
A) Is it still problematic to store records in the session even though they don't exist elsewhere? Even if I modify a model, can't I kill all existing sessions?
B) If it may cause problems, how should I temporarily store objects? Should I save them and use a boolean flag to indicate permanent vs. temporary status? Then cron a script to weed out stale temporary objects?
Thoughts?
Why don't you store them in the database, but in a way you know they are "incomplete"?
For example, you can add a added_to_cart_at datetime field. When the product is added to the cart, you save the record and you set the value for that field. Then, if the user completes the purchase, you clear the field and you associate the product to an order.
To clear stale record, you can setup a daily cron to delete all records where added_to_cart_at is older than 1.day.ago.
The problem of storing them in the session is that the session has limited space. ActiveRecord objects bring a lot of overhead in terms of space when stored in the session. If you're storing records with densely populated has_many relationships you will run into trouble.
I believe Simone Carletti's description of storing partial records in the database to be the best solution.
But, if you really want to store objects in the session, store the minimum amount of information possible. For example ids and updated fields only. Also store the information as a hash, instead of storing the object in the session directly.