Among countless problems with STEs I am now facing this one, which should be trivial, but is not:
For simplicity, lets assume a standard invoice, order, product scenario.
Lets say that I have a non-stateless (for whatever reason) tier that receives an invoice and adds some orders for some products before sending the invoice back to the tier it came from.
This sounds simple, but is actually a problem with STEs that I have found no simple solution for. The problem is how to keep a collection of product entities to assign to the orders. As I see it I would have to do it in one of these ways, that all have major drawbacks:
Query the database for a product entity for each order I want to place in the invoice-entity.
Drawbacks:
If the same product is used in several orders, EF will throw an exception, when the invoice changes are saved in the database, since multiple instances of the same product key is not allowed in the context. I could make sure that products occurring in multiple orders are only queried once and then shared among the orders, but it would definitely complicate things.
Another drawback is performance if the product entities (or equivalent entity in another domain scenario) are rather large or querying the database for products at the application tier is impractical for whatever reason.
Query the database for all product entities at the beginning of the application lifetime
This would be the typical scenario if you keep a list of products in a user application if the list of products rarely changes. In my case I don't want to query the database, since the "product" type list never changes in the lifetime of the application, performance is an important issue and the system should be robust to the database being temporarily unavailable, so I want to keep a cached collection of "product" equivalent entities.
Drawbacks: The major drawback here is that assigning cached products to new order entities will cause a memory leak. The reason for that is that the "Fixup" methods generated by the STE will hookup an event handler to the product's ChangeTracker in order to handle cascading deletions of a product. That event handler will keep the new order and the cached product connected accumulating all orders added for the lifetime of the cache.
A solution could be to implement some kind of "freeze" property of STEs, such that not only would change tracking be disabled, but the state of the entity and change tracker would remain unchanged even after navigation property assignments. Such a "freeze" modification could be difficult to write, since STE code comes with a lot of side effects making them hard to modify.
Create product entity clones
Here the products are being queried at the beginning of the application lifetime, but instead of using the product entities, clones are being made when assigning the products to the orders. This would solve the memory leak problem, but cloning would need to be implemented and maintained. It is possible that it would not be too hard rewriting the STE .tt scripts to support cloning. There would still be the problem with multiple entities in the entity context sharing the same key though.
Related
I'm a senior Comp. Sci. major working on a senior design project for our faculty. The name of this project is "Gradebook", and it is responsible for allowing instructors to record grades for students and for students to check their grades in a class. This project is written in Ruby on Rails, and this feature set is integrated into our current CS Website.
One requirement for our project is to constantly keep the course average and each of the student's averages updated. So I designed a CourseInfo class and a StudentInfo class to help with this process.
The CourseInfo class accepts a Gradebook (an ActiveRecord object) as a parameter and calculates the course average. It creates an Associative Array of StudentInfo objects, with each StudentInfo object containing the student's overall average in the class. The benefit of this is that I can calculate the Course Average with one line of code that initializes the class, and it is very clean.
But there is one issue that I'm mulling over. The problem is, the CourseInfo object does not survive when another HTTP request is made, I have to keep recreating it. Whether I'm adding an assignment, editing a category, or recording grades, I have to keep it updated because this project uses AJAX requests all the time. Instructors do not have to refresh any pages, because AJAX requests are created with every action.
For example, suppose I'm recording grades for a specific assignment. With each grade I record into the spreadsheet, an AJAX request is made and the course average updates with each new grade. But the problem is, if I want to update the Course Average after recording a student's grade, since the CourseInfo object does not stay alive in the next request, I have to recreate the object to keep the average updated. But that is a LOT of work. That involves calculating each of the student's average for EACH assignment, and then calculating the course average for EACH student. I know, a lot of work and could be simpler right?
So naturally, I want this CourseInfo object to live forever as long as the client is using the website. I've thought of many different ways to solve this problem:
1) Global Variables or Class Variables - I honestly want to stay away from this approach because I hear it is bad design. I also hear that this approach is not thread-safe. But it seems to provide a simple solution to my problem?
2) Serialize the Object in the Database - This is what I'm learning towards the most. I hear that sometimes people will serialize a Hash that contains user preferences in a web app, why not serialize my CourseInfo object? I've also done some research on the MessagePack gem, and I could potentially encode the CourseInfo object using MessagePack and then store it into the database. I feel like this would be a noticeable performance increase.
3) Use some kind of cache - Gems such as Redis act as a cache, and I liked Redis because it is a key value store. I can store a CourseInfo object for each Gradebook that was used during the session, and if I need to update the CourseInfo object, I can simply fetch the CourseInfo object by using the Gradebok's ID as a key. But I'm not sure if this is thread-safe. What if two instructors attempt to update two different grades at the same time? Will there be multiple instances of this CourseInfo object for each client using Gradebook?
4) Store it in the Session - Yeah I pretty much crossed this option off my list. I researched this approach, and I hear it is horrible to store a lot of data in the session. I don't want to do this.
What do you think? If I don't want to reinitialize this large object for each request, how can I make it live forever? What is the most efficient solution? What do you think about my design?
Help would be much appreciated! Thanks!
Use
2) Serialize the Object in the Database
due to agile philosophy of implementing the simplest thing that could possibly work first.
see Saving arrays, hashes, and other non-mappable objects in text columns
The course_average allways reflects the persistent state of the users records. Serializing it is a no braner in ActiveRecord. If you are using postgres , you can even use the native json store, which you can not only deserialize but also query through. No need for additional complexity to maintain an extra store. This solution has also the benefit of having a persistent counter cache.(no need to recalculate if nothing changes)
However using a cache is also a valuable option. Just remember, if you want to use redis as a cache store you have to explicitly configure a cache expiring policy, as by default none of the keys will expire and you will recieve an out of memory error, when redis grows beyound the size of RAM on the machine.
The redis-rails gem will setup rails to use redis for caching.
Storing this information in the session might also work, but watch out you session not getting to big. The whole session data is allways loaded completely into memory, regardles of some information in it is required or not. Allways loading megabytes of data into memory for every http connection might be not a great idea.
There is also a 5th option, i would evaluate first. Check, does the computation of averages really takes so long. Or can the peformance of it, pobably be improved, e.g. by reducing n+1 queries, setting proper indexes, doing the whole computation in sql or preparing the necessary data completly in sql, so that all the necessary data can be fetched in 1 query.
I'm currently building a Core Data app and I've hit a snag. I guess here's some context on the schema:
The app is to keep track of a therapist's session with her clients. So the schema is organized thus: there's a table of clients, clients have sessions, sessions have activities, and activities have metrics. In the app these metrics translate to simple counters, timers, and NSSliders.
The crux is that the client wants to be able to insert previously made activities into new sessions for new clients. So, I've tried just doing a simple fetch request and then moved on to an NSFetchedResultsController. I keep running into the issue that since Core Data is an object graph, I get a ton of activity entries with virtually the same data. The only differentiating property would be whatever the session is (and if you want to go further back, the client itself).
I'm not sure if this is something I need to change in the schema itself, or if there's some kind of workaround I can do within Core Data. I've already tried doing distinct fetch results with the NSFetchedResultsController by using the result type NSDictionaryResultType. It kind of accomplishes what I want but I only get the associated properties of the entity, and not any children entities associated with it (I need those metrics, you see).
Any help is appreciated, and I can post code if desired even though I don't really have a specific coding error.
I don't see the problem. If you modeled things with the Client, Session, Activity, and Metric entities, each having a to-many relationship to the one to its right and to-one/to-many inverse relationship to the one to its left (in the order I listed the entities), there is nothing stopping you from adding a particular activity into another session (of another client), is it?
Maybe I'm misunderstanding the question.
Just use a simple NSFetchRequest and set the predicate for exactly what you are looking for. You can set the fetch limit if you are getting too many results but your question doesn't exactly sounds like a question IMO.
I believe what you are looking for is an NSPredicate to narrow your results down. Once you fetch a specific object you can assign any relation or attribute to that object easily with dot notation then save the context.
Is there a way to ensure an ordered atomic change set from Simperium?
I have a data model that has complex relationships associated. It seems looking over things that it is possible for the object graph to enter in an invalid state if the communication pipe is severed. Is there a way to indicate to Simperium that a group of changes belong together? This would be helpful as the client or server would prevent applying those changes unless all the data from a "transaction" is present thus keeping the object graph in a valid state.
Presently it's expected that your relationships are marked as optional, which allows objects to be synced and stored in any order without technically violating your model structure. Relationships are lazily re-established by Simperium at first opportunity, even if the connection is severed and later restored.
But this approach does pass some burden to your application logic. The code is open source, and suggestions for changes in this regard are welcome.
I use Core Data to store objects in my app. Basically I store objects valuable to user, for example things that user liked or places where he wants to go. There is a hidden relationship between every of these objects and user himself. Everything is perfectly logical. But now I want to store (cache) some data that is not directly linked to user, for example result of search requests. These objects mostly are used only in one particular place and could have some expire time like cache or something like that. So I need to store these objects but be able to remove them from storage in future. The thing is that these objects are of the same entity as my valuable objects. Another thing is that there could be a relationship between these non-valuable objects and some valuable objects.
What is the best way to store such non-valuable objects and, most important, to clean Core Data Storage from them?
It depends on the use you need for these items.
basically, these are items that have an expiry date (temporary objects), you could use a timestamp on each object, or you could create an new entity relating to the items (one-to-one relationship) that you can query and delete by under conditions you specify.
When you open the store coordinator, you could have a cleanup rule (that you will need to implement) it will run in the background and remove items that are no longer relevant (make sure not to display them to the user, or access them when you perform the cleanup).
try to keep your cleanup in the background and perform it at times which will not disturbe the user.
in my own opinion, it is better to mark the items as hidden, then deleting them immediately as deletion is a heavy operation that might block the UI. also try and keep your deletions small (not all objects at once, but small batches of objects)
I have an old SL4/ria app, which I am looking to replace with breeze. I have a question about memory use and caching. My app loads lists of Jobs (a typical user would have access to about 1,000 of these jobs). Additionally, there are quite a few lookup entity types. I want to make sure these are cached well client-side, but updated per session. When a user opens a job, it loads many more related entities (anywhere from 200 - 800 additional entities) which compose multiple matrix-style views for the jobs. A user can view the list of jobs, or navigate to view 1 job at a time.
I feel that I should be concerned with memory management, especially not knowing how browsers might deal with this. Originally I felt this should all be 1 EntityManager and I would detachEntities when user navigates away from a job, but I'm thinking this might benefit from multiple managers by intended lifetime. Or perhaps I should create a new dataservice & EntityManager each time the user navigates to a new hash '/#/' area, since comments on clear() seems to indicate that this would be faster? If I did this, I suppose I will be using pub/sub to notify other viewmodels of changes to entities? This seems complex and defeating some of the benefits of breeze as the context.
Any tips or thoughts about this would be greatly appreciated.
I think I understand the question. I think I would use a multi-manager approach:
Lookups Manager - holds once-per session reference (lookup) entities
JobsView Manager - "readonly" list of Jobs in support of the JobsView
JobEditor Manager - One per edit session.
The Lookups Manager maintains the canonical copy of reference entities. You can fill it once with a single call to server (see docs for how). This Lookups Manager will Breeze-export these reference entities to other managers which Breeze-import them as they are created. I am assuming that, while numerous and diverse, the total memory footprint of reference entities is pretty low ... low enough that you can afford to have more than one copy in multiple managers. There are more complicated solutions if that is NOT so. But let that be for now.
The JobsView Manager has the necessary reference entities for its display. If you only displayed a projection of the Jobs, it would not have Jobs in cache. You might have an array and key map instead. Let's keep it simple and assume that it has all the Jobs but not their related entities.
You never save changes with this manager! When editing or creating a Job, your app always fires up a "Job Editor" view with its own VM and JobEditor Manager. Again, you import the reference entities you need and, when editing an existing Job, you import the Job too.
I would take this approach anyway ... not just because of memory concerns. I like isolating my edit sessions into sandboxes. Eases cancellation. Gives me a clean way to store pending changes in browser storage so that the user won't lose his/her work if the app/browser goes down. Opens the door to editing several Jobs at the same time ... without worrying about mutually dependent entities-with-changes. It's a proven pattern that we've used forever in SL apps and should apply as well in JS apps.
When a Job edit succeeds, You have to tell the local client world about it. Lots of ways to do that. If the ONLY place that needs to know is the JobsView, you can hardcode a backchannel into the app. If you want to be more clever, you can have a central singleton service that raises events specifically about Job saving. The JobsView and each new JobEditor communicate with this service. And if you want to be hip, you use an in-process "Event Aggregator" (your pub/sub) for this purpose. I'd probably be using Durandal for this app anyway and it has an event aggregator in the box.
Honestly, it's not that complicated to use and importing/exporting entities among managers is a ... ahem ... breeze. Well worth it compared to refreshing the Jobs List every time you return to it (although you'll want a "refresh button" too because OTHER users could be adding/changing those Jobs). You retain plenty of Breeze benefits: querying, validation, change-tracking, batch saves, entity navigation (those reference lists work "for free" in Breeze).
As a refinement, I don't know that I would automatically destroy the JobEditor view/viewmodel/manager when I returned to the JobsView. In my experience, people often return to the same Job that they just left. I might hold on to a view so you could go back and forth quickly. But now I'm getting tricky.