This is the way I am thinking of using DB4o. When I need to query, I would open the file, read and close:
using (IObjectContainer db = Db4oFactory.OpenFile(Db4oFactory.NewConfiguration(), YapFileName))
{
try
{
List<Pilot> pilots = db.Query<Pilot>().ToList<Pilot>();
}
finally
{
try { db.Close(); }
catch (Exception) { };
}
}
At some later time, when I need to insert, then
using (IObjectContainer db = Db4oFactory.OpenFile(Db4oFactory.NewConfiguration(), YapFileName))
{
try
{
Pilot pilot1 = new Pilot("Michael Schumacher", 100);
db.Store(pilot1);
}
finally
{
try { db.Close(); }
catch (Exception) { };
}
}
In this way, I thought I will keep the file more tidy by only having it open when needed, and have it closed most of the time. But I keep getting InvalidCastException
Unable to cast object of type 'Db4objects.Db4o.Reflect.Generic.GenericObject' to type 'Pilot'
What's the correct way to use DB4o?
No, it's not a good idea to work this way. db4o ObjectContainers are intended to be kept open all the time your application runs. A couple of reasons:
db4o maintains a reference system to identify persistent objects, so it can do updates when you call #store() on an object that is already stored (instead of storing new objects) . This reference system is closed when you close the ObjectContainer, so updates won't work.
Class Metadata would have to be read from the database file every time you reopen it. db4o would also have to analyze the structure of all persistent classes again, when they are used. While both operations are quite fast, you probably don't want this overhead every time you store a single object.
db4o has very efficient caches for class and field indexes and for the database file itself. If you close and reopen the file, you take no advantage of them.
The way you have set up your code there could be failures when you work with multiple threads. What if two threads would want to open the database file at exactly the same time? db4o database files can be opened only once. It is possible to run multiple transactions and multiple threads against the same open instance and you can also use Client/Server mode if you need multiple transactions.
Later on you may like to try Transparent Activation and Transparent Persistence. Transparent Activation lazily loads object members when they are first accessed. Transparent Persistence automatically stores all objects that were modified in a transaction. For Transparent Activation (TA) and Transparent Persistence (TP) to work you certainly have to keep the ObjectContainer open.
You don't need to worry about constantly having an open database file. One of the key targets of db4o is embedded use in (mobile) devices. That's why we have written db4o in such a way that you can turn your machine off at any time without risking database corruption, even if the file is still open.
Possible reasons why you are getting a GenericObject back instead of a Pilot object:
This can happen when the assembly name of the assembly that contains the Pilot object has changed between two runs, either because you let VisualStudio autogenerate the name or because you changed it by hand.
Maybe "db4o" is part of your assembly name? One of the recent builds was too agressive at filtering out internal classes. This has been fixed quite some time ago. You may like to download and try the latest release, "development" or "production" should both be fine.
In a presentation I once did I have once seen really weird symptoms when db4o ObjectContainers were opened in a "using" block. You probably want to work without that anyway and keep the db4o ObjectContainer open all the time.
It is ok to reopen the database multiple times. The problem would be performance and loosing the "identity". Also you can't keep a reference to a result of a query and try to iterate it after closing the db (based on you code, looks like you want to do that).
GenericObjects are instantiated when the class cannot be found.
Can you provide a full, minimalist, sample that fails for you?
Also, which db4o version are you using?
Best
Related
In our application, we are using grails framework and SQL server for database. We have multiple sites and those sites can have multiple users (a few users) and if they are accessing the same method via AJAX that can cause issue so we made the that method as synchronized method and to minimize the database interaction we are storing data in map on site basis since all the user from one site will get the same data, and if the data is older than 10 seconds we get the data from database and update the map object. Here we are getting a lot of database connection close issues on the very first line of synchronized method where we are getting site object from database. What is the issue here and how we can resolve the issue?
def synchronized getData(params){
Site site = Site.get(params.siteId)
// Here we are checking whether site data does not exists in map
// or the data expired (10 second older data) then we get data from
// database and update the map object
// Then here we create new list object from the data in map object
return list
}
Difficult to figure out the exact problem without more information here. Several things stand out...
I'm not especially familiar with using the synchronized keyword in front of a service method, I would recommend trying the synchronized annotation with a static object key:
private static final myLock = new Object()
#Synchronized("myLock")
void getData() {
//do stuff
}
or synchronizing explicitly within the method
void getData() {
synchronized(myLock) {
//do stuff
}
}
I don't know if that's related to your connection closing issues, but worth a try.
But also notably, grails and hibernate provide caching of database retrieves, so if you're loading the same data that's been loaded into hibernate cache, you don't need to cache this in a Map locally... grails is already doing that for you. Site site = Site.get(params.siteId) will NOT make a database call if it's been called recently and is already cached by the framework.
I would strongly suggest running some performance checks just making that call vs. caching in a Map object, especially if you're expiring in ~10s anyway.
We're using Realm (Swift binding currently in version 3.12.0) from the earliest days in our project. In some early versions before 1.0 Realm provided change listeners for Results without actually giving changeSets.
We used this a lot in order to find out if a specific Results list changed.
Later the guys at Realm exchanged this API with changeSet providing methods. We had to switch and are now mistreating this API just in order to find out if anything in a specific List changed (inserts, deletions, modifications).
Together with RxSwift we wrote our own implementation of Results change listening which looks like this:
public var observable: Observable<Base> {
return Observable.create { observer in
let token = self.base.observe { changes in
if case .update = changes {
observer.onNext(self.base)
}
}
observer.onNext(self.base)
return Disposables.create(with: {
observer.onCompleted()
token.invalidate()
})
}
}
When we now want to have consecutive updates on a list we subscribe like so:
someRealm.objects(SomeObject.self).filter(<some filter>).rx.observable
.subscribe(<subscription code that gets called on every update>)
//dispose code missing
We wrote the extension on RealmCollection so that we can subscribe to List type as well.
The concept is equal to RxRealm's approach.
So now in our App we have a lot of filtered lists/results that we are subscribing to.
When data gets more and more we notice significant performance losses when it comes to seeing a change visually after writing something into the DB.
For example:
Let's say we have a Car Realm Object class with some properties and some 1-to-n and some 1-to-1 relationships. One of the properties is a Bool, namely isDriving.
Now we have a lot of cars stored in the DB and bunch of change listeners with different filters listing to changes of the cars collection (collection observers listening for changeSets in order to find out if the list was changed).
If I take one car of some list and set the property of isDriving from false to true (important: we do writes in the background) ideally the change listener fires fast and I have the nearly immediate correct response to my write on the main thread.
Added with edit on 2019-06-19:
Let's make the scenario still a little more real:
Let's change something down the hierarchy, let's say the tires manufacturer's name. Let's say a Car has a List<Tire>, a Tire has a Manufacturer and a Manufacturer has aname.
Now we're still listing toResultscollection changes with some more or less complex filters applied.
Then we're changing the name of aManufacturer` which is connected to one of the tires which are connected to one of the cars which is in that filtered list.
Can this still be fast?
Obviously when the length of results/lists where change listeners are attached to gets longer Realm's internal change listener takes longer to calculate the differences and fires later.
So after a write we see the changes - in worst case - much later.
In our case this is not acceptable. So we are thinking through different scenarios.
One scenario would be to not use .observe on lists/results anymore and switch to Realm.observe which fires every time anything did change in the realm, which is not ideal, but it is fast because the change calculation process is skipped.
My question is: What can I do to solve this whole dilemma and make our app fast again?
The crucial thing is the threading stuff. We're always writing in the background due to our design. So the writes itself should be very fast, but then that stuff needs to synchronize to the other threads where Realms are open.
In my understanding that happens after the change detection for all Results has run through, is that right?
So when I read on another thread, the data is only fresh after the thread sync, which happens after all notifications were sent out. But I am not sure currently if the sync happens before, that would be more awesome, did not test it by now.
I'm sorry the title may mislead you, since I'm not so good at English. Let me describe my problem as below (You may skip to the TL;DR version at the bottom of this question).
In Coredata, I design a Product entity. In app, I download products from a server. It return JSON string, I defragment it then save to CoreData.
After sometimes has passed, I search a product from that server again, having some interaction with server. Now, I call the online product XProduct. This product may not exist in CoreData, and I also don't want to save it to CoreData since it may not belong to this system (it come from other warehouse, not my current warehouse).
Assume this XProduct has the same properties as Product, but not belong to CoreData, the developer from before has designed another Object, the XProduct, and copy everything (the code) from Product. Wow. The another difference between these two is, XProduct has some method to interact with server, like: - (void)updateStock:(NSInteger)qty;
Now, I want to upgrade the Product properties, I'll have to update the XProduct also. And I have to use these two separately, like:
id product = anArrayContainsProducts[indexPath.row];
if ([product isKindOfClass:[XProduct class]] {
// Some stuff with the xproduct
}
else {
// Probably the same display to the cell.
}
TL;DR
Basically, I want to create a scenario like this:
Get data from server.
Check existed in CoreData.
2 == true => add to array (also may update some data from server).
2 == false => create object (contains same structure as NSManagedObject from JSON dictionary => add to array.
The object created in step 4 will never exist in CoreData.
Questions
How can I create an NSManagedObject without having it add to NSMangedObjectContext and make sure the app would run fine?
If 1 is not encouragement, please suggest me a better approach to this. I really don't like to duplicate so many codes like that.
Update
I was thinking about inheritance (XProduct : Product) but it still make XProduct the subclass of NSManagedObject, so I don't think that is a good approach.
There are a couple of possibilities that might work.
One is just to create the managed objects but not insert them into a context. When you create a managed object, the context argument is allowed to be nil. For example, calling insertNewObjectForEntityForName(_:inManagedObjectContext:) with no context. That gives you an instance of the managed object that's not going to be saved. They have the same lifetime as any other object.
Another is to use a second Core Data stack for these objects, with an in-memory persistent store. If you use NSInMemoryStoreType when adding the persistent store (instead of NSSQLiteStoreType), you get a complete, working Core Data stack. Except that when you save changes, they only get saved in memory. It's not really persistent, since it disappears when the app exits, but aside from that it's exactly the same as any other Core Data stack.
I'd probably use the second approach, especially if these objects have any relationships, but either should work.
I'm building a tab bar application for iPhone and i'm using Core Data with two UIManagedDocuments. In the first tab, i write the data to database and in the second i read them into UITableView with UIFetchedResultsController.
At the start of application, if i write data first, and after then i read results, it works fine. Results appear in the second tab immediately. However, if i read some data first and after then if i write something to database, results appear in second tab with considerable delay (almost 1 minutes). If is there any synchronization problem between two UIManagedObjectContexts or two UIManagedDocuments, how does it works in the first condition? And, is there any solution for this delay?
The way that you can ensure that your UIManagedDocument is up to date is to make sure you're saving your changes properly. Given the information you've shown above, I'm not really sure about how you're managing your documents or your managedObjectContexts. There are just too many factors that could be affecting this to be able to give you a 100% concrete answer.
So without knowing what your code looks like and without knowing how you're managing your context, the only thing I can do is give you what I use in my own projects. This may or may not help you, but it helped me - more times than one - when it comes to handing core data by UIManagedDocument.
When it comes to Context:
I use a singleton to manage UIManagedDocument. I do this because I don't want to have to deal with what you're talking about above - having more than one managedObjectContext. When you start dealing with multiple contexts, you have the issue where the data will not be consistent unless you manage all of your contexts properly. If you save on one but don't update the other - then your data can become out of sync. You also have to make sure that each context is working on the property thread - the Apple Docs is a great resource for understanding the whys ad hows this even matters.
The point is, though, that this is one of the biggest problems with working with UIManagedDocument that isn't as bad as when you're working with pure core data and using a SQL persistent store. The main reason that I've found is because of how UIManagedDocument actually saves to its UIDocument store. It is very unpredictable about when it wants to save. This makes knowing when your UIManagedDocument will actually persist and have your data a freaking shot in the dark. You end having to do all kinds of stupid stuff just to make sure that it is always readily available.
Considering I have a belief (that many, maybe rightfully so, believe is an ignorant belief) that working with core data is hard, and UIManagedDocument makes it easier than it would be if you didn't work with it at all. That being said, I don't like it when working with something as simple as UIManagedDocument begins to get complicated - so I use the one thing that has always kept it simple, and that is a singleton, shared-instance of a single UIManagedDocument so that I only have 1 managedObjectContext, ever, to have to work with.
When it comes to saving:
Whenever I make any significant change to a model ( Create, Update, Delete, Edit ), I always make sure to call [document updateChangeCount:UIDocumentChangeDone]; I do this because I do not use the "undo manager" (NSUndoManager) when working with UIManagedDocument. Simply because I haven't needed to yet, plus because I hate all the "workaround" garbage you have to do with it.
Working only on the Main Thread:
Whenever I do anything with my UIManagedDocument or Core Data, I always make sure its on the main thread. I think I've already said it once, but I'll say it again: working in threads is helpful when you need it, but also when you actually understand threading in general. I like working in threads, but it comes at a cost of complexity that makes me not want to work in them when it comes to core data. With that being the case, I tend to stay strictly on the main thread as this keeps things simple and easy (for me).
Saving the Document
When I absolutely need to make sure that the UIManagedDocument is "saved" ( written to disk ), I have 2 methods that I wrote and use that are always readily available for me to call: saveDocument and forceSaveDocument.
The first one ( saveDocument ) merely checks the context for changes. If it has any, it then checks to see if we have any newly inserted objects. When insertedObjects are found, it obtains the perm ids for these items. You can think of this one as a good way to ensure that your core data model is up to date, and that your managed context is in a safe state, so that when your document is actually saved, that you get everything saved in the state that it needs to be in (your ids are realized, your contexts are clean, and what you are about to save represents a state of your model once all work has been complete on it).
Its big brother, forceSaveDocument, actually calls saveDocument first. Again, to make sure that your actual model/context is saved and proper. If it returns successful ( YES ), then it will actually do the real saving and write the document to disk by means of saveToUrl.
Some Code (hopefully it helps):
Here are those 2 methods in case it helps:
-(BOOL)saveDocument {
NSManagedObjectContext *context = self.document.managedObjectContext;
if(!context.hasChanges) return YES;
NSSet *inserts = [context insertedObjects];
if([inserts count] > 0) {
NSError *error = nil;
if(![context obtainPermanentIDsForObjects:[inserts allObjects] error:&error] && error) {
[self reportError:error];
return NO;
}
}
return YES;
}
-(void)forceSaveDocument {
if( [self saveDocument] ) {
[self.document saveToURL:self.document.fileURL forSaveOperation:UIDocumentSaveForOverwriting completionHandler:self.onSaveBlock ? self.onSaveBlock : nil];
}
}
General Rules/Guidelines
Overall, these are my guidelines that I follow ( and have worked for me for about 3 years now ) when working with UIManagedDocument and Core Data. I'm sure there are better out there from guys/gals much smarter than me, but these have what I use. The benefit I get out of them is that it makes me have to worry less about managing my data and gives me more freedom to work with everything else:
Use a singleton to manage my UIManagedDocument until the need of multiple threads is absolutely necessary - then migrate over to start using multiple contexts ( i've never needed to do this yet - but then again I try to keep things simple )
Always call updateChangeCount:UIDocumentChangeDone when I make any change to a model. It is very light weight and has little impact. If anything, it will help ensure your document stays up to date and never gets too out of sync with your data.
Don't use undo manager unless you actually need it ( I have yet to need it )
Use save/ForceSave sparingly, and only when absolutely necessary (deletes are a good reason to use it. Or if you create a new item on one view controller and need it on the next one, but can't wait for core data and the document to sync up - its kind of like kicking it in the ass and saying "I object to you saving whenever you want - save now lol.. )
Final Thoughts
All of the above is my own belief and understandings. These come from a lot of research, reading, and being a pain the ass when it comes to wanting to do things right, all while keeping it simple. Anyone can write a complex solution - but I think the fundamental question is always: do you really need the complexity, or do you just need it to work so you can focus on more complex issues?
I'm sure the above is way more than you probably wanted, and may even add more questions than you have. If you need some links and resources, let me know and I'll try to throw a few together.
Either way, hope that helps.
I don't understand this paradigm I guess?
For a small single server or development environment... I hate having to load 100's of thousands of records just to analyze it in a graph... am I missing the big picture here?
UPDATE (3/21/2012 10:38a):
My current setup:
Default Install
Default Configs
Server Setup
Creating nodes via REST API
How do you instantiate your database, embedded or server? Are you running ImpermanentGraphDatabase, because that's the in-memory test database. If you use the normal EmbeddedGraphDatabase your graph is persisted trasactionally along the way when you insert your data.
Please give a little more information.
If using Java embedded transactions must be closed when saving objects or they might get lost. In earlier versions this was done by calling finally { tx.finish(); }, later versions (2.1+) it should happen automatically when instantiated within the try-with-resource. (This makes it possible to run into problems if the Transaction tx is instantiated outside the try clause).
GraphDatabaseService graphDb = new GraphDatabaseFactory().newEmbeddedDatabase(DB_PATH);
try (Transaction tx = graphDb.beginTx()) {
// create some nodes here
}