The class GraphDatabaseService seems not provide any method to drop/clear the database. It there any other means to drop/clear the current embedded database with Java?
Just perform a GraphDatabaseService.shutdown() and after it has returned, remove the database files (using code like this).
You could also use getAllNodes() to iterate over all nodes, delete their relationships and the nodes themselves. Maybe avoid deleting the reference node.
If your use case is testing, then you could use the ImpermanentGraphDatabase, which will delete the database after shutdown.
To use ImpermanentGraphDatabase add the neo4j-kernel tests jar/dependency to your project. Look for the file with a name ending with "tests.jar" on maven central.
I think the easiest way is to delete a directory with neo4j database. I do it in my junit tests after running all tests. Here is a function I use where file is the neo4j directory:
public static void deleteFileOrDirectory( final File file ) {
if ( file.exists() ) {
if ( file.isDirectory() ) {
for ( File child : file.listFiles() ) {
deleteFileOrDirectory( child );
}
}
file.delete();
}
}
I think I found it on neo4j wiki. I have found in this discussion another solution. You can use Blueprint API, which provide method clear.
Like nawroth said, for testing you should use the ImpermanentGraphDatabase. It pretty much auto-fixes all your problems.
If you're not testing, there are two ways really. I generally have two methods available to me. One is the clearDB method, in which I recursively delete the DB path. I use the FileUtils library for this, and it's pretty much a single line of code :
FileUtils.deleteRecursively(new File(DB_PATH));
The other one is to remove every node in the database EXCEPT THE REFERENCE NODE, using the removeAllNodes method. There is a simple query for this, which you execute like this:
engine.execute("START n = node(*), ref = node(0) WHERE n<>ref DELETE n");
Important to note is that you have to call the clearDB method BEFORE you create a new EmbeddedGraphDatabase object. The removeAllNodes method is called AFTER you've created this object.
There is a helper class
Neo4jHelper.cleanDb(db);
(it comes from org.springframework.data.neo4j.support.node.Neo4jHelper and the db you reference is a GraphDatabaseService)
You also have the ability to dump it:
Neo4jHelper.dumpDb();
Related
I'm making my first steps in using Redis under ZF2.
I was wondering if there is a method to retrieve keys by pattern.
e.g.:
after setting multiple values with keys like: 'stackOverflow_'.time(), i would like to retrieve later all keys matching the 'stackOverflow_' pattern.
tried using getItems(array $keys) with wildcard in: \vendor\zendframework\zendframework\library\Zend\Cache\Storage\Adapter\AbstractAdapter.php
$redisKeyPattern = 'stackOverflow_';
$redis = $this->getServiceLocator()->get('Redis');
$values = $redis->getItems(array($redisKeyPattern.'*'));
with no succces.
any ideas?
UDPATE:
thanks guys. i ended up with duplicating the Redis adapter and adding my own functionality that utilizes the 'keys' function in the Redis extension:
public function getItemsByKeyPattern($pattern) {
$keys = $this->getRedisResource()->keys('*'.$pattern.'*');
if(empty($keys)) return null;
foreach($keys as &$key){
$key = explode(':', $key)[1];
}
$items = parent::getItems($keys);
return $items;
}
and it works for me :)
sadly to say there is no method present to return items with a wildcard, also redis don't support namespaces for stored items.
you need to define each item you want to receive, maybe you should look at a implementation like this
$receiveRedisKeys = [];
foreach($resultSet as $result)
{
$receiveRedisKeys[] = 'predefined_prefix_' . $result->getId();
}
$redisCacheResultSet = $redis->getItems($receiveRedisKeys);
i know that someone on github made a new repository where he modified redis to allow namespaces but this requires that you build the redis binarys by yourself from source. this leeds to a redis version you can't update anymore over apt-get
It's not possible, but there are some alternatives.
One idea is to keep a set with the keys you are interested in. That is the most common approach to this problem: each time you create one of the keys you will want to retrieve later, you add its name to a set. Then when you need to operate on one of those keys, you can grab it from the set. Read this article to get a general idea about this approach.
Another idea is to use the SCAN command to walk the keyspace with the pattern you are using, and as a second step retrieve the values with MGET followed by the keys you collected. This approach is good for administrative processes, but not as something that should be included in an application because the performance will be worse than that of the first idea. More about SCAN.
Finally, an option that is not recommended but I'm listing it just for completeness is to use the KEYS command to collect the keys you want, then proceed to get the values with MGET, as in the SCAN approach. This is not recommended as KEYS shouldn't be used in production environments. More about KEYS.
I've got a domain class, Widget, that I need to delete all instances out of -- clear it out. After that, I will load in fresh data. What do you suggest as a mechanism to do this?
P.S. Note this is not at bootstrap time, but at "run-time".
The easiest way is to use HQL directly:
DomainClass.executeUpdate('delete from DomainClass')
DomainClass.findAll().each { it.delete() }
If you want to avoid any GORM gotchas, such as needing to delete the object immediately and checking to make sure it actually gets deleted, add some arguments.
DomainClass.findAll().each { it.delete(flush:true, failOnError:true) }
Fairly old post, but still actual.
If your table is very large (millions of entries), iterating using findall()*.delete() might not be the best option, as you can run into transaction timeouts (e.g. MySQL innodb_lock_wait_timeout setting) besides potential memory problems stated by GreenGiant.
So at least for MySQL Innodb, much faster is to use TRUNCATE TABLE:
sessionFactory.currentSession
.createSQLQuery("truncate table ${sessionFactory.getClassMetadata(MyDomainClass).tableName}")
.executeUpdate()
This is only useful if your table is not referenced by other objects as a foreign key.
From what I learnt, I agree with #ataylor the below code is fastest IF there are no associations in your domain object (Highly unlikely in any real application):
DomainClass.executeUpdate('delete from DomainClass')
But if you have assiciations with other domains, then the safest way to delete (and also a bit slower than the one mentioned above) would be the following:
def domainObjects = DomainClass.findAll()
domainObjects.each {
it.delete(flush:it==domainObjects.last, failOnError:true)
}
If you have a list of objects and want to delete all elements, you can use * operator.
'*' will split the list and pass its elements as separate arguments.
Example.
List<Book> books = Book.findAllByTitle('grails')
books*.delete()
I have no idea if I'm doing this right, but this is how a Get method in my repository looks:
public IQueryable<User> GetUsers(IEnumerable<Expression<Func<User, object>>> eagerLoading)
{
IQueryable<User> query = db.Users.AsNoTracking();
if (eagerLoading != null)
{
foreach (var expression in eagerLoading)
{
query = query.Include(expression);
}
}
return query;
}
Lets say I also have a GeographyRepository that has GetCountries method, which is similar to this.
I have 2 separate service layer classes calling these 2 separate repositories, sharing the same DbContext (EF 4.1 code-first).
So in my controller, I'd do:
myViewModel.User = userService.GetUserById(1);
myViewModel.Countries = geoService.GetCountries();
This is 2 separate calls to the database. If I didn't use these patterns and tie up the interface and database, I'd have 1 call. I guess its something of a performance vs maintainability.
My question is, can this be pushed to 1 database call? Can we merge queries like this when views calls multiple repositories?
I'd say that if performance is the real issue then I'd try and avoid going back to the database altogether. I'm assuming the list returned from geoService.GetCountries() is fairly static, so I'd be inclined to cache it in the service after the initial load and remove the database hit altogether. The fact that you have a service there suggests that it would be the perfect place to abstract away such details.
Generally when asking questions about performance, it's rare that all perf related issues can be tarred with the same brush and you need to analyse each situation and work out an appropriate solution for the specific perf issue you're having.
This is the way I am thinking of using DB4o. When I need to query, I would open the file, read and close:
using (IObjectContainer db = Db4oFactory.OpenFile(Db4oFactory.NewConfiguration(), YapFileName))
{
try
{
List<Pilot> pilots = db.Query<Pilot>().ToList<Pilot>();
}
finally
{
try { db.Close(); }
catch (Exception) { };
}
}
At some later time, when I need to insert, then
using (IObjectContainer db = Db4oFactory.OpenFile(Db4oFactory.NewConfiguration(), YapFileName))
{
try
{
Pilot pilot1 = new Pilot("Michael Schumacher", 100);
db.Store(pilot1);
}
finally
{
try { db.Close(); }
catch (Exception) { };
}
}
In this way, I thought I will keep the file more tidy by only having it open when needed, and have it closed most of the time. But I keep getting InvalidCastException
Unable to cast object of type 'Db4objects.Db4o.Reflect.Generic.GenericObject' to type 'Pilot'
What's the correct way to use DB4o?
No, it's not a good idea to work this way. db4o ObjectContainers are intended to be kept open all the time your application runs. A couple of reasons:
db4o maintains a reference system to identify persistent objects, so it can do updates when you call #store() on an object that is already stored (instead of storing new objects) . This reference system is closed when you close the ObjectContainer, so updates won't work.
Class Metadata would have to be read from the database file every time you reopen it. db4o would also have to analyze the structure of all persistent classes again, when they are used. While both operations are quite fast, you probably don't want this overhead every time you store a single object.
db4o has very efficient caches for class and field indexes and for the database file itself. If you close and reopen the file, you take no advantage of them.
The way you have set up your code there could be failures when you work with multiple threads. What if two threads would want to open the database file at exactly the same time? db4o database files can be opened only once. It is possible to run multiple transactions and multiple threads against the same open instance and you can also use Client/Server mode if you need multiple transactions.
Later on you may like to try Transparent Activation and Transparent Persistence. Transparent Activation lazily loads object members when they are first accessed. Transparent Persistence automatically stores all objects that were modified in a transaction. For Transparent Activation (TA) and Transparent Persistence (TP) to work you certainly have to keep the ObjectContainer open.
You don't need to worry about constantly having an open database file. One of the key targets of db4o is embedded use in (mobile) devices. That's why we have written db4o in such a way that you can turn your machine off at any time without risking database corruption, even if the file is still open.
Possible reasons why you are getting a GenericObject back instead of a Pilot object:
This can happen when the assembly name of the assembly that contains the Pilot object has changed between two runs, either because you let VisualStudio autogenerate the name or because you changed it by hand.
Maybe "db4o" is part of your assembly name? One of the recent builds was too agressive at filtering out internal classes. This has been fixed quite some time ago. You may like to download and try the latest release, "development" or "production" should both be fine.
In a presentation I once did I have once seen really weird symptoms when db4o ObjectContainers were opened in a "using" block. You probably want to work without that anyway and keep the db4o ObjectContainer open all the time.
It is ok to reopen the database multiple times. The problem would be performance and loosing the "identity". Also you can't keep a reference to a result of a query and try to iterate it after closing the db (based on you code, looks like you want to do that).
GenericObjects are instantiated when the class cannot be found.
Can you provide a full, minimalist, sample that fails for you?
Also, which db4o version are you using?
Best
Using Symfony 1.4.5 with Doctrine
I have a model which includes an uploaded image as one of the columns - creating and updating the record is fine (using the doSave() method to deal with the upload and any changes to the file).
The problem I'm having is if the record is deleted - I want it to remove the associated file as well. But I can't find anyway to do this after several hours of hunting through documentation and Google.
Is there a way to specify some kind of post-delete code?
Final solution:
in /lib/model/doctrine/Image.class.php
class Image extends BaseImage
{
public function postDelete()
{
$filename = $this->getFilename();
$filepath = sfConfig::get('sf_upload_dir') . '/' . $filename;
#unlink($filepath);
}
}
Thanks to Colonel Sponz for pointing me in the right direction
It's a while since I last used Doctrine but I seem to remember there is a post delete hook function that you can use for this kind of thing. If you look into the source for the Doctrine base class you should be able to find the exact method name and usage.
EDIT: The method is postDelete() and is found in the Doctrine_Record class
Here's the section from the Symfony documentation that covers advanced Doctrine usage.
Hijacking Colonel Sponsz's answer, the postDelete() method is definitely the way to go. +1 to him :-) But, you'll need to enable Doctrine callbacks in your config/ProjectConfiguration.class.php. Add this method:
public function configureDoctrine(Doctrine_Manager $manager)
{
$manager->setAttribute(Doctrine_Core::ATTR_USE_DQL_CALLBACKS, true);
}
Clear your Symfony cache, and Doctrine will fire the callback methods such as postDelete() at the appropriate times.