Zf2 Redis Adapter, getItems using wildcards - zend-framework2

I'm making my first steps in using Redis under ZF2.
I was wondering if there is a method to retrieve keys by pattern.
e.g.:
after setting multiple values with keys like: 'stackOverflow_'.time(), i would like to retrieve later all keys matching the 'stackOverflow_' pattern.
tried using getItems(array $keys) with wildcard in: \vendor\zendframework\zendframework\library\Zend\Cache\Storage\Adapter\AbstractAdapter.php
$redisKeyPattern = 'stackOverflow_';
$redis = $this->getServiceLocator()->get('Redis');
$values = $redis->getItems(array($redisKeyPattern.'*'));
with no succces.
any ideas?
UDPATE:
thanks guys. i ended up with duplicating the Redis adapter and adding my own functionality that utilizes the 'keys' function in the Redis extension:
public function getItemsByKeyPattern($pattern) {
$keys = $this->getRedisResource()->keys('*'.$pattern.'*');
if(empty($keys)) return null;
foreach($keys as &$key){
$key = explode(':', $key)[1];
}
$items = parent::getItems($keys);
return $items;
}
and it works for me :)

sadly to say there is no method present to return items with a wildcard, also redis don't support namespaces for stored items.
you need to define each item you want to receive, maybe you should look at a implementation like this
$receiveRedisKeys = [];
foreach($resultSet as $result)
{
$receiveRedisKeys[] = 'predefined_prefix_' . $result->getId();
}
$redisCacheResultSet = $redis->getItems($receiveRedisKeys);
i know that someone on github made a new repository where he modified redis to allow namespaces but this requires that you build the redis binarys by yourself from source. this leeds to a redis version you can't update anymore over apt-get

It's not possible, but there are some alternatives.
One idea is to keep a set with the keys you are interested in. That is the most common approach to this problem: each time you create one of the keys you will want to retrieve later, you add its name to a set. Then when you need to operate on one of those keys, you can grab it from the set. Read this article to get a general idea about this approach.
Another idea is to use the SCAN command to walk the keyspace with the pattern you are using, and as a second step retrieve the values with MGET followed by the keys you collected. This approach is good for administrative processes, but not as something that should be included in an application because the performance will be worse than that of the first idea. More about SCAN.
Finally, an option that is not recommended but I'm listing it just for completeness is to use the KEYS command to collect the keys you want, then proceed to get the values with MGET, as in the SCAN approach. This is not recommended as KEYS shouldn't be used in production environments. More about KEYS.

Related

passing collections as parameters with neo4j

I have been using parameters to query node indexes as such (using the rest api in java)-
final QueryResult<Map<String,Object>> result = engine.query("start nd=node:name_index(name={src}) return nd.age as age", MapUtil.map("src", "Susan");
However I haven't been able to get this to work for a collection of nodes/names. I have been trying something along the lines of-
final QueryResult<Map<String,Object>> result = engine.query("start nd=node:name_index(name={src}) return nd.age as age", MapUtil.map("src", Arrays.asList("Susan","Brian", "Ian"));
But it refuses to compile. I as wondering if there is something wrong in my syntax or that parameters are not designed to work in this context.
The name= syntax in the start is meant to do an index lookup on a property. It won't do an IN lookup. The way you can do this sort of lookup is like this (note it depends on Apache's StringUtils):
List<String> names = Arrays.asList("Susan","Brian", "Ian");
String luceneQuery = "name:("+StringUtils.join(names, ",")+")";
engine.query("start nd=node:name_index({luceneQuery}) return nd.age as age", MapUtil.map("luceneQuery", luceneQuery));
Just a note, this is the "legacy" index way of doing things. In 2.0 they've introduced label-based indexes, which work entirely differently.
Thanks a lot; though it would still only return a non empty answer when I added a space after the comma in line 2. I used-
String luceneQuery = "name:("+StringUtils.join(names, ", ")+")";
and it returned the age of one person. When I tried this:
String luceneQuery = "fs:(fs:"+ StringUtils.join(names, " OR fs:")+")";
it gave me all three ages. However, I am still unsure about whether this query will be able to leverage the usual advantages of parameters , i.e. will the engine be able to reuse the query and execution path the next time around (this time we may want to query for 4 names instead of 3)

ServiceStack - Repository Injection By Name

All,
I have read up on the way SS uses Func to wire registrations. My current issue is that I am still not seeing how to call a specific instance from runtime.
What I would like to do is set up two different repository classes representing two different database systems, say SQLDB and MongoDB, both of which inherit from IDB, then be able to determine which database to use based on an app setting in the config file.
What I have right now in my Configure method is just
container.Register("SQLDB", new TestSQLDB("connectionName"));
container.Register("MongoDB", new TestMongoDB("mongoURL"));
If anyone can help me fill in the blanks I'd appreciate it. I already managed this with Ninject, but I would prefer not to have to add it if I don't need to.
Thanks,
Bs
Register via concrete type (or interface if you prefer):
container.Register<SQLDB>(new TestSQLDB("connectionName"));
container.Register<MongoDB>(new TestMongoDB("mongoURL"));
When you want one back:
var mySqlDB = container.Resolve<SQLDB>();
var myMongoDB = container.Resolve<MongoDB>();
Thanks for your help Gavin. You got me on the right track. Here is what worked.
1) In the config:
container.Register<ITest>("SQLDB", new TestSQLDB("xxxx"));
container.Register<ITest>("MongoDB", new TestMongoDB("mongo://127.0.0.1"));
2) In the service:
IAppHost appHost = base.GetAppHost();
var repository = appHost.GetContainer().ResolveNamed<ITest>("MongoDB");
List<Test> testlist = repository.LoadAll();
This way I can just replace "MongoDB" with something from the config file and I don't need to recompile to change data sources.

grails delete all data from table / domain class, i.e. "deleteAll"

I've got a domain class, Widget, that I need to delete all instances out of -- clear it out. After that, I will load in fresh data. What do you suggest as a mechanism to do this?
P.S. Note this is not at bootstrap time, but at "run-time".
The easiest way is to use HQL directly:
DomainClass.executeUpdate('delete from DomainClass')
DomainClass.findAll().each { it.delete() }
If you want to avoid any GORM gotchas, such as needing to delete the object immediately and checking to make sure it actually gets deleted, add some arguments.
DomainClass.findAll().each { it.delete(flush:true, failOnError:true) }
Fairly old post, but still actual.
If your table is very large (millions of entries), iterating using findall()*.delete() might not be the best option, as you can run into transaction timeouts (e.g. MySQL innodb_lock_wait_timeout setting) besides potential memory problems stated by GreenGiant.
So at least for MySQL Innodb, much faster is to use TRUNCATE TABLE:
sessionFactory.currentSession
.createSQLQuery("truncate table ${sessionFactory.getClassMetadata(MyDomainClass).tableName}")
.executeUpdate()
This is only useful if your table is not referenced by other objects as a foreign key.
From what I learnt, I agree with #ataylor the below code is fastest IF there are no associations in your domain object (Highly unlikely in any real application):
DomainClass.executeUpdate('delete from DomainClass')
But if you have assiciations with other domains, then the safest way to delete (and also a bit slower than the one mentioned above) would be the following:
def domainObjects = DomainClass.findAll()
domainObjects.each {
it.delete(flush:it==domainObjects.last, failOnError:true)
}
If you have a list of objects and want to delete all elements, you can use * operator.
'*' will split the list and pass its elements as separate arguments.
Example.
List<Book> books = Book.findAllByTitle('grails')
books*.delete()

How to drop the neo4j embedded database with java?

The class GraphDatabaseService seems not provide any method to drop/clear the database. It there any other means to drop/clear the current embedded database with Java?
Just perform a GraphDatabaseService.shutdown() and after it has returned, remove the database files (using code like this).
You could also use getAllNodes() to iterate over all nodes, delete their relationships and the nodes themselves. Maybe avoid deleting the reference node.
If your use case is testing, then you could use the ImpermanentGraphDatabase, which will delete the database after shutdown.
To use ImpermanentGraphDatabase add the neo4j-kernel tests jar/dependency to your project. Look for the file with a name ending with "tests.jar" on maven central.
I think the easiest way is to delete a directory with neo4j database. I do it in my junit tests after running all tests. Here is a function I use where file is the neo4j directory:
public static void deleteFileOrDirectory( final File file ) {
if ( file.exists() ) {
if ( file.isDirectory() ) {
for ( File child : file.listFiles() ) {
deleteFileOrDirectory( child );
}
}
file.delete();
}
}
I think I found it on neo4j wiki. I have found in this discussion another solution. You can use Blueprint API, which provide method clear.
Like nawroth said, for testing you should use the ImpermanentGraphDatabase. It pretty much auto-fixes all your problems.
If you're not testing, there are two ways really. I generally have two methods available to me. One is the clearDB method, in which I recursively delete the DB path. I use the FileUtils library for this, and it's pretty much a single line of code :
FileUtils.deleteRecursively(new File(DB_PATH));
The other one is to remove every node in the database EXCEPT THE REFERENCE NODE, using the removeAllNodes method. There is a simple query for this, which you execute like this:
engine.execute("START n = node(*), ref = node(0) WHERE n<>ref DELETE n");
Important to note is that you have to call the clearDB method BEFORE you create a new EmbeddedGraphDatabase object. The removeAllNodes method is called AFTER you've created this object.
There is a helper class
Neo4jHelper.cleanDb(db);
(it comes from org.springframework.data.neo4j.support.node.Neo4jHelper and the db you reference is a GraphDatabaseService)
You also have the ability to dump it:
Neo4jHelper.dumpDb();

Code re-use with Linq-to-Sql - Creating 'generic' look-up tables

I'm working on an application at the moment in ASP.NET MVC which has a number of look-up tables, all of the form
LookUp {
Id
Text
}
As you can see, this just maps the Id to a textual value. These are used for things such as Colours. I now have a number of these, currently 6 and probably soon to be more.
I'm trying to put together an API that can be used via AJAX to allow the user to add/list/remove values from these lookup tables, so for example I could have something like:
http://example.com/Attributes/Colours/[List/Add/Delete]
My current problem is that clearly, regardless of which lookup table I'm using, everything else happens exactly the same. So really there should be no repetition of code whatsoever.
I currently have a custom route which points to an 'AttributeController', which figures out the attribute/look-up table in question based upon the URL (ie http://example.com/Attributes/Colours/List would want the 'Colours' table). I pass the attribute (Colours - a string) and the operation (List/Add/Delete), as well as any other parameters required (say "Red" if I want to add red to the list) back to my repository where the actual work is performed.
Things start getting messy here, as at the moment I've resorted to doing a switch/case on the attribute string, which can then grab the Linq-to-Sql entity corresponding to the particular lookup table. I find this pretty dirty though as I find myself having to write the same operations on each of the look-up entities, ugh!
What I'd really like to do is have some sort of mapping, which I could simply pass in the attribute name and get out some form of generic lookup object, which I could perform the desired operations on without having to care about type.
Is there some way to do this to my Linq-To-Sql entities? I've tried making them implement a basic interface (IAttribute), which simply specifies the Id/Text properties, however doing things like this fails:
System.Data.Linq.Table<IAttribute> table = GetAttribute("Colours");
As I cannot convert System.Data.Linq.Table<Colour> to System.Data.Linq.Table<IAttribute>.
Is there a way to make these look-up tables 'generic'?
Apologies that this is a bit of a brain-dump. There's surely imformation missing here, so just let me know if you'd like any further details. Cheers!
You have 2 options.
Use Expression Trees to dynamically create your lambda expression
Use Dynamic LINQ as detailed on Scott Gu's blog
I've looked at both options and have successfully implemented Expression Trees as my preferred approach.
Here's an example function that i created: (NOT TESTED)
private static bool ValueExists<T>(String Value) where T : class
{
ParameterExpression pe = Expression.Parameter(typeof(T), "p");
Expression value = Expression.Equal(Expression.Property(pe, "ColumnName"), Expression.Constant(Value));
Expression<Func<T, bool>> predicate = Expression.Lambda<Func<T, bool>>(value, pe);
return MyDataContext.GetTable<T>().Where(predicate).Count() > 0;
}
Instead of using a switch statement, you can use a lookup dictionary. This is psuedocode-ish, but this is one way to get your table in question. You'll have to manually maintain the dictionary, but it should be much easier than a switch.
It looks like the DataContext.GetTable() method could be the answer to your problem. You can get a table if you know the type of the linq entity that you want to operate upon.
Dictionary<string, Type> lookupDict = new Dictionary<string, Type>
{
"Colour", typeof(MatchingLinqEntity)
...
}
Type entityType = lookupDict[AttributeFromRouteValue];
YourDataContext db = new YourDataContext();
var entityTable = db.GetTable(entityType);
var entity = entityTable.Single(x => x.Id == IdFromRouteValue);
// or whatever operations you need
db.SubmitChanges()
The Suteki Shop project has some very slick work in it. You could look into their implementation of IRepository<T> and IRepositoryResolver for a generic repository pattern. This really works well with an IoC container, but you could create them manually with reflection if the performance is acceptable. I'd use this route if you have or can add an IoC container to the project. You need to make sure your IoC container supports open generics if you go this route, but I'm pretty sure all the major players do.

Resources