I'm writing lua scripts for PowerDNS.
I wish to get some kind of a unique ID for each DNS query when my script intercepts a DNS query.
I'm intercepting the DNS queries with the "preresolve" and "postresolve" functions and they don't have any unique ID that will symbolize the current DNS query it's handling.
I saw that when using the function "ipfilter", I can use the GetID() function to get the DNS query ID.
This ability is missing in the "preresolve" and "postresolve" functions.
Any ideas?
After upgrading from PDNS Recursor version 3.0 to 4.0 I found my answer.
In version 4, 'preresolve' and 'postresolve' functions get the dq (DNSQuestion) object,
the dq object contains two fields that can help with the query ID.
One, you can use the original query ID by getting the value from
dq:getDH():getID()
Two, dq can contain a custom LUA object reference that is persistent throughout the lifetime of the dq object.
local data={}
data["domaindetails"]= dq.udpAnswer
dq.data=data
Find more details in their Scripting manual:
https://doc.powerdns.com/md/recursor/scripting/#the-dnsquestion-dq-object
Related
spring-data-neo4j query using dynamic key and dynamic value,
like following code:
public interface NodeReposity extends Neo4jRepository<Node,Long> {
#Query("MATCH(n:Node{{key}={value}})return n")
Iterable<Node> queryByProperty(#Param("key")String key,#Param("value") String value);
}
But it says the {key} must be something like variable in string, such as MATCH(n:Node{name={value}})return n.Can't be {key}. But My property's key is dynamic like the value, how to implement it and is it possible?
Short answer: The query will be send "as it is" to the database and because cypher does only support placeholders for values, this will cause an error.
Slightly longer answer: When it comes to executing the method Spring Data Neo4j will look if it has already pre-processed the query and either process and cache it or just load it from the cache. This is done to improve the time it takes to execute the method from the application.
Pre-processing means SDN knows what parameters are in there and just adds the values in the right place when the method is called.
If SDN would provide more features for the query than cypher, the query would have to be processed every time the method gets called to create a new query that can be used with Neo4j.
I was working through some code, and noticed:
return new Uri(host: server, path: apiPath, query: query, queryParameters: queryParams);
This code is executed regularly throughout the application, and the only difference was queryParams. So i printed it out:
{Id:[1234], enabled:false}
shows it is a key:value set of: Id:List, enabled:boolean.
The stack trace i get is:
which shows the map and then the trace. #6 points to the above line.
It is looking at false... something with iterating false is what breaks this.
When dealing with the URI and query parameters, it is looking for numerics, lists, and strings but not booleans. In order to resolve this and allow it to function correctly, you will need to do:
{"enabled": false.toString()}
// or
{"enabled": "false"}
and the uri class will set the query parameter accordingly.
The Uri class is located in core library for Dart. When we are using it, we are passing in the created Uri object into an action for a client class,
Client client = new BrowserClient();
which accepts the url as a part of the parameters.
While looking at the errors above though, the Uri class ultimately is unable to properly parse a false value to an accepted value.
When looking at the Code Docs for Uri as per the Dart languages: https://api.dartlang.org/dev/1.25.0-dev.7.0/dart-core/Uri/Uri.html
The query component is set through either query or queryParameters. When query is used, the provided string should be a valid URI query, but invalid characters, other than general delimiters, will be escaped if necessary. When queryParameters is used the query is built from the provided map. Each key and value in the map is percent-encoded and joined using equal and ampersand characters. A value in the map must be either a string, or an Iterable of strings, where the latter corresponds to multiple values for the same key.
Which makes sense to say all values must be String or an Iterable of Strings. The only thing which I cant figure out is that in Dartpad, true and false have toString functions, and yet you can also pass numerics in there.
The only conclusion is that while it accepts Strings and Iterables of Strings, it will also parse ints and other numerics because they will explicitly check for that type as it is common to see in URI.
One would think that the URI would understand booleans since those are also common place, but that is yet to be seen since I cant take an explicit look at the source code for dartlang. I did however manage to look at the source code for it and narrowed it down. writeComponent points to _Uri._uriEncode but when looking at that function, there is no code as much as just a definition.
HTH.
I'm making my first steps in using Redis under ZF2.
I was wondering if there is a method to retrieve keys by pattern.
e.g.:
after setting multiple values with keys like: 'stackOverflow_'.time(), i would like to retrieve later all keys matching the 'stackOverflow_' pattern.
tried using getItems(array $keys) with wildcard in: \vendor\zendframework\zendframework\library\Zend\Cache\Storage\Adapter\AbstractAdapter.php
$redisKeyPattern = 'stackOverflow_';
$redis = $this->getServiceLocator()->get('Redis');
$values = $redis->getItems(array($redisKeyPattern.'*'));
with no succces.
any ideas?
UDPATE:
thanks guys. i ended up with duplicating the Redis adapter and adding my own functionality that utilizes the 'keys' function in the Redis extension:
public function getItemsByKeyPattern($pattern) {
$keys = $this->getRedisResource()->keys('*'.$pattern.'*');
if(empty($keys)) return null;
foreach($keys as &$key){
$key = explode(':', $key)[1];
}
$items = parent::getItems($keys);
return $items;
}
and it works for me :)
sadly to say there is no method present to return items with a wildcard, also redis don't support namespaces for stored items.
you need to define each item you want to receive, maybe you should look at a implementation like this
$receiveRedisKeys = [];
foreach($resultSet as $result)
{
$receiveRedisKeys[] = 'predefined_prefix_' . $result->getId();
}
$redisCacheResultSet = $redis->getItems($receiveRedisKeys);
i know that someone on github made a new repository where he modified redis to allow namespaces but this requires that you build the redis binarys by yourself from source. this leeds to a redis version you can't update anymore over apt-get
It's not possible, but there are some alternatives.
One idea is to keep a set with the keys you are interested in. That is the most common approach to this problem: each time you create one of the keys you will want to retrieve later, you add its name to a set. Then when you need to operate on one of those keys, you can grab it from the set. Read this article to get a general idea about this approach.
Another idea is to use the SCAN command to walk the keyspace with the pattern you are using, and as a second step retrieve the values with MGET followed by the keys you collected. This approach is good for administrative processes, but not as something that should be included in an application because the performance will be worse than that of the first idea. More about SCAN.
Finally, an option that is not recommended but I'm listing it just for completeness is to use the KEYS command to collect the keys you want, then proceed to get the values with MGET, as in the SCAN approach. This is not recommended as KEYS shouldn't be used in production environments. More about KEYS.
Hello I am trying to execute a query using breezejs 1.3.4 . My query is the following
function getContacts() {
var query = breeze.EntityQuery
.from("Contacts").where("Desc", "startsWith", "P");
return manager.executeQuery(query)
.then(getSucceeded).fail(getFailed);
}
"Desc" is a string property in my "Contacts" C# backend model. Tha problem is that the Query URL is formatted like this .../api/Application/Contacts?$filter=startswith(Desc%2Ctime'P')%20eq%20true
The word time is added before "P" and I get a this exception in the Response
{"$id":"1","$type":"System.Web.Http.HttpError, System.Web.Http","Message":"The query specified in the URI is not valid.","ExceptionMessage":"Unrecognized 'Edm.Time' literal 'time'P''
If in the comparison I use a lower case "p" then the Url is costructed as it should be like this
"$filter=startswith(Desc%2C'p')%20eq%20true` .
I don't have the same problem when using other UpperCase letters of the english alphabet.
Does anyone have an idea what am I missing, I can't figure out why the word "time" is added in that specific query?
Thank you.
We were able to reproduce the problem.
While the exception message is confusing, I believe you might be getting the error because you have not associated that the resource name 'Contacts' with an EntityType of presumably 'Contact'.
What is happening is that when Breeze tries to build the query it will usually use its metadata to correctly format the url query string to send to the server. The critical part of this process involves determining the EntityType associated with the resourceName given in the 'from' clause of the query ('Contacts' in your case). Breeze uses a resource name to entity type map to do this, but if it cannot find a mapping, it will still continue to build the url because we still need to support ad-hoc requests against endpoints for which we have no metadata.
You can update this map yourself via the MetadataStore.setEntityTypeForResourceName method. If you are using the Entity Framework, this map is initially populated based on the EntityType name/Entity Set name mapping that is part of your EDMX model. So in your case you can either modify your EDMX model or call the setEntityTypeForResourceName method directly.
Unfortunately, without the metadata Breeze has to infer the datatypes in the query. So in your case
"Desc", "startsWith", "P"
since Breeze can't determine that "Desc" is a 'string' property of your Contract EntityType it tries to infer the 'dataType' of "Desc" based on the value 'P'. Unfortunately, 'P' is a valid ISO8601 'duration' value ( a way to express a 'Time' datatype), so Breeze incorrectly tries to construct a query string with 'P' treated as a 'Time' constant. This is why your code works with a lower case 'p' ( not a valid duration value).
This inference logic can certainly be improved and we have a fix that will allow us to do exactly that coming out in the next release. However... the better and more general solution to this type of issue is to get the 'resourceName/entityType' mappings correct in the first place.
Hope this helps.
I have a legacy dataset (ENRON data represented as GraphML) that I would like to query. In an comment in a related question, #StefanArmbruster suggests that I use Cypher to query the database. My query use case is simple: given a message id (a property of the Message node), retrieve the node that has that id, and also retrieve the sender and recipient nodes of that message.
It seems that to do this in Cypher, I first have to create an index of the nodes. Is there a way to do this automatically when the data is loaded from the graphML file? (I had used Gremlin to load the data and create the database.)
I also have an external Lucene index of the data (I need it for other purposes). Does it make sense to have two indexes? I could, for example, index the Neo4J node ids into my external index, and then query the graph based on those ids. My concern is about the persistence of these ids. (By analogy, Lucene document ids should not be treated as persistent.)
So, should I:
Index the Neo4j graph internally to query on message ids using Cypher? (If so, what is the best way to do that: regenerate the database with some suitable incantation to get the index built? Build the index on the already-existing db?)
Store Neo4j node ids in my external Lucene index and retrieve nodes via these stored ids?
UPDATE
I have been trying to get auto-indexing to work with Gremlin and an embedded server, but with no luck. In the documentation it says
The underlying database is auto-indexed, see Section 14.12, “Automatic Indexing” so the script can return the imported node by index lookup.
But when I examine the graph after loading a new database, no indexes seem to exist.
The Neo4j documentation on auto indexing says that a bunch of configuration is required. In addition to setting node_auto_indexing = true, you have to configure it
To actually auto index something, you have to set which properties
should get indexed. You do this by listing the property keys to index
on. In the configuration file, use the node_keys_indexable and
relationship_keys_indexable configuration keys. When using embedded
mode, use the GraphDatabaseSettings.node_keys_indexable and
GraphDatabaseSettings.relationship_keys_indexable configuration keys.
In all cases, the value should be a comma separated list of property
keys to index on.
So is Gremlin supposed to set the GraphDatabaseSettings parameters? I tried passing in a map into the Neo4jGraph constructor like this:
Map<String,String> config = [
'node_auto_indexing':'true',
'node_keys_indexable': 'emailID'
]
Neo4jGraph g = new Neo4jGraph(graphDB, config);
g.loadGraphML("../databases/data.graphml");
but that had no apparent effect on index creation.
UPDATE 2
Rather than configuring the database through Gremlin, I used the examples given in the Neo4j documentation so that my database creation was like this (in Groovy):
protected Neo4jGraph getGraph(String graphDBname, String databaseName) {
boolean populateDB = !new File(graphDBName).exists();
if(populateDB)
println "creating database";
else
println "opening database";
GraphDatabaseService graphDB = new GraphDatabaseFactory().
newEmbeddedDatabaseBuilder( graphDBName ).
setConfig( GraphDatabaseSettings.node_keys_indexable, "emailID" ).
setConfig( GraphDatabaseSettings.node_auto_indexing, "true" ).
setConfig( GraphDatabaseSettings.dump_configuration, "true").
newGraphDatabase();
Neo4jGraph g = new Neo4jGraph(graphDB);
if (populateDB) {
println "Populating graph"
g.loadGraphML(databaseName);
}
return g;
}
and my retrieval was done like this:
ReadableIndex<Node> autoNodeIndex = graph.rawGraph.index()
.getNodeAutoIndexer()
.getAutoIndex();
def node = autoNodeIndex.get( "emailID", "<2614099.1075839927264.JavaMail.evans#thyme>" ).getSingle();
And this seemed to work. Note, however, that the getIndices() call on the Neo4jGraph object still returned an empty list. So the upshot is that I can exercise the Neo4j API correctly, but the Gremlin wrapper seems to be unable to reflect the indexing state. The expression g.idx('node_auto_index') (documented in Gremlin Methods) returns null.
the auto indexes are created lazily. That is - when you have enabled the auto-indexing, the actual index is first created when you index your first property. Make sure you are inserting data before checking the existence of the index, otherwise it might not show up.
For some auto-indexing code (using programmatic configuration), see e.g. https://github.com/neo4j-contrib/rabbithole/blob/master/src/test/java/org/neo4j/community/console/IndexTest.java (this is working with Neo4j 1.8
/peter
Have you tried the automatic index feature? It's basically the use case you're looking for--unfortunately it needs to be enabled before you import the data. (Otherwise you have to remove/add the properties to reindex them.)
http://docs.neo4j.org/chunked/milestone/auto-indexing.html