Cypher query for externalized property values - neo4j

in our data model we have externalized some of our domain entities values into external nodes. The model for a service object looks like:
ref=node(0),
ref<-[:SERVICE]-subRefNode<-[:SERVICE]-aService-[:HAS_PROPERTY_VALUE]->propValueNode-[:IS_OF_TYPE]->propDefType,
ref<-[:SERVICE]-subRefNode-[:HAS_PROPERTY]->propDefType
The node subRefNode holds a relationship to all services. All possible properties for a service are defined through ref<-[:SERVICE]-subRefNode-[:HAS_PROPERTY]->propDefType. So it could be that a certain node doesn't have a propValueNode yet for a certain property (e.g. a comment) and others might have one but its empty (the user might have entered a comment and then cleared it).
So my question is how do I get the nodes with an empty propValueNode.value and also the ones that do not have a propertyValueNode yet? I thought about something like...
START ref=node(0) MATCH ref<-[:SERVICE]-subRef, aService-[pvRel?:HAS_PROPERTY_VALUE]->propValueNode-[pdRel:IS_OF_TYPE]->propDef<-[:HAS_PROPERTY]-subRef<-[:SERVICE]-aService WHERE (pvRel IS NOT NULL AND propDef.name = 'comment' AND propValueNode.value=~"^$") OR (pvRel IS NULL AND pdRel IS NULL AND propDef.name="comment") RETURN DISTINCT aService.debug
http://console.neo4j.org/r/7zeoay
...but this misses the ones without a propValueNode. Any hints are appreciated!
Regards,
Andi
If http://console.neo4j.org/r/7zeoay is not valid anymore, here's the initial graph setup:
start _0 = node(0) with _0
create
(_1 {type:"SubReferenceNode", name:"SubRef"}),
(_2 {type:"Service", debug:"S0 empty value"}),
(_3 {type:"Service", debug:"S1 missing value node"}),
(_4 {type:"Service", debug:"S2 with value"}),
(_5 {type:"PropertyDefintion", name:"comment"}),
(_6 {type:"PropertyDefintion", name:"name"}),
(_7 {type:"PropertyValue", value:"S0 empty value"}),
(_8 {type:"PropertyValue", value:"S1 missing value node"}),
(_9 {type:"PropertyValue", value:"S2 with value"}),
(_10 {type:"PropertyValue", value:""}),
(_11 {type:"PropertyValue", value:"This is a comment"}),
_0<-[:SERVICE]-_1,
_1<-[:SERVICE]-_2,
_1<-[:SERVICE]-_3,
_1<-[:SERVICE]-_4,
_1-[:HAS_PROPERTY]->_5,
_1-[:HAS_PROPERTY]->_6,
_2-[:HAS_PROPERTY_VALUE]->_7,
_7-[:IS_OF_TYPE]->_6,
_3-[:HAS_PROPERTY_VALUE]->_8,
_8-[:IS_OF_TYPE]->_6,
_4-[:HAS_PROPERTY_VALUE]->_9,
_9-[:IS_OF_TYPE]->_6,
_2-[:HAS_PROPERTY_VALUE]->_10,
_10-[:IS_OF_TYPE]->_5,
_4-[:HAS_PROPERTY_VALUE]->_11,
_11-[:IS_OF_TYPE]->_5

You can combine two queries, one is to return the service which has an empty value for the property "comment", and another returns the service which does not have a propery value that is of the type "comment", in other word, none of the service property values is of the the type "comment".
START ref=node(0)
MATCH ref<-[:SERVICE]-subRef, aService-[pvRel?:HAS_PROPERTY_VALUE]->propValueNode-[pdRel:IS_OF_TYPE]->propDef<-[:HAS_PROPERTY]-subRef<-[:SERVICE]-aService
WHERE (pvRel IS NOT NULL AND propDef.name = 'comment' AND propValueNode.value=~"^$")
RETURN DISTINCT aService.debug
UNION
START ref=node(0)
MATCH ref<-[:SERVICE]-subRef, aService-[pvRel:HAS_PROPERTY_VALUE]->propValueNode-[pdRel:IS_OF_TYPE]->propDef<-[:HAS_PROPERTY]-subRef<-[:SERVICE]-aService
with aService, collect(propDef.name) as propNames
WHERE NONE( propName in propNames where propName = 'comment')
RETURN DISTINCT aService.debug
The alternative solution without using the "UNION" clause is to get all value nodes for each service and returns only those service that either none of its values is of type "comment" or there is one value of the type "comment" and the value is empty, as shown in the "Where" clause below,
START ref=node(0)
MATCH ref<-[:SERVICE]-subRef, aService-[pvRel:HAS_PROPERTY_VALUE]->propValueNode-[pdRel?:IS_OF_TYPE]->propDef<-[:HAS_PROPERTY]-subRef<-[:SERVICE]-aService
WHERE propDef.name = 'comment'
WITH aService, collect(propValueNode) AS valueNodes, propDef
WHERE NONE (v IN valueNodes
WHERE v-[:IS_OF_TYPE]->propDef) OR SINGLE (v IN valueNodes
WHERE v.value=~"^$" AND v-[:IS_OF_TYPE]->propDef)
RETURN aService.debug

Related

How to exclude multiple values in OData call?

I am creating a SAPUI5 application. This application is connected to a backend SAP system via OData. In the SAPUI5 application I use a smart chart control. Out of the box the smart chart lets the user create filters for the underlying data. This works fine - except if you try to use multiple 'not equals' for one property. Is there a way to accomplish this?
I found out that all properties within an 'and_expression' (including nested or_expressions) must have unique name.
The reason why two parameters with the same property don't get parsed into the select options:
/IWCOR/CL_ODATA_EXPR_UTILS=>GET_FILTER_SELECT_OPTIONS takes the expression you pass and parses it into a table of select options.
The select option table returned is of type /IWCOR/IF_ODATA_TYPES=>EDM_SELECT_OPTION_T which is a HASHED TABLE .. WITH UNIQUE KEY property.
From: https://archive.sap.com/discussions/thread/3170195
The problem is that you cannot combine NE terms with OR. Because both parameters after the NE should not be shown in the result set.
So at the end the it_filter_select_options is empty and only the iv_filter_string is filled.
Is there a manual way of facing this problem (evaluation of the iv_filter_string) to handle multiple NE terms?
This would be an example request:
XYZ/SmartChartSet?$filter=(Category%20ne%20%27Smartphone%27%20and%20Category%20ne%20%27Notebook%27)%20and%20Purchaser%20eq%20%27CompanyABC%27%20and%20BuyDate%20eq%20datetime%272018-10-12T02%3a00%3a00%27&$inlinecount=allpages
Normally I want this to exclude items with the category 'Notebook' and 'Smartphone' from my result set that I retrieve from the backend.
If there is a bug inside /iwcor/cl_odata_expr_utils=>get_filter_select_options which makes it unable to treat multiple NE filters of the same component, and you cannot wait for an OSS. I would suggest to wrap it inside a new static method that will make the following logic (if you will be stuck with the ABAP implementation i would try to at least partially implement it when i get time):
Get all instances of <COMPONENT> ne '<VALUE>' inside a () (using REGEX).
Replace each <COMPONENT> with <COMPONENT>_<i> so there will be ( <COMPONENT>_1 ne '<VALUE_1>' and <COMPONENT>_2 ne '<VALUE_2>' and... <COMPONENT>_<n> ne '<VALUE_n>' ).
Call /iwcor/cl_odata_expr_utils=>get_filter_select_options with the modified query.
Modify the rt_select_options result by changing COMPONENT_<i> to <COMPONENT> again.
I can't find the source but I recall that multiple "ne" isn't supported. Isn't that the same thing that happens when you do multiple negatives in SE16, some warning is displayed?
I found this extract for Business ByDesign:
Excluding two values using the OR operator (for example: $filter=CACCDOCTYPE ne ‘1000’ or CACCDOCTYPE ne ‘4000’) is not possible.
The workaround I see is to select the Categories you actively want, not the ones you don't in the UI5 app.
I can also confirm that my code snippet I've used a long time for filtering also has the same problem...
* <SIGNATURE>---------------------------------------------------------------------------------------+
* | Instance Public Method ZCL_MGW_ABS_DATA->FILTERING
* +-------------------------------------------------------------------------------------------------+
* | [--->] IO_TECH_REQUEST_CONTEXT TYPE REF TO /IWBEP/IF_MGW_REQ_ENTITYSET
* | [<-->] CR_ENTITYSET TYPE REF TO DATA
* | [!CX!] /IWBEP/CX_MGW_BUSI_EXCEPTION
* | [!CX!] /IWBEP/CX_MGW_TECH_EXCEPTION
* +--------------------------------------------------------------------------------------</SIGNATURE>
METHOD FILTERING.
FIELD-SYMBOLS <lt_entityset> TYPE STANDARD TABLE.
ASSIGN cr_entityset->* TO <lt_entityset>.
CHECK: cr_entityset IS BOUND,
<lt_entityset> IS ASSIGNED.
DATA(lo_filter) = io_tech_request_context->get_filter( ).
/iwbep/cl_mgw_data_util=>filtering(
exporting it_select_options = lo_filter->get_filter_select_options( )
changing ct_data = <lt_entityset> ).
ENDMETHOD.

Neo4j cypher query fails with unknown syntax error

I have the following paramObj and dbQuery
paramObj = {
email: newUser.email,
mobilenumber: newUser.telephone,
password: newUser.password,
category: newUser.category,
name: newUser.name,
confirmuid: verificationHash,
confirmexpire: expiryDate.valueOf(),
rewardPoints: 0,
emailconfirmed: 'false',
paramVehicles: makeVehicleArray,
paramVehicleProps: vehiclePropsArray
}
dbQuery = `CREATE (user:Person:Owner {email:$email})
SET user += apoc.map.clean(paramObj,
['email','paramVehicles','paramVehiclesProps'],[])
WITH user, $paramVehicles AS vehicles
UNWIND vehicles AS vehicle
MATCH(v:Vehicles {name:vehicle})
CREATE UNIQUE (user)-[r:OWNS {since: timestamp()}]->(v)
RETURN user,r,v`;
Then I tried to execute
commons.session
.run(dbQuery, paramObj)
.then(newUser => {
commons.session.close();
if (!newUser.records[0]) {........
I am getting
Error: {"code":"Neo.ClientError.Statement.SyntaxError","name":"Neo4jError"}
which doesn't direct me anywhere. Can anyone tell me what am I doing wrong here?
This is actually the first time I am using the query format .run(dbQuery, paramObj) but this format is critical to my use case. I am using Neo4j 3.4.5 community with apoc plugin installed.
Ok...so I followed #inversFalcon suggestion to test in browser and came up with following parameters and query that closely match the ones above:
:params paramObj:[{ email:"xyz123#abc.com", mobilenumber:"8711231234",password:"password1", category:"Owner",name:"Michaell",vehicles:["Toyota","BMW","Nissan"],vehicleProps: [] }]
and query
PROFILE
CREATE (user:Person:Owner {email:$email})
SET user += apoc.map.clean($paramObj, ["email","vehicles","vehicleProps"],[])
WITH user, $vehicles AS vehicles
UNWIND vehicles AS vehicle
MATCH(v:Vehicles {name:vehicle})
MERGE (user)-[r:OWNS {since: timestamp()}]->(v)
RETURN user,r,v;
Now I get
Neo.ClientError.Statement.TypeError: Can't coerce `List{Map{name -> String("Michaell"), vehicles -> List{String("Toyota"), String("BMW"), String("Nissan")},.......
I also reverted to neo4j 3.2 (re: an earlier post by Mark Needham) and got the same error.
You should try doing an EXPLAIN of the query using the browser to troubleshoot it.
A few of the things I'm seeing here:
You're referring to paramObj, but it's not a parameter (rather, it's the map of parameters you're passing in, but it itself is not a parameter you can reference in the query). If you need to reference the entire set of parameters being passed in, then you need to use nested maps, and have paramObj be a key in the map that you pass as the parameter map (and when you do use it in the query, you'll need to use $paramObj)
CREATE UNIQUE is deprecated, you should use MERGE instead, though be aware that it does behave in a different manner (see the MERGE documentation as well as our knowledge base article explaining some of the easy-to-miss details of how MERGE works).
I am not sure what caused the coercion error to disappear but it did with the same query and I got a "expected parameter error" this was fixed by using $paramObj.email, etc. so the final query looks like this:
CREATE (user:Person:Owner {email: $paramObj.email})
SET user += apoc.map.clean($queryObj, ["email","vehicles","vehicleProps"],[])
WITH user, $paramObj.vehicles AS vehicles
UNWIND vehicles AS vehicle
MATCH(v:Vehicles {name:vehicle})
MERGE (user)-[r:OWNS {since: timestamp()}]->(v)
RETURN user,r,v;
which fixed my original problem of how to remove properties from a map when using SET += map.

Numeric sort in Manual (Legacy) index in Neo4j 3 is not working correctly

I'm using Legacy indexing (now called Manual indexing). After migration from Neo4j 2 to version 3 I have some problems with numeric sorting.
Example of correct statement in Neo4j 2:
queryContext.sort(new Sort(new SortField(AGE, SortField.INT, false)));
This stament should be changed for Neo4j 3 (Lucene 5):
queryContext.sort(new Sort(new SortField(AGE, SortField.Type.INT, false)));
But if you use this sort statement you will get an exception:
java.lang.IllegalStateException: unexpected docvalues type SORTED_SET for field 'firstName' (expected=SORTED). Use UninvertingReader or index with docvalues.
at org.apache.lucene.index.DocValues.checkField(DocValues.java:208)
at org.apache.lucene.index.DocValues.getSorted(DocValues.java:264)
at org.apache.lucene.search.FieldComparator$TermOrdValComparator.getSortedDocValues(FieldComparator.java:762)
at org.apache.lucene.search.FieldComparator$TermOrdValComparator.getLeafComparator(FieldComparator.java:767)
at org.apache.lucene.search.FieldValueHitQueue.getComparators(FieldValueHitQueue.java:183)
at org.apache.lucene.search.TopFieldCollector$SimpleFieldCollector.getLeafCollector(TopFieldCollector.java:164)
at org.neo4j.kernel.api.impl.index.collector.DocValuesCollector.replayTo(DocValuesCollector.java:297)
at org.neo4j.kernel.api.impl.index.collector.DocValuesCollector.getTopDocs(DocValuesCollector.java:275)
at org.neo4j.kernel.api.impl.index.collector.DocValuesCollector.getIndexHits(DocValuesCollector.java:150)
at org.neo4j.index.impl.lucene.legacy.LuceneLegacyIndex.search(LuceneLegacyIndex.java:346)
at org.neo4j.index.impl.lucene.legacy.LuceneLegacyIndex.query(LuceneLegacyIndex.java:261)
at org.neo4j.index.impl.lucene.legacy.LuceneLegacyIndex.query(LuceneLegacyIndex.java:205)
at org.neo4j.index.impl.lucene.legacy.LuceneLegacyIndex.query(LuceneLegacyIndex.java:217)
at org.neo4j.kernel.impl.api.StateHandlingStatementOperations.nodeLegacyIndexQuery(StateHandlingStatementOperations.java:1440)
at org.neo4j.kernel.impl.api.OperationsFacade.nodeLegacyIndexQuery(OperationsFacade.java:1162)
at org.neo4j.kernel.impl.coreapi.LegacyIndexProxy$Type$1.query(LegacyIndexProxy.java:83)
at org.neo4j.kernel.impl.coreapi.LegacyIndexProxy.query(LegacyIndexProxy.java:365)
I think this is caused by new added statement in Neo4j indexer class (Neo4j is indexing field for sorting automatically now?). See in:
org.neo4j.index.impl.lucene.legacy.IndexType CustomType addToDocument( Document document, String key, Object value )
new line:
document.add( instantiateSortField( key, value ) );
and method instantiateSortField is creating SortedSetDocValuesField
So I changed my code to:
queryContext.sort(new Sort(new SortedSetSortField(AGE, false)));
This runs OK but sorting is not working because numbers are sorted as string. I see that "value" parameter is String every time in method "addToDocument". I think the root cause is explained it this old comment:
see comment in class org.neo4j.index.impl.lucene.legacy.IndexType CustomType
// TODO We should honor ValueContext instead of doing value.toString() here.
// if changing it, also change #get to honor ValueContext.
Am I missing some new way how to index, search and sort data in Neo4j 3 or this is really problem that values are indexed as string in Neo4j?
Simple unit test for Neo4j 2 and Neo4j 3 can be downloaded
Solution added by MishaDemianenko at GH issue

Neo4j+PopotoJS: filter graph based-on predefined constraints

I have a question about the query based on the predefined constraints in PopotoJs. In this example, the graph can be filtered based on the constraints defined in the search boxes. The sample file in this example visualizations folder, constraint is only defined for "Person" node. It is specified in the sample html file like the following:
"Person": {
"returnAttributes": ["name", "born"],
"constraintAttribute": "name",
// Return a predefined constraint that can be edited in the page.
"getPredefinedConstraints": function (node) {
return personPredefinedConstraints;
},
....
In my graph I would like to apply that query function for more than one node. For example I have 2 nodes: Contact (has "name" attribute) and Delivery (has "address" attribute)
I succeeded it by defining two functions for each nodes. However, I also had to put two search box forms with different input id (like constraint1 and constraint2). And I had to make the queries in the associated search boxes.
Is there a way to make queries which are defined for multiple nodes in one search box? For example searching Contact-name and/or Delivery-adress in the same search box?
Thanks
First I’d like to specify that the predefined constraints feature is still experimental (but fully functional) and doesn’t have any documentation yet.
It is intended to be used in configuration to filter data displayed in nodes and in the example the use of search boxes is just to show dynamically how it works.
A common use of this feature would be to add the list of predefined constraint you want in the configuration for every node types.
Let's take an example:
With the following configuration example the graph will be filtered to show only Person nodes having "born" attribute and only Movie nodes with title in the provided list:
"Person": {
"getPredefinedConstraints": function (node) {
return ["has($identifier.born)"];
},
...
}
"Movie": {
"getPredefinedConstraints": function (node) {
return ["$identifier.title IN [\"The Matrix\", \"The Matrix Reloaded\", \"The Matrix Revolutions\"]"];
},
...
}
The $identifier variable is then replaced during query generation with the corresponding node identifier. In this case the generated query would look like this:
MATCH (person:`Person`) WHERE has(person.born) RETURN person
In your case if I understood your question correctly you are trying to use this feature to implement a search box to filter the data. I'm still working on that feature but it won't be available soon :(
This is a workaround but maybe it could work in your use case, you could keep the search box value in a variable:
var value = d3.select("#constraint")[0][0].value;
inputValue = value;
Then use it in the predefined constraint of all the nodes type you want.
In this example Person will be filtered based on the name attribute and Movie on title:
"Person": {
"getPredefinedConstraints": function (node) {
if (inputValue) {
return ["$identifier.name =~ '(?i).*" + inputValue + ".*'"];
} else {
return [];
}
},
...
}
"Movie": {
"getPredefinedConstraints": function (node) {
if (inputValue) {
return ["$identifier.title =~ '(?i).*" + inputValue + ".*'"];
} else {
return [];
}
},
...
}
Everything is in the HTML page of this example so you can view the full source directly on the page.
#Popoto, thanks for the descriptive reply. I tried your suggestion and it worked pretty much well. With the actual codes, when I make a query it was showing only the queried node and make the other node amount zero. I wanted to make a query which queries only the related node while the number of other nodes are still same.
I tried a temporary solution for my problem. What I did is:
Export the all the node data to JSON file, search my query constraint in the exported JSONs, if the file is existing in JSON, then run the query in the related node; and if not, do nothing.
With that way, of course I needed to define many functions with different variable names (as much as the node amount). Anyhow, it is not a propoer way, bu it worked for now.

py2neo create function creating duplicate nodes

I have a Neo4j database containing information on Congressmen. The problem I'm having is if there is a vacant position. When this happens I am using the same key:value in the "Congressmen" index. I tried the code below because in the py2neo documentation it states that the add function is idempotent
#Check if we have any vacancies and if so if they match the one that we currently want to add
query="start n=node:Congressmen('website:N/A') return n"
result= cypher.execute(graph_db, query.encode('utf-8', errors='ignore'))
#Match what we already have
if str(result[0]) != "[]":
#create is idempotent so will only create a new node if properties are different
rep, = graph_db.create({"name" : userName, "website" : web, "district" : int(district), "state" : child[2].text, "party" : child[4].text, "office" : child[5].text, "phone" : child[6].text, "house" : "House of Representatives"})
cong = graph_db.get_or_create_index(neo4j.Node, "Congressmen")
# add the node to the index
cong.add("website", web, rep)
When I checked the interface after running the code 3 times I have duplicate nodes.
Is it possible to prevent the nodes from duplicating and still be able to index them using the same key/value?
The Index.add method is certainly idempotent: the same entity can only be added once to a particular entry point. The GraphDatabaseService.create method is not however. Each time you run the create method, a new node is created and each run of add appends that new node to the index. You probably want to use the Index.add_if_none, Index.create_if_none or Index.get_or_create method instead.

Resources