Spring Data Neo4j 4 very slow for large hierarchical queries - neo4j

I am currently using SpringDataNeo4j–4.1.0.M1 and have a use case where I need to get a sub-graph (actually a tree) and return it in an hierarchical structure similar to this (these sub-graphs are usually around 500 nodes in size and have an unspecified depth):
<Pathway dbId="1" displayName="Name1">
<Pathway dbId="2" displayName="Name2">
<Pathway dbId="3" displayName="Name3">
<Reaction dbId="4" displayName="Name4" />
<Reaction dbId="5" displayName="Name5" />
<Pathway dbId="6" displayName="Name6">
<Reaction dbId="7" displayName="Name7" />
</Pathway>
</Pathway>
...
The data model:
#NodeEntity
public abstract class Event {
#GraphId
private Long id;
private Long dbId;
private String displayName;
... other relationships and properties
}
#NodeEntity
public class Pathway extends Event {
#Relationship(type="hasEvent", direction = Relationship.OUTGOING)
private List<Event> hasEvent;
... other relationships and properties
}
#NodeEntity
public class Reaction extends Event {
... other relationships and properties
}
To solve this I have created the following query:
String query = "Match (n:Pathway{dbId:123456})-[r:hasEvent*]->(m:Event) Return n,r,m";
Result result = neo4jTemplate.query(query,Collections.<~>emptyMap());
return result.iterator().next().get("n");
This is working and returns the result as expected. The only problem is the speed. While testing the query using the Neo4j-Browser I get a result within 50-100ms. Using SDN it takes over 2 seconds for Spring to map the result to objects.
The question now is: Is there any way I can speed up this query in Spring or is there a better solution where I can return this hierarchical structure in Cypher using something like nested Collections(since I only need the names and ids and not the objects themselves)?

Related

I want to add a Relationship between two already existing nodes in Neo4j - how to do that using Spring Data Neo4jRepository?

The cypher query to perform the above task would be as follows:
MATCH (a:Buyer), (b:Seller)
MERGE (a) -[:BUY {quantity: 150}]-> (b);
I want the equivalent Neo4jRepository function or some equivalent code that can serve the same above purpose. Please post the answer if you know some solution.
[Updates]
I have posted an answer below. But I am also expecting some other kind of solutions to this purpose. Please feel free to post alternative solutions as answers.
One Possible Way
If we have two Nodes/Entities as Buyer and Stock as POJO classes in SpringBoot Code, and if we are trying to add a relationship called [:HAS] between two such nodes then we can do the following.
#Node("Stock")
class Stock
{
#Id #GeneratedValue(...)
private Long id;
/* Other attributes here --- */
}
#Node("Buyer")
class Buyer
{
#Id #GeneratedValue(...)
private Long id;
/* Other Attribute Variables Here --- */
#Relationship(type="HAS")
private List<Stock> stockList;
public List<Stock> getStockList()
{
return stockList;
}
public void setStockList(List stockList)
{
this.stockList = stockList;
}
}
So we can do this to create the desired relationship.
buyer.getStockList().add(newStock);
buyerRepo.save(buyer);
Here buyerRepo is the object of a Repository that implements Neo4jRepository<Buyer, Long>

Spring Data Query into Elastic search for exact match

I am learning elasticsearch with spring data so can someone help me understand better what elasticsearch query is doing here. I am trying to return back only a set of results based off of a certain value, in this case env. It seems to me that this JPQL query, is not making a difference to only return what I ask for. I have also used an #Query with no difference.
-- here is part of my repository class
public interface MyFormRepo extends ElasticsearchRepository<MyForm, String> {
//??? these function calls are not effecting my return
#Query("{\"bool\": {\"must\": [{\"match\": {\"env\": \"?0\"}}]}}")
Page<MyForm> getAllByEnv(String env, Pageable pageable);
Page<MyForm> findAllByEnv(String env, Pageable pageable);
-- Here is part of my entity class
#Document(indexName = "my_form")
public class MyForm {
#Id
private String id;
#Field(type = Text)
private String schema;
#Field(type = Long)
private long version;
#Field(type = Text)
private String env;
Here is what I understand. Elasticsearch has this concept called Fuzziness (https://www.elastic.co/guide/en/elasticsearch/reference/current/common-options.html#fuzziness) so in it's searches which is based on Levenshtein distance (https://en.wikipedia.org/wiki/Levenshtein_distance). Spring data does not allow us to modify this out of the box and is considered Fuzziness.AUTO by default. https://docs.spring.io/spring-data/elasticsearch/docs/current/reference/html/#elasticsearch.misc. As for the queries neither of them will do anything different. Both findByAllEnv and getAllBeEnv has a fuziness.AUTO, as for the #Query I found a good reason stated at this site: What is difference between match and bool must match query in Elasticsearch. What I ended up finding is that I must implemented a custom repo for that I have found this example/explaination How to query Elastic with Spring-data-elastic.

How to implement bidirectional relationship in SDN 6

How should bidirectional relationships be implemented in SDN 6?
I have the following implementation of two entities and a repository
#Node("UserAccount")
class UserAccount(
#Id
val username: String,
) {
#Relationship(type = "HAS_ACCOUNT")
var person: Person? = null
}
#Node("Person")
class Person(
#Relationship(type = "HAS_ACCOUNT")
var userAccount: UserAccount? = null,
) {
#Id
#GeneratedValue
var personId: Long = -1
}
#Repository
interface PersonRepository : Neo4jRepository<Person, Long> {
fun findByPersonId(personId: Long): Optional<Person>
}
After fetching person by personId the following exception occurs:
org.springframework.data.mapping.MappingException: The node with id 646 has a logical cyclic mapping dependency. Its creation caused the creation of another node that has a reference to this.
The problem is rooted in the dependency chain (cycle) Person -> UserAccount -> Person, as you have already discovered.
Spring Data Neo4j has a dead-lock detection during mapping to avoid stack overflow / infinite recursion if it comes to cyclic mapping dependencies. This is why you see the exception.
What happens in detail?
SDN starts mapping the Person that depends on an UserAccount instance (constructor) and puts it in a set of "objects in construction".
The UserAccount instance gets created AND the properties of it populated. This has to be done because a property can -besides public visibility and setter- also be "set" by using a wither method (https://docs.spring.io/spring-data/neo4j/docs/current/reference/html/#mapping.fundamentals.property-population) and would result in a new (and different instance of the UserAccount)
The person property needs the Person-in-construction -> Exception
To avoid this, you can either remove the constructor dependency and define a field.
Another option is to discard the bi-directional relationship definition completely.

How to persist a List<CustomObject> as a property of a node?

I am trying to persist a list of objects of a class suppose xyz. when I do this in the NodeEntity Class:
#Property
List<xyz> listOfConditions
The Node table when loaded from the neo4j-database via the Neo4jOperations.load(entity) method, will return an error saying :- ERROR mapping GraphModel to NodeEntity type class.
Is there any way to persist a List of Objects onto a nodes properties in Neo4j?. I am using neo4j-ogm-embedded driver and Spring-data-neo4j.
Neo4j does not support storing another object as a nested property. Neo4j-OGM only supports
any primitive, boxed primitive or String or arrays thereof,
essentially anything that naturally fits into a Neo4j node property.
If you want to work around that, you may need to create a custom type convertor. For example,
import org.neo4j.ogm.typeconversion.AttributeConverter
class XYZ{
XYZ(Integer x, String y) {
this.x = x
this.y = y
}
Integer x
String y
}
public class XYZConverter implements AttributeConverter<XYZ, String> {
#Override
public String toGraphProperty(XYZ value) {
return value.x.toString() + "!##" + value.y
}
#Override
public XYZ toEntityAttribute(String value) {
String[] split = value.split("!##")
return new XYZ(Integer.valueOf(split[0]), split[1])
}
}
You can then annotate the #NodeEntity with a #Convert like this
#NodeEntity
class Node {
#GraphId
Long id;
#Property
String name
#Convert(value = XYZConverter.class)
XYZ xyz
}
On the flip side, its not a good practice to do this, since ideally you should link Node and XYZ with a 'hasA' relationship. Neo4j has been designed to optimally handle such kind of relationships, so it would be best to play with to strengths of neo4j
No, nested objects represented as properties on a single node are not supported by the OGM. The only option is to write a custom converter to serialize the nested object to a String representation and store it as a single property.
Otherwise, a list of objects on a node is treated as relationships from the node to those objects.
Here's a link to the manual for further reference: http://neo4j.com/docs/ogm-manual/current/

Neo4j Repository - Writing query with a dynamic where clause

I am relatively new to Neo4j, and have a doubt regarding writing of dynamic queries in Neo4j with spring.
From what I have read, the queries are annotated with #Query parameter in an interface that extends the GraphRepository class, and the dynamic parameters are supplied as argument.
But my requirement is such that I have to dynamically generate the number of where clauses.
For example,
#Query("match n where n.__type__='com.connectme.domain.Person' and n.age > {0} return n.id)
public List<Object> getPeopleWithAge(Integer age);//
My query can also change wherein age can also be less than some value, in which case, the query can become :
#Query("match n where n.__type__='com.connectme.domain.Person' and n.age > {0} and n.age <{1} return n.id)
public List<Object> getPeopleWithAge(Integer age1, Integer age2);//
In similar way, many clauses around the age parameter can lead to variation in where clauses.
How can i dynamically handle this as currently I am only aware of this annotated way of executing queries.
Can I override and write my own custom queries ?
You can write your own custom query logic. First you create an extra interface containing the custom query method, so you get two repository interfaces
public interface YourRepository extends GraphRepository<SomeClass> implements YourRepositoryExtension{
//inferred queries, annotated queries
}
public interface YourRepositoryExtension {
EndResult<SomeClass> customQuery();
Iterable<SomeClass> customTraversal();
}
Then you make an implementation:
#Repository
public class YourRepositoryImpl implements YourRepositoryExtension {
#Autowired
private YourRepository yourRepository;
#Override
public EndResult<SomeClass> customQuery(){
//your query logic, using yourRepository to make cypher calls.
return yourRepository.query("START n.. etc.. RETURN n", <optional params>);
}
#Override
public Iterable<SomeClass> customTraversal(){
SomeClass startNode = yourRepository.findOne(123);
return yourRepository.findAllByTraversal(startNode, <some custom traversal>);
}
}

Resources