Optimizing Neo4j Query for Path search - neo4j

Question

Neo4j version: 3.5.16
What kind of API / driver do you use: Python API with py2neo to run the query with graph.run()
Py2neo version: 4.3.0.
Hey all,
I'm trying to optimize a cypher query to retrieve a variable length path.
The graph is created each time data arrives and startNode and endNode are fixed on their name property. Once created the graph, I have a startNode and an endNode and the corolllary/objective is:
"From all the possible paths with a minimum length of X and a maximum length of Y, I want to find the shortest path that yields the highest aggregated relationship value".
What I actually have managed to do is: "get the path of length between X and Y that yields the highest aggregated relationship value" with the following cyper query:
MATCH path = (startN:Batch { name: $startNode })-[:CHANGES_TO*4..7]->(endN:Batch { name: $endNode })
RETURN path,
REDUCE (s = 1, r IN RELATIONSHIPS(path) | s * r.rateValue) AS finalBatchValue
ORDER BY finalBatchValue DESC
LIMIT 1
Howerver, it takes some time to run. ¿Could someone provide ideas on how to optimize this, both to accomplish the objetive of the shortestPath and to optimize the query for running faster if possible?
I tried to make it work with APOC methods like allShortestPaths or Dijkstra with no success; it ended up returning the shortest path and I wasn't able to fix the minimum amount of nodes to consider.
Any help is much appreciated.

Answer 1

Since you are basically looking for strongest one in shortest length. You can come around the performance problem with a trick.
You can query the graph with fixed length than variable length. But you have to query n2-n1+1 times , in your case , 4 times , first with length 4, and then 5 and so on .
You can stop querying if you find a path at any point.
This approach will tremendously decrease the data loaded each time. But you have to hit the graph multiple times.Its most likely that average time taken for four hits approach will be less than single hit with variable length.
The reason being you don't calculate all the paths of higher length if you find a lower length solution.
Since,longer the path gets, the time taken will grow exponentially.
This is not possible only using cypher . One way is writing neo4j procedure in java and using that in cypher query .
The second way is : hitting neo4j using different query.
i am writing python code for your case here ,
query = 'MATCH path = (startN:Batch { name: $startNode })-[:CHANGES_TO*LENGTH_PARAM]->(endN:Batch { name: $endNode })
RETURN path,
REDUCE (s = 1, r IN RELATIONSHIPS(path) | s * r.rateValue) AS finalBatchValue
ORDER BY finalBatchValue DESC
LIMIT 1'
for length in range(4,8):
query = query.replace('LENGTH_PARAM',str(x))
result = graph.run(query)
#if result size > 0
#your implementation
#final_result= result['path']
#return final_result
That's how it works, here in worst case,you need to hit graph four times for each start,end node pair . Network calls increase, but average time taken should be reduced.
With java plugins, it can be reduced to one hit like previous query , as you can do the loop part inside the java code .