Neo4j - Counting rows with where condition - neo4j

I'm trying to count the amount of rows that Neo4j will return but the count (or the query) is very slow.
Version 1 (70 sec):
MATCH (person:Person)-[:HAS_ORDER]->(order:Order)
WHERE order.timestamp >= 1632434400 AND size((order)<-[:HAS_ORDER]-(:OrderLine)-[:HAS_PRODUCT]->(:Product)) <= 20
WITH order
MATCH (order)<-[:HAS_ORDER]-(:OrderLine)-[:HAS_PRODUCT]->(product:Product)
RETURN COUNT(product);
Version 2 (68 sec.):
MATCH (person:Person)-[:HAS_ORDER]->(order:Order)
WITH size((order)<-[:HAS_ORDER]-(:OrderLine)-[:HAS_PRODUCT]->(:Product)) AS amount
WHERE order.timestamp >= 1632434400 AND amount <= 20
RETURN SUM(amount)
Using Neo4j 4.4 community with about 800000 orders and about 17000000 order lines.
Is there a more efficient way to count the rows?
These are the indexes:
CREATE INDEX idx_order_torder_id FOR (n:Order) ON (n.order_id);
CREATE INDEX idx_order_timestamp FOR (n:Order) ON (n.timestamp);
CREATE INDEX idx_person_person_id FOR (n:Person) ON (n.person_id);
CREATE INDEX idx_product_product_id FOR (n:Product) ON (n.product_id);
The amount of rows are equal to 4269011.
The EXPLAIN plan:

Please try below, hope it will give faster results
MATCH (person:Person)-[:HAS_ORDER]->(order:Order)
WHERE order.timestamp >= 1632434400
WITH order.order_id AS orderid
MATCH (o:Order { order_id: orderid })<-[:HAS_ORDER]-(:OrderLine)-[:HAS_PRODUCT]->(product:Product)
WITH COUNT(product) as productCount
WHERE productCount <= 20
RETURN productCount;

Because every order line has one product, i can skip the counting of the relation order lines to products:
MATCH (order:Order)
WHERE order.timestamp >= 1632434400
WITH order
MATCH (order)<-[:HAS_ORDER]-(orderLine:OrderLine)
WITH COUNT(orderLine) as productCount
WHERE productCount <= 20
RETURN SUM(productCount);
This query took 0m17.342s
But i managed to snoop some seconds with the following query:
MATCH (order:Order)
WHERE order.timestamp >= 1632434400
WITH order, size((order)<-[:HAS_ORDER]-(:OrderLine)) AS amount
WHERE amount <= 20
RETURN SUM(amount);
This query took 0m15.675s

Related

Finding the "GAP" in node values ? or next?

Let say I have a nodes with values a multiples of 10. I want to find the first GAP in the values.
Here is how I would do it in numpy :
> np.where(np.diff([11,21,31,51,61,71,91]) > 10)[0][0] + 2
> 4 i.e. 41
How would I do this in Cypher... ?
match (n) where n.val % 10 = 1
with n.val
order by val ....???
I'm using RedisGraph.
PS>
if no GAP it should return the next value i.e. biggest + 10, if possible !
I'm not sure if this is the most performant solution, but you can accomplish this using a combination of collect() and list comprehensions:
MATCH (n) WHERE n.val % 10 = 1 WITH n.val AS val ORDER BY n.val // collect ordered vals
WITH collect(val) AS vals // combine vals into array
WITH vals, [idx IN range(0, size(vals) + 1) WHERE vals[idx + 1] - vals[idx] > 10] AS gaps // find first index with diff > 10
RETURN vals[gaps[0]] + 10 // return missing value
To additionally return the next-biggest value if no gaps are found, change the RETURN clause to use a CASE statement:
RETURN CASE size(gaps) WHEN 0 THEN vals[-1] + 10 ELSE vals[gaps[0]] + 10 END

Cypher Neo4j query optimization

I use the following Cypher query:
MATCH (v:Value)-[:CONTAINS]->(hv:HistoryValue)
WHERE v.id = {valueId}
OPTIONAL MATCH (hv)-[:CREATED_BY]->(u:User)
WHERE {fetchCreateUsers}
WITH u, hv
ORDER BY hv.createDate DESC
WITH count(hv) as count, ceil(toFloat(count(hv)) / {maxResults}) as step, COLLECT({userId: u.id, historyValueId: hv.id, historyValue: hv.originalValue, historyValueCreateDate: hv.createDate}) AS data
RETURN REDUCE(s = [], i IN RANGE(0, count - 1, CASE step WHEN 0 THEN 1 ELSE step END) | s + data[i]) AS result, step, count
This query works fine and does exactly what I need.
Right now I'm concerned about two possible issues inside of this query from the performance point of view and Cypher best practices.
First of all, as you may see - I two times use the same count(hv) function. Will it cause the problems during the execution from the performance point of view or Cypher and Neo4j are smart enough to optimize it? If no, please show how to fix it.
And the second place is the CASE statement inside range() function? The same question here - will this CASE statement be executed only once or every time for every iteration over my range? Please show how to fix it if needed.
UPDATED
I tried to do a separator WITH for count but the query doesn't return the results(returns empty result)
MATCH (v:Value)-[:CONTAINS]->(hv:HistoryValue)
WHERE v.id = {valueId}
OPTIONAL MATCH (hv)-[:CREATED_BY]->(u:User)
WHERE {fetchCreateUsers}
WITH u, hv ORDER BY hv.createDate DESC
WITH u, hv, count(hv) as count
WITH u, hv, count, ceil(toFloat(count) / {maxResults}) as step, COLLECT({userId: u.id, historyValueId: hv.id, historyValue: hv.originalValue, historyValueCreateDate: hv.createDate}) AS data
RETURN REDUCE(s = [], i IN RANGE(0, count - 1, CASE step WHEN 0 THEN 1 ELSE step END) | s + data[i]) AS result, step, count
1 MATCH (v:Value)-[:CONTAINS]->(hv:HistoryValue)
2 WHERE v.id = {valueId}
3 OPTIONAL MATCH (hv)-[:CREATED_BY]->(u:User)
4 WHERE {fetchCreateUsers}
5 WITH u, hv
6 ORDER BY hv.createDate DESC
7 WITH count(hv) as count, ceil(toFloat(count(hv)) / {maxResults}) as step, COLLECT({userId: u.id, historyValueId: hv.id, historyValue: hv.originalValue, historyValueCreateDate: hv.createDate}) AS data
8 RETURN REDUCE(s = [], i IN RANGE(0, count - 1, CASE step WHEN 0 THEN 1 ELSE step END) | s + data[i]) AS result, step, count
(1) You need to pass hv in line 5, because it's values are collected in line 7. That said, you can still do something like this:
5 WITH u, collect(hv) AS hvs, count(hv) as count
UNWIND hvs AS hv
However, this is not very elegant and probably not worth doing.
(2) You can calculate the CASE expression in line 7:
7 WITH count, data, step, CASE step WHEN 0 THEN 1 ELSE step END AS stepFlag
8 RETURN REDUCE(s = [], i IN RANGE(0, count - 1, stepFlag) | s + data[i]) AS result, step, count

Conditional Order By Clause In Cypher Query Neo4j

Hi I want to sort my graph results by two filters..
My Cypher Query looks like this..
MATCH (n:User{user_id:304020})-[r:know]->(m:User) with m MATCH (m)-[s:like|create|share]->(o{is_active:1})
with m, s, o, (toInt(timestamp()/1000)-toInt(o.created_on))/86400 as days,
(toInt(timestamp()/1000)-toInt(o.created_on))/3600 as hours,
(1- round(o.impression_count_all/20)/50) as low_boost
with m,s,o,days,low_boost,hours,
CASE
WHEN days > 30 THEN 0.05
WHEN days >=20 AND days <=30 THEN 0.1
WHEN days >=10 AND days <=20 THEN 0.2
WHEN days >=5 AND days <=10 THEN 0.4
WHEN days >=2 AND days <=5 THEN 0.5
WHEN days =1 THEN 0.6
WHEN days < 1 THEN
CASE
WHEN hours <= 2 THEN 1
WHEN hours > 2 AND hours <= 8 THEN 0.9
WHEN hours > 8 AND hours <= 16 THEN 0.8
WHEN hours > 16 AND hours < 23 THEN 0.75
WHEN hours >= 23 AND hours <= 24 THEN 0.7
END
END as rs,
CASE
WHEN low_boost > 0 THEN low_boost
WHEN low_boost <= 0 THEN 0
END as lb
where has(o.trending_score_all) and has(o.impression_count_all) and not(o.is_featured=2)
RETURN distinct o.story_id as story_id,
(o.trending_score_all*4) as ts, (o.trending_score_all + rs + lb) as final_score,
count(s) as rel_count,max(s.activity_id) as id, toInt(o.created_on) as created_on
ORDER BY (CASE WHEN ts > 3 THEN final_score desc, rel_count desc ELSE ts) END) DESC
skip 0 limit 10;
Now If ts > 3 ,I want to sort results by both final_score and rel_count ELSE srt only by ts..
Please modify order by..
Does this much simplified query (which uses a single argument for ORDER BY) work for you?
MATCH (u:User)-[r:like]->(s:Story)
WITH s, (s.trending_score_all*4) AS ts
RETURN DISTINCT s.story_id, ts, TOINT(s.impression_count_72)
ORDER BY (CASE WHEN ts > 3 THEN ts ELSE TOINT(s.impression_count_72) END) DESC
LIMIT 10;
[EDITED]
If you need to sort by a varying number of values (depending on the situation) you have to use a workaround, as Cypher does not support that directly.
For example, suppose when (ts > 3) you wanted to order by ts DESC and then by s.story_id ASC. In this situation, you could change the above ORDER BY clause to this:
ORDER BY
(CASE WHEN ts > 3 THEN ts ELSE TOINT(s.impression_count_72) END) DESC,
(CASE WHEN ts > 3 THEN s.story_id ELSE NULL END) ASC
By using NULL (or any literal value) in this way, you can have any of the sub-sorts effectively do nothing.
1) You use pagination (skip and limit)
2) If I understand what you need, then add "else" to sort:
UNWIND RANGE(1,100) as i
WITH i, rand()*5 as x, toInt(rand()*10) as y
RETURN i, x, y, CASE WHEN x>3 THEN 1 ELSE 0 END as for_sort
ORDER BY
CASE
WHEN for_sort=1 THEN x ELSE y END DESC,
CASE
WHEN for_sort=1 THEN y ELSE x END DESC

Find top 3 nodes with maximum relationships

The structure of my data base is:
( :node ) -[:give { money: some_int_value } ]-> ( :Org )
One node can have multiple relations.
I need to find top 3 nodes with the most number of relations :give with their property money holding: vx <= money <= vy
Using ORDER BY and LIMIT should solve your problem:
Match ( n:node ) -[r:give { money: some_int_value } ]-> ( :Org )
RETURN n
ORDER BY count(r) DESC //Order by the number of relations each node has
LIMIT 3 //We only want the top 3 nodes
Instead of using the label 'node', maybe use something more descriptive like Person for the label so the datamodel is more clear:
MATCH (p:Person)-[r:give]->(o:Org)
WITH count(r) AS num, sum(r.money) AS total, p
RETURN p, num, total ORDER BY num DESC LIMIT 3;
I'm not sure what you mean by "their property money holding: vx <= money <= vy". If you could clarify I can update my answer accordingly. You can calculate the total of the money properties using the sum() function.
Edit
To only include relationships with money property with value greater than 10 and less 25:
MATCH (p:Person)-[r:give]->(o:Org)
WHERE r.money >= 10 AND r.money <= 25
WITH count(r) AS num, sum(r.money) AS total, p
RETURN p, num, total ORDER BY num DESC LIMIT 3;

Cypher: find a path which takes the maximum valued step each time

I am trying to write a cypher query that finds a path between nodes a and b such that each step has the maximum timestamp value out of all available alternatives that is less than 15.
Here is my query so far, it does everything except for select the maximum possible timestamp at each step. How do I express this condition?
MATCH path=(a:NODE)-[rs:PARENT*]->(b:NODE)
WHERE a.name = 'SOME_VALUE' and b.name = 'SOME_OTHER_VALUE' AND ALL (r IN rs
WHERE r.timestamp < 15)
RETURN path
This is just awful sudo code but I think it expresses what I am looking for
MATCH path=(a:NODE)-[rs:PARENT*]->(b:NODE)
WHERE a.name = 'SOME_VALUE' and b.name = 'SOME_OTHER_VALUE' AND ALL (r IN rs
WHERE r.timestamp < 15 AND r.timestamp = max(allPossibleRsForThisStep))
RETURN path
Can this kind of query be written in cypher?
It won't be fast in cypher, it's possible to compute all maximum values first and then do what you want to do by compare the max value in a list with the current value.
Something like this (not sure if it works)
WITH range(1,10) as max_vals // a list with 10 values (actual values are not that important)
MATCH (a:NODE)-[rs:PARENT*..10]->(b:NODE)
WHERE a.name = 'SOME_VALUE' and b.name = 'SOME_OTHER_VALUE'
WITH a,b,
map(idx in range(0,size(rs)) |
max_vals[idx] = case when max_vals[idx]<rs[idx].timestamp then rs[idx].timestamp else max_vals[idx] end ), max_vals
MATCH path=(a)-[rs:PARENT*..10]->(b)
AND ALL (idx in range(0,size(rs) WHERE rs[idx].timestamp < 15 AND rs[idx].timestamp = max_vals[idx])
RETURN path

Resources