Extra match clause causes very slow neo4j query - neo4j

Question

I have the following query, which is very slow:
MATCH (defn :WORKFLOW_DEFINITION { uid: '48a72b6b-6791-4da9-8d8f-dc4375d0fe2d' }),
(instance :WORKFLOW_INSTANCE :SUCCEEDED) -[:DEFINED_BY]-> (defn)
WITH instance, defn
ORDER BY instance.createdAt, instance.uid
LIMIT 1000
OPTIONAL MATCH (instance) -[:INPUT]-> (input :ARTIFACT) <-[:DEFINES_INPUT]- (inputDefn :WORKFLOW_ARTIFACT) <-[:TAKES_INPUT]- (defn)
WITH instance, defn,
collect([input.uid, inputDefn.label, input.bucket, input.key, input.blobUid]) AS inputs
RETURN instance, defn, inputs
The structure of my data is as follows:
WORKFLOW_DEFINITION elements define workflows; each has edges to
one-to-many WORKFLOW_ARTIFACTs (the relationship is TAKES_INPUT) defining the different inputs the definition takes.
WORKFLOW_INSTANCE elements are instances of workflow definitions; they link to their parent definition via DEFINED_BY.
each WORKFLOW_INSTANCE takes one-to-many (representing real/existing on disk files) ARTIFACTs (edges via INPUT relationships).
the number of input ARTIFACTs a WORKFLOW_INSTANCE
takes equals the number of WORKFLOW_ARTIFACTs its parent
WORKFLOW_DEFINITION links to.
each input ARTIFACT links to exactly one of
the parent's WORKFLOW_ARTIFACTs (the WORKFLOW_ARTIFACT provides
data on how the ARTIFACT is consumed when the workflow is run)
I currently have a handful of WORKFLOW_DEFINITIONS. Most WORKFLOW_DEFINITIONs have a few WORKFLOW_INSTANCEs, but one (the one I'm interested querying) has ~2000 WORKFLOW_INSTANCEs. The definition has three WORKFLOW_ARTIFACTs, and hence each WORKFLOW_INSTANCE for this definition has three ARTIFACTs linked to the WORKFLOW_ARTIFACTs.
When I run the query above it requires 50449598 total db hits and takes 16654 ms.
On the other hand, the following query, omitting only the '<-[:TAKES_INPUT]- (defn)' at the end of line 6, requires only 55598 total db hits and takes 82 ms. Here's the fast query with that bit omitted:
MATCH (defn :WORKFLOW_DEFINITION { uid: '48a72b6b-6791-4da9-8d8f-dc4375d0fe2d' }),
(instance :WORKFLOW_INSTANCE :SUCCEEDED) -[:DEFINED_BY]-> (defn)
WITH instance, defn
ORDER BY instance.createdAt, instance.uid
LIMIT 1000
OPTIONAL MATCH (instance) -[:INPUT]-> (input :ARTIFACT) <-[:DEFINES_INPUT]- (inputDefn :WORKFLOW_ARTIFACT)
WITH instance, defn,
collect([input.uid, inputDefn.label, input.bucket, input.key, input.blobUid]) AS inputs
RETURN instance, defn, inputs
Here are the profiled query plans for slow:
Slow query plan
and for fast:
Fast query plan
Why is that final edge back to the WORKFLOW_DEFINITION (to ensure we're getting the correct WORKFLOW_ARTIFACT, since an ARTIFACT may be used in different workflows) making the query so slow? Where's this combinatorial-explosion-looking db hits number coming from?
Thanks in advance for your help, and please let me know if there's anything else I can do to clarify!

Answer 1

Looking at the two query plans, there is a difference in how the query plans are processing the expansions.
If you look at the flow of operations in the slower query plan, you can see that it expands from defn to inputDefn, next from inputDefn to input, and finally an Expand(Into) from instance to input.
The faster query doesn't consider defn when it begins processing the right hand side (since it's not involved in any expansions of the OPTIONAL MATCH), so it can't use that same approach. Instead it expands from the other direction: instance to input, then input to inputDefn. Looks like traversal using this direction ends up being much more efficient.
The takeaway is that traversing from input to inputDefn is efficient, it looks to be a 1-1 correlation. However inputDefn to input seems to be a horrible expansion, looks like inputDefn tend to be supernodes, with an average of 2400 relationships to input nodes each.
As for how to fix this, while there aren't planner hints for explicitly avoiding an expansion in a certain direction, we may be able to use join hints to nudge the planner toward that effect. (you can try the join hint on different variables and EXPLAIN the plan to ensure you're avoiding inputDefn expansions to input).
If we add the following hint: USING JOIN ON inputDefn, then this means we'll expand toward inputDefn from both sides and do a hash join to get the final result set. This will avoid the costly outgoing expansion from inputDefn to input.
MATCH (defn :WORKFLOW_DEFINITION { uid: '48a72b6b-6791-4da9-8d8f-dc4375d0fe2d' }),
(instance :WORKFLOW_INSTANCE :SUCCEEDED) -[:DEFINED_BY]-> (defn)
WITH instance, defn
ORDER BY instance.createdAt, instance.uid
LIMIT 1000
OPTIONAL MATCH (instance) -[:INPUT]-> (input :ARTIFACT) <-[:DEFINES_INPUT]- (inputDefn :WORKFLOW_ARTIFACT) <-[:TAKES_INPUT]- (defn)
USING JOIN ON inputDefn
WITH instance, defn,
collect([input.uid, inputDefn.label, input.bucket, input.key, input.blobUid]) AS inputs
RETURN instance, defn, inputs