Performance: Creating a relationship in neo4j based on property ID - neo4j

For test purposes i imported data from another datasource into neo4j.
I imported the data only as nodes. Now i want to add the edges based on the imported ID. Every node has 2 fields
id: contains the identification as String
from: contains all connections as a String[]
For performance improvements i also created an index for the propertiy "id" and an index for the property "from"
First i created both properties as String (the from list as comma separated String).
This works, but is really slow:
MATCH (e:Test1),(r:Test2)
WHERE r.from CONTAINS e._id
MERGE (e)-[:HAS]->(r)
is there a better way?
PS: i tried also to store the from field as String[]. than i used the following query
MATCH (e:Test1),(r:Test2)
WHERE e._id IN r.from
MERGE (e)-[:HAS]->(r)
-> Performance is the same

The problem is that you take a combination of all components - the Cartesian product. In both cases. More would be better to split the string by comma to identifiers. For example:
MATCH (T2:Test2)
UNWIND split(T2.from, ",") as id
MATCH (T1:Test1) WHERE T1._id = id
MERGE (T1)-[:HAS]->(T2)
Or, if you keep the identifiers in the array:
MATCH (T2:Test2)
UNWIND T2.from as id
MATCH (T1:Test1) WHERE T1._id = id
MERGE (T1)-[:HAS]->(T2)
And, of course, do not forget about the index.

Actually, at import time, you should be creating the :HAS relationships instead of creating the from property (which forces you to make a wasteful additional query to create the relationships, and leaves you with redundant from properties that you would probably want to delete).
For example, if you are using LOAD CSV to import, and your import file has test2Id and from columns (a string and a string collection, respectively), this import query should create all the nodes and relationships:
LOAD CSV WITH HEADERS FROM "file:///input.csv" AS row
MERGE (t2:Test2 {id: row.test2Id})
WITH row, t2
UNWIND row.from AS t1Id
MERGE (t1:Test1 {id: t1Id})
MERGE (t1)-[:HAS]->(t2);
For better performance, you would want indexes on both :Test1(id) and :Test2(id).

Related

Loading sparse adjacency matrix on Neo4j

I'm trying to load a sparse (co-occurrence) matrix in Neo4j but after many failed queries, it's getting frustrating.
Raw data
Basically, I want to create the nodes from the ids, and the relationship weight against each other node (including itself) should be the value on the matrix.
So, for example, 'nhs' should have a self-relationship with weight 41 and 16 with 'england', and so on.
I was trying things like:
LOAD CSV WITH HEADERS FROM 'file:///matpharma.csv' AS row
MERGE (a: node{name: row.id})
MERGE (b: node{name: row.key})
MERGE (a)-[:w]-(b);
I'm not sure how to attach the edge values though (and not yet sure if the merges are producing the expected result).
Thanks in advance for the assistance
If you just need to add a property on a relationship, where the property value is in your CSV, then it's just a matter of adding a variable for the relationship that you MERGE in, and then using SET (or ON CREATE SET, if you only want to set the property if the relationship didn't exist and needed to be created). So something like:
LOAD CSV WITH HEADERS FROM 'file:///matpharma.csv' AS row
MERGE (a: node{name: row.id})
MERGE (b: node{name: row.key})
MERGE (a)-[r:w]-(b)
SET r.weight = row.weight
EDIT
Ah, took a look at the CSV clip. This is a very strange way to format your data. You have data in your header (that is, your headers are trying to define the other node to lookup) which is the wrong way to go about this. You should instead have, per row, one column that defines one of the two nodes to connect (like the "id" column) and then another column for the other node (something like an "id2"). That way you can just do two MATCHes to get your nodes, then a MERGE between them, and then setting the relationship property, similar to the sample query I provided above.
But if you're set on this format, then it's going to be a more complicated query, since we have to deal with dynamic access of the row keys and values.
Something like:
LOAD CSV WITH HEADERS FROM 'file:///matpharma.csv' AS row
MERGE (start:Node {name:row.id})
WITH start, row, [key in keys(row) WHERE key <> 'id'] as keys
FOREACH (key in keys |
MERGE (end:Node {name:key})
MERGE (start)-[r:w]-(end)
ON CREATE SET r.weight = row[key] )
This is a nice Cypher challenge :) Let's say that LOAD CSV is not really meant to do this and probably you would be happier by flattening your data
Here is what I came up with :
LOAD CSV FROM "https://gist.githubusercontent.com/ikwattro/a5260d131f25bcce97c945cb97bc0bee/raw/4ce2b3421ad80ca946329a0be8a6e79ca025f253/data.csv" AS row
WITH collect(row) AS rows
WITH rows, rows[0] AS firstRow
UNWIND rows AS row
WITH firstRow, row SKIP 1
UNWIND range(0, size(row)-2) AS i
RETURN firstRow[i+1], row[0], row[i+1]
You can take a look at the gist

How to match line of csv which is ignored by constraint and create only relationship

I have been created a graph having a constraint on primary id. In my csv a primary id is duplicate but the other proprieties are different. Based on the other properties I want to create relationships.
I tried multiple times to change the code but it does not do what I need.
USING PERIODIC COMMIT
LOAD CSV WITH HEADERS FROM 'file:///Trial.csv' AS line FIELDTERMINATOR '\t'
MATCH (n:Trial {id: line.primary_id})
with line.cui= cui
MATCH (m:Intervention)
where m.id = cui
MERGE (n)-[:HAS_INTERVENTION]->(m);
I already have the nodes Intervention in the graph as well as the trials. So what I am trying to do is to match a trial with the id from intervention and create only the relationship. Instead is creating me also the nodes.
This is a sample of my data, so the same primary id, having different cuis and I am trying to match on cui:
You can refer the following query which finds Trial and Intervention nodes by primary_id and cui respectively and creates the relationship between them.
USING PERIODIC COMMIT
LOAD CSV WITH HEADERS FROM 'file:///Trial.csv' AS line FIELDTERMINATOR '\t'
MATCH (n:Trial {id: line.primary_id}), (m:Intervention {id: line.cui})
MERGE (n)-[:HAS_INTERVENTION]->(m);
The behavior you observed is caused by 2 aspects of the Cypher language:
The WITH clause drops all existing variables except for the ones explicitly specified in the clause. Therefore, since your WITH clause does not specify the n node, n becomes an unbound variable after the clause.
The MERGE clause will create its entire pattern if any part of the pattern does not already exist. Since n is not bound to anything, the MERGE clause would go ahead and create the entire pattern (including the 2 nodes).
So, you could have fixed the issue by simply specifying the n variable in the WITH clause, as in:
WITH n, line.cui= cui
But #Raj's query is even better, avoiding the need for WITH entirely.

Create node and relationship given parent node

I am creating a word tree but when I execute this cypher query:
word = "MATCH {} MERGE {}-[:contains]->(w:WORD {{name:'{}'}}) RETURN w"
.format(parent_node, parent_node, locality[i])
where parent_node has a type Node
It throws this error:
py2neo.cypher.error.statement.InvalidSyntax: Can't create `n8823` with properties or labels here. It already exists in this context
formatted query looks like this:
'MATCH (n8823:HEAD {name:"sanjay"}) MERGE (n8823:HEAD {name:"sanjay"})-[:contains]->(w:WORD {name:\'colony\'}) RETURN w'
The formatted query is broken and won't work, but I also don't see how that could be what the formatted query actually looks like. When you do your string format you pass the same parameter (parent_node) twice so the final string should repeat whatever that parameter looks like. It doesn't, and instead has two different patterns for the match and merge clauses.
Your query should look something like
MATCH (n8823:Head {name: "sanjay"})
MERGE (n8823)-[:CONTAINS]->(w:Word {name: "colony"})
RETURN w
It's probably a bad idea to do string formatting on a Node object. Better to either use property values from your node object in a Cypher query to match the right node (and only the variable that you bind the matched node to in the merge clause) or use the methods of the node object to do the merge.
Although the MERGE clause is able to bind identifiers (like n8823), Cypher unfortunately does not allow MERGE to re-bind an identifier that had already been bound -- even if it would not actually change the binding. (On the other hand, the MATCH clause does allow "rebinding" to the same binding.) Simply re-using a bound identifier is OK, though.
So, the workaround is to change your Cypher query to re-use the bound identifier. Also, the recommended way to dynamically specify query data without changing the overall structure of a query is to use "query parameters". For py2neo, code along these lines should work for you (note that the parent_name variable would contain a name string, like "sanjay"):
from py2neo import Graph
graph = Graph()
cypher = graph.cypher
results = cypher.execute(
"MATCH (foo:{name:{a}}) MERGE (foo)-[:contains]->(w:WORD {{name:'{b}'}}) RETURN w",
a=parent_name, b=locality[i])

Avoid processing duplicate data when CSV importing via Cypher

I have a set of CSV files with duplicate data, i.e. the same row might (and does) appear in multiple files. Each row is uniquely identified by one of the columns (id) and has quite a few other columns that indicate properties, as well as required relationships (i.e. ids of other nodes to link to). The files all have the same format.
My problem is that, due to size and number of the files, I want to avoid processing the rows that already exist - I know that as long as id is the same, the contents of the rows will be the same across the files.
Can any cypher wizard advise how to write a query that would create the node, set all the properties and create all the relationship if a node with given id does not exist, but skip the action altogether if such node is found? I tried with MERGE ON CREATE, something along the lines of:
LOAD CSV WITH HEADERS FROM "..." AS row
MERGE (f:MyLabel {id:row.uniqueId})
ON CREATE SET f....
WITH f,row
MATCH (otherNode:OtherLabel {id : row.otherNodeId})
MERGE (f) -[:REL1] -> (otherNode)
but unfortunately that can only be applied to not setting the properties again, but I couldn't work out how to skip the merging part of relationships (only shown one here, but there are quite a few more).
Thanks in advance!
You can just optionally match the node and then skip with WHERE n IS NULL
Make sure you have an index or constraint on :MyLabel(id)
LOAD CSV WITH HEADERS FROM "..." AS row
OPTIONAL MATCH (f:MyLabel {id:row.uniqueId})
WHERE f IS NULL
MERGE (f:MyLabel {id:row.uniqueId})
ON CREATE SET f....
WITH f,row
MATCH (otherNode:OtherLabel {id : row.otherNodeId})
MERGE (f) -[:REL1] -> (otherNode)

Neo4J Arrays in MATCH query

The intention of my Query is to mark similar words.
CREATE CONSTRAINT ON (n:Word) ASSERT n.title IS UNIQUE
MATCH (n) WHERE ID(n)={id}
MERGE (o:Word{title:{title}})
WITH n,o MERGE n-[r:SIMILAR{location:'{location}'}]->o
RETURN ID(o)
n is a existing Word. I want to create the relationsship & the other Word (o) if they don't exist yet.
The Problem with this query is, that it works for one title, but if I use a Array with titles the title of the Word o is the whole Array.
Can you suggest me another Query that does the same and/or a way to pass multiple values to title.
I'm using the Neography Gem on Rails e.g. the REST API
To use individual values in a parameter array you can use FOREACH, something like
MATCH (n)
WHERE ID (n) = {id}
FOREACH (t IN {title} |
MERGE (o:Word {title:t})
MERGE n-[:SIMILAR]->o
)
If you want to pass location also as a parameter (it is actually a string literal in your current query), such that merge operations for n should happen for each title, location pair in a parameter array, you can try
FOREACH (map IN {maps} |
MERGE (o:Word {title:map.title})
MERGE n-[:SIMILAR {location:map.location}]->o
)
with a parameter that looks something like
{
"maps": [
{
"title":"neography",
"location":"1.."
},{
"title":"coreography",
"location":"3.."
}
]
}
Other suggestions:
It's usually not great to look up nodes by internal id from parameter. In some cases when chaining queries it may be fine, but in most cases label index lookup would be better: MATCH (n:Word {title:"geography"})
If you are not using the transactional cypher endpoint, give it a shot. You can then make one or more calls with one or more queries in each call within one transaction. Performance improves and you may find you don't need to send the more complex parameter object, but can send many simple queries.

Resources