Importing data from two files into neo4j using LOAD CSV - neo4j

I am trying to import data into neo4j using LOAD CSV
The resource file contains names of all the nodes I need to create
resource1
resource2
resource3
In another file I have all the properties of that resource
resource1,name,xyz
resource1,year,1920
resource1,age,100
resource2,length,300
resource2,age,30
I Managed to load the nodes into neo4j but how do I import the second file so that I can add the data to that particular node as properties, I tried setting the key dynamically
USING PERIODIC COMMIT
LOAD CSV FROM 'file:///infobox.csv' AS line
MERGE (:Node{line[1]:line[2]})
neo4j doesn't allow setting the key dynamically?
How do I solve this?

Natively, Neo4j doesn't allow setting the key dynamically. But you can install APOC Procedures use apoc.create.setProperty to do this.
Try something like:
USING PERIODIC COMMIT
LOAD CSV FROM 'file:///infobox.csv' AS line
// match the node by resource1, resource2, etc
MATCH(node:Node{resource_id : line[0]})
CALL apoc.create.setProperty(node, line[1], line[2])
RETURN *
Note: Remember to install APOC procedures according the version of Neo4j you are using. Take a look in the Version Compatibility Matrix.

Related

Joern:how to use ast's csv file to make cpg's csv file by joern?

As we all know,joern can easily analysis project written by c/c++,and it will generate CPG(code property graph).
Now I hava 2 csv file(nodes.csv and rels.csv),the 2 csv file can build AST(abstract syntax code)by neo4j and it can be shown in neo4j.But AST is not enough for me,what i need is CPG.How could i use the 2 csv file to build a CPG and show it in neo4j.I wanna to know how to use joern to make my ast'csv file become cpg'csv file.Or you hava other ways to make me do this.
I relly need your help,please.Thank you very much.

Cannot import csv file using neo4j-admin

Hello i am trying to import csv file using neo4j-admin tool but i am receiving this error graph.db' already contains a database
Does this mean that i need to create new database every new import ?
neo4j-admin import is only for creating a new db, it cannot be used to add to an existing db, so you would need to delete your graph.db folder prior to import.
You'll need to use an alternate means, like a LOAD CSV query if you want to add into an existing graph.

Neo4j APOC import error

I have a data model that starts with a single record, this has a custom "recordId" that's a uuid, then it relates out to other nodes and they then in turn relate to each other. That starting node is what defines the data that "belongs" together, as in if we had separate databases inside neo4j. I need to export this data, into a backup data-set that can be re-imported into either the same or a new database with ease
After some help, I'm using APOC to do the export:
call apoc.export.cypher.query("MATCH (start:installations)
WHERE start.recordId = \"XXXXXXXX-XXX-XXX-XXXX-XXXXXXXXXXXXX\"
CALL apoc.path.subgraphAll(start, {}) YIELD nodes, relationships
RETURN nodes, relationships", "/var/lib/neo4j/data/test_export.cypher", {})
There are then 2 problems I'm having:
Problem 1 is the data that's exported has internal neo4j identifiers to generate the relationships. This is bad if we need to import into a new database and the UNIQUE IMPORT ID values already exist. I need to have this data generated with my own custom recordIds as the point of reference.
Problem 2 is that the import doesn't even work.
call apoc.cypher.runFile("/var/lib/neo4j/data/test_export.cypher") yield row, result
returns:
Failed to invoke procedure apoc.cypher.runFile: Caused by: java.lang.RuntimeException: Error accessing file /var/lib/neo4j/data/test_export.cypher
I'm hoping someone can help me figure out what may be going on, but I'm not sure what additional info is helpful. No one in the Neo4j slack channel has been able to help find a solution.
Thanks.
problem1:
The exported file does not contain any internal neo4j ids. It is not safe to use neo4j ids out of the database, since they are not globally unique. So you should not use them to transfer data from one database to another.
If you are about to use globally uniqe ids, you can use an external plugin like GraphAware UUID plugin. (disclaimer: I work for GraphAware)
problem2:
If you cannot access the file, then possible reasons:
apoc.import.file.enabled=true is not set in neo4j.conf
os level
permission is not set

Gephi finds no node/relation properties on import

I want to visualize my Neo4j dataset with Gephi. After installing apoc and get it working, I called call apoc.export.graphml.all("/tmp/test2.graphml",{}) and I get the right file. Now I import/open this .graphml-file in Gephi 0.9.1 but in the import-window I can´t see any properties. Also in the graph itself there´re no properties on the nodes / relations.
Does anyknow know, what I´m doing wrong or have I forgot to set the right configuration-parameters?
Thanks in advance
UDPATE
this is my procedure call:
call apoc.export.graphml.all("/tmp/test2.graphml",{}) yield nodes, relationships, properties, time
this is the snapchot from the Neo4j browser
I´ve loaded this file from my server and openend it in Gephi, resulted in this:
Like you see, my properties are still not there...
Apoc has a custom procedure, that exports data to Gephi in one single step. You will need to download a graph streaming plugin for Gephi,so you will be able to easily export data from Neo4j to Gephi using apoc.gephi procedures.
Example:
MATCH path = (:Person)-[:KNOWS]->(:Person)
CALL apoc.gephi.add(null,'workspace1',path,'weight') yield nodes
RETURN distinct("success")
Check out the docs and this tutorial for more info.

Batch import in Neo4j

I am using the BatchInserter to insert data from a CSV file which works fine when the DB is completely empty (no files in the data directory). But using the BatchInserter I cannot get data into the database and it throws an exception mentioned below. This is with the DB service stopped. I tried several ways and failed. But I need to know if there is a way to import data from a CSV into an existing DB as the CSV is prone to change.
Exception in thread "main" java.lang.IllegalStateException: Misaligned file size 68 for DynamicArrayStore[fileName:neostore.nodestore.db.labels, blockSize:60], expected version length 25
at org.neo4j.kernel.impl.store.AbstractDynamicStore.verifyFileSizeAndTruncate(AbstractDynamicStore.java:265)
at org.neo4j.kernel.impl.store.CommonAbstractStore.loadStorage(CommonAbstractStore.java:217)
at org.neo4j.kernel.impl.store.CommonAbstractStore.<init>(CommonAbstractStore.java:118)
at org.neo4j.kernel.impl.store.AbstractDynamicStore.<init>(AbstractDynamicStore.java:92)
at org.neo4j.kernel.impl.store.DynamicArrayStore.<init>(DynamicArrayStore.java:64)
at org.neo4j.kernel.impl.store.StoreFactory.newNodeStore(StoreFactory.java:327)
at org.neo4j.kernel.impl.store.StoreFactory.newNodeStore(StoreFactory.java:316)
at org.neo4j.kernel.impl.store.StoreFactory.newNeoStore(StoreFactory.java:160)
at org.neo4j.unsafe.batchinsert.BatchInserterImpl.<init>(BatchInserterImpl.java:258)
at org.neo4j.unsafe.batchinsert.BatchInserters.inserter(BatchInserters.java:94)
at org.neo4j.unsafe.batchinsert.BatchInserters.inserter(BatchInserters.java:88)
at org.neo4j.unsafe.batchinsert.BatchInserters.inserter(BatchInserters.java:63)
at org.neo4j.unsafe.batchinsert.BatchInserters.inserter(BatchInserters.java:51)
at net.xalgo.neo4j.batchinserter.DrugDatabaseInserter.start(DrugDatabaseInserter.java:30)
at net.xalgo.neo4j.batchinserter.BatchInserterApp.main(BatchInserterApp.java:56)
The Batch Inserter only works with initial data import (when the database is empty). It avoids transactions and other checks to increase performance and therefore cannot be used with an existing database.
For importing data into an already existing Neo4j database you can use LOAD CSV Cypher.
Finally I had to remove BatchInserter and use spring data Neo4jTemplate with direct cypher in a non transactional way.

Resources