I have an employee database with 1800 employees and 45000 messages in between them. I am trying to export the results of the following query into a csv file by clicking the export option in the neo4j browser.
LOAD CSV WITH HEADERS FROM
'file:///employees.csv' AS line
WITH line
MATCH(e:Employee{pkey:line.profile_key})-[r:Message]->(b:Employee) RETURN
e.pkey, b.pkey, COUNT(r)
ORDER BY e.pkey;
But its not working. I only get the initial 100 rows.I have also changed the number of rows to 10000 in the browser settings, but then again after the execution of the query my browser stops responding and closes automatically. I am using neo4j community edition 3.2.1 on windows. Is there any other way to export the results other than the browser option in windows? Thanks in advance!
You might want to use the APOC procedures and run them from the Cypher shell. Example :
neo4j> CALL apoc.export.csv.query("MATCH(p:Part) RETURN p.name","/var/tmp/parts.csv", {});
This does require you to setup the apoc plugin and add the following parameter to neo4j.conf
apoc.export.file.enabled=true
Hope this helps,
Tom
Related
I have tried all possible ways to load the pole.dump file into Neo4J :
I have been doing the following for past 3 days now:
Opened the Neo4J Desktop and Using the Add Drop-Down Menu , I have added the pole.dump into the Neo4J Desktop.
Then I have selected the Import dump into existing DBMS -> which is my Graph3 Database.
Then I am going to Neo4J Desktop and from the Database Information, I selected the pole database but I am getting this error
Database "pole" is unavailable, its status is "offline".
I also tried this: https://community.neo4j.com/t5/graphacademy-discussions/cannot-create-new-database-from-dump-file/td-p/39914
i. Database-->Open Folder-->DBMS. Here you will see data/dumps folder
ii. Copy pole.dump file to data/dumps folder (Although there is no folder called dumps in the data folder)
iii. Close the browser. Click on ... and select Terminal.
iv. Terminal window shows up. Enter this command:
bin/neo4j-admin load --from=data/dumps/pole.dump --database=pole --force
v. If successful, close the Terminal window and open the db in browser.
vi. Click on the database icon on top left to see the databases from the dropdown box.
Here you will not see pole db.
vii. Select 'system' database. On the right pane run this Cypher:
CREATE DATABASE pole and execute the Cypher.
viii. Run SHOW DATABASES and you should see pole and check the status. Status should be 'online'.
ix. Select pole from the dropdown list. Once selected you should see all the nodes,
relationships on the left. Now you can start playing with it!!
But I could not pass after point iv as it says in the neo4j terminal if I open it from the Neo4J Desktop , that it could not load - in fact it says there is a parsing error.
I did check with the following:
C:\Users\Chirantan\.Neo4jDesktop\relate-data\dbmss\dbms-11aabb23-daca-4d35-9043-6c039d133a34\bin>neo4j-import Graph3 load --from=data/dumps/pole.dump
'neo4j-import' is not recognized as an internal or external command,
operable program or batch file.
I am coming to this platform because I have tried everything available:
https://neo4j.com/docs/operations-manual/current/tools/neo4j-admin/
'neo4j-admin' is not recognized as an internal or external command, operable program or batch file
https://www.youtube.com/watch?v=HPwPh5FUvAk
But could not get any luck.
After step 3, which is:
Then I am going to Neo4J Desktop and from the Database Information, I selected the pole database but I am getting this error
Did you try to start the database with the following command?
START DATABASE pole;
I have already solved the problem. The issue was existent even after I followed the steps provided in the OP. What I did was: I created random texts for all records for names of criminals/victims, friends of victims, friends of criminals, generated random phone numbers, generated random nhs numbers, also generated random addresses using :
https://fossbytes.com/tools/random-name-generator
https://www.randomlists.com/london-addresses?qty=699
Using this code I generated random nhs ids :
import string
import random
# initializing size of string
N = 7
list_str= []
for i in range(699):
# using random.choices()
# generating random strings
res = ''.join(random.choices(string.ascii_uppercase +
string.digits, k=N))
list_str.append(res)
Random Phone Number generated using:
https://fakenumber.in/united-kingdom
There is a better answer.
Go to this url : https://neo4j.com/sandbox/
Then Select the Pre-Installed Databases that come with sandbox- Crime Investigation being one of them with the POLE Database pre-installed.
You will be prompted to open the HOME from there with the POLE Database pre-installed.
You Finally open the Neo4J Browser from here using the drop down menu by pressing the Open Button and access the Neo4J Browser and voila! You can access POLE database using Neo4J
I am trying to import data into neo4j using LOAD CSV
The resource file contains names of all the nodes I need to create
resource1
resource2
resource3
In another file I have all the properties of that resource
resource1,name,xyz
resource1,year,1920
resource1,age,100
resource2,length,300
resource2,age,30
I Managed to load the nodes into neo4j but how do I import the second file so that I can add the data to that particular node as properties, I tried setting the key dynamically
USING PERIODIC COMMIT
LOAD CSV FROM 'file:///infobox.csv' AS line
MERGE (:Node{line[1]:line[2]})
neo4j doesn't allow setting the key dynamically?
How do I solve this?
Natively, Neo4j doesn't allow setting the key dynamically. But you can install APOC Procedures use apoc.create.setProperty to do this.
Try something like:
USING PERIODIC COMMIT
LOAD CSV FROM 'file:///infobox.csv' AS line
// match the node by resource1, resource2, etc
MATCH(node:Node{resource_id : line[0]})
CALL apoc.create.setProperty(node, line[1], line[2])
RETURN *
Note: Remember to install APOC procedures according the version of Neo4j you are using. Take a look in the Version Compatibility Matrix.
I have a data model that starts with a single record, this has a custom "recordId" that's a uuid, then it relates out to other nodes and they then in turn relate to each other. That starting node is what defines the data that "belongs" together, as in if we had separate databases inside neo4j. I need to export this data, into a backup data-set that can be re-imported into either the same or a new database with ease
After some help, I'm using APOC to do the export:
call apoc.export.cypher.query("MATCH (start:installations)
WHERE start.recordId = \"XXXXXXXX-XXX-XXX-XXXX-XXXXXXXXXXXXX\"
CALL apoc.path.subgraphAll(start, {}) YIELD nodes, relationships
RETURN nodes, relationships", "/var/lib/neo4j/data/test_export.cypher", {})
There are then 2 problems I'm having:
Problem 1 is the data that's exported has internal neo4j identifiers to generate the relationships. This is bad if we need to import into a new database and the UNIQUE IMPORT ID values already exist. I need to have this data generated with my own custom recordIds as the point of reference.
Problem 2 is that the import doesn't even work.
call apoc.cypher.runFile("/var/lib/neo4j/data/test_export.cypher") yield row, result
returns:
Failed to invoke procedure apoc.cypher.runFile: Caused by: java.lang.RuntimeException: Error accessing file /var/lib/neo4j/data/test_export.cypher
I'm hoping someone can help me figure out what may be going on, but I'm not sure what additional info is helpful. No one in the Neo4j slack channel has been able to help find a solution.
Thanks.
problem1:
The exported file does not contain any internal neo4j ids. It is not safe to use neo4j ids out of the database, since they are not globally unique. So you should not use them to transfer data from one database to another.
If you are about to use globally uniqe ids, you can use an external plugin like GraphAware UUID plugin. (disclaimer: I work for GraphAware)
problem2:
If you cannot access the file, then possible reasons:
apoc.import.file.enabled=true is not set in neo4j.conf
os level
permission is not set
I want to visualize my Neo4j dataset with Gephi. After installing apoc and get it working, I called call apoc.export.graphml.all("/tmp/test2.graphml",{}) and I get the right file. Now I import/open this .graphml-file in Gephi 0.9.1 but in the import-window I can´t see any properties. Also in the graph itself there´re no properties on the nodes / relations.
Does anyknow know, what I´m doing wrong or have I forgot to set the right configuration-parameters?
Thanks in advance
UDPATE
this is my procedure call:
call apoc.export.graphml.all("/tmp/test2.graphml",{}) yield nodes, relationships, properties, time
this is the snapchot from the Neo4j browser
I´ve loaded this file from my server and openend it in Gephi, resulted in this:
Like you see, my properties are still not there...
Apoc has a custom procedure, that exports data to Gephi in one single step. You will need to download a graph streaming plugin for Gephi,so you will be able to easily export data from Neo4j to Gephi using apoc.gephi procedures.
Example:
MATCH path = (:Person)-[:KNOWS]->(:Person)
CALL apoc.gephi.add(null,'workspace1',path,'weight') yield nodes
RETURN distinct("success")
Check out the docs and this tutorial for more info.
I have a Cypher script for populating a Neo4j (2.2.3) database. Currently, the names of all the CSV files are hard coded. Is there a way to parameterize the CSV files, in case I'd like to switch to a different web server or switch to using the local file system?
Update
I forgot to mention that my use case is via neo4j-shell. Is there also a way to define parameters for use by the shell or can that only be done through the REST API? Thanks!
You can use parameters in the shell, just export them as "environment" variables.
List them with env:
export name=Tim
env
match (p:Person {firstName:{name}}) return p;
Yes, the URL for the CSV file is a string in the Cypher query so you can parameterize it like any other Cypher query. Check out the docs here and here.