I have created 15.4 GB of csv files that I would like to import into fresh new Neo4j graph.db.
After executing neo4j-admin import --delimiter="|" --array-delimiter="&" --nodes="processes.*" command (I have 17229 csv files, that are named "processes_someHash.csv") I get this particular output:
..../pathWithCsvFiles: neo4j-admin import --delimiter="|" --array-delimiter="&" --nodes="processes.*"
WARNING: Max 1024 open files allowed, minimum of 40000 recommended. See the Neo4j manual.
For input string: "10059167292802359779483"
usage: neo4j-admin import [--mode=csv] [--database=<name>]
[--additional-config=<config-file-path>]
[--report-file=<filename>]
[--nodes[:Label1:Label2]=<"file1,file2,...">]
[--relationships[:RELATIONSHIP_TYPE]=<"file1,file2,...">]
[--id-type=<STRING|INTEGER|ACTUAL>]
[--input-encoding=<character-set>]
[--ignore-extra-columns[=<true|false>]]
[--ignore-duplicate-nodes[=<true|false>]]
[--ignore-missing-nodes[=<true|false>]]
[--multiline-fields[=<true|false>]]
[--delimiter=<delimiter-character>]
[--array-delimiter=<array-delimiter-character>]
[--quote=<quotation-character>]
[--max-memory=<max-memory-that-importer-can-use>]
[--f=<File containing all arguments to this import>]
[--high-io=<true/false>]
usage: neo4j-admin import --mode=database [--database=<name>]
[--additional-config=<config-file-path>]
[--from=<source-directory>]
environment variables:
NEO4J_CONF Path to directory which contains neo4j.conf.
NEO4J_DEBUG Set to anything to enable debug output.
NEO4J_HOME Neo4j home directory.
HEAP_SIZE Set JVM maximum heap size during command execution.
Takes a number and a unit, for example 512m.
Import a collection of CSV files with --mode=csv (default), or a database from a
pre-3.0 installation with --mode=database.
options:
--database=<name>
Name of database. [default:graph.db]
--additional-config=<config-file-path>
Configuration file to supply additional configuration in. [default:]
--mode=<database|csv>
Import a collection of CSV files or a pre-3.0 installation. [default:csv]
--from=<source-directory>
The location of the pre-3.0 database (e.g. <neo4j-root>/data/graph.db).
[default:]
--report-file=<filename>
File in which to store the report of the csv-i
... and more of a manual
What does the For input string: "10059167292802359779483" mean?
Have you checked the headers in your CSV files? That's been a problem for me when importing previously.
Any chance your delimiter character is also appearing in data values?
well, I tested neo4j importing with more compact dataset and it worked fine (when there was problem with delimiter for example, then the error message showed me what was the specific problem). I optimized my program for creating these csv files based on this low dataset and used it to make the mentioned bigger csv files, which doesn't work.
Related
I use OTB (Orfeo Tool Box) in QGis for classification. When I use the ImageTrainClassifier tool in a batch process, I have a problem for some images. Instead of returning a model in a xml/txt file format, it returns several files with those extensions : .xml_rates_1, .xml_samples_1.dbf, .xml_samples_1.prj, .xml_samples_1.shp, .xml_samples_1.shx, .xml_stats_1 (I have the same files with txt instead of xml if I use txt file format as output).
During the execution of the algorithms, I have only one warning message :
(WARNING): file ..\Modules\Learning\Sampling\src\otbSamplingRateCalculatorList.cxx, line 99, SamplingRateCalculatorList (00000283DDC3C0E0): All classes are empty !
And after that :
(FATAL) TrainImagesClassifier: No samples found in the inputs!
The problem is that after that, I want to use ImageClassifier, that takes the model of ImageTrainClassifier in input, that I don’t have.
Thanks for your help
Am using R Commander and trying to import a data set in Excel on a pc running Windows. Commands: Data > Import Data > from Excel file. When I do so I get the following R error message:
Loading required package: splines
Loading required package: RcmdrMisc
Loading required package: car
Loading required package: carData
Loading required package: sandwich
Loading required package: effects
lattice theme set by effectsTheme()
See ?effectsTheme for details.
Rcmdr Version 2.5-1
Attaching package: 'Rcmdr'
The following object is masked from 'package:car':
Confint
Error in xlsx_sheets(path) : Evaluation error: 'exdir' does not exist.
Don’t think it’s a problem with Excel file. Have installed R Commander three times and encountered problem consistently. Doesn’t work in csv format either. Mystery given I have RC on another pc and am not having this problem with same Excel file. What is the problem and how can I fix it?
Thanks, RB
Try using the package readxl from the tidyverse to load your file.
It's tried and true and easy to use.
First install the package from CRAN by running:
install.packages("readxl")
Then import the library: library(readxl)
And finally read your excel file into an environment variable
my_spreadsheet <- read_excel("my_excel_file.xlsx")
This should work for both .xlsx files and .xls files
For more information see https://readxl.tidyverse.org/
I'm trying to import a csv file into Neo4J. I'm following this guide and I've already imported a file containing thousands of nodes (users), now I'm trying to import relationships between these nodes but I keep getting this error:
Input error: 'TYPE '
Caused by:'TYPE '
java.lang.IllegalArgumentException: 'TYPE '
at org.neo4j.csv.reader.Extractors.valueOf(Extractors.java:152)
at org.neo4j.unsafe.impl.batchimport.input.csv.DataFactories$DefaultRelationshipFileHeaderParser.entry(DataFactories.java:361)
The csv file I'm trying to import looks like this:
ID,:START_ID,:END_ID,:TYPE
1,82513,82718,FRIEND_OF
2,48635,21154,FRIEND_OF
3,92784,96648,FRIEND_OF
4,55215,86714,FRIEND_OF
.......
And in order to import it I use the following command:
neo4j-import --into ./ --nodes my_users.csv --relationships my_rels.csv
which as I said before correctly imports all the nodes.
It seems you have trailing spaces or tabs in your file?
Please remove them.
If it is all the same rel-type you can also provide it
on the command line with --relationships:FRIEND_OF.
I try to import CSV in a Neo4j Database and I have a problem.
On my desktop computer (windows 7, java 1.8.0_40-b25), the LOAD CSV works great.
But on the server (windows 2012 R2, java 1.8.0_65-b17), i have this error message "URI is not hierarchical".
I try to put the data on C:, F: ... no change.
Here's the code :
USING PERIODIC COMMIT 100
LOAD CSV WITH HEADERS FROM
"file:F:/Neo4JData/Destination.csv"
AS line
MERGE (d:Destination {`Code`: line.`Code`});
Thanks for your help.
Are you using 2.3.0 Community Edition?
try:
USING PERIODIC COMMIT 10000 LOAD CSV FROM
'file:///F:\\Neo4JData\\Destination.csv
Create an import folder in the default path of the DB and place the file there that helped me.
For example: C:\Users\XXXXY\Documents\Neo4j\default.graphdb\import and put the csv there. In the query use
USING PERIODIC COMMIT
LOAD CSV WITH HEADERS FROM "file:///customers.csv" AS row
CREATE (:Customer {companyName: row.CompanyName, customerID: row.CustomerID, fax: row.Fax, phone: row.Phone});
I had the same problem. I solved it by putting /// instead of F:/ or F:///.
So if your source is
F:/FolderOne/FolderTwo/file.csv
It becomes
///FolderOne/FolderTwo/file.csv
Remember that in order to add the file you must put file: in front of the source.
So finally
file:///FolderOne/FolderTwo/file.csv
As specified above once you try with
USING PERIODIC COMMIT
LOAD CSV WITH HEADERS FROM "file:///E:/AdventureWorks/adventureworks-neo4j/data/products.csv" as row
CREATE (:Product {productName: row.ProductName, productNumber: row.ProductNumber, productId: row.ProductID, modelName: row.ProductModelName, standardCost: row.StandardCost, listPrice: row.ListPrice});
The "URI is not hierarchical" error disappears. Then most probably you will get an error saying that it couldnt load the resource like
TransientError.Statement.ExternalResourceFailure
In order to solve the same you should find the neo4j.conf file.
Since i'm using a windows 10 machine and community edition of neo4j i could find the same in the below path.
C:\Users\{username}\AppData\Roaming\Neo4j Community Edition
Edit the conf file and comment out the line
dbms.directories.import=import
Doing the above steps enabled me to load the csv file.
The file path seems wrong, can you try with :
"file:F:///Neo4JData/Destination.csv"
Try with:
file:///F:/Neo4JData/Destination.csv
Putting the .csv into the $NEO4JHOME/default.graphdb/import directory worked. You may have to create the folder, or maybe uncommenting the dbms.directories.import=import line in conf might do it. I dunno, did it the hard way :)
However, I found that I still had to include the drive specifier, i.e., file:///c:\csv2import.csv even if it wasn't in the root but in the import directorry
Use the path 'file:///F:/Neo4JData/Destination.csv' and add the Destination.csv file to the neo4jDB\import directory.If the import directory is not there create a new directory named as import and add the file.
Create a folder import under default.graphdb and place your csv files there. Later, you can use file:///fileName.csv in your LOAD CSV query
I was getting this problem as well so what I did was to copy the folder in which the .csv file was present to the following location:
C:\Users\Username\AppData\Roaming\Neo4j Desktop\Application\neo4jDatabases\database-27badd10-8989-482d-871b-cad746091f07\installation-3.3.3\import\
it seems when neo4j is installed and we try to import data from files by giving "file:///c://........" then neo4j starts looking for that file in the location C:\Users\Username\AppData\Roaming\Neo4j Desktop\Application\neo4jDatabases\database-27badd10-8989-482d-871b-cad746091f07\installation-3.3.3\import\
i.e the import folder. So we have to copy all the files needed to be imported by this this type of statement in that import folder.
This one worked at my end in windows
LOAD CSV FROM 'File:///order-details.csv' AS row
RETURN count(row);
I'm just getting started with Neo4j, and have been trying to create my first project in Neo4j Community with a small sample data from a CSV. I keep getting an invalid input/syntax error (see image below).
The problem could be several places:
I may not have set up my project correctly
I may not have the file in the right place
I may not be using the syntax correctly
Here is the Cypher I've been using to try to load the file:
LOAD CSV WITH HEADERS FROM 'C:\Users\Diana\Documents\Nattosphere\Natto_Sample.csv' AS line
CREATE (n: Natto_Variety{Product_UID: line.Product_UID, Product_Manufacturer: line.Product_Manufacturer, Product_Weight_g: line.Product_Weight_g, Product_Flavoring: line.Product_Flavoring})
I've tried several approaches, and created a simplified file, but am getting the same error each time:
Invalid input 's': expected org$neo4j$cypher$internal$compiler$v2_2$parser$Strings$$HexDigit (line 1, column 33 (offset: 32))
"LOAD CSV WITH HEADERS FROM 'C:\Users\Diana\Documents\Nattosphere\Natto_Sample.csv' AS line"
At the bottom of the GUI, another error reads:
"Neo.ClientError.Statement.InvalidSyntax"
Any idea what might be happening here?
-D
I think you have to do:
LOAD CSV WITH HEADERS FROM 'file:C:/Users/Diana/Documents/Nattosphere/Natto_Sample.csv'
If that doesn't work try:
LOAD CSV WITH HEADERS FROM 'file://C:\Users\Diana\Documents\Nattosphere\Natto_Sample.csv'
See: http://neo4j.com/developer/guide-import-csv/
Frist, you have to set true in setting dbms.security.allow_csv_import_from_file_urls
Second, you must to set where neo4J will search the csv in dbms.directories.import
When you setting those items, you have to copy you csv file into the folder where we set the dbms.directories.import
Later in Cypher:
LOAD CSV FROM 'file:///Natto_Sample.csv'
*if you hace folder into folder... use this character / to route your URL
The Syntaxe to load the csv with headers as described In the CSV Import Guide is :
LOAD CSV WITH HEADERS FROM "file-url" AS line
where the file-url , for local files, is :
file:///data.csv
Put the csv file in the import directory,and that should works
You don't need to write the C: reference:
LOAD CSV WITH HEADERS FROM 'file:/Users/Diana/Documents/Nattosphere/Natto_Sample.csv'