I'm testing the Neo 4J ETL Tool with several MySQL and MSSQL databases and so far mapping and data transfer takes a long time to the point that for some I can't get the mapping done and for none I have been able to finish the data transfer. The details below:
Target Database: Neo4j Graph database version 5.1.0
Processor: Intel(R) Xeon(R) Gold 5217 CPU # 3.00GHz 2.99 GHz (2 processors)
Ram: 64GB
Windows 10
For all tests, I see the Zulu Platform x64 Architecture running with significant CPU and Memory usage.
Test 1&2: MySQL and MSSQL database stuck on data transfer
After mapping and discarding some tables, I tested both import online import modes with the default configuration (Unwind Row Size: 1000, Transaction Batch Size: 10000): The process is stuck at
Creating nodes with label DspIexGmp
since more than 24h. The table itself has a size of 100MB, the database in total a size of 400 MB. The MSSQL database is on the same machine (localhost). The MySQL on another machine in the network.
Test 3: MSSQL database stuck on mapping
For a different MSSQL database, the mapping is stuck since 10h after the following lines:
Crawling routines
Retrieved 0 routines
Not retrieving synonyms, since this was not requested
Not retrieving sequences, since this was not requested
Server is hosted on a server in the local network
The Neo4j log gives me always the same lines over and over again during either tests mapping and data transfer:
[2022-10-28 13:15:56.009] [info] Online check request:
https://dist.neo4j.org/neo4j-desktop/win/latest.yml [2022-10-28
13:15:56.073] [info] Online check response: 200 version: 1.5.2 files
And sometimes
[2022-10-28 12:57:10.949] [info] Graph App[neo4j-browser-id]: Checking 5.0.0 for suitable desktop API version
[2022-10-28 12:57:10.952] [info] Graph App[neo4j-browser-id]: Version 5.0.0 satisfied. { desktopApiVersion: '1.4.0', packageDesktopApiVersion: '^1.4.0' }
[2022-10-28 12:57:10.956] [info] Graph App[f72f3631-b844-48c0-81ec-c2591b9c0852]: Checking 1.5.1 for suitable desktop API version
[2022-10-28 12:57:10.958] [info] Graph App[f72f3631-b844-48c0-81ec-c2591b9c0852]: Version 1.5.1 satisfied. { desktopApiVersion: '1.4.0', packageDesktopApiVersion: '^1.4.0' }
[2022-10-28 12:57:10.960] [info] Graph App[neo4j-bloom-id]: Checking 2.5.1 for suitable desktop API version
[2022-10-28 12:57:10.962] [info] Graph App[neo4j-bloom-id]: Version 2.5.1 satisfied. { desktopApiVersion: '1.4.0', packageDesktopApiVersion: '^1.4.0' }
[2022-10-28 12:57:11.069] [info] Graph App[neo4j-bloom-id]: Checking 2.5.1 for suitable desktop API version
[2022-10-28 12:57:11.072] [info] Graph App[neo4j-bloom-id]: Version 2.5.1 satisfied. { desktopApiVersion: '1.4.0', packageDesktopApiVersion: '^1.4.0' }
[2022-10-28 12:57:11.078] [warn] Failed to parse manifest file for Graph-App[neo4j-bloom]. Error: Graph-App[neo4j-bloom] does not contain manifest.json
at getManifest (C:\Users\XXX\AppData\Local\Programs\Neo4j Desktop\resources\app.asar\dist\main.prod.js:3828:15)
at async LocalProcessor.resolveConfig (C:\Users\XXX\AppData\Local\Programs\Neo4j Desktop\resources\app.asar\dist\main.prod.js:4268:30)
at async LocalProcessor.downloadUpdate (C:\Users\XXX\AppData\Local\Programs\Neo4j Desktop\resources\app.asar\dist\main.prod.js:4496:45)
[2022-10-28 12:57:11.080] [warn] update version: 2.5.1 already downloaded for neo4j-bloom
[2022-10-28 12:57:38.066] [info] Online check request: https://dist.neo4j.org/neo4j-desktop/win/latest.yml
[2022-10-28 12:57:38.135] [info] Online check response: 200 version: 1.5.2
Does anyone have any ideas what can help? I am open to alternative solutions on how to import a existing database to Neo4j but have very limited knowledge about how to do so.
Thanks in advance!
If more information is required feel free to ask, I can also send the full logs both from the Tool and the Desktop Application.
Related
We are in the process of installing a 4.x version of Neo4j community on an offline server.
We followed https://neo4j.com/docs/operations-manual/current/installation/linux/rpm/#linux-rpm-install-offline-install-download
The packages served by https://dist.neo4j.org/rpm/ doesn't seem to be signed.
Is that normal?? This seems dangerous for a product like this (I tested several versions, including lastest enterprise)
rpm -qip neo4j-enterprise-5.3.0-1.noarch.rpm
Name : neo4j-enterprise
Version : 5.3.0
Release : 1
Architecture: noarch
Install Date: (not installed)
Group : Unspecified
Size : 226422290
License : Proprietary
Signature : (none)
Source RPM : neo4j-enterprise-5.3.0-1.src.rpm
Build Date : Thu Dec 15 14:35:50 2022
Build Host : 385d2a9db634
Relocations : (not relocatable)
URL : http://neo4j.com/
Summary : Neo4j server is a database that stores data as graphs rather than tables.
Description :
Neo4j is a highly scalable, native graph database purpose-built to
leverage not only data but also its relationships.
downloaded via curl -O https://dist.neo4j.org/rpm/neo4j-enterprise-5.3.0-1.noarch.rpm then checked the package with curl -O https://dist.neo4j.org/rpm/neo4j-enterprise-5.3.0-1.noarch.rpm
expected a signed package
got an unsigned package
It depends on the provider and you if you think it is dangerous. The package does not need to be signed when you build or install it. It actually requires additional effort to sign it.
When the package is not signed, then you can be attacked using a man-in-the-middle attack.
It is the same as when you browse the web using HTTP vs. HTTPS.
I would be ok install not signed package to the developer ephemeral VM. I require signed packages for production machines.
You should ask Neo4j to sign the packages.
While running a query on Tosca Commander using Tosca DI, it returns an "Unknown" as the value where as the query returns valid data when run manually on Google Cloud Platform
Below is the link to the screenshot of the query run on Tosca
enter image description here
Now, we did raise ticket with the Tricentis support and they quote it to be JDBC Driver issue rather than Tosca DI Issue.
We are currently using:
Driver Version: Simba JDBC Driver Version 1.2.14
Database: SQL Server Management Studio v18.6
Tosca Version: 14
Any responses/solution will be highly appreciated.
Thank you!
I was running a large delete query and got an out of memory error, so the DB shutdown automatically. I restarted it, but it is still showing as 'offline' in Neo4j desktop.
Here are the log entries from the restart:
2021-08-01 23:47:03.506+0000 INFO Starting...
2021-08-01 23:47:06.804+0000 INFO ======== Neo4j 4.2.1 ========
Exception in thread "neo4j.Scheduler-1" java.lang.OutOfMemoryError: Java heap space
WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance.
2021-08-01 23:47:22.505+0000 INFO Sending metrics to CSV file at /Users/my_user/Library/Application Support/Neo4j Desktop/Application/relate-data/dbmss/dbms-########-####-####-####-##########/metrics
2021-08-01 23:47:22.524+0000 INFO Bolt enabled on localhost:7687.
2021-08-01 23:47:23.836+0000 INFO Remote interface available at http://localhost:7474/
2021-08-01 23:47:23.837+0000 INFO Started.
Similarly, when I attempt to connect from a browser it tells me that the Neo4j database is unavailable.
In the log I can see that there is a Java out of memory error. Why would this appear? Does Neo4j queue/cache incomplete queries? And how do I go about clearing it if I can't access the server?
The data is only test data, so I don't need to save it. I do need to understand if I can fix it, and how, since I am putting the product through its paces for a new project.
I was given a graph db I must open with Neo4j. Installed community version Neo4j-2.2.0-M02 on Win7 Ultimate laptop successfully. When I tried to start the server, after pointing to the graph db, I get the following error:
Starting Neo4j Server failed:
Component org.neo4j.server.database.LifecycleManagingDatabase#325ce05c was successfully initialized, but failed to start. Please see attached cause exception.
Logged in as windows local admin. Windows FWall is off. There's no antivirus SW installed on the machine. Local admin has full perms on entire root drive. When I point to the default db, Neo4j server starts normally and I can browse http://localhost:7474/ without a problem.
Any lead will be appreciated.
The aforementioned error message was the result of using a Neo4j version different to the one the graph db was originally developed. In particular, the original graph db was developed in ver 2.0.4 Community that due to my lack of knowledge I was trying to run on an installation of Neo4j ver. 2.2.0-M02 (for development only).
The code owner enlightened me to the fact and an attempt to bring up the graph db on ver. 2.0.4 Community was successful.
Thanks for your support.
Delete the folder "c:\Users\Yourname\MyDocuments\Neo4j"
I removed the plugins that were loaded from <NEO4J_HOME>/plugins as they were also loaded from under <DATABASE_HOME>/plugins
I'm using Neo4J 2.1.0 M01 64bit community edition on 64bit windows machine. So whenever i execute cypher queries it shows " disconnected from neo4j: please check if the cord is unpluged". When i searched for the same issue in the net ,they are saying to do some settings in the neo4j.properties file present in the conf folder.So what changes needs to be done to file?
My neo4j.properties content is :
# Default values for the low-level graph engine
#neostore.nodestore.db.mapped_memory=25M
#neostore.relationshipstore.db.mapped_memory=50M
#neostore.propertystore.db.mapped_memory=90M
#neostore.propertystore.db.strings.mapped_memory=130M
#neostore.propertystore.db.arrays.mapped_memory=130M
Here everything is commented out by default.
Please tell me what's needs to be done to resolve this issue?
Thanks,
Shree