All I'm trying to do is the following:
BatchInserter ins = BatchInserters.inserter("target/batchinserter-example");
where "target/batchinserter-example" is a given store directory. It compiles fine, but I get:
Exception in thread "main" java.lang.ExceptionInInitializerError
at org.neo4j.unsafe.batchinsert.BatchInserterImpl.rejectAutoUpgrade(BatchInserterImpl.java:556)
at org.neo4j.unsafe.batchinsert.BatchInserterImpl.<init>(BatchInserterImpl.java:219)
at org.neo4j.unsafe.batchinsert.BatchInserters.inserter(BatchInserters.java:94)
at org.neo4j.unsafe.batchinsert.BatchInserters.inserter(BatchInserters.java:88)
at org.neo4j.unsafe.batchinsert.BatchInserters.inserter(BatchInserters.java:63)
at org.neo4j.unsafe.batchinsert.BatchInserters.inserter(BatchInserters.java:51)
at NeoBatchInserter.main(NeoBatchInserter.java:26)
Caused by: java.lang.ArrayIndexOutOfBoundsException: 0
at org.neo4j.graphdb.factory.GraphDatabaseSettings.<clinit>(GraphDatabaseSettings.java:69)
... 7 more
I don't understand how this isn't working. I'm literally just calling a constructor that takes a String. I'm quite frustrated at the moment. If there is a better way to do this, I am welcome to trying it. All I want to do is set up a BatchInserter (from scratch) so that I can begin my work.
EDIT:
I just attempted to point it a graph.db from a test graph, and this still didn't work...
I'm also using kernel.2.1.3
EDIT 2:
Since my server is 2.1.2, I changed my jars to the 2.1.2 version, yet I still get the same error.
What version of Neo4j was the directory created with? I'm pretty sure that you cannot use batchinserter of 2.1.3 with an earlier version.
Install 2.1.3, set allow_store_upgrade=true in neo4j.properties, start the server and shut it down. It's crucial to have a clean shutdown. Then use your batchinserter code.
Related
I am trying to create several .qgs project files to be served at a later time by an instance of qgis Server.
To this end I need to start a new PyQGIS application several times upon request. The application runs smoothly the first time it is called, but if I try to run it a second time I get a Segmentation Fault error.
Here is an example code that triggers the problem:
from qgis.core import QgsApplication
import os
os.environ["QT_QPA_PLATFORM"] = "offscreen"
for i in range(2):
print(f'Iteration number {i}')
print('\tSet prefix path')
QgsApplication.setPrefixPath('/usr', False)
print('\tInstantiating application')
qgs = QgsApplication([], False)
print('\tInitializing application')
qgs.initQgis()
print('\tExiting')
qgs.exitQgis()
When executed, I get this output:
Iteration number 0
Set prefix path
Instantiating application
QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-root'
Initializing application
Exiting
Iteration number 1
Set prefix path
Instantiating application
Initializing application
Segmentation fault
Something similar happens if I enclose the content of the loop inside a function and call it multiple times. In this case the segmentation fault happens upon calling qgs.exitQgis() the second time (and any vector or raster layers added before that would be invalid).
Maybe the problem is that for some reason qgs.exitQgis() is not really cleaning up before exit?
The code is running on a Python:3.9 docker container that comes with Debian Bullseye.
Qgis has been installed following the instruction from the QGIS docs:
https://qgis.org/en/site/forusers/alldownloads.html#debian-ubuntu. QGIS version is QGIS 3.22.3-Białowieża 'Białowieża'.
To prevent an import error when loading qgis.core I had to set the environment variable PYTHONPATH = /usr/lib/python3/dist-packages/.
UPDATE: Following a suggestion of one comment found on this post:
https://gis.stackexchange.com/questions/250933/using-exitqgis-in-pyqgis,
I substituted qgs.exitQgis() with qgs.exit() and now the app can be instantiated again any number of times without crashing.
It is still not clear what causes the segmentation fault, but at least I found this workaround.
It seems like the problem was fixed in QGIS ver. 3.24 Tisler. Now qgs.exitQgis() can be called in a loop without triggering a segmentation fault.
I'm trying to migrate from SDN 4.2 to SDN 5 and OGM 3
Everything almost works perfectly except one case.
Right now in order to save the entity I have to use depth=2 instead of depth=1 like at SDN 4.2
It's hard to explain there so I have created a demo project at GitHub that reproduces this issue - https://github.com/Artgit/spring-boot-2.0.0.M4-sdn5-ogm3-saving-issue
Steps to reproduce:
If you want to use your own Neo4j instance please skip step #1 and start reading from step #2.
Run mvn docker:start -Dfile.encoding=UTF-8 in order to spin up Neo4j 3.2.5 in Docker container (Docker must be installed)
Execute test com.decisionwanted.domain.DecisionCharacteristicIT.testUpdateValue()
The test should fail with the assertion:
java.lang.AssertionError: expected:<BaseEntity [id=3, class=class com.decisionwanted.domain.model.user.User, createDate=Wed Oct 04 21:54:17 EEST 2017, updateDate=Wed Oct 04 21:54:17 EEST 2017]> but was:<BaseEntity [id=2, class=class com.decisionwanted.domain.model.user.User, createDate=Wed
As you can see from the following code:
rdbmsHorScalingValue = valueDao.update(rdbmsHorScalingValue, newStringValue2, newStringDescription2, user3,
null);
assertEquals(user3, rdbmsHorScalingValue.getUpdateUser());
rdbmsHorScalingValue = valueDao.getById(rdbmsHorScalingValue.getId());
assertEquals(user3, rdbmsHorScalingValue.getUpdateUser()); // Error here !!!!
I have updated rdbmsHorScalingValue with user3 and after getting Value by id (valueDao.getById()) I expect this user as rdbmsHorScalingValue.getUpdateUser() but for some unknown reason, it is not true.
But if we change at the following method: com.decisionwanted.domain.dao.decision.characteristic.value.history.HistoryValueDaoImpl.create(Value) saving depth from 1 to 2 - everything starts working fine.
Right now I don't know where is the issue and the only thing I know - it is working fine with saving depth = 1 with SDN 4.2.
Please tell me where is the issue(why it doesn't work with SDN 5) and how to solve it.
The issue is with you update method (com.decisionwanted.domain.dao.decision.characteristic.value.ValueDaoImpl#update)
You are changing a relationship (UPDATED_BY), which is not tracked in a current session (which is bound to a transaction). It will leave your old UPDATED_BY relationship - you end up with 2 relationships - check your graph in Neo4j directly. Behaviour for such case is undefined, OGM expects the graph model to mach the object model.
Why it works with saving depth 2 - the save will add the Value instance and the relationship to user into the session (with depth 1 it will do that only for the Value instance, not the relationship) and the subsequent change is then detected
You should load the value instance at the beginning of the update method (up to the depth you modify):
value = valueRepository.getById(value.getId());
If you use the ValueDao objects from #Transactional services you don't need that, but the *IT tests should themselves be transactional to reflect that.
I just created a new folder name-Test and started Neo4j server.
When i run the below script, i get the error - "Neo.ClientError.Statement.EntityNotFound"
and a message "Node with id 0"
start root=node(0)
create
(tatham {Name:'Tatham'}),
(tom {Name:'Tom'}),
(pat {Name:'Pat'}),
(chrissy {Name:'Chrissy'}),
(sailing {Name:'Sailing'}),
(mtb {Name:'MTB'}),
(rowing {Name:'Rowing'}),
(tennis {Name:'Tennis'}),
root-[:HAS_USER]->tatham,
root-[:HAS_USER]->tom,
root-[:HAS_USER]->pat,
root-[:HAS_USER]->chrissy,
tatham-[:FRIEND]->tom,
tom-[:FRIEND]->pat,
tatham-[:FRIEND]->chrissy,
tatham-[:LIKES]->sailing,
tatham-[:LIKES]->mtb,
tom-[:LIKES]->sailing,
pat-[:LIKES]->mtb,
tom-[:LIKES]->rowing,
pat-[:LIKES]->tennis,
chrissy-[:LIKES]->mtb,
chrissy-[:LIKES]->sailing
Can you kindly help me hot to fix this issue
A #WilliamLyon indicated:
A new DB has no nodes, and therefore has no node with the ID of 0.
The START clause is now deprecated.
You are apparently using a very old version of neo4j. If possible, you should install the latest version.
In addition:
Nodes now must always be specified within parentheses.
Try the following, instead, which should work with your version of neo4j as well as the latest versions:
CREATE
(tatham {Name:'Tatham'}),
(tom {Name:'Tom'}),
(pat {Name:'Pat'}),
(chrissy {Name:'Chrissy'}),
(sailing {Name:'Sailing'}),
(mtb {Name:'MTB'}),
(rowing {Name:'Rowing'}),
(tennis {Name:'Tennis'}),
(root)-[:HAS_USER]->(tatham),
(root)-[:HAS_USER]->(tom),
(root)-[:HAS_USER]->(pat),
(root)-[:HAS_USER]->(chrissy),
(tatham)-[:FRIEND]->(tom),
(tom)-[:FRIEND]->(pat),
(tatham)-[:FRIEND]->(chrissy),
(tatham)-[:LIKES]->(sailing),
(tatham)-[:LIKES]->(mtb),
(tom)-[:LIKES]->(sailing),
(pat)-[:LIKES]->(mtb),
(tom)-[:LIKES]->(rowing),
(pat)-[:LIKES]->(tennis),
(chrissy)-[:LIKES]->(mtb),
(chrissy)-[:LIKES]->(sailing);
The root node will be created automatically the first time it is encountered by the query, and then re-used.
The problem is the first line of your Cypher query: start root=node(0). That statement is saying "find a Node with id 0", however if you haven't inserted any data yet there is no Node to find, thus the error.
start has been deprecated and is no longer required so you can just remove it.
I'm trying to use the Java API for Neo4j but I seem to be stuck at IndexHits. If I query the DB with Cypher using
START n=node:types(type="Process") RETURN n;
I get all 2087 nodes of type "Process".
In my application I have the following lines
Index<Node> nodeIndex = db.index().forNodes("types");
IndexHits<Node> hits = nodeIndex.get("type", "Process");
System.out.println("Node index size: " + hits.size());
which leads my console to spit out a value of 0. Here, db is of course an instance of GraphDatabaseService.
I expected an object that included all 2087 nodes. What am I doing wrong?
The .size() question is just the prelude to my iterator
for(Node process : hits) { ... }
but that does not much when hits.size() == 0. According to http://api.neo4j.org/1.9.2/org/neo4j/graphdb/index/IndexHits.html this should be possible, provided there is something in hits.
Thanks in advance for your help.
I figured it out. Man, I feel so embarrassed...
It so happens that I had set up the DB_PATH to my default data folder, whereas the default storage folder is the default data folder plus graph.db. When I tried to run the code from that corrected DB_PATH I got an error saying that a lock file was in place because the Neo4j server was running. After shutting it down it worked perfectly.
So, if you happen to see the following error, just stop the server and run the code again:
Caused by: org.neo4j.kernel.StoreLockException: Could not create lock file
at org.neo4j.kernel.StoreLocker.checkLock(StoreLocker.java:74)
at org.neo4j.kernel.StoreLockerLifecycleAdapter.start(StoreLockerLifecycleAdapter.java:40)
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:491)
I found on several forums that you cannot run the Neo4j server and use the Java API to query it at the same time.
So I'm getting page load times in the range of 30-45 seconds.
Some history:
This was not always the case for this project. This project is in production so I haven't really touched the code in a while. I noticed it started happening the last time I was updating the code. I don't recall anything specific that I changed that should have anything to do with the problem. I have other projects that are running with the same Grails versions with no problem.
I think it started happening in 2.2.3. I am now running 2.2.4.
I am using x64 JDK 1.7.0_25, Windows 7 x64.
I'm not sure what else to put here that would be relevant. Any assistance is appreciated!
Edit: running with -noreloading has no effect.
Edit2: I've tried deleting my .grails folder entirely, running clean, and deleting my target folder and stacktrace log.
Edit3: It does seem that the amount of time it takes is dependent on the amount of data displayed/read. Small pages take 3-4 seconds. Medium pages 10-12 seconds...
Edit4: I'm running it via IntelliJ IDEA 12.1.4 x64 (idea64.exe). I've also tried it outside of IntelliJ with the same results.
Edit5: The database is Oracle enterprise that supports the entire company. It is managed by full time adminstrators. This isn't a MySQL server on my local machine.
Edit6: The application also functions normally when deployed in TEST (test war), but still is slow when ran with test run-app.
Starting to get somewhere:
I downloaded JDK 1.7.21 and ran the app with that and it started working no problem! I then ran clean which triggered a recompile and it stopped working... grr
Now with 1.7.21 still active, I tried -noreloading and it works!
Annnd... now it works even if I don't use -noreloading..........
I've gone back to 1.7.25.. ran clean, and it works. Sooooooo yeah... explain that.
And now it doesn't anymore.
This is under Linux but will maybe useful:
If you are running the code within an IDE:
ps auwx|grep java
-Dgrails.console.class=grails.build.logging.GrailsEclipseConsole -Dosgi.requiredJavaVersion=1.6 -Xms40m -Xmx768m -XX:MaxPermSize=256m -
As you can see the memory settings Xms and Xmx are quite low...
In your IDE there should be an INI file:
more STS.ini
1 -vm
2 /usr/lib/jvm/java-6-openjdk-amd64/bin/java
3 -startup
4 plugins/org.eclipse.equinox.launcher_1.3.0.v20120522-1813.jar
5 --launcher.library
6 plugins/org.eclipse.equinox.launcher.gtk.linux.x86_64_1.1.200.v20120913-144807
7 -product
8 org.springsource.sts.ide
9 --launcher.defaultAction
10 openFile
11 -vmargs
12 -Dgrails.console.enable.interactive=false
13 -Dgrails.console.enable.terminal=false
14 -Djline.terminal=jline.UnsupportedTerminal
15 -Dgrails.console.class=grails.build.logging.GrailsEclipseConsole
16 -Dosgi.requiredJavaVersion=1.6
17 -Xms40m
18 -Xmx768m
19 -XX:MaxPermSize=256m
You can up these value and try restarting your IDE...
I would also suggest you run something like nmon before/during and monitor whilst the code is running and monitor disk/cpu/network throughputs.
You may find you are hammering your dev box which is causing the issue.
If the production is fine I really don't see what the problem is..
E2A Ahhh forgot it was under windows so no nmon for windblows but hey not that I tried it - http://sourceforge.net/projects/jnmonanalyser/
E2A again:
1. Enable DataSource.groovy debugging:
dataSource {
pooled = true
driverClassName ="com.mysql.jdbc.Driver"
username = "aaa"
password = "aaaa"
//SQL Logging - refer to Config.groovy at hibernate.sql now
logSql=true
...
config.groovy - this will stop your app from running if you have issues with lets say records you are trying to add in your BootStrap
// Return error when it fails
//grails.gorm.failOnError=true
Enable log4j and use this or part of it:
// log4j configuration
log4j {
appender.stdout = "org.apache.log4j.ConsoleAppender"
appender.'stdout.layout'="org.apache.log4j.PatternLayout"
appender.'stdout.layout.ConversionPattern'='[%r] %c{2} %m%n'
appender.stacktraceLog = "org.apache.log4j.FileAppender"
appender.'stacktraceLog.layout'="org.apache.log4j.PatternLayout"
appender.'stacktraceLog.layout.ConversionPattern'='[%r] %c{2} %m%n'
appender.'stacktraceLog.File'="stacktrace.log"
appender.'stacktraceLog.MaxFileSize'="1MB"
rootLogger="error,stdout"
logger {
grails="error"
StackTrace="error,stacktraceLog"
org {
codehaus.groovy.grails.web.servlet="error" // controllers
codehaus.groovy.grails.web.pages="error" // GSP
codehaus.groovy.grails.web.sitemesh="error" // layouts
codehaus.groovy.grails."web.mapping.filter"="error" // URL mapping
codehaus.groovy.grails."web.mapping"="error" // URL mapping
codehaus.groovy.grails.commons="info" // core / classloading
codehaus.groovy.grails.plugins="error" // plugins
codehaus.groovy.grails.orm.hibernate="error" // hibernate integration
// Hibernate should be on - if you want to catch sql logs
springframework="off"
hibernate="on"
//hibernate.SQL = 'debug'
//hibernate.type = 'trace'
//hibernate.SQL = 'info,hibernate'
//hibernate.type = 'info,hibernate'
//hibernate = 'info,hibernate'
//apache.commons.digester.Digester = 'debug,javaclasses'
}
}
additivity.StackTrace=false
}
try and capture what it is doing, it is also worth running developer tools on your browser whether its firefox of chrome and trying to figure out on what elements it is taking that time - but between the logs and the browser developer tools should lie your answer.
Usually you can fix this by doing
grails clean
on the grails command line (I open it via CRTL+ALT+G in IntelliJ IDEA).
This erases all compiled files and will recompile your project from scratch (afaik), which usually erases errors like that. This is not a real fix for the underlying problem, but it solves the problem. Grails is highly experimental and unstable if you ask me, i have a lot of weird error that usually disappear when doing a clean. Btw i'm using 2.1.5 on Windows 7 x64, too.
Delete stacktrace file in the target folder of your project. It can
get huge. (At present mine is 48 GB).
Check if there is enough space in your C directory.
If you are hot swapping code, then page loads can get slow. So in such cases, restart the dev server (grails app).
Sometimes, requests to the server can hang, where focusing (left or right clicking on the cmd) on the command prompt seems to skip the pause. (weird)
Increasing the JVM permgen, heap spaces depending on your memory might help as well.
Try running the server using command prompt rather within an IDE.
Better use methods for actions than closures.
For a system with 3GB RAM, my environment variable setting is:
JAVA_OPTS
-Xms512m -Xmx1g
The STS.ini settings:
-startup
plugins/org.eclipse.equinox.launcher_1.2.0.v20110502.jar
--launcher.library
plugins/org.eclipse.equinox.launcher.win32.win32.x86_1.1.100.v20110502
-product
com.springsource.sts.ide
--launcher.defaultAction
openFile
--launcher.XXMaxPermSize
384M
-vmargs
-Dosgi.requiredJavaVersion=1.5
-Xmn128m
-Xms1024m
-Xmx1024m
-Xss2m
-XX:PermSize=256m
-XX:MaxPermSize=512m
-XX:MaxGCPauseMillis=10
-XX:MaxHeapFreeRatio=50
-XX:+UseConcMarkSweepGC
-XX:+CMSIncrementalMode
-XX:+CMSIncrementalPacing
-XX:+UserParallelGC
8) Maybe the problem is with the JDK and grails versions combination. There seems to be an error with OpenJDK 1.7u25 and spring loaded. Okay, you are not using OpenJDK, but try with other version anyway. Try with JDK1.7u03.
9) Try JVM with -server flag, and see if it improves runtime performance.
grails run-app -server
So the reason why this was happening:
JDK 1.7.25