Upgrading Neo4j data fails on "logs contains entries with prefix 2" - neo4j

Question

I'm running a database on Neo4j v3.5.11 CE via Docker volume on AWS. I want to upgrade to 4.4.9, so I created a tar of ./graph.db and brought it back to my dev box. I extracted to /var/lib/neo4j/data/databases. I mounted it to a neo4j v3.5.11 container and it starts fine. I can see all the data via localhost:7474.
Next I try mounting to neo4j v4.4.0 via:
docker run -d -p 7474:7474 -p 7687:7687 -v /var/lib/neo4j/data:/var/lib/neo4j/data -v /var/lib/neo4j/plugins:/plugins -v /var/lib/neo4j/logs:/var/log/neo4j -e NEO4J_AUTH=none -e NEO4J_dbms_allow__upgrade=true --name neo4j neo4j:4.0.0
Neo4j fails: "Transaction logs contains entries with prefix 2, and the highest supported prefix is 1. This indicates that the log files originates from a newer version of neo4j." This is odd because it was upgraded from 3.5.5 and has been running on 3.5.11--never touched by a newer version.
docker logs neo4j-apoc
Fetching versions.json for Plugin 'apoc' from https://neo4j-contrib.github.io/neo4j-apoc-procedures/versions.json
Installing Plugin 'apoc' from https://github.com/neo4j-contrib/neo4j-apoc-procedures/releases/download/4.0.0.7/apoc-4.0.0.7-all.jar to /plugins/apoc.jar
Applying default values for plugin apoc to neo4j.conf
Skipping dbms.security.procedures.unrestricted for plugin apoc because it is already set
Directories in use:
home: /var/lib/neo4j
config: /var/lib/neo4j/conf
logs: /logs
plugins: /plugins
import: /var/lib/neo4j/import
data: /var/lib/neo4j/data
certificates: /var/lib/neo4j/certificates
run: /var/lib/neo4j/run
Starting Neo4j.
2022-09-10 14:18:32.888+0000 WARN Unrecognized setting. No declared setting with name: apoc.export.file.enabled
2022-09-10 14:18:32.892+0000 WARN Unrecognized setting. No declared setting with name: apoc.import.file.enabled
2022-09-10 14:18:32.893+0000 WARN Unrecognized setting. No declared setting with name: apoc.import.file.use_neo4j_config
2022-09-10 14:18:32.921+0000 INFO ======== Neo4j 4.0.0 ========
2022-09-10 14:18:32.934+0000 INFO Starting...
2022-09-10 14:18:48.713+0000 ERROR Failed to start Neo4j: Starting Neo4j failed: Component 'org.neo4j.server.database.LifecycleManagingDatabaseService#123d7057' was successfully initialized, but failed to start. Please see the attached cause exception "Transaction logs contains entries with prefix 2, and the highest supported prefix is 1. This indicates that the log files originates from a newer version of neo4j.". Starting Neo4j failed: Component 'org.neo4j.server.database.LifecycleManagingDatabaseService#123d7057' was successfully initialized, but failed to start. Please see the attached cause exception "Transaction logs contains entries with prefix 2, and the highest supported prefix is 1. This indicates that the log files originates from a newer version of neo4j.".
I tried a couple things:
1.) Deleting the transaction logs: sudo rm graph.db/neostore.transaction.db.* It throws the same exact transaction log error, even though there are no transaction logs in the directory;
2.) Tried a database recovery by adding this to the run command: -e NEO4J_unsupported_dbms_tx__log_fail__on__corrupted__log__files=false This fails with "Unknown store version 'SF4.3.0'":
2022-09-10 15:39:48.458+0000 INFO Starting...
2022-09-10 15:40:34.529+0000 ERROR Failed to start Neo4j: Starting Neo4j failed: Component 'org.neo4j.server.database.LifecycleManagingDatabaseService#2a39aa2b' was successfully initialized, but failed to start. Please see the attached cause exception "Unknown store version 'SF4.3.0'". Starting Neo4j failed: Component 'org.neo4j.server.database.LifecycleManagingDatabaseService#2a39aa2b' was successfully initialized, but failed to start. Please see the attached cause exception "Unknown store version 'SF4.3.0'".
org.neo4j.server.ServerStartupException: Starting Neo4j failed: Component 'org.neo4j.server.database.LifecycleManagingDatabaseService#2a39aa2b' was successfully initialized, but failed to start. Please see the attached cause exception "Unknown store version 'SF4.3.0'".
Any ideas appreciated! Thanks!

Answer 1

Deleting transaction logs is never a good idea. What you want to do is add an environment variable:
dbms.allow_upgrade=true
Then it should work as the docs states that you can update the latest 3.5 to 4.0.0 Neo4j version.