Getting an unclear Error Message in Neo4j with APOC batch loader - neo4j

Question

I am loading a large number of Neo4j nodes into the system from a json file. It is failing with this error message "Failed to invoke procedure apoc.merge.node: Caused by: java.lang.NullPointerException" - I am not seeing enough information to figure out what I am doing wrong and as this is the first time i have used this, I just don't see it. This is the last 7 or so errors on the error stack. Looks like the error originates when merge_node is called.
File "F:\ClientSide\current\testload1.py", line 104, in <lambda>
nodes.apply(lambda h: merge_node(h), axis=1)
File "F:\ClientSide\current\testload1.py", line 61, in merge_node
ses.run("UNWIND $batch AS row CALL apoc.merge.node(['ProgNode', row.nodetype], {node:row.node}, apoc.map.removeKeys(properties(row), ['nodetype', 'node'])) YIELD node RETURN 1", batch=BATCH["batch"])
File "C:\Users\Bill Dickenson\AppData\Local\Programs\Python\Python37\lib\site-packages\neo4j\work\simple.py", line 217, in run
self._autoResult._run(query, parameters, self._config.database, self._config.default_access_mode, self._bookmarks, **kwparameters)
File "C:\Users\Bill Dickenson\AppData\Local\Programs\Python\Python37\lib\site-packages\neo4j\work\result.py", line 101, in _run
self._attach()
File "C:\Users\Bill Dickenson\AppData\Local\Programs\Python\Python37\lib\site-packages\neo4j\work\result.py", line 202, in _attach
self._connection.fetch_message()
File "C:\Users\Bill Dickenson\AppData\Local\Programs\Python\Python37\lib\site-packages\neo4j\io\_bolt3.py", line 326, in fetch_message
response.on_failure(summary_metadata or {})
File "C:\Users\Bill Dickenson\AppData\Local\Programs\Python\Python37\lib\site-packages\neo4j\io\_bolt3.py", line 512, in on_failure
raise Neo4jError.hydrate(**metadata)
neo4j.exceptions.ClientError: Failed to invoke procedure `apoc.merge.node`: Caused by: java.lang.NullPointerException
The "batch" data structure contains lists of variables like this one
{'EIEO': True, 'FILECOUNT': 1, 'KDM': 'data:Writes', 'changed': False, 'ctx': '113540257', 'level': 'code', 'location': [55, 8, 55, 94], 'node': 100, 'quvioDensity': 1.0, 'quviolations': 2, 'szAFP': '', 'szaep': 17, 'szlocs': 2, 'text': 'FilemavenWrapperPropertyFile=newFile(baseDirectory,MAVEN_WRAPPER_PROPERTIES_PATH);', 'type': 'localVariableDeclarationStatement'}
and the code that is processing it looks like this, including the print statement that generated the data above.
def merge_node(args):
global INNODE, NODECOUNT
"""
Function to create nodes from a batch.
"""
INNODE += 1
if (INNODE % 10000) == 0:
print("...Sent %s of %s for processing" % (INNODE, NODECOUNT))
if len(BATCH['batch']) == 4:
print(BATCH['batch'][3])
if (len(BATCH['batch']) > 1000) or (INNODE == NODECOUNT):
if INNODE == NODECOUNT:
print("...Final Record (%s) added and transmitted" % INNODE)
BATCH['batch'].append(args.to_dict())
with graphDB_Driver.session() as ses:
ses.run("UNWIND $batch AS row CALL apoc.merge.node(['ProgNode', row.nodetype], {node:row.node}, apoc.map.removeKeys(properties(row), ['nodetype', 'node'])) YIELD node RETURN 1", batch=BATCH["batch"])
reset_batch()
BATCH['batch'].append(args.to_dict())
Oddly enough, this runs locally with this error. When it runs against my remote Neo4j db, it processes fine ( no errors) but does NOT generate anything on the server. So I assume its failing up there but APOC is redirecting the console and just moving on.
Anyone see what I am doing incorrectly ?

Answer 1

The "batch" data structure in your question contains neither of the properties required by your Cypher code:
nodetype
node
You have to make sure that all the elements in the $batch list have at least those 2 properties if you want to use that Cypher code.