I would like to work with Neo4j packages for java.
I see that the function newEmbeddedDatabaseBuilder is deprecated.
What is the best way to work now with Neo4j using java code?
thanks
In Neo4j 3.0, you'll use the GraphDatabaseFactory-
graphDb = new GraphDatabaseFactory().newEmbeddedDatabase( DB_PATH );
The Neo4j Java manual is available here: http://neo4j.com/docs/java-reference/current/#tutorials-java-embedded
Related
My goal is to write transformed data from a MongoDB collection into Neo4j using Spark Structured Streaming. According to the Neo4j docs, this should be possible with the "Neo4j Connector for Apache Spark" version 4.1.2.
Batch queries so far work fine. However, with the following example below, I run into an error message:
spark-shell --packages org.mongodb.spark:mongo-spark-connector:10.0.2,org.neo4j:neo4j-connector-apache-spark_2.12:4.1.2_for_spark_3
val dfTxn = spark.readStream.format("mongodb")
.option("spark.mongodb.connection.uri", "mongodb://<IP>:<PORT>")
.option("spark.mongodb.database", "test")
.option("spark.mongodb.collection", "txn")
.option("park.mongodb.read.readPreference.name","primaryPreferred")
.option("spark.mongodb.change.stream.publish.full.document.only", "true")
.option("forceDeleteTempCheckpointLocation", "true").load()
val query = dfPaymentTx.writeStream.format("org.neo4j.spark.DataSource")
.option("url", "bolt://<IP>:<PORT>")
.option("save.mode", "Append")
.option("checkpointLocation", "/tmp/checkpoint/myCheckPoint")
.option("labels", "Account")
.option("node.keys", "txn_snd").start()
This gives me the following error message:
java.lang.UnsupportedOperationException: Data source org.neo4j.spark.DataSource does not support streamed writing
Although the Connector should officially support streaming starting with version 4.x. Does anybody have an idea what I'm doing wrong?
Thanks in advance!
Incase, if the connector doesnt support streaming writes, you can try something like below.
you can leverage foreachBatch() functionality from spark structured streaming and write the data into Neo4j in batch mode.
https://spark.apache.org/docs/latest/structured-streaming-programming-guide.html#using-foreach-and-foreachbatch
def process_entry(df, id):
df.write.ToNeo4j(url=url, table="mytopic", mode="append", properties=props)
query = df.writeStream.foreachBatch(process_entry).start()
In the above code, you can have your Neo4j Writer logic and you can write the data into database using batch mode.
in couchbase SDK version 3 I can't find consistency set in the insert and find query like java
N1qlParams adhoc2 = N1qlParams.build().consistency(ScanConsistency.STATEMENT_PLUS).adhoc(true);
is that deprecated?
and if it's deprecated how can I fix this problem?
regards.
checkout options of cluster.query()
https://docs.couchbase.com/sdk-api/couchbase-node-client/Cluster.html#query
Check out the class QueryScanConsistency
cluster.query(queryString, QueryOptions.queryOptions()
.scanConsistency(QueryScanConsistency.REQUEST_PLUS) )
.rowsAs(MyEntity.class)
You should use REQUEST_PLUS instead of STATEMENT_PLUS
I have a situation where I want to import my graph data to database.I am having janusgraph(latest version) running with cassandra(version 3) and elasticsearch(version 6.6.0) using Docker.I have been suggested to use gryo format.So I have tried this command
graph.io(IoCore.gryo()).reader().create().readGraph(ToInputStream.from("my_graph.kryo"), graph);
but ended up with an error
No such property: ToInputStream for class: Script4
The documentation I am following is here.Please take a look and put me in a right procedure. Thanks in advance!
ToInputStream is not a function of Gremlin or JanusGraph. I believe that it is only a function of IBM Compose so unless you are running JanusGraph on that specific platform, this command will not work.
Versions of JanusGraph that utilize TinkerPop 3.4.x will support the io() step and this is the preferred manner in which to load gryo (as well as graphson and graphml) files.
Graph graph = ... // setup JanusGraph instance
GraphTraversalSource g = traversal().withGraph(graph); // might use withRemote() here instead depending on how you are connecting I suppose
g.io("graph.kryo").read().iterate()
Note that if you are connecting remotely - it seems you are sending scripts to the Docker instance given your error - then be sure that that "graph.kryo" file path is accessible to Docker. That's what's nice about ToInputStream from Compose as it allows you to access remote sources.
In py2neo V2, it was possible to do this by using query.append() which add the query to a queue and all the queries can be executed later.
How can I do the same thing in v3? I cannot find an equivalent function in the documentation for this.
The following is in Python 3.6.3 using py2neo version 3.1.2:
from py2neo import Graph
g = Graph(host="localhost", user="neo4j", bolt=True) # Modify for your situation
transaction = g.begin(autocommit=False)
transaction.append("match(n) return count(n);")
This fails and returns the following warning:
/usr/local/bin/ipython3:1: DeprecationWarning: Transaction.append(...) is deprecated, use Transaction.run(...) instead
Hope this helps.
Below code worked for Dataflow 1.9 sdk, migrating to 2.X
PCollection<TableRow> tableRow = ...
tableRow.apply(BigQueryIO.Write()
.to(String.format("%1$s:%2$s.%3$s",projectId, bqDataSet, bqTable))
.withSchema(schema)
.withWriteDisposition(BigQueryIO.Write.WriteDisposition.WRITE_APPEND));
I get
The method apply(PTransform<? super PCollection<TableRow>,OutputT>) in the type PCollection<TableRow> is not applicable for the arguments (BigQueryIO.Write<Object>)
Release notes are not much of a help here and documentation on 2.X is non existant redirects to beam API page.
Have you tried using BigqueryIO.writeTableRows()?
Apache Beam 2.1.0 BigqueryIO documentation
https://beam.apache.org/documentation/sdks/javadoc/2.1.0/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.html
You can try providing TableRow type explicitly (BigQuery.<TableRow>write()...) or use BigQuery.writeTableRows() as suggested above.
Looks like the interface was made generic in 2.x. Earlier version had TableRow hard coded.