I am evaluating if I should use Spring Data Neo4j 4 or directly use the native APIs that Neo4j has. Is it possible to get the full potential of Neo4j when using Spring Data Neo4j 4 or will it limit my future usage of Neo4j?
I see the benefit with that POJO simplifies the storage of objects in the database.
The recently updated content on https://graphaware.com/spring-data-neo4j may provide you with additional information to consider.
In my view, yes, SDN allows you to make use of the full potential of Neo4j. That said, for use cases where needed you can also sidestep SDN and make direct use of the underlying OGM and/or Cypher directly. In other words, when making use of SDN you also have the freedom and flexibility to use alternate options of what best suits your needs, so your usage does not need to be an "all SDN" or "no SDN" approach; you can mix and match as needed.
There are 2 "native" APIs
There is the Java API, that you can access in unmanaged extensions or when using Neo4j as embedded
Neo4j java driver (a.k.a. Bolt) - this what Neo itself promotes the most
OGM (and therefore SDN) supports both embedded and bolt, with new features of Bolt being covered shortly after being released.
There are some features of the embedded database that can't be used (at least not directly, you may use them through user-defined procedures/functions). E.g. traversals etc..
You should also consider other aspects of your use case, like performance, if your domain model matches the graph model etc..
Related
After reading through Traversal API, I liked the idea the concept of BranchSelector, Expander and Uniqueness. It is something like describing how traversal should be made. In other words, I felt it like giving declarative description of traversal to be performed. The fluent API is well suited for this purpose. However it seems that it can only be used to target neo4j whose database can be accessed through file system, that is we need to specify graph.db folder path. This essentially means we can only use it in embedded mode. Can we use traversal API to perform traversal on graph running on remote machine?
Especially I want to have the convenience of API (BranchSelector, Expander and Uniqueness) available to perform traversal.
I read we can use bolt to access embedded neo4j. However this does not seem to mean that we can use embedded neo4j from remote machine.
So it seems that their is no way to use Traversal API if I cannot have access to physical (or directory) location of graph database. Is it so?
The Traversal API can only be used if the code is colocated with the data, otherwise it would usually be awfully slow. The problem is that it's not purely declarative, since you can provide implementations of PathExpander, Evaluator, etc. instead of only using pre-defined constants.
However, there are several ways of having this code colocated:
one is indeed by using Neo4j in embedded mode, as you have noted
another one is by extending Neo4j, using either used-defined procedures / functions, or unmanaged extensions
See APOC for a collection of procedures developed by the Neo4j community, including some traversals.
What are the comparative advantages of querying a neo4j DB via
REST API
JDBC
as a Spring Data plugin
Performance will be better within Java using JDBC as opposed to a REST API. Here's a good explanation of why:
When you add complexity the code will run slower. Introducing a REST
service if it's not required will slow the execution down as the
system is doing more.
Abstracting the database is good practice. If you're worried about
speed you could look into caching the data in memory so that the
database doesn't need to be touched to handle the request.
Before optimizing performance though I'd look into what problem you're
trying to solve and the architecture you're using, I'm struggling to
think of a situation where the database options would be direct access
vs REST.
Regarding using neo4j as a plugin you can certainly do so, but I have to imagine the performance would not be as good as using JDBC.
From the book "Graph Databases" - Ian Robinson
Queries run fastest when the portions of the graph needed to satisfy
them reside in main memory (that is, in the filesystem cache and the
object cache). A single graph database instance today can hold many
billions of nodes, relationships, and properties, meaning that some
graphs will be just too big to fit into main memory.
If you add another layer to the app, this will be reflected in performance, so the bare you can consumes your data the better the performance but also the complexity and understanding of the code.
I'm currently doing some R and D regarding moving some business functionality from an Oracle RDBMS to Neo4j to reduce join complexity in the application queries. Due to the maintenance and visibility requirements for the data, I believe the stand alone server is the best option.
My thought is that within a java program I would pull the relevant data out of the Oracle tables, map it to a node object and persist it to neo4j (creating the appropriate relationships in the process).
I'm curious, with SDN over REST not being an optimal solution, what options are available for persistence. Are server plugins or unmanaged extensions the preferred method or am I overcomplicating the issue as tends to happen from time to time.
Thank you!
REST refers to a way to query the data over a network, not a way to store the data. Typically, you're going to store the data on some machine; you then have the option of either making it accessible via RESTful services with the neo4j server, or just using java applications to access the data.
I assume by SDN you're referring to spring data neo4j. Spring is a framework used for java applications, and SDN then refers to a plugin if you will for spring that allows java programmers to store models in neo4j. One could indeed use spring-data-neo4j to read data in, and then store it in Neo4J - but again this is a method of how the data gets into neo4j, it's not storage by itself.
The storage model in most cases is pretty much always the same. This link describes aspects of how storage actually happens.
Now -- to your larger business objective. In order to do this with neo4j, you're going to need to take a look at your oracle data and decide how it is best modeled as a graph. There's a big difference between an oracle RDBMS and Neo4J in terms of how the data is represented. Once you've settled on a graph design, you can then load your data into neo4j (many different options for doing that).
Will all of this "reduce join complexity in the application queries"? Well, yes, in the sense that Neo4j doesn't do joins. Will it improve the speed/performance of your application? There's just no way to tell. The answer to that depends on what your app is, what the queries are, how you model the data as a graph, and how you express the resulting queries over that graph.
I'm hoping to hear from any of you who have architected and implemented a decent sized Neo4j app (10's millions nodes/rels) - and what your recommendations are particularly w.r.t modelling and the various APIs (vanilla java/groovy Neo4j vs Spring-Data-Neo4j vs Grails GORM/Neo4j).
I'm interested if it actually pays off to add the extra OGM (object-graph-mapping) layer and associated abstractions?
Has anyone's experience been that it is best to stick to 'plain' graph-modelling with nodes+properties, relationships+properties, traversals and (e.g.) Cypher to model and store their data?
My concern is that 'forcing' a particular OGM abstraction onto a graph database will affect future flexibility in adapting/changing the domain model and/or flexibility in querying the data.
We're a Grails shop, and I have experimented with GORM/Neo4J and also with spring-data-neo4j.
The primary purpose for the dataset will be to model and query relationships amongst v.large numbers of people, their aliases, their associates and all sorts of criminal activity and history. There will be more than 50 main domain classes. There must be flexibility in the model (which will need to evolve rapidly in the early phases of the project) and in speed and flexibility of querying.
I have to confess, I'm struggling to find a compelling reason to use a OGM layer when I can use (e.g.) POJOs or POGOs, a little Groovy magic and some simple hand-rolled domain object <-> node/relationship mapping code. As far as I can tell, I think I would be happy just dealing with nodes & traversals & Cypher (aka KISS). But I would be very happy to hear others' experiences and recommendations.
Thanks for your time & thoughts,
TP
since I'm the author of the Grails Neo4j plugin, I might be biased. The main reason for creating the plugin was to apply the ease of Grails domain classes with their powerful out-of-the-box scaffolding to Neo4j for ~80% of the use cases. For the other 20% where specific requirements require stuff like traversals etc. we're using Neo4j APIs directly (traversals/cypher) and do not use the GORM API.
The current version of the Neo4j plugin suffers from a supernode issue since each domain instance is connected to a subreference node. If multiple concurrent requests (aka threads) add new domain instances there is chance to get a locking exception. I'm about to fix that either by a sub-subreference approach or by using indexing.
Cypher can also be used in the Neo4j Grails plugin.
Spring-Data-Neo4j on the other hand is a more advanced approach with finer control over mapping details, but requires usage of specific annotations. And I found no easy way to integrate that into Grails in a way scaffolding works.
We're using the predecessor version of the plugin in a productive application with ~60k users and ~10^6 rels. Due to NDA I cannot provide more details on that.
We do not use grails, but do use a hybrid plain neo4j / spring-data-neo4j solution. The reason is based on the fact that some of our domain data has a fixed schema and some doesn't. SDN takes a lot of the burden away and can be mixed with plain neo4j if the need arises.
We have classes that describe a data model, the objects for these classes we persist using SDN, with no additional tricks, we just use the basics from SDN. Then we have classes that contain the data for the model that is not known beforehand. These are stored in nodes contain special properties for describing what model type the data refers to. When neo4j 2 gets released, we will probably move that info into labels. Between these nodes there can be relations, also described by the aforementioned data model managed by sdn. We also have relations from the generic nodes to SDN nodes, which works fine, as everything ends up being the same things: nodes.
We have not encountered any issues yet using this approach. The thing we love the most is that the data of which we do not know in advanced how it will be modelled, is stored in the way you would have wanted to store data when you would have known it in advance, making the data actually match the model chosen, which is very hard to do when using any other type of (non-graph) database.
I am brand new to NOSQL databases (or any kind of database) and I need to build a graph database in Java. I have never used SpringSource before either. Will using Spring Data neo4j make the process of creating a graph database easier or will it complicate things? Should I just try to work with neo4j directly?
Thank you very much.
It depends on your use-case. SDN is a good fit when you are already working in a Spring Environment and have a rich domain model which you want to map in the graph.
SDN is a good fit in all the cases where you mostly work with a results of a few hundred or thousand POJO objects which have to interact with existing libraries, ui-layers or other application parts that deal with POJO's.
If you're not working in a Spring environment it is up to you, it adds some complexity in setup and dependencies. There are also other solutions like jo4neo or Tinkerpop Frames that work on top of Neo4j.
It is a slower than the native Neo4j API due to the indirection introduced.
For highest performance you can always fall back onto the Neo4j API.
In general the Core-API is fastest, a good thing in between is the cypher-query language which is very expressive.