I want to achieve, with a Neo4j graph a RDBMS's ability to define and enforce a known schema. We know what our graph should look like (all the edge types and node types). So we simply want to prevent someone (developer/user) from adding an edge or node type which is "invalid" i.e. not part of the defined graph schema. How can we enforce a graphs schema? Note I am not asking about how to enforce the properties of an edge or a graph but simply how to enforce that the graph is made up if a specific set of known edge and node types.
Please help
This should probably be done on the application side. Build a wrapper/API that enforces this sort of thing, and make the developers use it. Sorry for the short answer...
Most of the language drivers or frameworks listed here provide means to define a schema:
http://www.neo4j.org/drivers
For Java we developed structr (https://github.com/structr/structr) where you define your schema in Java beans. You could start f.e. with the simple Maven archetype as shown in this screencast: http://vimeo.com/53235075
Cheers
Axel
It has to happen in a layer above Neo4j. I've been building one of those layers (Restagraph), which puts a REST interface on top of it.
It's a mite less mature than Structr, but may be worth a look. I package it in a Docker image, and it's designed so you can easily define your own schema in YAML files.
Related
I want to present release data complexity which is associated with each node like at epic, userstory etc in grafana in form of charts but grafana do not support neo4j database.Is there any way Directly or indirectly to present neo4j database in grafana?
I'm having the same issues and found this question among others. From my research I cannot agree with this answer completely, so I felt I should point some things out, here.
Just to clarify: a graph database may seem structurally different from a relational or time series database, but it is possible to build Cypher queries that basically return graph data as tables with proper columns as it would be with any other supported data source. Therefore this sentence of the above mentioned answer:
So what you want to do is just not possible.
is not absolutely true, I'd say.
The actual problem is, there is no datasource plugin for Neo4j available at the moment. You would need to implement one on your own, which will be a lot of work (as far as I can see), but I suspect it to be possible. For me at least, this will be too much work to do, so I won't use any approach to read data directly from Neo4j into Grafana.
As a (possibly dirty) workaround (in my case), a service will regularly copy relevant portions of the Neo4j graph into a relational database (or a time series database, if the data model is sufficiently simple for that), which Grafana is aware of (see datasource plugins), so I can query it from there. This is basically the replication idea also given in the above mentioned answer. In this case you obviously end up with at least 2 different database systems and an additional service, which is not so insanely great, but at the moment it seems to be the quickest way to resolve the problem with the missing datasource plugin. Maybe this is applicable in your case, too.
Using neo4j's graphite metrics you can actually configure data to be sent to grafana, and from there build whichever dashboards you like.
Up until recently, graphite/grafana wasn't supported, but it is now (in the recent 3.4 series releases), along with prometheus and other options.
Update July 2021
There is a new plugin called Node Graph Panel (currently in beta) that can visualise graph structures in Grafana. A prerequisite for displaying your graph is to make sure that you have an API that exposes two data frames, one for nodes and one for edges, and that you set frame.meta.preferredVisualisationType = 'nodeGraph' on both data frames. See the Data API specification for more information.
So, one option would be to setup an API around your Neo4j instance that returns the nodes and edges according to the specifications above. Note that I haven't tried it myself (yet), but it seems like a viable solution to get Neo4j data into Grafana.
Grafana support those databases, but not Neo4j : Graphite, InfluxDB, OpenTSDB, Prometheus, Elasticsearch, CloudWatch
So what you want to do is just not possible.
You can replicate your Neo4j data inside of those database, but the datamodel is really different ... (timeseries vs graph).
If you just want to have some charts, you can use Apache Zeppeline for that.
I am thinking of using a graph DB to store IFC data. Ideally, the DB should provide a way to define all the rule types defined in the IFC schema. However, I don't think there are any such databases because some of the rule types in IFC are very complex and requires querying the DB. Others are simple, like uniqueness of GUID, existence of mandatory attributes, or data validation. Neo4j seem to have a few constraint enforcing methods:
Neo4j helps enforce data integrity with the use of constraints. Constraints can be applied to either nodes or relationships. Unique node property constraints can be created, as well as node and relationship property existence constraints.
Are there other methods that can ensure compliance of entered data with a predefined schema?
Or are there other graph DBs that are more suitable for this job?
You can achieve pretty much everything you want by creating Transaction Event handlers.
http://neo4j.com/docs/java-reference/current/javadocs/org/neo4j/graphdb/event/TransactionEventHandler.html
You can also take a look at the GraphAware Framework and all its submodules for use-cases and also the ease of creating and deploying neo4j extensions.
Depends on whether you need the schema enforced by the database itself, or whether you're OK with that being done at the application layer.
I've just gotten Restagraph to the "working prototype" level, and my next trick is Dockerising it.
It's a framework of sorts, that enables you to define a schema by creating nodes and relationships in Neo4J with specific labels, and which dynamically creates a REST API to enforce it.
It's also written in Common Lisp, so I'll understand if you wait for the Docker image :)
Assuming i am working with neo4j, the only way i can think of that would visualize my mock up data is to generate cypher code and paste it into neo4j's data browser
Is there another (better, simplier?) way one can use to create visualization without using cypher? Generating cypher code seems like a complex enough task by itself.
Writing tests is of course another way of making sure relationships are set up right, but as i am learning the system, i'd like to visually see things to make sure they are set up as expected.
This gist contains an example on how to use the Neo4J Graphviz component to generate output in Graphviz DOT notation, which is supported by a range of graph visualizing software. (And of course Graphviz itself)
(Link to the original blog post where I found the example: http://blog.neo4j.org/2012/05/graph-this-rendering-your-graph-with.html)
There is a new solution to explore the content of a Neo4j graph database using a web browser: http://linkurio.us/
It allows you to search nodes by properties, inspect nodes, expand neighborhood...
Disclamer: I'm co-founder of Linkurious and Gephi.
There are some options listed on http://www.neo4j.org/develop/visualize also.
I'm hoping to hear from any of you who have architected and implemented a decent sized Neo4j app (10's millions nodes/rels) - and what your recommendations are particularly w.r.t modelling and the various APIs (vanilla java/groovy Neo4j vs Spring-Data-Neo4j vs Grails GORM/Neo4j).
I'm interested if it actually pays off to add the extra OGM (object-graph-mapping) layer and associated abstractions?
Has anyone's experience been that it is best to stick to 'plain' graph-modelling with nodes+properties, relationships+properties, traversals and (e.g.) Cypher to model and store their data?
My concern is that 'forcing' a particular OGM abstraction onto a graph database will affect future flexibility in adapting/changing the domain model and/or flexibility in querying the data.
We're a Grails shop, and I have experimented with GORM/Neo4J and also with spring-data-neo4j.
The primary purpose for the dataset will be to model and query relationships amongst v.large numbers of people, their aliases, their associates and all sorts of criminal activity and history. There will be more than 50 main domain classes. There must be flexibility in the model (which will need to evolve rapidly in the early phases of the project) and in speed and flexibility of querying.
I have to confess, I'm struggling to find a compelling reason to use a OGM layer when I can use (e.g.) POJOs or POGOs, a little Groovy magic and some simple hand-rolled domain object <-> node/relationship mapping code. As far as I can tell, I think I would be happy just dealing with nodes & traversals & Cypher (aka KISS). But I would be very happy to hear others' experiences and recommendations.
Thanks for your time & thoughts,
TP
since I'm the author of the Grails Neo4j plugin, I might be biased. The main reason for creating the plugin was to apply the ease of Grails domain classes with their powerful out-of-the-box scaffolding to Neo4j for ~80% of the use cases. For the other 20% where specific requirements require stuff like traversals etc. we're using Neo4j APIs directly (traversals/cypher) and do not use the GORM API.
The current version of the Neo4j plugin suffers from a supernode issue since each domain instance is connected to a subreference node. If multiple concurrent requests (aka threads) add new domain instances there is chance to get a locking exception. I'm about to fix that either by a sub-subreference approach or by using indexing.
Cypher can also be used in the Neo4j Grails plugin.
Spring-Data-Neo4j on the other hand is a more advanced approach with finer control over mapping details, but requires usage of specific annotations. And I found no easy way to integrate that into Grails in a way scaffolding works.
We're using the predecessor version of the plugin in a productive application with ~60k users and ~10^6 rels. Due to NDA I cannot provide more details on that.
We do not use grails, but do use a hybrid plain neo4j / spring-data-neo4j solution. The reason is based on the fact that some of our domain data has a fixed schema and some doesn't. SDN takes a lot of the burden away and can be mixed with plain neo4j if the need arises.
We have classes that describe a data model, the objects for these classes we persist using SDN, with no additional tricks, we just use the basics from SDN. Then we have classes that contain the data for the model that is not known beforehand. These are stored in nodes contain special properties for describing what model type the data refers to. When neo4j 2 gets released, we will probably move that info into labels. Between these nodes there can be relations, also described by the aforementioned data model managed by sdn. We also have relations from the generic nodes to SDN nodes, which works fine, as everything ends up being the same things: nodes.
We have not encountered any issues yet using this approach. The thing we love the most is that the data of which we do not know in advanced how it will be modelled, is stored in the way you would have wanted to store data when you would have known it in advance, making the data actually match the model chosen, which is very hard to do when using any other type of (non-graph) database.
I'd like to build an ontology for my web application, and I thought of using Neo4j to represent the ontology (since I use Neo4j anyway).
Right now, it seems I have to write it from scratch, including the entire ontology implementation, validation, reasoning, GUI editor. Definitely too much work.
Is there a framework or library that will allow me to manage my ontology on neo4j without implementing it from scratch?
If not, what is the most reasonable alternative? (I'm using Neo4j over Node.js)
ontology implementation, validation, reasoning, GUI editor.
You can use Tinkerpop's Blueprints stack, if you want those things (except the GUI editor), whilst still using the Neo4j as the DB.
It abstracts away from the "low-level" graph databases' API and provides a unified interface. This means you can change the DB in the future, while retaining the features.
To get started you'll need the main Blueprints lib, Neo4j impl. and Sail oupl.. At this point you should be using the GraphSail class and its Connection class for semantic graph management.
Note that by using the Sail ouplementation (I didn't come up with the term..) you may use Sesame's libs like (pasting from my maven project) sesame-rio-rdfxml, sesame-rio-ntriples, sesame-repository-sail, sesame-queryparser-sparql.
I'm not working for Tinkerpop, but I really enjoy their framework.
There is a lot you can do with the Neosemantics plugin. You may want to take a look.
You can import an existing ontology and extend it easily. You get visualisation for free with Bloom but no ontology editor as such. For that, you can use Protégé. Create your onto there and then import it into neo4j with neosemantics. It's a one line thing:
CALL n10s.onto.import.fetch("...path to your onto...", "Turtle")
//or whichever serialisation format you're using