Short question
Does Neo4j support constraints based on a domain-model?
Explanation
In the basic tutorial, it says "Please keep this picture at hand all the time. It details the domain-model for this tutorial." (https://stack.versal.com/api2/assets/fdc05cea-e18b-44ea-8ba9-e119d7a8f872).
But is there any way to check that data stored into the graph respect this domain-model?
For relational databases, we have "create" instructions to build the domain-model and "insert" instructions to store data in compliance with this domain-model.
For graph database in Neo4j, I only found "create" instructions where we can specify a type (that would be part of the domain-model).
What I need to do
I need to create a domain-model that prevent the creation of nodes which are not compliants with the domain-modeln for example:
the node type must be in the domain-model
a type of association can only link nodes with specific types
Example
With the movie domain-model coming from the tutorial (https://stack.versal.com/api2/assets/fdc05cea-e18b-44ea-8ba9-e119d7a8f872) :
A node can only be of type Person or Movie
A Movie can't have outgoing edges
A DIRECTED or ACTED_IN relationship can't link two Persons
...
Is this possible in Neo4j?
Or do I have to create checkers on the model?
You will have to create checkers of the model or an API that guarantees that only nodes matching the model are added.
Some things that you describe will be added in Neo4j in the future but it has not been decided when.
But I saw a presentation of the http://structr.org application framework today that allows you to model a schema with types, properties and relationships with cardinalities.
Related
I know how this would be accomplished in SQL but having difficulty wrapping my brain around how to do this in cypher..
Basically working on a master data setup where a user has a master_id (node) and need to use an existing relationship property to determine the master_id in order to create a new relationship between the master_id node and a location node.
Currently have master users created as nodes that contain a master_id property. A relationship is created between the master user and a brand, and the relationship has a property of brand_user_id.
I now have another file I need to import that contains data at the brand_user level, but need to create the relationship between the master_id and a location node. In order to do this because the file does not contain the master_id property, I am attempting to use the new file to lookup master_id's based on the existing relationship with the brand, then use that master_id to create the new relationship with the location.
Have this relationship:
(m:Master{master_id:12345})-[:IS_BRAND_USER{brand_user_id:9876}]->(b:Brand{name:"Acme"})
Have this file:
brand_user_id,location_id
9876,6
Need this relationship:
(m:Master{master_id:12345})-[:HAS_LOCATION]->(l:Location{id:6})
My approach:
LOAD CSV WITH HEADERS FROM "file:///brand_user_ids.csv" as buid
MATCH (m:Master)-[r:IS_BRAND_USER{brand_user_id:buid.id}]->(b:Brand)
WITH m, buid.location_id AS location_id
MATCH (l:Location {id: location_id})
CREATE (m)-[:HAS_LOCATION {source: 'abcdef'}]->(l)
Seems to run for an extremely long time and not seeing any real progress after an hour so I'm wondering if this is fundamentally the correct approach or not, or if I have inadvertently created some horrific cross join equivalent.
The problem is that you are trying to enter a graph on a relationship. And that always requires lots of "scanning the graph".
Now, I'm not a specialist in your domain, but you might be missing a type of nodes here ... BrandUser. And there could be several reasons for that :
Based on your data a Master can have many BrandUser id's. Potentially even more than one per Brand ? Do you have other properties that make sense on the BrandUser level ?
That Location data is strange. Wouldn't you agree it's actually the BrandUser that has a location and that a Master may have many locations ?
The most important reason is however ... if you're going to enter the graph on the brand_user_id all the time (and judging from the location example that may be the case) ... you've got the reason to turn it into a node right there.
So ... it's a modeling issue really.
Hope this helps.
Regards,
Tom
I'm totally new to Neo4j and I'm testing it in these days.
One issue I have with it is how to correctly implement a relationship which involves 3 different nodes using Spring Data. Suppose, for example, that I have 3 #NodeEntitys: User, Tag and TaggableObject.
As you can argue, a User can add a Tag to a TaggableObject; I model this operation with a #RelationshipEntity TaggingOperation.
However, I can't find a simple way to glue the 3 entities inside the relationship. I mean, the obvious choice is to set #StartNode User tagger and #EndNode TaggedObject taggedObject; but how can I also add the Tag to the relationship?
This is called a "hyperedge", I believe, and it's not something that Neo4j supports directly. You can create an additional node to support it, tough. So you could have a TagEvent node with a schema like so:
(:User)-[:PERFORMED]->(:TagEvent)
(:Tag)<-[:USED]-(:TagEvent)
(:TagObject)<-[:TAGGED]-(:TagEvent)
Another alternative is to store a foreign key as a property on a relationship or a node. Obviously that's not very graphy, but if you just need it for reference that might not be a bad solution. Just remember to not use the internal Neo4j ID as in future versions that may not be dependable. You should create your own ID for this purpose.
I'm exploring potential use cases for neo4j, and I find that the relationship model is great, but I'm curious if the database can support something along the lines of a business transaction log.
For instance, a video rental store:
Customer A rents Video A on 01/01/2014
Customer A returns Video A on 01/20/2014
Customer B rents Video A on 01/25/2014
Customer B returns video A on 02/15/2014
Customer C rents Video A on 03/10/2014
etc...
The business requirement would be to track all rental transaction relationships relating to the Video A node.
This seems to be technically possible. Would one create a new relationship for every time that a new rental occurs? Are there better ways to approach this? Is this a misuse of the technology?
Nice! This is the exact use case that led me to develop FlockData (github link). FD uses Neo4J to track event type activity against a domain document (Rental in your example). Then use Tags to create Nodes that represent Meta Data associated with the domain doc (Movie/Person). You have an event node for each change in state of the Rental. Couple of graphs over here on LinkedIn showing "User Created", "User Approved" and "User Audited".
FD uses 3 databases to achieve its goals - Neo4j for the network of relationships, KV store for the bulky data (Redis or Riak) and ElasticSearch to let users find their Business Context Document (the Rental) via free text.
In terms of your specific question exercise caution with nodes that have a lot of relationships. Checkout this article on modelling dates. Peter Neubauer has a similar article somewhere in the Neo4j docs.
I'd look at it depending on what you're trying to get out of it. If you're looking to develop a recommendation engine, or see the relationships between users and/or movies, a graphDB is a pretty natural solution. If you're looking at tracking the state changes of Video A over time, a Temporal database is modeled for that (http://en.wikipedia.org/wiki/Temporal_database). For a straight up transactional system, a traditional relational database will work easily. Personally, I think you'll have better options with a graphDB. In your example, you would have 3 consumer nodes, 1 video node, 3 relationships of type :RENTS and two of :RETURNS. You'd want to make sure that your property model supports the same user re-renting the same movie (store the date in an array, not a single value). Just some thoughts...
I can't seem to find any discussion on this. I had been imagining a database that was schemaless and node based and heirarchical, and one day I decided it was too common sense to not exist, so I started searching around and neo4j is about 95% of what I imagined.
What I didn't imagine was the concept of relationships. I don't understand why they are necessary. They seem to add a ton of complexity to all topics centered around graph databases, but I don't quite understand what the benefit is. Relationships seem to be almost exactly like nodes, except more limited.
To explain what I'm thinking, I was imagining starting a company, so I create myself as my first nodes:
create (u:User { u.name:"mindreader"});
create (c:Company { c.name:"mindreader Corp"});
One day I get a customer, so I put his company into my db.
create (c:Company { c.name:"Customer Company"});
create (u:User { u.name:"Customer Employee1" });
create (u:User { u.name:"Customer Employee2"});
I decide to link users to their customers
match (u:User) where u.name =~ "Customer.*"
match (c:Company) where c.name =~ "Customer.*
create (u)-[:Employee]->(c);
match (u:User where name = "mindreader"
match (c:Company) where name =~ "mindreader.*"
create (u)-[:Employee]->(c);
Then I hire some people:
match (c:Company) where c.name =~ "mindreader.*"
create (u:User { name:"Employee1"})-[:Employee]->(c)
create (u:User { name:"Employee2"})-[:Employee]->(c);
One day hr says they need to know when I hired employees. Okay:
match (c:Company)<-[r:Employee]-(u:User)
where name =~ "mindreader.*" and u.name =~ "Employee.*"
set r.hiredate = '2013-01-01';
Then hr comes back and says hey, we need to know which person in the company recruited a new employee so that they can get a cash reward for it.
Well now what I need is for a relationship to point to a user but that isn't allowed (:Hired_By relationship between :Employee relationship and a User). We could have an extra relationship :Hired_By, but if the :Employee relationship is ever deleted, the hired_by will remain unless someone remembers to delete it.
What I could have done in neo4j was just have a
(u:User)-[:hiring_info]->(hire_info:HiringInfo)-[:hired_by]->(u:User)
In which case the relationships only confer minimal information, the name.
What I originally envisioned was that there would be nodes, and then each property of a node could be a datatype or it could be a pointer to another node. In my case, a user record would end up looking like:
User {
name: "Employee1"
hiring_info: {
hire_date: "2013-01-01"
hired_by: u:User # -> would point to a user
}
}
Essentially it is still a graph. Nodes point to each other. The name of the relationship is just a field in the origin node. To query it you would just go
match (u:User) where ... return u.name, u.hiring_info.hiring_date, u.hiring_info.hired_by.name
If you needed a one to many relationship of the same type, you would just have a collection of pointers to nodes. If you referenced a collection in return, you'd get essentially a join. If you delete hiring_info, it would delete the pointer. References to other nodes would not have to be a disorganized list at the toplevel of a node. Furthermore when I query each user I will know all of the info about a user without both querying for the user itself and also all of its relationships. I would know his name and the fact that he hired someone in the same query. From the database backend, I'm not sure much would change.
I see quite a few questions from people asking whether they should use nodes or relationships to model this or that, and occasionally people asking for a relationship between relationships. It feels like the XML problem where you are wondering if a pieces of information should be its own tag or just a property its parent tag.
The query engine goes to great pains to handle relationships, so there must be some huge advantage to having them, but I can't quite see it.
Different databases are for different things. You seem to be looking for a noSQL database.
This is an extremely wide topic area that you've reached into, so I'll give you the short of it. There's a spectrum of database schemas, each of which have different use cases.
NoSQL aka Non-relational Databases:
Every object is a single document. You can have references to other documents, but any additional traversal means you're making another query. Times when you don't have relationships between your data very often, and are usually just going to want to query once and have a large amount of flexibly-stored data as the document that is returnedNote: These are not "nodes". Node have a very specific definition and implies that there are edges.)
SQL aka Relational Databases:
This is table land, this is where foreign keys and one-to-many relationships come into play. Here you have strict schemas and very fast queries. This is honestly what you should use for your user example. Small amounts of data where the relationships between things are shallow (You don't have to follow a relationship more than 1-2 times to get to the relevant entry) are where these excel.
Graph Database:
Use this when relationships are key to what you're trying to do. The most common example of a graph is something like a social graph where you're connecting different users together and need to follow relationships for many steps. (Figure out if two people are connected within a depth for 4 for instance)
Relationships exist in graph databases because that is the entire concept of a graph database. It doesn't really fit your application, but to be fair you could just keep more in the node part of your database. In general the whole idea of a database is something that lets you query a LOT of data very quickly. Depending on the intrinsic structure of your data there are different ways that that makes sense. Hence the different kinds of databases.
In strongly connected graphs, Neo4j is 1000x faster on 1000x the data than a SQL database. NoSQL would probably never be able to perform in a strongly connected graph scenario.
Take a look at what we're building right now: http://vimeo.com/81206025
Update: In reaction to mindreader's comment, we added the related properties to the picture:
RDBM systems are tabular and put more information in the tables than the relationships. Graph databases put more information in relationships. In the end, you can accomplish much the same goals.
However, putting more information in relationships can make queries smaller and faster.
Here's an example:
Graph databases are also good at storing human-readable knowledge representations, being edge (relationship) centric. RDF takes it one step further were all information is stored as edges rather than nodes. This is ideal for working with predicate logic, propositional calculus, and triples.
Maybe the right answer is an object database.
Objectivity/DB, which now supports a full suite of graph database capabilities, allows you to design complex schema with one-to-one, one-to-many, many-to-one, and many-to-many reference attributes. It has the semantics to view objects as graph nodes and edges. An edge can be just the reference attribute from one node to another or an edge can exist as an edge object that sits between two nodes.
An edge object can have any number of attribute and can have references off to other objects, as shown in the diagram below.
Being able to "hang" complex objects off of an edge allows Objectivity/DB to support weighted queries where the edge-weight can be calculated using a user-defined weight calculator operator. The weight calculator operator can build the weight from a static attribute on the edge or build the weight by digging down through the objects connected to the edge. In the picture, above, we could create a edge-weight calculator that computes the sum of the CallDetail lengths connected to the Call edge.
I am playing around with Neo4j but trying to get my head around the graph concepts. As a learning process I want to port a small Postgres relational database schema to Neo4j. Is there any way I can port it and issues "equivalent" relational queries to Neo4j?
Yes, you can port your existing schema to a graph database. Keep in mind that this is not necessarily the best model for your data, but it is a starting point.
How easy it is depends a lot on the quality of your existing schema.
The tables corresponding to entities in an entity-relationship-diagram define your types of nodes. In the upcoming neo4j 2.0, you can labels them with the name of the entity to make a lookup easier. In older versions you can use an index or a manual label property.
Assuming a best case, where all your relationships between data is modelled using foreign keys, any 1:1 relationship between nodes can be identified and ported next.
For tables modelling n:m relationships, identify the corresponding nodes and add a direct relationship between them.
So as an example assume tables Author[id, name, publisher foreign key], Publisher[id, name] and Book[id, title] and written_by[author foreign key, book foreign key].
Every row in Author, Publisher and Book becomes a node.
Every Author node gets a relationship to the publisher identified by the foreign key relationship.
For every row in written_by you add a relationship between the Author node and Book node referenced
For queries in neo4j I recommend cypher due to its expressiveness. A (2.0) query looking for books by some author would look like:
MATCH (author:Author)-[:written_by]-(book:Book)
WHERE author.name='Hugh Laurie'
RETURN book.title
You actually have several options at hand:
use the Talend connector for Neo4J
export your schema+data in CSV files consumable by the batch importer
or you can do it programmatically
I'm afraid not. The relational data model and the graph data model are two different ways of modelling a real-world domain. It requires a human brain (at least as of 2013) to understand the domain in order to model it.
I suggest that you take a piece of paper and capture, using circles and arrows, what your entities are (nodes) and how they relate to each other (relationships). Then, have a look at that piece of paper. Voila, your new Neo4j data model.
Then, take a query that you want to be answered and try to figure out how you would do that without a computer, just by tracing your nodes and relationships with a finger on that piece of paper. Once you've figured that out, translate what you've done to a Cypher query.
Have a look at neo4j.org, there are plenty of examples.
Check this out:
The musicbrainz -> neo4j
https://github.com/redapple/sql2graph/tree/master/examples/musicbrainz
Neo4j Sql-importer
https://github.com/peterneubauer/sql-import
Good Luck!
This tool does exactly that.
Import any relational db into neo4j
https://github.com/jexp/neo4j-rdbms-import