I am currently evaluating Neo4j against Cosmos db graph .
As the present system lies in cosmos thus we started building graph in cosmos .
But in recent times came to know about certain tinkerpop3 queries which are not supported in cosmos db graph like regex, filter and other lambda operations.
Do we have a list of such supported/unsupported operations anywhere so that we are in better place to choose between the two databases without compromising on the features we wish to build.
The full list of Gremlin steps supported by CosmosDB can be found here. It is worth clarifying that TinkerPop does not support regex natively. You can only do that through a lambda expression with filter() step:
gremlin> g = TinkerFactory.createModern().traversal()
==>graphtraversalsource[tinkergraph[vertices:6 edges:6], standard]
gremlin> p = Pattern.compile("(marko|j.*h)")
==>(marko|j.*h)
gremlin> g.V().values('name').filter{p.matcher(it.get()).matches()}
==>marko
==>josh
While this works, there is no graph that I am aware of that will optimize that particular traversal (i.e. it will not be index based). In fact, no graph will optimize any lambda - it is arbitrary code that the graph will simply execute. You will need to look for graphs that natively support regex or some form of full-text search. The only ones that I know of that have such support in Gremlin itself would be JanusGraph and DSE Graph. Other graphs do have such support natively, but it isn't necessarily exposed in a way that it could be used in Gremlin directly.
TinkerPop is adding native support for text predicates now that more graphs seem to be supporting this feature and the pattern for doing so is relative consistent. We should see that in TinkerPop 3.4.0 when that is released.
I am learning neo4j. I am accessing neo4j via REST api(s) supported by the server mode. CRUD operations are implemented using neo4jOperations. For experimentation , I have benchmarked its read operations but I have found that methods : 'query' and 'queryForObjects' are taking huge execution time, although I am querying via a field which is indexed. Traversals are not complex.
I have : around 500K+ nodes, 900K+ relationships.
neo4j version : 3.0.8.
Is there any solution to improve the performance of query on neo4j in server mode?
Without looking at your actual queries and model it is hard to say why the performance would not be up to your expectations. Try to run the queries through the Neo4j browser and either EXPLAIN or PROFILE them, that may give you a hint of where the issue is.
Having said that, you really should move to version 3.2.1 and access the server over the bolt:/ protocol. That by itself should already significantly improve things.
Regards,
Tom
I'm trying to understand cypher planner, and I'm not sure of few things.
should I ever change it, or let the Cypher engine to control it?
what is the difference between the COST and the RULE planner?
Since every version of neo4j may tweak the planners, the only way to know for sure which planner works better for a specific query and a specific neo4j version would be to use PROFILE and performance testing. Also, since the plan generated by the COST planner depends on the actual characteristics of your data, you may also want to periodically test query performance with both planners even when you do not upgrade to a newer neo4j version.
This neo4j blog entry provides some details on the planners.
I'm using neo4j 2.1.2 community edition. I have loaded the CSV file which is having 2500 rows and i have created nodes and relationships among the columns. When i run the below cypher query
match (n) return count(*);
I'll get the nodes count as 17275. So when i match the nodes like match (n) return n and try to get the corresponding graph in a neo4j browser, it says
Resultset too large (over 1000 rows)
I know it's due to the nodes requested is more than 1000. So if i want to see the complete graph in neo4j browser, how can i do it?
The same query i tried in the neo4j web-admin, i wan able to get the data in tabular format but i wanted to see the data as a graph.
Also I'm not able to find neo4j-Shell in my neo4j installation bin directory. Why is that?
Thanks
Update 1
The Neo4J Web UI is built on top of D3.js using SVG: due to SVG performances in a browser when you have more than 500 nodes in a network, the user experience starts to degrade quite quickly.
Handling more than 1000 nodes adds to the technical challenge: in fact with so many nodes what happens most of the time is the "hairball" effect.
This is a blog post that might be useful (disclaimer: I am a developer for KeyLines) about visualizing big network with some design hints.
As you can imagine visualizing more than 1000 nodes is not that easy and that's why some companies such Cambridge Intelligence (KeyLines), Tom Sawyer (Perspective) or Linkourius came up with specific products for that.
You can of course build the visualization yourself for fun with open source libraries but keep in mind that it can take a very long time.
If your Neo4J project is not commercial I can suggest to have a look to Gephi to visualize it: it is a Desktop Application and it has a Neo4J adapter plugin. It can easily handle huge datasets but of course it lacks the same portability of a webapp.
In case you need ONLY a storage for your graph/data than a visualization is not required, you're right.
Original Answer
I think you might have to implement a custom visualization too see such graph in the browser, using one option of those in this page: http://www.neo4j.org/develop/visualize .
Alternatively have a look to this most extensive list here: Big data visualization using "search, show context, and expand on demand" concept
Or maybe have a different visualization approach with one of the following: Data Visualization libraries
Look at settings in neo4j browser. You can change Graph Visualization how you like. But browser can work much slower if you wanna see the complete graph.
I'm starting to develop with Neo4j using the REST API.
I saw that there are two options for performing complex queries - Cypher (Neo4j's query language) and Gremlin (the general purpose graph query/traversal language).
Here's what I want to know - is there any query or operation that can be done by using Gremlin and can't be done with Cypher? or vice versa?
Cypher seems much more clear to me than Gremlin, and in general it seems that the guys in Neo4j are going with Cypher.
But - if Cypher is limited compared to Gremlin - I would really like to know that in advance.
For general querying, Cypher is enough and is probably faster. The advantage of Gremlin over Cypher is when you get into high level traversing. In Gremlin, you can better define the exact traversal pattern (or your own algorithms) whereas in Cypher the engine tries to find the best traversing solution itself.
I personally use Cypher because of its simplicity and, to date, I have not had any situations where I had to use Gremlin (except working with Gremlin graphML import/export functions). I expect, however, that even if i would need to use Gremlin, I would do so for a specific query I would find on the net and never come back to again.
You can always learn Cypher really fast (in days) and then continue with the (longer-run) general Gremlin.
We have to traverse thousands of nodes in our queries. Cypher was slow. Neo4j team told us that implementing our algorithm directly against the Java API would be 100-200 times faster. We did so and got easily factor 60 out of it. As of now we have no single Cypher query in our system due to lack of confidence. Easy Cypher queries are easy to write in Java, complex queries won't perform. The problem is when you have multiple conditions in your query there is no way in Cypher to tell in which order to perform the traversals. So your cypher query may go wild into the graph in a wrong direction first.
I have not done much with Gremlin, but I could imagine you get much more execution control with Gremlin.
The Neo4j team's efforts on Cypher have been really impressive, and it's come a long way. The Neo team typically pushes people toward it, and as Cypher matures, Gremlin will probably get less attention. Cypher is a good long-term choice.
That said- Gremlin is a Groovy DSL. Using it through its Neo4j REST endpoint allows full, unfettered access to the underlying Neo4j Java API. It (and other script plugins in the same category) cannot be matched in terms of low-level power. Plus, you can run Cypher from within the Gremlin plugin.
Either way, there's a sane upgrade path where you learn both. I'd go with the one that gets you up and running faster. In my projects, I typically use Gremlin and then call Cypher (from within Gremlin or not) when I need tabular results or expressive pattern matching- both are a pain in the Gremlin DSL.
I initially started using Gremlin. However, at the time, the REST interface was a little unstable, so I switched to Cypher. It has much better support for Neo4j. However, there are some types of queries that are simply not possible with Cypher, or where Cypher can't quite optimize the way you can with Gremlin.
Gremlin is built over Groovy, so you can actually use it as a generic way to get Neo4j to execute 'Java' code and perform various tasks from the server, without having to take the HTTP hit from the REST interface. Among others, Gremlin will let you modify data.
However, when all I want is to query data, I go with Cypher as it is more readable and easier to maintain. Gremlin is the fallback when a limitation is reached.
Gremlin queries can be generated programmatically.
(See http://docs.sqlalchemy.org/en/rel_0_7/core/tutorial.html#intro-to-generative-selects to know what I mean.)
This seems to be a bit more tricky with Cypher.
Cypher only works for simple queries. When you start incorporating complex business logic into your graph traversals it becomes prohibitively slow or stops working altogether.
Neo4J clearly knows that Cypher isn't cutting it, because they also provide the APOC procedures which include an alternate path expander (apoc.path.expand, apoc.path.subgraphAll, etc).
Gremlin is harder to learn but it's more powerful than Cypher and APOC. You can implement any logic you can think of in Gremlin.
I really wish Neo4J shipped with a toggleable Gremlin server (from reading around, this used to be the case). You can get Gremlin running against a live Neo4J instance, but it involves jumping through a lot of hoops. My hope is that since Neo4J's competitors are allowing Gremlin as an option, Neo4J will follow suit.
Cypher is a declarative query language for querying graph databases. The term declarative is important because is a different way of programming than programming paradigms like imperative.
In a declarative query language like Cypher and SQL we tell the underlying engine what data we want to fetch and we do not specify how we want the data to be fetched.
In Cypher a user defines a sub graph of interest in the MATCH clause. Then underlying engine runs a pattern matching algorithm to search for the similar occurrences of sub graph in the graph database.
Gremlin is both declarative and imperative features. It is a graph traversal language where a user has to give explicit instructions as to how the graph is to be navigated.
The difference between these languages in this case is that in Cypher we can use a Kleene star operator to find paths between any two given nodes in a graph database. In Gremlin however we will have to explicitly define all such paths. But we can use a repeat operator in Gremlin to find multiple occurrences of such explicit paths in a graph database. However, doing iterations over explicit structures in not possible in Cypher.
If you use gremlin, then it allow you to migrate the to different graph databases,
Since most of the graph databases supports the gremlin traversal, Its good idea to chose the gremlin.
Long answer short : Use cypher for query and gremlin for traversal. You will see the response timing yourself.