I am working on a research domain called knowledge managment and i am using neo4j.
I want to link my neo4j base with other database that requires physical data storage (PostgreSQL, MySQL...). Is this possible?
In general sure, it depends on how you want to set up the linking.
Perhaps you can detail your use-case more?
Normally people sync data between other datastores and Neo4j e.g. by triggering updates or polling.
For Postgres there is also a foreign data wrapper.
You can also use an event-sourced system, where data is written to your relational databases and relationships also to Neo4j. (also)
Related
I recently came across an application which uses NEO4j as the backend. In my experience with SQL and other Key-value based databases, I have developed an understanding(which could be refined) that other databases store data and your application derives the information while with NEO4J you store the information. This means that the logic of deriving the information is already captured in the model of NEO4J. I am not able to get my head around this because now I cannot have logic that can be composed and most importantly something that can be tested with unit tests. I can sure have component level tests using embedded neo4j but then that's not the same. Can someone please help me understand the application development philosophy/methodology with NEO4J.
...other databases store data and your application derives the information while with NEO4J you store the information.
Hmmm.... Define data and define information. Mostly it goes: Data is something that requires further processing to become information (that is, something informative - something you can derived some conclusion or insights from).
Anyhow, doubt this has anything to do with Graph databases vs relational/aggregate databases. A database, as the name suggests, stores data.
This means that the logic of deriving the information is already captured in the model of NEO4J.
I'm not sure what you mean by "the logic... is already captured". Some queries are much easier with Neo+Cypher that with say SQL; like "Find all the friends of my friends that live in Berlin", but I would hardly relate this to 'logic'.
I cannot have logic that can be composed and most importantly something that can be tested with unit tests.
What do you mean by 'logic that can be composed'? And unit tests has nothing to do with this I'm afraid - there's no logic being tested if you talk about graph vs other databases.
Can someone please help me understand the application development philosophy/methodology with NEO4J.
There's really not much to it. Neo4J is a database like any other database, only that it uses a different model from relational/aggregate databases.
To highlight two of its strengths:
No joins - That's a pain point with relational/aggregate databases, especially with complex queries. Essentially, nearly all system involve a data model that is a graph (you only need one many-to-many relationship in your data model for that), and not using a graph database is a form of dimensionality reduction. The reasons relational databases prevailed for so many years is nothing short of a set of historical coincidences.
Easier DB migrations - and that's for being a schema-less data base. You ripe the same benefits with any other schema-less database.
I strongly recommend you read the 'NOSQL Overview' appendix of the free Graph Databases. It focus on a lot of these points.
I have some normalized Master Data in PostgreSQL.
I want a graph visualization layer in Neo4j without migrating any Data to Neo4j. Kind of like a view. Lazy fetching of data at runtime.
Neo4j will not commit any changes and only meant for viewing.
Can Neo4j use something like a PostgreSQL JDBC connector and provide a visualization?
Thanks.
You could with apoc.load.jdbc and virtual nodes/relationships created from the data.
But it would be a bit involved as you need to load all tables and then connect them.
With the Neo4j-ETL tool you can do a quick (few min) one-time import to visualize.
https://neo4j.com/blog/neo4j-etl-1-2-0-release-whats-new-and-demo/
Esp. if you don't just visualize but also query you need to transfer the data anyway.
You can use ETL tool, from Neo4j.
You need to ask for an activation key via email at devrel#neo4j.com
I'm currently doing some R and D regarding moving some business functionality from an Oracle RDBMS to Neo4j to reduce join complexity in the application queries. Due to the maintenance and visibility requirements for the data, I believe the stand alone server is the best option.
My thought is that within a java program I would pull the relevant data out of the Oracle tables, map it to a node object and persist it to neo4j (creating the appropriate relationships in the process).
I'm curious, with SDN over REST not being an optimal solution, what options are available for persistence. Are server plugins or unmanaged extensions the preferred method or am I overcomplicating the issue as tends to happen from time to time.
Thank you!
REST refers to a way to query the data over a network, not a way to store the data. Typically, you're going to store the data on some machine; you then have the option of either making it accessible via RESTful services with the neo4j server, or just using java applications to access the data.
I assume by SDN you're referring to spring data neo4j. Spring is a framework used for java applications, and SDN then refers to a plugin if you will for spring that allows java programmers to store models in neo4j. One could indeed use spring-data-neo4j to read data in, and then store it in Neo4J - but again this is a method of how the data gets into neo4j, it's not storage by itself.
The storage model in most cases is pretty much always the same. This link describes aspects of how storage actually happens.
Now -- to your larger business objective. In order to do this with neo4j, you're going to need to take a look at your oracle data and decide how it is best modeled as a graph. There's a big difference between an oracle RDBMS and Neo4J in terms of how the data is represented. Once you've settled on a graph design, you can then load your data into neo4j (many different options for doing that).
Will all of this "reduce join complexity in the application queries"? Well, yes, in the sense that Neo4j doesn't do joins. Will it improve the speed/performance of your application? There's just no way to tell. The answer to that depends on what your app is, what the queries are, how you model the data as a graph, and how you express the resulting queries over that graph.
I am building an rails application which will have huge amount of user and their activities and tasks and dynamic attributes. Currently I am using postgresql but I know it is not a good choice this kind of application.
I am a confused between Graph databases and 'Nosql databases` which one to choose.
A graph database is a NoSQL database, as is a document database, column-store database, and a key/value database. Regarding which one to choose: Unfortunately that cannot be answered so simply, as a lot will depend on your specific application.
But... why choose one? Each type of NoSQL data store has its specific advantages. You can build a system based on multiple data stores, each one used to its specific advantage(s). This concept is known as polyglot persistence.
The Microsoft Patterns & Practices team published guidance around this very topic, and I'd suggest reading through it. You can also check out the book NoSQL Distilled which also goes into this topic and the specifics of each data store classification.
If you want to see an example of an app built on several data stores, check out Cloud Ninja Polyglot Persistence. Members of my team (including myself) got together and built this as a learning/training exercise.
So this is more or less an implementation question, this is the senario I have, basically we have an app which uses MySQL as it's datastore, user accounts, transactions etc, but we want to add in a robust charting feature and the data will be stored in Redis, now basically my question is:
Is it possible, and what are the best practices for integrating another datastore into an app which already depends on another one. Can I use Rack to generate the reports? etc...
I want to turn this into a sort of open discussion because I think the need for a solution like this is going to rise as we see more and more key/value stores that offer benefits far different than a RDBMS, an NoSQL stores as well. They all have their benefits but no one solution covers them all.
Thoughts?
You can have models that do not inherit ActiveRecord::Base. Add your preferred Redis client gem, do whatever config is necessary, and start writing Redis models.
I can try to reopen this topic, because should be very practical.
Have same issue with this. I want to replicate data from SQL to NoSQL. SQL used as main database storage, because data integrity, relations etc. And NoSQL as secondary database storage set for reading. In SQL you have much associations divided to much tables. Many one-to-one association saved in different tables for better readability. This associations should be saved as one document with NoSQL. It gives unbelievable speed. Only one load. Great for data exchange for API.
Do someone positive experience with replication SQL data to more consistent NoSQL documents?