Building a rails application which will be used to edit documents, two persons may edit the same document concurrently but on their own branch, each can't see others' change until they are ready to push and merge it back to the master branch, and until they pull the latest changes from the master into their own branch.
what's the best NoSQL DB or solution for this rails application?
You could do all of this with the filesystem and git so I'm not sure why you'd even need a database here except for auxiliary functions. There's nothing in your requirements that would promote one DB over another.
I'd go with whatever you know best. Even a regular SQL DB would have zero trouble handling this.
Are the documents stored as plain text or encoded in a format (such that it needs to be binary in the DB) that requires the server to decoded and perform the merge itself?
The scenario you are describing it some what documented collection oriented in that you're effectively pushing the same type of metadata but potentially with different key/values. You may be interested in something like CouchDB or MongoDB. Both express a document in a JSON (BSON in MongoDB) like fashion, allowing each document to have differing keys.
Related
I would like to disable my rails application's oracle database partitioning, but :
I don't know how to tell whether my app is using database partitioning
I don't know how to find the place my app use partitioning, since I didn't write most of the application's code myself
Can I just system search the code base for the keyword 'Partition' and look for any result that has the key word partition in raw SQL statement?
How should I go about this?
Thanks!
Update:
I have 2 answers below and they seem to understand my question differently
I am confused now as well. I want to disable the partition feature of my Oracle Database (https://www.oracle.com/technetwork/database/options/partitioning/overview/index.html), does that means I cannot use the 'partition by' keyword (Oracle "Partition By" Keyword) anymore?
Partitioning is done at the schema level declaratively. Usually, one would not expect the application code to directly need anything specific to use partitioning since it is done at the data definition level. You can connect to the schema owner account and check the data dictionary views USER_PART_TABLES for partitioned tables owned by the user and USER_PART_INDEXES for the indexes.
I've started off with a single Rails application. Very simple, largely read-only, front end for a line of business application (using view backed to retrieve data) - with a few standard tables to augment the views.
I now have a need to use the same set of data in a new application (the 2 applications, whilst sharing the same data, work differently enough its not trivial to try and merge them into the same application).
I figured it would be easiest to split the models I can reuse into their own engine and have the 2 applications share the database.
Adding an API and having both applications query that is an option, but in practice I'm not sure I can provide an API that will satisfy both applications properly, as they use the data in different ways.
On that basis I figured if I gave each application a table prefix, or use different schemas to namespace them - that way each application has it's own distinct tables for stuff they don't share, but I can easily reuse the existing views without having to duplicate them.
Both options seems to work great, except I forgot about migrations for the common views and data.
So the only things I can think of are:
Since the 2 applications are kinda tightly coupled anyway, I have no migrations in my common data engine at all - any changes to the views/tables will be dealt with by the "first" application. This seems a bit nasty, since the models are now contained in a separate engine.
I dislike the idea of having migrations in the engine and then copying them into one or the other, since that's basically the same thing.
I use pivotal labs advice, but add some code to detect if it's in the "first" application, and only apply to that. If I fail to do that I end up with both applications including the engine migrations, which results in both applications trying to run the same migrations, and causing nothing but pain.
I actually split off the common data into it's own database. So application #1 uses db #1, application #2 uses db #2, and the common data is housed in db #3 and accessible by both applications. With a bit of faffing, I'm guessing I can end up with 3 dbs, 3 schema_migrations, and I can just blindly leave my migrations in the engine, and include them in both applications as per pivotal labs - my plan is to do something like this to make all this work, and have the common models setup to connect to their own DB, rather than the application DB
Stick with 1 db and multiple schemas, and somehow setup a task to run the engine migrations only, using an account locked down to it's own schema only - that way it creates it's own schema_migrations.
I kinda feel paralyzed as I'm not sure what is the least shitty option. 3 or 4 feels "best", but not great.
I think 3 is the best option. However, I'd expose the common db #3 data via an api. That is, have an application that is used to manage the shared data (HTML admin interface, and json feeds for the other apps to connect on).
Keep the shared data app really simple. Just CRUD actions on the data. So the other two apps will grab the data from the shared data app, manipulate to match local requirements, and then display it; or allow input - manipulate the input to match the shared structure, and then persist via the shared data app's API update/create actions.
As an update to this, due to how tightly coupled the 2 applications are, and other needs that came up yesterday, I've ended up pulling the migrations out of both and using active_record_migrations to manage and orchestrate the migrations for both applications.
Effectively I could've done the same by having a dummy or master application, however this feels somewhat cleaner.
However, this comes at the expense of being able to use models effectively in the migrations. For my use case this doesn't matter at all.
I am writing a simple grails app and there is no need to use DB (it is too complex for such task and on production enviroment also better use simple file and not install some database). Can I configure grails (maybe use some plug-ins) to store all data in file(s) in local directory?
You can use H2. As per the site, "H2 Database is probably the best Java in-memory database." Note, you can actually save this via locally using
dbCreate = "update"
url = "jdbc:h2:file:~/.h2" //or any other path
I know you're not looking for a full scale DB, but if you're using H2, you get all the nice parts of a DB without the management side. Its good if you're looking for simple table structures and doing simple queries, etc.
Can I use MongoDB and a PostgreSQL in one rails app? Specifically I will eventually want to use something like MongoHQ. So far I have failed to get this working in experimentation. And it concerns me that the MongoDB documentation specifically says I have to disable ActiveRecord. Any advice would be appreciated.
You don't need to disable ActiveRecord to use MongoDB. Check out Mongoid and just add the gem plus any models along side any of your existing ActiveRecord models. You should note that MongoHQ is just a hosting service for MongoDB and can be used alongside any Object Document Mapper (ODM).
For further details check http://mongoid.org/en/mongoid/docs/installation.html. Just skip the optional 'Getting Rid of Active Record' step.
On a recent client site I worked with a production system that merged MySQL and MongoDB data with a single Java app. To be honest, it was a nightmare. To join data between the two databases required complex Java data structures and lots of code, which is actually databases do best.
One use-case for a two database system is to have the pure transactional data in the SQL database, and the aggregate the data into MongoDB for reporting etc. In fact this had been the original plan at the client, but along the way the databases became interrelated for transactional data.
The system has become so difficult to maintain that is is planned to be scrapped and replaced with a MongoDB-only solution (using Meteor.js).
Postgres has excellent support for JSON documents via it's jsonb datatype, and it is fully supported under Rails 4.2, out of the box. I have also worked with this and I find it a breeze, and I would recommend this approach.
This allows an easy mix of SQL and NoSQL transactions, eg
select id, blast_results::json#>'{"BlastOutput2","report","results","search","hits"}'
from blast_caches
where id in
(select primer_left_blast_cache_id
from primer3_output_pairs where id in (185423,185422,185421,185420,185419) )
It doesn't offer the full MongoDB data manipulation features, but probably is enough for most needs.
Some useful links here:
http://nandovieira.com/using-postgresql-and-jsonb-with-ruby-on-rails
https://dockyard.com/blog/2014/05/27/avoid-rails-when-generating-json-responses-with-postgresql
There are also reports that it can outperform MongoDB on json:
http://www.slideshare.net/EnterpriseDB/the-nosql-way-in-postgres
Another option would be to move your Rails app entirely to MongoDB, and Rails has very good support for MongoDB.
I would not recommend running two databases, based on personal observations on how it can go bad.
Let's say your app enables users to create their own tables in the database to hold their own, custom data. Each table would have it's own schema. What are some good approaches?
My first stab involved dynamically creating migration files and model files bu I'd like to run this on heroku where you can't write to the filesystem.
I'm thinking eval may be the way to go to create and run the migration class and the model class. But I want to make sure the model class exists when a new process of the app is spawned. Can probably do this by storing these class definition with each user as they create new tables and then run through them all at startup. But now it's convulted enough that I may be missing something obvious.
It's probably a better idea not to generate new classes on runtime. Besides all of the security risks, each thread's startup time will be abominable if you ever get a significant number of users.
I would suggest rethinking your app design and aim at generic tables to hold the user's custom data. If you have examples of data structures that users can create we might be able to help.
Have you thought about a non-sql database for those tables? Look at CouchDB - there are several plugins on Github that integrate it with rails. Records in the database are JSON documents, with arbitrary key-value structure. May be perfect for a user-defined schema.
There is (was?) a cool Wiki project, called Informl. It was a Wiki, not just for web pages but for web applications. (Get it? It's informal because it's a Wiki, it's got forms because it is an application, and it's user-generated, thus Web 2.0, which means that according to an official UN resolution it is legally required to have a name which is missing a vwl.)
So, in other words, it was not just about user-generated content, but also user-generated structured data.
They did this by generating PostgreSQL-specific SQL at runtime to create new tables and then have ActiveRecord reload the schemas.
The code is up on RubyForge. It's based on Rails 1.2.3. I guess you could do much better than that today, especially with the upcoming extensibility interfaces in Rails 3.