I'm trying to build an app in Ruby on Rails that stores specified tweets in a database.
I think I'm going to use the Twitter gem, but I'm not sure how to go from displaying tweets to actually storing them.
Any help or resource links are greatly appreciated!
And which database would be best suited for this application?
When iterating results, you can store them in model objects, which in turn map to tables in sql databases (mysql, postgres, oracle), or documents in nosql dbs like mongodb.
Tweaking an example in the twitter gem's README:
# Find and store the 3 most recent marriage proposals to #justinbieber
Twitter::Search.new.containing("marry me").to("justinbieber").result_type("recent").per_page(3).each do |r|
MyLocalTweetModel.create!(:from_user => r.from_user, :text => #{r.text}")
end
This can be done with any database, really.
The decision over which database to use depends on many other factors, such as where your app will be hosted, what traffic you expect, how you plan to scale it... and taste is definitely one of these factors, and not an unimportant one. I hate Active Record migrations, for instance, though I like its many other niceties. Active Record is the default ORM rails uses for sql databases.
If you're new to all this, just start on a simple sqlite database with Active Record. There's more of a learning-curve to the alternatives.
Related
I want to create an website where users can create their own teaching resources, e.g. blocks of text with embedded images etc.
How should I store this in a database in rails? I've heard mongoDB is good for storing documents but I was planning on using postgresql for the user database etc. and have read that generally you shouldn't mix different types of db
I'm sure this is an obvious question but I couldn't find an answer anywhere...
Thanks,
Graham
There are several things you could do.
1. Use PostgreSQL for both the Users table and the TeachingResources table. You could simply use a content column of type text to save all the data.
2. Use PostgreSQL but use the HStore functionality to basically store a hash of objects of your choosing, this gives you more flexibility. Rails 4 will support this by default, but there is also a gem you can use.
3. Use a combination of PostgreSQL and MongoDB (or any other NoSQL solution) in your app. I don't see this as a bad solution, but it does put you outside of the "new user constraints" in Rails, so this might not be the best route to start with
4. Go NoSQL all the way. There is no reason you shouldn't be able to use MongoDB for your User model. However, you are right that this type of datastorage can not give full ACID guarantees, so be careful with product planning and know it's vulnerabilities (but also its strengths).
Can I use MongoDB and a PostgreSQL in one rails app? Specifically I will eventually want to use something like MongoHQ. So far I have failed to get this working in experimentation. And it concerns me that the MongoDB documentation specifically says I have to disable ActiveRecord. Any advice would be appreciated.
You don't need to disable ActiveRecord to use MongoDB. Check out Mongoid and just add the gem plus any models along side any of your existing ActiveRecord models. You should note that MongoHQ is just a hosting service for MongoDB and can be used alongside any Object Document Mapper (ODM).
For further details check http://mongoid.org/en/mongoid/docs/installation.html. Just skip the optional 'Getting Rid of Active Record' step.
On a recent client site I worked with a production system that merged MySQL and MongoDB data with a single Java app. To be honest, it was a nightmare. To join data between the two databases required complex Java data structures and lots of code, which is actually databases do best.
One use-case for a two database system is to have the pure transactional data in the SQL database, and the aggregate the data into MongoDB for reporting etc. In fact this had been the original plan at the client, but along the way the databases became interrelated for transactional data.
The system has become so difficult to maintain that is is planned to be scrapped and replaced with a MongoDB-only solution (using Meteor.js).
Postgres has excellent support for JSON documents via it's jsonb datatype, and it is fully supported under Rails 4.2, out of the box. I have also worked with this and I find it a breeze, and I would recommend this approach.
This allows an easy mix of SQL and NoSQL transactions, eg
select id, blast_results::json#>'{"BlastOutput2","report","results","search","hits"}'
from blast_caches
where id in
(select primer_left_blast_cache_id
from primer3_output_pairs where id in (185423,185422,185421,185420,185419) )
It doesn't offer the full MongoDB data manipulation features, but probably is enough for most needs.
Some useful links here:
http://nandovieira.com/using-postgresql-and-jsonb-with-ruby-on-rails
https://dockyard.com/blog/2014/05/27/avoid-rails-when-generating-json-responses-with-postgresql
There are also reports that it can outperform MongoDB on json:
http://www.slideshare.net/EnterpriseDB/the-nosql-way-in-postgres
Another option would be to move your Rails app entirely to MongoDB, and Rails has very good support for MongoDB.
I would not recommend running two databases, based on personal observations on how it can go bad.
I'm learning Rails, and the target of my experiments is to realize something similar to Zoho Creator, Flexlist or Mytaskhelper, i.e. an app where the user can create his own database schema and views. What's the best strategy to pursue this?
I saw something about the Entity-Attribute-Value (EAV) but I'm not sure whether it's the best strategy or if there is some support in Rails for it.
If there was any tutorial in Rails about a similar project it would be great.
Probably it's not the easiest star for learning a new language and framework, but it would be something I really plan to do since a long time.
Your best bet will be MongoDB. It is easy to learn (because the query language is JavaScript) and it provides a schema-less data store. I would create a document for each form that defines the structure of the form. Then, whenever a user submits the data, you can put the data into a generic structure and store it in a collection based on the name of the form. In MongoDB collections are like tables, but you can create them on the fly. You can also create indexes on the fly to speed searches.
The problem you are trying to solve is one of the primary use cases for document oriented databases which MongoDB is. There are several other document oriented databases out there, but in my opinion MongoDB has the best API at the moment.
Give the MongoDB Ruby tutorial a read and I am sure you will want to give it a try.
Do NOT use a relational database to do this. Creating tables on the fly will be miserable and is a security hazard, not just for your system, but for the data of your users as well. You can avoid creating tables on the fly by creating a complex schema that tracks the form structures and each field type would require its own table. Rails makes this less painful with polymorphic associations, but it definitely is not pretty.
I think it's not exactly what you want, but this http://github.com/LeonB/has_magic_columns_fork but apparently this does something similar and you may get some idea to get started.
Using a document store like mongodb or couchdb would be the best way forward, as they are schema-less.
It should be possible to generate database tables by sending DDL-statements directly to the server or by dynamical generating a migration. Then you can generate the corresponding ActiveRecord models using Class.new(ActiveRecord::Base) do ... end. In principle this should work, but it has to be done with some care. But this definitely no job for a beginner.
A second solution could be to use MongoMapper and MongoDB. My idea is to use a collection to store the rows of your table and since MongoDB is schema less you can simply add attributes.
Using EntryAttributeValue allows you to store any schema data in a set amount of tables, however the performance implications and maintenance issues this creates may very well not be worth it.
Alternately you could store your data in XML and generate an XML schema to validate against.
All "generic" solutions will have issues with foreign keys or other constraints, uless you do all of that validation in memory before storage.
I am about to create an application with Ruby on Rails and I would like to use multiple databases, basically is an accounting app that will have multiple companies for each user. I would like to create a database for each company
I found this post http://programmerassist.com/article/302
But I would like to read more thoughts about this issue.
I have to decide between MySQL and PosgreSQL, which database might fit better my problem.
There are several options for handling a multi-tenant app.
Firstly, you can add a scope to your tables (as suggested by Chad Birch - using a company_id). For most use-cases this is fine. If you are handling data that is secure/private (such as accounting information) you need to be very careful about your testing to ensure data remains private.
You can run your system using multiple databases. You can have a single app that uses a database for each client, or you can have actually have a seperate app for each client. Running a database for each client cuts a little against the grain in rails, but it is doable. Depending on the number of clients you have, and the load expectations, I would actually suggest having a look at running individual apps. With some work on your deployment setup (capistrano, chef, puppet, etc) you can make this a very streamlined process. Each client runs in a completely unique environment, and if a particular client has high loads you can spin them out to their own server.
If using PostgreSQL, you can do something similar using schemas.
PostgresQL schemas provide a very handy way of islolating your data from different clients. A database contains one or more named schemas, which in turn contain tables. You need to add some smarts to your migrations and deployments, but it works really well.
Inside your Rails application, you attach filters to the request that switch the current user's schema on or off.
Something like:
before_filter :set_app
def set_app
current_app = App.find_by_subdomain(...)
schema = current_app.schema
set_schema_path(schema)
end
def set_schema_path(schema)
connection = ActiveRecord::Base.connection
connection.execute("SET search_path TO #{schema}, #{connection.schema_search_path}")
end
def reset_schema_path
connection = ActiveRecord::Base.connection
connection.execute("SET search_path TO #{connection.schema_search_path}")
end
The problem with answers about multiple databases is when they come from people who don't have a need or experience with multiple databases. The second problem is that some databases just don't allow for switching between multiple databases, including allowing users to do their own backup and recovery and including scaling to point some users to a different data server. Here is a link to a useful video
http://aac2009.confreaks.com/06-feb-2009-14-30-writing-multi-tenant-applications-in-rails-guy-naor.html
This link will help with Ruby on Rails with Postgresql.
I currently have a multi-tenant, multi-database, multi-user (many logons to the same tenant with different levels of access), and being an online SaaS application. There are actually two applications one is in the accounting category and the other is banking. Both Apps are built on the same structure and methods. A client-user (tenant) can switch databases under that user's logon. An agent-user such as a tax accountant can switch between databases for his clients only. A super-user can switch to any database. There is one data dictionary i.e. only one place where tables and columns are defined. There is global data and local data. Global data such as a master chart-of-accounts which is available to everyone (read only). Local data is the user's database. A new user can get a clone of a master database. There are multiple clones to choose from. A super-user can maintain the clone databases.
The problem is that it is in COBOL and uses ISAM files and uses the CGI method. The problem with this is a) there is a perception that COBOL is outdated, b) getting trained people, c) price and d) online help. Otherwise it works and I'm happy with it.
So I'm researching what to replace it with and what a minefield that is.
It has past time and the decission for this has been to use PostgreSQL schemas, making multitenant applications, I have a schema called common where related data is stored.
# app/models/organisation.rb
class Organisation < ActiveRecord::Base
self.table_name = 'common.organisations'
# set relationships as usual
end
# app/models/user.rb
class User < ActiveRecord::Base
self.table_name = 'common.users'
# set relationships as usual
end
Then for migrations I have done that with this excellent tutorial. http://timnew.github.com/blog/2012/07/17/use-postgres-multiple-schema-database-in-rails/ use this, this is way better than what I saw in other places even the way Ryan Bates did on railscasts.
When a new organisation is created then a new schema is created with the name of the subdomain the organisation. I have read in the past that it's not a good idea to use different schemas but it depends on the job you are doing, this app has almost no soccial component so it's a good fit.
No, you shouldn't use multiple databases.
I'm not really sure what advice to give you though, it seems like you have some very basic misunderstandings about database design, you may want to educate yourself on the basics of databases first, before going further.
You most likely just want to add a "company id" type column to your tables to identify which company a particular record belongs to.
Our app currently spawns a new database for each client. We're starting to wonder whether we should consider refactoring this to a multi-tenant system.
What benefits / trade-offs should we be considering? What are the best practices for implementing a multi-tenant app in Rails?
I've been researching the same thing and just found this presentation to offer an interesting solution: Using Postgre's schemas (a bit like namespaces) to separate data at the DB level while keeping all tenants in the same DB and staying (mostly) transparent to rails.
Writing Multi-Tenant Applications in Rails - Guy Naor
Multi-tenant systems will introduce a whole range of issues for you. My quick thoughts are below
All SQL must be examined and
refactored to include a ClientId
value.
All Indexes must be examined to
determine if the ClientId needs to be
included
An error in a SQL statement by a
developer/sysadmin in production will
affect all of your customers.
A database corruption/problem will
affect all of your customers
You have some data privacy issues
whereby poor code/implementation could
allow customerA to see data belonging
to CustomerB
A customer using your system in a
heavy/agressive manner may affect
other customers perception of performance
Tailoring static data to an individual customers preference becomes more complex.
I'm sure there are a number of other issues but these were my initial thoughts.
It really depends upon what you're doing.
We are making a MIS program for the print industry that tracks inventory, employees, customers, equipment, and does some serious calculations to estimate costs of performing jobs based on a lot of input variables.
We are anticipating very large databases for each customer, and we currently have 170 tables. Adding another column to almost every table just to store the client_id hurts my brain.
We are currently in the beta stage of our program, and here are some things that we have encountered:
Migrations: A Rails assumption is that you will only have 1 database. You can adapt it for multiple databases, and migrations is one of them. You need a custom rake task to apply migrations to all existing databases. Be prepared to do a lot of trouble shooting because a migration may succeed on one DB, but fail on another.
Spawning Databases: How do you create a new db? From a SQL file, copying an existing db, or running all migrations? How do you keep you schema consistent between your table creation system, and your live databases?
Connecting to the appropriate database: We use a cookie to store a unique value that maps to the correct DB. We use a before filter in an Authorized controller that inheirits from ActionController that gets the db from that unique value and uses the establish_connection method on a Subclass of ActiveRecord::Base. This allows us to have some models pull from a common db and others from the client's specific db.
If you have specific questions about any of these, I can help.
I don't have any experience with this personally, but during the lightning talks at the 2009 Ruby Hoedown, Andrew Coleman presented a plugin he designed and uses for multi-tenant databases in rails w/ subdomains. You can check out the lightning talk slides and here's the acts_as_restricted_subdomain repository.
Why would you? Do you have heavy aggregation between users or are you spawning too many DBs? Have you considered using SQLite files per tenant instead of shared DB servers (since multitenant apps often are low-profile and don't need that much concurrency)?