I'm getting some objects from an external library and I need to store those objects in a database. Is there a way to create the tables and relationships starting from the objects, or I have to dig into them and create migrations and models by hand?
Thanks!
Roberto
Even if you could dynamically create tables on the fly like that (not saying that you can). I wouldn't want to do that. There is so much potential for error there.
I would create the migrations by hand and have the tables and fields pre-created and fill them in with rows as needed.
Note: This is a TERRIBLE hack and you'll be ironing out the bugs for years to come, but it is however pretty easy:
This relies on rails ActiveSupport, and ActiveRecord already being loaded
Say you get a random object from a third party library which has 2 instance variables - it's class might look like this:
class Animal
attr_accessor :name, :number_of_legs
end
a = SomeThirdPartyLibrary.get_animal
You can use reflection to figure out it's name, and columns:
table_name = a.class.to_s.tableize
column_names = a.instance_variables.map{ |n| n[1..-1] } # remove the #
column_types = a.instance_variables.map{ |n| a.instance_variable_get(n).class
}.map{ |c| sql_type_for_class(c) } # go write sql_type_for_class please
Then you can use ActiveRecord migrations to create your table, like this:
ActiveRecord::Migration.class_eval do
create_table table_name do |t|
column_names.zip(column_types).each do |colname, coltype|
t.column colname, coltype
end
end
end
Then you can finally declare an activerecord class which will then interface with the just-created table.
# Note we declare a module so the new classes don't conflict with the existing ones
module GeneratedClasses; end
eval "class GeneratedClasses::#{a.class} < ActiveRecord::Base; end"
Presto!
Now you can do this:
a = GeneratedClasses::Animal.new
a.update_attributes whatever
a.save
PS: Don't do this!
Apart from being awful, if your rails app restarts it will lose all concept of the Generated Classes, so you'll need to devise some mechanism of persisting those too.
I have this exact situation. I have to read data external to the application and the performance hit is so big, that I store locally. I have gone with a solution where I have, over time, developed a schema and migrations by hand, that work with the data, and allow me to persist the data to the tables. I have developed a caching scheme that works for my data and the performance has increased significantly.
All that to say, I did everything by hand and I don't regret it. I can have confidence that my database is stable and that I am not re-creating db tables on the fly. Because of that, I have no concern about the stability of my application.
Depending on what you're trying to do with the objects, you can store objects directly into the database by serializing them.
Try looking at some ORM solutions. Or store as XML.
Related
I have a relational database table that holds a product lookup. This table powers multiple systems, only one of which is a Rails app. In the Rails app, I want to use the product lookup as an ActiveRecord class with instance members with the product code - for example, the key code field is a 4-digit alphanumeric. It would be nice to be able to refer to instances by the code like this: ProductCode.01A3. I don't want to simply declare them in the Rails code, of course, because the DB is the system of record for multiple systems. Also, how would Ruby react to a non-existent product code? If ProductCode.ABCD doesn't exist, does it just silently return a nil, and I'd need nil checks everywhere? And then there's the issue of releasing a new ProductCode into production. Updating the table would require reloading the class instance variables.
Thoughts? Can this be done? Should this be done? I've searched for a library but maybe my Google-fu isn't that good.
Lookup tables are a great tool to reduce the number of database queries and I often use them in long worker processes. I wouldn't recommend using them in a webapp unless the data is often accessed rarely changes and is expensive to get.
My implementation for you problem would look like this:
class ProductCode
#product_codes = {}
class << self
attr_accessor :product_codes
end
def self.get(code)
#product_codes[code.to_s]
end
def self.cache_all
# whatever you do here
#product_codes = {'01A3' => 42}
end
end
ProductCodes.cache_all
ProductCodes.get('01A3')
Can someone give me a short introduction to doing DB migrations in Rails using Mongoid? I'm particularly interested in lazy per document migrations. By this, I mean that whenever you read a document from the database, you migrate it to its latest version and save it again.
Has anyone done this sort of thing before? I've come across mongoid_rails_migrations, but it doesn't provide any sort of documentation, and although it looks like it does this, I'm not really sure how to use it.
I should point out I'm only conceptually familiar with ActiveRecord migrations.
If you want to do the entire migration at once, then mongoid_rails_migrations will do what you need. There isn't really much to document, it duplicates the functionality of the standard ActiveRecord migration. You write your migrations, and then you use rake db:migrate to apply them and it handles figuring out which ones have and haven't been ran. I can answer further questions if there is something specific you want to know about it.
For lazy migrations, the easiest solution is to use the after_initialize callback. Check if a field matches the old data scheme, and if it does you modify it the object and update it, so for example:
class Person
include Mongoid::Document
after_initialize :migrate_data
field :name, :type => String
def migrate_data
if !self[:first_name].blank? or !self[:last_name].blank?
self.set(:name, "#{self[:first_name]} #{self[:last_name]}".strip)
self.remove_attribute(:first_name)
self.remove_attribute(:last_name)
end
end
end
The tradeoffs to keep in mind with the specific approach I gave above:
If you run a request that returns a lot of records, such as Person.all.each {|p| puts p.name} and 100 people have the old format, it will immediately run 100 set queries. You could also call self.name = "#{self.first_name} #{self.last_name}".strip instead, but that means your data will only be migrated if the record is saved.
General issues you might have is that any mass queries such as Person.where(:name => /Foo/).count will fail until all of the data is migrated. Also if you do Person.only(:name).first the migration would fail because you forgot to include the first_name and last_name fields.
Zachary Anker has explained a lot in his answer.using mongoid_rails_migrations is a good option for migration.
Here are some links with example that will be useful for you to go through and use mongoid_rails_migrations
Mongoid Migrations using the Mongo Driver
Embedding Mongoid documents and data migrations
Other then this the Readme is should be enough with this example to implement mongoid migration
I have the same need.
Here is what I came up with: https://github.com/nviennot/mongoid_lazy_migration
I would gladly appreciate some feedback
I'm writing a Rails application against a legacy database. One of the tables in this legacy database has a column named object_id. Unfortunately object_id is also an attribute of every object in Ruby, so when ActiveRecord is trying to use these objects to formulate a query it is using the Ruby defined object_id, rather than the value that is in the database.
The legacy application is immense at well over a million lines of code, so simply changing the name of the column in the database would be an option of last resort.
Questions:
1. Is there any way to make ActiveRecord/Rails use an alias or synonym for this column?
2. Is there any way in Ruby to make the object_id method behave differently, depending on who is calling it?
3. Can I simply override the behavior of the object_id method in my model (I assume this is a terrible idea, but had to ask)
Any suggestions are greatly appreciated.
I'm just kind of spitballing here, but you might try something like this:
class Legacy < ActiveRecord::Base
#... all the other stuff
#give yourself a way to access the DB version of object_id
def oid
attributes[:object_id]
end
def oid=(val)
attributes[:object_id]=val
end
#restore ruby's default #object_id implementation
def object_id
super
end
end
Check out alias_attribute http://www.railstips.org/blog/archives/2008/06/20/alias-attribute/ I believe that it does what you are looking for.
Apparently, include and select can't be used simultaneously on a Rails find query, and this has been repeatedly marked as wontfix:
http://dev.rubyonrails.org/ticket/7147
http://dev.rubyonrails.org/ticket/5371
This strikes me as very inconvenient, because the times I'd want to use include are exactly the same times I'd want to use select - when every bit of performance counts.
Is there any way to work around this and manually generate a combined include-with-select using find_by_sql, or any other method? The trouble is, I'm not aware of any way to emulate the functionality of include, where it instantiates models in memory to hold the included associated models, such that I can enter model1.associated_models and have it not hit the database again.
Have you considered creating model for database view? For example:
Create database view, with your complicated SQL query:
CREATE VIEW production_plan_items AS
SELECT * FROM [...]
INNER JOIN [...];
Create model for this view:
# app/view_model.rb
class ViewModel < ActiveRecord::Base
self.abstract_class = true
def readonly?
true
end
def before_destroy
raise ActiveRecord::ReadOnlyRecord
end
end
# app/models/logical/production_plan_item.rb
module Logical
class ProductionPlanItem < ::ViewModel
end
end
Use as always, but remember that these records are READ ONLY!
Logical::ProductionPlanItem.where( ... )
If performance still be an issue in the future, you can quite easily convert DB views to materialized views using triggers and stored procedures. This will give your application enormous speed boost, and you don't have to change even one line of Rails code.
Enterprise Rails is highly recommended reading:
http://www.amazon.com/Enterprise-Rails-Dan-Chak/dp/0596515200/ref=sr_1_1?ie=UTF8&qid=1293140116&sr=8-1
thanks for your time first...after all the searching on google, github and here, and got more confused about the big words(partition/shard/fedorate)，I figure that I have to describe the specific problem I met and ask around.
My company's databases deals with massive users and orders, so we split databases and tables in various ways, some are described below:
way database and table name shard by (maybe it's should be called partitioned by?)
YZ.X db_YZ.tb_X order serial number last three digits
YYYYMMDD. db_YYYYMMDD.tb date
YYYYMM.DD db_YYYYMM.tb_ DD date too
The basic concept is that databases and tables are seperated acording to a field(not nessissarily the primary key), and there are too many databases and too many tables, so that writing or magically generate one database.yml config for each database and one model for each table isn't possible or at least not the best solution.
I looked into drnic's magic solutions, and datafabric, and even the source code of active record, maybe I could use ERB to generate database.yml and do database connection in around filter, and maybe I could use named_scope to dynamically decide the table name for find, but update/create opertions are bounded to "self.class.quoted_table_name" so that I couldn't easily get my problem solved. And even I could generate one model for each table, because its amount is up to 30 most.
But this is just not DRY!
What I need is a clean solution like the following DSL:
class Order < ActiveRecord::Base
shard_by :order_serialno do |key|
[get_db_config_by(key), #because some or all of the databaes might share the same machine in a regular way or can be configed by a hash of regex, and it can also be a const
get_db_name_by(key),
get_tb_name_by(key),
]
end
end
Can anybody enlight me? Any help would be greatly appreciated~~~~
Case two (where only db name changes) is pretty easy to implement with DbCharmer. You need to create your own sharding method in DbCharmer, that would return a connection parameters hash based on the key.
Other two cases are not supported right away, but could be easily added to your system:
You implement sharding method that knows how to deal with database names in your sharded dabatase, this would give you an ability to do shard_for(key) calls to your model to switch db connection.
You add a method like this:
class MyModel < ActiveRecord::Base
db_magic :sharded => { :sharded_connection => :my_sharding_method }
def switch_shard(key)
set_table_name(table_for_key(key)) # switch table
shard_for(key) # switch connection
end
end
Now you could use your model like this:
MyModel.switch_shard(key).first
MyModel.switch_shard(key).count
and, considering you have shard_for(key) call results returned from the switch_shard method, you could use it like this:
m = MyModel.switch_shard(key) # Switch connection and get a connection proxy
m.first # Call any AR methods on the proxy
m.count
If you want that particular DSL, or something that matches the logic behind the legacy sharding you are going to need to dig into ActiveRecord and write a gem to give you that kind of capability. All the existing solutions that you mention were not necessarily written with your situation in mind. You may be able to bend any number of solutions to your will, but in the end you're gonna have to probably write custom code to get what you are looking for.
Sounds like, in this case, you should consider not use SQL.
If the data sets are that big and can be expressed as key/value pairs (with a little de-normalization), you should look into couchDB or other noSQL solutions.
These solutions are fast, fully scalable, and is REST based, so it is easy to grow and backup and replicate.
We all have gotten into solving all our problems with the same tool (Believe me, I try to too).
It would be much easier to switch to a noSQL solution then to rewrite activeRecord.