I've recently started looking into Ruby on Rails, and I've set up a basic system to scan an parse and XML datasource, storing the elements in a MySQL database.
I'm intending to run the script as a rake task at set intervals, so want to track additions and updates, outputting the new, or changed, values to a text file.
I initially looked at using the before_save in order to write self.changes to a file, however the complexity arises as I'm retrieving data from two different pages and want to group the log output, e.g note each pricing row is a different record in the same table, ignore the variable names these are examples.
Item GUID
- Price US: #{old price} to #{new price}
- Price UK: #{old price} to #{new price}
The solution I'm currently looking to implement is appending a logged column to the table, if the data changes I can set this to changed, or new if the record has been added, and use this in a query to find records in which logged is not NULL, and group them by GUID. However as this will execute after the object has been saved I lose knowledge of the past values.
Is there a different approach I could take to achieve something like this?
Yes, there is a better way to do this. Take a look at these options you've got:
audited gem: https://github.com/collectiveidea/audited
paper_trail gem: https://github.com/airblade/paper_trail
espinita gem: https://github.com/continuum/espinita
Related
Currently I have a units table with the following columns:
id: PRIMARY Integer
name: String
serial_id: String
. . .
As well as a statuses table. This Unit's data is sent to the server from our server. The statuses table has the following columns:
id: PRIMARY Integer
internal_temp: Float
battery_level: Integer
unit_id: FOREIGN KEY Integer
serial_id: String
...
Currently, I want to void out old units, so that way I can save new units with duplicate/same serial_id. However, I still want to preserve the relationship that voided units and statuses have and maintain the relationship/data in an organized, scalable manner.
My first idea, was to simply add is_void to the units table and then do something like:
Unit.where(serial_id: serial).find_by_is_void(false).statuses
but, I am not sure if this will scale well. Currently, a unit makes nearly 2000 status entries per day.
I was thinking along the lines of creating a join table that can manage the relationship between the unit and status but I was unsure and wanted to seek some advice.
If you're using Rails 5 or lower, you could use the paranoia gem to implement the soft delete functionality. This gem requires us to add a column called deleted_at to the model which we want to soft_delete. It then uses a default scope to fetch only those records which have not been soft deleted. It provides various utility methods to easily fetch deleted records, really destroy records etc.
In case you are using Rails 6 or want to implement the soft deletion feature by yourself, here's what you would need to do:
Add a column called deleted_at to the model which you want to soft delete
Add a default scope in that model as default_scope -> { where(deleted_at: nil) }
You could then write utility methods to check if records are deleted or not, or scopes to fetch those records which are deleted as well. This API could be anything that you want
For scalability, you could even extract out all the common methods that you would want, associated with soft deleted models, into a concern which could simply be plugged in wherever required
As you can see, this implementation is extremely similar to what the paranoia gem provides out of the box. In case you're doing this for learning and have lots of time, I'd recommend building your own feature. If this is something that needs to be deployed soon in a professional project, I'd recommend using the gem since it may speed up the process.
This is a nice article which describes the paranoia gem and its features: Rails Soft Delete with Paranoia gem
I'm looking for some feedback on my current plan of implementing custom fields in rails. I'm new to rails and app development in general and would appreciate any comments from more experienced individuals.
Background
The app: Keep track of food and beverage tastings.
What I'm trying to model:
User creates a new sample type.
They call it: "Wine"
They decide for their company, they'd like to keep track of the following attributes: Origin, Grape Type, Company, Elevation,Temperature Kept, and more.
The only assumptions about a sample type that my database has made is that it has a Name. (eg. coffee, wine, etc.) the rest are all custom fields specified by the user.
Now that a sample type has been created.
The user begins to create samples of sample type wine.
They choose create sample, choose of type Wine.
The fields they must fill in are the ones they specified earlier.
In Origin they put: France, in Grape type: they put chardonnay, etc..
--
My plan of approach is as follows:
When a user creates the sample type, store the custom fields as an array or in some string format and keep it under a column called data.
SampleType
name
wine
data
[origin, grape_type, company, ...]
When a user wants to create a sample of type Wine:
I look up the sample type wine, for each key in the data column, it creates form fields.
When the user submits the data, I create a hash of all the custom fields names and their corresponding data. I serialize it and store it in a hash in a data column like such:
Sample
type
wine
data
{ origin: "France", grape_type: "Pinot Grigio, ... }
My plan at the moment is to use PostgreSQL's hstore to implement the hashing in the data column.
My questions are:
Is this a valid solution for what I'm trying to do?
Will I run into trouble when users change what custom fields they want?
Any other concerns I should take into account?
Is mongodb and other such db's a better choice for this type of model?
I've been using the following links as a reference:
http://schneems.com/post/19298469372/you-got-nosql-in-my-postgres-using-hstore-in-rails
http://blog.artlogic.com/2012/09/13/custom-fields-in-rails/
As well as many other stack overflow posts, however none seem to be using it in the way I mention above.
Any comments are appreciated.
jtgi, having done something like this more times than I want to remember, my first response was, "run away!" In my experience, the whole user-defined field thing is an ugly, hacky, nightmare. Soon, someone will ask, "can I search on grape?" or "I want to be able to input multiple values for grape." And on and on, and you will hate yourself for ever stepping down this path. :-)
That said, I think your approach is pretty decent. To answer your questions directly:
Yes, this is a valid approach.
Yes, you will run into trouble when users change the custom fields they want. (see above)
See some notes below.
Might be. I went there even before I read your 4th question. With your field => value hash, you're kind of implementing a noSQL solution anyhow, but it'll be non-trivial to implement lookups, searches, etc.
Some thoughts:
I think I would marshal the data into a db column, rather than using a db function. That way, it's pure Ruby and not dependent on the db type. See http://www.ruby-doc.org/core-1.9.3/Marshal.html. I'm doing this to cache some data in an app right now, and it's pretty slick. You may need to marshal(l) the data anyhow, if you want to wind up storing Ruby objects more complex than strings.
You'll probably get there soon anyhow, so I would plan on storing some "metadata" about the attributes while you're at it. E.g., "grape" is a String, max length 20, "rating" is an integer between 0 and 100. That way you can make your form a little prettier and do some rudimentary validation.
When you come to hate this feature, you can remember me. :-)
Normally to create/alter a table in database I use migrations (manually run rake db:migrate) and then in my code I use ActiveRecord. This is very cool as I don't have to worry about representation of the data in db and about a specific kind of db (sqlserver, pg or other).
But now a customer wants to be able to create "things" on-fly himself like, say, he starts selling computers, so he wants to an interface where he can dynamically create an object "computer" with properties like "Name, RAM, HD, ...". It seems to be quite natural to create a separate table in db with all these fields. But how can I do that in RoR and keep all these nice things about ActiveRecord?
Please suggest.
The usual way is to do exactly the opposite:
Have a table for object types
Have a table for field names for each object type
Have a very big table with all the custom attributes for each object of any type
This is called EAV (Entity-attribute-value model, see http://en.wikipedia.org/wiki/Entity-attribute-value_model). And it scales pretty bad.
Alternatively, you can use a store text column instead of the big EAV table (see http://api.rubyonrails.org/classes/ActiveRecord/Store.html) so you don't have to make those difficult attribute retrievals, typical of EAV. You still need to store somewhere the "object types" definitions, so the expected fields etc are available when building forms and tables.
The problem with this approach is that you are not able to query (where/join/select) on those attributes because they are not columns. There are a number of solutions to that:
Don't do filtering on those attributes (meh...)
Have an external search server that's able to do faceted search
(as #Amar correctly says) Use a document database
Use postgreSQL and use hstore instead of a simple serialized column.
NoSQL database(Document Database Mongodb,CouchDB) can be best fit for this or use redis. As per my thoughts you can use Vertical Table concept Try to run Rails 2.x Demo of application for MySQL.
You can try with Mongodb, check if this is needed.
I want to alter a record, but I don't want to read it out of the database first, because frankly I don't see the point. It's huge and across a network that is slow and expensive.
Can it be done (easily)?
Lets say I have a record with 100 fields (for arguments sake) and I want to alter one field in the table. I have a really bad connection to the database (this is true) because it's housed on a different box and there's nothing I can do to change this.
Right now I pull down the record and rails validates its contents (because I have serialized bits) I then alter one field (one of the hundred depending on X condition) and save the record again. Which I suppose writes the whole record to the database again, with no knowledge of the fact that I only changed one small bit. (this last bit is assumption)
Now to change one record it's sending a huge amount of data across the network, and it could be that I'm only changing one small small thing..
Also it's doing two queries on the database. First the select * then the update..
So my question.. are there smarter base classes that do this right, to write without read?
Top of my head I would think a setter method for each field with a bool flag for changed.
When saving, walk the flags and where true... Does this happen now, if so how do I make use of it?
Rails models have the update method, which does a SQL select then an SQL update.
Its use is simple (suppose you have a model named Person)
Person.update(person_id, :user_name => 'New Name')
The drawback is that you have to know beforehand the person id.
But then, you will always have to do a query to find out that. You could write your own SQL to change the column value, with a WHERE clause that searches for the param you have.
Rails models have the update_all method, which does only a SQL update without loading the data.
But I wouldn't recommend using it because it doesn't trigger any callbacks or validations that your model may have.
You could use the update_all like this:
Book.update_all "author = 'David'", "title LIKE '%Rails%'"
ps: examples were all taken from the Rails official documentation. If you search for update, or update_all, you'll find both methods in no time.
Take a look there, it's a good place to find out what rails can do.
Rails API link
It's not obvious from the documentation, but update_all is the answer to this question.
Book.where(id: 123).update_all(title: "War and Peace")
results in exactly one query:
UPDATE `books` SET `books`.`title` = "War and Peace" WHERE `books`.`id` = 123
It's been a while since this question was asked, but this is so for Rails 4/5/6.
Say I wanted to allow an administrative user to add a field to an ActiveRecord Model via an interface in the Rails app. I believe the normal ActiveRecord::Migration code would be adequate for modifying the AR Model's table structure (something that would not be wise for many applications - I know). Of course, only certain types of fields could be added...in theory.
Obviously, the forms that add (or edit) records to this newly modified ActiveRecord Model would need to be build dynamically at run-time. A common form_for approach won't do. This discussion suggests this can only be accomplished with JavaScript.
http://groups.google.com/group/rubyonrails-talk/browse_thread/thread/fc0b55fd4b2438a5
I've used Ruby in the past to query an object for it's available methods. I seem to remember it was insanely slow. I'm too green with Ruby and Rails to know an elegant way to approach this. I hope someone here may. I'm also open to entirely different approaches to this problem that don't involve modifying the database.
To access the columns which are currently defined for a model, use the columns method - it will give you, for each column, its name, type and other information (such as whether it is a primary key, etc.)
However, modifying the schema at runtime is delicate.
The schema is pre-loaded (and cached, from the DB driver) by each model class when it is first loaded. In production mode, Rails only does this once per model, around startup.
In order to force Rails to refresh its cached schema following your modification, you should force Ruby to reload the affected model's class (pretty much what Rails does for you automatically, after each request, when running in development mode - see how to reload a class using remove_const followed by load.)
If you have a Mongrel cluster, you also have to inform the other processes in the cluster, which run in their own separate memory space, to also reload their model's classes (some clusters will allow you to create a 'restart.txt' file, which will cause an automatic soft-restart of all processes in your cluster with no additional work required on your behalf.)
Now, these having been said, depending on the actual problem that you need to solve you may not need to dynamically alter the schema after all. Instead of adding, say, columns col1, col2 and col3 to some table entries (model Entry), you can use a table called dyn_attribs, where Entry has_many :dyn_attribs, and where dyn_attribs has both a key column (which in this case can have values col1, col2 or col3) and a value column (which lists the corresponding values for col1, col2 etc.)
Thus, instead of:
my_entry = Entry.find(123)
col1 = my_entry.col1
#do something with col1
you would use:
my_entry = Entry.find(123, :include => :dyn_attribs)
dyn_attribs = my_entry.dyn_attribs.inject(HashWithIndifferentAccess.new) { |s,a|
s[a.key] = a.value ; s
}
col1 = dyn_attribs[:col1]
#do something with col1
The above inject call can be factored away into the model, or even into a base class inherited from by all models that may require additional, dynamic columns/attributes (see Polymorphic associations on how to make several models share the same dyn_attribs table for dynamic attributes.)
UPDATE
Adding or renaming a column via a regular HTML form.
Assume that you have a DynAttrTable model representing a table with dynamic attributes, as well as a DynAttrDef defining the dynamic attribute names for a given table.
Run:
script/generate scaffold_resource DynAttrTable name:string
script/generate scaffold_resource DynAttrDef name:string
rake db:migrate
Then edit the generated models:
class DynAttrTable < ActiveRecord::Base
has_many :dyn_attr_defs
end
class DynAttrDef < ActiveRecord::Base
belongs_to :dyn_attr_table
end
You may continue to edit the controllers and the views like in this tutorial, replacing Recipe with DynAttrTable, and Ingredient with DynAttrDef.
Alternatively, use one of the plugins reviewed here to automatically put the dyn_attr_tables and dyn_attr_defs tables under management by an automated interface (with all its bells and whistles), with virtually zero implementation effort on your behalf.
This should get you going.
Say I wanted to allow an
administrative user to add a field to
an ActiveRecord Model via an interface
in the Rails app.
I've solved this sort of problem before by having an extra model called AdminAdditions. The table includes an id, an admin user id, a model name string, a type string, and a default value string.
I override the model's find and save methods to add attributes from its admin_additions, and save them appropriately when changed. The model table has a large text field, initially empty, where I save nondefault values of the added attributes.
Essentially the views and controllers can pretend that every attribute of the model has its own column. This means form_for and so on all work.
ActiveRecord::Migration.add_column(User, "email", :string)
You could use Flex Attributes for this, though if you want to be able to search or order by these new columns you'll have to write (a lot of) custom SQL.
I have seen the dynamic alteration/migration of tables offered as a solution many times but I have never actually seen it implemented. There are many reasons why this solution is rarely implemented.
If the table is large then the table may/will be locked for extended periods of what is supposed to be up-time.
Why is your model changing dynamically? It is quite rare for a models structure to need to change dynamically. It is more often an indication that you are trying to model something specific in a generalised way.
This is often an attempt a producing a "Categorised" model than could be better solved by another approach.
DDL statements are often not allowed by the same user that is being used for day to day DML requirements. Whilst this could be the case, and often is in the ROR arena it is not always the "right" way to do it.
What are you trying to achieve here? A better understanding of the problem would probably reveal a more natural solution.
If you were doing this with PostgreSQL now you could probably get away with a JSON type field and then just store whatever in the json hash.