I'm trying to create a data visualization site for time series data.
I want the user to be able to upload their own csv's of time series data. So that my web application can create graphs from it.
The amount of fields the user can have should be variable. A user can have just temperature, or he can have temperature and humidity. I want to use only one model to handle the variable amount of fields. A template model maybe?
I think the problem here is with ActiveRecord and the way I want it. I'd probably have to create a separate Model for each table the user wants to handle different number of fields.
Is there a better way to handle this using ActiveRecord? Suggestions? I'm lost and I haven't been able to move forward at all. Any help is appreciated.
Thanks,
Ejay
EDIT:
Another I thought of was a model that represents each data column. So that every instance would contain the same number attributes. Something like, :name, :type, :timestamp, :data. However, this seems highly inefficient because I would have to access a number of tables in order to upload one csv file.
Is this the correct way I should be thinking about this?
My suggesstion is having only model, called DataSet. DataSet has three columns, :name, :data and :user_id. The type of :name should be something like varchar(255). The type of :data is just text, and it's purpose is to store a json blob representing the CSV.
Let's say the CSV looks something like this:
num,value
1,4
2,5
3,4
4,100
5,60
You parse this into a hash that looks like this
{
1 => 4,
2 => 5,
3 => 4,
4 => 100,
5 => 60
}
When you want to write this to the database, just do:
DataSet s = DataSet.new
s.name = 'Whatever'
s.data = parse_csv(csv_file).to_json # "{\"1\":4,\"2\":5,\"3\":4,\"4\":100,\"5\":60}"
s.save
To get the data back out, you can do this:
ActiveSupport::JSON.decode s.data
You can ever edit your model to do this conversion automatically:
class DataSet < ActiveRecord::Base
belongs_to :user
def data=(data_hash)
write_attribute :data, data_hash.to_json
end
def data
ActiveSupport::JSON.decode read_attribute(:data)
end
end
Related
I have an Appointment model, whose available_dates can include multiple dates (say, available in Jan 1, 5, 6 and 7).
My question is very basic: how should I store available dates for each event?
What I can think of is a table Avaiable_Dates with two columns: event_id and date. Each event would have multiple rows, one date per row. It seems to be cumbersome to query entire table to make sure we got all dates of an event. A Hash {event => {date1, date2, date3}} would be faster, but I don't know how to implement it using ActiveRecord.
Thank you.
It might not be a bad idea to just use the separate model for available times, but if you decide to go the hash route you can do so using the serialize keyword. You have to tell ActiveRecord to serialize the variable, and then it will do the serialization and deserialization automatically whenever you access the hash.
Saving arrays, hashes, and other non-mappable objects in text columns
Active Record can serialize any object in text columns using YAML. To do so, you must specify this with a call to the class method serialize. This makes it possible to store arrays, hashes, and other non-mappable objects without doing any additional work.
class User < ActiveRecord::Base
serialize :preferences
end
user = User.create(:preferences => { "background" => "black", "display" => large })
User.find(user.id).preferences # => { "background" => "black", "display" => large }
You can also specify a class option as the second parameter that’ll raise an exception if a serialized object is retrieved as a descendant of a class not in the hierarchy.
class User < ActiveRecord::Base
serialize :preferences, Hash
end
user = User.create(:preferences => %w( one two three ))
User.find(user.id).preferences # raises SerializationTypeMismatch
When you specify a class option, the default value for that attribute will be a new instance of that class.
class User < ActiveRecord::Base
serialize :preferences, OpenStruct
end
user = User.new
user.preferences.theme_color = "red"
From rubyonrails.org
From a design perspective, if you think you will ever add any more data to the available time object then you should make it its own model. Otherwise a serialized hash seems fine.
I don't see a problem with the separate table that you mentioned, I would go with that. It will also be easier to extend later which you will appreciate when the time comes.
I'm getting started with parsing data and getting some structure from user supplied strings (mostly pulling out digits and city names).
I've run a bit of code in the ruby interpreter, and now I want to use that same code in a web application.
I'm struggling as to where in the code my parsing should be, or how it is structured.
My initial instinct was that it belongs in the model, because it is data logic. For example, does the entry have an integer, does it have two integers, does it have a city name, etc. etc.
However, my model would need to inherit both ActiveRecord, and Parslet (for the parsing), and Ruby apparently doesn't allow multiple inheritance.
My current model is looking like this
#concert model
require 'parslet'
class concert < Parlset::Parser
attr_accessible :date, :time, :city_id, :band_id, :original_string
rule(:integer) {match('[0-9]').repeat(1)}
root(:integer)
end
Really not much there, but I think I'm stuck because I've got the structure wrong and don't know how to connect these two pieces.
I'm trying to store the original string, as well as components of the parsed data.
I think what you want is:
#concert model
require 'parslet'
class concert < ActiveRecord::Base
before_save :parse_fields
attr_accessible :date, :time, :city_id, :band_id, :original_string
rule(:integer) {match('[0-9]').repeat(1)}
root(:integer)
private
def parse_fields
date = Parlset::Parser.method_on_original_string_to_extract_date
time = Parlset::Parser.method_on_original_string_to_extract_time
city_id = Parlset::Parser.method_on_original_string_to_extract_city_id
band_id = Parlset::Parser.method_on_original_string_to_extract_band_id
end
end
It looks to me as though you need several parsers (one for city names, one for digits). I would suggest that you create an informal interface for such parsers, such as
class Parser
def parse(str) # returning result
end
end
Then you would create several Ruby classes that each do a parse task in ./lib.
Then in the model, you'd require all these ruby classes, and put them to the task, lets say in a before_save hook or such.
As the author of parslet, I might add that parsing digits or city names is probably not the sweet spot for parslet. Might want to consider regular expressions there.
As an extension of the question
How to retrieve the hash values in the views in rails
I have some doubts of keeping hash values in the table..
I have a user detail table where i am maintaining the additional details of the user in a column named additional_info in a hash format .. Will it be good in keeping like so...
As if the user scenario changes if the user wants to find all the users under a particular project where i kept the project to which the user belongs in the hash format..
Give some suggestions..
Simple solution is to serialiaze it:
class FooBar < ActiveRecord::Base
# ...
serialize :additional_info
#...
end
This internally uses the YAML serializer. You can assign any object that can be serialized using YAML.
foo = FooBar.first
foo.additional_info = {:foo => 'Lorem', :bar => 'ipsum'}
foo.save
foo.additional_info[:foo] # Gives 'Lorem'
I have a model called List which has many records:
class List
has_many :records
end
class Record
end
The table Record has 2 permanent fields: name, email.
Besides these 2 fields, for each List a Record can have 'n' custom fields.
For example: for list1 I add address(text), dob(date) as custom fields.
Then while adding records to list one, each record can have values for address and dob.
Is there any ActiveRecord plugin which provides this type of functionality?
Or else could you share your thoughts on how to model this?
Thanks in advance,
Pankaj
You should take a look in schemaless database solutions. One that i think is great is mongoDB.
There is a railscast explaining how to use it with rails. Take a look, it works great.
If your custom fields don't need to be real database columns, you could use serialize: http://railsapi.com/doc/rails-v2.3.5/classes/ActiveRecord/Base.html#M000924 You would use it like:
class Record < ActiveRecord::Base
serialize :custom_fields, Hash
end
r = Record.create :custom_fields => {:name => 'John Doe', :birth_date => Date.new(1970,1,1)}
r.custom_fields[:name]
# => 'John Doe'
r.custom_fields[:birth_date]
# => #<Date: 4881175/2,0,2299161>
Pro: easy to use
Con: since the custom fields are not db columns, you can't find records in the db based on their values (e.g. Record.find_by_name('John Doe') does not work)
Maybe this? has_magic_columns. I havn't tested it myself but seems like it can do what you need.
Im looking for a good way to solve my performance issues in my rails application. I have three tables which have: one to one to many connections in between. If I want to fill in 130 items of the first table with all the data for the underneath tables, It results in about 1000 queries and takes about 10 seconds (SQLite DB).
I found the
accept_nested_attributes_for
statement, witch lets you enter data for multiple tables in one line of code. My question is, wether this is a good option in a performance point of view. Does somebody have any experience with it?
Thanks
Markus
accept_nested_attributes_for add the possibility to ActiveRecord to be able to write into association directly from one model.
exemple :
You have models like :
class User
accepts_nested_attributes_for :cars
end
class Car
belongs_to :user
end
and a hash like :
param[:user] = {}
params[:user][:name] = "Mike"
params[:user][:car] = {}
params[:user][:car][:brand] = "Nissan"
User.create(params[:user])
This will create a new user and a new car,
without accepts_nested_attributes_for :
#user = User.create(params[:user])
#car = Car.create(params[:user][:car])
#user.car = #car
This function is usually with fields_for in HTML forms so you can easily handle the creation of an object and his associations.
In your case I imagine your models like that (regarding your XML) :
class Card
has_one :front_side, :class => "Side"
has_one :back_side, :class => "Side"
end
class Side
belongs_to :card
has_many :card_side_entry
end
class CardSideEntry
belongs_to :side
end
I don't know where your XML come from (your data are extracted from it ??), but I imagine you could use accepts_nested_attributes_for so you could have each card hash generating the associations.
But I'm not sure to understand all the problem and if this is the best solution
here it is:
Table: cards
front_side_id
back_side_id
Table: card_sides
Table: card_side_entries
card_side_id
I have now a XML like this:
<Cards>
<Card>
<FrontSide>
<CardSideEntries>
<CardSideEntrie/>
...
</CardSideEntries>
</FrontSide>
<BackSide>
<CardSideEntries>
<CardSideEntrie/>
...
</CardSideEntries>
</BackSide>
</Card>
...
</Cards>
In my solution I parse the whole XML file line by line, and because I sometimes need a card_id I have to save a certain table entry twice... Does anybody now something about accept_nested_attributes_for?
Thanks,
Markus