I have the following data:
[{"uid"=>"12406664"}, {"uid"=>"13715056"}, {"uid"=>"20911274"}, {"uid"=>"20921750"}, {"uid"=>"144901695"}, {"uid"=>"200002261"}, {"uid"=>"583131545"}, {"uid"=>"584667098"}, {"uid"=>"585043552"}, {"uid"=>"593713530"}, {"uid"=>"645734146"}, {"uid"=>"649596998"}, {"uid"=>"663955553"}, {"uid"=>"698033741"}, {"uid"=>"1024899231"}, {"uid"=>"1032611215"}, {"uid"=>"1076202442"}, {"uid"=>"1168728549"}, {"uid"=>"1283882122"}, {"uid"=>"1296965460"}, {"uid"=>"1417999220"}, {"uid"=>"1420197620"}, {"uid"=>"1455766774"}, {"uid"=>"1479820827"}, {"uid"=>"1568075339"}, {"uid"=>"100000804563736"}, {"uid"=>"100001055926570"}, {"uid"=>"100001633945205"}]
How do I store this data using Ruby on Rails? I'd like to keep the data accesable, where
>> x[1]
=> {"uid"=>"13715056"}
Currently I'm using a postgreSQL heroku server, I have the User.user_id column serialized as a Hash and the data is a mess:
"[{\"uid\"=>\"12406664\"}, {\"uid\"=>\"13715056\"}, {\"uid\"=>\"20911274\"}, {\"uid\"=>\"20921750\"}, {\"uid\"=>\"144901695\"}, {\"uid\"=>\"200002261\"}, {\"uid\"=>\"583131545\"}, {\"uid\"=>\"584667098\"}, {\"uid\"=>\"585043552\"}, {\"uid\"=>\"593713530\"}, {\"uid\"=>\"645734146\"}, {\"uid\"=>\"649596998\"}, {\"uid\"=>\"663955553\"}, {\"uid\"=>\"698033741\"}, {\"uid\"=>\"1024899231\"}, {\"uid\"=>\"1032611215\"}, {\"uid\"=>\"1076202442\"}, {\"uid\"=>\"1168728549\"}, {\"uid\"=>\"1283882122\"}, {\"uid\"=>\"1296965460\"}, {\"uid\"=>\"1417999220\"}, {\"uid\"=>\"1420197620\"}, {\"uid\"=>\"1455766774\"}, {\"uid\"=>\"1479820827\"}, {\"uid\"=>\"1568075339\"}, {\"uid\"=>\"100000804563736\"}, {\"uid\"=>\"100001055926570\"}, {\"uid\"=>\"100001633945205\"}]"
>> User.last.uid.last
=> "]"
How can I correct this?
Thank you!
you can serialize as you wish: Hash, Array, custom class... you're quite free.
As of your expectations, serialize as an Array.
Related
I have a model with an :extra_fields column that is :jsonb datatype, I want to add in the attr hashes to the column, something like this below but I am unsure of the syntax to cast the hash values' datatypes here, and if not here what is the best practice for casting hash value data ?
instance = Model.find_or_create_by(ref_id: hash[:ref_id]) do |a|
a.extra_fields = {
'attr1' : hash[:attr1], <-- //possible to cast type here ie ::type ?
'attr2' : hash[:attr2] <--
}
instance.save!
end
Bonus: how would I cast the hash values as type :decimal, :string, :boolean, :date for example?
All incoming parameters in Rails/Rack are strings. Well except except array/hash parameters which still have strings as values. Rails does the actual casting when you pass parameters to models.
You can cast strings to any other type in Ruby with the .to_x methods:
irb(main):006:0> "1.23".to_f
=> 1.23
irb(main):007:0> "1.23".to_d
=> #<BigDecimal:7ff7dea40b68,'0.123E1',18(18)>
irb(main):008:0> 1.23.to_s
=> "1.23"
irb(main):008:0> 1.23.to_i
=> 1
Boolean casting is a Rails feature. You can do it by:
# Rails 5
ActiveModel::Type::Boolean.new.cast(value)
ActiveModel::Type::Boolean.new.cast("true") # true
ActiveModel::Type::Boolean.new.cast("t") # true
ActiveModel::Type::Boolean.new.cast("false") # false
ActiveModel::Type::Boolean.new.cast("f") # false
# This is somewhat surprising
ActiveModel::Type::Boolean.new.cast("any arbitrary string") # true
# Rails 4.2
ActiveRecord::Type::Boolean.new.type_cast_from_database(value)
# Rails 4.1 and below
ActiveRecord::ConnectionAdapters::Column.value_to_boolean(value)
Note that this is very different then the Ruby boolean coercion done by ! and !!.
irb(main):008:0> !!"false"
(irb):8: warning: string literal in condition
=> true
In Ruby everything except nil and false are true.
Dates are somewhat more complex. The default Rails date inputs use multi-parameters to send each part of the date (year, month, day) and a special setter that constructs a date from these inputs.
Processing by PeopleController#create as HTML
Parameters: { "person"=>{"birthday(1i)"=>"2019", "birthday(2i)"=>"2", "birthday(3i)"=>"16"}, ...}
You can construct a date from these parameters by:
date_params = params.fetch(:person).permit("birthday")
Date.new(*date_params.values.map(&:to_i))
what is the best practice for casting hash value data ?
There is no best practice here. What you instead should be pondering is the use of a JSON column. Since you seem to be want to apply some sort of schema to the data it might be a good idea to actually create a separate table and model. You are after all using a relational database.
JSON columns are great for solving some complex issues like key/value tables or storing raw JSON data but they should not be your first choice when modelling your data.
See PostgreSQL anti-patterns: Unnecessary json/hstore dynamic columns for a good write up on the topic.
I'm using Toptal's Chewy gem to connect and query my Elasticsearch, just like an ODM.
I'm using Chewy along with Elasticsearch 6, Ruby on Rails 5.2 and Active Record.
I've defined my index just like this:
class OrdersIndex < Chewy::Index
define_type Order.includes(:customer) do
field :id, type: "keyword"
field :customer do
field :id, type: "keyword"
field :name, type: "text"
field :email, type: "keyword"
end
end
end
And my model:
class Order < ApplicationRecord
belongs_to :customer
end
The problem here is that when I perform any query using Chewy, the customer data gets deserialized as a hash instead of an Object, and I can't use the dot notation to access the nested data.
results = OrdersIndex.query(query_string: { query: "test" })
results.first.id
# => "594d8e8b2cc640bb78bd115ae644637a1cc84dd460be6f69"
results.first.customer.name
# => NoMethodError: undefined method `name' for #<Hash:0x000000000931d928>
results.first.customer["name"]
# => "Frederique Schaefer"
How can I access the nested association using the dot notation (result.customer.name)? Or to deserialize the nested data inside an Object such as a Struct, that allows me to use the dot notation?
try to use
results = OrdersIndex.query(query_string: { query: "test" }).objects
It converts query result into active record Objects. so dot notation should work. If you want to load any extra association with the above result you can use .load method on Index.
If you want to convert existing ES nested object to accessible with dot notation try to reference this answer. Open Struct is best way to get things done in ruby.
Unable to use dot syntax for ruby hash
also, this one can help too
see this link if you need openStruct to work for nested object
Converting the just-deserialized results to JSON string and deserializing it again with OpenStruct as an object_class can be a bad idea and has a great CPU cost.
I've solved it differently, using recursion and the Ruby's native Struct, preserving the laziness of the Chewy gem.
def convert_to_object(keys, values)
schema = Struct.new(*keys.map(&:to_sym))
object = schema.new(*values)
object.each_pair do |key, value|
if value.is_a?(Hash)
object.send("#{key}=", convert_to_object(value.keys, value.values))
end
end
object
end
OrdersIndex.query(query_string: { query: "test" }).lazy.map do |item|
convert_to_object(item.attributes.keys, item.attributes.values)
end
convert_to_object takes an array of keys and another one of values and creates a struct from it. Whenever the class of one of the array of values items is a Hash, then it converts to a struct, recursively, passing the hash keys and values.
To presence the laziness, that is the coolest part of Chewy, I've used Enumerator::Lazy and Enumerator#map. Mapping every value returned by the ES query into the convert_to_object function, makes every entry a complete struct.
The code is very generic and works to every index I've got.
I am currently working on a database API which is returning me the result of an ActiveRecord::Base class call. Right now when I required a data from the PostGres DB, I am using the following command:
ApiResponse.select([:id, :ts, :response])
.where(userid: 'toto')
.limit(limit).to_a
This code is working find and I get a array of hash :
[
[ 0] #<ApiResponse:0x007fb27e5d0c40> {
:id => "7cbf6ce2-ec5e-41e0-8185-030f0a70b247",
:ts => 2017-08-28 20:21:09 UTC,
:response => {}}, ...
...
]
ApiResponse is just a class to access to the DB using ActiveRecord and defined as:
class ApiResponse < ActiveRecord::Base
Right now, based on the result I have to run the below code to make it JSON and modify the ts field
new_list = list.map do | e |
{
id: e['id'],
ts: e['ts'].strftime("%F %T.%6N%z"),
response: e['response']
}
end
This solution is not the best I think because I am running through a map (list is the result of the select command) to modify one field [ts] to match the date/time format and secondly, I am doing a
return new_list.to_json
the need of modify the date/time is because the PostGres DB use PostGres Date/Time format but ActiveRecord use ISO8601.
I do have 2 questions:
Is there a faster way to parse all the array to apply the 'strftime' on each ts field? other than using map
Is there a way to avoid using 'to_json' to_json is amazingly slow. For privacy, the response is null but in my code, it contains a lot of data which slow down the system.
Thanks
So I am iterating through a set of data and building a hash from it:
clean_response = Array.new
response.each_with_index do |h, idx|
clean_response <<
{
:lat => h["location"]["latitude"],
:lg => h["location"]["longitude"],
:place => h["location"]["name"],
#This grabs the entire hash at "location" because we are wanting all of that data
:profile_picture => h["user"]["profile_picture"],
:hash_tags => h["tags"],
:username => h["user"]["username"],
:fullname => h["user"]["full_name"],
:created_time => (Time.at(h["created_time"].to_i)).to_s,
:image => h["images"]["low_resolution"]["url"] # we can replace this with whichever resolution.
}
end
Which return an array of hashes like so:
[{:lat=>40.7486382,
:lg=>-73.9487686,
:place=>"The Cliffs at LIC",
:profile_picture=>"http://scontent.cdninstagram.com/hphotos-xaf1/t51.2885-19/s150x150/12104940_1653775014895036_286845624_a.jpg",
:hash_tags=>["bouldering"],
:username=>"denim_climber",
:fullname=>"DenimClimber",
:created_time=>2015-10-13 22:58:09 -0400,
:image=>"https://scontent.cdninstagram.com/hphotos-xaf1/t51.2885-15/s320x320/e35/11856571_1062082890510188_611068928_n.jpg"},
{:lat=>40.7459602,
:lg=>-73.9574966,
:place=>"SHI",
:profile_picture=>"http://scontent.cdninstagram.com/hphotos-xaf1/t51.2885-19/11348212_1453525204954535_631200718_a.jpg",
:hash_tags=>["cousins", "suchafunmoment", "johnlennonstyle"],
:username=>"xiomirb",
:fullname=>"Xiomi",
:created_time=>2015-10-13 22:57:21 -0400,
:image=>"https://scontent.cdninstagram.com/hphotos-xaf1/t51.2885-15/s320x320/e35/11375290_1688934151392424_2009781937_n.jpg"}]
I'd like to convert this data to json and then serve it to a specific view.
How can I convert this? I tried the .to_json method but it doesn't return a well formatted one since my UI isn't binding to the data.
You can convert a Ruby hash into JSON using to_json:
require 'json'
your_hash.to_json # gives you a JSON object
But, in your case the data is an array of hashes, but NOT a hash. So, your to_json would not work.
I am not quite sure how you want to do this, but one possibility is to loop through the array of hashes, get each hash and convert that to a JSON object using to_json call (like shown above) and build a new array of JSON objects. This way, you can build an array of JSON objects from an array of hashes.
array_of_json = []
# loop through the array of hashes
clean_response.each do |hash|
array_of_json << hash.to_json
end
array_of_json # array of JSON objects
If by "serve it to a specific view" you mean pass it to a .haml or .erb template, you can pass the array of hashes as is. Both haml and erb will allow you to iterate over the array, and even the hash if you want.
If you mean you want to hand a json string to the browser, #to_json should work fine. Other options are jbuilder or oat when you want to refine what is sent, but to_json should "serve" you well!
My code looks like this:
hash = MyModel.count(:group => 'id', :conditions => 'bla = "bla"')
The returned Hash has keys that are strings. I want them to be ints. I know it would be possible to convert the Hash manually using something like a map construct.
Edit:
Thanks for the responses. Have realised it was a json conversion process that was turning the ids into Strings and rails does in fact use the Fixnum as one might expect.
hash = MyModel.count(group: 'id', conditions: 'bla = "bla"')
should have Fixnum keys by default since id is an instance of Fixnum.
What happens is that ActiveRecord always fetch result as strings and then Rails takes care of converting them to other datatypes according to the type of the database column (we say that they are typecast).
So it's maybe a Rails bug or the 'id' column is not set as integer(which would be surprising).
If you can't fix it, convert them manually:
hash.each_with_object({}) do |(key, value), hash|
hash[key.to_i] = value
end
When I use your code I get integer keys (rails 3.07), what's the column type of id?
If you want to do it manually:
new_hash = hash.inject({}){|h,a| h[a.first.to_i] = a.last; h}
new_hash = Hash[hash.map { |k, v| [k.to_i, v] }