I'm having problems with weird behaviour in RoR. I'm having a Hash that i'm converting to json using to_json() like so:
data = Hash.new
# ...
data = data.to_json()
This code appears inside a model class. Basically, I'm converting the hash to JSON when saving to database. The problem is, the string gets saved to database with its surrounding quotes. For example, saving an empty hash results in: "{}". This quoted string fails to parse when loading from the database.
How do I get rid of the quotes?
The code is:
def do_before_save
#_data = self.data
self.data = self.data.to_json()
end
EDIT:
Due to confusions, I'm showing my entire model class
require 'json'
class User::User < ActiveRecord::Base
after_find { |user|
user.data = JSON.parse(user.data)
}
after_initialize { |user|
self.data = Hash.new unless self.data
}
before_save :do_before_save
after_save :do_after_save
private
def do_before_save
#_data = self.data
self.data = self.data.to_json()
end
def do_after_save
self.data = #_data
end
end
The data field is TEXT in mysql.
I'm willing to bet money that this is the result of you calling .to_json on the same data twice (without parsing it in between). I've had a fair share of these problems before I devised a rule: "don't mutate data in a lossy way like this".
If your original data was {}, then first .to_json would produce "{}". But if you were to jsonify it again, you'd get "\"{}\"" because a string is a valid json data type.
I suggest that you put a breakpoint in your before_save filter and see who's calling it the second time and why.
Update
"call .to_json twice" is a general description and can also mean that there are two subsequent saves on the same object, and since self.data is reassigned, this leads to data corruption. (thanks, #mudasobwa)
It depends on your model's database field type.
If the field is string type (like VARCHAR or TEXT) it should be stored as string (no need to get rid of the quotes - they are fine). Make sure calling to_json once.
If the field is Postgres JSON type, then you can just assign a hash to the model's field, no need to call to_json at all.
If you are saving hash as a JSON string in a varchar column you can use serialize to handle marshalling/unmarshaling the data:
class Thing < ActiveRecord::Base
serialize :foo, JSON
end
Knowing exactly when to convert the data in the lifecycle of a record is actually quite a bit harder than your naive implementation. So don't reinvent the wheel.
However a huge drawback is that the data cannot be queried in the DB*. If you are using Postgres or MySQL you can instead use a JSON or JSONB (postgres only) column type which allows querying. This example is from the Rails guide docs:
# db/migrate/20131220144913_create_events.rb
create_table :events do |t|
t.json 'payload'
end
# app/models/event.rb
class Event < ApplicationRecord
end
# Usage
Event.create(payload: { kind: "user_renamed", change: ["jack", "john"]})
event = Event.first
event.payload # => {"kind"=>"user_renamed", "change"=>["jack", "john"]}
## Query based on JSON document
# The -> operator returns the original JSON type (which might be an object), whereas ->> returns text
Event.where("payload->>'kind' = ?", "user_renamed")
use {}.as_json instead of {}.to_json
ex:
a = {}
a.as_json # => {}
a.to_json # => "{}"
http://api.rubyonrails.org/classes/ActiveModel/Serializers/JSON.html#method-i-as_json
Related
I am trying to not jsonify a string within a hash. It already has been escaped.
Reading around the way of handling this for PORO is to overwrite as_json. So I wrapped the string in another object. But as I'm dealing with just a string that leads to a stack level too deep when I return the object. When I return the already encoded string, it obviously tries to escape it. activesupport-6.0.3.2/lib/active_support/json/encoding.rb
def jsonify(value)
case value
when String
EscapedString.new(value)
when Numeric, NilClass, TrueClass, FalseClass
value.as_json
when Hash
Hash[value.map { |k, v| [jsonify(k), jsonify(v)] }]
when Array
value.map { |v| jsonify(v) }
else
jsonify value.as_json
end
end
What I could do is monkey patch the above method to accept the class I have wrapped the string in and do nothing.
I guess the other option is to parse the JSON string back into a hash and let it follow the normal procedure. That would impact our performance too much though.
So my question is, Is there are more elegant way of telling Rails to do nothing when encountering a specified object when it tries to jsonify it.?
Edit:
Im using this already jsonified string like:
{foo: MyStringJsonWrapper.new('{"bar":"foobar"}')}.to_json
I was looking around in the file activesupport-6.0.3.2/lib/active_support/json/encoding.rb and noticed you can configure which class is used to perform the json encoding. It is currently JSONGemEncoder which is contained within that class.
So I made my own class:
class MyJSONEncoder < ActiveSupport::JSON::Encoding::JSONGemEncoder
class AlreadyEscapedString < String
def to_json(*)
self
end
end
private
def jsonify(value)
if value.is_a?(AlreadyEncodedStringWrapper)
AlreadyEscapedString.new(value.already_jsonified)
else
super
end
end
end
and
class AlreadyEncodedStringWrapper
attr_accessor :already_jsonified
def initialize already_jsonified
#already_jsonified = already_jsonified
end
def as_json(options = {})
self
end
end
The extra class of AlreadyEscapedString exists to overwrite the behaviour of EscapedString(found in the same encoding class) but do nothing.
Then just set:
ActiveSupport.json_encoder = MyJSONEncoder
Example:
jsonified_hash = {foobar: "barbaz"}.to_json
already_encoded_string = AlreadyEncodedStringWrapper.new(jsonified_hash)
{foo: already_encoded_string}.to_json
=> "{\"foo\": {\"foobar\": \"barbaz\"}}"
I'm using Toptal's Chewy gem to connect and query my Elasticsearch, just like an ODM.
I'm using Chewy along with Elasticsearch 6, Ruby on Rails 5.2 and Active Record.
I've defined my index just like this:
class OrdersIndex < Chewy::Index
define_type Order.includes(:customer) do
field :id, type: "keyword"
field :customer do
field :id, type: "keyword"
field :name, type: "text"
field :email, type: "keyword"
end
end
end
And my model:
class Order < ApplicationRecord
belongs_to :customer
end
The problem here is that when I perform any query using Chewy, the customer data gets deserialized as a hash instead of an Object, and I can't use the dot notation to access the nested data.
results = OrdersIndex.query(query_string: { query: "test" })
results.first.id
# => "594d8e8b2cc640bb78bd115ae644637a1cc84dd460be6f69"
results.first.customer.name
# => NoMethodError: undefined method `name' for #<Hash:0x000000000931d928>
results.first.customer["name"]
# => "Frederique Schaefer"
How can I access the nested association using the dot notation (result.customer.name)? Or to deserialize the nested data inside an Object such as a Struct, that allows me to use the dot notation?
try to use
results = OrdersIndex.query(query_string: { query: "test" }).objects
It converts query result into active record Objects. so dot notation should work. If you want to load any extra association with the above result you can use .load method on Index.
If you want to convert existing ES nested object to accessible with dot notation try to reference this answer. Open Struct is best way to get things done in ruby.
Unable to use dot syntax for ruby hash
also, this one can help too
see this link if you need openStruct to work for nested object
Converting the just-deserialized results to JSON string and deserializing it again with OpenStruct as an object_class can be a bad idea and has a great CPU cost.
I've solved it differently, using recursion and the Ruby's native Struct, preserving the laziness of the Chewy gem.
def convert_to_object(keys, values)
schema = Struct.new(*keys.map(&:to_sym))
object = schema.new(*values)
object.each_pair do |key, value|
if value.is_a?(Hash)
object.send("#{key}=", convert_to_object(value.keys, value.values))
end
end
object
end
OrdersIndex.query(query_string: { query: "test" }).lazy.map do |item|
convert_to_object(item.attributes.keys, item.attributes.values)
end
convert_to_object takes an array of keys and another one of values and creates a struct from it. Whenever the class of one of the array of values items is a Hash, then it converts to a struct, recursively, passing the hash keys and values.
To presence the laziness, that is the coolest part of Chewy, I've used Enumerator::Lazy and Enumerator#map. Mapping every value returned by the ES query into the convert_to_object function, makes every entry a complete struct.
The code is very generic and works to every index I've got.
In my Rails 4 app, I actually send an active record relation in JSON with:
[...]
wine['varietals'] = record.varietals
#wines << wine
format.json { render :json => { :success => "OK", :items => #wines } }
[...]
wine['varietals'] is an array of AR relations. My problem is the varietal model contains a field named grape_id that is an integer. I need to send it in string for my WS. I don't want to make a custom conversion to JSON just for this field.
How to force this field to be string before the automatic JSON conversion ? If possible I don't want to make an array of hashes and keep the AR style with dot: model.field
wine['varietals'].each do |varietal|
varietal.grape_id.to_s
end
Of course this doesn't work.
All Rails models have an as_json method that gets called when rednering the model to JSON. You can override this method within your models to set up custom JSON formatting. In your case, you may want to add something like this to your Wine model:
def as_json(opts = {})
json = super(opts)
json["grape_id"] = self.grape_id.to_s
json
end
The method gives you the default model JSON when you call the super method and set it to the json variable, then stringifies grape_id and sets it in the JSON, and finally returns the updated JSON.
Now, any time a controller returns a JSON version of single Wine model, or an association of multiple Wine models, the JSON will be formatted through this updated method and the grape_id will be stringified every time.
I have the following function to sum all the records of an :amount field in my Pack model for that given user:
user.rb
def total_money_spent_cents
amount = self.packs.map(&:amount).sum
return amount
end
However, when I use this function I receive the following error:
nil can't be coerced into Fixnum
Any suggestions?
EDIT
I am still having issues in regards to Fixnum in my tests, and have another question open here.
This suggests that one of your packs has an amount field which has not yet been set, so is nil. When you try and add it to something else, it undergoes Type coercion, to see if Ruby can massage its type into one that can be added to numbers, but it can't, and so you have this error.
One solution is this:
def total_amount_spent_cents
packs.map(&:amount).compact.sum
end
Array#compact removes the nil elements.
This may be fixing the symptom and not the actual problem though. It could be the case that you shouldn't have nil's in there at all, in which case you should check the initialisation of your Pack model (or perhaps its validations, to ensure that amount is mandatory).
I added some extra methods into Array and Hash for this sort of thing: they're like compact but they remove all values returning true for blank? rather than just nil: so will remove empty strings, empty arrays, hashes etc.
class Hash
def compact_blank!
self.each{|k,v| self.delete(k) if v.blank? }
self
end
def compact_blank
self.dup.compact_blank!
end
end
class Array
def compact_blank!
self.delete_if(&:blank?)
end
def compact_blank
self.dup.compact_blank!
end
end
use like
["1", "abc", "", nil, []].compact_blank
=> ["1", "abc"]
it's useful with params especially, where you might get a lot of empty strings through.
I would like to save query result into redis using JSON serialization and query it back.
Getting query results to json is pretty easy:
JSON.generate(Model.all.collect {|item| item.attributes})
However I did not find a proper way to deserialize it back to ActiveRecord. The most straight-forward way:
JSON.parse(#json_string).collect {|item| Model.new.from_json(item)}
Gives me an error:
WARNING: Can't mass-assign protected attributes: id
So id gets empty. I thought of just using OpenStruct for the views instead of ActiveRecord but I am sure there is a better way.
You could instantiate the new object from JSON and then assign the id afterwards. Probably best to create your own method for this:
class Model
def self.from_json_with_id(params = {})
params = JSON.parse(params)
model = new(params.reject {|k,v| k == "id"})
model.id = params["id"]
model
end
end
Or maybe just override the from_json() method.
Why not like this:
JSON.parse(#json_string).each do |item|
item.delete(:id) # I tested it in my case it also works without this line
object=Model.create(item)
end
If the host that created the JSON adds a JSON root you might have to use item[1] instead of item.