I have this problem with my Rails (5.1.6) application that is running on a PostgreSQL instance.
I have a model with a JSON type column (t.json :meta). The model has a store accessor like
store :meta, accessors: [:title], coder: JSON
The problem is now when I set this value that it shows up in the database as
"{\"title\":\"I am a title\"}"
making it text rather than a JSON value, which in turn makes that I cannot use the JSON query operator (->>) to query my JSON fields. I already tried without the coder option, but this results in it being saved as YAML.
The serialize function also did not change anything for me (adding serialize :meta, JSON)
Any and all help is appreciated!
serialize and store are not intended to be used for native JSON columns. Their purpose is to marshal and un-marshal data into string columns.
This was a "poor mans" JSON storage before native JSON support existed (and was supported by ActiceRecord). Using it on a JSON column will result in a double encoded string as you have noticed.
You don't actually have to do anything to use a JSON column. Its handled by the adapter.
See:
http://api.rubyonrails.org/classes/ActiveRecord/AttributeMethods/Serialization/ClassMethods.html#method-i-serialize
http://guides.rubyonrails.org/active_record_postgresql.html#json-and-jsonb
Related
I have a JSON field in my PostgreSQL database. If I do #profile.json, then I will get something like:
{ {"name"=>"jhon", "degree"=>"12312"}, "1480103144467"=>{"name"=>"", "degree"=>""}}`
It has all the => and other symbols, which I can not parse. How can I convert to normal format?
If you've declared your column of type json that's a signal to Rails to automatically serialize and decode your column on-demand, transparently. What you're seeing here is a traditional Ruby Hash structure, which is to be expected.
Inside the database itself it's stored as JSON.
If you need to re-emit this as JSON for whatever reason, like for an API, try this:
#profile.json.to_json
Calling your column something other than json is probably advisable, too.
Im hard pressed to store data as 'JSON' format into my sqlite database for an rails application. I have searched for how to store data as JSON in my sqlite database but am not seeing many alternatives which are promising. Any one who can guide me on how this can be done?
You need to generate a string from your JSON and then save that string in your database as a regular string.
require 'json'
my_hash = {:hello => "goodbye"}
puts JSON.generate(my_hash) => "{\"hello\":\"goodbye\"}"
When you need to use that JSON object, you select your json string and convert it to JSON object using:
json_object = JSON.parse(string)
You can read about JSON objects here:
http://ruby-doc.org/stdlib-2.0.0/libdoc/json/rdoc/JSON.html
Tried first using 'JSON' as an data type but since SQLite had no support for 'JSON' data type it failed. Then again I performed an migration with the data type for the attribute set up as type 'string' and it started to work.
I am setting up stripe connect with the example from https://github.com/rfunduk/rails-stripe-connect-example and am running into a problem using serialize to store stripe_account_status which should be stored as an array.
This is how it should be stored (Generated from the above example link)
{"details_submitted"=>false, "charges_enabled"=>true, "transfers_enabled"=>false, "fields_needed"=>["legal_entity.first_name", "legal_entity.last_name", "legal_entity.dob.day", "legal_entity.dob.month", "legal_entity.dob.year", "legal_entity.address.line1", "legal_entity.address.city", "legal_entity.address.postal_code", "bank_account"], "due_by"=>nil}
And this is how my application is storing it
{:details_submitted=>false, :charges_enabled=>true, :transfers_enabled=>false, :fields_needed=>["legal_entity.first_name", "legal_entity.last_name", "legal_entity.dob.day", "legal_entity.dob.month", "legal_entity.dob.year", "legal_entity.address.line1", "legal_entity.address.city", "legal_entity.address.postal_code", "bank_account"], :due_by=>nil}
As far as I am concerned everything is set up the same. The only difference is that the first example uses
serialize :stripe_account_status, JSON
and my app just has
serialize :stripe_account_status
The reason for this is that when I add JSON I this error:
JSON::ParserError - 795: unexpected token at '':
I have tried finding out the JSON error including changing the config/initializers/cookies_serializer.rb to use :hybrid but this is giving me the same error.
Could someone point me into the right direction of either fixing the JSON issue OR finding a way to make sure the stripe_account_status is stored as an array correctly.
Below is the methods used to store the array:
if #account
user.update_attributes(
currency: #account.default_currency,
stripe_account_type: 'managed',
stripe_user_id: #account.id,
secret_key: #account.keys.secret,
publishable_key: #account.keys.publishable,
stripe_account_status: account_status
)
end
def account_status
{
details_submitted: account.details_submitted,
charges_enabled: account.charges_enabled,
transfers_enabled: account.transfers_enabled,
fields_needed: account.verification.fields_needed,
due_by: account.verification.due_by
}
end
Thanks I really appreciate any direction you could point me!
When you ask Rails to serialize an attribute on a model, it will default to storing the object as YAML string.
You can ask Rails to serialize differently, as you have noticed by providing a class to do the serialization e.g
serialize :stripe_account_status, JSON
The reason why this isn't working when you add it is because you presumably already have a record in the database using the YAML and so Rails can't parse this as a valid JSON string when reading from the DB. If it's just development data that you don't need, you can delete the records and then use JSON, otherwise you will need to convert the current YAML strings to JSON.
Rails will also symbolize the keys of a hash when parsing a serialized string in the database. This is the only difference between the hashes in your question and shouldn't matter in practise. Should you need String keys for some reason, you can use the #stringify_keys method on the hash provided by Rails.
I'm using Faraday to retrieve JSON encoded objects from a web service, but the web service uses horrible key strings for the key:value pairs, like 'Memo' where I want 'description' and 'projectManager' where I want 'manager'.
How can I transform the keys as they are retrieved by Faraday and inserted the result into a Ruby hash?
I'm hoping this is a solved problem, but it sure is hard to find. (I see a lot of posts asking about symbols being converted to strings in JSON.parse() ).
consider mapping the data into Virtus objects
https://github.com/solnic/virtus
class Foo
include Virtus.model
attribute :manager, String
attribute :description, String
def self.map(json)
# you only need to map the ones that differ
new(json.merge!({
manager: json["projectManager"],
description: json["Memo"]
})
end
end
then loop through your data as needed (or create a top level virtus object if it makes sense)
payload = ... # json retrieved by Faraday as array of ruby hashes
foo_list = payload.map do |item|
Foo.map(item)
end
When I assign an Array or Hash to an attribute of a Mongo document, it gets properly
serialized except for Symbols when they are used as Hash keys. Simple example:
irb>MyMongoModel.create :some_attr => {:a => [:b,:c]}
=> #<MyMongoModel _id: 4d861c34c865a1f06a000001, some_attr: {:a=>[:b, :c]}>
irb>MyMongoModel.last
=> #<MyMongoModel _id: 4d861c34c865a1f06a000001, some_attr: {"a"=>[:b, :c]}>
Please, note that some_attr is retrieved as {"a"=>[:b, :c]}, not as
{:a=>[:b, :c]}
This also happens for nested Hashes (e.g., inside of Arrays or other Hashes). Is there a way to preserve Symbols in such cases?
Solution
I'm using YAML to manually serialize some_attr - YAML.dump (or Object#to_yaml) before storing, and YAML::load after reading the attribute. YAML preserves the serialized object better. ActiveRecord is using YAML to implement its serialize class method on ActiveRecord::Base.
More than likely this has to do with the ORM you are using to provide the persistance layer for the model. You can probably wrap some_attr with a method that returns it in the form of a HashWithIndifferentAccess which you can then access with either strings or arrays. Since you are using Rails, this functionality can be activated by calling the with_indifferent_access method on the Hash object. (If you have an array of Hash objects, you'll need to call it on each one of course) The method will return the same hash, but then symbol lookups will work.
From your code:
new_hash = MyMongoModel.last.some_attr.with_indifferent_access
new_hash[:a] # Will return the same as new_hash['a']
Hope this helps!
the culprit here is the BSON serialization. when you serialize a symbol used as a key for hashes, it is actually translated to a string and when you ask it back you get the string instead of the symbol.
i'm having the same problem as you and i'm thinking of extending the Hash class to include a method to convert all the "string" keys to :symbols.
unfortunately i'm not on Rails so i cannot use the with_indifferent_access as suggested by ctcherry.
I'm not sure about preserving symbols but you can convert the strings back to symbols.
str.to_sym
Found this, works well and you have define the field as Hash:
https://github.com/mindscratch/mongoid-indifferent-access