I'd like to understand how to understand the way to apply transformation (underscore in my case) on each key when parsing a JSON file with Oj.
For example in ruby/rails :
require 'oj'
Oj.optimize_rails()
def transform_keys(hash)
hash.deep_transform_keys { |k| k.to_s.underscore }
end
data = '{ "JsonKey": "JsonValue", "JsonKey2": { "JsonSubKey2": "JsonSubValue" }, "Array": ["ValueArray1","ValueArray2"] }'
data2 = '[{ "JsonKey": "JsonValue", "JsonKey2": { "JsonSubKey2": "JsonSubValue" }, "Array": ["ValueArray1","ValueArray2"] }]'
json_data = JSON.parse(data)
case json_data
when Hash
transform_keys(json_data)
when Array
json_data.map { |hash| transform_keys(hash) }
end
This do the job but it iterate twice : one during parsing and for apply the transformation.
Checking oj, I found :
::Oj::ScHandler http://www.ohler.com/oj/doc/Oj/ScHandler.html
::Oj::Saj http://www.ohler.com/oj/doc/Oj/Saj.html
Let's try :
class UnderscoreKeyHandler < ::Oj::ScHandler
def hash_start
{}
end
def hash_set(h,k,v)
h[k.to_s.underscore] = v
end
def array_start
[]
end
def array_append(a,v)
a << v
end
def add_value(v)
end
end
handler = UnderscoreKeyHandler.new
Oj.sc_parse(handler, data)
It work! Ok, let's benchmark this :
require 'oj'
Oj.optimize_rails()
hash = Hash.new
key = 'abcd'
1000.times { hash[key.succ!] = hash.keys }
json = hash.to_json; nil
def transform_keys(hash)
hash.deep_transform_keys { |k| k.to_s.underscore }
end
10000 * Benchmark.realtime {
json_data = JSON.parse(json)
case json_data
when Hash
transform_keys(json_data)
when Array
json_data.map { |hash| transform_keys(hash) }
end
} # Between 2400 and 3500
10000 * Benchmark.realtime { Oj.sc_parse(handler, json) } # Between 700 and 1600
Perfect? I seem to be but i feel insecure...
In fact i just want to override each key for a hash and not rewrite a full handler. Is there a way?
Moreover :
What is the difference between ::Oj::ScHandler and ::Oj::Saj ?
What are the risk of using a custom handler
What is the purpose of the add_value method.
How can i raise another error than the Oj::ParseError default error?
Related
I have an array of strings of this format ['config = 3', 'config_b.root.a.b.c = 13'] ;
my goal is to create the following json object from them
{
"config": 3,
"config_b": {
"root": {
"a": {
"b": {
"c": 13
}
}
}
}
}
this is my current working approach
# inputs is an array of strings
def create_config(inputs)
hash={}
inputs.each do |x|
value = x.split("=")[1]
keys = x.split("=")[0].strip.split(".")
add_item(keys,value,hash)
end
print hash
end
# recusive function for adding items
def add_item(keys,value,hash)
current = keys.shift
if keys.empty?
hash[current] = value
else
hash[current] = {}
add_item(keys,value,hash[current])
end
end
I would like to know if anyone has a better approach for solving this, Thanks
I think I have a solution.
def create_config(inputs)
inputs.map do |e|
keys, value = e.split(' = ')
keys.split('.').reverse.inject(value) { |assigned_value, key| { key => assigned_value } }
end.reduce(:merge)
end
I tried it with
['config = 3', 'config_b.root.a.b.c = 13']
and got
{"config"=>"3", "config_b"=>{"root"=>{"a"=>{"b"=>{"c"=>"13"}}}}}
I need the next hash:
x = {
params: {
user_params1: { name: "stephen", dir: "2001", dir2: nil },
user_params2: { name: "josh", dir: "jhon", dir2: nil }
}
to return a new hash of arrays like this:
x = {
params: {
user_params1: ["stephen","201", ""],
user_params2: ["josh","jhon",""]
}
Given:
x = {
params: {
user_params1: { name: "stephen", dir: "2001", dir2: nil },
user_params2: { name: "josh", dir: "jhon", dir2: nil }
}
}
Try:
x[:params] = x[:params].each_with_object({}) do |(k,v), returning|
returning[k] = v.map{|k,v| v}
end
Which will yield:
{:params=>{:user_params1=>["stephen", "2001", nil], :user_params2=>["josh", "jhon", nil]}}
If you want empty strings instead of nils (as in your example), do:
x[:params] = x[:params].each_with_object({}) do |(k,v), returning|
returning[k] = v.map{|k,v| v.to_s}
end
If you don't want to modify x, then just create a new hash and do the same:
y ={}
y[:params] = x[:params].each_with_object({}) do |(k,v), returning|
returning[k] = v.map{|k,v| v.to_s}
end
Since you're not doing anything with that k in v.map, you could just do v.values.map(&:to_s) (stolen shamelessly from Gerry's answer) - which is cleaner, IMO, but costs you one extra character(!) - and end up with:
y ={}
y[:params] = x[:params].each_with_object({}) do |(k,v), returning|
returning[k] = v.values.map(&:to_s)
end
As Sebastian points out, there is syntactic sugar for this:
y[:params] = x[:params].transform_values do |value|
# Then use one of:
# hash.values.map { |value| value.nil? ? '' : value }
# hash.values.map { |value| value ? value : '' }
# hash.values.map { |value| value || '' }
# hash.values.map(&:to_s)
end
Interestingly, if you look at the source code,
you'll see that the each_with_object and tranform_values mechanics are quite similar:
def transform_values
return enum_for(:transform_values) unless block_given?
result = self.class.new
each do |key, value|
result[key] = yield(value)
end
result
end
You could imagine this re-written as:
def transform_values
return enum_for(:transform_values) unless block_given?
each_with_object(self.class.new) do |(key, value), result|
result[key] = yield(value)
end
end
Which, at its root (IMO), is pretty much what Gerry and I came up with.
Seems to me this cat is well-skinned.
You use each_with_object (twice in case you have more thane one key on the top level); for example:
x.each_with_object({}) do |(k, v), result|
result[k] = v.each_with_object({}) do |(k1, v1), result1|
result1[k1] = v1.values.map(&:to_s)
end
end
#=> {:params=>{:user_params1=>["stephen", "2001", ""], :user_params2=>["josh", "jhon", ""]}}
I've got a class that looks like this that turns a collection into a nested array of hashes:
# variable_stack.rb
class VariableStack
def initialize(document)
#document = document
end
def to_a
#document.template.stacks.map { |stack| stack_hash(stack) }
end
private
def stack_hash(stack)
{}.tap do |hash|
hash['stack_name'] = stack.name.downcase.parameterize.underscore
hash['direction'] = stack.direction
hash['boxes'] = stack.boxes.indexed.map do |box|
box_hash(box)
end.reverse_if(stack.direction == 'up') # array extensions
end.delete_if_key_blank(:boxes) # hash extensions
end
def box_hash(box)
{}.tap do |hash|
hash['box'] = box.name.downcase.parameterize.underscore
hash['content'] = box.template_variables.indexed.map do |var|
content_array(var)
end.join_if_any?
end.delete_if_key_blank(:content)
end
def content_array(var)
v = #document.template_variables.where(master_id: var.id).first
return unless v
if v.text.present?
v.text
elsif v.photo_id.present?
v.image.uploaded_image.url
else
''
end
end
end
# array_extensions.rb
class Array
def join_if_any?
join("\n") if size.positive?
end
def reverse_if(boolean)
reverse! if boolean
end
end
# hash_extensions.rb
class Hash
def delete_if_key_blank(key)
delete_if { |_, _| key.to_s.blank? }
end
end
This method is supposed to return a hash that looks like this:
"stacks": [
{
"stack_name": "stack1",
"direction": "down",
"boxes": [
{
"box": "user_information",
"content": "This is my name.\n\nThis is my phone."
}
},
{
"stack_name": "stack2",
"direction": "up",
"boxes": [
{
"box": "fine_print",
"content": "This is a test.\n\nYeah yeah."
}
]
}
Instead, often the boxes key is null:
"stacks": [
{
"stack_name": "stack1",
"direction": "down",
"boxes": null
},
{
"stack_name": "stack2",
"direction": "up",
"boxes": [
{
"box": "fine_print",
"content": "This is a test.\n\nYeah yeah."
}
]
}
I suspect it's because I can't "single-line" adding to arrays in Rails 5 (i.e., they're frozen). The #document.template.stacks is an ActiveRecord collection.
Why can't I map records in those collections into hashes and add them to arrays like hash['boxes']?
The failing test
APIDocumentV3 Instance methods #stacks has the correct content joined and indexed
Failure/Error:
expect(subject.stacks.first['boxes'].first['content'])
.to include(document.template_variables.first.text)
expected "\n" to include "#1"
Diff:
## -1,2 +1 ##
-#1
The presence of \n means the join method works, but it shouldn't join if the array is empty. What am I missing?
reverse_if returns nil if the condition is false. Consider this:
[] if false #=> nil
You could change it like this:
def reverse_if(condition)
condition ? reverse : self
end
delete_if_key_blank doesn't look good for me. It never deletes anything.
Disclaimer. I don't think it's a good idea to extend standard library.
So thanks to Danil Speransky I solved this issue, although what he wrote doesn't quite cover it.
There were a couple of things going on here and I solved the nil arrays with this code:
hash['boxes'] = stack.boxes.indexed.map do |box|
box_hash(box) unless box_hash(box)['content'].blank?
end.reverse_if(stack.direction == 'up').delete_if_blank?
end
That said, I'm almost certain my .delete_if_blank? extension to the Array class isn't helping at all. It looks like this, FYI:
class Array
def delete_if_blank?
delete_if(&:blank?)
end
end
I solved it by thowing the unless box_hash(box)['content'].blank? condition on the method call. It ain't pretty but it works.
I've got a class that looks like this:
class VariableStack
def initialize(document)
#document = document
end
def to_array
#document.template.stacks.each { |stack| stack_hash stack }
end
private
def stack_hash(stack)
stack_hash = {}
stack_hash['stack_name'] = stack.name
stack_hash['boxes'] = [stack.boxes.each { |box| box_hash box }]
stack_hash
end
def box_hash(box)
box_hash = {}
content = []
box.template_variables.indexed.each { |var| content << content_array(var) }
content.delete_if(&:blank?)
box_hash.store('content', content.join("\n"))
return if box_hash['content'].empty?
box_hash
end
def content_array(var)
v = #document.template_variables.where(master_id: var.id).first
return unless v
if v.text.present?
v.format_text
elsif v.photo_id.present?
v.image.uploaded_image.url
end
end
end
The document I'm testing with has two template_variables so the desired result should be a nested hash like so:
Instead I'm getting this result:
=> [#<Stack id: 1, name: "User information">]
i.e., I'm not getting the boxes key nor it's nested content. Why isn't my method looping through the box_hash and content fields?
That's because the to_array method uses each method, which returns the object it's been called on (in this case #document.template.stacks)
Change it to the map and you may get the desired result:
def to_array
#document.template.stacks.map { |stack| stack_hash stack }
end
I have the following array of hashes and I want to use transform_keys to strip the beginning of each key using a regex:
array_of_hashes = [{"a_0_abc"=>"1",
"a_0_def"=>"1",
"a_0_hij"=>"1",},
{"a_1_abc”=>"2",
"a_1_def"=>"2",
"a_1_hij"=>"2"}]
and I want the following:
transformed_hash_keys = [{"abc"=>"1",
"def"=>"1",
"hij"=>"1",},
{"abc"=>"2",
"def"=>"2",
"hij"=>"2"}]
I have the following method but it results in array_of_hashes instead of transformed_hash_keys:
def strip
s = array_of_hashes.each { |hash| hash.transform_keys { |key| key.sub(/^a_(\d+)_/, '') } }
end
Can anyone tell me what I'm doing wrong in this method?
transform_keys doesn't operate in place and each returns the original iterator, not the result of the block.
You could do what you want with map instead of each.
def strip
s = array_of_hashes.map { |hash| hash.transform_keys { |key| key.sub(/^a_(\d+)_/, '') } }
end
Or, you could use transform_keys! to modify the contents of array_of_hashes
def strip
s = array_of_hashes.each { |hash| hash.transform_keys! { |key| key.sub(/^a_(\d+)_/, '') } }
end
Here's a pure Ruby solution.
arr = [{ "a_0_abc"=>"1", "a_0_def"=>"1", "a_0_hij"=>"1" },
{ "a_1_abc"=>"2", "a_1_def"=>"2", "a_1_hij"=>"2" }]
arr.map { |h| h.map { |k,v| [k[/[[:alpha:]]+\z/], v] }.to_h }
#=> [{"abc"=>"1", "def"=>"1", "hij"=>"1"}, {"abc"=>"2", "def"=>"2", "hij"=>"2"}]
or
arr.map { |h| h.each_with_object({}) { |(k,v),g| g[k[/[[:alpha:]]+\z/]] = v } }
# => [{"abc"=>"1", "def"=>"1", "hij"=>"1"}, {"abc"=>"2", "def"=>"2", "hij"=>"2"}]
This lets you do any kind of transformation on the keys just passing a block to it:
def strip_keys(object)
deep_transform_keys_in_object!(object) { |key| key.sub(/^a_(\d+)_/, '') }
end
def deep_transform_keys_in_object!(object, &block)
case object
when Hash
object.keys.each do |key|
value = object.delete(key)
object[yield(key)] = deep_transform_keys_in_object!(value, &block)
end
object
when Array
object.map! { |e| deep_transform_keys_in_object!(e, &block) }
else
object
end
end