I have an initial hash which has a structure given below
Initial Hash
initial_hash = {
`section1`:{
'person_name1':{
'city': 'City1',
'country': 'Country1'
},
'person_name2':{
'city': 'City2',
'country': 'Country2'
},
...
},
`section2`:{
'person_name12':{
'city': 'City12',
'country': 'Country12'
},
'person_name23':{
'city': 'City23',
'country': 'Country23'
},
...
}
}
Final Hash
final_hash = {
`section1`:{
'country1':{
'city': 'City1',
'person_name': 'person_name1'
},
'country2':{
'city': 'City2',
'person_name': 'person_name2'
},
...
},
`section2:{
'country12':{
'city': 'City12',
'person_name': 'person_name12'
},
'country23':{
'city': 'City23',
'person_name': 'person_name23'
},
...
}
}
As you can see that the final_hash has been restructured so as country and person_name has taken the place of each other. So far my attempt for it is as below:
My attempt:
final_hash = {}
initial_hash.each do |h|
final_hash[h[0]] = {}
final_hash[h[0]] = h[1].group_by{|x| x[1]['country']}.each{|_, v| v.map!{|h| h[1]}}
end
The above attempt helps me getting this structure:
final_hash = {
'section1':{
'country'1: {
'city': 'City1',
'country': 'Country1'
},
'country2': {
'city': 'City2',
'country': 'Country2'
},
...
},
'section2':{
'country'12: {
'city': 'City12',
'country': 'Country12'
},
'country23': {
'city': 'City23',
'country': 'Country23'
},
...
}
}
I'm not able to understand how to place the person_name in place of country. I tried to add up each to result of map! block. But no luck so far. To add to this problem, i have a json data which consist of 1000 records, so performance is a concern here.
Thanks in advance
try to use inject:
1) only inject for inner hash:
initial_hash.inject({}){ |h,(section,inner_hash)| h.merge section => inner_hash.inject({}) { |inner_h,(k,v)| inner_h.merge v.delete(:country) => v.merge(person_name: k) }}
2) use map & inject:
Hash[initial_hash.map { |section, inner_hash| [section, inner_hash.inject({}) { |inner_h, (k, v)| inner_h.merge v.delete(:country) => v.merge(person_name: k) }]}]
benchmark(for 1000):
user system total real
injects: 0.060000 0.010000 0.070000 ( 0.057551)
map&inject: 0.060000 0.000000 0.060000 ( 0.053678)
Related
I'm interning at a company right now, and I have a database that I connect to in order to get some specific data about our customers. The query is all worked out, the database is returning the data I need, but now I need to figure out how to consolidate the data into the necessary format. We are using Ruby 3.1.2 for reference.
This is the format that I receive from our database (the real data will be much larger, so I'm only using a small dataset until the logic is solid).
[{ "product_id"=>1, "customer_id"=>001 },
{ "product_id"=>2, "customer_id"=>001 },
{ "product_id"=>1, "customer_id"=>002 },
{ "product_id"=>3, "customer_id"=>002 },
{ "product_id"=>1, "customer_id"=>003 },
{ "product_id"=>2, "customer_id"=>003 },
{ "product_id"=>3, "customer_id"=>003 }]
When I get this data, I need to get a list of each distinct "customer_id" with a list of every "product_id" they have assigned to them. Example of what I need to get back below.
{001=>[1, 2], 002=>[1, 3], 003=>[1, 2, 3]}
I thought I had a solution with the line below, but it doesn't seem to work how I expected.
data.group_by(&:customer_id).transform_values { |p| p.pluck(:product_id) }
In addition to #group_by one might use #each_with_object to iteratively build the needed hash.
data.each_with_object({}) { |x, h|
h[x["customer_id"]] ||= []
h[x["customer_id"]] << x["product_id"]
}
# => {1=>[1, 2], 2=>[1, 3], 3=>[1, 2, 3]}
I would do this:
array = [{ "product_id"=>1, "customer_id"=>001 },
{ "product_id"=>2, "customer_id"=>001 },
{ "product_id"=>1, "customer_id"=>002 },
{ "product_id"=>3, "customer_id"=>002 },
{ "product_id"=>1, "customer_id"=>003 },
{ "product_id"=>2, "customer_id"=>003 },
{ "product_id"=>3, "customer_id"=>003 }]
array.group_by { |hash| hash['customer_id'] }
.transform_values { |values| values.map { |value| value['product_id'] } }
#=> { 1 => [1, 2], 2 => [1, 3], 3 => [1, 2, 3] }
The other answers are correct in their own ways but in my opinion code-readability should also always be considered and considering that factor the accepted answer looks more nearer and my solution below is also based on the same logic with the only difference that I have moved out the string keys outside the loop because the string literal inside the loop should create, in each iteration, different String instances in memory for those keys.
array = [{ "product_id"=>1, "customer_id"=>001 },
{ "product_id"=>2, "customer_id"=>001 },
{ "product_id"=>1, "customer_id"=>002 },
{ "product_id"=>3, "customer_id"=>002 },
{ "product_id"=>1, "customer_id"=>003 },
{ "product_id"=>2, "customer_id"=>003 },
{ "product_id"=>3, "customer_id"=>003 }]
transformed_data = {}
customer_id_key = "customer_id"
product_id_key = "product_id"
array.each do |h|
customer_id = h[customer_id_key]
product_id = h[product_id_key]
transformed_data[customer_id] ||= []
transformed_data[customer_id] << product_id
end
transformed_data
I want to create a nested hash using four values type, name, year, value. ie, key of the first hash will be type, value will be another hash with key name, then value of that one will be another hash with key year and value as value.
The array of objects I'm iterating looks like this:
elements = [
{
year: '2018',
items: [
{
name: 'name1',
value: 'value1',
type: 'type1',
},
{
name: 'name2',
value: 'value2',
type: 'type2',
},
]
},
{
year: '2019',
items: [
{
name: 'name3',
value: 'value3',
type: 'type2',
},
{
name: 'name4',
value: 'value4',
type: 'type1',
},
]
}
]
And I'm getting all values together using two loops like this:
elements.each do |element|
year = element.year
element.items.each |item|
name = item.name
value = item.value
type = item.type
# TODO: create nested hash
end
end
Expected output is like this:
{
"type1" => {
"name1" => {
"2018" => "value1"
},
"name4" => {
"2019" => "value4"
}
},
"type2" => {
"name2" => {
"2018" => "value2"
},
"name3" => {
"2019" => "value3"
}
}
}
I tried out some methods but it doesn't seems to work out as expected. How can I do this?
elements.each_with_object({}) { |g,h| g[:items].each { |f|
h.update(f[:type]=>{ f[:name]=>{ g[:year]=>f[:value] } }) { |_,o,n| o.merge(n) } } }
#=> {"type1"=>{"name1"=>{"2018"=>"value1"}, "name4"=>{"2019"=>"value4"}},
# "type2"=>{"name2"=>{"2018"=>"value2"}, "name3"=>{"2019"=>"value3"}}}
This uses the form of Hash#update (aka merge!) that employs a block (here { |_,o,n| o.merge(n) } to determine the values of keys that are present in both hashes being merged. See the doc for definitions of the three block variables (here _, o and n). Note that in performing o.merge(n) o and n will have no common keys, so a block is not needed for that operation.
Assuming you want to preserve the references (unlike in your desired output,) here you go:
elements = [
{
year: '2018',
items: [
{name: 'name1', value: 'value1', type: 'type1'},
{name: 'name2', value: 'value2', type: 'type2'}
]
},
{
year: '2019',
items: [
{name: 'name3', value: 'value3', type: 'type2'},
{name: 'name4', value: 'value4', type: 'type1'}
]
}
]
Just iterate over everything and reduce into the hash. On the structures of known shape is’s a trivial task:
elements.each_with_object(
Hash.new { |h, k| h[k] = Hash.new(&h.default_proc) } # for deep bury
) do |h, acc|
h[:items].each do |item|
acc[item[:type]][item[:name]][h[:year]] = item[:value]
end
end
#⇒ {"type1"=>{"name1"=>{"2018"=>"value1"},
# "name4"=>{"2019"=>"value4"}},
# "type2"=>{"name2"=>{"2018"=>"value2"},
# "name3"=>{"2019"=>"value3"}}}
I have a json object. It has multiple fields "passthrough_fields" which is unnecessary for me and I want to remove them. Is there a way to get all those attributes filtered out?
JSON:
{
"type": "playable_item",
"id": "p06s0lq7",
"urn": "urn:bbc:radio:episode:p06s0mk3",
"network": {
"id": "bbc_radio_five_live",
"key": "5live",
"short_title": "Radio 5 live",
"logo_url": "https://sounds.files.bbci.co.uk/v2/networks/bbc_radio_five_live/{type}_{size}.{format}",
"passthrough_fields": {}
},
"titles": {
"primary": "Replay",
"secondary": "Bill Shankly",
"tertiary": null,
"passthrough_fields": {}
},
"synopses": {
"short": "Bill Shankly with Sue MacGregor in 1979 - five years after he resigned as Liverpool boss.",
"medium": null,
"long": "Bill Shankly in conversation with Sue MacGregor in 1979, five years after he resigned as Liverpool manager.",
"passthrough_fields": {}
},
"image_url": "https://ichef.bbci.co.uk/images/ic/{recipe}/p06qbz1x.jpg",
"duration": {
"value": 1774,
"label": "29 mins",
"passthrough_fields": {}
},
"progress": null,
"container": {
"type": "series",
"id": "p06qbzmj",
"urn": "urn:bbc:radio:series:p06qbzmj",
"title": "Replay",
"synopses": {
"short": "Colin Murray unearths classic sports commentaries and interviews from the BBC archives.",
"medium": "Colin Murray looks back at 90 years of sport on the BBC by unearthing classic commentaries and interviews from the BBC archives.",
"long": null,
"passthrough_fields": {}
},
"activities": [],
"passthrough_fields": {}
},
"availability": {
"from": "2018-11-16T16:18:54Z",
"to": null,
"label": "Available for over a year",
"passthrough_fields": {}
},
"guidance": {
"competition_warning": false,
"warnings": null,
"passthrough_fields": {}
},
"activities": [],
"uris": [
{
"type": "latest",
"label": "Latest",
"uri": "/v2/programmes/playable?container=p06qbzmj&sort=sequential&type=episode",
"passthrough_fields": {}
}
],
"passthrough_fields": {}
}
Is there a way I can remove all those fields and store the updated json in a new variable?
You can do this recursively to tackle nested occurances of passthrough_fields, whether they're found in an array or a sub hash. Inline comments to explain things a little as it goes:
hash = JSON.parse(input) # convert the JSON to a hash
def remove_recursively(hash, *to_remove)
hash.each do |key, val|
hash.except!(*to_remove) # the heavy lifting: remove all keys that match `to_remove`
remove_recursively(val, *to_remove) if val.is_a? Hash # if a nested hash, run this method on it
if val.is_a? Array # if a nested array, loop through this checking for hashes to run this method on
val.each { |el| remove_recursively(el, *to_remove) if el.is_a? Hash }
end
end
end
remove_recursively(hash, 'passthrough_fields')
To demonstrate, with a simplified example:
hash = {
"test" => { "passthrough_fields" => [1, 2, 3], "wow" => '123' },
"passthrough_fields" => [4, 5, 6],
"array_values" => [{ "to_stay" => "I am", "passthrough_fields" => [7, 8, 9]}]
}
remove_recursively(hash, 'passthrough_fields')
#=> {"test"=>{"wow"=>"123"}, "array_values"=>[{"to_stay"=>"I am"}]}
remove_recursively(hash, 'passthrough_fields', 'wow', 'to_stay')
#=> {"test"=>{}, "array_values"=>[{}]}
This will tackle any arrays, and will dig for nested hashes however deep it needs to go.
It takes any number of fields to remove, in this case a single 'passthrough_fields'.
Hope this helps, let me know how you get on.
I think that the easiest solution would be to:
convert JSON into hash (JSON.parse(input))
use this answer to extend the functionality of Hash (save it in config/initializers/except_nested.rb)
on the hash from 1st step, call:
without_passthrough = your_hash.except_nested('passthrough_fields')
covert hash to JSON (without_passthrough.to_json)
Please keep in mind that it will work for passthrough_fields that is nested directly in hashes. In your JSON, you have the following part:
"uris" => [
{
"type"=>"latest",
"label"=>"Latest",
"uri"=>"/v2/programmes/playable?container=p06qbzmj&sort=sequential&type=episode",
"passthrough_fields"=>{}
}
]
In this case, the passthrough_fields will not be removed. You have to find a more sophisticated solution :)
You can do something like this:
def nested_except(hash, except_key)
sanitized_hash = {}
hash.each do |key, value|
next if key == except_key
sanitized_hash[key] = value.is_a?(Hash) ? nested_except(value, except_key) : value
end
sanitized_hash
end
json = JSON.parse(json_string)
sanitized = nested_except(json, 'passthrough_fields')
See example:
json = { :a => 1, :b => 2, :c => { :a => 1, :b => { :a => 1 } } }
nested_except(json, :a)
# => {:b=>2, :c=>{:b=>{}}}
This helper can easily be converted to support multiple keys to except, simply by except_keys = Array.wrap(except_key) and next if except_keys.include?(key)
I would like to get rid of the value: <value> key value inside each of the attributes in the hash. And make it like this: "total_interactions": 493.667
Below is the incorrect format followed by the expected good format I hope to achieve in json.
{
"3": {
"total_interactions": {
"value": 493.667
},
"shares": {
"value": 334
},
"comments": {
"value": 0
},
"likes": {
"value": 159.66666666666666
},
"total_documents": 6
},
"4": {
"total_interactions": {
"value": 701
},
"shares": {
"value": 300
},
"comments": {
"value": 0
},
"likes": {
"value": 401
},
"total_documents": 1
}
}
I want it to be like this:
{
"3": {
"total_interactions": 493.6666666666667,
"shares": 334,
"comments": 0,
"likes": 159.66666666666666,
"total_documents": 6
},
"4": {
"total_interactions": 701,
"shares": 300,
"comments": 0,
"likes": 401,
"total_documents": 1
}
}
Here is the code that is supposed to do this but is not working. Nothing affected. Not sure what is wrong
# the result_hash variable is the first hash with value: <value>
result_hash.each do |hash_item|
hash_item.each do |key,value_hash|
if( !value_hash.nil? )
value_hash.each do |k,v|
hash_item[key] = v
end
end
end
end
Max Williams' code is perfect for in-place. You can also do this in functional style to get a new, corrected hash:
hash.merge(hash) {|k,v| v.merge(v) {|kk,vv| vv.is_a?(Hash) && vv['value'] ? vv['value'] : vv }}
hash = {"3"=>{"total_documents"=>6, "comments"=>{"value"=>0}, "total_interactions"=>{"value"=>493.667}, "shares"=>{"value"=>334}, "likes"=>{"value"=>159.666666666667}},
"4"=>{"total_documents"=>1, "comments"=>{"value"=>0}, "total_interactions"=>{"value"=>701}, "shares"=>{"value"=>300}, "likes"=>{"value"=>401}}}
hash.each do |k,v|
v.each do |k2, v2|
if v2.is_a?(Hash) && v2["value"]
hash[k][k2] = v2["value"]
end
end
end
after this:
hash = {"3"=>{"total_documents"=>6, "comments"=>0, "total_interactions"=>493.667, "shares"=>334, "likes"=>159.666666666667},
"4"=>{"total_documents"=>1, "comments"=>0, "total_interactions"=>701, "shares"=>300, "likes"=>401}}
If you do not wish to mutate your initial hash, h,you can do this:
h.each_with_object({}) { |(k,v),g|
g[k] = v.each_with_object({}) { |(kk,vv),f|
f[kk] = (Hash === vv) ? vv[:value] : vv } }
#=> {:"3"=>{:total_interactions=>493.667,
# :shares=>334,
# :comments=>0,
# :likes=>159.66666666666666,
# :total_documents=>6},
# :"4"=>{:total_interactions=>701,
# :shares=>300,
# :comments=>0,
# :likes=>401,
# :total_documents=>1}}
Below is json I translated from ruby hash for ease of representation for this question using hash.to_json. Notice how the key range is being repeated since the values in the nested doc are different. How do I merge the ranges so that for the weight key both "gt": 2232, "lt": 4444 fall under the one hash key weight inside range. Is there some union or collapse method in ruby to sort of "compactify" hashes?
{
"must": [
{
"match": {
"status_type": "good"
}
},
{
"range": {
"created_date": {
"lte": 43252
}
}
},
{
"range": {
"created_date": {
"gt": "42323"
}
}
},
{
"range": {
"created_date": {
"gte": 523432
}
}
},
{
"range": {
"weight": {
"gt": 2232
}
}
},
{
"range": {
"weight": {
"lt": 4444
}
}
}
],
"should": [
{
"match": {
"product_age": "old"
}
}
]
}
Want to change the above to this:
{
"must": [
{
"range": {
"created_date": {
"gte": 523432,
"gt": "42323"
}
}
},
{
"range": {
"weight": {
"gt": 2232,
"lt": 4444
}
}
}
],
"should": [
{
"match": {
"product_age": "old"
}
}
]
}
I don't know of a built in way to handle something like this, but you could write a method that does something like this:
def collapse(array, key)
# Get only the hashes with :range
to_collapse = array.select { |elem| elem.has_key? key }
uncollapsed = array - to_collapse
# Get the hashes that :range points to
to_collapse = to_collapse.map { |elem| elem.values }.flatten
collapsed = {}
# Iterate through each range hash and their subsequent subhashes.
# Collapse the values into the collapsed hash as necessary
to_collapse.each do |elem|
elem.each do |k, v|
collapsed[k] = {} unless collapsed.has_key? k
v.each do |inner_key, inner_val|
collapsed[k][inner_key] = inner_val
end
end
end
[uncollapsed, collapsed].flatten
end
hash[:must] = collapse hash[:must], :range
Note that this is a specific solution that's mainly applicable to the presented problem. It only works for the hash/array depths specified here. You could probably write a recursive solution that could potentially work at any level of depth with a bit more work.