Related
I am working on requirements we have data in hash around 100+ keys. we need to generate CSV file as per user-defined header with some transformation, we may end up having 100+ template
Main changes will be
1) Change column name such as Fname –> First name
2) Data transformation like Full name – > First name + Last name (adding 2 column)
3) Fixing the position of a column – Fname should be at 35 positions etc.
please suggest is it possible to define declarative way or any gem available. Can you let me know any design pattern we can apply here?
Some sample scenarios
I have input like this with many columns (100+)
[ {:employee_id=>"001", :first_name=>"John",:last_name=>"Dee" :date_of_birth=>"10/10/1983", :salary=>"100000",:bounus =>"50000",......},
{:employee_id=>"002", :first_name=>"Alex",:last_name=>"Peck" :date_of_birth=>"11/01/1988", :salary=>"120000",:bounus =>"70000", .........},
]
Some customer need CSV as
Employee ID, First Name, Last Name, Date of birth, Salary, Bonus
001,John,Dee,10/10/1983,100000,50000,...
002,Alex,Peck,11/01/1988,120000,70000,...
Others (only header change)
ID, FName, LName, Dob, Salary, Bounus
001,John,Dee,10/10/1983,100000,50000,...
002,Alex,Peck,11/01/1988,120000,70000,...
another (merge of colum FName, LName -> Fullname)
ID, Fullname, Dob, Salary, Bounus
001,John Dee,10/10/1983,100000,50000,...
002,Alex Peck,11/01/1988,120000,70000,...
anothers (merge of column Salary, Bonus -> Salary+ Bonus)
ID, FName, LName, Dob, Salary
001,John,Dee,10/10/1983,150000,...
002,Alex,Peck,11/01/1988,190000,...
anothers ( column order changed also insted of Dob need age)
FName, LName, ID, age, Salary
John,Dee,001,36,150000,...
Alex,Peck,003,32,190000,...
Like many variations with the same input
Thanks for help
What you need is the presenter design pattern.
Your controller will request the data and store it in a local variable, and then your will have to load a presenter for your client passing it the data variable.
In response you'll get the final CSV to return to the client.
Let's say you clients have uniq codes, so that a Client model instance has a code attribute which is a string.
So your controller would looks like this:
app/controllers/exports_controller.rb
class ExportsController < ApplicationController
def export
data = MyService.fetchData # <== data contains the data you gave as an example
# Gets the right presenter, initialise it, and build the CSV
csv = PresenterFactory.for(current_client).new(data).present
respond_to do |format|
format.html
format.csv { send_data csv, filename: "export-name-for-#{current_client.code}.csv" }
end
end
end
The PresenterFactory class would be something like that:
app/models/presenter_factory.rb
class PresenterFactory
def self.for(client)
# For client with code "ABCD" it will return Presenters::Abcd class
"Presenters::#{client.code.capitalize}".constantize
end
end
The factory return the client's presenter class
And here is an example for a client's presenter class, for a client having the code ABCD:
app/models/presenters/abcd.rb
module Presenters
class Abcd
def initialize(data)
#data = data
end
def present
CSV.generate(headers: true) do |csv|
# Here is the client's specific CSV header
csv << [
'Employee ID',
'First Name',
# ...
]
#data.each do |row|
# Here is the client's specific CSV row
csv << [
row[:employee_id],
row[:first_name],
# ...
]
end
end
end
end
end
You can achieve your objective by constructing a transformation hash whose keys are the names of the columns in the desired CSV file, in order, and whose values are procs, which when called with an argument equal to an element of the given array of hashes, returns an element to be written in a row of the CSV file in the column corresponding to the key.
Code
require 'csv'
def construct_csv(fname, arr, transform)
CSV.open(fname, "wb") do |csv|
keys = transform.keys
csv << keys
arr.each { |h| csv << keys.map { |k| transform[k].call(h) } }
end
end
Examples
I will now illustrate how this method is used with various transformations.
Common data
arr = [{:employee_id=>"001", :first_name=>"John", :last_name=>"Dee",
:date_of_birth=>"10/10/1983", :salary=>"100000", :bonus=>"50000" },
{:employee_id=>"002", :first_name=>"Alex", :last_name=>"Peck",
:date_of_birth=>"11/01/1988", :salary=>"120000", :bonus=>"70000" }]
FName = 'temp.csv'
Write a CSV file with the same keys, in the same order, and the same values
keys = arr.first.keys
#=> [:employee_id, :first_name, :last_name, :date_of_birth, :salary, :bonus]
transform = keys.each_with_object({}) { |k,g| g[k] = ->(h) { h[k] } }
#=> {:employee_id=>#<Proc:0x00005bd270a0e710#(irb):451 (lambda)>,
# :first_name=>#<Proc:0x00005bd270a13260#(irb):451 (lambda)>,
# ...
# :bonus=>#<Proc:0x00005bd270a19cc8#(irb):451 (lambda)>}
construct_csv(FName, arr, transform)
Let's see what was written.
puts File.read(FName)
employee_id,first_name,last_name,date_of_birth,salary,bonus
001,John,Dee,10/10/1983,100000,50000
002,Alex,Peck,11/01/1988,120000,70000
Write a CSV file with the columns reordered1
col_order = [:last_name, :first_name, :employee_id, :salary, :bonus,
:date_of_birth]
keys = arr.first.keys
order_map = col_order.each_with_object({}) { |k,h| h[k] = keys.index(k) }
#=> {:last_name=>2, :first_name=>1, :employee_id=>0, :salary=>4,
# :bonus=>5, :date_of_birth=>3}
transform = col_order.each_with_object({}) { |k,g|
g[k] = ->(h) { h[keys[order_map[k]]] } }
#=> {:last_name=>#<Proc:0x00005bd270f8e5a0#(irb):511 (lambda)>,
# :first_name=>#<Proc:0x00005bd270f8e550#(irb):511 (lambda)>,
# ...
# :date_of_birth=>#<Proc:0x00005bd270f8e3c0#(irb):511 (lambda)>}
construct_csv(FName, arr, transform)
puts File.read(FName)
last_name,first_name,employee_id,salary,bonus,date_of_birth
Dee,John,001,100000,50000,10/10/1983
Peck,Alex,002,120000,70000,11/01/1988
Write a CSV file with a subset of keys, renamed and reordered
keymap = { :FirstName=>:first_name, :LastName=>:last_name, :ID=>:employee_id,
:Salary=>:salary, :Bonus=>:bonus }
transform = keymap.each_with_object({}) { |(new,old),g| g[new] = ->(h) { h[old] } }
#=> {:FirstName=>#<Proc:0x00005bd270d50298#(irb):391 (lambda)>,
# :LastName=>#<Proc:0x00005bd270d50220#(irb):391 (lambda)>,
# ...
# :Bonus=>#<Proc:0x00005bd270d830f8#(irb):391 (lambda)>}
construct_csv(FName, arr, transform)
puts File.read(FName)
FirstName,LastName,ID,Salary,Bonus
John,Dee,001,100000,50000
Alex,Peck,002,120000,70000
Write a CSV file after removing keys and adding keys whose values are computed
keys_to_remove = [:first_name, :last_name]
keys_to_add = [:full_name, :compensation]
keys = arr.first.keys + keys_to_add - keys_to_remove
#=> [:employee_id, :date_of_birth, :salary, :bonus, :full_name,
# :compensation]
transform = keys.each_with_object({}) do |k,h|
h[k] =
case k
when :full_name
->(h) { h[:first_name] + " " + h[:last_name] }
when :compensation
->(h) { h[:salary].to_i + h[:bonus].to_i }
else
->(h) { h[k] }
end
end
#=> {:employee_id=>#<Proc:0x00005bd271001000#(irb):501 (lambda)>,
# :date_of_birth=>#<Proc:0x00005bd271000f88#(irb):501 (lambda)>,
# :salary=>#<Proc:0x00005bd271000f10#(irb):501 (lambda)>,
# :bonus=>#<Proc:0x00005bd271000ec0#(irb):501 (lambda)>,
# :full_name=>#<Proc:0x00005bd271000e20#(irb):497 (lambda)>,
# :compensation=>#<Proc:0x00005bd271000dd0#(irb):499 (lambda)>}
construct_csv(FName, arr, transform)
puts File.read(FName)
employee_id,date_of_birth,salary,bonus,full_name,compensation
001,10/10/1983,100000,50000,John Dee,150000
002,11/01/1988,120000,70000,Alex Peck,190000
1. I don't understand the reason for doing this but it was mentioned as a possible requirement.
I do have an array with orders, each with a date. Like:
[
#<Order id: 1, date: '2019-10-07'>,
#<Order id: 2, date: '2019-10-08'>,
#<Order id: 3, date: '2019-10-10'>,
#<Order id: 4, date: '2019-10-10'>,
#<Order id: 5, date: '2019-10-12'>
]
I want to display it like this:
2019-10-05:
2019-10-06:
2019-10-07: id 1
2019-10-08: id 2
2019-10-09:
2019-10-10: id 3, id 4
2019-10-11:
2019-10-12: id 5
2019-10-13:
What is the best way to do this?
I can think of the following options:
date_range.each do ... and check if there are any corresponding orders on that date.
First sort the array of orders, then do orders.each do ... and check if there are any dates skipped.
Is there some 3rd way, that is walking through both arrays simultaneously? Like starting with the dates, when there is a corresponding order, start continue with the orders until there is a new date?
Similar to what Michael Kohl and arieljuod describe in their answers. First group your dates based on date, then loop through the dates and grab the groups that are relevant.
# mock
orders = [{id: 1, date: '2019-10-07'}, {id: 2, date: '2019-10-08'}, {id: 3, date: '2019-10-10'}, {id: 4, date: '2019-10-10'}, {id: 5, date: '2019-10-12'}]
orders.map!(&OpenStruct.method(:new))
# solution
orders = orders.group_by(&:date)
orders.default = []
date_range = Date.new(2019, 10, 5)..Date.new(2019, 10, 13)
date_range.map(&:iso8601).each do |date|
ids = orders[date].map { |order| "id: #{order.id}" }.join(', ')
puts "#{date}: #{ids}"
end
# 2019-10-05:
# 2019-10-06:
# 2019-10-07: id: 1
# 2019-10-08: id: 2
# 2019-10-09:
# 2019-10-10: id: 3, id: 4
# 2019-10-11:
# 2019-10-12: id: 5
# 2019-10-13:
#=> ["2019-10-05", "2019-10-06", "2019-10-07", "2019-10-08", "2019-10-09", "2019-10-10", "2019-10-11", "2019-10-12", "2019
I'd start with something like this:
Group the array of orders by date: lookup = orders.group_by(:date)
Iterate over your date range, use date as key into lookup, so at least you don't need to traverse the orders array repeatedly.
I would do a mix of both:
# rearange your array of hashes into a hash with [date, ids] pairs
orders_by_date = {}
orders.each do |id, date|
orders_by_date[date] ||= []
orders_by_date[date] << id
end
# iterate over the range and check if the previous hash has the key
date_range.each do |date|
date_s = date.strftime('%Y-%m-%d')
date_ids = orders_by_date.fetch(date_s, []).map { |x| "id: #{x}" }.join(', ')
puts "#{date_s}: #{date_ids}"
end
Try group_by. You can find the documentation at https://apidock.com/ruby/Enumerable/group_by
grouped_orders = orders.group_by{|ords| ords[:date]}
(start_date..end_date).each do |order_date|
puts order_date
grouped_orders.fetch(order_date).map{|m| puts m.id}
end
Data
We are given an array of instances of the class Order:
require 'date'
class Order
attr_reader :id, :date
def initialize(id,date)
#id = id
#date = date
end
end
arr = ['2019-10-07', '2019-10-08', '2019-10-10', '2019-10-10', '2019-10-12'].
map.each.with_index(1) { |s,i| Order.new(i, Date.iso8601(s)) }
#=> [#<Order:0x00005a49d68ad8b8 #id=1,
# #date=#<Date: 2019-10-07 ((2458764j,0s,0n),+0s,2299161j)>>,
# #<Order:0x00005a49d68ad6d8 #id=2,
# #date=#<Date: 2019-10-08 ((2458765j,0s,0n),+0s,2299161j)>>,
# #<Order:0x00005a49d68ad3b8 #id=3,
# #date=#<Date: 2019-10-10 ((2458767j,0s,0n),+0s,2299161j)>>,
# #<Order:0x00005a49d68ad138 #id=4,
# #date=#<Date: 2019-10-10 ((2458767j,0s,0n),+0s,2299161j)>>,
# #<Order:0x00005a49d68aceb8 #id=5,
# #date=#<Date: 2019-10-12 ((2458769j,0s,0n),+0s,2299161j)>>]
and start and end dates:
start_date = '2019-10-05'
end_date = '2019-10-13'
Assumption
I assume that:
Date.iso8601(start_date) <= arr.first.date &&
arr.first.date <= arr.last.date &&
arr.last.date <= Date.iso8601(end_date)
#=> true
There is no need for the elements of arr to be sorted by date.
Code
h = (start_date..end_date).each_with_object({}) { |d,h| h[d] = d + ':' }
arr.each do |inst|
date = inst.date.strftime('%Y-%m-%d')
h[date] += "#{h[date][-1] == ':' ? '' : ','} id #{inst.id}"
end
h.values
#=> ["2019-10-05:",
# "2019-10-06:",
# "2019-10-07: id 1",
# "2019-10-08: id 2",
# "2019-10-09:",
# "2019-10-10: id 3, id 4",
# "2019-10-11:",
# "2019-10-12: id 5",
# "2019-10-13:"]
Explanation
The first step is to construct the hash h:
h = (start_date..end_date).each_with_object({}) { |d,h| h[d] = d + ':' }
#=> {"2019-10-05"=>"2019-10-05:", "2019-10-06"=>"2019-10-06:",
# "2019-10-07"=>"2019-10-07:", "2019-10-08"=>"2019-10-08:",
# "2019-10-09"=>"2019-10-09:", "2019-10-10"=>"2019-10-10:",
# "2019-10-11"=>"2019-10-11:", "2019-10-12"=>"2019-10-12:",
# "2019-10-13"=>"2019-10-13:"}
Now we will loop through the elements inst (instances of Order) of arr, and for each will alter the value of the key in h that equals inst.date converted to a string:
arr.each do |inst|
date = inst.date.strftime('%Y-%m-%d')
h[date] += "#{h[date][-1] == ':' ? '' : ','} id #{inst.id}"
end
Resulting in:
h #=> {"2019-10-05"=>"2019-10-05:",
# "2019-10-06"=>"2019-10-06:",
# "2019-10-07"=>"2019-10-07: id 1",
# "2019-10-08"=>"2019-10-08: id 2",
# "2019-10-09"=>"2019-10-09:",
# "2019-10-10"=>"2019-10-10: id 3, id 4",
# "2019-10-11"=>"2019-10-11:",
# "2019-10-12"=>"2019-10-12: id 5",
# "2019-10-13"=>"2019-10-13:"}
All that remains is to extract the values of the hash h:
h.values
#=> ["2019-10-05:",
# "2019-10-06:",
# "2019-10-07: id 1",
# "2019-10-08: id 2",
# "2019-10-09:",
# "2019-10-10: id 3, id 4",
# "2019-10-11:",
# "2019-10-12: id 5",
# "2019-10-13:"]
this is my first try using ruby, this is probably a simple problem, I have been stuck for an hour now, I have a ruby array with some objects in it, and I want that array to sort by the first character in the objects name property (which I make sure is always a number.)
the names are similar to:
4This is an option
3Another option
1Another one
0Another one
2Second option
I have tried:
objectArray.sort_by {|a| a.name[0].to_i}
objectArray.sort {|a,b| a.name[0].to_i <=> b.name.to_i}
In both cases my arrays sorting doesnt change.. (also used the destructive version of sort! and sort_by!)
I looped through the array like this:
objectArray.each do |test|
puts test.name[0].to_i
puts "\n"
end
and sure enough I see the integer value it should have
Tried with an array like this one:
[
{ id: 5, name: "4rge" },
{ id: 7, name: "3gerg" },
{ id: 0, name: "0rege"},
{ id: 2, name: "2regerg"},
{ id: 8, name: "1frege"}
]
And I don't have any issues with #sagarpandya82's answer:
arr.sort_by { |a| a[:name][0] }
# => [{:id=>0, :name=>"0rege"}, {:id=>8, :name=>"1frege"}, {:id=>2, :name=>"2regerg"}, {:id=>7, :name=>"3gerg"}, {:id=>5, :name=>"4rge"}]
Just sort by name. Since strings are sorted in lexicographic order, the objects will be sorted by name's first character :
class MyObject
attr_reader :name
def initialize(name)
#name = name
end
def to_s
"My Object : #{name}"
end
end
names = ['4This is an option',
'3Another option',
'1Another one',
'0Another one',
'2Second option']
puts object_array = names.map { |name| MyObject.new(name) }
# My Object : 4This is an option
# My Object : 3Another option
# My Object : 1Another one
# My Object : 0Another one
# My Object : 2Second option
puts object_array.sort_by(&:name)
# My Object : 0Another one
# My Object : 1Another one
# My Object : 2Second option
# My Object : 3Another option
# My Object : 4This is an option
If you want, you could also define MyObject#<=> and get the correct sorting automatically :
class MyObject
def <=>(other)
name <=> other.name
end
end
puts object_array.sort
# My Object : 0Another one
# My Object : 1Another one
# My Object : 2Second option
# My Object : 3Another option
# My Object : 4This is an option
I have the following code which takes a hash and turns all the values in to strings.
def stringify_values obj
#values ||= obj.clone
obj.each do |k, v|
if v.is_a?(Hash)
#values[k] = stringify_values(v)
else
#values[k] = v.to_s
end
end
return #values
end
So given the following hash:
{
post: {
id: 123,
text: 'foobar',
}
}
I get following YAML output
--- &1
:post: *1
:id: '123'
:text: 'foobar'
When I want this output
---
:post:
:id: '123'
:text: 'foobar'
It looks like the object has been flattened and then been given a reference to itself, which causes Stack level errors in my specs.
How do I get the desired output?
A simpler implementation of stringify_values can be - assuming that it is always a Hash. This function makes use of Hash#deep_merge method added by Active Support Core Extensions - we merge the hash with itself, so that in the block we get to inspect each value and call to_s on it.
def stringify_values obj
obj.deep_merge(obj) {|_,_,v| v.to_s}
end
Complete working sample:
require "yaml"
require "active_support/core_ext/hash"
def stringify_values obj
obj.deep_merge(obj) {|_,_,v| v.to_s}
end
class Foo
def to_s
"I am Foo"
end
end
h = {
post: {
id: 123,
arr: [1,2,3],
text: 'foobar',
obj: { me: Foo.new}
}
}
puts YAML.dump (stringify_values h)
#=>
---
:post:
:id: '123'
:arr: "[1, 2, 3]"
:text: foobar
:obj:
:me: I am Foo
Not sure what is the expectation when value is an array, as Array#to_s will give you array as a string as well, whether that is desirable or not, you can decide and tweak the solution a bit.
There are two issues. First: the #values after the first call would always contain an object which you cloned in the first call, so in the end you will always receive a cloned #values object, no matter what you do with the obj variable(it's because of ||= operator in your call). Second: if you remove it and will do #values = obj.clone - it would still return incorrect result(deepest hash), because you are overriding existing variable call after call.
require 'yaml'
def stringify_values(obj)
temp = {}
obj.each do |k, v|
if v.is_a?(Hash)
temp[k] = stringify_values(v)
else
temp[k] = v.to_s
end
end
temp
end
hash = {
post: {
id: 123,
text: 'foobar',
}
}
puts stringify_values(hash).to_yaml
#=>
---
:post:
:id: '123'
:text: foobar
If you want a simple solution without need of ActiveSupport, you can do this in one line using each_with_object:
obj.each_with_object({}) { |(k,v),m| m[k] = v.to_s }
If you want to modify obj in place pass obj as the argument to each_with_object; the above version returns a new object.
If you are as aware of converting values to strings, I would go with monkeypatching Hash class:
class Hash
def stringify_values
map { |k, v| [k, Hash === v ? v.stringify_values : v.to_s] }.to_h
end
end
Now you will be able to:
require 'yaml'
{
post: {
id: 123,
text: 'foobar'
},
arr: [1, 2, 3]
}.stringify_values.to_yaml
#⇒ ---
# :post:
# :id: '123'
# :text: foobar
# :arr: "[1, 2, 3]"
In fact, I wonder whether you really want to scramble Arrays?
I have a YAML file containing:
cat:
name: Cat
description: catlike reflexes
dog:
name: Dog
description: doggy breath
I want to parse it and break the description into key1 and key2 like so:
cat:
name: Cat
description: catlike reflexes
info:
key1: catlike
key2: reflexes
dog:
name: Dog
description: doggy breath
info:
key1: doggy
key2: breath
But, for some reason, I cannot do this correctly. What I've tried so far are variations of the code below, which I think I'm overcomplicating:
# to get the original file's data
some_data = YAML.load(File.open("#{Rails.root}/config/some_data.yml"))
new_data = some_data.collect do |old_animal|
animal = old_animal.second
if animal && animal["description"]
new_blocks = Hash.new
blocks = animal["description"].split(" ")
new_blocks["key1"] = blocks.first
new_blocks["key2"] = blocks.second
animal["info"] = new_blocks
end
old_animal.second = animal
old_animal
end
# to write over the original file
File.write("#{Rails.root}/config/some_data.yml", new_data.to_yaml)
You don't say whether or not you can have multiple words in the description, but it's kind of common-sense you would, so I'd do something like this:
require 'yaml'
data = YAML.load(<<EOT)
cat:
name: Cat
description: catlike reflexes rules
dog:
name: Dog
description: doggy breath
EOT
data # => {"cat"=>{"name"=>"Cat", "description"=>"catlike reflexes rules"}, "dog"=>{"name"=>"Dog", "description"=>"doggy breath"}}
At this point the data from the YAML file is loaded into a hash. Iterate over each hash key/value pair:
data.each do |(k, v)|
descriptions = v['description'].split
keys = descriptions.each_with_object([]) { |o, m| m << "key#{(m.size + 1)}" }
hash = keys.each_with_object({}) { |o, m| m[o] = descriptions.shift }
data[k]['info'] = hash
end
This is what we got back:
data # => {"cat"=>{"name"=>"Cat", "description"=>"catlike reflexes rules", "info"=>{"key1"=>"catlike", "key2"=>"reflexes", "key3"=>"rules"}}, "dog"=>{"name"=>"Dog", "description"=>"doggy breath", "info"=>{"key1"=>"doggy", "key2"=>"breath"}}}
And what it'd look like if it was output:
puts data.to_yaml
# >> ---
# >> cat:
# >> name: Cat
# >> description: catlike reflexes rules
# >> info:
# >> key1: catlike
# >> key2: reflexes
# >> key3: rules
# >> dog:
# >> name: Dog
# >> description: doggy breath
# >> info:
# >> key1: doggy
# >> key2: breath
each_with_object is similar to inject but a little cleaner to use because it doesn't require we return the object we're accumulating into.