Can you help me to implement one Accumulator from List of maps?.
[
%{
score: 1,
name: "Javascript",
},
%{
score: 2,
name: "Elixir",
},
%{
score: 10,
name: "Elixir",
}
]
The result should be:
[
%{
score: 12,
name: "Elixir",
},
%{
score: 1,
name: "Javascript",
}
]
I will appreciate your suggestion.
Regards
Assuming your original list is stored in input local variable, one might start with Enum.reduce/3 using Map.update/4 as a reducer.
Enum.reduce(input, %{}, fn %{score: score, name: name}, acc ->
Map.update(acc, name, score, & &1 + score)
end)
#⇒ %{"Elixir" => 12, "Javascript" => 1}
Whether you insist on having a list of maps as a result (which is way less readable IMSO,) go further and Enum.map/2 the result:
Enum.map(%{"Elixir" => 12, "Javascript" => 1}, fn {name, score} ->
%{name: name, score: score}
end)
#⇒ [%{name: "Elixir", score: 12},
# %{name: "Javascript", score: 1}]
To sum it up:
input
|> Enum.reduce(%{}, fn %{score: score, name: name}, acc ->
Map.update(acc, name, score, & &1 + score)
end)
|> Enum.map(& %{name: elem(&1, 0), score: elem(&1, 1)})
#⇒ [%{name: "Elixir", score: 12},
# %{name: "Javascript", score: 1}]
Sidenote: maps in erlang (and, hence, in elixir) are not ordered. That means, if you want the resulting list to be sorted by name, or by score, you should explicitly Enum.sort/2 it:
Enum.sort(..., & &1.score > &2.score)
#⇒ [%{name: "Elixir", score: 12},
# %{name: "Javascript", score: 1}]
A simple way could be to use Enum.group_by/3 to group the items by name, then Enum.sum/1 to sum the scores:
list
|> Enum.group_by(& &1.name, & &1.score)
|> Enum.map(fn {name, score} -> %{name: name, score: Enum.sum(score)} end)
Output:
[%{name: "Elixir", score: 12}, %{name: "Javascript", score: 1}]
If you were looking to create & use a more generalized solution, you could create your own Merger module.
defmodule Merger do
def merge_by(enumerable, name_fun, merge_fun) do
enumerable
|> Enum.group_by(name_fun)
|> Enum.map(fn {_name, items} -> Enum.reduce(items, merge_fun) end)
end
end
list = [
%{score: 1, name: "Javascript"},
%{score: 2, name: "Elixir"},
%{score: 10, name: "Elixir"}
]
Merger.merge_by(list, & &1.name, &%{&1 | score: &1.score + &2.score})
# => [%{name: "Elixir", score: 12}, %{name: "Javascript", score: 1}]
Related
Suppose we have two arrays:
foo = [1, 2] # can be mapped in [{id: 1}, {id: 2}]
bar = [{id: 2}, {id: 4}]
As a result I need to get following array:
res = [ [1, nil], [2, {id: 2}], [nil, {id: 4}] ]
# Is also ok [ [{id: 1}, nil], [{id: 2}, {id: 2}], [nil, {id: 4}] ]
I there any standard Ruby method to get such combinations? At first I'm looking for Ruby way, elegant, short solution.
In other words, I need to get an array diff map, which shows absent and newly added elements.
The elements:
elements = (foo + bar.map{ |h| h.values.first }).uniq
#=> [1, 2, 4]
and the combinations:
elements.map { |i| [foo.include?(i) ? i : nil, bar.include?({id: i}) ? {id: i} : nil] }
#=> [[1, nil], [2, {:id=>2}], [nil, {:id=>4}]]
Or as Sebastian suggested, you can also use #find instead of #includes?:
elements.map { |i| [foo.find { |e| e == i }, bar.find { |e| e == { id: i } }] }
I would transform the two arrays to lookup hashes. Then collect all the ids to iterate. Map the ids into the foo and bar values, then zip them together.
foo_lookup = foo.to_h { |id| [id, {id: id}] } # change `{id: id}` into `id` for the other output
bar_lookup = bar.to_h { |item| [item[:id], item] }
ids = foo_lookup.keys | bar_lookup.keys
res = ids.map(&foo_lookup).zip(ids.map(&bar_lookup))
#=> [[{:id=>1}, nil], [{:id=>2}, {:id=>2}], [nil, {:id=>4}]]
Just another option.
foo = [1, 2]
bar = [{id: 2}, {id: 4}]
(foo + bar).group_by { |e| e.is_a?(Hash) ? e[:id] : e }.values.map { |e| e.size == 1 ? e << nil : e }
#=> [[1, nil], [2, {:id=>2}], [{:id=>4}, nil]]
The first part returns
(foo + bar).group_by { |e| e.is_a?(Hash) ? e[:id] : e }.values
#=> [[1], [2, {:id=>2}], [{:id=>4}]]
If you need to sort the sub arrays, just map using Array#unshift or Array#push (prepend or append nil) depending in the element already there.
I am trying to sum array of array and get average at the same time. The original data is in the form of JSON. I have to parse my data to array of array in order to render the graph. The graph does not accept array of hash.
I first convert the output to JSON using the definition below.
ActiveSupport::JSON.decode(#output.first(10).to_json)
And the result of the above action is shown below.
output =
[{"name"=>"aaa", "job"=>"a", "pay"=> 2, ... },
{"name"=>"zzz", "job"=>"a", "pay"=> 4, ... },
{"name"=>"xxx", "job"=>"a", "pay"=> 6, ... },
{"name"=>"yyy", "job"=>"a", "pay"=> 8, ... },
{"name"=>"aaa", "job"=>"b", "pay"=> 2, ... },
{"name"=>"zzz", "job"=>"b", "pay"=> 4, ... },
{"name"=>"xxx", "job"=>"b", "pay"=> 6, ... },
{"name"=>"yyy", "job"=>"b", "pay"=> 10, ... },
]
Then I retrieved the job and pay by converting to array of array.
ActiveSupport::JSON.decode(output.to_json).each { |h|
a << [h['job'], h['pay']]
}
The result of the above operation is as below.
a = [["a", 2], ["a", 4], ["a", 6], ["a", 8],
["b", 2], ["b", 4], ["b", 6], ["b", 10]]
The code below will give me the sum of each element in the form of array of array.
a.inject({}) { |h,(job, data)| h[job] ||= 0; h[job] += data; h }.to_a
And the result is as below
[["a", 20], ["b", 22]]
However, I am trying to get the average of the array. The expected output is as below.
[["a", 5], ["b", 5.5]]
I can count how many elements in an array and divide the sum array by the count array. I was wondering if there is an easier and more efficient way to get the average.
output = [
{"name"=>"aaa", "job"=>"a", "pay"=> 2 },
{"name"=>"zzz", "job"=>"a", "pay"=> 4 },
{"name"=>"xxx", "job"=>"a", "pay"=> 6 },
{"name"=>"yyy", "job"=>"a", "pay"=> 8 },
{"name"=>"aaa", "job"=>"b", "pay"=> 2 },
{"name"=>"zzz", "job"=>"b", "pay"=> 4 },
{"name"=>"xxx", "job"=>"b", "pay"=> 6 },
{"name"=>"yyy", "job"=>"b", "pay"=> 10 },
]
output.group_by { |obj| obj['job'] }.map do |key, list|
[key, list.map { |obj| obj['pay'] }.reduce(:+) / list.size.to_f]
end
The group_by method will transform your list into a hash with the following structure:
{"a"=>[{"name"=>"aaa", "job"=>"a", "pay"=>2}, ...], "b"=>[{"name"=>"aaa", "job"=>"b", ...]}
After that, for each pair of that hash, we want to calculate the mean of its 'pay' values, and return a pair [key, mean]. We use a map for that, returning a pair with:
They key itself ("a" or "b").
The mean of the values. Note that the values list has the form of a list of hashes. To retrieve the values, we need to extract the last element of each pair; that's what list.map { |obj| obj['pay'] } is used for. Finally, calculate the mean by suming all elements with .reduce(:+) and dividing them by the list size as a float.
Not the most efficient solution, but it's practical.
Comparing the answer with #EricDuminil's, here's a benchmark with a list of size 8.000.000:
def Wikiti(output)
output.group_by { |obj| obj['job'] }.map do |key, list|
[key, list.map { |obj| obj['pay'] }.reduce(:+) / list.size.to_f]
end
end
def EricDuminil(output)
count_and_sum = output.each_with_object(Hash.new([0, 0])) do |hash, mem|
job = hash['job']
count, sum = mem[job]
mem[job] = count + 1, sum + hash['pay']
end
result = count_and_sum.map do |job, (count, sum)|
[job, sum / count.to_f]
end
end
require 'benchmark'
Benchmark.bm do |x|
x.report('Wikiti') { Wikiti(output) }
x.report('EricDuminil') { EricDuminil(output) }
end
user system total real
Wikiti 4.100000 0.020000 4.120000 ( 4.130373)
EricDuminil 4.250000 0.000000 4.250000 ( 4.272685)
This method should be reasonably efficient. It creates a temporary hash with job name as key and [count, sum] as value:
output = [{ 'name' => 'aaa', 'job' => 'a', 'pay' => 2 },
{ 'name' => 'zzz', 'job' => 'a', 'pay' => 4 },
{ 'name' => 'xxx', 'job' => 'a', 'pay' => 6 },
{ 'name' => 'yyy', 'job' => 'a', 'pay' => 8 },
{ 'name' => 'aaa', 'job' => 'b', 'pay' => 2 },
{ 'name' => 'zzz', 'job' => 'b', 'pay' => 4 },
{ 'name' => 'xxx', 'job' => 'b', 'pay' => 6 },
{ 'name' => 'yyy', 'job' => 'b', 'pay' => 10 }]
count_and_sum = output.each_with_object(Hash.new([0, 0])) do |hash, mem|
job = hash['job']
count, sum = mem[job]
mem[job] = count + 1, sum + hash['pay']
end
#=> {"a"=>[4, 20], "b"=>[4, 22]}
result = count_and_sum.map do |job, (count, sum)|
[job, sum / count.to_f]
end
#=> [["a", 5.0], ["b", 5.5]]
It requires 2 passes, but the created objects aren't big. In comparison, calling group_by on a huge array of hashes isn't very efficient.
How about this (Single pass iterative average calculation)
accumulator = Hash.new {|h,k| h[k] = Hash.new(0)}
a.each_with_object(accumulator) do |(k,v),obj|
obj[k][:count] += 1
obj[k][:sum] += v
obj[k][:average] = (obj[k][:sum] / obj[k][:count].to_f)
end
#=> {"a"=>{:count=>4, :sum=>20, :average=>5.0},
# "b"=>{:count=>4, :sum=>22, :average=>5.5}}
Obviously average is just recalculated on every iteration but since you asked for them at the same time this is probably as close as you are going to get.
Using your "output" instead looks like
output.each_with_object(accumulator) do |h,obj|
key = h['job']
obj[key][:count] += 1
obj[key][:sum] += h['pay']
obj[key][:average] = (obj[key][:sum] / obj[key][:count].to_f)
end
#=> {"a"=>{:count=>4, :sum=>20, :average=>5.0},
# "b"=>{:count=>4, :sum=>22, :average=>5.5}}
as Sara Tibbetts comment suggests, my first step would be to convert it like this
new_a = a.reduce({}){ |memo, item| memo[item[0]] ||= []; memo[item[0]] << item[1]; memo}
which puts it in this format
{a: [2, 4, 6, 8], b: [2, 4, 6, 20]}
you can then use slice to filter the keys you want
new_a.slice!(key1, key2, ...)
Then do another pass through to do get the final format
new_a.reduce([]) do |memo, (k,v)|
avg = v.inject{ |sum, el| sum + el }.to_f / v.size
memo << [k,avg]
memo
end
I elected to use Enumerable#each_with_object with the object being an array of two hashes, the first to compute totals, the second to count the number of numbers that are totalled. Each hash is defined Hash.new(0), zero being the default value. See Hash::new for a fuller explanation, In short, if a hash defined h = Hash.new(0) does not have a key k, h[k] returns 0. (h is not modified.) h[k] += 1 expands to h[k] = h[k] + 1. If h does not have a key k, h[k] on the right of the equality returns 0.1
output =
[{"name"=>"aaa", "job"=>"a", "pay"=> 2},
{"name"=>"zzz", "job"=>"a", "pay"=> 4},
{"name"=>"xxx", "job"=>"a", "pay"=> 6},
{"name"=>"yyy", "job"=>"a", "pay"=> 8},
{"name"=>"aaa", "job"=>"b", "pay"=> 2},
{"name"=>"zzz", "job"=>"b", "pay"=> 4},
{"name"=>"xxx", "job"=>"b", "pay"=> 6},
{"name"=>"yyy", "job"=>"b", "pay"=>10}
]
htot, hnbr = output.each_with_object([Hash.new(0), Hash.new(0)]) do |f,(g,h)|
s = f["job"]
g[s] += f["pay"]
h[s] += 1
end
htot.merge(hnbr) { |k,o,n| o.to_f/n }.to_a
#=> [["a", 5.0], ["b", 5.5]]
If .to_a at the end is dropped the the hash {"a"=>5.0, "b"=>5.5} is returned. The OP might find that more useful than the array.
I've used the form of Hash#merge that uses a block to determine the values of keys that are present in both hashes being merged.
Note that htot={"a"=>20, "b"=>22} and hnbr=>{"a"=>4, "b"=>4}.
1 If the reader is wondering why h[k] on the left of = doesn't return zero as well, it's a different method: Hash#[]= versus Hash#[]
I am quiet new to Elixir programming and stuck badly at splitting into two elements tuple.
Given a list of integers, return a two element tuple. The first element is a list of the even numbers from the list. The second is a list of the odd numbers.
Input : [ 1, 2, 3, 4, 5 ]
Output { [ 2, 4], [ 1, 3, 5 ] }
I have reached to identify the odd or even but not sure how do I proceed.
defmodule OddOrEven do
import Integer
def task(list) do
Enum.reduce(list, [], fn(x, acc) ->
case Integer.is_odd(x) do
:true -> # how do I get this odd value listed as a tuple element
:false -> # how do I get this even value listed as a tuple element
end
#IO.puts(x)
end
)
end
You can use Enum.partition/2:
iex(1)> require Integer
iex(2)> [1, 2, 3, 4, 5] |> Enum.partition(&Integer.is_even/1)
{[2, 4], [1, 3, 5]}
If you really want to use Enum.reduce/2, you can do this:
iex(3)> {evens, odds} = [1, 2, 3, 4, 5] |> Enum.reduce({[], []}, fn n, {evens, odds} ->
...(3)> if Integer.is_even(n), do: {[n | evens], odds}, else: {evens, [n | odds]}
...(3)> end)
{[4, 2], [5, 3, 1]}
iex(4)> {Enum.reverse(evens), Enum.reverse(odds)}
{[2, 4], [1, 3, 5]}
Or you can use the Erlang :lists module:
iex> :lists.partition(fn (n) -> rem(n, 2) == 1 end, [1,2,3,4,5])
{[1,3,5],[2,4]}
This is just probably easy one.
For example this sys.config into mix config.exs:
{gsms, [{interfaces,
[
{gsms_0705, 1, [{device,"/dev/tty.usbserial-FTF5DP2J"},
{bnumber, "<phone-number>"},{baud,19200},
{reopen_timeout, 5000}]}
{gsms_0705, 2, [{device,"/dev/tty.HUAWEIMobile-Pcui"},
{bnumber, "<phone-number>"},
{reopen_timeout, 5000}]}
{gsms_0705, 3, [{device, "/dev/tty.usbserial"},
{bnumber, "<phone-number>"},
{baud, 9600}]}
]}
]}
If you fix the syntax errors (missing commas) and add the mandatory trailing dot, you can even parse the Erlang term from Elixir and let the parser do the conversion for you:
erlang_term = '{gsms, [{interfaces, [
{gsms_0705, 1, [{device,"/dev/tty.usbserial-FTF5DP2J"},
{bnumber, "<phone-number>"},{baud,19200},
{reopen_timeout, 5000}]},
{gsms_0705, 2, [{device,"/dev/tty.HUAWEIMobile-Pcui"},
{bnumber, "<phone-number>"},
{reopen_timeout, 5000}]},
{gsms_0705, 3, [{device, "/dev/tty.usbserial"},
{bnumber, "<phone-number>"},
{baud, 9600}]}
]}
]}.'
{:ok, tokens, _} = :erl_scan.string(erlang_term)
{:ok, result} = :erl_parse.parse_term(tokens)
IO.inspect result, limit: :infinity
This prints:
{:gsms,
[interfaces: [{:gsms_0705, 1,
[device: '/dev/tty.usbserial-FTF5DP2J', bnumber: '<phone-number>',
baud: 19200, reopen_timeout: 5000]},
{:gsms_0705, 2,
[device: '/dev/tty.HUAWEIMobile-Pcui', bnumber: '<phone-number>',
reopen_timeout: 5000]},
{:gsms_0705, 3,
[device: '/dev/tty.usbserial', bnumber: '<phone-number>', baud: 9600]}]]}
The first part of the tuple is the application name :gsms, the other part of the tuple is proplist containing the actual options. You can leave off the surrounding brackets and reformat this a bit and you will get:
config :gsms, interfaces: [
{:gsms_0705, 1, [
device: '/dev/tty.usbserial-FTF5DP2J',
bnumber: '<phone-number>',
baud: 19200,
reopen_timeout: 5000]},
{:gsms_0705, 2, [
device: '/dev/tty.HUAWEIMobile-Pcui',
bnumber: '<phone-number>',
reopen_timeout: 5000]},
{:gsms_0705, 3, [
device: '/dev/tty.usbserial',
bnumber: '<phone-number>',
baud: 9600]}]
I think this would be it:
config :gsms, :interfaces, [
{:gsms_0705, 1, [device: '/dev/tty.usbserial-FTF5DP2J',
bnumber: '<phone-number>',
baud: 19200,
reopen_timeout: 5000]},
# and so on
]
where :gsms is the application you're configuring, :interfaces is the key you'll retrieve through Application.get_env/2 (Application.get_env(:gsms, :interfaces)) and the list of {:gsms_*, ...} tuples is the value at that key.
Below is the text file:
Old count: 56
S id: 1
M id: 1
New count: 2
Old count: 56
S id: 1
M id: 2
New count: 20
Old count: 56
S id: 1
M id: 2
New count: 32
-----------------------------
Old count: 2
S id: 2
M id: 1
New count: 4
--------------------------------
.
.
.
.
I have used delimiter "---------------" for each ids.
How to parse the value such that the lines with in the delimiter "-----" that is new count is added like this: 2+20+32 = 54
Hash array: count << {'new count' => 54} for first block and so on for remaining blocks.
I have tried like this..
begin
f=File.open("out2", "r")
f.each_line do |line|
#data+=line
end
s_rec=#data.split("------")
s_rec.each do |rec|
row_s=rec.split(/\n/)
row_s.each do |row|
if r.include?"New count"
rv=row.split(":")
#db=rv[1]
end
end
end
Not sure what output format you are trying to achieve, but given the text:
text = <<__
Old count: 56
S id: 1
M id: 1
New count: 2
Old count: 56
S id: 1
M id: 2
New count: 20
Old count: 56
S id: 1
M id: 2
New count: 32
-----------------------------
Old count: 2
S id: 2
M id: 1
New count: 4
--------------------------------
.
.
.
.
__
this:
text
.split(/^-{5,}/)
.map{|s| s.scan(/\bNew count: (\d+)/).map{|match| match.first.to_i}.inject(:+)}
gives:
[
54,
4,
nil
]
In response to the comment, still not clear what you want because what you wrote is not a valid Ruby object, but this:
text
.scan(/^S id: (\d+).+?^New count: (\d+)/m)
.inject(Hash.new(0)){|h, (k, v)| h[k.to_i] += v.to_i; h}
.map{|k, v| {"S id" => k, "new count" => v}}
gives:
[
{
"S id" => 1,
"new count" => 54
},
{
"S id" => 2,
"new count" => 4
}
]
I'd start with:
data = 'Old count: 56
S id: 1
M id: 1
New count: 2
Old count: 56
S id: 1
M id: 2
New count: 20
Old count: 56
S id: 1
M id: 2
New count: 32
-----------------------------
Old count: 2
S id: 2
M id: 1
New count: 4
--------------------------------
'
ary = data.split("\n").slice_before(/^---/).map{ |a| a.select{ |s| s['New count:'] }.map{ |s| s[/\d+/].to_i }.inject(:+) }.compact
Which gives me an array:
[
[0] 54,
[1] 4,
]
compact is needed because there's a trailing ---- block delimiter that results in an empty array when slice_before does its magic.
From that point it's easy to create an array of hashes:
Hash[ ary.map.with_index(1) { |v, i| ["S #{ i }", "new count #{ v }" ] } ]
Which looks like:
{
"S 1" => "new count 54",
"S 2" => "new count 4"
}
Breaking it down, the code through slice_before returns:
[
[0] [
[ 0] "--------------------------------",
[ 1] "Old count: 56",
[ 2] "S id: 1",
[ 3] "M id: 1 ",
[ 4] "New count: 2",
[ 5] "Old count: 56",
[ 6] "S id: 1",
[ 7] "M id: 2",
[ 8] "New count: 20",
[ 9] "Old count: 56",
[10] "S id: 1",
[11] "M id: 2",
[12] "New count: 32"
],
[1] [
[0] "-----------------------------",
[1] "Old count: 2",
[2] "S id: 2",
[3] "M id: 1",
[4] "New count: 4"
]
]
From there it's straightforward, selecting the lines that are needed in each sub-array, extracting out the values, and summing them using inject.
Once that's done it's simply using map and with_index to build the string and name/value pairs, then let Hash turn them into a hash.