I try to create tombstone message by using below steps in ksqldb 0.27.2 , but the tombstone message missing key value. The reproduce steps are as follows.
Origin topic message:
key: f8b6c12812ef4edc9c2c10ab2f7adceb, value: {"op_type": "I", "ID": "f8b6c12812ef4edc9c2c10ab2f7adceb", "CLASS_ID": "89b65aec513e4610908073dfde5baf46"}
key: f8b6c12812ef4edc9c2c10ab2f7adceb, value: {"op_type": "D", "ID": "f8b6c12812ef4edc9c2c10ab2f7adceb", "CLASS_ID": "89b65aec513e4610908073dfde5baf46"}
Step 1: create source stream
set 'auto.offset.reset'='earliest';
CREATE STREAM STREAM_ORIGIN_OBJ_ATTRIBUTE WITH (KAFKA_TOPIC='SIT.CPLM.OBJ_ATTRIBUTE',VALUE_FORMAT='AVRO');
Step 2: create tombstone stream
CREATE STREAM STREAM_ETL_TOMBSTONE_OBJ_ATTRIBUTE WITH (KAFKA_TOPIC='SIT.CPLM.OBJ_ATTRIBUTE', VALUE_FORMAT='KAFKA') AS SELECT CAST(NULL AS VARCHAR) FROM STREAM_ORIGIN_OBJ_ATTRIBUTE WHERE op_type = 'D';
The output message of tombstone stream:
rowtime: 2022/08/02 06:56:08.867 Z, key: f8b6c12812ef4edc9c2c10ab2f7adceb, value: {"op_type": "I", "ID": "f8b6c12812ef4edc9c2c10ab2f7adceb", "CLASS_ID": "89b65aec513e4610908073dfde5baf46"}, partition: 0
rowtime: 2022/08/02 06:56:19.877 Z, key: f8b6c12812ef4edc9c2c10ab2f7adceb, value: {"op_type": "D", "ID": "f8b6c12812ef4edc9c2c10ab2f7adceb", "CLASS_ID": "89b65aec513e4610908073dfde5baf46"}, partition: 0
rowtime: 2022/08/02 06:56:19.877 Z, key: <null>, value: <null>, partition: 0
Expected tombstone message:
rowtime: 2022/08/02 06:56:19.877 Z, key: f8b6c12812ef4edc9c2c10ab2f7adceb, value: <null>, partition: 0
Related
Can you help me to implement one Accumulator from List of maps?.
[
%{
score: 1,
name: "Javascript",
},
%{
score: 2,
name: "Elixir",
},
%{
score: 10,
name: "Elixir",
}
]
The result should be:
[
%{
score: 12,
name: "Elixir",
},
%{
score: 1,
name: "Javascript",
}
]
I will appreciate your suggestion.
Regards
Assuming your original list is stored in input local variable, one might start with Enum.reduce/3 using Map.update/4 as a reducer.
Enum.reduce(input, %{}, fn %{score: score, name: name}, acc ->
Map.update(acc, name, score, & &1 + score)
end)
#⇒ %{"Elixir" => 12, "Javascript" => 1}
Whether you insist on having a list of maps as a result (which is way less readable IMSO,) go further and Enum.map/2 the result:
Enum.map(%{"Elixir" => 12, "Javascript" => 1}, fn {name, score} ->
%{name: name, score: score}
end)
#⇒ [%{name: "Elixir", score: 12},
# %{name: "Javascript", score: 1}]
To sum it up:
input
|> Enum.reduce(%{}, fn %{score: score, name: name}, acc ->
Map.update(acc, name, score, & &1 + score)
end)
|> Enum.map(& %{name: elem(&1, 0), score: elem(&1, 1)})
#⇒ [%{name: "Elixir", score: 12},
# %{name: "Javascript", score: 1}]
Sidenote: maps in erlang (and, hence, in elixir) are not ordered. That means, if you want the resulting list to be sorted by name, or by score, you should explicitly Enum.sort/2 it:
Enum.sort(..., & &1.score > &2.score)
#⇒ [%{name: "Elixir", score: 12},
# %{name: "Javascript", score: 1}]
A simple way could be to use Enum.group_by/3 to group the items by name, then Enum.sum/1 to sum the scores:
list
|> Enum.group_by(& &1.name, & &1.score)
|> Enum.map(fn {name, score} -> %{name: name, score: Enum.sum(score)} end)
Output:
[%{name: "Elixir", score: 12}, %{name: "Javascript", score: 1}]
If you were looking to create & use a more generalized solution, you could create your own Merger module.
defmodule Merger do
def merge_by(enumerable, name_fun, merge_fun) do
enumerable
|> Enum.group_by(name_fun)
|> Enum.map(fn {_name, items} -> Enum.reduce(items, merge_fun) end)
end
end
list = [
%{score: 1, name: "Javascript"},
%{score: 2, name: "Elixir"},
%{score: 10, name: "Elixir"}
]
Merger.merge_by(list, & &1.name, &%{&1 | score: &1.score + &2.score})
# => [%{name: "Elixir", score: 12}, %{name: "Javascript", score: 1}]
I am trying to iterate data from my values received from server. But I am getting following error:
flutter: type 'List<dynamic>' is not a subtype of type 'String'
Received data is
{-L_Ct1pu8k2qeVEbaiwN: [{calorie: 0-30, food: {calorie: 0, carb: 0, fat: 0, image: https://firebasestorage.googleapis.com/v0/b/yourstrulycare.appspot.com/o/food_item_LN5T8kDMQEJtJ7KfAGQEiJXtLRBXcGksLrn8we4tN9mx9qqINz.png?alt=media&token=499fce97-4a70-4805-8e83-f5094043db13, key: -LSOFQCZGLMpFdgLvsu0, name: Warm water with Cinnamon, protien: 0, qty: 500, range: 0-30, uom: ml}, notes: 1, serving: 1, time: Pre-Breakfast, total_calorie: 0}, {calorie: 0-30, food: {calorie: 18, carb: 1, fat: 1.4, image: https://firebasestorage.googleapis.com/v0/b/yourstrulycare.appspot.com/o/food_item_w2nj0GCjlg5zsP7jisS45IY3XTyLZhoX5R9OAG7W55czyXFDbU.png?alt=media&token=024631ed-2912-4935-9476-2bde51dbc20d, key: -LSIyhpzxUYOrzf4lUwx, name: Flax Seeds, protien: 0.6, qty: 200, range: 0-30, uom: Gram}, notes: 1, serving: 1, time: Pre-Breakfast, total_calorie: 18}]}
My code is
try{
widget.mealData.forEach((key, value) {
print('Value -----'+value);
if (value != null)
_mealList.add(MeanIntervalData.checkData(
tempId: key,
tempData: value,
));
});
}catch(error){
print(error);
}
It is generating me an error
flutter: type 'List<dynamic>' is not a subtype of type 'String'
How I can iterate this?
I need to send a 2d array in this format:
[[1, 4], [2, 5]]
this is passed to parameter
params[:order][:product_ids_and_quantities]
I have tried
Key: order[product_ids_and_quantities][][]
Value: [1, 4]
Key: order[product_ids_and_quantities][][]
Value: [2, 5]
And
Key: order[product_ids_and_quantities]
Value: [[1,4],[2,5]]
But it's not working. I always get a string in rails
This is just probably easy one.
For example this sys.config into mix config.exs:
{gsms, [{interfaces,
[
{gsms_0705, 1, [{device,"/dev/tty.usbserial-FTF5DP2J"},
{bnumber, "<phone-number>"},{baud,19200},
{reopen_timeout, 5000}]}
{gsms_0705, 2, [{device,"/dev/tty.HUAWEIMobile-Pcui"},
{bnumber, "<phone-number>"},
{reopen_timeout, 5000}]}
{gsms_0705, 3, [{device, "/dev/tty.usbserial"},
{bnumber, "<phone-number>"},
{baud, 9600}]}
]}
]}
If you fix the syntax errors (missing commas) and add the mandatory trailing dot, you can even parse the Erlang term from Elixir and let the parser do the conversion for you:
erlang_term = '{gsms, [{interfaces, [
{gsms_0705, 1, [{device,"/dev/tty.usbserial-FTF5DP2J"},
{bnumber, "<phone-number>"},{baud,19200},
{reopen_timeout, 5000}]},
{gsms_0705, 2, [{device,"/dev/tty.HUAWEIMobile-Pcui"},
{bnumber, "<phone-number>"},
{reopen_timeout, 5000}]},
{gsms_0705, 3, [{device, "/dev/tty.usbserial"},
{bnumber, "<phone-number>"},
{baud, 9600}]}
]}
]}.'
{:ok, tokens, _} = :erl_scan.string(erlang_term)
{:ok, result} = :erl_parse.parse_term(tokens)
IO.inspect result, limit: :infinity
This prints:
{:gsms,
[interfaces: [{:gsms_0705, 1,
[device: '/dev/tty.usbserial-FTF5DP2J', bnumber: '<phone-number>',
baud: 19200, reopen_timeout: 5000]},
{:gsms_0705, 2,
[device: '/dev/tty.HUAWEIMobile-Pcui', bnumber: '<phone-number>',
reopen_timeout: 5000]},
{:gsms_0705, 3,
[device: '/dev/tty.usbserial', bnumber: '<phone-number>', baud: 9600]}]]}
The first part of the tuple is the application name :gsms, the other part of the tuple is proplist containing the actual options. You can leave off the surrounding brackets and reformat this a bit and you will get:
config :gsms, interfaces: [
{:gsms_0705, 1, [
device: '/dev/tty.usbserial-FTF5DP2J',
bnumber: '<phone-number>',
baud: 19200,
reopen_timeout: 5000]},
{:gsms_0705, 2, [
device: '/dev/tty.HUAWEIMobile-Pcui',
bnumber: '<phone-number>',
reopen_timeout: 5000]},
{:gsms_0705, 3, [
device: '/dev/tty.usbserial',
bnumber: '<phone-number>',
baud: 9600]}]
I think this would be it:
config :gsms, :interfaces, [
{:gsms_0705, 1, [device: '/dev/tty.usbserial-FTF5DP2J',
bnumber: '<phone-number>',
baud: 19200,
reopen_timeout: 5000]},
# and so on
]
where :gsms is the application you're configuring, :interfaces is the key you'll retrieve through Application.get_env/2 (Application.get_env(:gsms, :interfaces)) and the list of {:gsms_*, ...} tuples is the value at that key.
Below is the text file:
Old count: 56
S id: 1
M id: 1
New count: 2
Old count: 56
S id: 1
M id: 2
New count: 20
Old count: 56
S id: 1
M id: 2
New count: 32
-----------------------------
Old count: 2
S id: 2
M id: 1
New count: 4
--------------------------------
.
.
.
.
I have used delimiter "---------------" for each ids.
How to parse the value such that the lines with in the delimiter "-----" that is new count is added like this: 2+20+32 = 54
Hash array: count << {'new count' => 54} for first block and so on for remaining blocks.
I have tried like this..
begin
f=File.open("out2", "r")
f.each_line do |line|
#data+=line
end
s_rec=#data.split("------")
s_rec.each do |rec|
row_s=rec.split(/\n/)
row_s.each do |row|
if r.include?"New count"
rv=row.split(":")
#db=rv[1]
end
end
end
Not sure what output format you are trying to achieve, but given the text:
text = <<__
Old count: 56
S id: 1
M id: 1
New count: 2
Old count: 56
S id: 1
M id: 2
New count: 20
Old count: 56
S id: 1
M id: 2
New count: 32
-----------------------------
Old count: 2
S id: 2
M id: 1
New count: 4
--------------------------------
.
.
.
.
__
this:
text
.split(/^-{5,}/)
.map{|s| s.scan(/\bNew count: (\d+)/).map{|match| match.first.to_i}.inject(:+)}
gives:
[
54,
4,
nil
]
In response to the comment, still not clear what you want because what you wrote is not a valid Ruby object, but this:
text
.scan(/^S id: (\d+).+?^New count: (\d+)/m)
.inject(Hash.new(0)){|h, (k, v)| h[k.to_i] += v.to_i; h}
.map{|k, v| {"S id" => k, "new count" => v}}
gives:
[
{
"S id" => 1,
"new count" => 54
},
{
"S id" => 2,
"new count" => 4
}
]
I'd start with:
data = 'Old count: 56
S id: 1
M id: 1
New count: 2
Old count: 56
S id: 1
M id: 2
New count: 20
Old count: 56
S id: 1
M id: 2
New count: 32
-----------------------------
Old count: 2
S id: 2
M id: 1
New count: 4
--------------------------------
'
ary = data.split("\n").slice_before(/^---/).map{ |a| a.select{ |s| s['New count:'] }.map{ |s| s[/\d+/].to_i }.inject(:+) }.compact
Which gives me an array:
[
[0] 54,
[1] 4,
]
compact is needed because there's a trailing ---- block delimiter that results in an empty array when slice_before does its magic.
From that point it's easy to create an array of hashes:
Hash[ ary.map.with_index(1) { |v, i| ["S #{ i }", "new count #{ v }" ] } ]
Which looks like:
{
"S 1" => "new count 54",
"S 2" => "new count 4"
}
Breaking it down, the code through slice_before returns:
[
[0] [
[ 0] "--------------------------------",
[ 1] "Old count: 56",
[ 2] "S id: 1",
[ 3] "M id: 1 ",
[ 4] "New count: 2",
[ 5] "Old count: 56",
[ 6] "S id: 1",
[ 7] "M id: 2",
[ 8] "New count: 20",
[ 9] "Old count: 56",
[10] "S id: 1",
[11] "M id: 2",
[12] "New count: 32"
],
[1] [
[0] "-----------------------------",
[1] "Old count: 2",
[2] "S id: 2",
[3] "M id: 1",
[4] "New count: 4"
]
]
From there it's straightforward, selecting the lines that are needed in each sub-array, extracting out the values, and summing them using inject.
Once that's done it's simply using map and with_index to build the string and name/value pairs, then let Hash turn them into a hash.