I have a condition where I need to get the taskId value of the dictionary where taskcode is "LIFE_MAX_DAYS". How can I do ?
Dictionary
replFlag: null (Text)
taskCode: "LIFE_MAX_DAYS"
createdBy: "Administrator"
createdOn: 3/20/2020 1:54 AM EDT
lastModifiedBy: "Administrator"
lastModifiedOn: 3/20/2020 1:54 AM EDT
actionId: null (Number (Integer))
priorityId: 5
statusId: null (Number (Integer))
concatKey: null (Text)
taskId: 5980
batchId: null (Number (Integer))
id: 4
Dictionary
replFlag: null (Text)
taskCode: "LIFE_MAX_DAYS"
createdBy: "Administrator"
createdOn: 3/20/2020 1:54 AM EDT
lastModifiedBy: "Administrator"
lastModifiedOn: 3/20/2020 1:54 AM EDT
actionId: null (Number (Integer))
priorityId: 5
statusId: null (Number (Integer))
concatKey: null (Text)
taskId: 5980
batchId: null (Number (Integer))
id: 5
wherecontains finds the indicies where an array value is provided.
Furthermore, lists support multi-indexing (e. g. {"a", "b", "c", "d"}[{1, 3}] returns {"a", "c"}).
Finally lists support projected indexing (e. g. {{x: 1}, {x: 2}, {x: 3}}.x returns {1, 2, 3})
These three features allow you to do this:
with(
/* Pull out just the task codes of all the tasks */
taskCodesOfTasks: listOfTasks.taskCode,
/* Get the indicies where the task code is what we're looking for */
indicies: wherecontains("LIFE_MAX_DAYS", taskCodesOfTasks),
/* Pull out the task data */
selectedTasks: listOfTasks[indicies],
/* Return the task IDs */
selectedTasks.taskId
)
Which can of course be spelled with much less ceremony:
listOfTasks[wherecontains("LIFE_MAX_DAYS", listOfTasks.taskCode)].taskId
Related
For example I have list as:
List<Map<String, dynamic>> employees = [
{"name": 'Kris', 'departmentId': 18, CityId: 1},
{"name": 'Ana', 'departmentId': 18, CityId: 2},
{"name": 'Monty', 'departmentId': 18, CityId: 3},
{"name": 'John', 'departmentId': 18, CityId: 4},
];
If you have a List, and you want to make sure that all items in the list have a certain property, you can use List.every. For example:
bool isEven(int n) => n % 2 == 0;
[1, 2, 3, 4].every(isEven) // false, some values are not even
[2, 4, 6, 8].every(isEven) // true, every value is even
In your case, you can check that every value of departmentId has the same value as the first (if you don't know that id ahead of time).
bool allHaveSameDepartment(List<Map<String, dynamic>> employees) {
bool sameIdAsFirst(Map<String, dynamic> employee) => employee['departmentId'] == employees[0]['departmentId'];
return employees.every(sameIdAsFirst);
}
Note that in this case, an empty list will return true (https://en.wikipedia.org/wiki/Vacuous_truth). You may wish to override this behaviour by checking employees.isEmpty at the start and maybe throwing an error or returning false.
I got the solution by using toSet().
Solution: employees.map((value) => value.departmentId).toSet().length.
I have a json as mentioned below,
{
"list": [{
"notificationId": 123,
"userId": 444
},
{
"notificationId": 456,
"userId": 789
}
]
}
I need to write a postgres procedure which interates through the list and perform either update/insert based on notification id is already present or not in DB.
I have a notification table which has notificationid and userID as columns.
Can anyone please tell me on how to perform this using postgres json operators.
Try this query:
SELECT *
FROM yourTable
WHERE col->'list'#>'[{"notificationId":123}]';
You may replace the value 123 with whatever notificationId you want to search. Follow the link below for a demo showing that this logic works:
Demo
Assuming you have a unique constraint on notificationid (e.g. because it's the primary key, there is no need for stored function or loop:
with data (j) as (
values ('
{
"list": [{
"notificationId": 123,
"userId": 444
},
{
"notificationId": 456,
"userId": 789
}
]
}'::jsonb)
)
insert into notification (notificationid, userid)
select (e.r ->> 'notificationId')::int, (e.r ->> 'userId')::int
from data d, jsonb_array_elements(d.j -> 'list') as e(r)
on conflict (notificationid) do update
set userid = excluded.userid;
The first step in that statement is to turn the array into a list of rows, this is what:
select e.*
from data d, jsonb_array_elements(d.j -> 'list') as e(r)
does. Given your sample JSON, this returns two rows with a JSON value in each:
r
--------------------------------------
{"userId": 444, "notificationId": 123}
{"userId": 789, "notificationId": 456}
This is then split into two integer columns:
select (e.r ->> 'notificationId')::int, (e.r ->> 'userId')::int
from data d, jsonb_array_elements(d.j -> 'list') as e(r)
So we get:
int4 | int4
-----+-----
123 | 444
456 | 789
And this result is used as the input for an INSERT statement.
The on conflict clause then does an insert or update depending on the presence of the row identified by the column notificationid which has to have a unique index.
Meanwhile i tried this,
CREATE OR REPLACE FUNCTION insert_update_notifications(notification_ids jsonb) RETURNS void AS
$$
DECLARE
allNotificationIds text[];
indJson jsonb;
notIdCount int;
i json;
BEGIN
FOR i IN SELECT * FROM jsonb_array_elements(notification_ids)
LOOP
select into notIdCount count(notification_id) from notification_table where notification_id = i->>'notificationId' ;
IF(notIdCount = 0 ) THEN
insert into notification_table(notification_id,userid) values(i->>'notificationId',i->>'userId');
ELSE
update notification_table set userid = i->>'userId' where notification_id = i->>'notificationId';
END IF;
END LOOP;
END;
$$
language plpgsql;
select * from insert_update_notifications('[{
"notificationId": "123",
"userId": "444"
},
{
"notificationId": "456",
"userId": "789"
}
]');
It works.. Please review this.
Below query takes long time to create temporary table, its only have "228000" distinct record.
DECLARE todate,fromdate DATETIME;
SET fromdate=DATE_SUB(UTC_TIMESTAMP(),INTERVAL 2 DAY);
SET todate=DATE_ADD(UTC_TIMESTAMP(),INTERVAL 14 DAY);
SET SESSION TRANSACTION ISOLATION LEVEL READ COMMITTED;
DROP TEMPORARY TABLE IF EXISTS tempabc;
SET max_heap_table_size = 1024*1024*1024;
CREATE TEMPORARY TABLE IF NOT EXISTS tempabc
-- (index using BTREE(id))
ENGINE=MEMORY
AS
(
SELECT SQL_NO_CACHE DISTINCT id
FROM abc
WHERE StartTime BETWEEN fromdate AND todate
);
I already created index on 'startTime' coulmn, still it tooks 20 sec to create table. Kindly help me out to reduce the creation time.
More Info:-
I changed my query earlier I was using "tempabc" temporary table to get my output, now I am using IN clause instead of temporary table and now it is taking 12 sec to execute, but still more than expected time..
Earlier(taking 20-30 sec)
DECLARE todate,fromdate DATETIME;
SET fromdate=DATE_SUB(UTC_TIMESTAMP(),INTERVAL 2 DAY);
SET todate=DATE_ADD(UTC_TIMESTAMP(),INTERVAL 14 DAY);
SET SESSION TRANSACTION ISOLATION LEVEL READ COMMITTED;
DROP TEMPORARY TABLE IF EXISTS tempabc;
SET max_heap_table_size = 1024*1024*1024;
CREATE TEMPORARY TABLE IF NOT EXISTS tempabc
-- (index using BTREE(id))
ENGINE=MEMORY
AS
(
SELECT SQL_NO_CACHE DISTINCT id
FROM abc
WHERE StartTime BETWEEN fromdate AND todate
);
SELECT DISTINCT p.xyzID
FROM tempabc s
JOIN xyz_tab p ON p.xyzID=s.ID AND IFNULL(IsGeneric,0)=0;
Now(taking 12-14 sec)
DECLARE todate,fromdate Timestamp;
SET fromdate=DATE_SUB(UTC_TIMESTAMP(),INTERVAL 2 DAY);
SET todate=DATE_ADD(UTC_TIMESTAMP(),INTERVAL 14 DAY);
SELECT p.xyzID FROM xyz_tab p
WHERE id IN (
SELECT DISTINCT id FROM abc
WHERE StartTime BETWEEN fromdate AND todate )
AND IFNULL(IsGeneric,0)=0 GROUP BY p.xyxID;
But we need to achieve 3-5 sec of execution time.
This is my explain output.
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: abc
partitions: NULL
type: index
possible_keys: ix_starttime_id,IDX_Start_time,IX_id_starttime,IX_id_starttime_prgsvcid
key: IX_id_starttime
key_len: 163
ref: NULL
rows: 18779876
filtered: 1.27
Extra: Using where; Using index; Using temporary; Using filesort; LooseScan
*************************** 2. row ***************************
id: 1
select_type: SIMPLE
table: p
partitions: NULL
type: eq_ref
possible_keys: PRIMARY,IX_seriesid
key: PRIMARY
key_len: 152
ref: onconnectdb.abc.ID
rows: 1
filtered: 100.00
Extra: Using where
Explain in JSON format
EXPLAIN: {
"query_block": {
"select_id": 1,
"cost_info": {
"query_cost": "10139148.44"
},
"grouping_operation": {
"using_temporary_table": true,
"using_filesort": true,
"cost_info": {
"sort_cost": "1.00"
},
"nested_loop": [
{
"table": {
"table_name": "abc",
"access_type": "index",
"possible_keys": [
"ix_starttime_tmsid",
"IDX_Start_time",
"IX_id_starttime",
"IX_id_starttime_prgsvcid"
],
"key": "IX_id_starttime",
"used_key_parts": [
"ID",
"StartTime",
"EndTime"
],
"key_length": "163",
"rows_examined_per_scan": 19280092,
"rows_produced_per_join": 264059,
"filtered": "1.37",
"using_index": true,
"loosescan": true,
"cost_info": {
"read_cost": "393472.45",
"eval_cost": "52812.00",
"prefix_cost": "446284.45",
"data_read_per_join": "2G"
},
"used_columns": [
"ID",
"StartTime"
],
"attached_condition": "(`onconnectdb`.`abc`.`StartTime` between <cache>(fromdate#1) and <cache>(todate#0))"
}
},
{
"table": {
"table_name": "p",
"access_type": "eq_ref",
"possible_keys": [
"PRIMARY",
"IX_seriesid"
],
"key": "PRIMARY",
"used_key_parts": [
"ID"
],
"key_length": "152",
"ref": [
"onconnectdb.abc.ID"
],
"rows_examined_per_scan": 1,
"rows_produced_per_join": 1,
"filtered": "100.00",
"cost_info": {
"read_cost": "9640051.00",
"eval_cost": "0.20",
"prefix_cost": "10139147.44",
"data_read_per_join": "2K"
},
"used_columns": [
"ID",
"xyzID",
"IsGeneric"
],
"attached_condition": "(ifnull(`onconnectdb`.`p`.`IsGeneric`,0) = 0)"
}
}
]
}
}
}
Please suggest.
I'm currently executing a SQL statement in Rails and while it works I've come to realize I need a different format and am attempting to accomplish this in PostreSQL. This is my query:
sql = "SELECT one_month_high - one_month_low as one_month,
three_month_high - three_month_low as three_month,
six_month_high - six_month_low as six_month,
twelve_month_high - twelve_month_low as twelve_month,
ytd_high - ytd_low as ytd,
saved_on
FROM daily_high_lows
ORDER BY saved_on DESC;"
Which returns:
#<PG::Result:0x007fdb4aea1fa0 status=PGRES_TUPLES_OK ntuples=380 nfields=6 cmd_tuples=380>
...
{"one_month"=>"544", "three_month"=>"214", "six_month"=>"9","twelve_month"=>"122",
"ytd"=>"143", "saved_on"=>"2016-06-09 00:00:00"}
{"one_month"=>"1283", "three_month"=>"475", "six_month"=>"22","twelve_month"=>"189",
"ytd"=>"517", "saved_on"=>"2016-06-08 00:00:00"}
I've come to realize that I require a format:
[
{
name: "One Month",
data: {
2016-06-09 00:00:00: 544,
2016-06-08 00:00:00: 1283
}
},
{
name: "Three Month",
data: {
2016-06-09 00:00:00: 214,
2016-06-08 00:00:00: 475
}
}, etc...
]
I've been trying to research how to do this but it's a bit beyond me currently so I could use some direction.
You should be able to use the to_json method available in rails. So response.to_json should yield what you need.
I think if you have a lot of records, building the JSON inside Postgres is a great approach. Postgres didn't really get very useful JSON-building functions until 9.4 though, so I recommend you be at least there if not higher. Anyway, this seems to give you what you want:
WITH seqs AS (
SELECT json_object_agg(saved_on::text, one_month_high ORDER BY saved_on) one_month_highs,
json_object_agg(saved_on::text, one_month_low ORDER BY saved_on) one_month_lows
FROM daily_high_lows
)
SELECT json_agg(j)
FROM (
SELECT json_build_object('name', 'One Month Highs', 'data', one_month_highs)
FROM seqs
UNION ALL
SELECT json_build_object('name', 'One Month Lows', 'data', one_month_lows)
FROM seqs
) x(j)
;
Tested like so:
t=# create table daily_high_lows (one_month_high integer, one_month_low integer, three_month_high integer, three_month_low integer, six_month_high integer, six_month_low integer, twleve_month_high integer, twelve_month_low integer, ytd_high integer, ytd_low integer, saved_on timestamp);
t=# insert into daily_high_lows (one_month_high, one_month_low, three_month_high, three_month_low, saved_on) values (1, 10, 3, 6, '2016-06-08');
t=# insert into daily_high_lows (one_month_high, one_month_low, three_month_high, three_month_low, saved_on) values (2, 9, 3, 8, '2016-03-09');
t=# with seqs as (select json_object_agg(saved_on::text, one_month_high order by saved_on) one_month_highs, json_object_agg(saved_on::text, one_month_low order by saved_on) one_month_lows from daily_high_lows) select json_agg(j) from (select json_build_object('name', 'One Month Highs', 'data', one_month_highs) from seqs union all select json_build_object('name', 'One Month Lows', 'data', one_month_lows) from seqs) x(j);
json_agg
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
[{"name" : "One Month Highs", "data" : { "2016-03-09 00:00:00" : 2, "2016-06-08 00:00:00" : 1 }}, {"name" : "One Month Lows", "data" : { "2016-03-09 00:00:00" : 9, "2016-06-08 00:00:00" : 10 }}]
(1 row)
I'm trying to use the Aerospike bulk loader to seed a cluster with data from a tab-separated file.
The source data looks like this:
set key segments
segment 123 10,20,30,40,50
segment 234 40,50,60,70
The third column, 'segments', contains a comma separated list of integers.
I created a JSON template:
{
"version" : "1.0",
"input_type" : "csv",
"csv_style": { "delimiter": " " , "n_columns_datafile": 3, "ignore_first_line": true}
"key": {"column_name":"key", "type": "integer"},
"set": { "column_name":"set" , "type": "string"},
"binlist": [
{"name": "segments",
"value": {"column_name": "segments", "type": "list"}
}
]
}
... and ran the loader:
java -cp aerospike-load-1.1-jar-with-dependencies.jar com.aerospike.load.AerospikeLoad -c template.json data.tsv
When I query the records in aql, they seem to be a list of strings:
aql> select * from test
+--------------------------------+
| segments |
+--------------------------------+
| ["10", "20", "30", "40", "50"] |
| ["40", "50", "60", "70"] |
+--------------------------------+
The data I'm trying to store is a list of integers. Is there an easy way to convert the objects stored in this bin to a list of integers (possibly a Lua UDF) or perhaps there's a tweak that can be made to the bulk loader template?
Update:
I attempted to solve this by creating a Lua UDF to convert the list from strings to integers:
function convert_segment_list_to_integers(rec)
for i=1, table.maxn(rec['segments']) do
rec['segments'][i] = math.floor(tonumber(rec['segments'][i]))
end
aerospike:update(rec)
end
... registered it:
aql> register module 'convert_segment_list_to_integers.lua'
... and then tried executing against my set:
aql> execute convert_segment_list_to_integers.convert_segment_list_to_integers() on test.segment
I enabled some more verbose logging and notice that the UDF is throwing an error. Apparently, it's expecting a table and it was passed userdata:
Dec 04 2015 23:23:34 GMT: DEBUG (udf): (udf_rw.c:send_result:527) FAILURE when calling convert_segment_list_to_integers convert_segment_list_to_integers ...rospike/usr/udf/lua/convert_segment_list_to_integers.lua:2: bad argument #1 to 'maxn' (table expected, got userdata)
Dec 04 2015 23:23:34 GMT: DEBUG (udf): (udf_rw.c:send_udf_failure:407) Non-special LDT or General UDF Error(...rospike/usr/udf/lua/convert_segment_list_to_integers.lua:2: bad argument #1 to 'maxn' (table expected, got userdata))
It seems that maxn isn't an applicable method to a userdata object.
Can you see what needs to be done to fix this?
To convert your lists with string values to lists of integer values you can run the following record udf:
function convert_segment_list_to_integers(rec)
local list_with_ints = list()
for value in list.iterator(rec['segments']) do
local int_value = math.floor(tonumber(value))
list.append(list_with_ints, int_value)
end
rec['segments'] = list_with_ints
aerospike:update(rec)
end
When you edit your existing lua module, make sure to re-run register module 'convert_segment_list_to_integers.lua'.
The cause of this issue is within the aerospike-loader tool: it will always assume/enforce strings as you can see in the following java code:
case LIST:
/*
* Assumptions
* 1. Items are separated by a colon ','
* 2. Item value will be a string
* 3. List will be in double quotes
*
* No support for nested maps or nested lists
*
*/
List<String> list = new ArrayList<String>();
String[] listValues = binRawText.split(Constants.LIST_DELEMITER, -1);
if (listValues.length > 0) {
for (String value : listValues) {
list.add(value.trim());
}
bin = Bin.asList(binColumn.getBinNameHeader(), list);
} else {
bin = null;
log.error("Error: Cannot parse to a list: " + binRawText);
}
break;
Source on Github: http://git.io/vRAQW
If you prefer, you can modify this code and re-compile to always assume integer list values. Change line 266 and 270 to something like this (untested):
List<Integer> list = new ArrayList<Integer>();
list.add(Integer.parseInt(value.trim());