Is it possible to store a multidimensional array in a column.
I have tried the following and received the error below coming from creating the records column.
migration_file.rb
create_table :balance_sheets_details do |t|
t.string :headers, array: true, default: []
t.string :records, array: true, default: [[]]
t.timestamps
end
Raised error
PG::InvalidTextRepresentation: ERROR: malformed array literal: "{{}}"
From the docs on arrays (emphasis added):
The syntax for CREATE TABLE allows the exact size of arrays to be specified, for example:
CREATE TABLE tictactoe (
squares integer[3][3]
);
However, the current implementation ignores any supplied array size limits, i.e., the behavior is the same as for arrays of unspecified length.
The current implementation does not enforce the declared number of dimensions either. Arrays of a particular element type are all considered to be of the same type, regardless of size or number of dimensions. So, declaring the array size or number of dimensions in CREATE TABLE is simply documentation; it does not affect run-time behavior.
Thus, there isn't really a multidimensional array type. To fix your issue, just change the default from {{}} to {}.
This means a varchar[][] is the same type as a varchar[]:
db=# select pg_typeof(a), pg_typeof(b) from (values ('{{hello},{world}}'::varchar[][], '{foo}'::varchar[])) x(a, b);
pg_typeof | pg_typeof
---------------------+---------------------
character varying[] | character varying[]
(1 row)
You will still be able to store multidimensional data, though.
A one and two dimensional array are not the same:
db=# select '{{foo}}'::varchar[] = '{foo}'::varchar[];
?column?
----------
f
(1 row)
Related
I am using compound datatypes with h5py, with some elements being variable-length arrays. I can't find a way to set the item. The following MWE shows 6 various ways to do that (sequential indexing — which would not work in h5py anyway, fused indexing, read-modify-commit for columns/rows), neither of which works.
What is the correct way? Why is h5py saying Cannot change data-type for object array when writing integer list to int32 list?
with h5py.File('/tmp/test-vla.h5','w') as h5:
dt=np.dtype([('a',h5py.vlen_dtype(np.dtype('int32')))])
dset=h5.create_dataset('test',(5,),dtype=dt)
dset['a'][2]=[1,2,3] # does not write the value back
dset[2]['a']=[1,2,3] # does not write the value back
dset['a',2]=[1,2,3] # Cannot change data-type for object array
dset[2,'a']=[1,2,3] # Cannot change data-type for object array
tmp=dset['a']; tmp[2]=[1,2,3]; dset['a']=tmp # Cannot change data-type for object array
tmp=dset[2]; tmp['a']=[1,2,3]; dset[2]=tmp # 'list' object has no attribute 'dtype'
When working with compound datasets, I've discovered it's best to add all row data in a single statement. I tweaked your code and to show how add 3 rows of data (each of different length). Note how I: 1) define the row of data with a tuple; 2) define the list of integers with np.array(); and 3) don't reference the field name ['a'].
with h5py.File('test-vla.h5','w') as h5:
dt=np.dtype([('a',h5py.vlen_dtype(np.dtype('int32')))])
dset=h5.create_dataset('test',(5,),dtype=dt)
print (dset.dtype, dset.shape)
dset[0] = ( np.array([0,1,2]), )
dset[1] = ( np.array([1,2,3,4]), )
dset[2] = ( np.array([0,1,2,3,4]), )
For more info, take a look at this post on the HDF Group Forum under HDF5 Ancillary Tools / h5py:
Compound datatype with int, float and array of floats
if a table of N integer is present how to check if an element is repeating if present it shows message that table has repeating elements, if this is to be achieved in minimum time complexity
Hash table is the way to go (ie normal Lua table). Just loop over each integer and place it into the table as the key but first check if the key already exists. If it does then you have a repeat value. So something like:
values = { 1, 2, 3, 4, 5, 1 } -- input values
local htab = {}
for _, v in ipairs(values) do
if htab[v] then print('duplicate value: ' .. v)
else htab[v] = true end
end
With small integer values the table will use an array so will be O(1) to access. With larger and therefore sparser values the values will be in the hash table part of the table which can just be assumed to be O(1) as well. And since you have N values to insert this is O(N).
Getting faster than O(N) should not be possible since you have to visit each value in the list at least once.
A user model has a goal field that is an array:
t.integer "goal", default: [], array: true
A valid entry for goal is [[],[],[]]. If I do
Match.create(goal: [[],[],[]])
I get an exception:
ActiveRecord::StatementInvalid: PG::InvalidTextRepresentation: ERROR: malformed array literal: "{{},{},{}}"
DETAIL: Unexpected "}" character.
I know that I can work around this by saving it as goal: [], but I lose information about the number of sub-arrays. Is there a way to fix this?
You’ve gone from trying to store an array of numbers to an array of fixed length which includes null values or even sub arrays. Therefore, a t.integer field is no good. You need to use a t.jsonb or t.json field.
See the Postgres docs on JSON datatypes
Background: We use PaperTrail to keep the history of our changing models. Now I want to query for a Item, which belonged to a certain customer. PaperTrail optionally stores the object_changes and I need to query this field to understand, when something was created with this ID or changed to this ID.
My table looks simplified like this:
item_type | object_changes
----------|----------------------------------------------------------
"Item" | {"customer_id": [null, 5], "other": [null, "change"]}
"Item" | {"customer_id": [4, 5], "other": ["unrelated", "change"]}
"Item" | {"customer_id": [5, 6], "other": ["asht", "asht"]}
How do I query for elements changed from or to ID 5 (so all rows above)? I tried:
SELECT * FROM versions WHERE object_changes->'customer_id' ? 5;
Which got me:
ERROR: operator does not exist: jsonb ? integer
LINE 1: ...T * FROM versions WHERE object_changes->'customer_id' ? 5;
^
HINT: No operator matches the given name and argument type(s).
You might need to add explicit type casts.
For jsonb the contains operator #> does what you ask for:
Get all rows where the number 5 is an element of the "customer_id" array:
SELECT *
FROM versions
WHERE object_changes->'customer_id' #> '5';
The #> operator expects jsonb as right operand - or a string literal that is valid for jsonb (while ? expects text). The numeric literal without single quotes you provided in your example (5) cannot be coerced to jsonb (nor text), it defaults to integer. Hence the error message. Related:
No function matches the given name and argument types
PostgreSQL ERROR: function to_tsvector(character varying, unknown) does not exist
This can be supported with different index styles. For my query suggested above, use an expression index (specialized, small and fast):
CREATE INDEX versions_object_changes_customer_id_gin_idx ON versions
USING gin ((object_changes->'customer_id'));
This alternative query works, too:
SELECT * FROM versions WHERE object_changes #> '{"customer_id": [5]}';
And can be supported with a general index (more versatile, bigger, slower):
CREATE INDEX versions_object_changes_gin_idx ON versions
USING gin (object_changes jsonb_path_ops);
Related:
Index for finding an element in a JSON array
Query for array elements inside JSON type
According to the manual, the operator ? searches for any top-level key within the JSON value. Testing indicates that strings in arrays are considered "top-level keys", but numbers are not (keys have to be strings after all). So while this query would work:
SELECT * FROM versions WHERE object_changes->'other' ? 'asht';
Your query looking for a number in an array will not (even when you quote the input string literal properly). It would only find the (quoted!) string "5", classified as key, but not the (unquoted) number 5, classified as value.
Aside: Standard JSON only knows 4 primitives: string, number, boolean and null. There is no integer primitive (even if I have heard of software adding that), integer is a just a subset of number, which is implemented as numeric in Postgres:
https://www.postgresql.org/docs/current/static/datatype-json.html#JSON-TYPE-MAPPING-TABLE
So your question title is slightly misleading as there are no "integer" members, strictly speaking.
Use a lateral join and the jsonb_array_elements_text function to process each row's object_changes:
SELECT DISTINCT v.* FROM versions v
JOIN LATERAL jsonb_array_elements_text(v.object_changes->'customer_id') ids ON TRUE
WHERE ids.value::int = 5;
The DISTINCT is only necessary if the customer_id you're looking for could appear multiple times in the array (if a different field changed but customer_id is tracked anyway).
How to sort the Mongoid model by the length of the array which is a field inside the model.
Mongo documentation says:
You cannot use $size to find a range of sizes (for example: arrays
with more than 1 element). If you need to query for a range, create an
extra size field that you increment when you add elements. Indexes
cannot be used for the $size portion of a query, although if other
query expressions are included indexes may be used to search for
matches on that portion of the query expression.
So we cannot order by using mongo's $size.
You can solve your task by adding new field, which will store array size.
class Post
include Mongoid::Document
field :likes, type: Array, default: []
field :likes_size, type: Integer
before_save do
self.likes_size = likes.size
end
end
Sort posts by likes_size:
Post.order_by(likes_size: :desc)
Document says that you can't orderby using size.
Try adding a new column containing the value of size and sort it which will work as order by.
In ruby, you can sort an array like this :
my_array.sort_by(&:my_attr)
It will sort the array my_array by the attribute my_attr of each element inside the array.
You can also write it like this :
my_array.sort_by{|element| element.my_attr }
Which is exactly the same, it will sort by the my_attr attribute of each element. This second syntax is for when you want a more complex sort condition than just the result of a method of each element.
Documentation : http://ruby-doc.org/core-2.3.1/Enumerable.html#method-i-sort_by