I have data consisting of n key-value pairs in a table, A
I'd like to produce another table, B of size (n+1) x (n+1), where the first row/column are the keys of the original table, and entry (i,j) is some function of the ith and jth value
Ex:
A:
K|V
---
a 1
b 2
c 3
B:
a b c
a f(1,1) f(1,2) f(1,3)
b f(2,1) f(2,2) f(2,3)
c f(3,1) f(3,2) f(3,3)
Depends on the function you need. Assuming B2:B4 contains {1,2,3} , The following can be done. Each will use different functions and will add or subtract in different ways. The first one is the only one, that'll work in the exact way you asked, but that's just for Matrix multiplication - You could maybe use that as base and do other functions on it as needed.
=ARRAYFORMULA(MMULT(--(B2:B4), --TRANSPOSE(B2:B4)))
=ARRAYFORMULA((--(B2:B4)/--TRANSPOSE(B2:B4)))
=ARRAYFORMULA((--(B2:B4)+--TRANSPOSE(B2:B4)))
=ARRAYFORMULA((--(B2:B4)---TRANSPOSE(B2:B4)))
Related
The reference manual has this to say about the table.move function, introduced in Lua 5.3:
table.move (a1, f, e, t [,a2])
Moves elements from table a1 to table a2, performing the equivalent to the following multiple assignment: a2[t],··· = a1[f],···,a1[e]. The default for a2 is a1. The destination range can overlap with the source range. The number of elements to be moved must fit in a Lua integer.
This description leaves a lot to be desired. I'm hoping for a general, canonical explanation of the function that goes into more detail than the reference manual. (Oddly, I could not find such an explanation anywhere on the web, perhaps because the function is fairly new.)
Particular points I am still confused on after reading the reference manual's explanation a few times:
When it says "move", that means the items are being removed from their original location, correct? Do the indices of items above the removed items shift down to fill the gaps? If so, and we're moving within the same table, does t point to the original location before anything starts moving?
Is there some significance to the choice of index letters f, e, and t?
There is no similar function in any other language I know. What's an example of how I might use this? Since it's one of only seven table functions, I presume it's quite useful.
Moves elements from table a1 to table a2, performing the equivalent to the following multiple assignment a2[t],··· = a1[f],···,a1[e]
Maybe they could have added the information this is done using consecutive integer values from f to e.
If you know Lua a bit more you'll know that a Lua table has no order. So the only way to make that code work is to use consecutive integer keys. Especially as the documentation mentions a source range.
Giving the equivalent syntax is the most unambiguous way of describing a function.
If you know the very basic concept of multiple assignment in Lua (see 3.3.3. Assignment) , you know what this function does.
table.move(a1, 1, 4, 6, a2) would copy a1[1], a1[2], a1[3], a1[4] into a2[6], a2[7], a2[8], a2[9]
The most common usecase is probably to get a subset of a list.
local values = {1,45,1,44,123,2354,321,745,1231}
old syntax:
local subset = {}
for i = 3, 7 do
table.insert(subset, values[i])
end
new:
local subset = table.move(values, 5, 7, 1, {})
Or maybe you quickly want to remove the last 3 values from a table?
local a = {1,2,3,4,5,6,7}
table.move({}, 1,3,#a-2, a)
How long will it take for a program to print a truth table for n propositional symbols?
(Symbols: P1, P2, ..., Pn)
Can't seem to crack this question, not quite sure how to calculate this instance.
It will take time proportional to at least n*2^n. Each of the propositional symbols can assume one of two values - true or false. The portion of the truth table that lists all possible assignments of n such variables must have at least 2 * 2 * … * 2 (n times) = 2^n rows of n entries each; and that's not even counting the subexpressions that make up the rest of the table. This lower bound is tight since we can imagine the proposition P1 and P2 and … and Pn and the following procedure taking time Theta(n*2^n) to write out the answer:
fill up P1's column with 2^(n-1) TRUE and then 2^(n-1) FALSE
fill up P2's column with 2^(n-2) TRUE and then 2^(n-2) FALSE, alternating
…
fill up Pn's column with 1 TRUE and 1 FALSE, alternating
fill up the last column with a TRUE at the top and FALSE all the way down
If you have more complicated propositions then you should probably take the number of subexpressions as another independent variable since that could have an asymptotically relevant effect (using n propositional symbols, you can have arbitrarily many unique subexpressions that must be given their own columns in a complete truth table).
I have a data in below format where 1st column represents the products node, all the following columns represent properties of the products. I want to apply content based filtering algo using cosine similarity in Neo4j. For that, I believe, I need to define the fx columns as the properties of each product node and then call these properties as a vector and then apply cosine similarity between the products. I am having trouble doing two things:
1. How to define these columns as properties in one go(as the columns could be more than 100).
2. How to call all the property values as a vector to be able to apply cosine similarity.
Product f1 f2 f3 f4 f5
P1 0 1 0 1 1
P2 1 0 1 1 0
P3 1 1 1 1 1
P4 0 0 0 1 0
You can use LOAD CSS to input your data.
For example, this query will read in your data file and output for each input line (ignoring the header line) a name string and a props collection:
LOAD CSV FROM 'file:///data.csv' AS line FIELDTERMINATOR ' '
WITH line SKIP 1
RETURN HEAD(line) AS name, [p IN TAIL(line) | TOFLOAT(p)] AS props
Even though your data has a header line, the above query skips over it, as it is not needed. In fact, we don't want to use the WITH HEADERS option of LOAD CSV, since that would convert each data line into a map, whereas it is more convenient for our current purposes to get each data line as a collection of values.
The above query assumes that all the columns are space-separated, that the first column will always contain a name string, and that all other columns contain the numeric values that should be put into the same collection (named props).
If you replace RETURN with WITH, you can append additional clauses to the query that make use of the name and props values.
I have a table of objects, and a user can choose an object in the table at any given order in the table and place it in another slot in the table. When that happens I need the table to shift from the selected dropped slot and fill the empty slot. Not a swap, that's easy, but a shift at the point of placement.
so if I have this as a highly simplified example of my table
t = {a, b, c, d, e, f}
and the user chooses, say e, and wants to drop it into slot b. how best would I
have e take the b slot
have all the values from "b to d shift right and then also fill
the empty e slot?
how would I handle this shift no matter what one is chosen and where
its moved in the table efficiently no matter what size the table
might be?
Here is an implementation of shift using table.move which is efficient and available in Lua 5.3 as #lhf mentioned:
function shift(t, old, new)
local value = t[old]
if new < old then
table.move(t, new, old - 1, new + 1)
else
table.move(t, old + 1, new, old)
end
t[new] = value
end
If you want to move the item at position old to position new as you describe, you can use this:
table.insert(t, new, table.remove(t,old))
Here is your example:
t = {10,20,30,40,50,60}
print(table.concat(t, ','))
old = 5
new = 2
table.insert(t, new, table.remove(t,old))
print(table.concat(t, ','))
As for efficiency, the code above does shift some elements twice when they could have stayed where they were, but this will probably not matter unless the table is huge.
In Lua 5.3, you can probably do something better with table.move.
I want to use the FOREACH like:
a:{a_attr:chararray}
b:{b_attr:int}
FOREACH a {
res = CROSS a, b;
-- some processing
GENERATE res;
}
By this I mean to make for each element of a a cross-product with all the elements of b, then perform some custom filtering and return tuples.
==EDIT==
Custom filetering = res_filtered = FILTER res BY ...;
GENERATE res_filtered.
==EDIT-2==
How to do it with a nested CROSS no more no less inside a FOR loop without prior GROUP or COGROUP?
Depending on the specifics of your filtering, you may be able to design a limited set of disjoint classes of elements in a and b, and then JOIN on those. For example:
If your filtering rules are
if a_attr starts with "Foo" and b is 4, accept
if a_attr starts with "Bar" and b is greater than 17, accept
if a_attr begins with a letter in [m-z] and b is less than 0, accept
otherwise, reject
Then you can write a UDF that will return 1 for items satisfying the first rule, 2 for the second, 3 for the third, and NULL otherwise. Your CROSS/FILTER then becomes
res = JOIN a BY myUDF(a), b BY myUDF(b);
Pig drops null values in JOINs, so only pairs satisfying your filtering criteria will be passed.
CROSS generates a cross-product of all the tuples in each relation. So there is no need to have a nested FOREACH. Just do the CROSS and then FILTER:
a: {a_attr: chararray}
b: {b_attr: int}
crossed = CROSS a, b;
crossed: {a::a_attr: chararray,b::b_attr: int}
res = FILTER crossed BY ... -- your custom filtering
If you have the FILTER immediately after the CROSS, you should not have (unnecessary) excessive IO trouble from the CROSS writing the entire cross-product to disk before filtering. Records that get filtered will never be written at all.