Why does adding 20 elements cause allocation, but adding 19 does not - memory

I noted that an #generated function I was working with would sometimes allocate and sometimes not depending on one of the parameters.
I have narrowed it down to this simple case.
The cut off for allocation free addition seems to come down to adding 20 or more elements.
19 is fine 20 is not.
adding 10 twice is also fine.
See the code;
function add19(x)
#inbounds ret = x[1]+ x[2]+ x[3]+ x[4]+ x[5]+ x[6]+ x[7]+ x[8]+ x[9]+ x[10]+ x[11]+ x[12]+ x[13]+ x[14]+ x[15]+ x[16]+ x[17]+ x[18]+ x[19]
ret
end
function add20(x)
#inbounds ret = x[1]+ x[2]+ x[3]+ x[4]+ x[5]+ x[6]+ x[7]+ x[8]+ x[9]+ x[10]+ x[11]+ x[12]+ x[13]+ x[14]+ x[15]+ x[16]+ x[17]+ x[18]+ x[19]+ x[20]
ret
end
function add10twice(x)
#inbounds ret = (x[1]+ x[2]+ x[3]+ x[4]+ x[5]+ x[6]+ x[7]+ x[8]+ x[9]+ x[10]) + ( x[11]+ x[12]+ x[13]+ x[14]+ x[15]+ x[16]+ x[17]+ x[18]+ x[19]+ x[20])
ret
end
timings/allocations (for a = rand(Int8, 100)
#btime add19($a) 52.542 ns (0 allocations: 0 bytes)
#btime add20($a) 789.929 ns (20 allocations: 336 bytes)
#btime add10twice($a) 80.959 ns (0 allocations: 0 bytes)
So what is going on here?
(I have a theory it related to how varargs are handled)

Related

finding minimum values from a cut table Lua 5.1.5

I have a Lua script that turns a table into segments:
function tablecut(t, n)
local result = {}
local j = 0
for i = 1, #t do
if (i-1) % n == 0 then
j = j + 1
result[j] = {}
end
result[j][#result[j]+1] = t[i]
end
return result
end
output = tablecut({'15', '62', '14', '91', '33', '55', '29', '4'}, 4)
for i = 1, #output do
for j = 1, #output[i] do
io.write(tostring(output[i][j])..' ')
end
print()
end
output:
15 62 14 91
33 55 29 4
And I am trying to find the minima from the cut lists so the output would look like this:
15 62 14 91
min = 14
33 55 29 4
min = 4
Edit: If its of any importance this is how I got it to work on Lua 5.3 but there is no table.move function on Lua 5.1. I can't remember how my thought function worked when I wrote this code.
function indexOf(array, value)
for i, v in ipairs(array) do
if v == value then
return i
end
end
return nil
end
Indicies = {}
Answers = {}
function chunks(lst, size)
local i = 1
local count = 0
return function()
if i > #lst then return end
local chunk = table.move(lst, i, i + size -1, 1, {})
i = i + size
count = count + 1
return count, chunk
end
end
local a = {91,52,19,59,38,29,58,11,717,91,456,49,30,62,43,8,17,15,26,22,13,10,2,23} --Test list
for i, chunk in chunks(a, 4) do
x=math.min(a)
print(string.format("#%d: %s", i, table.concat(chunk, ",")))
table.sort(chunk)
print(math.min(chunk[1]))
table.insert(Answers, chunk[1])
table.insert(Indicies, (indexOf(a, chunk[1])))
Output:
#1: 91,52,19,59
19
#2: 38,29,58,11
11
#3: 717,91,456,49
49
your table cut function could be simplified, and your output for loop needs you use an iterator if you want to get an output simply like you do in your 5.3 script.
function cuttable(t,n)
local binned = {}
for i=1,#t,n do
local bin = {}
for j=1,n do
table.insert(bin, t[i + ((j - 1) % n)])
end
table.insert(binned, bin)
end
return binned
end
For the for loop, we can use ipairs on the output of cuttable keeping things pretty simple, then we just do the same steps of concat then sort and print out our results.
for k, bin in ipairs(cuttable(a,4)) do
local output = "#" .. k .. ":" .. table.concat(bin, ",")
table.sort(bin)
print(output)
print(bin[1])
end
Output
#1:91,52,19,59
19
#2:38,29,58,11
11
#3:717,91,456,49
49
#4:30,62,43,8
8
#5:17,15,26,22
15
#6:13,10,2,23
2
One way to implement the cutting would be using a for loop & unpack. I have handled the case of the length not being divisible by 4 after the for loop to (1) maximize performance (check doesn't need to be done every iteration) and (2) be able to directly pass the values to math.min, which doesn't accept nils.
for i = 1, math.floor(#t / 4), 4 do
print(unpack(t, i, i+4))
print("min = " .. math.min(unpack(t, i, i+4)))
end
-- If #t is not divisible by 4, deal with the remaining elements
local remaining = #t % 4
if remaining > 0 then
print(unpack(t, #t - remaining, remaining))
print("min = " .. math.min(unpack(t, #t - remaining, remaining)))
end

How to get the latest x entries of a table in Lua?

If I have (for example) a table with 300 entries, how would I get the latest x entries only?
I was thinking of doing the next, but I'm wondering if there is a better/more optimized way to do this exact thing.
local TestTable = {}
-- Populate table
for i = 1, 300, 1 do
print('Adding: ' .. i)
table.insert(TestTable , i)
end
-- Get latest x of table
function GetLatestFromTable(OriginalTable, Amount)
local TableLength = #OriginalTable
local Retval = {}
for i = 1, Amount, 1 do
if TableLength - i <= 0 then break end -- Dont allow to go under 0
table.insert(Retval, OriginalTable[TableLength - i])
print("Adding to Retval: " .. OriginalTable[TableLength - i] .. ' (Index: ' .. TableLength - i .. ')')
end
return Retval
end
print(#TestTable)
local LatestTable = GetLatestFromTable(TestTable, 10)
print(#LatestTable)
For keys in sequence (and values are string/number) a call to table.concat() allows range parameter.
local tab = {"One", "Two", "Three", "Four", "Five"}
print(table.concat(tab, '\n', #tab - 1, #tab)) -- Last two entries
See: table.concat()
As mentioned by #Luke100000, one way could be to use Lua custom iterators. In Lua, an iterator is a special function which, when called, will return the next value. It is made possible by the fact that functions are first-class citizen in Lua and they can refer to previous scope with a mecanism named closure.
To answer the question, one could start implement a general iterator over a given range.
function IterateRange (Table, Min, Max)
local ClosureIndex = Min - 1
local ClosureMax = math.min(Max, #Table)
local function Closure ()
if (ClosureIndex < ClosureMax) then
ClosureIndex = ClosureIndex + 1
return Table[ClosureIndex]
end
end
return Closure
end
IterateRange is a function returning an anonymous function. The anonymous function does not take any parameter. It simply update the ClosureIndex index defined in the local scope of IterateRange and return the table value.
The first thing that the anonymous function do is to increment ClosureIndex. For that reason, ClosureIndex must be initialized to Min - 1.
This function works as one might expect:
TestTable = {}
for i = 1, 300, 1 do
print('Adding: ' .. i)
table.insert(TestTable , i)
end
for Value in IterateRange(TestTable, 290, 300) do
print(Value)
end
290
291
292
293
294
295
296
297
298
299
300
Now, it's trivial to reuse this general iterator to iterate over the last N entries of a given table:
function IterateLastEntries (Table, Count)
local TableSize = #Table
local StartIndex = (TableSize - Count)
return IterateRange(Table, StartIndex, TableSize)
end
It also work as one might expect:
TestTable = {}
for i = 1, 300, 1 do
print('Adding: ' .. i)
table.insert(TestTable , i)
end
for Value in IterateLastEntries(TestTable, 10) do
print(Value)
end
290
291
292
293
294
295
296
297
298
299
300
And finally, to summarize all this in a fully copy & pasteable solution:
TestTable = {}
for i = 1, 300, 1 do
print('Adding: ' .. i)
table.insert(TestTable , i)
end
function IterateRange (Table, Min, Max)
local ClosureIndex = Min - 1
local ClosureMax = math.min(Max, #Table)
local function Closure ()
if (ClosureIndex < ClosureMax) then
ClosureIndex = ClosureIndex + 1
return Table[ClosureIndex]
end
end
return Closure
end
function IterateLastEntries (Table, Count)
local TableSize = #Table
local StartIndex = (TableSize - Count)
return IterateRange(Table, StartIndex, TableSize)
end
for Value in IterateLastEntries(TestTable, 10) do
print(Value)
end
This should return:
290
291
292
293
294
295
296
297
298
299
300
I will let the OP update the code in order to achieve the same results for 30 entries.

is there a way to split list to chunks?

I have a very large list (~1M items) and I want to unpack it.
I saw that it is impossible to unpack a list larger than 8000 items so I wanted to chunk it.
Basically this function in python
def chunks(lst, size):
for i in range(0, len(lst), size):
yield lst[i:i+size]
(for reference I am running the lua script on redis using eval)
You could write an iterator that gives you chunks of a certain size.
-- an iterator function that will traverse over lst
function chunks(lst, size)
-- our current index
local i = 1
-- our chunk counter
local count = 0
return function()
-- abort if we reached the end of lst
if i > #lst then return end
-- get a slice of lst
local chunk = table.move(lst, i, i + size -1, 1, {})
-- next starting index
i = i + size
count = count + 1
return count, chunk
end
end
-- test
local a = {1,2,3,4,5,6,7,8,9,10,11}
for i, chunk in chunks(a, 2) do
print(string.format("#%d: %s", i, table.concat(chunk, ",")))
end
Output:
#1: 1,2
#2: 3,4
#3: 5,6
#4: 7,8
#5: 9,10
#6: 11
Read this:
https://www.lua.org/pil/7.1.html
https://www.lua.org/manual/5.4/manual.html#3.3.5

How to get the bytes that an unsigned integer is composed of?

Supposing I've this number:
local uint = 2000;
how can I get the bytes that it's composed of? For instance:
print(sepbytes(uint));
-- 7, 208
My try:
local function sepbytes(cur)
local t = {};
repeat
cur = cur / 16;
table.insert(t, 1, cur);
until cur <= 0
return table.unpack(t);
end
print(sepbytes(2000));
This results in:
0 9.8813129168249e-324, +(lots of numbers)...
Expected result:
7, 208
Basing in the comments, if I want 2 fixed bytes (that's the current case), I may use #ajcr solution:
local function InParts(num)
return ((num & (0xff << 8)) >> 8), (num & 0xff);
end
#EgorSkriptunoff (Lua 5.3) solution works for any amount of bytes.
local function InParts(num)
return string.pack(">I2", uint):byte(1, -1);
end

formatting strings in lua in a pattern

I want to make a script that takes any number, counts up to them and returns them in a format.
so like this
for i = 1,9 do
print(i)
end
will return
1
2
3
4
5
6
7
8
9
however I want it to print like this
1 2 3
4 5 6
7 8 9
and I want it to work even with things more than 9 so things like 20 would be like this
1 2 3
4 5 6
7 8 9
10 11 12
13 14 15
16 17 18
19 20
I'm sure it can be done using the string library in lua but I am not sure how to use that library.
Any help?
function f(n,per_line)
per_line = per_line or 3
for i = 1,n do
io.write(i,'\t')
if i % per_line == 0 then io.write('\n') end
end
end
f(9)
f(20)
The for loop takes an optional third step:
for i = 1, 9, 3 do
print(string.format("%d %d %d", i, i + 1, i + 2))
end
I can think of 2 ways to do this:
local NUMBER = 20
local str = {}
for i=1,NUMBER-3,3 do
table.insert(str,i.." "..i+1 .." "..i+2)
end
local left = {}
for i=NUMBER-NUMBER%3+1,NUMBER do
table.insert(left,i)
end
str = table.concat(str,"\n").."\n"..table.concat(left," ")
And another one using gsub:
local NUMBER = 20
local str = {}
for i=1,NUMBER do
str[i] = i
end
-- Makes "1 2 3 4 ..."
str = table.concat(str," ")
-- Divides it per 3 numbers
-- "%d+ %d+ %d+" matches 3 numbers divided by spaces
-- (You can replace the spaces (including in concat) with "\t")
-- The (...) capture allows us to get those numbers as %1
-- The "%s?" at the end is to remove any trailing whitespace
-- (Else each line would be "N N N " instead of "N N N")
-- (Using the '?' as the last triplet might not have a space)
-- ^ e.g. NUMBER = 6 would make it end with "4 5 6"
-- The "%1\n" just gets us our numbers back and adds a newline
str = str:gsub("(%d+ %d+ %d+)%s?","%1\n")
print(str)
I've benchmarked both code snippets. The upper one is a tiny bit faster, although the difference is almost nothing:
Benchmarked using 10000 interations
NUMBER 20 20 20 100 100
Upper 256 ms 276 ms 260 ms 1129 ms 1114 ms
Lower 284 ms 280 ms 282 ms 1266 ms 1228 ms
Use a temporary table to contain the values until you print them:
local temp = {}
local cols = 3
for i = 1,9 do
if #temp == cols then
print(table.unpack(temp))
temp = {}
end
temp[#temp + 1] = i
end
--Last minute check for leftovers
if #temp > 0 then
print(table.unpack(temp))
end
temp = nil

Resources