After reading this topic and after experimenting a bit, I am trying to understand how the Lua length operator works when a table contains nil values.
Before I started to investigate, I thought that the length was simply the number of consecutive non-nil elements, starting at index 1:
print(#{nil}) -- 0
print(#{"o"}) -- 1
print(#{"o",nil}) -- 1
print(#{"o","o"}) -- 2
print(#{"o","o",nil}) -- 2
That looks pretty simple, right?
But my headache started when I accidentally added an element after a nil-terminated table:
print(#{"o",nil,"o"})
My guess was that it should probably print 1 because it would stop counting when the first nil is found. Or maybe it should print 2 if the length operator is greedy enough to look for non-nil elements after the first nil. But the above code prints 3.
So I’ve ran several other tests to see what happens:
-- nil before the end
print(#{nil,"o"}) -- 2
print(#{nil,"o","o"}) -- 3
print(#{"o",nil,"o"}) -- 3
-- several nil elements
print(#{"o",nil,nil}) -- 1
print(#{nil,"o",nil}) -- 0
print(#{nil,nil,"o"}) -- 3
I should mention that repl.it currently uses Lua 5.1.5 which is rather old, but if you test with the Lua demo, which currently uses Lua 5.3.5, you’ll get the same results.
By looking at those results and by looking at this answer, I assume that:
if the last element is not nil, the length operator returns the full size of the table, including nil entries if any
if the last element is nil, it counts the number of consecutive non-nil and stops counting at the first nil
Are those assumptions correct?
Can we predict a 100% well-defined behavior when a table contains one or several nil values?
The Lua documentation states that the length of a table is only defined if the table is a sequence. Does that mean that the length operator has undefined behavior for non-sequences?
Apart from the length operator, can nil values cause any trouble in a table?
We can predict some behaviour, but it is not standardised, and as such you should never rely on it. It's quite possible that the behaviour may change within this major version of Lua.
Should you ever need to fill a table with nil values, I suggest wrapping the table and replace holes with a unique placeholder value (eg. NIL={}; if v==nil then t[k]=NIL end, this is quite cheap to test against and safe.).
That said...
As there is even a difference in the result of # depending on how the table is defined, you'll have to distinguish between statically defined (constant) tables and dynamic defined (muted) tables.
Static table definitions:
#{nil,nil,nil,nil,nil, 1} -- 6
#{3, 2, nil, 1} -- 4
#{nil,nil,nil, 1, 1,nil} -- 0
#{nil,nil, 1, 1, 1,nil} -- 5
#{nil, 1, 1, 1, 1,nil} -- 5
#{nil,nil,nil,nil, 1,nil} -- 0
#{nil,nil, 1,nil, 1,nil,nil} -- 5
#{nil,nil,nil, 1,nil,nil, 1,nil} -- 4
Using this kind of definition, as long as the last value is non-nil, you will get a length equal to the position of the last value. If the last value is nil, Lua starts a (non-linear) search from the tail until it finds the first non-nil value.
Dynamic data definition
local x={}; x[5]=1;print(#x) -- 0
local x={}; x[1]=1;x[2]=1;x[3]=1;x[5]=1;print(#x) -- 3
local x={}; x[1]=1;x[2]=1;x[4]=1;x[5]=1;print(#x) -- 5
#{[5]=1} -- 0
local x={nil,nil,nil,1};x[5]=1;print(#x) -- 0
As soon as the table was changed once, the operator works the other way (that includes static definitions with []). If the first element is nil, # always returns 0, but if not it starts a search that I did not investigate further (I guess you can check the sources, though I don't think it's a standard binary search), until it finds a nil value that is preceded by a non-nil value.
As said before, relying on this behaviour is not a good idea, and invites lots of issues down the road. Though if you want to make a nasty unmaintainable program to mess with a colleague, that's a sure way to do it.
When a table is a sequence (all numeric keys start at 1 and there are no nil gaps), # is defined to be precisely the count of those elements.
For non-sequence tables, it is a bit more complicated. Lua 5.2 seems to leave the result as undefined. For 5.1 and 5.3, the result of the operation is a border.
A border in a table is any positive index that contains a non-nil value followed by nil, or 0 if the first element is nil. # is defined to return any value that satifies these conditions.
Looking at it from another perspective, since tables contain an "array" part and a "map" part, Lua has no way of knowing where the "map" indices start. For example, you can create a table with 1000 values and then set the first 999 of them to nil; that could leave you with a table of "size" 1000. However, you can also start with an empty table and set the 1000th element, having a table of "size" 0 but still structurally equivalent to the first one. The result of # is then simply the first valid value the internal algorithm finds.
The length operator produces undefined behaviour for tables that aren't sequences (i.e. tables with nil elements in the middle of the array). This means that even if the Lua implementation always behaves in a certain way, you shouldn't rely on that behaviour, as it may change in future versions of Lua, or in different implementations like LuaJIT.
You can use nils in tables - there is nothing wrong with that - just don't use the length operator on a table which might have nils before non-nil values.
The post you linked to contains more details about how the actual algorithm works. It mentions counting elements with a "binsearch", i.e. a binary search. This is not the same as just counting the elements one by one - if there are nils in the table, then depending on their exact position, the binary search algorithm may treat them as the end of the table, or may just ignore them.
To sum up, the algorithm is harder to predict than you were assuming, and even though it is technically possible to predict what will happen in any given case, you shouldn't rely on that behaviour as it is liable to change.
Related
I am working with a third party device which has some implementation of Lua, and communicates in BACnet. The documentation is pretty janky, not providing any sort of help for any more advanced programming ideas. It's simply, "This is how you set variables...". So, I am trying to just figure it out, and hoping you all can help.
I need to set a long list of variables to certain values. I have a userdata 'ME', with a bunch of variables named MVXX (e.g. - MV21, MV98, MV56, etc).
(This is all kind of background for BACnet.) Variables in BACnet all have 17 'priorities', i.e., every BACnet variable is actually a sort of list of 17 values, with priority 16 being the default. So, typically, if I were to say ME.MV12 = 23, that would set MV12's priority-16 to the desired value of 23.
However, I need to set priority 17. I can do this in the provided Lua implementation, by saying ME.MV12_PV[17] = 23. I can set any of the priorities I want by indexing that PV. (Corollaries - what is PV? What is the underscore? How do I get to these objects? Or are they just interpreted from Lua to some function in C on the backend?)
All this being said, I need to make that variable name dynamic, so that i can set whichever value I need to set, based on some other code. I have made several attempts.
This tells me the object(MV12_PV[17]) does not exist:
x = 12
ME["MV" .. x .. "_PV[17]"] = 23
But this works fine, setting priority 16 to 23:
x = 12
ME["MV" .. x] = 23
I was trying to attempt some sort of what I think is called an evaluation, or eval. But, this just prints out function followed by some random 8 digit number:
x = 12
test = assert(loadstring("MV" .. x .. "_PV[17] = 23"))
print(test)
Any help? Apologies if I am unclear - tbh, I am so far behind the 8-ball I am pretty much grabbing at straws.
Underscores can be part of Lua identifiers (variable and function names). They are just part of the variable name (like letters are) and aren't a special Lua operator like [ and ] are.
In the expression ME.MV12_PV[17] we have ME being an object with a bunch of fields, ME.MV12_PV being an array stored in the "MV12_PV" field of that object and ME.MV12_PV[17] is the 17th slot in that array.
If you want to access fields dynamically, the thing to know is that accessing a field with dot notation in Lua is equivalent to using bracket notation and passing in the field name as a string:
-- The following are all equivalent:
x.foo
x["foo"]
local fieldname = "foo"
x[fieldname]
So in your case you might want to try doing something like this:
local n = 12
ME["MV"..n.."_PV"][17] = 23
BACnet "Commmandable" Objects (e.g. Binary Output, Analog Output, and o[tionally Binary Value, Analog Value and a handful of others) actually have 16 priorities (1-16). The "17th" you are referring to may be the "Relinquish Default", a value that is used if all 16 priorities are set to NULL or "Relinquished".
Perhaps your system will allow you to write to a BACnet Property called "Relinquish Default".
I have an array named xTain of size nDatax1
I initialize it as
xTrain = torch.linspace(-1,1,nData)
To access the array, the author uses xTrain[{{i}}]
can you please explain this notation? Why not simply xTrain[i] ?
Please refer the author's code here on Pg No 21- https://www.cs.ox.ac.uk/people/nando.defreitas/machinelearning/lecture4.pdf
As an additional note-
xTrain=torch.linespace(-1,1,10)
when I do
th> print(xTrain[1])
-1
th> print(xTrain[{1}])
-1
th> print(xTrain[{{1}}])
-1
[torch.DoubleTensor of size 1]
Why does it also print [torch.DoubleTensor of size 1] in 3rd case. My guess is in first 2 case its returning a scalar value at that location and in 3rd case a DoubleTensor
Good place to start is the Lua manual, it's syntax and explessions. You can see, what is meaning of {...} in Lua:
{...} -- creates a list with all vararg parameters
So in short your {1} creates a list with single value 1. Repeating it once more you got a list containing list containing single number 1.
If the xTrain would be simple table, it would probably fail, because it is hard to index using lists, but Lua supports metatables so the actual value is not used for indexing the table, but passed to some function which takes care of the lists.
Also reading further about the Tensor class, which is returned from the torch.linespace() function is a good place to see. The indexing using "array access" is explained in the section [Tensor] [{ dim1,dim2,... }] or [{ {dim1s,dim1e}, {dim2s,dim2e} }]
This question already has answers here:
Lua for loop reduce i? Weird behavior [duplicate]
(3 answers)
Closed 7 years ago.
im trying this in lua:
for i = 1, 10,1 do
print(i)
i = i+2
end
I would expect the following output:
1,4,7,10
However, it seems like i is getting not affected, so it gives me:
1,2,3,4,5,6,7,8,9,10
Can someone tell my a bit about the background concept and what is the right way to modify the counter variable?
As Colonel Thirty Two said, there is no way to modify a loop variable in Lua. Or rather more to the point, the loop counter in Lua is hidden from you. The variable i in your case is merely a copy of the counter's current value. So changing it does nothing; it will be overwritten by the actual hidden counter every time the loop cycles.
When you write a for loop in Lua, it always means exactly what it says. This is good, since it makes it abundantly clear when you're doing looping over a fixed sequence (whether a count or a set of data) and when you're doing something more complicated.
for is for fixed loops; if you want dynamic looping, you must use a while loop. That way, the reader of the code is aware that looping is not fixed; that it's under your control.
When using a Numeric for loop, you can change the increment by the third value, in your example you set it to 1.
To see what I mean:
for i = 1,10,3 do
print(i)
end
However this isn't always a practical solution, because often times you'll only want to modify the loop variable under specific conditions. When you wish to do this, you can use a while loop (or if you want your code to run at least once, a repeat loop):
local i = 1
while i < 10 do
print(i)
i = i + 1
end
Using a while loop you have full control over the condition, and any variables (be they global or upvalues).
All answers / comments so far only suggested while loops; here's two more ways of working around this problem:
If you always have the same step size, which just isn't 1, you can explicitly give the step size as in for i =start,end,stepdo … end, e.g. for i = 1, 10, 3 do … or for i = 10, 1, -1 do …. If you need varying step sizes, that won't work.
A "problem" with while-loops is that you always have to manually increment your counter and forgetting this in a sub-branch easily leads to infinite loops. I've seen the following pattern a few times:
local diff = 0
for i = 1, n do
i = i+diff
if i > n then break end
-- code here
-- and to change i for the next round, do something like
if some_condition then
diff = diff + 1 -- skip 1 forward
end
end
This way, you cannot forget incrementing i, and you still have the adjusted i available in your code. The deltas are also kept in a separate variable, so scanning this for bugs is relatively easy. (i autoincrements so must work, any assignment to i below the loop body's first line is an error, check whether you are/n't assigning diff, check branches, …)
The, well, special specification of Lua's length operator made me wonder whether Lua would be "allowed" to return a negative value in a situation like
#{[-5]=1,[-1]=3}
It says:
The length of a table t is defined to be any integer index n such that t[n] is not nil and t[n+1] is nil;
n=-5 and n=-1 would meet this criterion in my example, right?
moreover, if t[1] is nil, n can be zero.
Right, it can be zero, but it's not guaranteed, right?
For a regular array, with non-nil values from 1 to a given n, its length is exactly that n, the index of its last value.
This isn't the case here, so it doesn't apply.
If the array has "holes" (that is, nil values between other non-nil values), then #t can be any of the indices that directly precedes a nil value (that is, it may consider any such nil value as the end of the array).
This is the case here, so again, n=-5 and n=-1 would be valid return values, right?
Can I be entirely certain that Lua always returns 0 for the example table, or any other table containing only negative indices? If (hypothetically) I'd be writing a Lua interpreter and would return either of those values, would I be conforming with the specifications?
Edit
Obviously, the way Lua is implemented, it does not return negative values. I felt the length operator is somewhat underdocumented and I see that Lua 5.2's documentation has changed. It now says:
Unless a __len metamethod is given, the length of a table t is only defined if the table is a sequence, that is, the set of its positive numeric keys is equal to {1..n} for some integer n. In that case, n is its length. Note that a table like
{10, 20, nil, 40}
is not a sequence, because it has the key 4 but does not have the key 3.
So, it now talks about positive numeric keys, that's much clearer. I'm left wiser but not totally happy with the documentation. When it says the "length is only defined if the table is a sequence", it should also state that even if the table is not a sequence a value is returned, but the behavior is undefined. Also, this table looks pretty much like a sequence:
a = setmetatable(
{0},
{
__index = function(t,k)
return k < 10 and k or nil
end
}
)
i = 1
while a[i] do
print(a[i])
i = i+1
end
--[[ prints:
0
2
3
4
5
6
7
8
9
]]
print(#a)
-- prints: 1
However, this is becoming nitpicking as it's pretty clear that it wouldn't make sense to take into account what mess __index might make. And Stackoverflow is certainly not the place to complain about documentation that could be more precise.
As you have noted, the specification of the length operator has changed between 5.1 and 5.2.
Can I be entirely certain that Lua always returns 0 for the example table, or any other table containing only negative indices?
You can for the current reference implementation, which ensures that for ilen defined
function ilen (xs)
local i=0
while xs[i+1] do i=i+1 end
return i
end
we always have #xs >= ilen(xs) - see the definition of luaH_getn in the ltable.c source. But the specification now deliberately does not promise this behaviour: a conformant implementation can return nil or raise an exception for attempts to find the length of tables that are not sequences.
From the text in reference link. The answer is NO.
I think your confusing the fact that if a NIL is found then the length of the table is deemed to be position the NIL was found -1.
Therefore if t(1) is NIL then 1 - 1 = 0 so the table length is 0.
If the length of a table was 5 then the next position or t(6) IS or WOULD BE NIL
The length of a table t is defined to be any integer index n such that t[n] is not nil and t[n+1] is nil; moreover, if t[1] is nil, n can be zero.
In Lua, there seem to be two ways of appending an element to an array:
table.insert(t, i)
and
t[#t+1] = i
Which should I use, and why?
Which to use is a matter of preference and circumstance: as the # length operator was introduced in version 5.1, t[#t+1] = i will not work in Lua 5.0, whereas table.insert has been present since 5.0 and will work in both. On the other hand, t[#t+1] = i uses exclusively language-level operators, wheras table.insert involves a function (which has a slight amount of overhead to look up and call and depends on the table module in the environment).
In the second edition of Programming in Lua (an update of the Lua 5.0-oriented first edition), Roberto Ierusalimschy (the designer of Lua) states that he prefers t[#t+1] = i, as it's more visible.
Also, depending on your use case, the answer may be "neither". See the manual entry on the behavior of the length operator:
If the array has "holes" (that is, nil values between other non-nil values), then #t can be any of the indices that directly precedes a nil value (that is, it may consider any such nil value as the end of the array).
As such, if you're dealing with an array with holes, using either one (table.insert uses the length operator) may "append" your value to a lower index in the array than you want. How you define the size of your array in this scenario is up to you, and, again, depends on preference and circumstance: you can use table.maxn (disappearing in 5.2 but trivial to write), you can keep an n field in the table and update it when necessary, you can wrap the table in a metatable, or you could use another solution that better fits your situation (in a loop, a local tsize in the scope immediately outside the loop will often suffice).
The following is slightly on the amusing side but possibly with a grain of aesthetics. Even though there are obvious reasons that mytable:operation() is not supplied like mystring:operation(), one can easily roll one's own variant, and get a third notation if desired.
Table = {}
Table.__index = table
function Table.new()
local t = {}
setmetatable(t, Table)
return t
end
mytable = Table.new()
mytable:insert('Hello')
mytable:insert('World')
for _, s in ipairs(mytable) do
print(s)
end
insert can insert arbitrarily (as its name states), it only defaults to #t + 1, where as t[#t + 1] = i will always append to the (end of the) table. see section 5.5 in the lua manual.
'#' operator only use indexed key table.
t = {1, 2 ,3 ,4, 5, x=1, y=2}
at above code
print(#t) --> print 5 not 7
'#' operator whenever not using.
If you want to '#' operator, then check it to table elements type.
Insert function can using any type use.But element count to work slow than '#'