I have a closure that update captured external variable. It works well when the closure is called in the same thread that variable is defined. But when I pass the closure to a child thread, it doesn't update the external variable as expected. What's happening when a closure is passed to a thread? Any docs about it?
# nim c -r --threads:on testClosure.nim
import strutils
template hexDumpAddrOf[T](v: T): string =
# echo fmt"size of addr: {sizeof(v.unsafeAddr)}, sizeof ByteAddress: {sizeof(ByteAddress)}"
var p = cast[array[sizeof(ByteAddress), uint8]](v.unsafeAddr)
var result = ""
for x in p:
result = x.toHex() & result
result
proc closureThreadProcWrapper(closure: proc() ) =
closure()
proc testClosureThread() =
var thr: Thread[proc()]
var output = #["first"] # to be updated by thead
echo " original addr of output: ", hexDumpAddrOf(output)
proc localClosure() =
# The address of captured output is different from the one when running in a child thread.
echo " localClosure addr of output: ", hexDumpAddrOf(output)
output.add "anything"
localClosure() # print the same addr as the original one. and print
echo "invoked closure directly and external var is updated by closure: ", output # print #["first", "anything"]
createThread(thr, closureThreadProcWrapper, localClosure) # print different addr of output ???
thr.joinThread
echo "invoked closure in child thread. external var doesn't update as expected: ", output # print #["first", "anything"] ???
when isMainModule:
testClosureThread()
The output is:
original addr of output: 00007F63349C8060
localClosure addr of output: 00007F63349C8060
invoked closure directly and external var is updated by closure: #["first", "anything"]
localClosure addr of output: 00007F63348C9060
invoked closure in child thread. external var doesn't update as expected: #["first", "anything"]
From the manual about the memory model for threads:
Nim's memory model for threads is quite different than that of other common programming languages (C, Pascal, Java): Each thread has its own (garbage collected) heap, and sharing of memory is restricted to global variables. This helps to prevent race conditions. GC efficiency is improved quite a lot, because the GC never has to stop other threads and see what they reference.
In your example output is a local variable, not global. Hence you're modifying the threads copy not the one from the "outside".
Related
Are the two codes equivalent?
Code 1:
with cp.cuda.Stream(non_blocking=False) as stream:
# do stuff
stream.synchronize()
Code 2:
with cp.cuda.Stream(non_blocking=False) as stream:
# do stuff
stream.synchronize()
Yes, these two are equivalent. with cp.cuda.Stream(non_blocking=False) as stream: switches the current stream of CuPy, i.e., all the code within the block will be performed on stream.
In CuPy, streams are not synchronized automatically on exiting the context manager so that users can fully control the synchronization timing. This is handy when dealing with multiple streams, e.g.:
s1 = cp.cuda.Stream(non_blocking=False)
s2 = cp.cuda.Stream(non_blocking=False)
for i in range(10):
with s1:
# do some work
with s2:
# do some work
s1.synchronize()
s2.synchronize()
I'm putting some code into a module so I can draw and maintain multiple copies. I'm getting this common error but I can't see why. I understand what it's saying to a basic level, but as I'm able to see a print out from the table being created, I don't understand why calling a function that module contains would throw this error.
I've read through all the answers on SO, but I'm still at a loss. I've tried printing out at various stages to see where the issue is, everything works as if I had created an instance of the module, but the error persists.
Code below is cleaned of extraneous stuff.
local orbitalCircle = include('lib/orbital_circle')
function init()
c1 = orbitalCircle.new(20, 42, 18, 1.7, 16, 62, 15, c1Sequence)
<-- at this point print code from the module's init function works
c1:doFunc(param) <-- this will call the error
The module:
local Orbital_Circle = {}
-- set up variables
local some Vars Are here
function Orbital_Circle.new(x, y, diameter, scale_factor, number_of_notes, beats_per_second, frames_per_second, sequence_data)
print("Orbital_Circle running")
end
function Orbital_Circle:doFunc(param)
self.var = param <-- I update a local var here
print("self.var") <-- I then print the updated number for sanity checking
end
return Orbital_Circle
I expect the var in my instance of this module to update and the functions code to run, but... no joy. I get the error.
Cheers.
I'm putting some code into a module so I can draw and maintain multiple copies.
I think there's a bit of a misunderstanding about how Lua modules work here. It's an easy mistake to make.
When you require a module in Lua, each subsequent require of the same file refers to the same code. So (eg) these two variables contain exactly the same code:
local orbitalCircle1 = require('lib/orbital_circle')
local orbitalCircle2 = require('lib/orbital_circle')
Which means that you can't use Lua modules by themselves to create OOP type objects as you are trying to do. Your new function must return something that can be used like an instance of a class, a unique table for each call:
local Orbital_Circle = {}
local shared_variable = 1
function Orbital_Circle.new(x, y)
-- create unique table
local obj = {}
-- access these from table/object methods with self.xxx
obj.x = x or 0
obj.y = y or 0
obj.var = "initial value"
-- now define functions with an explicit 'self' parameter...
function obj.doFunc(self, param)
self.var = self.var .. " " .. param
shared_variable = shared_variable + 1
end
-- ... or with the syntactic 'self' sugar, ':'
function obj:printVars()
print("self.var = " .. self.var)
print("shared_variable = " .. shared_variable)
print("self.x = " .. self.x)
end
return obj
end
return Orbital_Circle
You can also define the methods as local functions outside the new function that have self parameter and have a list of entries such as:
obj.anotherMethod = functionDeclaredAtTopOfFile
… to keep things tidier, if you like.
Your code is completely messed up.
<-- will cause an error for unexpected symbol.
c1 = orbitalCircle.new(20, 42, 18, 1.7, 16, 62, 15, c1Sequence)
will give you an error for indexing a global nil value c1 because orbitalCircle.new has no return value.
your init function is incomplete and you don't call it so the provided code does not do anything even if you fix the above errors.
The reported error is not caused by any line of code you provided here.
Code below is cleaned of extraneous stuff.
I'm afraid you removed too much.
The error message tells you that you're indexing local n, a nil value from within a local function that has been defined in n's scope.
This code for example:
local n
function test()
local b = n.a
end
test()
would result in the error message:
input:3: attempt to index a nil value (upvalue 'n')
n is an upvalue for test because it is a local variable defined outside the functions body, but not a global variable.
I've been stuck on this for a while now and I cannot seem to find good resources to my problem. I am coming from and "only C" background, so most of the web dev stuff is completely new for me.
I wrote a C function float editDistance(char *str1, char *str2) that returns the edit distance of 2 char arrays. Right now the goal is to successfully call this function from a JS environment.
After ensuring that the code works with the recommended Emscipten ccall method, I decided to move on. Now
I use Emscripten to compile the C code with flags -O3, -s WASM=1, -s EXPORTED_FUNCTIONS="['_editDistance']", and -s SIDE_MODULE=1 -s to Wasm. The JS code I'm trying to wrap around my WebAssembly is:
// Allocate memory for the wasm module to run in. (65536*256 bit)
let wasmMemory = new WebAssembly.Memory({
initial: 256
});
let info = {
env: {
abort: function() {},
memoryBase: 0,
tableBase: 0,
memory: wasmMemory,
table: new WebAssembly.Table({initial: 2, element: 'anyfunc'}),
}
}
// Define the strings
let str1 = "abcd";
let str2 = "abcd";
// Allocate memory on the wasm partition for the HEAPU8
let HEAPU8 = new Uint8Array(wasmMemory.buffer);
// Create the char arrays on the heap from the strings
let stackPtr = 0;
let str1Ptr = stackPtr;
stackPtr = stringToASCIIArray(str1, HEAPU8, stackPtr);
let str2Ptr = stackPtr;
stackPtr = stringToASCIIArray(str2, HEAPU8, stackPtr);
// Read the wasm file and instantiate it with the above environment setup. Then
// call the exported function with the string pointers.
let wasmBinaryFile = 'bin/edit_distanceW.wasm';
fetch(wasmBinaryFile, {credentials:"same-origin"})
.then((response) => response.arrayBuffer())
.then((binary) => WebAssembly.instantiate(binary,info))
.then((wa) => alert(wa.instance.exports._editDistance(str1Ptr, str2Ptr)));
// Converts a string to an ASCII byte array on the specified memory
function stringToASCIIArray(str, outU8Array, idx){
let length = str.length + 1;
let i;
for(i=0; i<length; i++){
outU8Array[idx+i] = str.charCodeAt(i);
}
outU8Array[idx+i]=0;
return (idx + length);
}
The generated wasm file when converted to wat demands these imports:
(import "env" "abort" (func (;0;) (type 0)))
(import "env" "memoryBase" (global (;0;) i32))
(import "env" "tableBase" (global (;1;) i32))
(import "env" "memory" (memory (;0;) 256))
(import "env" "table" (table (;0;) 2 anyfunc))
.. and exports these:
(export "__post_instantiate" (func 7))
(export "_editDistance" (func 9))
(export "runPostSets" (func 6))
(elem (;0;) (get_global 1) 8 1))
Now, when I test the code the strings are passed to the C module without a problem. A few function calls are even made on them (strLen) before things go south. In the C function there is this nasty nested loop that does the main computation, iterating thru a 2D array while reading the characters from the strings (C code just been ported from a paper with an ugly pseudo code, so pardon me the variable names):
do{
for(p=0; p<editDistance; p++){
// Do stuff
}
// Do more stuff
editDistance++;
} while(fkp[len2*2-len1][editDistance] != len1);
Before the function enters the for() loop, the module still has the strings on memory str1Ptr=0x00 and str2Ptr=0x05 with the correct length and content. On the contrary, immediately after entering the for() loop the memory gets overwritten by garbage (mostly 0s), corrupting the end result. I suspect some stack saving and restoration problems on the scope change, as the exact same code compiled to my PC using gcc works like a charm.
Any idea what setup I'm missing that hinders the correct completion of the C function?
If you are starting out you probably want to use the emscripten-generated JS glue. That is, don't use SIDE_MODULE=1 and instead output to a files calle .js. The emscripten compiler will then generate both a .js and a .wasm file. You can then include the .js file in your project and it will handle all the loading and setup for you.
If you try to load the wasm file yourself, you will need to do a lot of work to replicate the emscripten environment, which will require a lot of internal details of emscripten. Also, those internal details of subject to change when you update to the new version of emscripten so you are creating more work for yourself.
With the below code from the Lua demo page, I was trying to get the name of the function being pcalled.
function test()
local info = debug.getinfo(1);
for k, v in pairs(info) do
print(k, v);
end;
end;
pcall(function()
test();
end);
This was a success, as I received the following output, containing the name:
source =input
func function: 0x25a1830
nparams 0
short_src input
isvararg false
name test
namewhat global
istailcall false
linedefined 1
lastlinedefined 7
nups 1
currentline 2
what Lua
If I change the code to the following, I no longer get that information:
function test()
local info = debug.getinfo(1);
for k, v in pairs(info) do
print(k, v);
end;
end;
pcall(test);
The output is as follows:
func function: 0x21ee790
linedefined 1
nups 1
short_src input
namewhat
lastlinedefined 7
isvararg false
istailcall false
what Lua
source =input
currentline 2
nparams 0
If, however, I change the code to the following, I can obtain the name of the function passed to pcall:
function test()
local traceback = debug.traceback();
print(traceback);
end;
pcall(test);
With the output being as follows:
stack traceback:
input:2: in function 'test'
[C]: in function 'pcall'
input:7: in main chunk
[C]: in function 'pcall'
demo.lua:49: in main chunk
[C]: in ?
How does debug.traceback get this extra information, and using solely Lua is there a way to get it without extracting it from debug.traceback's return value?
debug.getinfo and debug.traceback get their info from various sources, some of them hacky. For instance, the function name is generally extracted from the source code of the calling function: whatever name the code used to look up the thing it called, that's what gets used as the name. (That's why your second code snippet didn't give you the name: pcall doesn't have Lua bytecode backing it up, so it can't tell you "test" unless there's a function in the middle to call it "test".) Lua functions do not have inherent names, any more than Lua integers do; a function is just another kind of value, which can be automatically assigned to a variable via some syntactic sugar.
Functions are values. Values don't have names.
Functions are not "declared" as in some other languages. A function value is created when a function definition is evaluated.
Debugging output is just trying to be helpful in simple cases by giving the name of a variable associated with calling the function value.
I'm attempting to sandbox some functions using setfenv, and I recieve the following output:
123
nil
Why is testValue when calling sandboxTest() nil, but it's 123 when it's accessed in callSandboxedTest()?
Using LuaJIT 2.1.0-beta2 (Lua 5.1)
function sandboxTest()
print(testValue)
end
local aNumber = 123
function callSandboxedTest()
setfenv(1, {
print = print,
testValue = aNumber,
sandboxTest = sandboxTest
})
print(testValue)
sandboxTest()
end
callSandboxedTest()
Environments aren't part of the call stack. Every function has its own environment. So sandboxTest has an environment, as does callSandboxTest. Changing the environment of one function has no effect on the environment of another.
sandboxTest will continue to use the default environment, so it will access the regular global table to find testValue. And since testValue is never set in the global table, sandboxTest will get nil.
That's why, when maintaining a sandbox, it is very important to carefully choose what functionality to expose to the sandbox. If a function needs to be part of the sandbox, then that function needs to have its environment set.
That's why it's best to sandbox based on compiled Lua chunks rather than individual functions. When creating functions, the functions created will inherit the current environment.
You haven't modified the environment that sandboxTest is using; you only modified the environment of the current function. You can use setfenv to set the environment of a particular function by passing a function name (passing a number modifies the environment of a function in the call stack):
setfenv(sandboxTest, {
print = print,
testValue = aNumber,
sandboxTest = sandboxTest
})
This will print 123 123.