I learned that require only loads a file once it's loaded (in package.loaded). However I see the following behavior and would like to get some pointers on what is happening.
Say I have a file lib/q.lua:
local q = {}
local function dep()
return 1;
end
q.dep = dep;
return q;
Another file lib/p.lua:
local q = require('./lib/q');
local p = {}
local function result()
return q;
end
p.result = result;
return p;
Another file main.lua:
local p = require('./lib/p');
local q = require('./lib/q');
local assert = require('luassert');
local result = p.result();
print(result);
print(package.loaded['./lib/q']);
print(q);
assert.is_equal(q, result);
The value of package.loaded['./lib/q'] is the same as result. However its value is different from the local q in main.lua. I expected them to be the same. Is there documentation on this or I'm missing something.
New to Lua. Any help is greatly appreciated. Thanks.
I need some help converting a luajit pointer to a string and back.
First I define the ctype:
ffi.cdef[[
typedef struct {
unsigned char Bytes[16];
} EncryptionKeys[100000000];
void* malloc(size_t);
void free(void*);
]]
Then use malloc to allocate some memory and then create the 'EncryptionKeys' variable.
local EncryptionKeyMemoryAddress = ffi.C.malloc(ffi.sizeof("EncryptionKeys"))
local EncryptionKeys = ffi.cast("EncryptionKeys(&)", EncryptionKeyMemoryAddress)
I first convert the variable into a lua string using:
ffi.string(EncryptionKeyMemoryAddress)
But I can't figure out how to convert it back!
Can someone please help me?
FYI: I am passing the 'EncryptionKeyMemoryAddress' variable to one of the function parameters for a lua lane (https://lualanes.github.io/lanes/).
Edit:
Here is the section of code that I am working on:
This is for the client managers module of my server that manages a list of lua states that all have access to any clients connected to the server. They all use a shared section of memory that I want them to have access to using a pointer.
local ClientFFIString = [[
typedef struct {
unsigned char Bytes[16];
} EncryptionKeys[100000000];
void* malloc(size_t);
void free(void*);
]]
ffi.cdef(Matchpools.FFIString)
local EncryptionKeyMemoryAddress = ffi.C.malloc(ffi.sizeof("EncryptionKeys"))
--------------------------------------------
function ClientManagers.CreateNewClientManager()
local EncryptionKeys = ffi.cast("EncryptionKeys(&)", EncryptionKeyMemoryAddress)
EncryptionKeys[0].Bytes[0] = 24
print("___a", EncryptionKeys[0].Bytes[0])
local NewIndex = #ClientManagers.List+1
ClientManagers.List[NewIndex] = ClientManagerFunc(
ClientFFIString,
ffi.string(EncryptionKeysMemoryAddress)
)
end
--------------------------------------------
local ClientManagerFunc = Lanes.gen("*", function(ClientFFIString, EncryptionKeysMemoryAddress)
ffi = require("ffi")
ffi.cdef(ClientFFIString)
local EncryptionKeys = ffi.cast("EncryptionKeys(&)", EncryptionKeyMemoryAddress)
print("___a", EncryptionKeys[0].Bytes[0])
-- I want this to be 24 just like it is in the function that created this lua state
local ClientManagerRunning = true
while ClientManagerRunning do
--local dt = GetDt()
--UpdateClientData(dt)
--UpdateMatchmaking(dt)
end
end)
You can convert Lua string to your structure pointer (and later use it as an array):
ffi.cdef"typedef struct {unsigned char Bytes[16];} Key;"
ptr=ffi.cast("Key *", your_string)
print("First Byte of the First Key in your Array:", ptr[0].Bytes[0])
UPDATE:
Let's test how it works for an array containing three keys:
local ffi = require'ffi'
ffi.cdef"typedef struct {unsigned char Bytes[16];} Key;"
local your_string = string.char(11):rep(16)..string.char(22):rep(16)..string.char(33):rep(16)
local ptr=ffi.cast("Key *", your_string)
print("First Byte of the First Key in your Array:", ptr[0].Bytes[0])
print("First Byte of the Second Key in your Array:", ptr[1].Bytes[0])
print("First Byte of the Third Key in your Array:", ptr[2].Bytes[0])
It prints 11, 22, 33
UPDATE 2:
Pass the address of buffer instead of the content of buffer
In the main thread
-- allocate the buffer
local Keys = ffi.cast("EncryptionKeys&", ffi.C.malloc(ffi.sizeof("EncryptionKeys")))
-- write to the buffer
Keys[2].Bytes[5] = 42
-- create string containing 64-bit address
local string_to_send = tostring(ffi.cast("uint64_t", Keys))
-- Send string_to_send to a lane
Inside the lane
-- receive the string
local received_string = .....
-- restore the buffer pointer
local Keys = ffi.cast("EncryptionKeys&", loadstring("return "..received_string)())
-- read the data from the buffer
print(Keys[2].Bytes[5])
I am using the following program to copy one file another. I am often seeing source and destination is not exactly the same (md5sum is different). Are there anything wrong with the below code?
local size = 2^13 -- good buffer size (8K)
local params = {...}
local srcfile = params[1]
local outfile = params[1] .. "_copy"
print (srcfile)
print (outfile)
local inf = io.open(srcfile, "r")
local of = io.open(outfile, "w")
while true do
local block = inf:read(size)
print(size)
if not block then break end
of:write(block)
end
inf:close()
of:close()
Thanks,
GL
You may want to use binary mode to ensure endline characters were not modified.
local inf = io.open(srcfile, "rb")
local of = io.open(outfile, "wb")
I wrote simple class to compress data. Here it is:
LZWCompressor = {}
function LZWCompressor.new()
local self = {}
self.mDictionary = {}
self.mDictionaryLen = 0
-- ...
self.Encode = function(sInput)
self:InitDictionary(true)
local s = ""
local ch = ""
local len = string.len(sInput)
local result = {}
local dic = self.mDictionary
local temp = 0
for i = 1, len do
ch = string.sub(sInput, i, i)
temp = s..ch
if dic[temp] then
s = temp
else
result[#result + 1] = dic[s]
self.mDictionaryLen = self.mDictionaryLen + 1
dic[temp] = self.mDictionaryLen
s = ch
end
end
result[#result + 1] = dic[s]
return result
end
-- ...
return self
end
And i run it by:
local compressor = LZWCompression.new()
local encodedData = compressor:Encode("I like LZW, but it doesnt want to compress this text.")
print("Input length:",string.len(originalString))
print("Output length:",#encodedData)
local decodedString = compressor:Decode(encodedData)
print(decodedString)
print(originalString == decodedString)
But when i finally run it by lua, it shows that interpreter expected string, not Table. That was strange thing, because I pass argument of type string. To test Lua's logs, i wrote at beggining of function:
print(typeof(sInput))
I got output "Table" and lua's error. So how to fix it? Why lua displays that string (That i have passed) is a table? I use Lua 5.3.
Issue is in definition of method Encode(), and most likely Decode() has same problem.
You create Encode() method using dot syntax: self.Encode = function(sInput),
but then you're calling it with colon syntax: compressor:Encode(data)
When you call Encode() with colon syntax, its first implicit argument will be compressor itself (table from your error), not the data.
To fix it, declare Encode() method with colon syntax: function self:Encode(sInput), or add 'self' as first argument explicitly self.Encode = function(self, sInput)
The code you provided should not run at all.
You define function LZWCompressor.new() but call CLZWCompression.new()
Inside Encode you call self:InitDictionary(true) which has not been defined.
Maybe you did not paste all relevant code here.
The reason for the error you get though is that you call compressor:Encode(sInput) which is equivalent to compressor.Encode(self, sInput). (syntactic sugar) As function parameters are not passed by name but by their position sInput inside Encode is now compressor, not your string.
Your first argument (which happens to be self, a table) is then passed to string.len which expects a string.
So you acutally call string.len(compressor) which of course results in an error.
Please make sure you know how to call and define functions and how to use self properly!
I'd like to do the equivalent of:
foo=$(echo "$foo"|someprogram)
within lua -- ie, I've got a variable containing a bunch of text, and I'd like to run it through a filter (implemented in python as it happens).
Any hints?
Added: would really like to do this without using a temporary file
As long as your Lua supports io.popen, this problem is easy. The solution is exactly as you have outlined, except instead of $(...) you need a function like this one:
function os.capture(cmd, raw)
local f = assert(io.popen(cmd, 'r'))
local s = assert(f:read('*a'))
f:close()
if raw then return s end
s = string.gsub(s, '^%s+', '')
s = string.gsub(s, '%s+$', '')
s = string.gsub(s, '[\n\r]+', ' ')
return s
end
You can then call
local foo = ...
local cmd = ("echo $foo | someprogram"):gsub('$foo', foo)
foo = os.capture(cmd)
I do stuff like this all the time. Here's a related useful function for forming commands:
local quote_me = '[^%w%+%-%=%#%_%/]' -- complement (needn't quote)
local strfind = string.find
function os.quote(s)
if strfind(s, quote_me) or s == '' then
return "'" .. string.gsub(s, "'", [['"'"']]) .. "'"
else
return s
end
end
I stumbled on this post while trying to do the same thing and never found a good solution, see the code below for how I solved my issues. This implementation allows users to access stdin, stdout, stderr and get the return status code. A simple wrapper is called for simple pipe calls.
require("posix")
--
-- Simple popen3() implementation
--
function popen3(path, ...)
local r1, w1 = posix.pipe()
local r2, w2 = posix.pipe()
local r3, w3 = posix.pipe()
assert((r1 ~= nil or r2 ~= nil or r3 ~= nil), "pipe() failed")
local pid, err = posix.fork()
assert(pid ~= nil, "fork() failed")
if pid == 0 then
posix.close(w1)
posix.close(r2)
posix.dup2(r1, posix.fileno(io.stdin))
posix.dup2(w2, posix.fileno(io.stdout))
posix.dup2(w3, posix.fileno(io.stderr))
posix.close(r1)
posix.close(w2)
posix.close(w3)
local ret, err = posix.execp(path, unpack({...}))
assert(ret ~= nil, "execp() failed")
posix._exit(1)
return
end
posix.close(r1)
posix.close(w2)
posix.close(w3)
return pid, w1, r2, r3
end
--
-- Pipe input into cmd + optional arguments and wait for completion
-- and then return status code, stdout and stderr from cmd.
--
function pipe_simple(input, cmd, ...)
--
-- Launch child process
--
local pid, w, r, e = popen3(cmd, unpack({...}))
assert(pid ~= nil, "filter() unable to popen3()")
--
-- Write to popen3's stdin, important to close it as some (most?) proccess
-- block until the stdin pipe is closed
--
posix.write(w, input)
posix.close(w)
local bufsize = 4096
--
-- Read popen3's stdout via Posix file handle
--
local stdout = {}
local i = 1
while true do
buf = posix.read(r, bufsize)
if buf == nil or #buf == 0 then break end
stdout[i] = buf
i = i + 1
end
--
-- Read popen3's stderr via Posix file handle
--
local stderr = {}
local i = 1
while true do
buf = posix.read(e, bufsize)
if buf == nil or #buf == 0 then break end
stderr[i] = buf
i = i + 1
end
--
-- Clean-up child (no zombies) and get return status
--
local wait_pid, wait_cause, wait_status = posix.wait(pid)
return wait_status, table.concat(stdout), table.concat(stderr)
end
--
-- Example usage
--
local my_in = io.stdin:read("*all")
--local my_cmd = "wc"
--local my_args = {"-l"}
local my_cmd = "spamc"
local my_args = {} -- no arguments
local my_status, my_out, my_err = pipe_simple(my_in, my_cmd, unpack(my_args))
-- Obviously not interleaved as they would have been if printed in realtime
io.stdout:write(my_out)
io.stderr:write(my_err)
os.exit(my_status)
There is nothing in the Lua standard library to allow this.
Here is an in-depth exploration of the difficulties of doing bidirectional communication properly, and a proposed solution:
if possible, redirect one end of the stream (input or output) to a file. I.e.:
fp = io.popen("foo >/tmp/unique", "w")
fp:write(anything)
fp:close()
fp = io.open("/tmp/unique")
x = read("*a")
fp:close()
You may be interested in this extension which adds functions to the os and io namespaces to make bidirectional communication with a subprocess possible.
Aha, a possibly better solution:
require('posix')
require('os')
require('io')
function splat_popen(data,cmd)
rd,wr = posix.pipe()
io.flush()
child = posix.fork()
if child == 0 then
rd:close()
wr:write(data)
io.flush()
os.exit(1)
end
wr:close()
rd2,wr2 = posix.pipe()
io.flush()
child2 = posix.fork()
if child2 == 0 then
rd2:close()
posix.dup(rd,io.stdin)
posix.dup(wr2,io.stdout)
posix.exec(cmd)
os.exit(2)
end
wr2:close()
rd:close()
y = rd2:read("*a")
rd2:close()
posix.wait(child2)
posix.wait(child)
return y
end
munged=splat_popen("hello, world","/usr/games/rot13")
print("munged: "..munged.." !")
A not very nice solution that avoids a temporary file...
require("io")
require("posix")
x="hello\nworld"
posix.setenv("LUA_X",x)
i=popen('echo "$LUA_X" | myfilter')
x=i.read("*a")
Here is how I solved the problem, it require lua posix.
p = require 'posix'
local r,w = p.pipe()
local r1,w1 = p.pipe()
local cpid = p.fork()
if cpid == 0 then -- child reads from pipe
w:close()
r1:close()
p.dup(r, io.stdin)
p.dup(w1 ,io.stdout)
p.exec('./myProgram')
r:close()
w1:close()
p._exit(0)
else -- parent writes to pipe
IN = r1
OUT = w
end
During myProgram execution, you'l read and write from normal io and after this part of code you just have to write/read on IN and OUT to comunicate with child program.
For a system I have running Lua 5.1 and luaposix 35.0-1, I started with the solution from Anthony Towns from this current page and made it work for this luaposix version for the purpose of calling openssl for encryption on a system without any other encryption capabilities. I have attempted to make the code more explicit in order to allow others to handle any potential API changes in luaposix in the future.
local posix = require('posix');
require('os');
require('io');
local function getOutputFromProcessProvidedInput( dataForProcess, command, commandArguments )
local MAXIMUM_BYTE_READ_COUNT = 100;
local readFileHandle1,writeFileHandle1 = posix.pipe()
io.flush();
local childProcessId1 = posix.fork();
if (childProcessId1 == 0)
then
posix.close( readFileHandle1 );
posix.write( writeFileHandle1, dataForProcess );
io.flush();
os.exit( 1 );
end
posix.close( writeFileHandle1 );
local readFileHandle2,writeFileHandle2 = posix.pipe();
io.flush();
local childProcessId2 = posix.fork();
if (childProcessId2 == 0)
then
posix.close( readFileHandle2 );
posix.dup2( readFileHandle1, posix.fileno( io.stdin ) );
posix.dup2( writeFileHandle2, posix.fileno( io.stdout ) );
posix.execp( command, commandArguments );
os.exit( 2 );
end
posix.close( writeFileHandle2 );
posix.close( readFileHandle1 );
local dataFromProcess = posix.read( readFileHandle2, MAXIMUM_BYTE_READ_COUNT );
posix.close( readFileHandle2 );
posix.wait( childProcessId2 );
posix.wait( childProcessId1 );
return dataFromProcess;
end
-- Command being executed
-- echo -n AAAAAAAAAAAAAAAA | openssl aes-128-cbc -e -nopad -a -K 30313233343536373839616263646566 -iv 1FF1ECB9000000000000000000000000
-- Expected result
-- 28iudIC31lHfDDxfa1/g9w==
result = openReadWritePipe("AAAAAAAAAAAAAAAA","openssl",{"aes-128-cbc", "-e", "-nopad", "-a", "-K", "30313233343536373839616263646566", "-iv", "1FF1ECB9000000000000000000000000"});
print("Result: "..result);
It's easy, no extensions necessary (tested with lua 5.3).
#!/usr/bin/lua
-- use always locals
local stdin = io.stdin:lines()
local stdout = io.write
for line in stdin do
stdout (line)
end
save as inout.lua and do chmod +x /tmp/inout.lua
20:30 $ foo=$(echo "bla"| /tmp/inout.lua)
20:30 $ echo $foo
bla