i have some shell scripts those printing some message to stdout, and i want to spawn some other process if output matches some regexp, i may also want to hang the shell for a while untill the other process give some feed back. my requirement basically behaves like pipe in shell and tcl expect, or pexpect in python, and my question is, does lua provide such feature?
i've considered a little lua's coroutine but it cannot yield during the middle of executing a shell script, so i did not dig.
Sadly, Lua doesn't provide piping support out of the box, so you'll have to choose between reading and writing. The closest you can get is by iterating through the :lines() of a io.popen()ed process:
for line in io.popen('/some/other/process'):lines() do
-- previous line will block until output is available
if line:match '^some regex' then
-- match found! do some stuff
end
end -- EOF reached
If you have access to C modules, luaposix provides an interface to pipe() throughposix.pipe()
However, bear in mind Lua may not be the most appropriate tool for the job. IMHO you'll be better off using TCL or Python, or even a bash script.
Related
As it is known, Lua 5.3 handles interactive REPL to differentiate expressions and statements this way:
In interactive mode, Lua repeatedly prompts and waits for a line. After reading a line, Lua first try to interpret the line as an expression. If it succeeds, it prints its value. Otherwise, it interprets the line as a statement. If you write an incomplete statement, the interpreter waits for its completion by issuing a different prompt.
However, this is not the behavior I want. For example, I have some codes "f()" to evaluate, where f will through error, no matter what happens. it also change the internal state of lua. The above approach will cause bugs because it will change the internal state twice.
So, I'd like to know, is there a way to implement a REPL that auto disambiguate expressions and statements? Do I have to add some syntax analysis to achieve this?
Interpreting code in Lua REPL is a two step process. First you convert the input to a runnable code with loadstring or a similar function. The code has been validated to be correct but didn’t run yet. Then you explicitly invoke it.
fn = loadstring(“return 42”);
fn()
To summarize, parsing and validating code with loadstring() or a similar function is side effect free, as long as you don’t invoke the result.
I wrote the following code in the file "orgin.lua"
if test==nil then
print(aa["bb"]["cc"]) -- to produce a crash
end
print(1120)
when it crash ,it will generate the following information:
lua: origin.lua:3: attempt to index global 'aa' (a nil value)
In order to prevent decompilation and make sure the code is safe,I use the following command to convert my code:
luac -o -s test.lua origin.lua
I know the argument -s is strip debug information, then it do not show the number of rows when crash:
lua: ?:0: attempt to index global 'aa' (a nil value)
but how to bring debugging information when encryption then lua code use luac?Is there any solution?
There is no way to do this built into Lua, but there are some work-arounds.
If you only need line numbers, then one option is to leave the line numbers in the chunk. Line numbers are not that useful for reverse engineering (unluac currently doesn't use them at all), so it shouldn't affect security. Lua doesn't provide an option for this, but it is easy to modify Lua to leave them in when stripping. From ldump.c
n = (D->strip) ? 0 : f->sizelineinfo;
can be changed to
n = f->sizelineinfo;
(Disclaimer: untested)
A more complicated option would be to modify the Lua runtime to output the virtual machine program counter instead of the line number, and also output information describing the location of the current function in the chunk (e.g. top level, first function, second function nested in third function, etc). Then the line number could be looked up by the developer in a non-stripped version of the chunk. (Here is a reference to someone using this approach on lua-l -- no source code was provided, though.)
Note that preventing decompilation is not true security. It may help against casual attacks, but Lua bytecode is not hard to read.
luac does not encrypt the output. It compiles your Lua source code to bytecode, that's all. The code is neither encrypted nor does it run any faster, only the loadtime is shorter since the compilation step is not needed.
If you want your code to be encrypted, I suggest to encrypt the bytecode using e.g. AES-256 and then decode it in memory just before handing it to the Lua state. This way the bytecode is encrypted on disk, but decripted in memory.
The overhead is low. We use this technique since years.
I have a log file that may be very large (10+ GB). I'd like to find the last occurrence of an expression. Is it possible to do this with standard posix commands?
Here are some potential answers, from similar questions, that aren't quite suitable.
Use tail -n <x> <file> | grep -m 1 <expression>: I don't know how far back the expression is, so I don't know what <x> would be. It could be several GB previous, so then you'd be tailing the entire file. I suppose you could loop and increment <x> until it's found, but then you'd be repeatedly reading the last part of the file.
Use tac <file> | grep -m 1 <expression>: tac reads the entire source file. It might be possible to chain something on to sigkill tac as soon as some output is found? Would that be efficient?
Use awk/sed: I'm fairly sure these both always start from the top of the file (although I may be wrong, my sed-fu is not strong).
"There'd be no speed up so why bother": I think that's incorrect, since file systems can seek to the end of a file without reading the whole thing. There'd be a little trial and error/buffering to find each new line, but that shouldn't slow things down much, compared to reading (e.g.) 10 GB that are never used.
Write a python/perl script to do it: this is my fall-back if no one can suggest anything better. I'd rather stick to something that can be done straight through the command line, since I'm executing it straight through ssh, and I'd rather not have to upload a script file as well. Using mmap's rfind() in python, I think we can do it in a few lines, provided the expression to find is static (which mine, unfortunately, is not). A regex requires a bit more work, something like this.
If it helps, the expression is anchored at the start of a line, eg: "^foo \d+$".
Whatever script you write will almost certainly be slower than:
tac file | grep -m 1 '^foo [0-9][0-9]*$'
This awk script will search through the whole file and print the last line matching the given /pattern/:
$ awk '/pattern/ { line=$0 } END { print $line }' gigantic.log
Using tac will be a better option (this uses GNU sed to output the first (i.e. last) found match with '/pattern/', after which it terminates, killing the pipeline):
$ tac gigantic.log | gsed -n '/pattern/{p;q}'
Using Perl or C or some other language, you could seek to the end of the file, step back 4kb (or something), and then
read forwards 4kb,
step back 8kb
repeat until pattern is found, making sure that handle reading partial lines correctly.
(This, apart from looking for a pattern, may actually be what tac does: one implementation of tac)
In bash, php/{composer,sismo} expands to php/composer php/sismo. Is there any way to do this with /bin/sh (which I believe is dash), the system shell ? I'm writing git hooks and would like to stay away from bash as long as I can.
You can use printf.
% printf 'str1%s\t' 'str2' 'str3' 'str4'
str1str2 str1str3 str1str4
There doesn't seem to be a way. You will have to use loops to generate these names, perhaps in a function. Or use variables to substitute common parts, maybe with "set -u" to prevent typos.
I see that you prefer dash for performance reasons, however you don't seem to provide any numbers to substantiate your decision. I'd suggest you measure actual performance difference and reevaluate. You might be falling for premature optimization, as well. Consider how much implementation and debugging time you'll save by using Bash vs. possible performance drop.
I really like the printf solution provided by #mikeserv, but I thought I'd provide an example using a loop.
The below would probably be most useful if you wish to execute one command for each expanded string, rather than provide both strings as args to the same command.
for X in composer sismo; do
echo "php/$X" # replace 'echo' with your command
done
You could, however, rewrite it as
ARGS="$(for X in composer sismo; do echo "php/$X"; done)"
echo $ARGS # replace 'echo' with your command
Note that $ARGS is unquoted in the above command, and be aware that this means that its content is wordsplitted (i.e. if any your original strings contain spaces, it will break).
This question already has answers here:
Confused about stdin, stdout and stderr?
(11 answers)
Closed 5 years ago.
I'm trying to understand stdin stdout and stderr.
I see them used in people's code all the time and I can't understand exactly what they are. I am assuming that they have something to do with input/output but have been searching for an explanation online and can't find one. Does anybody know of a good link with an explanation or if it is simple enough to explain it would be a great help to me.
Since I am learning Python 3, examples in that would be helpful.
sys.stdin
sys.stdout
sys.stderr
File objects used by the interpreter for standard input, output and errors:
stdin is used for all interactive input (including calls to input());
stdout is used for the output of print() and expression statements and for the prompts of input();
The interpreter’s own prompts and its error messages go to stderr.
For your more understanding:
>>> import sys
>>> for i in (sys.stdin, sys.stdout, sys.stderr):
... print i
...
<open file '<stdin>', mode 'r' at 0x103451150>
<open file '<stdout>', mode 'w' at 0x1034511e0>
<open file '<stderr>', mode 'w' at 0x103451270>
mode r means reading and mode w means writing
Does this explain it well enough?
sys.stdin
sys.stdout
sys.stderr
File objects corresponding to the interpreter’s standard input, output and error streams. stdin is used for all interpreter input except for scripts but including calls to input() and raw_input(). stdout is used for the output of print and expression statements and for the prompts of input() and raw_input(). The interpreter’s own prompts and (almost all of) its error messages go to stderr. stdout and stderr needn’t be built-in file objects: any object is acceptable as long as it has a write() method that takes a string argument. (Changing these objects doesn’t affect the standard I/O streams of processes executed by os.popen(), os.system() or the exec*() family of functions in the os module.)
To clairify
if I tell the python shell to do a print:
print 'what is your name?'
"what is your name" will go to stdout, whatever that is. If you haven't made any redirects that's by default the terminal you're using. You can interact with the standard streams in various different ways, for example:
sys.stdout.flush()
Tells the python shell to force any buffered information that I've printed to go to stdout right away.