I want to load a file from a .fsx script into into the F# Interactive Session but I can't use #load since I only want to load it if a certain condition is true.
Is there a function like FSI.LoadFile or something instead of the compiler directive?
Looking at the source code (fsi.fs, line 1710 here
| IHash (ParsedHashDirective("load",sourceFiles,m),_) ->
fsiDynamicCompiler.EvalSourceFiles (istate, m, sourceFiles, lexResourceManager),Completed
Now, some of these parameters are probably easy to fake - in particular sourceFiles and m. I suspect that the other parameters are harder to fake.
I suspect that you are not trying to solve the problem in a good way. An alternative solution may be to do something like
let conditionally_load fname cond =
if cond then System.IO.File.Copy(fname,"dummy.fs")
else System.IO.File.Create("dummy.fs")
conditionally_load "something.fs" true
#load "dummy.fs"
Although you might need a ;; before the #load to ensure that the function runs before the #load
What you are looking to do is tricky, since you want the decision on whether to load additional code to be taken at evaluation time. A solution to this would be to modify the shell so that script loading can be scheduled at the next iteration of the REPL. In essence, your FSI.LoadFile function would simply add the path to a global queue, in the form of a ParsedHashDirective("load",["foo.fsx"], ...) expression.
The queue could then be prepended to the actions list next time the ExecInteractions method is called in line 1840. This should work fine, but clearly you won't be getting any intellisense support.
Related
In python, there is a common construction of the form if __name__ == "__main__": to detect whether the file was imported or executed directly. Usually, the only action taken in this conditional is to execute some "sensible, top level" function. This allows the same file to be used as a basic script and as a library module (and also as something an interactive user can import and use).
I was wondering if there is a clean and reliable way to do this in lua. I thought I could use the _REQUIREDNAME global variable, but it turns out that this was changed in Lua 5.1. Currently, the lua require passes arguments (in the variadic ...), so in principle, these can be examined. However, this is either not reliable, not clean, or probably both, because obviously when a script is executed arguments can be passed. So to do this safely, you would have to examine the arguments.
FWIW, require passes the module name as argument 1 (the string you called require on), and the path to the file it eventually found as argument 2. So there is a obviously some examination that can be done to try to detect this, which if not nearly as nice as if __name__ == "__main__": and can always be bypassed by a user by passing two suitably constructed arguments to the script. Not exactly a security threat, but I would hope there is a better solution.
I also experimented with another method, which I found very ugly but promising. This was to use debug.traceack(). If the script is executed directly, the traceback is very predictable, in fact, it only has 3 lines. I thought this might be it, although, like I said, an ugly hack for sure.
Do any more frequent lua users have advice? In effect, if I am writing module X, I want to either return X.main_func() in script mode or return X in import mode.
EDIT:
I took out an item which was actually incorrect (and makes my traceback solution workable). Additionally, the link provided in the comment by Egor Skriptunoff did provide another trick from the debug library which is even cleaner than using the traceback. Other than that, it seems that everyone ran into the same issues as me and the lua team has been disinterested in providing an official means to support this.
Based on the links provided by Egor, the current cleanest and safest way to do this seems to be as outlined here:
How to determine whether my code is running in a lua module?
which I repeat for ease of reference:
if pcall(debug.getlocal, 4, 1) then
print("in package")
else
print("in main script")
end
There is a whole thread about it here:
http://lua.2524044.n2.nabble.com/Modules-with-standalone-main-program-td7681497.html
Like I said, it seems this is a popular feature which is going to remain unsupported for the time being, but the debug.getlocal method seems to be what common lua developers have settled on for now.
A require() returning in package.loaded.
So simply check package.loaded for what you want.
Also as a good start for more experience in that methodic i suggest to write package.preload functions for require() stuff without manipulating the path.
A simple example for showing this...
# /usr/local/bin/lua -i
Lua 5.4.3 Copyright (C) 1994-2021 Lua.org, PUC-Rio
> package.preload.mymod=function() return {dump=function(...)
>> local args={...}
>> local test,dump=pcall(assert,args[1])
>> if test then
>> for key,value in pairs(dump) do
>> io.write(string.format("%s=%s\n",key,value)):flush()
>> end
>> return true
>> else
>> return test,dump
>> end
>> end} end
> mymod=require('mymod')
> mymod.dump(package.loaded)
string=table: 0x56693590
mymod=table: 0x566aa890
utf8=table: 0x56694d80
package=table: 0x56691ed0
math=table: 0x56693cb0
table=table: 0x566920d0
_G=table: 0x56690730
debug=table: 0x566950e0
coroutine=table: 0x56692310
os=table: 0x56692e60
io=table: 0x566921b0
true
I'm reading some source codes of a project, which is a combination of c++ and lua, they are interwined through luabind.
There is a la.lua file, in which there is a function exec(arg). The lua file also uses functions/variables from other lua file, so it has statements as below in the beginning
module(..., package.seeall);
print("Loading "..debug.getinfo(1).source.."...")
require "client_config"
now I want to run la.exec() from interactive terminal(on linux), but I get errors like
attempt to index global 'lg' (a nil value)
if I want to import la.lua, I get
require "la"
Loading #./la.lua...
./la.lua:68: attempt to index global 'ld' (a nil value)
stack traceback:
./lg.lua:68: in main chunk
[C]: in function 'require'
stdin:1: in main chunk
[C]: ?
what can I do?
Well, what could be going wrong?
(Really general guesswork following, there's not much information in what you provided…)
One option is that you're missing dependencies because the files don't properly require all the things they depend on. (If A depends on & requires B and then C, and C depends on B but doesn't require it because it's implicitly loaded by A, directly loading C will fail.) So if you throw some hours at tracking down & fixing dependencies, things might suddenly work.
(However, depending on how the modules are written this may be impossible without a lot of restructuring. As an example, unless you set package.loaded["foo"] to foo's module table in foo before loading submdules, those submodules cannot require"foo". (Luckily, module does that, in newer code without module that's often forgotten – and then you'll get an endless loop (until the stack overflows) of foo loading other modules which load foo which loads other modules which …) Further, while "fixing" things so they load in the interpreter you might accidentally break the load order used by the program/library under normal operation which you won't notice until you try to run that one normally again. So it may simply cost too much time to fix dependencies. You might still be able to track down enough to construct a long lua -lfoo-lbar… one-off dependency list which might get things to run, but don't depend on it.)
Another option is that there are missing parts provided by C(++) modules. If these are written in the style of a Lua library (i.e. they have luaopen_FOO), they might load in the interpreter. (IIRC that's unlikely for C++ because it expects the main program to be C++-aware but lua is (usually? always?) plain C.) It's also possible that these modules don't work that way and need to be loaded in some other way. Yet another possibility might be that the main program pre-defines things in the Lua state(s) that it creates, which means that there is no module that you could load to get those things.
While there are some more variations on the above, these should be all of the general categories. If you suspect that your problem is the first one (merely missing dependency information), maybe throw some more time at this as you have a pretty good chance of getting it to work. If you suspect it's one of the latter two, there's a very high chance that you won't get it to work (at least not directly).
You might be able to side-step that problem by patching the program to open up a REPL and then do whatever it is you want to do from there. (The simplest way to do that is to call debug.debug(). It's really limited (no multiline, no implicit return, crappy error information), but if you need/want something better, something that behaves very much like the normal Lua REPL can be written in ~30 lines or so of Lua.)
What is the recommended way to load+reload fsx files? Just experimenting... yes yes right language right job ect ect..
I love how the following can be done in FSI:
#load "script.fsx";
open Script
> let p = script.x 1
Error: This expression was expected to have type string but here has int...
(* edit script.fsx x to make it int -> int *)
>
> #load "script.fsx"
> let p = script.x 1
val it : int = 2
But how do we do this for an application that we are running via fsi blah.fsx? Maybe something that is sitting in a while loop. It seems #load and #use must not be inside let or module.. i.e. you cannot use #load like let reload script = #load script, wonder why?
My original method was to have .fs files and recompile + relaunch each time I wanted to add/fix something. This method feels primitive.
Second method was to attempt to use the #load directive inside of a module, which turns out to not work (kind of makes sense in terms of scoping)...
module test1 =
#load #"C:\users\pc\Desktop\test.fsx"
open Test
module test2 =
...
Another way would be to create a new process for every module by loading fsi module.fsx with process diagnostics, but this seems horrible, inefficient and ugh.
I have a feeling deep in my heart that this will not be trivial inside .NET, but I would like to pose the question anyway, FSI does it... I wonder if I can leverage the FSI API or something (or at the least to copy their code)?
TL;DR I read the following about erlang and want it for myself in F#.
Erlang: Is there a way to reload changed modules into an already running node with rebar?
"...any time a module in your program changes on disk, the reloader will replace the running copy automatically."
I don't know if this would work in FS but in ML you can load a master file that loads all your files in your project and then executes any code that you need to use to knit them together and runs your application. To see an example of a massive app run from inside of a REPL look at the Isabelle/HOL site at the Cambridge laboratory of Computational Science http://www.cl.cam.ac.uk/research/hvg/Isabelle/installation.html. After downloading the app look in the src code directory for any file called root.ml. There will be half a dozen of them that control various levels of implementation. This is recursive because a top level file can call a file in several sub-directories that loads that particular sub-feature. This allows targeting your application to various scenarios depending on which top level file is executed.
Typical .NET Framework applications cannot unload/reload assemblies unless they are in an App Domains that are separate from the primary one that starts up with the application. This is essentially how most plugin systems are designed for applications that run on the full .NET Framework. Things may be changing post .NET Standard 2.0 in .NET Core with the Collectible Assemblies feature.
References:
https://github.com/dotnet/coreclr/issues/552
https://github.com/dotnet/corefx/issues/19773
I've read on the Lua wiki / here / etc. on how to sandbox lua code generally. But I haven't been able to find something that disallows function creation. For example, the example here provides a sample code as:
assert(run [[function f(x) return x^2 end; t={2}; t[1]=f(t[1])]])
And that's with an empty environment. But I want to eliminate the ability to create a function (the 1st part of the code) - e.g., just allow expressions. Any idea on how to do that? Does it have to be in C somehow? Thanks in advance!
If you want to evaluate expressions only, you could try this:
function run(s) return loadstring("return "..s)() end
(error handling omitted)
This simple solution will prevent most `attacks', but not eliminate them because one can say
(function () f=function(x) print"hello" end end)()
which defines a new function named f.
Your best bet is to use a sandbox and not worry about what the user does to the environment, because it'll not be your environment.
You can try detecting the creation of functions by looking for the string "function" before allowing the execution of the lua script. For example from your C/C++ backend.
If "function" appears throw a "you are not allowed to create functions" error and don't execute the code.
A couple notes:
You might want to try to customize the detection a bit more - only throw errors if you detect function followed by blanks and an opening parenthesis, for example. I'm leaving that as an exercise.
You should be aware that there are some standard lua functions that kindof expect the users to be able to create functions - for example, the string table has several of those. Without creating functions, it'll be very difficult for your users to work with strings (it is already difficult enough with functions...)
Is it possible to check if a lua script contains errors without executing it? I have fallowing code:
if(luaL_loadbuffer(L, data, size, name))
{
fprintf (stderr, "%s", lua_tostring (L, -1));
lua_pop (L, 1);
}
if(lua_pcall(L, 0, 0, 0))
{
fprintf (stderr, "%s", lua_tostring (L, -1));
lua_pop (L, 1);
}
But if the script contains errors it passes first if and it is executed. I want to know if it contains errors when I load it, not when I execute it. Is this possible?
You can use the LUA Compiler. It will only compile your file to bytecode without executing it.
Your program will also have the advantage the run faster if it is compiled.
You can even use the -p option to only perform a syntax checking, according to the linked man page :
-p load files but do not generate any output file. Used mainly for syntax checking or testing precompiled chunks: corrupted files will probably generate errors when loaded. For a thourough integrity test, use -t.
(This was originally meant as a reply to the first comment to Krtek's question, but I ran out of space there and to be honest it works as an answer just fine.)
Functions are essentially values, and thus a named function is actually a variable of that name. Variables, by their very definition, can change as a script is executed. Hell, someone might accidentally redefine one of those functions. Is that bad? To sum my thoughts up: depending on the script, parameters passed and/or actual implementations of those pre-defined functions you speak of (one might unset itself or others, for example), it is not possible to guarantee things work unless you are willing to narrow down some of your demands. Lua is too dynamic for what you are looking for. :)
If you want a flawless test: create a dummy environment with all bells and whistles in place, and see if it crashes anywhere along the way (loading, executing, etc). This is basically a sort of unit test, and as such would be pretty heavy.
If you want a basic check to see if a script has a valid syntax: Krtek gave an answer for that already. I am quite sure (but not 100%) that the lua equivalent is to loadfile or loadstring, and the respective C equivalent is to try and lua_load() the code, each of which convert readable script to bytecode which you would already need to do before you could actually execute the code in your normal all-is-well usecase. (And if that contained function definitions, those would need to be executed later on for the code inside those to execute.)
However, these are the extent of your options with regards to pre-empting errors before they actually happen. Lua is a very dynamic language, and what is a great strength also makes for a weakness when you want to prove correctness. There are simply too many variables involved for a perfect solution.
In general it is not possible, as Lua is a dynamic language, and most of errors happen in runtime.
If you want to check for syntax errors, use luac -p option. I use it as a part of my pre-commit hook, for example.
Other common errors are triggering by misusing the global variables. You may analyze output of luac -l to catch these cases. See here: http://lua-users.org/wiki/DetectingUndefinedVariables.
If you want something more advanced, there are several more-or-less functional static analysis tools for Lua code. Start with LuaInspect.
In any case, you are advised to write unit tests instead of just relying on static code checks. Less pain, more gain.