How much memory, if any, is used when importing the sys module?
Does the CPython interpreter allocate memory resources when I import the sys module, or are its functions and data permanently in memory? Since the module is quite low level, and since (I believe) it is implemented in C, it seems possible that it is built into the interpreter -- and is therefore always in memory. But if not, approximately how much additional memory is taken up when importing it? And if it is built in, does that indeed mean that little extra memory is used when importing it?
Knowing the resources used by sys will help me decide whether or not to import it when I only need it for some non-critical purpose.
No additional memory is used when you import the sys module. As you guessed, the sys module is created as soon as you create the interpreter, you can see this happening during interpreter creation in the cpython source.
Related
In C language, we use #include statement and using statement in C# to be able to attach its build-in function. But in Lua, we dont need to do anything, then we can use the coroutine, table, io, etc.
It's because Lua "pre-imports" some of the libraries for you. You can re-configure your copy of Lua to load a different set of libraries. See lualib.h file in the Lua distribution for the list of pre-loaded libraries.
The Lua interpreter exports all basic functions and library tables to Lua programs by calling luaL_openlibs before running the program.
EDIT:
Why don't all languages do this?
It's a trade-off. If a language exposes its entire standard library by default, it saves us a lot of boilerplate. On the other hand, it can pollute the namespace, use more memory, and increase startup time. Lua's standard library is small, so it doesn't cost very much. Most compiled languages try to be as lean as possible, so they make us import everything explicitly.
DASK has a very powerful distributed api. As far as I can understand it can only support though native python code and modules.
Does anyone know if distributed DASK can support c++ workers?
I could not find anything in the docs.
Would there be any other approach apart from adding python bindings to cpp code to use that functionality?
You are correct, if you wanted to call into C++ code using Dask, you would do it by calling from python, which usually means writing some form of binding layer to make the calling convenient. If there is also a C API, you could use ctypes or cffi.
In theory, the scheduler is agnostic of the language of the client and workers, so long as they agree with each other, but no one has implemented a C++ client/worker. This has been done, at leats a POC, for Julia.
I'm trying to understand why adding traits to Dart would cause the shape of objects in memory to change, and am therefore curious how it loads in code right now.
Dart is a dynamically typed language that generates its own machine language equivalents straight from source code with no intermediate byte-code step. There is no generic bytecode (like the JVM or llvm) and instead it is directly compiled into machine code.
I would add that despite compiling straight to machine code, the language itself is not designed in a way that would allow a C/C++ style compiler to effectively generate fast efficient code. This is by design as Dart seems to be an attempt to fill the gap between JavaScript and Java rather than the gap between Java and C/C++. Dart addresses many issues that make JavaScript hard to optimize most importantly typing of numeric variables.
There are some efforts to port the Dart environment to various platforms beyond Windows/Mac/Linux but I have yet to see an actual straight to machine language compiler for Dart. That doesn't mean they don't exist, I just haven't seen anything other than ports of the Linux Dart environment onto Beagleboard and other small Linux distros.
From the Dart FAQ
Q. Why didn’t Google build a bytecode VM targetable by multiple
languages including Dart? Each approach has advantages and
disadvantages, but we feel that in the context of Dart it made sense
to build a language-specific VM for the following reasons:
Google already works on a multi-language bytecode: LLVM bitcode in
PNaCl.
Even if a bytecode VM is specialized for Dart, a language VM will be
simpler and faster because it can work under stronger assumptions—for
instance, a structured control flow. These assumptions make the
implementation cleaner and optimizations easier.
A general-purpose bytecode VM would be even larger and slower, as it
generalizes assumptions and adds functionality that for Dart is dead
code: for example, multithreading with a shared heap.
No bytecode VM is truly general-purpose; they all make assumptions
that privilege some class of languages. A language VM leaves more room
to improve the VM and make deep changes to optimization of the
language. Some Dart engineers wrote an article talking about the VM
question in more detail.
A pretty good presentation on Compiling Dart to Efficient
Machine Code
In some Redis loading tests, LuaJIT 2.0 (beta) is performing quite nicely, at about 60% of the runtime of a similar single threaded Python script.
When using Python's multiprocessing module to chunk large text files, results in a significant performance improvement splitting the works across cores.
I am assuming using the same approach in Lua would perform even better, but as a Lua beginner, I have not found the correct approach. Can anyone point me in the right direction?
There is LuaThreads, however, as explained here by Mike Pall, it isn't really the best solution for multi-threading and interlinked task (as all threads will hammer the single lock on the lua state).
However LuaLanes may provide what you need, but, seeing as you are using/favorable to LuaJIT and not just plain Lua, you can probably leverage the FFI to spawn system threads straight from LuaJIT and pass them a Lua callback (I'm not sure on the safety of this however, thats something for the LuaJIT mailing list).
How can I get the meomory of any Process in Lua?
Is it possible in Lua? Is C# it is possible but I am not sure.
In C# we can use PerformanceCounter class to get the
PerformanceCounter WorkingSetMemoryCounter = new PerformanceCounter("Process",
"Working Set", ReqProcess.ProcessName);
What is equivalent of this in Lua?
Suppose there are 5 process of IE (Internet Explorer) running . How to get a List of those process?
To elaborate on what others have said, Lua is a scripting language designed for embedding. That means that it runs inside another application.
Lua.exe is one such application. But it is not the only application that Lua scripts can be written to be executed on.
When within a Lua script, you have access to exactly and only what the surrounding application environment allows. If the application does not explicitly allow you to access things like "files" or "the operating system", then you don't get to access them. Period.
Lua's standard library (which an application can forbid scripts to use. Lua.exe allows it, but there are some embedded environments that do not) is very small. It doesn't offer a lot of amenities, which make Lua ideal for embedded environments: small standard libraries mean smaller executables. Which is why you see a lot more Lua in mobile applications than, say, Python. Also, the standard library is cross-platform, so it does not access platform-specific libraries.
Modules, user-written programs (either in Lua or C/C++) can be loaded into Lua.exe's environment. Such a module could give your Lua script access to things like "processes", how much memory the process is taking, and so forth. But if you do not have access to such a module, then you're not getting that info from within a Lua script.
The most you are going to be able to do is get the size of the memory that this particular Lua environment is directly allocating and using, as #lhf said: collectgarbage "count".
To get the memory allocated by Lua in Kbytes, use collectgarbage"count".
Lua does not comes with this built-in functionality. You could write a binding to a library that provides you that or you could interface with a program to do that (like reading the output of "ps -aux | grep firefox").