I am searching a lot about this great language and got myself on this page https://github.com/dart-lang/sdk/wiki/Snapshots but it didn't answer some questions:
What is the difference between a Snapshot and AOT? Which is faster and why?
Why the AOT generated on a Windows machine doesn't run on a Linux but a snapshot generated through dart --snapshot_kind=kernel does? What is "kernel"? Is it slower?
What is the difference from a Kernel Snapshot and a JIT Snapshot? Which is faster?
With snapshots, nothing of your code (kernel snapshots) or only portions of your code (JIT snapshots) are pre-compiled into binary format. With AOT, all of your code gets pre-compiled to binary (platform specific) format. Pre-compiled binary code (AOT) is faster, because there is no need to compile code at runtime.
As mentioned above, AOT compiled code is transformed to platform specific binary code. Therefor you can not run a program AOT compiled for Windows on a Linux machine. With kernel snapshots, nothing of your code is pre-compiled. So it is portable between platforms, but must be compiled at runtime. So yes, this is slower.
Also as mentioned above, with kernel snapshots nothing of your code gets pre-compiled. With JIT snapshots, the program is executed in a test run and each part of the code that is executed, gets pre-compiled to platform specific binary format. So JIT snapshots is faster then kernel snapshots.
Related
I'm reading about dart compile, and it has a few option: Executable, AOT snapshot, JIT snapshot and Kernel snapshot and JavaScript.
What is the difference between an executable and a snapshot? Is it purely the fact executables contain the Dart runtime/ VM, whereas a snapshot doesn't. Why is it called a snapshot?
2 highly related question (which I found after posting this question) are:
What is the difference between Dart's snapshots and Java bytecode?
What is the snapshot concept in dart?
This question is different to Dart: Snapshots vs AOT, since is asking the difference between a Snapshot and AOT, but actually AOT files are snapshots. It also primarily asked about the differences between Snapshot options (AOT, Kernel, JIT).
An executable (created by dart compile exe) is a combination of an AOT snapshot, and the Dart runtime. The Dart runtime is needed to run any Dart code, as it performs critical tasks like managing memory (including garbage collection) and performing runtime type checks.
The three kinds of snapshots (AOT, kernel, and JIT) all contain just the compiled source code. They all need a runtime to be run (typically you just use dart run <snapshot>).
The snapshots should perhaps have been named differently. Would 'module' have been easier to understand?
When Dart code is compiled, it is compiled into intermediary byte code rather than native code. This means that in order for the code to be executed in any arbitrary environment, the compiler needs to include the Dart virtual machine inside the executable along with the user code as well as any portions of the core library the program requires. Usually, this is fine because the Dart runtime is actually quite compact, but the obvious downside to this is that the executable will be larger and start-up time will be longer as the runtime needs to be extracted and warmed up before the user code can be run.
If, however, you are compiling code for an environment where you can guarantee that a Dart runtime will be present (such as a server machine or an IoT device), you can omit the runtime from the compiled program by building a snapshot file rather than an executable. This results in a smaller compiled file with a faster start-up time, although it requires a command to execute and as such is less convenient. You can learn more about snapshots and how to build and execute them on the Dart GitHib wiki page on snapshots.
There are three different kinds of snapshots: Kernel snapshots, which contain only AST information rather than compiled byte code making them usable by Dart runtimes on any supported architecture (portable but slow); JIT snapshots, which contain just the parts of the program necessary for startup and leaves the rest to be interpreted at runtime (fastest startup but slower execution); and AOT snapshots, which fully compiles the entire program into byte code (slower startup but fastest execution).
As for why it's called a "snapshot", I couldn't say. If I had to guess, that's because it's a "snapshot" of the program in its compiled state but without the instructions necessary to run it as a standalone executable.
(The above is based on my quick research on the subject and may be missing some key details. If a member of the Dart team happens upon this question, they will probably be able to offer a more detailed and technical explanation.)
My understanding is that Xamarin's Ahead-of-Time (AOT) Compiler compiles Xamarin.iOS applications directly to native ARM assembly code (How Xamarin works).
What I don't get, however, is why it needs to be called "Ahead-of-Time" as opposed to just being an ordinary compiler. Is there any distinction between Xamarin's AOT compiler and a traditional compiler or is this just a marketing term?
How AOT compares to a traditional JIT compiler
Ahead-of-Time (AOT) compilation is in contrast to Just-in-Time compilation (JIT).
In a nutshell, .NET compilers do not generate platform specific assembly code, they generate .NET bytecode, instructions which are interpreted by the .NET virtual machine. This bytecode is portable, any .NET VM can run it, be it Windows Phone, Mono on Linux, or a JavaScript-based implementation. Unfortunately, because the code has to be interpreted by the VM it is slower than native code which can be executed by the processor itself. That's where JIT and AOT come in.
When a .NET application starts up, the JIT compiler analyzes the bytecode, identifies areas that could be sped up by being translated to native code, and compiles them. During execution, the compiler can also identify hot paths for compilation.
Unfortunately for .NET, Java, and any platform that would benefit from JIT, dynamic code generation is disallowed by the App Store terms of service. Since Xamarin can't perform JIT on the device and they know they're shipping to ARM devices, they can run a JIT-type compiler ahead of time (AOT) and bundle it into the binary.
How AOT compares to a machine code compiler
As mentioned above, AOT translates part of an interpreted bytecode to machine code. It doesn't eliminate the need for a virtual machine bytecode interpreter. The VM will run just as it would if, but occasionally see an instruction that says "Execute this chunk of machine code".
Is this just a marketing term?
No. The message that Xamarin was conveying in that paragraph was that their code performs faster than a simple byte code based language. For both iOS and Android, they are able to execute native code on hot code paths to improve performance. The terms AOT and JIT are technical details about how they do that.
How can I build and compile my own Lua files on Windows? And make them executable.
I am reading Beginning Lua programming, and I have Windows 7 and MacOS Lion both installed. I am having the hard time to follow the instructions. They do not work for me.
On MacOS I open the terminal and put these in:
export LUA_DIR=/usr/local/lib/lua/5.1
mkdir -p /usr/local/lib/lua/5.1 (it tells me, mkdir: illegal option) and I can not follow from here
SET LUA_DIR=”c:\program files\lua\5.1”
As for Windows I do this according to the book.
This what I see in my shell c:\Users\bd>
mkdir "c:\program files\utility" and it tells me access is denied
I have tried to right click on this folder and check off read only, but it does not work.
Any clues would be appreciated, this part has been really confusing for me.
To package your Lua files into an executable on Windows you have several options. There is srlua, there is wxLuaFreeze from wxLua (available as a binary for Windows), and there are more options in this SO answer.
Essentially, the main two options are: (1) append your Lua code to a precompiled exe file, such that it will be loaded and executed when that exe file is run, and (2) convert your Lua code into real executable by compiling it to bytecode, then to C, and then to your target platform.
As to your MacOS issue, mkdir -p means that mkdir is asked to create intermediate directories (for example, you asked to create /a/b/c, it will also create /a/b if those don't exist). As you don't say which version of MacOS you run, it's difficult to provide more detailed answer.
For now the standard distribution of Lua does not compile a script to native executable code; it execute your scripts by first compiling it to bytecode, then by interpreting the bytecode with a reasonnably fast static interpret (this also means that it is easily portable across native or virtual systems, and very resistant to attacks (that could be targetting bugs in the native compiler itself).
Also Lua still does not feature a runtime JIT compiler like Java and .Net: Lua still does not features a VM to produce a safe sandbox.
There exists Lua packages that convert your bytecode (or directly a source script) to a C source that can be used to convert a Lua library into native mode via the same C compiler used to compile the Lua engine itself (this is how the builtin libraries are produced, though they are slightly optimized manually in some time-critical parts).
However it is possible to compile Lua to a javascript source, and run it with fast performance using Javascript, because today's Javascript interprets do have good performance with their implemented VM featuring a JIT compiler for their own bytecodes.
It is also possible by converting it the Lua bytecode to a .Net or Java source that can then be executed directly from Lua (for that you need a version of Lua that has been ported to .Net or Java or Javascript, something that is not so complicate than developing in C/C++ directly a VM with a JIT compiler (a moderately complex part is the bytecode verifier, but the really complex part is the memory manager its garbage collector and its sandbox so that your Lua script will be fully isolated from the Lua engine itself for itw own memory, but the most complex part if the runtime optimizer and collection of profiling statistics: this has been done in the modern VMs for Java, .Net, Javascript, PHP/Zend, Python, Perl...).
I dont know which other language VM would offer the best performance to port Lua and implement on it a compiler to their own bytecode running at near native speed in their VM. But my own small experience with programs (in a much simpler language) self-generating a bytecode that they can run themselves, has always shown me Java winning in performance over .Net and Javascript. This is most probably because Java features an profiling-based dynamic code optimizer
(On the opposite the .Net optimizer runs only once during program installation, using some profiling data collected during the installation of the .Net VM itself, or at first instanciation of the script, without really knowing any profiling data collected during execution of the compiled program itself, and based on some cheked assumptions about the platform capabilities).
I also don't if would be faster in PHP, Python or Perl; the comparison with newer Javascript engines was never attempted though. Porting/compiling a Lua program to Javascript is relatively easy because it implements closures relatively easy for the resolution of linkages. Then the generated Javascript will compile to native code with the excellent Javascript's JIT compilers we have today (and never cease to improve in performance, so much that I've seen various appliactions running now faster in Javascript than before when they were written in C++ or plain C; as well the memory footprint has largely been reduced, we no longer have memory leaks, and even if there's a garbage collector, today's Javascript VM have a very efficient one, which is even better than the GC implemented in the native Lua).
But Lua remains useful as it is easy to secure and sandbox and offers various security benefits (but there are security issues in Lua as well for some kinds of applications, where Javascript offers some solutions, notably for side-channel attacks based on variation of time of execution; but these side-channel attacks are very hard to solve and can affect any system, any program, any programming language, and this starts becoming a critical issue because they are now more esily exploitable; the reason of that comes from hardware optimizations that we depend more and more today when we want to maximize the performances). And with Lua you may be more immune to these problems that a sandboxing sofware environment cannot solve alone.
Probably later we'll see a true VM implementation of Lua with a JIT and self-generating code and the possibility to instanciate new sandboxed VMs to run their self-generated code. It will take more time to generate an EXE file for distribution; notably because it generally requires adding also an installer and a distribution manager.
So for now we could imagine distributing Lua applications compiled to the bytecode of another JIT-capable VM: this generated bytecode would be faster than the Lua bytecode, and would then be extremely complex to reverse-engineer to the semantics of Lua because it would require two separate reverse engineering first from the bytecode of the other VM to the bytecode of Lua, both bytecodes loosing some easiy inferable rules and options tested and foll, and then again to sme Lua source
For the OSX terminal issue:
This command should work
export LUA_DIR=/usr/local/lib/lua/5.1
This command will probably give you permission problems:
mkdir -p /usr/local/lib/lua/5.1
You may try this to solve that. You will be prompted for your password:
sudo mkdir -p /usr/local/lib/lua/5.1
This command has nothing to do with OSX and will not work. This is a windows command:
SET LUA_DIR=”c:\program files\lua\5.1”
You have a permissions problem with Windows- try creating your cmd or PowerShell in Administrator mode. C:\Program Files is a protected directory that a regular user account doesn't have permission to write to.
As for the OS X issue, check out the mkdir OS X manual page to make sure you have the command correct.
So, if I understood your question correctly, you are trying to build Lua on Windows.
This is of course possible, but not easy for beginners. I would highly recommend you to use a binary distribution, which is much easier to install, unless you have special requirements.
Here are several Windows distributions :
Lua Binaries (Lua 5.1 and 5.2)
LuaForWindows (Lua 5.1)
LuaDist (Lua 5.2)
If I compile a regular .lua file with luac, can the result be ran without the Lua library or interpreter installed?
No. You can run it on a version of Lua that was built without the compiler, but you still need the Lua interpreter to execute the code.
Incidentally, the compiled Lua bytecode is also machine-specific; i.e. you can't compile on one architecture and then run that output on another architecture unless you understand the subtleties (endianness, sizes of types, etc.).
If your code doesn't use any dynamic load-based facility (that's loadstring, loadfile, require, etc.) you can strip Lua library to just a VM, because what compiler emits is code to be run on this virtual machine. This can easily cut Lua already small footprint to 1/3 fraction of original.
However, since this is NOT a native binary code for any currently existing architecture, you still CAN'T run it directly without assistance of VM.
I have an application which i build using gcc on linux host for ARM target processor. This generated arm executable i execute on a ARM development board i have.
I want to do some code coverage analysis:
Will gcov show code coverage if i have ARM assembly source files in my build environment?
If my build environment has some X86 assembly source files, then will gcov show code coverage data?
Thank you.
-AD.
AFAIK, gcov works by preprocessing your C or C++ source code.
If you have pure assembly language files, I don't think gcov ever
sees them.
If it does, I'd be suprised if it understand how
to safely insert code in arbitrary-target assembly code,
with ARM being common enough so there's a faint chance.
The problem with instrumentating assembly code is the
test coverage probe code itself may require registers,
and there isn't a safe way to know, for an arbitrary piece
of assemblers, a) what registers are available, and b)
if there's an inserted instruction, will some other instruction
break because of the extra space (e.g., a hardwired jump relative
across the inserted instruction).