No Erlang compile time errors for missing functions - erlang

Why is there no compile time errors or warnings when I call a function in another module that doesn't exist or has the wrong arity?
The compiler has all of the exports information in a module to make this possible. Is it just not implemented yet or is there a technical reason why it is not possible that I am not seeing?

I don't know why it's missing (probably because modules are completely separate and compilation of one doesn't depend on the other really - but that's just speculation). But I believe you can find problems like this with dialyzer static analysis. Have a look at http://www.erlang.org/doc/man/dialyzer.html
It's part of the system itself, so try including it in your workflow.

It is as others have said. Modules are compiled separately and there is absolutely no guarantee that the environment which exists at compile-time is the same as the one that will exit at run-time. This implies that doing checks at compile-time about the existence of a module, or of a function in it, is basically meaningless. At run-time that module may or may not be loaded, the function you call may or may not be defined in the module, or it may do something completely different from what you expected.
All this is due to the very dynamic nature of Erlang systems. There is no real way as such to define what is in system at run-time. Hot code-loading is a part of this and works properly because of the dynamic nature of the system. It means you can redefine the system at run-time, you can load in new versions of existing modules with a different interface and you can load in completely new modules and remove existing modules.
For this to work all checks about the existence of a module or function must be done at run-time.
Tools like dialyzer can help with this but they do assume that you don't do anything "funny" at run-time and the system you check is the same as the system you run. Which is of course all good, but very static. And against Erlang's nature which is to be dynamic, in everything.
Unfortunately, in this case, you can't both have your cake and eat it.

You may use the xref application to check the usage of deprecated, undefined and unused functions (and more!).
Compile the module with debug_info:
Eshell V6.2 (abort with ^G)
1> c(test, debug_info).
{ok,test}
Check the module with xref:m/1:
2> xref:m(test).
[{deprecated,[]},
{undefined,[{{test,start,0},{erlang,foo,0}}]},
{unused,[]}]
You may want to check out more about xref here:
Erlang -- Xref - The Cross Reference Tool (Tools User's Guide)
Erlang -- xref (Tools Reference Manual)

It is due hot code loading. Each module can be loaded in any particular time. So when you have in your module A code which calls function B:F then you can't tell it is wrong in compile time when your source code of module B has no function B:F. Imagine this: You compile module A with call to B:F. You load module B into memory without function B:F. Then you load module A which contain call B:F but don't call it. Then compile new version of module B with B:F. Then load this new module and then you can call B:F and everything is perfectly right. Imagine your module A makes module B on fly and load it. You can't tell in any particular time that it is wrong that module A contain call to nonexistent function B:F.

In my opinion most, if not all, compiler does not verify that a function exists at compilation. What it is required in general is a prototype declaration of the function: the type of the return value, the list and type of all arguments. This is done in C/C++ by including some_file.h in each module definition (not the .c or .cpp).
In Erlang this type verification is done dynamically, while the program is running, so it is not necessary to include these definitions. It is even totally useless because Erlang allows to upgrade the application in run, so the function type may change, or the function may disappear, on purpose or by mistake, during application life time; it is why the Erlang designer have chosen to make this verification at run time and not at build time.
The error you speak about generally occurs during the link phase of the code generation, when the "compiler" tries to gather all together some individual pieces of object code to build an executable file or a library, during this phase the linker solves all the external addresses (for shared variable, static call...). This phase does not exist in Erlang, a module is totally self contained; it does no share anything with the rest of the application, no variable nor function address.
Of course, it is mandatory to use some tools and make some test before updating a running production program, but I consider that these verifications have exactly the same level of importance than the correctness of the algorithm itself.

When you compile e.g. module alpha which has a call to beta:some_function(...), the compiler cannot assume some specific version of beta to be in use at runtime. Maybe you will compile a newer version of beta after you compiled alpha and this will have the correct some_function exported. Maybe you will upload alpha to be used on a different host, which has all the other modules.
The compiler therefore just compiles the remote call and any errors (non-existent module or function) are resolved at run time, when some version of beta will be loaded.

Related

How to use PathCchCanonicalizeEx with C++Builder 10.2?

I have a legacy Windows project using the legacy 32 Bit C++ compiler. For various reasons I need to use the Windows 8+ function PathCchCanonicalizeEx. C++Builder seems to provide the header and some module definition file for that, but I can't find any library to link against:
[ilink32 Error] Error: Unresolved external 'PathCchCanonicalizeEx' referenced from C:\[...]\WIN32\DEBUG\TMP\FILE.OBJ
How am I supposed to fix this? Do I need to add a Windows 8.1 SDK? Is the necessary lib simply named differently and I can't find it? Something completely different?
According my tests, one has two options:
IMPLIB/MKEXP
I'm developing/testing a some Windows 10 21H2, which provides an implementation for PathCchCanonicalizeEx in some DLL already. So if that source DLL is known, one can use IMPLIB or MKEXP to create an import library manually. I did that and after adding the created library from IMPLIB to my project, the linker errors were instantly gone.
Though, it's not that easy to know where PathCchCanonicalizeEx is placed in. One pretty easily finds the api-ms-win-core-path-l1-1-0.dll, but that thing is NOT an actual file on the disk and therefore can't be used by IMPLIB or MKEXP. That name is only a virtual concept for the library loader to address the same named API set of modern Windows, the extension .dll doesn't mean it's a file at all.
You can use an API set name in the context of a loader operation such as LoadLibrary or P/Invoke instead of a DLL module name to ensure a correct route to the implementation no matter where the API is actually implemented on the current device. However, when you do this you must append the string .dll at the end of the contract name. This is a requirement of the loader to function properly, and is not considered actually a part of the contract name. Although contract names appear similar to DLL names in this context, they are fundamentally different from DLL module names and do not directly refer to a file on disk.
https://learn.microsoft.com/en-us/windows/win32/apiindex/windows-apisets#api-set-contract-names
What you really need to work with is KernelBase.dll, which is even documented by MS.
implib "KernelBase x86.lib" C:\Windows\SysWOW64\KernelBase.dll
implib "KernelBase x86-64.lib" C:\Windows\System32\KernelBase.dll
Module Definition File
The downside of manually creating LIB files is that one needs to maintain those with the project. Things depend on if the target is 32 or 64 Bit, DEBUG or RELEASE, so paths might become a bit complex, one might need to create relative paths for libraries in the project settings using placeholders for the target and stuff like that.
It seems that all of this can be avoided with Module Definition Files, which's purpose is to provide IMPORT and EXPORT statements to either consume exported functions by other DLLs or make that possible for others with own functions. I've successfully resolved my linker problems by simply creating a file named like my app using the extension .def alongside my other project files. That file needs to be added to the project, though.
dbxml.cbproj
dbxml.cbproj.local
dbxml.cpp
dbxml.def
dbxml.res
[...]
The following content made the app use the correct function from the correct DLL. Though, what didn't work was using the API set name, which resulted in an error message by the linker.
IMPORTS
KernelBase.PathCchCanonicalizeEx
IMPORTS
api-ms-win-core-path-l1-1-0.PathCchCanonicalizeEx
[ilink32 Error] Invalid command line switch for "ilink32". Parameter "ItemSpec" cannot be null.
[ilink32 Error] Fatal: Error processing .DEF file
The latter is after restarting C++Builder, so I guess the format of the file is simply wrong because of the API set name.

Rewriting symbols in static iOS libraries

I am working on an iOS app which links several static libraries. The challenge is, those linked libraries define same method names with different implementations. Oddly, I don't get any duplicate symbol definition errors; but, to no surprise, I end up with access to only one implementation of the method.
To be more clear, say I have libA and libB and they both define a global C method called func1()
When I link both libA and libB, and make a call to func1(), it resolves to either libA's or libB's implementation without any compilation warning. I, however, need to be able to access both libA's func1() and libB's func1() separately.
There's a similar SO post that explains how it can be done in C (via symbol renaming) but unfortunately, as I found out, objcopy tool doesn't work for ARM architecture (hence iPhone).
(I will submit it to the App Store, hence, dynamic linking is not an option)
It appears that you are in luck - you can still rename symbols with the ARM binary format, it's just a bit more hacky than the objcopy method...
NOTE: This has only been tested minimally, and I would strongly advise you to make a backup of all libraries in question before trying this!
Also note that this only works for files not compiled with the C++ compiler! This will fail if the C++ compiler was used on these files.
First, you will need a decent hex editor, for this example, I will be using Hex Fiend.
Next, you will open up a copy of your of of your libraries, let's call it lib1-renamed.a, and do the following with it:
Find the name of the symbol you wish to re-name. It can be found using the nm tool, or, if you know the header name, you should be set.
Next, you will use hex fiend, and to a textual replace of the old name (in this case foo), and give it a new name (in this case, bar). These names must have the same length, or it will corrupt the binary's offsets!
Note: if there is more than one function that contain's foo's name in it, you may have problems.
Now, you must edit the headers of the library you changed, to use the new function name (bar) instead of the old one.
If you have done the three simple† steps above properly, you should now be able to compile & link the two files successfully, and call both implementations.
If you are trying to do this with a universal binary (e.g. one the works on the simulator as well), you'd be best off using lipo to separate the two binaries, using objcopy on the i386/x64 binary, and then using my method on the ARM binary, and lipo it back together.
†: Simplicity is not guaranteed, nor is it covered by the Richard J. Ross III super warranty. For more information about the super warranty, call 1-800-FREE-WARRANTY now. That's 1-800-FREE-WARRANTY now!

How to resolve "Main module of program is empty: nothing will happen when it is run"

I have two projects in an F# solution.
1. main project with [EntryPoint] and set as the StarUp project.
2. support, the second project, holds a group of support modules. I.e. they are only called and never initiate anything nor serve as the entry point nor are the StartUp project.
For the last module in the support project, compiling in Visual Studio gives
warning FS0988: Main module of program is empty; nothing will happen
when it is run
While using compiler option nowarn inline as #nowarn "988" in the module causing the warning does suppress the message I would rather add something like a dummy function with comments that resolves the issue.
How does one make such a dummy function to resolve the warning?
EDIT
Jack is correct in that my support project was setup as a Console Application instead of a Class Library. Changing to Class Library resolved the warning. It is also nice to know about do () for the other case.
EDIT
While it seemed odd that I would have set a support project as a Console Application, I recently found that for some reason when I made a change to the code in the project, something changed the Output type from Class Library to Console Application. I suspect it has to do with the F# PowerPack and it's build rules, but it's only a guess.
Are you building the support project as a Library or as a Console Application? (This is set via the project properties page.)
If you're building it as a library, then you may need to add a do() at the end of the last file in the project. This is necessary to make the F# compiler happy in a few specific scenarios, like when you create a module which contains only assembly-level attributes (because they're applied to the assembly, the module appears "empty" to the compiler).
You can see an example in my code here:
https://github.com/jack-pappas/FSharp.Compatibility/blob/master/FSharp.Compatibility.OCaml/AssemblyInfo.fs

Bundling additional Lua libraries for embedded and statically linked Lua runtime

I have embedded Lua on Win32 in my project by means of statically linking it in (no, I can't switch to DLL). I would like to bundle more Lua extensions that use native code - not just pure .lua files. Specifically, I want to bundle Steve Donovan's winapi which comes as some lua files and some .c files.
How to do it?
You need to do two things. First, you have to compile the Lua DLL projects into non-DLL projects. Since they're intended to be DLL modules, they probably won't have provisions for this in their build systems. That means you have to do it yourself. Get rid of the DLL main functions and other specialized DLL functions (but take note of what they do and make sure you replicate it if it's important). And make sure that you change any #defines that try to include Lua with dynamic linking.
All Lua module DLLs export one or more functions of the form luaopen_*, where * is the name of the module to load. This function will likely be decorated with declspec() notation. Typically, the notation is done via a preprocessor macro, but it may not be. Either way, remove it, turning it into a normal function declaration.
Now, once you have created your lua_State object, just call that luaopen_* function with your lua_State.

Native code execution by JVM/CLR

How does JVM/CLR execute JIT compiled native code? Is it by some code injection or by copying code to executable memory? What are the system calls that allows dynamic code execution?
I can explain how we do it in CACAO VM (a research JIT-only JVM). First, the machine code for a method is generated into some heap-allocated memory block. After compilation, the final code length is known, and a chunk of executable memory is allocated using mmap and the PROT_EXEC flag (relevant CACAO code here). Then, the machine code is copied into the mmapped area. After that, many architectures require some machine-specific cache flushing mechanism. As an example, have a look at the cache-flushing function for PowerPC 64. Notably, on i386 and x86_64, there is nothing to do. After this step, the processor is ready to execute the newly-generated code. Alternatively, already allocated memory pages can be marked executable with mprotect. Note that mmap/mprotect are Unix facilities.
I don't know specifically how Java does it, but in general you'd insert "trap" opcodes into the interpreter's instruction stream. There are two opcodes in the JVM spec that seem tailor-made for this purpose.
If you want to know for sure, there's no better answer than the source: http://download.java.net/jdk6/source/
The Common Language Runtime has a methodtable for each type with entries pointing to native code or a native stub to JIT managed code and then fixup the methodtable with the pointer to the just created native code.
MSDN has a more in depth explanation in the MethodDesc section
This blog entry by Dave Notario explains how the CLR JIT compiler works.

Resources