I'm having trouble getting started with Metal's shader compilation.
How to make a MTLLibrary that can link to a MTLDynamicLibrary (or MTLLinkedFunctions), in particular a library that declares extern functions that are to be resolved at runtime when providing preloadedLibraries (or linkedFunctions) in the compute pipeline descriptor? For example, I can compile the following to air using xcrun metal (with option -c), but then invoking xcrun metallib (even with option --split-module-without-linking) gives the error LLVM ERROR: Undefined symbol: _Z3addjj. In other words, how do I make a 'partially bound' metal library?
// shader.h
extern uint add(uint a, uint b);
/// shader.metal
#include "shader.h"
kernel void kernel_func(uint gid [[ thread_position_in_grid ]]) { add(gid,2); }
WWDC2021 mentions this extern technique, but the Dynamic Library Code Sample from the previous year doesn't use extern (or the installName), so I don't make sense of it.
When creating an executable library that uses a dynamic library, there are two points where you must include the dynamic library (I thought there was only one).
The process is different depending on whether the executable source is compiled at build or runtime. I'll describe for the case of runtime, because I haven't yet figured out the case for the executable library created from a metallib file.
The first point is when you compile the executable, where you must include the dynamic library in the libraries field of the CompileOptions. The library is there at this point just as a dummy, to check that you have a dynamic library that defines the declarations allowing for proper linkage, though that linkage doesn't occur at this stage, just the checking.
The second point is when you create the pipeline state, where you must include the dynamic library in the preloadedLibraries field of the pipeline descriptor. This time, the dynamic library is not a dummy but the real library you plan to use, as it will be linked with the executable during pipeline creation.
Related
First question here.
I have some troubles with the XCode Build System, specifically with preprocessor definitions.
I'm trying to define a macro for the objective-c runtime to avoid enforcing the dispatch functions to be cast to an appropriate function pointer type. The usual way to go would be to use #define OBJC_OLD_DISPATCH_PROTOTYPES and then include the header on the next line. Once the header gets included, the macro is already defined and the header is configured accordingly.
But that's where it starts to get weird!
The macro is not recognized at all and the header gets included as if the #define statement was not there so it fails to #define OBJC_OLD_DISPATCH_PROTOTYPES and it gets (re?)defined as 0.
main.c
#include <stdio.h>
#define OBJC_OLD_DISPATCH_PROTOTYPES 1
#include <objc/objc-runtime.h>
int main(int argc, const char * argv[]) {
// From there:
// - Build System: OBJC_OLD_DISPATCH_PROTOTYPES is always 0, except if defined in build settings
// - Clang (only): OBJC_OLD_DISPATCH_PROTOTYPES is 1
printf("%d\n", OBJC_OLD_DISPATCH_PROTOTYPES);
}
The build system acts as expected when the preprocessor macro is defined in the project build settings under the "Apple Clang - Preprocessing" section. It defines the global macro using the -D parameter of clang making it available to any files used by the project.
However, source code compiles correctly when I use clang from a terminal using clang main.c.
Could someone tell me what I need to configure for the build system to behave normally?
It gives a warning when building with Xcode IDE:
Ambiguous expansion of macro 'OBJC_OLD_DISPATCH_PROTOTYPES'
and the output is indeed 0 using Xcode directly, but 1 with clang main.c. The difference is that Xcode uses clang with enabled modules by default: You get the same warning on the command line if you enable modules there:
clang -fmodules main.c
Solution
In Xcode, select the target, go to the "Build Settings" tab and in the "Apple Clang - Language - Modules" section, switch the "Enable Modules (C and Objective-C)" entry to 'NO':
Then you get the expected result in both cases, regardless of whether you use Xcode or Clang on the command line.
Explanation:
If you use modules the following happens:
instead of the preprocessor including the text and compiling the result, a binary representation of the module is used
modules are (independently) precompiled, i.e. they use the definitions from the time the module was precompiled
consequently, preprocess definitions from the code before the include/import statement have no effect on the module (nor on other imported modules).
if modules are enabled, not only #imports are affected, but also #includes are translated into module imports under the hood
So you have a contradictory definitions for the OBJC_OLD_DISPATCH_PROTOTYPES.
The precompiled module uses a 0 for OBJC_OLD_DISPATCH_PROTOTYPES and you redefine it as 1.
BTW: if you use
#define OBJC_OLD_DISPATCH_PROTOTYPES 0
then you use the same definition that the precompiled module is using and therefore there is no warning about an ambiguous expansion of the macro even if modules are enabled.
Without enabled modules, the preprocessor includes the text, compiles the result and returns the expected result, i.e. in objc.h the desired typedef are used.
I am facing quite a strange situation trying to build a mixed-language dynamic iOS framework with Bazel.
The source code of the framework consists of *.c, *.cpp, *.m, *.mm, *.h, *.hpp files.
My first naive attempt was to declare a single objc_library rule referencing all the sources. That failed with an error message reading something about conflicting rules. Then I declared four distinct objc_library rules for *.m, *.mm, *.c, and *.cpp files respectively, and then referenced all these four rules as dependencies for the final ios_framework rule.
At this point, everything compiled and linked just fine (barring several compiler warnings which were expected). However, now the black magic begins:
The static library resulting from compiling ObjectiveC++ sources does contain all the necessary symbols (verified with the "nm" tool)
The .apple_binary_lipobin file resulting from the linking phase does not contain symbols from ObjectiveC++ sources
I believe I carefully went through the command lines used for compiling and linking, both for Bazel and xcodebuild.
The only anomalies I was able to spot:
xcodebuild passes a -single_module argument when linking, while Bazel cross tool does not.
Bazel cross tool adds '-stdlib=libc++' '-std=gnu++11' command line arguments when invoking wrapped_clang_pp during linking, while xcodebuild does not pass these arguments. I guess Bazel does this because there're these lines in the default Apple CROSSTOOL:
action_config {
config_name: "objc++-executable"
action_name: "objc++-executable"
tool {
tool_path: "wrapped_clang_pp"
execution_requirement: "requires-darwin"
}
flag_set {
flag_group {
flag: "-stdlib=libc++"
flag: "-std=gnu++11"
}
but I am not sure if these flags are really needed when invoking the linker.
I must admit I've run out of ideas except for trying to patch the CROSSTOOl file to make it behave as close as xcodebuild as possible.
Please help.
Can you try adding alwayslink = 1 to the objc_library target containing the C++ symbols? The linker is deadstripping the C++ symbols as they are not being referenced anywhere in the binary.
I'v been trying to get OmniThreadLibrary to run in builder, i've built it with all the c++ required files it builds ok but when i use it in an c++ builder app i get a bunch of error messages that look like the following
[bcc32 Error] DSiWin32.hpp(385): E2040 Declaration terminated incorrectly
one points at this line of code in the generated hpp file
static const System::Int8 CSIDL_ADMINTOOLS = System::Int8(0x30);
has anyone had this working in C++ builder or know the best way to resolve these issues
I'm using c++ builder settle and OmniThreadLibrary version 3.06
The Win32 API (and Delphi, for that matter) already declares CSIDL_ADMINTOOLS, Omni should not be declaring it at all. It should be using Delphi's Shlobj unit instead.
The Win32 API declares CSIDL_ADMINTOOLS using a #define statement:
#define CSIDL_ADMINTOOLS 0x0030
So the declaration in Omni's .hpp is getting modified by the C++ preprocessor to this:
static const System::Int8 0x0030 = System::Int8(0x30);
Thus the "Declaration terminated incorrectly" compiler error.
When Delphi code declares something that already exists in C++, it needs to be declared as either {$EXTERNALSYM} or {$NODECLARE} to avoid duplicate declarations, and then optionally use {$HPPEMIT} to output a relevant #include statement in a generated .hpp file. Delphi's units already do that for its Win32 declarations.
If Omni is not already doing that (and the error would suggest it is not) then it needs to be updated accordingly.
I have application that contain statically linked lua 5.2 inteperpreter (and haven't access to code).
When I trying to write extension with next code:
#define LUA_LIB
#define LUA_BUILD_AS_DLL
#include "lua.hpp"
extern "C"
{
static int test(lua_State* state)
{
return 1;
}
static const struct luaL_Reg functions[] = {
{"test", test},
{NULL, NULL},
};
int __declspec(dllexport) luaopen_test(lua_State* state)
{
luaL_newlibtable(state, functions);
luaL_setfuncs(state, functions, 0);
return 0;
}
}
And compile it with statically linked lua52.lib .
I get "multiple vms detected" error when I trying to require it fromn lua code.
What I can do in this situation?
You can't compile it with statically linked lua52.lib as the main application loads its own version of lua52.lib and when this module is "required", it loads its own copy, which leads to "multiple VMs detected" message.
With statically compiled VM you have two options (on Windows): (1) include all your modules statically, or (2) compile your modules against Lua52.dll, but instead of the actual DLL include a "proxy" DLL that will forward Lua API calls to the methods in the statically compiled executable (the API methods also need to be exported in the executable).
See this thread for the discussion on how the executable needs to be compiled and LuaProxyDllFour page for the proxy DLL.
On Linux you don't need to have a proxy library, but you still need to avoid linking Lua interpreter into the library and export symbols from the executable by using -Wl,-E linker option; see lhf's answer for details.
I have the following trivial Lua program which I copied from the book Programming In Lua
#include <stdio.h>
#include <lua.h>
#include <lauxlib.h>
#include <lualib.h>
int main (void)
{
char buff[256];
int error;
lua_State *L = luaL_newstate(); /* opens Lua */
luaL_openlibs(L); /* opens the standard libraries */
while (fgets(buff, sizeof(buff), stdin) != NULL)
{
error = luaL_loadbuffer(L, buff, strlen(buff), "line") ||
lua_pcall(L, 0, 0, 0);
if (error)
{
fprintf(stderr, "%s", lua_tostring(L, -1));
lua_pop(L, 1); /* pop error message from the stack */
}
}
lua_close(L);
return 0;
}
my environment is cywin
my make file looks like this:
CC=gcc
INCLUDE='-I/home/xyz/c_drive/Program Files/Lua/5.1/include'
LINKFLAGS='-L/home/xyz/c_drive/Program Files/Lua/5.1/lib' -llua51
li.o:li.c
$(CC) $(INCLUDE) -c li.c
main:li.o
$(CC) -o main $(LINKFLAGS) li.o
clean:
rm *.o
rm main
My /home/xyz/c_drive/Program Files/Lua/5.1/lib directory contains lua5.1.dll lua5.1.lib lua51.dll and lua51.lib
Trying to build my main target I am getting the following errors:
li.o:li.c:(.text+0x35): undefined reference to `_luaL_newstate'
li.o:li.c:(.text+0x49): undefined reference to `_luaL_openlibs'
li.o:li.c:(.text+0xaf): undefined reference to `_luaL_loadbuffer'
li.o:li.c:(.text+0xd9): undefined reference to `_lua_pcall'
li.o:li.c:(.text+0x120): undefined reference to `_lua_tolstring'
li.o:li.c:(.text+0x154): undefined reference to `_lua_settop'
li.o:li.c:(.text+0x167): undefined reference to `_lua_close'
Any ideas about what I might be doing wrong here?
The problem is that you have named the libraries on the link command line before the object files that require them. The linker loads modules from left to right on the command line. At the point on the line where you name -llua51, no undefined symbols that could be satisfied by that library are known. Then you name li.o, which does have unknown symbols.
Some Unix-like environments don't treat this as an error because part of the link process is deferred to the program load when reference to .so files are satisfied. But Cygwin, MinGW, and Windows in general must treat this as an error because DLLs work quite differently from .so files.
The solution is to put -llua51 after all the .o files on your link line.
Edit: Incidentally, it appears you are linking against the Lua for Windows distribution, but building with GCC under Cygwin. You will want to use Dependency Walker to make sure that your program does not depend on the Cygwin runtime, and that it does depend on the same C runtime as the lua51.dll from Lua for Windows. IIRC, that will be the runtime for the previous version of Visual Studio. It is possible to make GCC link against that, but you will need to be using the MinGW port (which you can use from Cygwin), and link against a couple of specific libraries to get that version. I'm away from my usual PC, or I'd quote an exact link line. (I believe you need -lmoldname -lmsvcr80 or something like that, as the last items on the link line.)
It will cause mysterious and very hard to diagnose problems if more than one C runtime library is in use. The easy answer is to use the same one as your preferred Lua DLL. Another alternative is that the Lua Binaries project has pre-compiled Lua DLLs for a wide array of C toolchains on Windows. If you need a Lua application that understands the Cygwin environment, you will want one that is built by GCC for Cygwin and not the Lua for Windows flavor. Lua Binaries will be your friend, or you can build Lua your self from source.
The names in the Lua API do not have those leading underscores. Try compiling with -fno-leading-underscore.