I am trying to hunt down some deadlock in multi-threaded code which uses a condition variable. Somebody advised using thread sanitizer. So I compiled LLVM from source and enabled thread sanitizer
LLVM built fine, but when I try to build my project I run into a lot of warnings of this kind:
: && /home/dzenan/LLVM-rel/bin/clang++ -fsanitize=thread -Wall -Wcast-align -Wdisabled-optimization -Wextra -Wformat=2 -Winvalid-pch -Wno-format-nonliteral -Wpointer-arith -Wshadow -Wunused -Wwrite-strings -funit-at-a-time -Wno-strict-overflow -Wno-deprecated -Wno-invalid-offsetof -Wno-undefined-var-template -Woverloaded-virtual -w -O2 -g -DNDEBUG -fsanitize=thread Modules/ThirdParty/VNL/src/vxl/v3p/netlib/tests/CMakeFiles/netlib_lsmrTest2.dir/lsmrTest2.cxx.o -o bin/netlib_lsmrTest2 -Wl,-rpath,/home/dzenan/ITK-git-clang/lib lib/libitkv3p_netlib-5.0.so.1 -lm && :
/usr/bin/ld: cannot find /home/dzenan/LLVM-rel/lib/clang/7.0.0/lib/linux/libclang_rt.tsan-x86_64.a: No such file or directory
/usr/bin/ld: cannot find /home/dzenan/LLVM-rel/lib/clang/7.0.0/lib/linux/libclang_rt.tsan_cxx-x86_64.a: No such file or directory
clang-7: error: linker command failed with exit code 1 (use -v to see invocation)
[1507/5739] Building C object Modules/ThirdParty/VNL/src/vxl/v3p/netlib/CMakeFiles/itknetlib.dir/triangle.c.o
FAILED: bin/netlib_lsqrTest1
What am I doing wrong and how to fix it?
I had this problem too. In my case, although these libraries were present in the system, they were not in a place where the linker could find them.
I solved that problem in a one-shot way by running this in the terminal I was using for compilation (replace the path with your local equivalent): export LD_LIBRARY_PATH=/lib64/clang/14.0.3/lib/linux/
It is only applied to the current terminal though. You may need to repeat that process each time, or use some persistent way of setting this for it to work everywhere.
Related
I'm trying to compile a wasm binary with multivalue and O3, however, I keep getting [wasm-validator error in module] unexpected false: Imported multivalue function requires multivalue [--enable-multivalue].
The current flags that I'm using are --target=wasm32-unknown-unknown-wasm -O3 -nostdlib -funroll-loops -Wall -Wno-comment -mllvm -polly -mmultivalue -Xclang -target-abi -Xclang experimental-mv -ffunction-sections -fdata-sections -flto -s -Wl,--export-dynamic,--allow-undefined,--gc-sections.
No, sadly there is no way (today) to modify the wasm-opt command line that clang uses.
This seems like a bug in llvm and really the target-features section of binary should contains multivalue, which should then enable it automatically in wasm-opt. Could you open an llvm bug about this?
For now your best bet might be to run clang without wasm-opt in your PATH.
I wrote C program that calculates the time step iterations of wavefunctions to solve the time depended Schrödinger equation.
At some step I need to do forward and backward Fast Fourier Transformations (FFT) for which I used the library kissfft.
(https://github.com/mborgerding/kissfft)
My programs structure is somewhat like this:
TDSE (working directory)
modules
include
scripts
test
inttest_analytical.c
kissfft
libkissfft-double.dylib
Now when I compile my inttest_analytical.c it works.
But when trying to run the executable afterwards I get the following error:
(base) user TDSE % ./inttest_analytical
dyld: Library not loaded: libkissfft-double.dylib
Referenced from: /Users/user/Documents/Uni/HU Berlin/Computational Physics 2/Project 3 - Time-dependet Schroedinger Equation/TDSE/./inttest_analytical
Reason: image not found
zsh: abort ./inttest_analytical
After running otool -L ./inttest_analytical I get
/inttest_analytical:
libkissfft-double.dylib (compatibility version 0.0.0, current version 0.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1281.100.1)
As far as I understand from reading about this in other questions on google searches, libkissfft-double.dylib is a static library but I don't manage to tell gcc where to find the library. And the path it was given (by the compiler or linker?) is the working directory TDSE instead of TDSE/kissfft
For compilation I ran:
gcc -g -Wall -fPIC -I include -I kissff ./modules/wavefunction.c ./modules/integrator.c ./modules/geometry.c ./modules/linearalgebra.c ./modules/assert.c ./modules/hamiltonian.c ./modules/conjugategradient.c ./test/inttest_analytical.c -Lkissfft -lkissfft-double -o inttest_analytical
So I guess I am using the flags -L and -l wrong??
Thanks for any help.
I guess I'll use this question to write a canonical answer for all "image not found" issues.
1. The issue
Let's start with a minimal setup consisting of a main binary and a library, like so:
main.c:
#include <stdio.h>
extern int f(void);
int main(void)
{
printf("%u\n", f());
return 0;
}
xyz.c:
int f(void)
{
return 42;
}
command line:
% cc -Wall -O3 -shared -o libxyz.dylib xyz.c
% cc -Wall -O3 -o main main.c -L. -lxyz
This works. You can run ./main and it will print 42.
However, if you now create a folder lib, move libxyz.dylib there and recompile main like so:
% cc -Wall -O3 -o main main.c -Llib -lxyz
Then the compilation will still succeed, however launching it will not:
% ./main
dyld: Library not loaded: libxyz.dylib
Referenced from: /private/tmp/./main
Reason: image not found
But if you go back and recompile libxyz.dylib to the lib folder directly and then rebuild main, like so:
% cc -Wall -O3 -shared -o lib/libxyz.dylib xyz.c
% cc -Wall -O3 -o main main.c -Llib -lxyz
Then it will once again work. But just to illustrate, this is the error you get if you move libxyz.dylib once more:
% ./main
dyld: Library not loaded: lib/libxyz.dylib
Referenced from: /private/tmp/./main
Reason: image not found
Just to make things worse though, you can also produce this error without even moving the library: simply cd lib and invoke ../main.
Also note the difference to before, libxyz.dylib vs lib/libxyz.dylib. This brings us to the core of the issue.
2. The reason
On macOS, each shared library has an "install name", i.e. the path at which it is expected to be found at runtime. This path can take three forms:
Absolute, e.g. /usr/lib/libxyz.dylib.
Relative, e.g. lib/libxyz.dylib.
Magic, e.g. #rpath/libxyz.dylib.
This path is embedded in the Mach-O header of the library, via the LC_ID_DYLIB load command. It can be viewed with otool like so:
% otool -l /tmp/lib/libxyz.dylib | fgrep -B1 -A5 LC_ID_DYLIB
Load command 2
cmd LC_ID_DYLIB
cmdsize 48
name lib/libxyz.dylib (offset 24)
time stamp 1 Thu Jan 1 01:00:01 1970
current version 0.0.0
compatibility version 0.0.0
This load command is created by the linker, whose man page (man ld) tells us the following:
-install_name name
Sets an internal "install path" (LC_ID_DYLIB) in a dynamic library.
Any clients linked against the library will record that path as the
way dyld should locate this library. If this option is not specified,
then the -o path will be used. This option is also called
-dylib_install_name for compatibility.
This tells us the three steps of how install names work:
The linker embeds the name when the library is built.
The linker copies the name into binaries linking against that library when those are built.
Dyld uses that name to try and load the library.
This will obviously cause issues if libraries are moved, or aren't even being compiled with the install name matching the path at which they will end up.
3. The solution
The solution is to change the install name path. Where and how depends on your setup. You can change it by two means:
Recompile the library with the correct install name (either -Wl,-install_name,... or outright -o ...), then recompile the main binary to link against that.
Use install_name_tool. This is a bit more involved.
In either case, you need to decide what form of install name you want to use:
Absolute.
This is recommended for libraries in global paths, shared by all users. You can also use this to point to your user directory, but it's a bit ugly since you can't move the binaries around or distribute them to someone else.
Relative.
Being relative to your working directory means this is entirely unreliable.
Never use this. Just don't.
Magic.
There are three "special" tokens that go beyond absolute and relative paths:
#executable_path is the runtime directory of the main binary of the process. This is the simplest form, but only works if your libraries are only used in a single main binary.
#loader_path is the runtime directory of the binary depending on the library. I recommend not using this, as it breaks if you have two binaries in different folders that want to link to the same library.
#rpath is a list of runtime directories assembled from LC_RPATH load commands. This is a bit more complex, but it's the most flexible solution, since it can itself contain #executable_path and #loader_path.
Use of those allows you to build binaries that can be moved around freely, so long as they all retain their relative position.
For a full description of them, see man dyld.
With that out of the way, let's look at implementing the possible solutions. We have:
cc -Wl,-install_name,... to specify an install name at compile time.
install_name_tool -id ... to change the path embedded in a library.
install_name_tool -change old new to change the path embedded in a binary linking against a library.
3.1 Absolute paths
If you can recompile both the library and the main binary:
% cc -Wall -O3 -shared -o /tmp/lib/libxyz.dylib xyz.c
% cc -Wall -O3 -o main main.c -L/tmp/lib -lxyz
If you can only recompile the main binary:
% install_name_tool -id '/tmp/lib/libxyz.dylib' /tmp/lib/libxyz.dylib
% cc -Wall -O3 -o main main.c -L/tmp/lib -lxyz
If you cannot recompile either:
% install_name_tool -id '/tmp/lib/libxyz.dylib' /tmp/lib/libxyz.dylib
% install_name_tool -change 'libxyz.dylib' '/tmp/lib/libxyz.dylib' main
3.2 #executable_path
If you can recompile both the library and the main binary:
% cc -Wall -O3 -shared -o lib/libxyz.dylib xyz.c -Wl,-install_name,'#executable_path/lib/libxyz.dylib'
% cc -Wall -O3 -o main main.c -Llib -lxyz
If you can only recompile the main binary:
% install_name_tool -id '#executable_path/lib/libxyz.dylib' lib/libxyz.dylib
% cc -Wall -O3 -o main main.c -Llib -lxyz
If you cannot recompile either:
% install_name_tool -id '#executable_path/lib/libxyz.dylib' lib/libxyz.dylib
% install_name_tool -change 'libxyz.dylib' '#executable_path/lib/libxyz.dylib' main
3.3 #rpath
Rpath requires manual addition of runtime paths, which requires some planning. Suppose you have the follwing file hierarchy:
a
bin/
b
libx.dylib
lib/
liby.dylib
libz.dylib
a and b are binaries that both link against libx and liby, which in turn both link against libz. For the install name of libz, you can use neither #executable_path (because a and b are in different directories) nor #loader_path (because libx and liby are in different directories). But you can use either of them inside #rpath, and here is the decision you have to make:
You can either embed an rpath of #executable_path in a and #executable_path/.. in b. Then you can use #rpath to refer to the project root from all binaries. libz would have an install name of #rpath/lib/libz.dylib.
Or you can embed an rpath of #loader_path/lib in libx and #loader_path in liby. Then you can use #rpath to refer to the directory containing each binary. libz would have an install name of #rpath/libz.dylib.
I generally find the former to be easier to deal with, but the latter may be preferable if you have a large number of binaries scattered over many directories and only a few libraries.
To actually add an rpath to a binary, you can use:
cc -Wl,-rpath,... at compile time.
install_name_tool -add_rpath ... afterwards.
So if you can recompile both the library and the main binary:
% cc -Wall -O3 -shared -o lib/libxyz.dylib xyz.c -Wl,-install_name,'#rpath/lib/libxyz.dylib'
% cc -Wall -O3 -o main main.c -Llib -lxyz -Wl,-rpath,'#executable_path'
If you can only recompile the main binary:
% install_name_tool -id '#rpath/lib/libxyz.dylib' lib/libxyz.dylib
% cc -Wall -O3 -o main main.c -Llib -lxyz -Wl,-rpath,'#executable_path'
If you cannot recompile either:
% install_name_tool -id '#rpath/lib/libxyz.dylib' lib/libxyz.dylib
% install_name_tool -change 'libxyz.dylib' '#rpath/lib/libxyz.dylib' main
% install_name_tool -add_rpath '#executable_path' main
Note that if any of your binaries are signed, this will of course invalidate that signature. Use codesign -f ... to replace the existing signature(s).
recently I updated my armadillo to version 5.200.1, but when i tried to compile my code using
g++ -std=c++11 -m64 -O3 -Wall -I. -I/opt/OpenBLAS/include -fopenmp -o code.cpp.o -c code.cpp
g++ -std=c++11 -m64 -O3 -Wall -I. -I/opt/OpenBLAS/include -fopenmp -o main.cpp.o -c main.cpp
g++ -std=c++11 -m64 -O3 -o code.cpp.o main.cpp.o -lgomp -L/opt/OpenBLAS/lib -lopenblas -larmadillo
I got error message in the final linking step
/usr/lib/gcc/x86_64-linux-gnu/4.9/../../../../lib/libarmadillo.so:
undefined reference to `pthread_atfork'
collect2: error: ld returned 1 exit status
make: *** [a.out] Error 1
This error appears in my machine with Ubuntu 14.04LTS + gcc 4.9.2.
The Armadillo itself is linked with OpenBLAS 0.2.14 (I followed the procedure provided by Armadillo in installation)
such error doesn't occur in my macosx laptop (the same library). It means something problems with my ubuntu machine. I tried to re-install the Armadillo 4.650.2 (this version worked before), but the same error kept appearing (now older doesn't work like before). The only thing that i did before updating those libraries were "apt-self update" and "apt-self upgrade" of my ubuntu machine
is there any suggestion to fix this problem?
I also use armadillo on Ubuntu Linux
I had the same problem first when I started to program with armadillo. For g++ I only had to add -larmadillo as command line option. such as this
g++ -O2 -o armadillo_example armadillo_example.cpp -larmadillo
I'm writing an iOS application that runs an xml-rpc server. I'm using the C library for xml-rpc available here : http://xmlrpc-c.sourceforge.net/ . The library uses ./configure to generate the makefile according to the host architecture, and make, make install to build/install the library.
I need to use this library with my iOS application. I understand that XCode uses a completely different build system, and I'm new to iOS development, so I'm not quite sure how to tie in the library with my application. What options do I need to pass in with ./configure to compile the library for iOS architectures (I understand there are three targets, armv7, armv7s and i386)?
Thanks in advance.
Here are the configure script options for the simulator and device platforms.
Simulator (i386) :
$ ./configure --host=i386-apple-darwin --prefix <path_to_binaries> CFLAGS="-arch i386 -isysroot /Applications/XCode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator7.1.sdk/ -miphoneos-version-min=4.0 --disable-cplusplus
$ make
$ make install
iOS device (arm) : this is a bit trickier, and there's an unresolved issue with this, but here are the general steps:
$ ./configure --host=arm-apple-darwin --prefix <path_to_binaries> CFLAGS="-arch armv7s -isysroot /Applications/XCode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS7.1.sdk -miphoneos-version-min=4.0" --disable-curl-client --disable-cplusplus LDFLAGS='-arch armv7s -miphoneos-version-min=4.0 -isysroot /Applications/XCode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS7.1.sdk’
$ make
Notes specific to this library : We need to build gennmtab against x86, since it defines the symbol table and is to be run on the machine compiling the program - run the following commands (starting from the base xmlrpc-xx.xxx directory)
$ cd lib/expat/gennmtab/
$ rm gennmtab gennmtab.o
$ gcc -c gennmtab.c -o gennmtab.o -DNDEBUG -Wall -W -Wno-uninitialized -Wundef -Wimplicit -Winline -Wno-unknown-pragmas -Wmissing-declarations -Wstrict-prototypes -Wmissing-prototypes -fno-common -g -O3 -D_THREAD -I/Users/sradhakrishnan/dev/xmlrpc-c-1.25.28 -Isrcdir/lib/util/include -I/Users/sradhakrishnan/dev/xmlrpc-c-1.25.28 -Isrcdir/lib/util/include
Now that gennmtab has been built against x86, continue with previous make, by running the following:
$ cd ../../../ (you are back in the base xmlrpc-xx.xxx directory)
$ make
$ make install
I am trying to do a gem install therubyracer-heroku -v '0.8.1.pre3', but I am getting errors while building, it is a big log but here is the last failing lines (pasted full log in the end):
g++ -o obj/release/bootstrapper.o -c -Wall -Werror -W -Wno-unused-parameter -Wnon-virtual-dtor -pedantic -m32 -O3 -fomit-frame-pointer -fdata-sections -ffunction-sections -ansi -fno-rtti -fno-exceptions -fvisibility=hidden -Wall -Werror -W -Wno-unused-parameter -Wnon-virtual-dtor -pedantic -m32 -O3 -fomit-frame-pointer -fdata-sections -ffunction-sections -ansi -DV8_TARGET_ARCH_IA32 -DENABLE_VMSTATE_TRACKING -DENABLE_LOGGING_AND_PROFILING -DENABLE_DEBUGGER_SUPPORT -Isrc src/bootstrapper.cc
src/bootstrapper.cc: In static member function 'static bool v8::internal::Genesis::CompileScriptCached(v8::internal::Vector<const char>, v8::internal::Handle<v8::internal::String>, v8::internal::SourceCodeCache*, v8::Extension*, v8::internal::Handle<v8::internal::Context>, bool)':
src/bootstrapper.cc:1002:18: error: variable 'result' set but not used [-Werror=unused-but-set-variable]
src/bootstrapper.cc: In member function 'bool v8::internal::Genesis::InstallNatives()':
src/bootstrapper.cc:1227:24: error: variable 'name' set but not used [-Werror=unused-but-set-variable]
cc1plus: all warnings being treated as errors
scons: *** [obj/release/bootstrapper.o] Error 1
scons: building terminated because of errors.
make: *** [build/v8/libv8.a] Error 2
*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of
necessary libraries and/or headers. Check the mkmf.log file for more
details. You may need configuration options.
I have googled about his for many hours and looked into a lot of SO similar questions, but nothing helped, also I don't know what is mkmf log mean and where to find it.
Additional information:
OS: Ubuntu, 12.04
I already have nodeJs installed, version is v0.8.14 (Just giving this info, since it may be linked with this)
Ruby version: 1.9.3
RVM is used