I am trying to compile a shared library using static libraries from Boost and OpenCV. Here below the command I am using to compile my library.
g++ -fPIC libsaliency.cpp -shared -o libsaliency.so \
-I/home/poiesi/data/libraries/boost_1_66_0/installed_w_contrib_static/include -I/home/poiesi/data/libraries/opencv-3.4.0/installed_w_contrib_static/include \
-Wl,--whole-archive \
/home/poiesi/data/libraries/boost_1_66_0/installed/lib/libboost_graph.a \
/home/poiesi/data/libraries/boost_1_66_0/installed/lib/libboost_filesystem.a \
/home/poiesi/data/libraries/boost_1_66_0/installed/lib/libboost_system.a \
/home/poiesi/data/libraries/opencv-3.4.0/installed_w_contrib_static/lib/libopencv_core.a \
/home/poiesi/data/libraries/opencv-3.4.0/installed_w_contrib_static/lib/libopencv_highgui.a \
/home/poiesi/data/libraries/opencv-3.4.0/installed_w_contrib_static/lib/libopencv_imgproc.a \
/home/poiesi/data/libraries/opencv-3.4.0/installed_w_contrib_static/lib/libopencv_imgcodecs.a \
/home/poiesi/data/libraries/opencv-3.4.0/installed_w_contrib_static/lib/libopencv_features2d.a \
/home/poiesi/data/libraries/opencv-3.4.0/installed_w_contrib_static/lib/libopencv_video.a \
-Wl,--no-whole-archive
However, I have this error:
usr/bin/ld: /home/poiesi/data/libraries/boost_1_66_0/installed/lib/libboost_graph.a(read_graphviz_new.o): relocation R_X86_64_32 against `.rodata.str1.8' can not be used when making a shared object; recompile with -fPIC
/home/poiesi/data/libraries/boost_1_66_0/installed/lib/libboost_graph.a(read_graphviz_new.o): error adding symbols: Bad value
collect2: error: ld returned 1 exit status
Makefile:7: recipe for target 'saliency' failed
make: *** [saliency] Error 1
Does this mean I have to recompile Boost using -fPIC command? I checked this online but I haven't found much info about it. This makes me wonder if I am searching for the right thing. Do you have any suggestion?
EDIT: As suggested below by Mike, I recompiled Boost like this:
./b2 cxxflags="-fPIC" link=static install
and I can now compile my .so library.
Does this mean I have to recompile Boost using -fPIC command?
Yes. All code that is linked into a shared library must be Position Independent Code. Object
files within static libraries normally are not, as shared libraries normally
link other shared libraries.
But there is nothing in principle to stop you from building boost static libraries
from -fPIC-compiled object files.
It would be simpler, of course, to link the shared versions of the boost libraries.
With boost 1.72.0, fixed this problem by recompile boost static library with -fPIC.
./bootstrap.sh --prefix=/usr/
sudo ./b2 cxxflags=-fPIC cflags=-fPIC link=static -a
sudo ./b2 install
Related
I am trying to build LLVM compilers so that I can enable OpenMP on the Apple M1.
I am using the LLVM development tree, (since I saw some OpenMP runtime go into that for this recently).
I have ended up with this script to invoke cmake:
# Xcode, Ninja
BUILD_SYSTEM=Ninja
BUILD_TAG=Ninja
cmake ../llvm \
-G$BUILD_SYSTEM -B ${BUILD_TAG}_build \
-DCMAKE_OSX_ARCHITECTURES='arm64' \
-DCMAKE_C_COMPILER=`which clang` \
-DCMAKE_CXX_COMPILER=`which clang++` \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_BUILD_WITH_INSTALL_RPATH=1 \
-DCMAKE_INSTALL_PREFIX=$HOME/software/clang-12.0.0/arm64 \
-DLLVM_ENABLE_WERROR=FALSE \
-DLLVM_TARGETS_TO_BUILD='AArch64' \
-DLLVM_ENABLE_PROJECTS='clang;openmp,polly' \
-DLLVM_DEFAULT_TARGET_TRIPLE='aarch64-apple-darwin20.1.0'
The compilers used here are
$ /usr/bin/clang --version
Apple clang version 12.0.0 (clang-1200.0.32.27)
Target: arm64-apple-darwin20.1.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
ninja can then successfully build clang, clang++ and the OpenMp runtime and install them. (As simple, Arm64 images targeting Arms64)
$ file ~/software/clang-12.0.0/arm64/bin/clang
/Users/jcownie/software/clang-12.0.0/arm64/bin/clang: Mach-O 64-bit executable arm64
$ ~/software/clang-12.0.0/arm64/bin/clang --version
clang version 12.0.0 (https://github.com/llvm/llvm-project.git 879c15e890b4d25d28ea904e92497f091f796019)
Target: aarch64-apple-darwin20.1.0
Thread model: posix
InstalledDir: /Users/jcownie/software/clang-12.0.0/arm64/bin
Which all looks sane, except that when I try to compile anything with them they are missing the include path to get system headers.
$ ~/software/clang-12.0.0/arm64/bin/clang hello.c
hello.c:1:10: fatal error: 'stdio.h' file not found
#include <stdio.h>
^~~~~~~~~
1 error generated.
So, after all that,
Does anyone know how to fix that include path problem?
Does anyone know how to configure and build a fat binary for the compilers (and libraries) so that the x86_64 embedded compiler targets x86_64 and the aarch64 binary aarch64? (This is what the Xcode clang and clang++ do...)
My attempt at this ended up with a compiler fat binary where both architectures targeted x86_64 :-(
Thanks
You can set -DDEFAULT_SYSROOT=/path/to/MacOSX11.1.sdk at build time or do export SDKROOT=/path/to/MacOSX11.1.sdk at runtime.
You need to compile with clang -arch arm64 -arch x86_64 to get a fat binary out of clang. You need to do this for Apple clang as well.
UPDATED 8 Feb 2021
Homebrew now supports the M1 based Arm machines, so using that is a better answer than the one below.
The info below is potentially still useful if you want to do this on your own, but using brew is likely to be much simpler.
Pre-brew answer
I haven't found a clean solution, but in case it helps anyone else, I do have a horrible hack.
The full recipe, then is configure with this script, then build and install.
# Xcode, Ninja
BUILD_SYSTEM=Ninja
BUILD_TAG=ninja
INSTALLDIR=$HOME/software/clang-12.0.0/arm64
cmake ../llvm \
-G$BUILD_SYSTEM -B ${BUILD_TAG}_build \
-DCMAKE_OSX_ARCHITECTURES='arm64' \
-DCMAKE_C_COMPILER=`which clang` \
-DCMAKE_CXX_COMPILER=`which clang++` \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX=$INSTALLDIR \
-DLLVM_LOCAL_RPATH=$INSTALLDIR/lib \
-DLLVM_ENABLE_WERROR=FALSE \
-DLLVM_TARGETS_TO_BUILD='AArch64' \
-DLLVM_DEFAULT_TARGET_TRIPLE='aarch64-apple-darwin20.1.0' \
-DDEFAULT_SYSROOT="$(xcrun --show-sdk-path)" \
-DLLVM_ENABLE_PROJECTS='clang;openmp;polly;clang-tools-extra;libcxx;libcxxabi' \
# -DLLVM_ENABLE_PROJECTS='clang;openmp;polly'
That gives a compiler that finds the right headers, but won't link successfully if OpenMP is used because it doesn't pass on any useful -L path or add a necessary rpath.
To overcome that I created a small shell script that sits in my ~/bin, at the front of my $PATH, which adds those extra linker flags.
#
# A truly awful hack, but it seems necessary.
# Install this with execute permissions as clang and clang++ in
# a directory early in your path, so that it is executed when clang or
# clang++ is needed.
#
# For brew...
INSTALLDIR=/usr/local/opt/llvm
# For a local build.
INSTALLDIR=${HOME}/software/clang-12.0.0/arm64/
# Find out the name of this file, and then invoke the same file in the
# compiler installation, adding the necessary linker directives
CMD=`echo $0 | sed "s/\/.*\///"`
${INSTALLDIR}/bin/${CMD} -L${INSTALLDIR}/lib -Wl,-rpath,${INSTALLDIR}/lib $*
I am not recommending this particularly; there should clearly be a better way to make it work, but it'll do for now, and lets me get back to using the compiler rather than building it!
I was able to build with -DDEFAULT_SYSROOT="$(xcrun --show-sdk-path)" -DCMAKE_INSTALL_PREFIX=/Users/foo/lokal/ and install into the lokal/bin lokal/lib path. Once that is done you can use LD_LIBRARY_PATH=/Users/foo/lokal/lib and all the libraries should be found without mucking with anything else rpath related.
I'm trying to add a .lo object file compiled through libtool with clang into a shared library file.
$ libtool --tag=CC --mode=compile clang -c newobject.c -shared
Is there an equivalent command to
$ ar r libmylib.a newobject.o
for shared libraries?
Alternatively, is there a way to dump all the .lo files that are already contained in a .so file so I can re-create tne shared library, say, using this command?
$ libtool --mode=link ld -soname libmylib.so -o libmylib.so.1 libmylib.so.0 newobject.o
There is no way to incrementally alter a shared library. You need all the component object files and rebuild the shared library from those each time.
I don't know of a way to extract object files from a shared library.
Environment: Ubuntu 18.04 bionic.
after git clone llvm-project from https://github.com/llvm/llvm-project.
I generated build credentials using Cmake commandline like this:
cmake -DLLVM_TARGETS_TO_BUILD=X86 \
-DLLVM_TARGET_ARCH=X86 \
-DCMAKE_BUILD_TYPE="Release" \
-DLLVM_BUILD_EXAMPLES=1 \
-DCLANG_BUILD_EXAMPLES=1 \
-G "Unix Makefiles" \
../llvm/
after makefile was generated, I then use make to build the project make -j8. somehow after build completion, clang and clang++ etc. are nowhere to be found in /build/bin/.
To my knowledge, if I didn't specify LLVM_ENABLE_PROJECTs, it will build LLVM and clang both by default. why are clang executables missing?
You need to explicitly specify LLVM_ENABLE_PROJECT and put the source code folder of clang in the same level of llvm source folder. The folder name must match the value of LLVM_ENABLE_PROJECT
I am adding some pthreads code into my Linux application that I'm building with autotools. I was getting an error about not linking in libpthreads. So I want to specify the pthreads dependency and compiler/linker flags in autotools.
I found some references that say use an ACX_PTHREAD macro. GNU provides an AX_PTHREAD macro. Both are very similar in concept. But I've tried both (on Ubuntu 13.04 64-bit), and found that they set -pthread in $PTHREAD_CFLAGS, but for some reason they don't set the -lpthread linker flag in $PTHREAD_LIBS.
The build fails. When I run make, I get:
...
/bin/sh ../libtool --tag=CXX --mode=link g++ -g -O2 -o myapp main.o ... -lconfuse -llog4cpp -lnsl -lpopt -lfuse -L/usr/local/lib -lrt
libtool: link: g++ -g -O2 -o .libs/myapp main.o ... -lconfuse -llog4cpp -lnsl /usr/lib/x86_64-linux-gnu/libpopt.so -lfuse -L/usr/local/lib -lrt
/usr/bin/ld: app-fuse.o: undefined reference to symbol 'pthread_kill##GLIBC_2.2.5'
/usr/bin/ld: note: 'pthread_kill##GLIBC_2.2.5' is defined in DSO /lib/x86_64-linux-gnu/libpthread.so.0 so try adding it to the linker command line
/lib/x86_64-linux-gnu/libpthread.so.0: could not read symbols: Invalid operation
collect2: error: ld returned 1 exit status
...
In this case, the ./configure step shows:
...
checking for the pthreads library -lpthreads... no
checking whether pthreads work without any flags... no
checking whether pthreads work with -Kthread... no
checking whether pthreads work with -kthread... no
checking for the pthreads library -llthread... no
checking whether pthreads work with -pthread... yes
checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE
checking if more special flags are required for pthreads... no
checking for PTHREAD_PRIO_INHERIT... yes
...
I notice it checks for -lpthreads, but shouldn't it be checking for -lpthread?
I've found that I can use:
AC_CHECK_LIB(pthread, pthread_create, [PTHREAD_LIBS+=-lpthread])
and then the build succeeds. But I assume this isn't the best way to make it work on the widest variety of platforms.
I see Ubuntu also has a package libpthread-stubs0-dev. But I'm not sure what it's for.
What is the "right way" to use pthreads with autotools?
Thanks to Peter Simons who asked on the autoconf mailing list, we have a somewhat official answer:
Compiler flags and linker flags are not mutually-exclusive sets, not
least because linking is typically done via the compiler frontend (cc)
and not by invoking the linker (ld) directly. Any flags that you can
use in the compile step (e.g. -O2, -DFOO, -I/tmp/include) will
generally be accepted in the linking step, even if it's not applicable
then. (The reverse may not be true, e.g. -lfoo.)
Given that, it's a lot less error-prone to use PTHREAD_CFLAGS (and
other CFLAGS variables) when linking, rather than duplicating the
applicable flags into PTHREAD_LIBS/LDFLAGS/etc. variables and not
using any CFLAGS variables then.
So just use PTHREAD_CFLAGS for your linker, too.
I bumped into this same issue when I added a first C++ source to an otherwise working C project (a shared library). Adding this C++ file caused libtool to switch from linking with gcc to linking with g++. Seems that linking with gcc a '-pthread' is enough to add the dynamic dependency to libpthread, but when linking with g++, it is not.
I tried with the above patch to a local ax_pthread.m4, but this didn't help. Passing '-lpthread' to g++ would fix the issue.
Edit: for some reason, ax_pthread.m4 forces C as the test language even if the AC_LANG is set as C++. This patch makes things work for me:
--- m4/ax_pthread.m4_orig 2013-06-15 20:03:36.000000000 +0300
+++ m4/ax_pthread.m4 2013-06-15 20:03:51.000000000 +0300
## -87,7 +87,6 ##
AU_ALIAS([ACX_PTHREAD], [AX_PTHREAD])
AC_DEFUN([AX_PTHREAD], [
AC_REQUIRE([AC_CANONICAL_HOST])
-AC_LANG_PUSH([C])
ax_pthread_ok=no
# We used to check for pthread.h first, but this fails if pthread.h
## -313,5 +312,4 ##
ax_pthread_ok=no
$2
fi
-AC_LANG_POP
])dnl AX_PTHREAD
It seems that the AX_PTHREAD macro is finding the -pthread compiler flag, and using that. But it looks as though for that particular flag, it should be specified to the linker as well (it apparently does the equivalent of -lpthread in the linker). I modified the macro as follows, so that the -pthread flag is specified as a linker flag too:
diff --git a/m4/ax_pthread.m4 b/m4/ax_pthread.m4
index 6d400ed..f426654 100644
--- a/m4/ax_pthread.m4
+++ b/m4/ax_pthread.m4
## -172,6 +172,12 ## for flag in $ax_pthread_flags; do
AC_MSG_CHECKING([whether pthreads work without any flags])
;;
+ -pthread)
+ AC_MSG_CHECKING([whether pthreads work with $flag])
+ PTHREAD_CFLAGS="$flag"
+ PTHREAD_LIBS="$flag"
+ ;;
+
-*)
AC_MSG_CHECKING([whether pthreads work with $flag])
PTHREAD_CFLAGS="$flag"
I guess I should submit this to the macro authors.
Expanding on a suggestion above (https://stackoverflow.com/a/20253318/221802) with exact script, this error went away for me after updating my PKbuild.sh script with pthread args:
./bootstrap && \
CPPFLAGS=" -g3 -Wall -pthread "\
CFLAGS=" -pthread -g3 -Wall "\
LDFLAGS=" -lpthread "\
./configure --enable-maintainer-mode \
--enable-debug \
--prefix=/usr \
--mandir=/usr/share/man \
--enable-pie \
--prefix=/usr \
--enable-library \
--enable-test \
......... [and so on]
I used the advice from another post: autoconf with -pthread
Here they mentioned you could download this file:
http://svn.sleuthkit.org/repos/sleuthkit/trunk/configure.ac
Place it into your m4 directory.
Then include this in your configure.ac:
ACX_PTHREAD
Finally, add this to your Makefile.am:
bin_PROGRAMS = main
main_SOURCES = main.c
main_CFLAGS = $(PTHREAD_CFLAGS)
main_LDADD = $(PTHREAD_LIBS)
I'm trying real hard to install vowpal wobbit and it fails when i run the make file, throwing:
cd library; make; cd ..
g++ -g -o ezexample temp2.cc -L ../vowpalwabbit -l vw -l allreduce -l boost_program_options -l z -l pthread
ld: library not found for -lboost_program_options collect2: ld returned 1 exit status make[1]: *** [ezexample] Error 1'
I then added the links to the boost library here by specifying -L/usr/local/lib
Now I get the following error:
g++ -g -o ezexample temp2.cc -L/usr/local/lib ../vowpalwabbit -l vw -l allreduce -l boost_program_options -l z -l pthread
ld: library not found for -lvw
collect2: ld returned 1 exit status
make: *** [ezexample] Error 1
I happened to get everything working on OS X 10.7 as follows:
Make sure you have a working Boost installation. As indicated on the Getting started page, usually we only need header files, but some Boost libraries must be built separately, including the program_options library which is used to process options from command line or config file. Go into your boost folder, and then at your shell prompt:
$ ./bootstrap.sh
$ ./bjam
This will compile and build everything. You should now have a bin.v2/ directory in your boost directory, with all built libraries for your system (static and threaded libs).
$ ls bin.v2/libs/
date_time iostreams python serialization test
filesystem math random signals thread
graph program_options regex system wave
More importantly, extra Boost libraries are made available in the stage/lib/ directory. For me, these are Mach-O 64-bit dynamically linked shared library x86_64.
The include path should be your_install_dir/boost_x_xx_x, where boost_x_xx_x is the basename of your working Boost. (I personally have boost_1_46_1 in /usr/local/share/ and I symlinked it to /usr/local/share/boost to avoid having to remember version number.) The library path (for linking) should read your_install_dir/boost_x_xx_x/stage/lib. However, it might be best to symlink or copy (which is what I did) everything in usual place, i.e. /usr/local/include/boost for header files, and /usr/local/lib for libraries.
Edit the Makefile from the vowpal_wabbit directory, and change the include/library paths to reflect your current installation. The Makefile should look like this (first 12 lines):
COMPILER = g++
UNAME := $(shell uname)
ifeq ($(UNAME), FreeBSD)
LIBS = -l boost_program_options -l pthread -l z -l compat
BOOST_INCLUDE = /usr/local/include
BOOST_LIBRARY = /usr/local/lib
else
LIBS = -l boost_program_options -l pthread -l z
BOOST_INCLUDE = /usr/local/share/boost # change path to reflect yours
BOOST_LIBRARY = /usr/local/share/boost/stage/lib # idem
endif
Then, you are ready to compile vowpal_wabbit (make clean in case you already compiled it):
$ make
$ ./vw --version
6.1
$ make test
You can also install vowpal wabbit on OS X using brew:
brew install vowpal-wabbit
Or you can just install boost, and then install vw from the github repo.
brew install boost
For installation on CentOS 7 (6.5 perl version is too old for latest vw source code), I've found the instructions at http://wkoplitz.blogspot.be/2012/12/vowpal-wabbit-on-centos.html to work fine:
yum install zlib-devel boost-devel
yum groupinstall "Development Tools"
git clone git://github.com/JohnLangford/vowpal_wabbit.git
cd vowpal_wabbit
./autogen.sh
make
make test
Good news:
As of the latest release VowpalWabbit version 9.1.0, vw no longer relies on Boost program_options
From the release highlights:
Removal of Boost Program Options dependency
For a long time we have depended on Boost Program Options
for command line options parsing. In this release, we have > replaced this dependency with our own implementation of
command line parsing. Apart from one place where we depend > on Boost Math in standalone mode, this means that VW core
and the command line tool are free of Boost dependencies
hopefully making the code a bit easier to build and package.
Vowpal Wabbit 9.1.0 release notes