autotools for pthreads not setting correct linker flags - pthreads

I am adding some pthreads code into my Linux application that I'm building with autotools. I was getting an error about not linking in libpthreads. So I want to specify the pthreads dependency and compiler/linker flags in autotools.
I found some references that say use an ACX_PTHREAD macro. GNU provides an AX_PTHREAD macro. Both are very similar in concept. But I've tried both (on Ubuntu 13.04 64-bit), and found that they set -pthread in $PTHREAD_CFLAGS, but for some reason they don't set the -lpthread linker flag in $PTHREAD_LIBS.
The build fails. When I run make, I get:
...
/bin/sh ../libtool --tag=CXX --mode=link g++ -g -O2 -o myapp main.o ... -lconfuse -llog4cpp -lnsl -lpopt -lfuse -L/usr/local/lib -lrt
libtool: link: g++ -g -O2 -o .libs/myapp main.o ... -lconfuse -llog4cpp -lnsl /usr/lib/x86_64-linux-gnu/libpopt.so -lfuse -L/usr/local/lib -lrt
/usr/bin/ld: app-fuse.o: undefined reference to symbol 'pthread_kill##GLIBC_2.2.5'
/usr/bin/ld: note: 'pthread_kill##GLIBC_2.2.5' is defined in DSO /lib/x86_64-linux-gnu/libpthread.so.0 so try adding it to the linker command line
/lib/x86_64-linux-gnu/libpthread.so.0: could not read symbols: Invalid operation
collect2: error: ld returned 1 exit status
...
In this case, the ./configure step shows:
...
checking for the pthreads library -lpthreads... no
checking whether pthreads work without any flags... no
checking whether pthreads work with -Kthread... no
checking whether pthreads work with -kthread... no
checking for the pthreads library -llthread... no
checking whether pthreads work with -pthread... yes
checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE
checking if more special flags are required for pthreads... no
checking for PTHREAD_PRIO_INHERIT... yes
...
I notice it checks for -lpthreads, but shouldn't it be checking for -lpthread?
I've found that I can use:
AC_CHECK_LIB(pthread, pthread_create, [PTHREAD_LIBS+=-lpthread])
and then the build succeeds. But I assume this isn't the best way to make it work on the widest variety of platforms.
I see Ubuntu also has a package libpthread-stubs0-dev. But I'm not sure what it's for.
What is the "right way" to use pthreads with autotools?

Thanks to Peter Simons who asked on the autoconf mailing list, we have a somewhat official answer:
Compiler flags and linker flags are not mutually-exclusive sets, not
least because linking is typically done via the compiler frontend (cc)
and not by invoking the linker (ld) directly. Any flags that you can
use in the compile step (e.g. -O2, -DFOO, -I/tmp/include) will
generally be accepted in the linking step, even if it's not applicable
then. (The reverse may not be true, e.g. -lfoo.)
Given that, it's a lot less error-prone to use PTHREAD_CFLAGS (and
other CFLAGS variables) when linking, rather than duplicating the
applicable flags into PTHREAD_LIBS/LDFLAGS/etc. variables and not
using any CFLAGS variables then.
So just use PTHREAD_CFLAGS for your linker, too.

I bumped into this same issue when I added a first C++ source to an otherwise working C project (a shared library). Adding this C++ file caused libtool to switch from linking with gcc to linking with g++. Seems that linking with gcc a '-pthread' is enough to add the dynamic dependency to libpthread, but when linking with g++, it is not.
I tried with the above patch to a local ax_pthread.m4, but this didn't help. Passing '-lpthread' to g++ would fix the issue.
Edit: for some reason, ax_pthread.m4 forces C as the test language even if the AC_LANG is set as C++. This patch makes things work for me:
--- m4/ax_pthread.m4_orig 2013-06-15 20:03:36.000000000 +0300
+++ m4/ax_pthread.m4 2013-06-15 20:03:51.000000000 +0300
## -87,7 +87,6 ##
AU_ALIAS([ACX_PTHREAD], [AX_PTHREAD])
AC_DEFUN([AX_PTHREAD], [
AC_REQUIRE([AC_CANONICAL_HOST])
-AC_LANG_PUSH([C])
ax_pthread_ok=no
# We used to check for pthread.h first, but this fails if pthread.h
## -313,5 +312,4 ##
ax_pthread_ok=no
$2
fi
-AC_LANG_POP
])dnl AX_PTHREAD

It seems that the AX_PTHREAD macro is finding the -pthread compiler flag, and using that. But it looks as though for that particular flag, it should be specified to the linker as well (it apparently does the equivalent of -lpthread in the linker). I modified the macro as follows, so that the -pthread flag is specified as a linker flag too:
diff --git a/m4/ax_pthread.m4 b/m4/ax_pthread.m4
index 6d400ed..f426654 100644
--- a/m4/ax_pthread.m4
+++ b/m4/ax_pthread.m4
## -172,6 +172,12 ## for flag in $ax_pthread_flags; do
AC_MSG_CHECKING([whether pthreads work without any flags])
;;
+ -pthread)
+ AC_MSG_CHECKING([whether pthreads work with $flag])
+ PTHREAD_CFLAGS="$flag"
+ PTHREAD_LIBS="$flag"
+ ;;
+
-*)
AC_MSG_CHECKING([whether pthreads work with $flag])
PTHREAD_CFLAGS="$flag"
I guess I should submit this to the macro authors.

Expanding on a suggestion above (https://stackoverflow.com/a/20253318/221802) with exact script, this error went away for me after updating my PKbuild.sh script with pthread args:
./bootstrap && \
CPPFLAGS=" -g3 -Wall -pthread "\
CFLAGS=" -pthread -g3 -Wall "\
LDFLAGS=" -lpthread "\
./configure --enable-maintainer-mode \
--enable-debug \
--prefix=/usr \
--mandir=/usr/share/man \
--enable-pie \
--prefix=/usr \
--enable-library \
--enable-test \
......... [and so on]

I used the advice from another post: autoconf with -pthread
Here they mentioned you could download this file:
http://svn.sleuthkit.org/repos/sleuthkit/trunk/configure.ac
Place it into your m4 directory.
Then include this in your configure.ac:
ACX_PTHREAD
Finally, add this to your Makefile.am:
bin_PROGRAMS = main
main_SOURCES = main.c
main_CFLAGS = $(PTHREAD_CFLAGS)
main_LDADD = $(PTHREAD_LIBS)

Related

Linking Boost static libraries

I am trying to compile a shared library using static libraries from Boost and OpenCV. Here below the command I am using to compile my library.
g++ -fPIC libsaliency.cpp -shared -o libsaliency.so \
-I/home/poiesi/data/libraries/boost_1_66_0/installed_w_contrib_static/include -I/home/poiesi/data/libraries/opencv-3.4.0/installed_w_contrib_static/include \
-Wl,--whole-archive \
/home/poiesi/data/libraries/boost_1_66_0/installed/lib/libboost_graph.a \
/home/poiesi/data/libraries/boost_1_66_0/installed/lib/libboost_filesystem.a \
/home/poiesi/data/libraries/boost_1_66_0/installed/lib/libboost_system.a \
/home/poiesi/data/libraries/opencv-3.4.0/installed_w_contrib_static/lib/libopencv_core.a \
/home/poiesi/data/libraries/opencv-3.4.0/installed_w_contrib_static/lib/libopencv_highgui.a \
/home/poiesi/data/libraries/opencv-3.4.0/installed_w_contrib_static/lib/libopencv_imgproc.a \
/home/poiesi/data/libraries/opencv-3.4.0/installed_w_contrib_static/lib/libopencv_imgcodecs.a \
/home/poiesi/data/libraries/opencv-3.4.0/installed_w_contrib_static/lib/libopencv_features2d.a \
/home/poiesi/data/libraries/opencv-3.4.0/installed_w_contrib_static/lib/libopencv_video.a \
-Wl,--no-whole-archive
However, I have this error:
usr/bin/ld: /home/poiesi/data/libraries/boost_1_66_0/installed/lib/libboost_graph.a(read_graphviz_new.o): relocation R_X86_64_32 against `.rodata.str1.8' can not be used when making a shared object; recompile with -fPIC
/home/poiesi/data/libraries/boost_1_66_0/installed/lib/libboost_graph.a(read_graphviz_new.o): error adding symbols: Bad value
collect2: error: ld returned 1 exit status
Makefile:7: recipe for target 'saliency' failed
make: *** [saliency] Error 1
Does this mean I have to recompile Boost using -fPIC command? I checked this online but I haven't found much info about it. This makes me wonder if I am searching for the right thing. Do you have any suggestion?
EDIT: As suggested below by Mike, I recompiled Boost like this:
./b2 cxxflags="-fPIC" link=static install
and I can now compile my .so library.
Does this mean I have to recompile Boost using -fPIC command?
Yes. All code that is linked into a shared library must be Position Independent Code. Object
files within static libraries normally are not, as shared libraries normally
link other shared libraries.
But there is nothing in principle to stop you from building boost static libraries
from -fPIC-compiled object files.
It would be simpler, of course, to link the shared versions of the boost libraries.
With boost 1.72.0, fixed this problem by recompile boost static library with -fPIC.
./bootstrap.sh --prefix=/usr/
sudo ./b2 cxxflags=-fPIC cflags=-fPIC link=static -a
sudo ./b2 install

AM_PROG_LEX and undefined yywrap

My program uses the documented autoconf macro AM_PROG_LEX. It builds fine on RHEL 6.5 and other distros, but fails on RHEL 6.6 and later.
The configure script cannot compile its tests. When it tries gcc with -ll, -lfl, linking fails with:
/usr/bin/ld: cannot find -lfl
When it tries gcc without extra libraries, linking fails with:
undefined reference to `yywrap'
libfl.a or libfl.so is missing from official repos of those systems. On RHEL 6.5 it's part of flex package.
RHEL 6.5
configure:5334: checking whether yytext is a pointer
configure:5351: gcc -o conftest -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -O0 conftest.c -lfl >&5
configure:5351: $? = 0
configure:5359: result: yes
RHEL 6.8
configure:5196: checking whether yytext is a pointer
configure:5217: gcc -o conftest -g -O2 conftest.c >&5
/tmp/ccNJtVgv.o: In function `input':
/home/git/rpmbuild/BUILD/snacc-1.3.1_16_g23ba7a6/lex.yy.c:1168: undefined reference to `yywrap'
/tmp/ccNJtVgv.o: In function `yylex':
/home/git/rpmbuild/BUILD/snacc-1.3.1_16_g23ba7a6/lex.yy.c:867: undefined reference to `yywrap'
/tmp/ccNJtVgv.o: In function `main':
/home/git/rpmbuild/BUILD/snacc-1.3.1_16_g23ba7a6/conftest.l:17: undefined reference to `yywrap'
collect2: ld returned 1 exit status
configure:5224: $? = 1
configure: failed program was:
...
configure:5246: result: no
libfl contains two and only two functions, both of which are normally unnecessary in production use of flex:
int main() { extern int yylex(void); while (yylex()) ; return 0; }
int yywrap(void) { return 1; }
The yywrap implementation (which essentially disables the yywrap functionality) is not necessary if you use the option
%option noyywrap
in your flex definition, or if you pass the command-line option --noyywrap to flex.
For quick-and-dirty flex scanners, or for debugging, it is sometimes handy to be able to use libfl to fill in the above functions. But it also can create problems on systems which provide both 32- and 64-bit environments. For this reason, libfl was removed from the RHEL flex rpm in 2014. See this RedHat bug fix advisory for details.
So you could install the appropriate flex-devel rpm in order to have libfl available. Or you could compile it yourself using the above code (which is not precisely the source code you'll find in the flex source bundle, but should produce precisely the same library).
Or you could try to fix autoconf so that it doesn't depend on libfl. It didn't used to have any such dependency; if it couldn't find libfl, it would just assume that it wasn't required for the program being compiled.
Workaround is to install flex-devel package containing libfl.a. RHEL version available to subscribers ony. Alternative is CentOS package or recompiling from source.

OpenCV with CMake version 3.5.2 vs CMake 2.X.X

I am at a loss to solve a particular issue I have: I cannot pinpoint the culprit.
System: Jetson TX1, arm64 kernel, 32b userspace, opencv4tegra
Situation: Have been building projects using:
find_package( OpenCV )
And this has worked fine and compiled.
Fault: I built from source and installed CMake 3.5.2. Now I can no-longer build any projects that depend on OpenCV. I get linker errors that point cannot find:
opencv_dep_cudart
I am assuming the issues are caused in OpenCVCConfig.cmake, around this point:
# Import target "opencv_core" for configuration "Release"
set_property(TARGET opencv_core APPEND PROPERTY IMPORTED_CONFIGURATIONS RELEASE)
set_target_properties(opencv_core PROPERTIES
IMPORTED_LINK_INTERFACE_LIBRARIES_RELEASE "opencv_dep_cudart;opencv_dep_nppc;opencv_dep_nppi;opencv_dep_npps;dl;m;pthread;rt;tbb"
IMPORTED_LOCATION_RELEASE "${_IMPORT_PREFIX}/lib/libopencv_core.so.2.4.12"
IMPORTED_SONAME_RELEASE "libopencv_core.so.2.4"
)
Out of the file: /usr/share/OpenCV/OpenCVModules-release.cmake
However, this file doesn't change between CMake versions as it is an OpenCV file. So this must be how it is processed.
Reverting my CMake back to 2.8.12.2 manually allowed me to build again. Here is an example of a make command that uses OpenCV. Note the correct cuda libs:
Linking CXX executable DuoInterfaceTest
/usr/local/bin/cmake -E cmake_link_script CMakeFiles/DuoInterfaceTest.dir/link.txt --verbose=1
/usr/bin/c++ -O2 -g -DNDEBUG -std=gnu++11 CMakeFiles/DuoInterfaceTest.dir/src/mainTest.cpp.o -o DuoInterfaceTest -L/home/ubuntu/catkin_ws/duointerface/lib/linux/arm -rdynamic libDuoInterface.a /usr/lib/libopencv_vstab.so.2.4.12 /usr/lib/libopencv_tegra.so.2.4.12 /usr/lib/libopencv_imuvstab.so.2.4.12 /usr/lib/libopencv_facedetect.so.2.4.12 /usr/lib/libopencv_esm_panorama.so.2.4.12 /usr/lib/libopencv_detection_based_tracker.so.2.4.12 /usr/lib/libopencv_videostab.so.2.4.12 /usr/lib/libopencv_video.so.2.4.12 /usr/lib/libopencv_ts.a /usr/lib/libopencv_superres.so.2.4.12 /usr/lib/libopencv_stitching.so.2.4.12 /usr/lib/libopencv_photo.so.2.4.12 /usr/lib/libopencv_objdetect.so.2.4.12 /usr/lib/libopencv_ml.so.2.4.12 /usr/lib/libopencv_legacy.so.2.4.12 /usr/lib/libopencv_imgproc.so.2.4.12 /usr/lib/libopencv_highgui.so.2.4.12 /usr/lib/libopencv_gpu.so.2.4.12 /usr/lib/libopencv_flann.so.2.4.12 /usr/lib/libopencv_features2d.so.2.4.12 /usr/lib/libopencv_core.so.2.4.12 /usr/lib/libopencv_contrib.so.2.4.12 /usr/lib/libopencv_calib3d.so.2.4.12 /usr/lib/libopencv_tegra.so.2.4.12 /usr/lib/libopencv_stitching.so.2.4.12 /usr/lib/libopencv_gpu.so.2.4.12 /usr/lib/libopencv_photo.so.2.4.12 /usr/lib/libopencv_legacy.so.2.4.12 /usr/local/cuda-7.0/lib/libcufft.so /usr/lib/libopencv_video.so.2.4.12 /usr/lib/libopencv_objdetect.so.2.4.12 /usr/lib/libopencv_ml.so.2.4.12 /usr/lib/libopencv_calib3d.so.2.4.12 /usr/lib/libopencv_features2d.so.2.4.12 /usr/lib/libopencv_highgui.so.2.4.12 /usr/lib/libopencv_imgproc.so.2.4.12 /usr/lib/libopencv_flann.so.2.4.12 /usr/lib/libopencv_core.so.2.4.12 /usr/local/cuda-7.0/lib/libcudart.so /usr/local/cuda-7.0/lib/libnppc.so /usr/local/cuda-7.0/lib/libnppi.so /usr/local/cuda-7.0/lib/libnpps.so -ldl -lm -lpthread -lrt -ltbb -lDUO -Wl,-rpath,/home/ubuntu/catkin_ws/duointerface/lib/linux/arm:/usr/local/cuda-7.0/lib
Thoughts? I would like to be able to keep the newer CMake on my system but don't understand enough to fix the problem. If you think this is too system-unique I will withdraw the question.
As noted by Michael Mairegger, you have to cmake in the build directory by doing
sudo cmake .. -DCUDA_USE_STATIC_CUDA_RUNTIME=false
But one extra thing I noticed is that if I try to make afterwards it won't work unless I do the cmake command twice.

How to use coverage analysis with Cython

I'm trying to run coverage analysis on some Cython code using pytest-cov and coveralls.io. I've got as far as building the extension modules with tracing enabled, and running the analysis with the help of the links below:
http://docs.cython.org/src/tutorial/profiling_tutorial.html
http://blog.behnel.de/posts/coverage-analysis-for-cython-modules.html
However, I'm getting some results that I can't explain. It seems that many of the def/cdef/cpdef lines in the code are showing as not running, despite code within them being OK. The results aren't even consistent as some lines seem OK.
Example report: https://coveralls.io/files/1871744040
I don't know if I'm calling something wrong, if this is a bug, or if I'm just not interpreting the results correctly.
In the example above, the get_cost method seems OK, but the __set__ method for the property above is not called, despite the lines within the function having been called.
Update: It seems the issue is with Cython classes. If the class is defined with def rather than cdef the problem goes away. I guess there isn't full support for this yet.
If the Cython tracing facility does not seem to work as intended, it should be possible to use gcov for the coverage analysis of cython code. This way one could verify if some line of the generated C code is executed or not.
With a simple main.pyx
import mymod
def main():
mymod.test()
and mymod.pyx
def test():
return 42
and then
cython --embed main.pyx
cython mymod.pyx
gcc -O1 -fPIC -fprofile-arcs -ftest-coverage -Wall -I/usr/include/python2.7 -c -o main.o main.c
gcc main.o -fprofile-arcs -lpython2.7 -lgcov -o main
gcc -O1 -fPIC -fprofile-arcs -ftest-coverage -Wall -I/usr/include/python2.7 -c -o mymod.o mymod.c
gcc -shared mymod.o -fprofile-arcs -lgcov -lpython2.7 -o mymod.so
an executable was created. After executing ./main main.gcda and mymod.gcda were created for gcov.

How to add directories to ld search path for a cross-compilation to ARM?

I am trying to configure util-linux to cross compile using arm-none-linux-gnueabi from CodeSourcery. My only problem so far is that it can't find my ncurses library which I compiled.
How can I add a directory to the ld search path? I've tried adding to my LIBRARY_PATH and LD_LIBRARY_PATH variables, but neither does anything. I know that I can add the -L flag to gcc and it will add to the linker path, but is there any way to do this globally, so that I can do it once, and not have to worry about it again?
Here is the output of arm-none-linux-gnueabi-gcc -print-search-dirs | grep libraries | sed 's/:/\n/g':
libraries
=/tools/bin/../lib/gcc/arm-none-linux-gnueabi/4.6.1/
/tools/bin/../lib/gcc/
/tools/bin/../lib/gcc/arm-none-linux-gnueabi/4.6.1/../../../../arm-none-linux-gnueabi/lib/arm-none-linux-gnueabi/4.6.1/
/tools/bin/../lib/gcc/arm-none-linux-gnueabi/4.6.1/../../../../arm-none-linux-gnueabi/lib/
/tools/bin/../arm-none-linux-gnueabi/libc/lib/arm-none-linux-gnueabi/4.6.1/
/tools/bin/../arm-none-linux-gnueabi/libc/lib/
/tools/bin/../arm-none-linux-gnueabi/libc/usr/lib/arm-none-linux-gnueabi/4.6.1/
/tools/bin/../arm-none-linux-gnueabi/libc/usr/lib/
I would like to add /arm/usr/lib and /arm/usr/local/lib to my ld search path.
If you need output from any other commands, just ask!
EDIT: I just found out about the CFLAGS environment variable--do all configure scripts/makefiles honor it?
Thank you!
If the ncurses library you compiled are going to be linked to the ARM binary you are cross-compiling you can not use LD_LIBRARY_PATH! LD_LIBRARY_PATH is only used by the current run-time and is in no way used by the compiler or linker when building your application.
The use of CFLAGS depends on creator of Makefile. CFLAGS are not automatically used even if they are defined as an environment variable. Only tools like the autoconf tools can pick them up from the environment and use them automagically. In the Makefiles find something like:
$(CC) $(CFLAGS) ....
if this fragment exists then the Makefile uses the CFLAGS variable. LDFLAGS is the more appropriate environment variable to use for link-time options.

Resources