AM_PROG_LEX and undefined yywrap - flex-lexer

My program uses the documented autoconf macro AM_PROG_LEX. It builds fine on RHEL 6.5 and other distros, but fails on RHEL 6.6 and later.
The configure script cannot compile its tests. When it tries gcc with -ll, -lfl, linking fails with:
/usr/bin/ld: cannot find -lfl
When it tries gcc without extra libraries, linking fails with:
undefined reference to `yywrap'
libfl.a or libfl.so is missing from official repos of those systems. On RHEL 6.5 it's part of flex package.
RHEL 6.5
configure:5334: checking whether yytext is a pointer
configure:5351: gcc -o conftest -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -O0 conftest.c -lfl >&5
configure:5351: $? = 0
configure:5359: result: yes
RHEL 6.8
configure:5196: checking whether yytext is a pointer
configure:5217: gcc -o conftest -g -O2 conftest.c >&5
/tmp/ccNJtVgv.o: In function `input':
/home/git/rpmbuild/BUILD/snacc-1.3.1_16_g23ba7a6/lex.yy.c:1168: undefined reference to `yywrap'
/tmp/ccNJtVgv.o: In function `yylex':
/home/git/rpmbuild/BUILD/snacc-1.3.1_16_g23ba7a6/lex.yy.c:867: undefined reference to `yywrap'
/tmp/ccNJtVgv.o: In function `main':
/home/git/rpmbuild/BUILD/snacc-1.3.1_16_g23ba7a6/conftest.l:17: undefined reference to `yywrap'
collect2: ld returned 1 exit status
configure:5224: $? = 1
configure: failed program was:
...
configure:5246: result: no

libfl contains two and only two functions, both of which are normally unnecessary in production use of flex:
int main() { extern int yylex(void); while (yylex()) ; return 0; }
int yywrap(void) { return 1; }
The yywrap implementation (which essentially disables the yywrap functionality) is not necessary if you use the option
%option noyywrap
in your flex definition, or if you pass the command-line option --noyywrap to flex.
For quick-and-dirty flex scanners, or for debugging, it is sometimes handy to be able to use libfl to fill in the above functions. But it also can create problems on systems which provide both 32- and 64-bit environments. For this reason, libfl was removed from the RHEL flex rpm in 2014. See this RedHat bug fix advisory for details.
So you could install the appropriate flex-devel rpm in order to have libfl available. Or you could compile it yourself using the above code (which is not precisely the source code you'll find in the flex source bundle, but should produce precisely the same library).
Or you could try to fix autoconf so that it doesn't depend on libfl. It didn't used to have any such dependency; if it couldn't find libfl, it would just assume that it wasn't required for the program being compiled.

Workaround is to install flex-devel package containing libfl.a. RHEL version available to subscribers ony. Alternative is CentOS package or recompiling from source.

Related

How to use the installed (pre-compiled) drake as external with bazel?

I am working on a C++ project with drake, using bazel as the build system. Previously, I use the drake source code as the external, following the drake_bazel_external example. Everything works fine.
Since I want to use the SNOPT solver in drake, I want to change to use the pre-compiled drake. I follow the drake_bazel_installed example. However, I got the following errors.
Compiling kuka/diffIK_controller.cc failed: (Exit 1): gcc failed: error executing command /usr/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer -g0 -O2 '-D_FORTIFY_SOURCE=1' -DNDEBUG -ffunction-sections ... (remaining 27 arguments skipped)
Use --sandbox_debug to see verbose messages from the sandbox
In file included from bazel-out/k8-opt/bin/external/drake/_virtual_includes/.drake_headers/drake/common/default_scalars.h:3,
from bazel-out/k8-opt/bin/external/drake/_virtual_includes/.drake_headers/drake/systems/framework/leaf_system.h:14,
from ./kuka/diffIK_controller.h:3,
from kuka/diffIK_controller.cc:3:
bazel-out/k8-opt/bin/external/drake/_virtual_includes/.drake_headers/drake/common/autodiff.h:12:10: fatal error: Eigen/Core: No such file or directory
12 | #include <Eigen/Core>
| ^~~~~~~~~~~~
compilation terminated.
I also find that the apps in the drake_bazel_external cannot be compiled successfully by drake_bazel_installed setting. The error message is
ERROR: error loading package 'app': Label '#drake//tools/skylark:py.bzl' is invalid because 'tools/skylark' is not a package; perhaps you meant to put the colon here: '#drake//:tools/skylark/py.bzl'?
Update
The bug can be produced by both the http_archive fetched drake and the apt installed drake (the latest stable drake I think, since I just installed it yesterday). I have isolated the relevant code to reproduce the bug in a github repo. Currently, I can get the original apps in drake_bazel_installed to work.
Update
By adding
# solve the eigen not found bug
build --cxxopt=-I/usr/include/eigen3
to the .bazelrc file, I can solve the above problem. However, when I try to build a program that uses iiwa_status_receiver.h, I encounter a new problem.
ERROR: /home/chenwang/repro_drake_bazel_external/drake_bazel_installed/apps/BUILD.bazel:102:10: Compiling apps/connection_test.cc failed: (Exit 1): gcc failed: error executing command /usr/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer -g0 -O2 '-D_FORTIFY_SOURCE=1' -DNDEBUG -ffunction-sections ... (remaining 32 arguments skipped)
Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
In file included from apps/connection_test.cc:10:
bazel-out/k8-opt/bin/external/drake/_virtual_includes/.drake_headers/drake/manipulation/kuka_iiwa/iiwa_status_receiver.h:6:10: fatal error: drake/lcmt_iiwa_status.hpp: No such file or directory
6 | #include "drake/lcmt_iiwa_status.hpp"
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
INFO: Elapsed time: 2.967s, Critical Path: 0.24s
INFO: 2 processes: 2 internal.
FAILED: Build did NOT complete successfully
This problem is also a missing header file problem. I have update the github repo to reproduce this problem.
This is a bug in Drake (filed as https://github.com/RobotLocomotion/drake/issues/17965 now).
To work around it, pass --cxxopt=-I/usr/include/eigen3 on all of your bazel commands, e.g., by adding this line to your projects' .bazelrc file:
build --cxxopt=-I/usr/include/eigen3
Edit: The nightly builds of Apt packages as of 20220923 should have this fixed as well.

How to exclude old version library in /usr/local?

Disclamer: using -nostdcincl isn't possible because it excludes needed system libraries. Here instead the problem seems to be that tthe compiler ignores my -I directives
I have installed a library (OpenCV) in ~/local on a remote machine, since I don't have sudo access there. Notice that an older version of the same library is installed in /usr/local.
I'm trying to compile this code:
g++ -DCC_DISABLE_CUDA -I/home/spm1428/CloudCache -I/home/spm1428/local/include/opencv -I/home/spm1428/local/include/opencv2 -I/usr/include/boost -I/home/spm1428/vlfeat -O3 -g -Wall -c -fopenmp -std=c++11 -c -o Descriptor.o ../Descriptors/Descriptor.cpp
However, the returned error is:
In file included from /usr/local/include/opencv2/opencv.hpp:77:0,
from /home/spm1428/CloudCache/Utilities/Utility.hpp:11,
from ../Descriptors/Descriptor.cpp:17:
/usr/local/include/opencv2/highgui/highgui.hpp:165:25: error: redeclaration of ‘IMREAD_UNCHANGED’
IMREAD_UNCHANGED =-1,
^
In file included from ../Descriptors/Descriptor.cpp:13:0:
/home/spm1428/local/include/opencv2/imgcodecs.hpp:65:8: note: previous declaration ‘cv::ImreadModes IMREAD_UNCHANGED’
IMREAD_UNCHANGED = -1, //!< If set, return the loaded image as is (with alpha channel,
^
In file included from /usr/local/include/opencv2/opencv.hpp:77:0,
from /home/spm1428/CloudCache/Utilities/Utility.hpp:11,
from ../Descriptors/Descriptor.cpp:17:
/usr/local/include/opencv2/highgui/highgui.hpp:167:24: error: redeclaration of ‘IMREAD_GRAYSCALE’
IMREAD_GRAYSCALE =0,
I think that this happens because there is another version installed. How can I solve this?
I think this error happens for the same reason (the old version doesn't have cv::xfeatures2d::SURF).

How is -fPIE passed along to llc?

I'm working on an LLVM backend for a new architecture and we need to have position independent executables. I can pass '-fPIE' on the clang command line, but I don't see any indication of this show up in the resulting LLVM IR. For example, if I run:
clang -v -emit-llvm -fPIC -O0 -S global_dat.c -o global_dat_x86_pic.ll
And then take a look at the resulting global_dat_x86_pic.ll file, I see the following near the bottom:
!0 = !{i32 1, !"PIC Level", i32 2}
Ok, makes sense.
However if I run:
clang -v -emit-llvm -fPIE -O0 -S global_dat.c -o global_dat_x86_pie.ll
I see that the two .ll files are identical. Near the bottom of global_cat_x86_pie.ll I see:
!0 = !{i32 1, !"PIC Level", i32 2}
Which is identical to the case where I ran with -fPIE. There's no indication of "PIE Level" in the .ll file. If this .ll file were passed on to llc how would llc know that -fPIE had been set on the clang command line?
I have run in gdb and see that in fact in the second case with -fPIE on the clang commandline there is an Opts.PIELevel (in $LLVM_HOME/tools/clang/lib/Frontend/CompilerInvocation.cpp) that gets set to 2 (in fact, both Opts.PIELevel and Opts.PICLevel are set to 2 in that case whereas in -fPIC is passed to clang only Opts.PICLevel is set to 2)
This depends on your default target triple, which I can't tell from your question. You can see what happens if you cut off the default architecture (or run a native clang on an arch that does support PIE)
For example, bare X86-64 shows this,
$ clang -c hello.c -target x86_64 -fPIE -emit-llvm -###
if you run that command, you'll find "-pie-level" "2" in the output, which is how llc (well, it's internal equivalent) knows about it.
The key here is that you'll have to arrange for your backend to do something with this flag. Certain platforms (like Darwin) just ignore it. If you happen to be experiment on an OSX host, you won't see -pie-level in the bogbrush output.

autotools for pthreads not setting correct linker flags

I am adding some pthreads code into my Linux application that I'm building with autotools. I was getting an error about not linking in libpthreads. So I want to specify the pthreads dependency and compiler/linker flags in autotools.
I found some references that say use an ACX_PTHREAD macro. GNU provides an AX_PTHREAD macro. Both are very similar in concept. But I've tried both (on Ubuntu 13.04 64-bit), and found that they set -pthread in $PTHREAD_CFLAGS, but for some reason they don't set the -lpthread linker flag in $PTHREAD_LIBS.
The build fails. When I run make, I get:
...
/bin/sh ../libtool --tag=CXX --mode=link g++ -g -O2 -o myapp main.o ... -lconfuse -llog4cpp -lnsl -lpopt -lfuse -L/usr/local/lib -lrt
libtool: link: g++ -g -O2 -o .libs/myapp main.o ... -lconfuse -llog4cpp -lnsl /usr/lib/x86_64-linux-gnu/libpopt.so -lfuse -L/usr/local/lib -lrt
/usr/bin/ld: app-fuse.o: undefined reference to symbol 'pthread_kill##GLIBC_2.2.5'
/usr/bin/ld: note: 'pthread_kill##GLIBC_2.2.5' is defined in DSO /lib/x86_64-linux-gnu/libpthread.so.0 so try adding it to the linker command line
/lib/x86_64-linux-gnu/libpthread.so.0: could not read symbols: Invalid operation
collect2: error: ld returned 1 exit status
...
In this case, the ./configure step shows:
...
checking for the pthreads library -lpthreads... no
checking whether pthreads work without any flags... no
checking whether pthreads work with -Kthread... no
checking whether pthreads work with -kthread... no
checking for the pthreads library -llthread... no
checking whether pthreads work with -pthread... yes
checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE
checking if more special flags are required for pthreads... no
checking for PTHREAD_PRIO_INHERIT... yes
...
I notice it checks for -lpthreads, but shouldn't it be checking for -lpthread?
I've found that I can use:
AC_CHECK_LIB(pthread, pthread_create, [PTHREAD_LIBS+=-lpthread])
and then the build succeeds. But I assume this isn't the best way to make it work on the widest variety of platforms.
I see Ubuntu also has a package libpthread-stubs0-dev. But I'm not sure what it's for.
What is the "right way" to use pthreads with autotools?
Thanks to Peter Simons who asked on the autoconf mailing list, we have a somewhat official answer:
Compiler flags and linker flags are not mutually-exclusive sets, not
least because linking is typically done via the compiler frontend (cc)
and not by invoking the linker (ld) directly. Any flags that you can
use in the compile step (e.g. -O2, -DFOO, -I/tmp/include) will
generally be accepted in the linking step, even if it's not applicable
then. (The reverse may not be true, e.g. -lfoo.)
Given that, it's a lot less error-prone to use PTHREAD_CFLAGS (and
other CFLAGS variables) when linking, rather than duplicating the
applicable flags into PTHREAD_LIBS/LDFLAGS/etc. variables and not
using any CFLAGS variables then.
So just use PTHREAD_CFLAGS for your linker, too.
I bumped into this same issue when I added a first C++ source to an otherwise working C project (a shared library). Adding this C++ file caused libtool to switch from linking with gcc to linking with g++. Seems that linking with gcc a '-pthread' is enough to add the dynamic dependency to libpthread, but when linking with g++, it is not.
I tried with the above patch to a local ax_pthread.m4, but this didn't help. Passing '-lpthread' to g++ would fix the issue.
Edit: for some reason, ax_pthread.m4 forces C as the test language even if the AC_LANG is set as C++. This patch makes things work for me:
--- m4/ax_pthread.m4_orig 2013-06-15 20:03:36.000000000 +0300
+++ m4/ax_pthread.m4 2013-06-15 20:03:51.000000000 +0300
## -87,7 +87,6 ##
AU_ALIAS([ACX_PTHREAD], [AX_PTHREAD])
AC_DEFUN([AX_PTHREAD], [
AC_REQUIRE([AC_CANONICAL_HOST])
-AC_LANG_PUSH([C])
ax_pthread_ok=no
# We used to check for pthread.h first, but this fails if pthread.h
## -313,5 +312,4 ##
ax_pthread_ok=no
$2
fi
-AC_LANG_POP
])dnl AX_PTHREAD
It seems that the AX_PTHREAD macro is finding the -pthread compiler flag, and using that. But it looks as though for that particular flag, it should be specified to the linker as well (it apparently does the equivalent of -lpthread in the linker). I modified the macro as follows, so that the -pthread flag is specified as a linker flag too:
diff --git a/m4/ax_pthread.m4 b/m4/ax_pthread.m4
index 6d400ed..f426654 100644
--- a/m4/ax_pthread.m4
+++ b/m4/ax_pthread.m4
## -172,6 +172,12 ## for flag in $ax_pthread_flags; do
AC_MSG_CHECKING([whether pthreads work without any flags])
;;
+ -pthread)
+ AC_MSG_CHECKING([whether pthreads work with $flag])
+ PTHREAD_CFLAGS="$flag"
+ PTHREAD_LIBS="$flag"
+ ;;
+
-*)
AC_MSG_CHECKING([whether pthreads work with $flag])
PTHREAD_CFLAGS="$flag"
I guess I should submit this to the macro authors.
Expanding on a suggestion above (https://stackoverflow.com/a/20253318/221802) with exact script, this error went away for me after updating my PKbuild.sh script with pthread args:
./bootstrap && \
CPPFLAGS=" -g3 -Wall -pthread "\
CFLAGS=" -pthread -g3 -Wall "\
LDFLAGS=" -lpthread "\
./configure --enable-maintainer-mode \
--enable-debug \
--prefix=/usr \
--mandir=/usr/share/man \
--enable-pie \
--prefix=/usr \
--enable-library \
--enable-test \
......... [and so on]
I used the advice from another post: autoconf with -pthread
Here they mentioned you could download this file:
http://svn.sleuthkit.org/repos/sleuthkit/trunk/configure.ac
Place it into your m4 directory.
Then include this in your configure.ac:
ACX_PTHREAD
Finally, add this to your Makefile.am:
bin_PROGRAMS = main
main_SOURCES = main.c
main_CFLAGS = $(PTHREAD_CFLAGS)
main_LDADD = $(PTHREAD_LIBS)

RT5370 SH4 Cross Compile Driver Errors

I try to build 2.5.0.3 driver from file 2011_0719_RT3070_RT3370_RT5370_RT5372_Linux_STA_V2.5.0.3_DPO.bz2 to RT5370 chipset.
Install STLinux 2.4 under Ubuntu 10.04.4 x32, make under kernel linux-sh4-2.5.32.59_stm24_0211. But I wrote /opt/STM/STLinux-2.2/devkit/sources/kernel/linux-sh4-2.5.32.59_stm24_0211 in path instead of STLinux-2.4, because Makefile have some defects:
install:
ifeq ($(TARGET), LINUX)
ifneq (,$(findstring 2.4,$(LINUX_SRC)))
$(MAKE) -C $(RT28xx_DIR)/os/linux -f Makefile.4 install
else
$(MAKE) -C $(RT28xx_DIR)/os/linux -f Makefile.6 install
endif
endif
The mention in a path 2.4 build a kernel as 2.4 that in my case a mistake.
Wrote in Makefile:
PLATFORM = ST
...
LINUX_SRC = /opt/STM/STLinux-2.2/devkit/sources/kernel/linux-sh4-2.5.32.59_stm24_0211
CROSS_COMPILE = /opt/STM/STLinux-2.2/devkit/sh4/bin/sh4-linux-
In ./os/linux/config.mk wrote:
HAS_WPA_SUPPLICANT=y
HAS_NATIVE_WPA_SUPPLICANT_SUPPORT=y
CC := sh4-linux-gcc
LD := sh4-linux-ld
Build at run make command. But have the error:
script/Makefile.build:49: *** CFLAGS was changed in "/home/vitaliy/drv_src/os/linux/Makefile". Fix it to use EXTRA_CFLAGS.
Founded strings at ./os/linux/config.mk:
ifeq ($(PLATFORM),ST)
CFLAGS := -D__KERNEL__ -I$(LINUX_SRC)/include -I$(RT28xx_DIR)/include -Wall -O2 -Wundef -Wstrict-prototypes -Wno-trigraphs -Wdeclaration-after-statement -Wno-pointer-sign -fno-strict-aliasing -fno-common -fomit-frame-pointer -ffreestanding -m4-nofpu -o $(WFLAGS)
export CFLAGS
endif
And change CFLAGS to EXTRA_CFLAGS in them.
Again error:
sh4-linux-gcc: error: -pg and -fomit-frame-pointer are incompatible.
Ok. Remove flag -fomit-frame-pointer.
Again error:
error: cpu/cache.h: No such file or directory.
In string:
WFLAGS := -DAGGREGATION_SUPPORT -DPIGGYBACK_SUPPORT -DWMM_SUPPORT -DLINUX -Wall -Wstrict-prototypes -Wno-trigraphs
Remove flag -DLINUX.
Again error with unknown types (for example: ./os/linux/../../common/crypt_md5.c:638:1: error: unknown type name 'VOID' and so on with types 'UCHAR', 'ULONG' etc).
Second way to build with help of
KBUILD_NOPEDANTIC=1 make without changes in sources files of drivers.
Error in this case too:
./os/linux/../../common/crypt_md5.c:28:23: fatal error: rt_config.h: No such file or directory.
What is wrong in my building? Or can I fix sources code and build driver for SH4-platform.
Thank you!
I recently got myself the same adapter, and I was trying to cross compile for ARM, and run into the same problem.
Basically, you just have to add the include folder from the root of the driver package.
I did these modifications to get it working:
in DRIVER_DIR/Makefile, added:
PLATFORM = MYPLATFORM
All other platforms are commented out.
Later in same file:
ifeq ($(PLATFORM),MYPLATFORM)
LINUX_SRC = /DIR_TO_MY_KERNEL_SRC/freescale_mainline/linux-mainline
CROSS_COMPILE = /DIR_TO_MY_CROSS_COMPILER/arm-unknown-linux-uclibcgnueabi-
CROSS_COMPILE_INCLUDE = /DRIVER_DIR/include /*Might not be necessary*/
endif
Then in DRIVER_DIR/os/linux/config.mk, added:
ifeq ($(PLATFORM),MYPLATFORM)
EXTRA_CFLAGS := $(WFLAGS) -I$(RT28xx_DIR)/include
endif
Also, please note that in your kernel configuration you need to have couple of flags enabled:
CONFIG_WEXT_CORE=y
CONFIG_WEXT_PROC=y
CONFIG_WEXT_SPY=y
CONFIG_WEXT_PRIV=y
You can find them under Device Drivers-->Network Device Support-->Wireless LAN-->IEEE 802.11 for Host AP
I compile it just like this now:
DRIVER_DIR$ ARCH=arm make
Hope it helps!

Resources