Linking Error with ROS PCL and Drake: Different VTK versions - ros

I have been facing linking issues when including various ROS packages and Drake as libraries, and have narrowed the issue down to VTK.
It seems that ROS(noetic) PCL depends on VTK 7.1.0 whereas Drake depends on VTK 9.1.0 after running ldd:
linux-vdso.so.1 (0x00007fffa88ea000)
libdrake.so => /opt/drake/lib/libdrake.so (0x00007f644589d000)
libpcl_io.so.1.10 => /lib/x86_64-linux-gnu/libpcl_io.so.1.10 (0x00007f64455d7000)
libvtkCommonCore-7.1.so.7.1p => /lib/x86_64-linux-gnu/libvtkCommonCore-7.1.so.7.1p (0x00007f644528b000)
libvtkFiltersGeneral-7.1.so.7.1p => /lib/x86_64-linux-gnu/libvtkFiltersGeneral-7.1.so.7.1p (0x00007f6444f41000)
libvtkRenderingCore-7.1.so.7.1p => /lib/x86_64-linux-gnu/libvtkRenderingCore-7.1.so.7.1p (0x00007f6444c30000)
libvtkRenderingOpenGL2-7.1.so.7.1p => /lib/x86_64-linux-gnu/libvtkRenderingOpenGL2-7.1.so.7.1p (0x00007f6444907000)
libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f6444725000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f6444533000)
libdrake_lcm.so => /opt/drake/lib/libdrake_lcm.so (0x00007f644451a000)
libdrake_marker.so => /opt/drake/lib/libdrake_marker.so (0x00007f6444517000)
libvtksys-9.1.so.1 => /opt/drake/lib/libvtksys-9.1.so.1 (0x00007f64444c2000)
libvtkfmt-9.1.so.1 => /opt/drake/lib/libvtkfmt-9.1.so.1 (0x00007f644449b000)
libvtkkissfft-9.1.so.1 => /opt/drake/lib/libvtkkissfft-9.1.so.1 (0x00007f6444493000)
libvtkloguru-9.1.so.1 => /opt/drake/lib/libvtkloguru-9.1.so.1 (0x00007f6444459000)
...
My CMakeLists.txt is defined as:
cmake_minimum_required(VERSION 3.10)
project(my_project)
find_package(catkin REQUIRED COMPONENTS
roscpp
pcl_ros
)
find_package(drake CONFIG REQUIRED PATHS /opt/drake)
catkin_package(
CATKIN_DEPENDS roscpp pcl_ros
DEPENDS drake
)
include_directories(${catkin_INCLUDE_DIRS})
add_executable(my_test test.cpp)
target_link_libraries(my_test drake::drake ${catkin_LIBRARIES})
and package.xml as
....
<buildtool_depend>catkin</buildtool_depend>
<depend>roscpp</depend>
<depend>pcl_ros</depend>
</package>
Simply including vtk headers causes immediate an segmentation fault:
#include "drake/systems/framework/diagram_builder.h"
#include <vtkPolyData.h>
int main() {
drake::systems::DiagramBuilder<double> builder;
return 0;
}

That's correct -- VTK's library symbols are not version-namespaced, so you cannot load two different VTK versions into the same process, even for two completely independent uses. This is not unique to Drake; it's generally true about VTK.
The only challenge with Drake is that (1) Drake's pre-compiled binaries are built against a specific version of VTK that differs from ROS PCL's VTK, and (2) even for a Drake source build, Drake doesn't make it very easy to change which version of VTK will be linked. Unlike Eigen or spdlog or fmt, Drake's VTK dependency is not designed to be end-user-serviceable.
To avoid problems like this, Drake is actively working on linking VTK privately. Follow #16502 for updates. ETA for the fix should be within the next month or two.
In the meantime, there are not many good work-arounds.
The only easy one is to carefully link to only the parts of PCL that do not depend on VTK. For example, the PCL KDTree does not use VTK (should be safe), but the PCL IO does use VTK (will segfault). If your problem is only over-linking, then fixing your build to link fewer unnecessary things might resolve it.
The other work-around is to put a process boundary between your Drake-using code and PCL-with-VTK-using-code, with some kind of a message channel inbetween. That is, rework your code to only ever load Drake in some programs, and PCL in others, and never the two shall meet.
Hopefully, the Drake feature 16502 will be resolved soon, making this all moot.

As of tonight's nightly build (20221117 or greater) or the next stable release (v1.11.0 or newer), Drake will use Ubuntu's build of VTK 9.1 when running on Ubuntu 22.04 (aka "Jammy"). In case you can run a version of ROS on Ubuntu 22.04, hopefully it would use a compatible VTK 9.1 and PCL would work.

Related

Yocto: cmake is missing header - how to DEPEND dev-package?

I am including opencv with custom build parameters in my Yocto image. For that I have an opencv_4.1.0.bbappend recipe, in which I set custom options, specifically FFMPEG. The recipe goes something like this:
DEPENDS += "ffmpeg libpng"
EXTRA_OECMAKE_append += "-DWITH_FFMPEG=ON -DWITH_GTK=OFF" # and some other options
During configure I get cmake errors and can't seem to figure out, how to satisfy the header dependencies. The errors go like this (I assume this is the reason for do_configure to fail):
CheckIncludeFile.c:1:10: fatal error: /home/janos/dev/yocto/build/tmp/work/core2-64-poky-linux/opencv/4.1.0-r0/recipe-sysroot/usr/include/libpng/png.h: No such file or directory
1 | #include </home/janos/dev/yocto/build/tmp/work/core2-64-poky-linux/opencv/4.1.0-r0/recipe-sysroot/usr/include/libpng/png.h>
CheckIncludeFile.c:1:10: fatal error: sys/videoio.h: No such file or directory
1 | #include <sys/videoio.h>
Focusing on the missing png.h header first, I am tempted to depend libpng-dev, as I also would apt install it. But there is no package for it.
When I search oe-pkgdata-util list-pkg-files -p libpng, I can find the header in a libpng-dev package:
...
libpng-dev:
/usr/bin/libpng-config
/usr/bin/libpng16-config
/usr/include/libpng16/png.h
/usr/include/libpng16/pngconf.h
/usr/include/libpng16/pnglibconf.h
/usr/include/png.h
...
...
I can also find it in libpng-src and also ffmpeg-src package (oe-pkgdata-util find-path "*png.h" was my friend). But all of these -dev and -src packages I cannot depend on in DEPENDS.
How can I get my recipe to know those headers?
Target machine is raspberrypi4-64, on which the recipe is configuring and compiling well - it fails when I build for qemux86-64, which I use for testing. Namely, my test command is MACHINE="qemux86-64" bitbake opencv.
It doesn't really answer the question which I though was the question - but this is how the opencv recipe is easily configured:
PACKAGECONFIG = "python3 libav libv4l v4l"
Looking into the opencv 4.1.0 recipe (opencv_4.1.0.bb), I could see that FFMPEG gets enabled with the libav configurable option.
As a result of depending FFMEPG, I had to whitelist "commercial" licenses in my local.conf file:
LICENSE_FLAGS_WHITELIST = "commercial"
Looking into ./build/tmp/work/aarch64-poky-linux/opencv/4.1.0-r0/temp/log.do_configure shows that opencv is correctly configured without GUI, with v4l/v4l2:, FFMPEG, python3, etc.
And so python3 in the resulting image:
import cv2
print(cv2.getBuildInformation())

OpenCV test failed in Yocto-Jethro

I have added meta-openembedded recipes in the Yocto project, at Jethro branch OpenCV version is 2.4.
After building while testing the sample applications provided by opencv-samples_2.4.bb gives these samples :
adaptiveskindetector contours facedetect find_obj_calonder morphology mushroom pyramid_segmentation
bgfg_codebook convert_cascade fback_c find_obj_ferns motempl one_way_sample smiledetect
blobtrack_sample delaunay find_obj latentsvmdetect mser_sample polar_transforms tree_engine
but while testing I got this error :
OpenCV Error: Unspecified error (The function is not implemented.
Rebuild the library with Windows, GTK+ 2.x or Carbon support. If you
are on Ubuntu or Debian, install libgtk2.0-dev and pkg-config, then
re-run cmake or configure script) in cvNamedWindow, file
/home/ymj/opencv/modules/highgui/src/window.cpp, line 516
I have added in local.conf:
IMAGE_INSTALL_append += " opencv opencv-samples python-opencv"
Is anything more needed to get full-fledged opencv operations.

gtk/gtk.h: No such file or directory

In opencv compilation, we need gtk. I have gtk-2.0 and gtk-3.0 installed.
This command gave me
dpkg -l libgtk2.0-0 libgtk-3-0
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-==============-============-============-=================================
ii libgtk-3-0:amd 3.10.8-0ubun amd64 GTK+ graphical user interface lib
ii libgtk2.0-0:am 2.24.23-0ubu amd64 GTK+ graphical user interface lib
ii libgtk2.0-0:i3 2.24.23-0ubu i386 GTK+ graphical user interface lib
I have gtk-2.0 and gtk-3.0 folder inside /usr/include/.
They exist as /usr/include/gtk-2.0 and /usr/include/gtk-3.0
But my compilation for opencv gave me error as
src/window_gtk.cpp:48:21: fatal error: gtk/gtk.h: No such file or directory
#include "gtk/gtk.h"
^
compilation terminated.
I have gtk/gtk.h inside both gtk-2.0 and gtk-3.0 folders.
What could be wrong?
First, use brackets instead of double quotes searches in your include if that's not a file that belongs to your project, but a third-party one.
We also need to know which version of opencv you're trying to compile. OpenCV 2.x support GTK+ 2 only, whereas OpenCV 3.x support GTK+ 3.
We also need how you call cmake when you configure the OpenCV project, so please add the command line and the logs of that call too.
This will help determine the arguments given to the compiler, to check it uses the right include locations, and also check that it's trying to use the GTK+ backend and not the Qt one for example.

OpenCV won't build with CUDA even though WITH_CUDA=ON in CMake

I have tried to build OpenCV 3.1 using CMake (the gui version) to enable Cuda. I have installed Cuda version 7.5 64-bit and CMake automatically found the correct path to the Cuda toolkit. I made sure that the WITH_CUDA value was set to ON, and pressed configure. This is what I got concerning Cuda:
CUDA detected: 7.5
CUDA NVCC target flags: -gencode;arch=compute_20,code=sm_20;-gencode;arch=compute_20,code=sm_21;-gencode;arch=compute_30,code=sm_30;-gencode;arch=compute_35,code=sm_35;-gencode;arch=compute_30,code=compute_30
...
Extra dependencies: comctl32 gdi32 ole32 setupapi ws2_32 vfw32 cudart nppc nppi npps cufft -LC:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v7.5/lib/x64
...
Other third-party libraries:
Use IPP: 9.0.1 [9.0.1]
at: C:/OpenCV-3.1.0/opencv/sources/3rdparty/ippicv/unpack/ippicv_win
Use IPP Async: NO
Use Eigen: NO
Use Cuda: YES (ver 7.5)
Use OpenCL: YES
Use custom HAL: NO
NVIDIA CUDA
Use CUFFT: YES
Use CUBLAS: NO
USE NVCUVID: NO
NVIDIA GPU arch: 20 21 30 35
NVIDIA PTX archs: 30
Use fast math: YES
Then I generate using Visual Studio 12 2013 Win64.
Next I open the newly generated OpenCV.sln project in Visual Studio 2013 and build the project. It completes without any errors, but 103 warnings like this:
LINK : warning LNK4044: unrecognized option '/LC:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v7.5/lib/x64'; ignored
OpenCV builds just fine and I can build programs with it. I can also include "opencv2/core/cuda.hpp" without any problem. However, when I try to use cuda::getDevice() i get this error:
OpenCV Error: No CUDA support (The library is compiled without CUDA support) in throw_no_cuda, file C:\builds\master_PackSlave-win64-vc12-shared\opencv\modules\core\include\opencv2/core/private.cuda.hpp, line 97
When I print the information from cv::getBuildInformation() i get:
Other third-party libraries:
Use IPP: 9.0.1 [9.0.1]
at: C:\builds\master_PackSlave-win64-vc12-shared\opencv\3rdparty/ippicv/unpack/ippicv_win
Use IPP Async: NO
Use Eigen: NO
Use Cuda: NO
Use OpenCL: YES
Use custom HAL: NO
It seems Cuda is disabled somewhere in the process, but I can't figure out why. I have tried to reconfigure and rebuild several times with the same results. Would love some help on this!
The problem is CMake: it generates wrong link options for Visual Studio.
LINK : warning LNK4044: unrecognized option '/LC:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v7.5/lib/x64'; ignored
It should be "LIBPATH:C:" instead of "LC:".
To fix the problem:
Generate the solution with CMake
Go to the build directory
Open an IDE/Text editor which is able to do a global search in this directory
For each occurrence of "-LC:" in any file, replace it by "-LIBPATH:C:".
========= EDIT =========
I found the problem, it is in CMakeLists.txt :
foreach(p ${CUDA_LIBS_PATH})
set(OPENCV_LINKER_LIBS ${OPENCV_LINKER_LIBS} -L${p})
endforeach()
Instead of using "${CMAKE_LIBRARY_PATH_FLAG}" which automatically put -LIBPATH, someone put "-L"...
So to fix it:
Open CMakeLists.txt
Replace "-L" by "${CMAKE_LIBRARY_PATH_FLAG}"
Configure & Generate the solution with CMake and compile with VS.
Hope it will help!
You can successfully compile OpenCV with Cuda after manually editing CMakeLists.txt as Dubrzr say.
Also You will have the same result if you download and compile OpenCV straight from github. It is already fixed.

How to compile OpenCV statically linked with libstdc++

I need to compile OpenCV statically linked with libstdc++ to avoid problems of different dll versions of libstdc++-6.dll needed by Qt5 and OpenCV. Following the steps of this article: http://www.argong.com/docs/how-to-OpenCV-2.2.0.pdf and adding the lines below to CMakeLists.txt i expected to get the OpenCV DLLs statically linked with libstdc++, but the OpenCV continues dependent of the libstdc++-6.dll. What i'm doing wrong to get the OpenCV libraries statically linked with libstdc++?
if (MINGW)
set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -static-libgcc")
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -static-libgcc -static-libstdc++")
set(CMAKE_SHARED_LIBRARY_LINK_C_FLAGS "${CMAKE_SHARED_LIBRARY_LINK_C_FLAGS} -static-libgcc -s")
set(CMAKE_SHARED_LIBRARY_LINK_CXX_FLAGS "${CMAKE_SHARED_LIBRARY_LINK_CXX_FLAGS} -static-libgcc -static-libstdc++ -s")
endif()
First ensure you're not pulling dependencies on libraries that link on shared system libraries. You can for example enable compilation of bundled source components like zlib, jpeg, etc., with flags BUILD_ZLIB, BUILD_JPEG, etc. You can also disable dependencies on optional components like Vtk (WITH_VTK flag) and others that may be already present in the system as shared libraries. Then, if you are building OpenCV as a shared library, push the setting on the shared linker flags variable (CMAKE_SHARED_LINKER_FLAGS) in the cache with cmake -C command:
set(CMAKE_SHARED_LINKER_FLAGS "-static -static-libgcc -static-libstdc++" CACHE INTERNAL "" FORCE)
Alternatively, if you are compiling OpenCV statically with -DBUILD_SHARED_LIBS=FALSE, you can plug gcc linker flags for static linking on your final shared object or executable.

Resources