luajit2.0.0 -- Segmentation fault: 11 - lua

I use a simple example from http://lua-users.org/wiki/SimpleLuaApiExample to make a test. The sample can be statically linked with libluajit.a with a success, but this error message occurs when you run it:
Segmentation fault: 11
I use LuaJIT-2.0.0 released at 2012-11-08. My OS is Mac OSX Lion 10.7.5.
$ uname -a
Darwin macmatoMacBook-Pro.local 11.4.2 Darwin Kernel Version 11.4.2: Thu Aug 23 16:25:48 PDT 2012; root:xnu-1699.32.7~1/RELEASE_X86_64 x86_64
The test steps:
compile luajit-2.0.0
$ cd lj2
$ ls
COPYRIGHT Makefile README doc dynasm etc src
$ make
==== Building LuaJIT 2.0.0 ====
make -C src
DYNLINK libluajit.so
LINK luajit
OK Successfully built LuaJIT
==== Successfully built LuaJIT 2.0.0 ====
$ rm src/*.so # force to use the static version: libluajit.a
$ cd ..
compile and run the sample app
Both test.c and script.lua come from here. The folder lj2 contains the source code of the above luajit-2.0.0, just compiled successfully.
$ ls
lj2 script.lua test.c
use clang compiler
$ clang -o test test.c -I./lj2/src -L./lj2/src -lluajit
$ ./test
Segmentation fault: 11
use gcc compiler
$ gcc -o test test.c -I./lj2/src -L./lj2/src -lluajit
$ ./test
Segmentation fault: 11
But if I replace lj2/src/luajit.c with test.c, it will give me a success. This is very strange. See below:
$ cd lj2
$ make clean
$ mv src/luajit.c src/luajit.c.orig
$ cp ../test.c src/luajit.c
$ make
$ cp src/luajit ../
$ cd ..
$ ./luajit
The table the script received has:
1 2
2 4
3 6
4 8
5 10
Returning data back to C
Script returned: 30

Problem resolved. There is an section which explains how to Embedding LuaJIT in this page:
http://luajit.org/install.html
If you're building a 64 bit application on OSX which links directly or indirectly against > LuaJIT, you need to link your main executable with these flags:
-pagezero_size 10000 -image_base 100000000
Also, it's recommended to rebase all (self-compiled) shared libraries which are loaded at runtime on OSX/x64 (e.g. C extension modules for Lua). See: man rebase
Now, let me test it again:
$ clang -o test test.c -O3 -I./lj2/src -L./lj2/src -lluajit -pagezero_size 10000 -image_base 100000000
$ ./test
The table the script received has:
1 2
2 4
3 6
4 8
5 10
Returning data back to C
Script returned: 30
And valgrind returns
$ valgrind ./test
bad executable (__PAGEZERO is not 4 GB)
valgrind: ./test: cannot execute binary file
That's another question.

Related

LLVM14: Profile Guided Optimization yields "Malformed Instrumentation Profile Data"

Windows 10, Ryzen 3700x, gcc 8.1.0 (Posix, SEH-enabled)
I am building clang, llvm, and compiler-rt (the PGO tools) from source. I have downloaded the clang+llvm source for 14.0.0, and built it successfully with the following:
cmake -G "MinGW Makefiles" -DLLVM_ENABLE_PROJECTS="clang;lld;compiler-rt" -DCMAKE_BUILD_TYPE=Release -DLLVM_TARGETS_TO_BUILD=X86 ../llvm
After this, I can invoke clang, and build projects, ranging from simple "Hello, World", to a much more complex one. I am able to make use of -flto, with the addition of -fuse-ld=lld.
However, if I attempt to do ANY sort of PGO building, I fail. For example, here is the minimal example to demonstrate the problem.
Andrew#Ryzen3700x MINGW64 ~/Desktop
$ cat test.c
#include <stdio.h>
int main() {
printf("Hello, World!");
return 0;
}
Andrew#Ryzen3700x MINGW64 ~/Desktop
$ clang -fprofile-instr-generate test.c
Andrew#Ryzen3700x MINGW64 ~/Desktop
$ ./a.exe
Hello, World!
Andrew#Ryzen3700x MINGW64 ~/Desktop
$ llvm-profdata merge -output=test.profdata default.profraw
warning: default.profraw: malformed instrumentation profile data
error: no profile can be merged
Andrew#Ryzen3700x MINGW64 ~/Desktop
$ llvm-profdata show default.profraw
error: default.profraw: malformed instrumentation profile data: function name is empty
I am aware of many answers to this question, and none of them seem to apply. My .profraw file is not empty.
I will note that when I installed LLVM/Clang directly (not building on my own), the PGO portions DID work. However, after many hours I could not resolve linking issues regarding -flto.

Building Drake From Source Fails With "ModuleNotFoundError: No module named 'yaml'"

I am following the instructions here: https://drake.mit.edu/from_source.html. I already ran
./setup/mac/install_prereqs.sh
in my python virtualenv (drake-venv) and it succeeded. I then managed to build and run the inclined plane example with Bazel. But trying to build some of the other examples results in errors involving YAML like this:
(drake-venv) benq:acrobot % bazel build acrobot_input --subcommands --verbose_failures --sandbox_debug
INFO: Analyzed target //examples/acrobot:acrobot_input (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
SUBCOMMAND: # //examples/acrobot:acrobot_input_codegen [action 'Action examples/acrobot/gen/acrobot_input.cc', configuration: f8bba554e4e3784a5a24e83c682b75e9b6104059526c94f74d854527a53436a6, execution platform: #local_config_platform//:host]
(cd /private/var/tmp/_bazel_benq/a35a7fa5c4830c980dbc52ab349cb0bc/execroot/drake && \
exec env - \
bazel-out/host/bin/tools/vector_gen/lcm_vector_gen '--src=examples/acrobot/acrobot_input_named_vector.yaml' '--out=bazel-out/darwin-opt/bin/examples/acrobot/gen/acrobot_input.cc' '--out=bazel-out/darwin-opt/bin/examples/acrobot/gen/acrobot_input.h' '--include_prefix=drake')
# Configuration: f8bba554e4e3784a5a24e83c682b75e9b6104059526c94f74d854527a53436a6
# Execution platform: #local_config_platform//:host
ERROR: /Users/benq/Documents/drake/examples/acrobot/BUILD.bazel:30:28: Action examples/acrobot/gen/acrobot_input.cc failed: (Exit 1): sandbox-exec failed: error executing command
(cd /private/var/tmp/_bazel_benq/a35a7fa5c4830c980dbc52ab349cb0bc/sandbox/darwin-sandbox/176/execroot/drake && \
exec env - \
TMPDIR=/var/folders/s0/tfqtn2s54135x0qzt5kxnzs00000gn/T/ \
/usr/bin/sandbox-exec -f /private/var/tmp/_bazel_benq/a35a7fa5c4830c980dbc52ab349cb0bc/sandbox/darwin-sandbox/176/sandbox.sb /var/tmp/_bazel_benq/install/ebbb2540c6000feeb8873385c487a79c/process-wrapper '--timeout=0' '--kill_delay=15' bazel-out/host/bin/tools/vector_gen/lcm_vector_gen '--src=examples/acrobot/acrobot_input_named_vector.yaml' '--out=bazel-out/darwin-opt/bin/examples/acrobot/gen/acrobot_input.cc' '--out=bazel-out/darwin-opt/bin/examples/acrobot/gen/acrobot_input.h' '--include_prefix=drake')
Traceback (most recent call last):
File "/private/var/tmp/_bazel_benq/a35a7fa5c4830c980dbc52ab349cb0bc/sandbox/darwin-sandbox/176/execroot/drake/bazel-out/host/bin/tools/vector_gen/lcm_vector_gen.runfiles/drake/tools/vector_gen/lcm_vector_gen.py", line 10, in <module>
import yaml
ModuleNotFoundError: No module named 'yaml'
Target //examples/acrobot:acrobot_input failed to build
INFO: Elapsed time: 1.059s, Critical Path: 0.58s
INFO: 5 processes: 5 internal.
FAILED: Build did NOT complete successfully
But I'm not sure why this is happening considering that importing yaml in Terminal works:
(drake-venv) benq:acrobot % which python
/Users/benq/Documents/drake/drake-venv/bin/python
(drake-venv) benq:acrobot % python --version
Python 3.9.10
(drake-venv) benq:acrobot % python -c 'import yaml'
(drake-venv) benq:acrobot %
I've already tried reinstalling PyYaml but that didn't help.
Relevant Info:
Operating System: macOS Monterey (12.3)
Architecture: x86_64
Python: Python 3.9.10
Bazel version:
% which bazel; bazel version
/usr/local/bin/bazel
Build label: 5.0.0-homebrew
Build target: bazel-out/darwin-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Tue Jan 1 00:00:00 1980 (315532800)
Build timestamp: 315532800
Build timestamp as int: 315532800
Bazel C++ compiler: Apple clang version 13.1.6 (clang-1316.0.21.2)
Git revision: 06dd087b40
The lcm_vector_gen in the error message is a code-generation tool that's run as part of the build.
It's probably not obeying your which python, but instead using the hard-coded /usr/local/bin/python3.9 from https://github.com/RobotLocomotion/drake/blob/master/tools/py_toolchain/interpreter_paths.bzl.
We don't run or test our builds within a virtual environment, so you've stumbled into a novel situation.
Possibly editing that bzl file linked above (interpreter_paths.bzl), to point MACOS_I386_INTERPRETER_PATH to your venv python (/Users/benq/Documents/drake/drake-venv/bin/python), would fix the error.

How to install llvm#13 with Homerew on macOS High Sierra 10.13.6? Got "Built target lldELF" error

Although High Sierra is no longer supported by Homebrew, but I need to install llvm#13 formula as a dependency for other formulas. So I tried to install it this way:
$ brew install llvm
...
==> Downloading https://github.com/llvm/llvm-project/releases/download/llvmorg-13.0.0/llvm-project-13.0.0.src.tar.xz
Already downloaded: /Users/username/Library/Caches/Homebrew/downloads/8fd68fc8f968137c5080826db6e58682326235960fd8469363eb27d0799978ca--llvm-project-13.0.0.src.tar.xz
...
==> Installing llvm
==> cmake -G Unix Makefiles .. -DLLVM_ENABLE_PROJECTS=clang;clang-tools-extra;lld;lldb;mlir;polly -DLLVM_ENABLE_RUNTIMES=compiler-rt;libcxx;libcxxabi;libunwind;openmp -DLLVM_POLLY_L
==> cmake --build .
...
[ 79%] Built target lldELF
make: *** [all] Error 2
An error is occurred after a long time of compilation. I also found this error in ~/Library/Logs/Homebrew/llvm/02.cmake:
/tmp/llvm-20211109-12151-m0zvtm/llvm-project-13.0.0.src/lldb/source/Host/macosx/objcxx/HostInfoMacOSX.mm:246:52: error: use of undeclared identifier 'CPU_SUBTYPE_ARM64E'
if (cputype == CPU_TYPE_ARM64 && cpusubtype == CPU_SUBTYPE_ARM64E) {
^
1 error generated.
make[2]: *** [tools/lldb/source/Host/macosx/objcxx/CMakeFiles/lldbHostMacOSXObjCXX.dir/HostInfoMacOSX.mm.o] Error 1
make[1]: *** [tools/lldb/source/Host/macosx/objcxx/CMakeFiles/lldbHostMacOSXObjCXX.dir/all] Error 2
How can I fix that compilation error?
Install llvm with debug mode enabled:
$ brew install --debug llvm
Installation process encounters with the same error mentioned in the question, but some options are provided to fix the issue. Choose option 5:
- raise
- ignore
- backtrace
- irb
- shell
Choose an action: 5
It gives a shell access to the current build directory of llvm formula. Find the current folder:
$ pwd
/private/tmp/llvm-20211109-12151-m0zvtm/llvm-project-13.0.0.src
Change the location to the build directory:
cd llvm/build
Edit the HostInfoMacOSX.mm and remove the second part of condition:
vi ../../lldb/source/Host/macosx/objcxx/HostInfoMacOSX.mm
You need to change the line 246 from:
if (cputype == CPU_TYPE_ARM64 && cpusubtype == CPU_SUBTYPE_ARM64E) {
to:
if (cputype == CPU_TYPE_ARM64) {
Then re-run the last command:
$ cmake --build .
It takes some time to be completed:
...
[100%] Linking CXX executable ../../../../bin/lldb-vscode
cd /tmp/llvm-20211109-12151-m0zvtm/llvm-project-13.0.0.src/llvm/build/tools/lldb/tools/lldb-vscode && /usr/local/Cellar/cmake/3.21.4/bin/cmake -E cmake_link_script CMakeFiles/lldb-v
scode.dir/link.txt --verbose=1
/usr/local/Homebrew/Library/Homebrew/shims/mac/super/clang++ -stdlib=libc++ -fPIC -fvisibility-inlines-hidden -Werror=date-time -Werror=unguarded-availability-new -Wall -Wextra -Wn
o-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic -Wno-long-long -Wc++98-compat-extra-semi -Wimplicit-fallthrough -Wcovered-switch-default -Wno-c
lass-memaccess -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wsuggest-override -Wstring-conversion -Wmisleading-indentation -Wno-deprecated-declarations -Wno-unkn
own-pragmas -Wno-strict-aliasing -Wno-deprecated-register -Wno-vla-extension -O3 -DNDEBUG -Wl,-search_paths_first -Wl,-headerpad_max_install_names -stdlib=libc++ -Wl,-sectcreate,__
TEXT,__info_plist,/tmp/llvm-20211109-12151-m0zvtm/llvm-project-13.0.0.src/llvm/build/tools/lldb/tools/lldb-vscode/lldb-vscode-Info.plist -Wl,-dead_strip CMakeFiles/lldb-vscode.dir/
lldb-vscode.cpp.o CMakeFiles/lldb-vscode.dir/BreakpointBase.cpp.o CMakeFiles/lldb-vscode.dir/ExceptionBreakpoint.cpp.o CMakeFiles/lldb-vscode.dir/FifoFiles.cpp.o CMakeFiles/lldb-vsc
ode.dir/FunctionBreakpoint.cpp.o CMakeFiles/lldb-vscode.dir/IOStream.cpp.o CMakeFiles/lldb-vscode.dir/JSONUtils.cpp.o CMakeFiles/lldb-vscode.dir/LLDBUtils.cpp.o CMakeFiles/lldb-vsco
de.dir/OutputRedirector.cpp.o CMakeFiles/lldb-vscode.dir/ProgressEvent.cpp.o CMakeFiles/lldb-vscode.dir/RunInTerminal.cpp.o CMakeFiles/lldb-vscode.dir/SourceBreakpoint.cpp.o CMakeFi
les/lldb-vscode.dir/VSCode.cpp.o -o ../../../../bin/lldb-vscode -Wl,-rpath,#loader_path/../lib ../../../../lib/liblldb.13.0.0.dylib -lpthread ../../../../lib/libclang-cpp.dylib ../
../../../lib/libLLVM.dylib
[100%] Built target lldb-vscode
/usr/local/Cellar/cmake/3.21.4/bin/cmake -E cmake_progress_start /tmp/llvm-20211109-12151-m0zvtm/llvm-project-13.0.0.src/llvm/build/CMakeFiles 0
Then run the install command:
$ cmake --build . --target install
The tail of the result should be:
...
-- Installing: /usr/local/Cellar/llvm/13.0.0_1/lib/cmake/llvm/./CheckAtomic.cmake
-- Installing: /usr/local/Cellar/llvm/13.0.0_1/lib/cmake/llvm/./FindSphinx.cmake
-- Installing: /usr/local/Cellar/llvm/13.0.0_1/lib/cmake/llvm/./FindGRPC.cmake
-- Installing: /usr/local/Cellar/llvm/13.0.0_1/lib/cmake/llvm/./TableGen.cmake
Execute the last command:
$ cmake --build . --target install-xcode-toolchain
The tail of the results should be:
...
-- Installing: /usr/local/Cellar/llvm/13.0.0_1/Toolchains/LLVM13.0.0.xctoolchain//usr/lib/cmake/llvm/./CheckAtomic.cmake
-- Installing: /usr/local/Cellar/llvm/13.0.0_1/Toolchains/LLVM13.0.0.xctoolchain//usr/lib/cmake/llvm/./FindSphinx.cmake
-- Installing: /usr/local/Cellar/llvm/13.0.0_1/Toolchains/LLVM13.0.0.xctoolchain//usr/lib/cmake/llvm/./FindGRPC.cmake
-- Installing: /usr/local/Cellar/llvm/13.0.0_1/Toolchains/LLVM13.0.0.xctoolchain//usr/lib/cmake/llvm/./TableGen.cmake
Built target install-xcode-toolchain
/usr/local/Cellar/cmake/3.21.4/bin/cmake -E cmake_progress_start /tmp/llvm-20211109-12151-m0zvtm/llvm-project-13.0.0.src/llvm/build/CMakeFiles 0
Then press control+d to return to debug menu. Because the two last commands were run manually, you need to ignore the rest of errors by choosing the option 2:
- raise
- ignore
- backtrace
- irb
- shell
Choose an action: 2
==> cmake --build . --target install
...
cmake
--build
.
--target
install
Error: could not load cache
BuildError: Failed executing: cmake --build . --target install
1. raise
2. ignore
3. backtrace
4. irb
5. shell
Choose an action: 2
==> cmake --build . --target install-xcode-toolchain
...
cmake
--build
.
--target
install-xcode-toolchain
Error: could not load cache
BuildError: Failed executing: cmake --build . --target install-xcode-toolchain
1. raise
2. ignore
3. backtrace
4. irb
5. shell
Choose an action: 2
It will continue to install to the rest:
==> Fixing /usr/local/Cellar/llvm/13.0.0_1/bin/FileCheck permissions from 755 to 555
==> Fixing /usr/local/Cellar/llvm/13.0.0_1/bin/analyze-build permissions from 755 to 555
...
==> Changing dylib ID of /usr/local/Cellar/llvm/13.0.0_1/lib/libunwind.1.0.dylib
from #rpath/libunwind.1.dylib
to /usr/local/opt/llvm/lib/libunwind.1.dylib
/usr/local/Homebrew/Library/Homebrew/brew.rb (Formulary::FromPathLoader): loading /usr/local/opt/llvm/.brew/llvm.rb
==> Caveats
To use the bundled libc++ please add the following LDFLAGS:
LDFLAGS="-L/usr/local/opt/llvm/lib -Wl,-rpath,/usr/local/opt/llvm/lib"
llvm is keg-only, which means it was not symlinked into /usr/local,
because macOS already provides this software and installing another version in
parallel can cause all kinds of trouble.
If you need to have llvm first in your PATH, run:
echo 'export PATH="/usr/local/opt/llvm/bin:$PATH"' >> ~/.zshrc
For compilers to find llvm you may need to set:
export LDFLAGS="-L/usr/local/opt/llvm/lib"
export CPPFLAGS="-I/usr/local/opt/llvm/include"
==> Summary
🍺 /usr/local/Cellar/llvm/13.0.0_1: 10,907 files, 1.8GB, built in 1418 minutes 39 seconds
It can be verified this way, the default llvm#10 pre-installed:
$ /usr/bin/clang --version
Apple LLVM version 10.0.0 (clang-1000.11.45.5)
Target: x86_64-apple-darwin17.7.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
And the new Homebrew version of llvm#13:
$ /usr/local/opt/llvm/bin/clang --version
Homebrew clang version 13.0.0
Target: x86_64-apple-darwin17.7.0
Thread model: posix
InstalledDir: /usr/local/opt/llvm/bin
#HamidRohani provides a great solution for those still tinkering in High Sierra (10.13). Getting a recent version of LLVM to compile on my old MAC with an older XCode (clang version 10.0.1 in my case) was a great help. My nominal contribution...
Alternatively, you could define the symbol after line 41 in HostInfoMacOSX.mm:
// Kludge: Symbol definition extracted from a modern machine.h
#ifndef CPU_SUBTYPE_ARM64E
# define CPU_SUBTYPE_ARM64E ((cpu_subtype_t) 2)
#endif
Now, there's no need to modify line 246. And the definition would resolve any (possible) subsequent references. And let me aggregate the steps shown above conducted in brew's debug-shell:
cmake . -DLLVM_CREATE_XCODE_TOOLCHAIN=On
cmake --build .
cmake --build . --target install
cmake --build . --target install-xcode-toolchain
Regarding the LLVM-related variable, setting LLVM_CREATE_XCODE_TOOLCHAIN to On directs CMake to generate a target named 'install-xcode-toolchain'. 1 The target is a work-around to System Integrity Protection (SIP); "Xcode toolchains are a mostly-undocumented feature that allows multiple copies of low level tools to be installed to different locations, and users can easily switch between them." 2
Brew's Caveats
Brew gives you few caveats necessary to use the new compiler: "because macOS already provides this software and installing another version in parallel can cause all kinds of trouble." To use your new compiler, "You need to have llvm first in your PATH and for compilers to find llvm you may need to set" LDFLAGS and CDFLAGS. But since these gems-of-wisdom appear near the end of a million-lines of output, let me re-iterate here:
export PATH="/usr/local/opt/llvm/bin:$PATH"
export LDFLAGS="-L/usr/local/opt/llvm/lib"
export CPPFLAGS="-I/usr/local/opt/llvm/include"
Setting PATH is straight forward. I however, didn't need to set LDFLAGS or CPPFLAGS. Further, no joy with this additional caveat, "To use the bundled libc++ please add the following LDFLAGS":
export LDFLAGS="-L/usr/local/opt/llvm/lib -Wl,-rpath,/usr/local/opt/llvm/lib"
Anyway, moving on... To demonstrate that all's good, a C++ foo program that incorporates <filesystem>; a library not in High Sierra:
#include <iostream>
// C++17: Modern C++ compiler has std filesystem
#include <filesystem>
namespace fs = std::filesystem;
typedef std::filesystem::path my_path;
using namespace std;
int main ()
{
fs::path path{"/tmp"};
path /= "foo.txt";
ofstream ofs(path);
ofs << "Hello World." << endl;
ofs.close();
return 0;
}
Clearly, a nonsensical program, But to compile:
unset CPPFLAGS
unset LDFLAGS
clang++ -std=c++17 -L/usr/local/opt/llvm/lib foo.cpp -o foo
Again, showing That I didn't need CPPFLAGS and LDFLAGS. And so, The executable links to the correct libc++ library:
MacIntel:c++fs mjo$ otool -L foo
foo:
/usr/local/opt/llvm/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.50.4)
Enjoy.

"command not found" in zsh after adding a command into /usr/local/bin

~ ll /usr/local/bin| grep vrl
-rwxrwxrwx 1 root root 31K 9月 29 09:38 vrlsubmit
➜ ~ echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
➜ ~ vrlsubmit
zsh: command not found: vrlsubmit
Finally somebody helped me:
Root cause is:
I am trying to exec a 32bit command in 64bit system
Debug:
> file submit # < submit stands for the command >
submit: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux.so.2, for GNU/Linux 2.2.5, not stripped
> ls /lib | grep ld
compat-ld
gold-ld
Based on above "file submit", it tells "interpreter /lib/ld-linux.so.2" needed. But there are no /lib/ld-linux.so.2 file at all!!!
Solution:
sudo apt-get install lib32z1 -y
submit
Error: Missing one or more required arguments.
submit
Finished to resolve this issue!

Errors due to vowpal wabbit's dependencies on boost library

I'm trying real hard to install vowpal wobbit and it fails when i run the make file, throwing:
cd library; make; cd ..
g++ -g -o ezexample temp2.cc -L ../vowpalwabbit -l vw -l allreduce -l boost_program_options -l z -l pthread
ld: library not found for -lboost_program_options collect2: ld returned 1 exit status make[1]: *** [ezexample] Error 1'
I then added the links to the boost library here by specifying -L/usr/local/lib
Now I get the following error:
g++ -g -o ezexample temp2.cc -L/usr/local/lib ../vowpalwabbit -l vw -l allreduce -l boost_program_options -l z -l pthread
ld: library not found for -lvw
collect2: ld returned 1 exit status
make: *** [ezexample] Error 1
I happened to get everything working on OS X 10.7 as follows:
Make sure you have a working Boost installation. As indicated on the Getting started page, usually we only need header files, but some Boost libraries must be built separately, including the program_options library which is used to process options from command line or config file. Go into your boost folder, and then at your shell prompt:
$ ./bootstrap.sh
$ ./bjam
This will compile and build everything. You should now have a bin.v2/ directory in your boost directory, with all built libraries for your system (static and threaded libs).
$ ls bin.v2/libs/
date_time iostreams python serialization test
filesystem math random signals thread
graph program_options regex system wave
More importantly, extra Boost libraries are made available in the stage/lib/ directory. For me, these are Mach-O 64-bit dynamically linked shared library x86_64.
The include path should be your_install_dir/boost_x_xx_x, where boost_x_xx_x is the basename of your working Boost. (I personally have boost_1_46_1 in /usr/local/share/ and I symlinked it to /usr/local/share/boost to avoid having to remember version number.) The library path (for linking) should read your_install_dir/boost_x_xx_x/stage/lib. However, it might be best to symlink or copy (which is what I did) everything in usual place, i.e. /usr/local/include/boost for header files, and /usr/local/lib for libraries.
Edit the Makefile from the vowpal_wabbit directory, and change the include/library paths to reflect your current installation. The Makefile should look like this (first 12 lines):
COMPILER = g++
UNAME := $(shell uname)
ifeq ($(UNAME), FreeBSD)
LIBS = -l boost_program_options -l pthread -l z -l compat
BOOST_INCLUDE = /usr/local/include
BOOST_LIBRARY = /usr/local/lib
else
LIBS = -l boost_program_options -l pthread -l z
BOOST_INCLUDE = /usr/local/share/boost # change path to reflect yours
BOOST_LIBRARY = /usr/local/share/boost/stage/lib # idem
endif
Then, you are ready to compile vowpal_wabbit (make clean in case you already compiled it):
$ make
$ ./vw --version
6.1
$ make test
You can also install vowpal wabbit on OS X using brew:
brew install vowpal-wabbit
Or you can just install boost, and then install vw from the github repo.
brew install boost
For installation on CentOS 7 (6.5 perl version is too old for latest vw source code), I've found the instructions at http://wkoplitz.blogspot.be/2012/12/vowpal-wabbit-on-centos.html to work fine:
yum install zlib-devel boost-devel
yum groupinstall "Development Tools"
git clone git://github.com/JohnLangford/vowpal_wabbit.git
cd vowpal_wabbit
./autogen.sh
make
make test
Good news:
As of the latest release VowpalWabbit version 9.1.0, vw no longer relies on Boost program_options
From the release highlights:
Removal of Boost Program Options dependency
For a long time we have depended on Boost Program Options
for command line options parsing. In this release, we have > replaced this dependency with our own implementation of
command line parsing. Apart from one place where we depend > on Boost Math in standalone mode, this means that VW core
and the command line tool are free of Boost dependencies
hopefully making the code a bit easier to build and package.
Vowpal Wabbit 9.1.0 release notes

Resources