Using Dart FFI I'm trying to dynamically load the linux/posix 'stat' function.
I've assumed that the function is in the libc.so.6 library but when I attempt to load it I get the error:
Invalid argument(s): Failed to lookup symbol (/lib/x86_64-linux-gnu/libc.so.6: undefined symbol: stat)
I'm successfully loading other functions from the libc.so.6 library so my dynamic loading technique is working correctly.
I have two theories:
stat is a macro for xstat and as such stat no longer exists.
stat is in another library that I've not been able to groc.
Ideally I want to use stat rather than xstat as I need this code to also work on osx which as far as I can tell doesn't support xstat.
Help?
I have two theories:
There is no need to theorize: you can just look:
echo "#include <sys/stat.h>" | gcc -xc - -E -dD | less
nm -AD /lib/x86_64-linux/gnu/*.so* | grep ' stat$'
will tell you everything you need to know (your first theory is correct).
I want to use stat rather than xstat
You can't: it doesn't exist (when using GLIBC).
I need this code to also work on osx which as far as I can tell doesn't support xstat.
Your code can detect the platform it's running on and adjust. This is the price of using non-portable mechanisms, such as FFI.
Related
We got rejected by Apple. Happens all the time right? But this time we are a bit stumped. The old ways of sussing this out aren't producing a clue to the solution.
From Apple:
5 PERFORMANCE: SOFTWARE REQUIREMENTS
Performance - 2.5.1
Your app uses or references the following non-public APIs:
transition:didComplete:
The use of non-public APIs is not permitted on the App Store because
it can lead to a poor user experience should these APIs change.
This app has been around half a decade and over the years, mostly due to business needs, it has a lot of references to 3rd party SDKs. This is where we are focusing our attention, but the trail dries up quick and is turning into a massive removal of everything until we find the pieces that include this old code.
What we know is it is not a symbol, otool and nm don’t find anything. strings does locate a match (1 time in the debug build and 2 times in our final release build if that is a clue or makes a difference.) This would appear to be a call is UIKit so I assuming that would not be the case.
Does anyone have any suggestions on how to proceed?
We're going through every archive/lib/binary we can find referenced in the project and doing string searches. If that fails we are about to rip every SDK out and do a destructive binary search to find the guilty party... If there was a hot tip on how to solve this I am all ears!
Here is the command line output (strings, otool, and nm):
Dev-MBP:helloworld.app codemonkey$ otool -ov helloworld | grep -C 10 "transition:didComplete"
Dev-MBP:helloworld.app codemonkey$ nm helloworld | grep -C 10 "transition:didComplete"
Dev-MBP:helloworld.app codemonkey$ strings helloworld | grep -C 3 "transition:didComplete"
destinationLayout
prepareTransition:
performTransition:
transition:didComplete:
destinationViewController
sourceViewController
isViewTransition
--
--
destinationLayout
prepareTransition:
performTransition:
transition:didComplete:
destinationViewController
sourceViewController
isViewTransition
Dev-MBP:helloworld.app codemonkey$ strings helloworld | grep "transition:didComplete"
transition:didComplete:
transition:didComplete:
Dev-MBP:helloworld.app codemonkey$
The lib containing the string "transition:didComplete:" was coming from the 4.x Google Play Games libs for c++. We also found it in the latest 5.1.1.
The command/script I ran to find this string inside of a file is probably the most useful part of this answer:
Dev-MBP:helloworldproj codemonkey$ find . -type f | while read i; do grep 'transition:didComplete' "$i" >/dev/null; [ $? == 0 ] && echo $i; done
Run this from the root of your iOS project (assuming your frameworks you added are all under that directory)
We can now deliberate on the most efficient way to write this command. I've already had one suggestion:
From a friend:
Naturally, it could be improved upon. The -l option to grep does the
same thing, so ...
find . -type f |
while read i
do
grep -l 'transition:didComplete' "$i"
done
I have been using clang-3.5 to happily build bitcode versions of musl libc and
use the result to produce nice stand alone executables.
Recent attempts with clang-3.8 have not been so happy. It seems that
the bitcode clang-3.8 generates uses functions defined in
compiler-rt/lib/builtins
Typical examples of functions I find polluting the bitcode are mulxc3, mulsc3, and muldc3. I can solve this by linking against libgcc, or even the llvm alternative if I had any clear idea of what that was. Though I would rather prevent the problem from happening in the first place.
I have seen mention of flags like rtlib=compiler-rt etc, but have found precious little documentation on the subject.
So here are some simple questions.
Is it possible to prevent clang from using the compiler-rt/lib/builtins
in the emitted bitcode? Or if not
Does llvm produce a version of libgcc that I could use. Actually I would
probably build a bitcode version of it, but that is besides the point.
Love to hear some guidance on this.
Added 12/8/2016: So I will illustrate my issues with a particular workflow that
people can reproduce if they wish, or, more likely, just point out where I am being stupid.
So start by checking out:
musllv
and follow the instructions in the README.to compile (here I am using clang-3.8 on ubuntu 14.04)
WLLVM_CONFIGURE_ONLY=1 CC=wllvm ./configure --target=LLVM --build=LLVM
make
cd lib
extract-bc -b libc.a
you will also need the bitcode of a simple executable. I will use nweb.c here.
wllvm nweb.c -o nweb
extract-bc nweb
Now we can do things like:
clang -static -nostdlib nweb.bc libc.a.bc crt1.o libc.a -o nweb
This workflow goes smoothly for clang-3.5 but for clang-3.8 we get:
clang -static -nostdlib nweb.bc libc.a.bc crt1.o libc.a -o nweb
/tmp/libc-f734a3.o: In function `cpowl':
libc.a.bc:(.text+0xbb9a): undefined reference to `__mulxc3'
/tmp/libc-f734a3.o: In function `cpowf':
libc.a.bc:(.text+0x38f7d): undefined reference to `__mulsc3'
/tmp/libc-f734a3.o: In function `csqrt':
libc.a.bc:(.text+0x78fc3): undefined reference to `__muldc3'
/tmp/libc-f734a3.o: In function `cpow':
libc.a.bc:(.text+0xafafc): undefined reference to `__muldc3'
clang-3.8: error: linker command failed with exit code 1 (use -v to seeinvocation)
So as #paul-brannan points out we could try
clang -static -nostdlib --rtlib=compiler-rt nweb.bc libc.a.bc crt1.o libc.a -o nweb
But this is where I am probably being stupid, because I get:
clang-3.8: warning: argument unused during compilation: '--rtlib=compiler-rt'
irregardless of whether I use it as a linking or compiling flag.
OK so I finally managed to make headway on this. I built llvm-3.8.1 together with the compiler-rt project using wllvm and wllvm++.
One of the build products was libclang_rt.builtins-x86_64.a,
and from this archive I was able to extract the bitcode module
libclang_rt.builtins-x86_64.bc
using the command:
extract-bc -b libclang_rt.builtins-x86_64.a
This bitcode module has definitions for those pesky instrinsics like
__mulxc3, __mulsc3, and __muldc3.
Hallelujah!
This is basically the same issue as in mingw ld cannot find some library which is exist in the search path, MinGW linker can't find MPICH2 libraries - and I'm aware that there are heaps of posts on StackOverflow regarding the issue of static and dynamic linking with MinGW - but I couldn't find anything that explains how I can troubleshoot.
I am building a project with a huge linker command like (via g++) on MinGW, in a MSYS2 shell (git-bash.exe). The process fails with, among others:
/z/path/to/../../../../i686-w64-mingw32/bin/ld.exe: cannot find -lssl
I add -Wl,--verbose to the g++ linker call (to be passed to ld), and I can see for the -L/z/path/to/libs/openssl/lib/mingw -lssl:
...
attempt to open /z/path/to/libs/openssl/lib/mingw/libssl.a failed
...
/z/path/to/libs/openssl/lib/mingw/ssl.dll failed
attempt to open /z/path/to/libs/openssl/lib/mingw\libssl.a failed
...
But this is weird, because the file exists?
$ file /z/path/to/libs/openssl/lib/mingw/libssl.a
/z/path/to/libs/openssl/lib/mingw/libssl.a: current ar archive
(... and it was built with the same compiler on the same machine)?
Weirdly, once it attempts to open with forward slash .../libssl.a, once with backslash ...\libssl.a - but at least the first path checks out in a bash shell, as shown above?
It gets even worse if I try to specify -l:libssl.a -- or if I specify -L/z/path/to/libs/openssl/lib/mingw -Wl,-Bstatic -lssl -- instead; then all attempts to open are with a backslash:
...
attempt to open /z/path/to/scripts/other/build/openssl/build/mingw/lib\libssl.a failed
attempt to open /z/path/to/libs/openssl/lib/mingw\libssl.a failed
...
To top it all off, if I look it up manually through the command line using ld, it is found ?!:
$ ld -L/z/path/to/libs/openssl/lib/mingw -lssl --verbose
attempt to open Z:/path/to/libs/openssl/lib/mingw/libssl.dll.a failed
attempt to open Z:/path/to/libs/openssl/lib/mingw/ssl.dll.a failed
attempt to open Z:/path/to/libs/openssl/lib/mingw/libssl.a succeeded
Does anyone have an idea why this happens, and how can I get ld to finally find these libraries? Or rather - how can I troubleshoot, and understand why these libraries are not found, when they exist at the paths where ld tries to open them?
OK, found something more - not sure if this is a bug; but my problem is that I'm actually reading arguments from a file (otherwise I get g++: Argument list too long). So, to simulate that:
$ echo " -Wl,--verbose -L/z/path/to/libs/openssl/lib/mingw -lssl -lcrypto " > tmcd3
$ g++ #tcmd3 2>&1 | grep succeeded | grep ssl
# nothing
$ g++ `cat tcmd3` 2>&1 | grep succeeded | grep ssl
attempt to open Z:/path/to/libs/openssl/lib/mingw/libssl.a succeeded
attempt to open Z:/path/to/libs/openssl/lib/mingw/libcrypto.a succeeded
... it turns out, if the very same arguments are fed on the command line, then static library lookup succeeds - but if the arguments are read from file through the # at-sign, then static library lookup fails?! Unfortunately, I cannot use on my actual project, since even with cat, I'd still get g++: Argument list too long ... So how can I fix this?
MSYS has special handling of directories as arguments when they are used in the shell. This translates e.g. /<drive_letter>/blabla to the proper Windows style paths. This is to accomodate Unix programs that don't handle Z: style directory root.
What you see here is that MSYS isn't performing this interpretation for string read from a file. When you think about it, it's very logical, but as you have experienced first-hand, also sometimes annoying.
Long story short: don't put Unix style paths in files with command arguments. Instead, pass them through e.g. cygpath -w, which works in MSYS2 (which should be the MSYS that Git for Windows 2+ comes with).
Ok, with some more experiments, I noticed that:
-L/z/path/to/libs/openssl/lib/mingw, the Unix path specification, tends to fail - while if we specify the same, except starting with a Windows drive letter, that is:
-LZ:/path/to/libs/openssl/lib/mingw, then things work - also from an arguments file with # at-sign:
$ echo " -Wl,--verbose -LZ:/path/to/libs/openssl/lib/mingw -lssl -lcrypto " > tmcd3
$ g++ #tcmd3 2>&1 | grep succeeded | grep ssl
attempt to open Z:/path/to/libs/openssl/lib/mingw/libssl.a succeeded
attempt to open Z:/path/to/libs/openssl/lib/mingw/libcrypto.a succeeded
I guess, since the shell is MSYS2/git-bash.exe, entering full POSIX paths on the shell with /z/... is not a problem, because the shell will convert them - but in a file, there is nothing to convert them, so we must use Windows/MingW convention to specify them...
I'm searching recursively some location e.g. /cygdrive/c/dev/maindir/dir/
There's a loop inside that directory structure i.e. there's a link .../maindir/dir/loopedDir/loopedDir pointing to .../maindir/dir/loopedDir.
When I run:
grep --exclude="/cygdrive/c/dev/maindir/dir/loopedDir/loopedDir" 'myPattern' -R /cygdrive/c/dev/maindir/dir/
...it works fine, like expected and finds what I need.
However, I also get a warning:
grep: warning: /cygdrive/c/dev/maindir/dir/loopedDir/loopedDir: recursive directory loop
...and I'm wondering why is that. Shouldn't dir exclusion prevent this particular looping occurance? How should I modify my query in order not to get the warning?
Add grep's option -s to suppress this and other error messages.
I want to use Mecab in Lua,but I really can't figure out the procedure of bindings,i am work no windows 7,is bingdings meaning that to create a shared library? if so? how to?i see some binding files about Java,the files in package org.chasen.mecab shows that it create by swig,it make me confused.so where does it derive from ? or just write by ourself?after bindings,what else should i do if i want to use in Lua,by the way I use mingw.can someone give some simple steps to me that I can keep trying to work on it.
to greatwolf:
i use followings command to do it
swig -lua -c++ MeCab.i
g++ -c MeCab_wrap.cxx -I C:\Lua\5.1\include -I ..\src
g++ -LC:\Lua\5.1\lib -shared MeCab_wrap.o -llua51 -o MeCab.dll
and i got errors below.
MeCab_wrap.o:MeCab_wrap.cxx:(.text+0x2960): undefined reference to `_imp___ZN5MeCab12createTaggerEPKc'
MeCab_wrap.o:MeCab_wrap.cxx:(.text+0x2981): undefined reference to `_imp___ZN5MeCab12getLastErrorEv'
MeCab_wrap.o:MeCab_wrap.cxx:(.text+0x29c9): undefined reference to `_imp___ZN5MeCab12createTaggerEPKc'
MeCab_wrap.o:MeCab_wrap.cxx:(.text+0x29d9): undefined reference to `_imp___ZN5MeCab12getLastErrorEv'
MeCab_wrap.o:MeCab_wrap.cxx:(.text+0x2a8d): undefined reference to `_imp___ZN5MeCab11createModelEPKc'
MeCab_wrap.o:MeCab_wrap.cxx:(.text+0x2aae): undefined reference to `_imp___ZN5MeCab12getLastErrorEv'
MeCab_wrap.o:MeCab_wrap.cxx:(.text+0x2af6): undefined reference to `_imp___ZN5MeCab11createModelEPKc'
MeCab_wrap.o:MeCab_wrap.cxx:(.text+0x2b06): undefined reference to `_imp___ZN5MeCab12getLastErrorEv'
MeCab_wrap.o:MeCab_wrap.cxx:(.text+0x2b6d): undefined reference to `_imp___ZN5MeCab13createLatticeEv'
MeCab_wrap.o:MeCab_wrap.cxx:(.text+0x11050): undefined reference to `MeCab::Model::version()'
MeCab_wrap.o:MeCab_wrap.cxx:(.text+0x11457): undefined reference to `MeCab::Model::create(int, char**)'
MeCab_wrap.o:MeCab_wrap.cxx:(.text+0x11732): undefined reference to `MeCab::Model::create(char const*)'
MeCab_wrap.o:MeCab_wrap.cxx:(.text+0x122bf): undefined reference to `MeCab::Tagger::parse(MeCab::Model const&, MeCab::Lattice*)'
MeCab_wrap.o:MeCab_wrap.cxx:(.text+0x17737): undefined reference to `MeCab::Tagger::create(int, char**)'
MeCab_wrap.o:MeCab_wrap.cxx:(.text+0x17a12): undefined reference to `MeCab::Tagger::create(char const*)'
MeCab_wrap.o:MeCab_wrap.cxx:(.text+0x17d83): undefined reference to `MeCab::Tagger::version()'
collect2: ld returned 1 exit status
it seems that these function declared in mecab.h，but i don't know how to handle it.
It looks like mecab provides a swig file for automatically generated bindings. As luck would have it, lua is one of swig's supported targets.
A reasonable starting point would be to check out the corresponding makefile to see how bindings get created for other languages. From mecab's swig makefile:
SWIG = swig
PREFIX = MeCab
all: perl ruby python java csharp
# ...
perl:
$(SWIG) -perl -shadow -c++ $(PREFIX).i
mv -f $(PREFIX)_wrap.cxx ../perl
mv -f $(PREFIX).pm ../perl
ruby:
$(SWIG) -ruby -c++ $(PREFIX).i
mv -f $(PREFIX)_wrap.cxx ../ruby/$(PREFIX)_wrap.cpp
python:
$(SWIG) -python -shadow -c++ $(PREFIX).i
mv -f $(PREFIX)_wrap.cxx ../python
mv -f $(PREFIX).py ../python
# ...
Extrapolating from the above, you can try generating swig bindings something like the following:
swig -lua -shadow -c++ MeCab.i
This will substantially reduce the effort you would otherwise need from manually creating the bindings yourself.
I don't know of any Lua bindings for mecab. Try googling for them first.
To create the bindings yourself you must be proficient both with Lua C API and C (or C++). A deep understanding of Lua itself (the language, I mean) is advisable.
Search the Lua WIKI for some more pointers. In particular BindingCodeToLua page.
I recently needed this and since I didn't find anything I wrote a module so you can use Mecab with Lua.
It works like this:
mecab = require "mecab"
parser = mecab:new("") -- you can pass mecab config options here, like "-Owakati"
print(parser:parse("吾輩は猫である"))
You can also install it via LuaRocks as mecab.
It just provides access to the parse method of the Tagger class, but when working with Mecab that's all I've ever needed. If you'd like support for other Mecab features please feel free to file an issue on Github.