jemalloc and JVM_FindSignal - memory

Question

As already answered in this question:
JVM_FindSignal function continuously allocates native memory
jemalloc reporting leaks from JVM_FindSignal is related to missing debug symbols. I certainly have debugging symbols installed, see:
rbs42#rbs42-VirtualBox:/usr/lib/jvm/java-1.8.0-openjdk-amd64/jre/lib/amd64/server$ gdb libjvm.so -ex 'info address UseG1GC'
GNU gdb (Ubuntu 9.1-0ubuntu1) 9.1
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from libjvm.so...
Reading symbols from /usr/lib/debug/.build-id/16/240e0172c3fc0dd6e974325c8ad1d93723ccac.debug...
(No debugging symbols found in /usr/lib/debug/.build-id/16/240e0172c3fc0dd6e974325c8ad1d93723ccac.debug)
Installing openjdk unwinder
Traceback (most recent call last):
File "/usr/share/gdb/auto-load/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so-gdb.py", line 52, in <module>
class Types(object):
File "/usr/share/gdb/auto-load/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so-gdb.py", line 66, in Types
nmethodp_t = gdb.lookup_type('nmethod').pointer()
gdb.error: No type named nmethod.
Symbol "UseG1GC" is at 0xd189b2 in a file compiled without debugging.
still my jeprof output looks as following:
rbs42#rbs42-VirtualBox:/media/rbs42/data/Gebos/RBS42/run/sms50$ jeprof --show_bytes /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java jeprof.22104.0.f.heap
Using local file /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java.
Using local file jeprof.22104.0.f.heap.
Welcome to jeprof! For help, type 'help'.
(jeprof) top
Total: 33502504 B
20958503 62.6% 62.6% 21342909 63.7% JVM_FindSignal
8388608 25.0% 87.6% 8388608 25.0% SNX11B1A
1481379 4.4% 92.0% 1481379 4.4% inflate
1151253 3.4% 95.5% 1151253 3.4% Java_java_util_zip_ZipFile_getZipMessage
426303 1.3% 96.7% 426303 1.3% SNE00B1A
404065 1.2% 97.9% 404065 1.2% inflateInit2_
253077 0.8% 98.7% 20393297 60.9% SUNWprivate_1.1
176271 0.5% 99.2% 176271 0.5% std::__throw_ios_failure
131713 0.4% 99.6% 131713 0.4% _dl_new_object
131328 0.4% 100.0% 131328 0.4% _dl_check_map_versions
(jeprof)
Is there anything else to consider?

Answer 1

It turns out that
openjdk-8-dbg package installs files with debug symbols into /usr/lib/debug/.build-id
jeprof looks for debug symbols in /usr/lib/debug/{FULL_SO_PATH}
So, it's a combination of a bug in jeprof that does not parse .note.gnu.build-id section, and a problem of the dbg package that does not include full path symlinks to debug libraries.
To work around this, you may create the corresponding symlink manually:
ln -s /usr/lib/debug/.build-id/16/240e0172c3fc0dd6e974325c8ad1d93723ccac.debug /usr/lib/debug/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
(the file names are taken from gdb output)
However, even when jeprof is able to read debug symbols, this doesn't often help in case of Java applications. The problem is that jemalloc doesn't know how to unwind Java stacks. See this presentation for an example.
Consider trying async-profiler that can show mixed Java+native stacks. Profiling malloc, mprotect and mmap calls with async-profiler can be helpful in finding native memory leaks in Java applications. See this answer for details.

Answer 2

Here is a patch for jeprof that makes it smarter about finding debug symbols. It's based on #apangin's answer.
https://github.com/jemalloc/jemalloc/pull/2059/files