Error while generating trace file in Vampirtrace - environment-variables

I want to do profiling for my parallel processing code that I have wrote using OpenMPI library. I am trying to use vampirtrace for this purpose. I compiled my code using command
vtcxx -vt:cxx mpic++ ../src/parallel_encode_nonBlocking.cpp -o parallel_encode_nonBlocking_trace.exe
and I am trying to run my code using following command to generate trace file,
mpirun -np 4 parallel_encode_nonBlocking_trace.exe ducks.m2v ducks11.h264
here last 2 names are names of input and output files that are required by code. But after running code I am getting following errors,
/usr/bin/nm: '/home/rishikesh/Videos/Documents/Rishikesh_MTP_data/Harshad': No such file
/usr/bin/nm: 'MTP/MTP/encoder_experiments/ffmpeg_transcode/bin/parallel_encode_nonBlocking_trace.exe': No such file
VampirTrace: FATAL: Failed to execute /usr/bin/nm -B --demangle --line-numbers /home/rishikesh/Videos/Documents/Rishikesh_MTP_data/Harshad MTP/MTP/encoder_experiments/ffmpeg_transcode/bin/parallel_encode_nonBlocking_trace.exe
Please set the environment variable VT_GNU_NM to the 'nm' command including command line switches which lists symbol/addresses of an object file in BSD-style or set VT_GNU_NMFILE to a pre-created symbol list file.
/usr/bin/nm: '/home/rishikesh/Videos/Documents/Rishikesh_MTP_data/Harshad': No such file
/usr/bin/nm: 'MTP/MTP/encoder_experiments/ffmpeg_transcode/bin/parallel_encode_nonBlocking_trace.exe': No such file
/usr/bin/nm: '/home/rishikesh/Videos/Documents/Rishikesh_MTP_data/Harshad': No such file
/usr/bin/nm: 'MTP/MTP/encoder_experiments/ffmpeg_transcode/bin/parallel_encode_nonBlocking_trace.exe': No such file
VampirTrace: FATAL: Failed to execute /usr/bin/nm -B --demangle --line-numbers /home/rishikesh/Videos/Documents/Rishikesh_MTP_data/Harshad MTP/MTP/encoder_experiments/ffmpeg_transcode/bin/parallel_encode_nonBlocking_trace.exe
Please set the environment variable VT_GNU_NM to the 'nm' command including command line switches which lists symbol/addresses of an object file in BSD-style or set VT_GNU_NMFILE to a pre-created symbol list file.
/usr/bin/nm: VampirTrace: FATAL: Failed to execute /usr/bin/nm -B --demangle --line-numbers /home/rishikesh/Videos/Documents/Rishikesh_MTP_data/Harshad MTP/MTP/encoder_experiments/ffmpeg_transcode/bin/parallel_encode_nonBlocking_trace.exe
Please set the environment variable VT_GNU_NM to the 'nm' command including command line switches which lists symbol/addresses of an object file in BSD-style or set VT_GNU_NMFILE to a pre-created symbol list file.
'/home/rishikesh/Videos/Documents/Rishikesh_MTP_data/Harshad': No such file
/usr/bin/nm: 'MTP/MTP/encoder_experiments/ffmpeg_transcode/bin/parallel_encode_nonBlocking_trace.exe': No such file
VampirTrace: FATAL: Failed to execute /usr/bin/nm -B --demangle --line-numbers /home/rishikesh/Videos/Documents/Rishikesh_MTP_data/Harshad MTP/MTP/encoder_experiments/ffmpeg_transcode/bin/parallel_encode_nonBlocking_trace.exe
Please set the environment variable VT_GNU_NM to the 'nm' command including command line switches which lists symbol/addresses of an object file in BSD-style or set VT_GNU_NMFILE to a pre-created symbol list file.
--------------------------------------------------------------------------
mpirun noticed that the job aborted, but has no info as to the process
that caused that situation.
--------------------------------------------------------------------------

There seems to be a bug in VampirTrace with spaces in the absolute path name to your application binary. To work around that you can either remove the space from the "Harshad MTP" directory, or manually execute
nm parallel_encode_nonBlocking_trace.exe > nm.file
export VT_GNU_NMFILE="$PWD/nm.file"
The error message is quite clear about that if you actually carefully read it.

Related

cannot open shared object file: No such file or directory, even after modifying LD_LIBRARY_PATH variable

I have been trying to use PETSc for some MPI enhanced matrix operations, but I keep getting this error when I submit the job using slurm on a supercomputer:
ERROR: ERROR: LoadError: LoadError: could not load library
"/home/20ph92r03/petsc/arch-linux-c-debug/lib/libpetsc.so"
liblapack.so.3: cannot open shared object file: No such file or directorycould not
load library "/home/20ph92r03/petsc/arch-linux-c-debug/lib/libpetsc.so"
liblapack.so.3: cannot open shared object file: No such file or directory
And this is my jobscript
#!/bin/bash
#SBATCH -J slepcTest
#SBATCH -p standard-low
#SBATCH --output=slepcTest-out
#SBATCH -n 2
#SBATCH -t 1:00:00
module load compiler/intel-mpi/mpi-2020-v4
export LD_LIBRARY_PATH=/home/21ph92r03/julia-1.8.0/lib/julia:$LD_LIBRARY_PATH
time mpiexec -n 2 /home/20ph92r03/julia-1.8.0/bin/julia mpiSlepc.jl
I have followed the earlier threads related to the same error and modified the LD_LIBRARY_PATH environment variable in the .bashrc file, like this:
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:'/home/20ph92r03/petsc/arch-linux-c-
debug/lib/:/usr/lib64/'
But the error persists. I don't know what else to try!
Thanks!

Unix. Parse file with full paths to SHA256 checksums files. Run command in each path/file

I have a file file.txt with filenames ending with *.sha256, including the full paths of each file. This is a toy example:
file.txt:
/path/a/9b/x3.sha256
/path/7c/7j/y2.vcf.gz.sha256
/path/e/g/7z.sha256
Each line has a different path/file. The *.sha256 files have checksums.
I want to run the command "sha256sum -c" on each of these *.sha256 files and write the output to an output_file.txt. However, this command only accepts the name of the .sha256 file, not the name including its full path. I have tried the following:
while read in; do
sha256sum -c "$in" >> output_file.txt
done < file.txt
but I get:
"sha256sum: WARNING: 1 listed file could not be read"
which is due to the path included in the command.
Any suggestion is welcome
#!/bin/bash
while read in
do
thedir=$(dirname "$in")
thefile=$(basename "$in")
cd "$thedir"
sha256sum -c "$thefile" >>output_file.txt
done < file.txt
Modify your code to extract the directory and file parts of your in variable.

Aldec Active-HDL VSimSA Converting BDE files into VHDL files

I'm trying to automate compiling my Aldec block designs using the VSimSA shell instead of using the Aldec GUI. Currently, I am able to get the following command to work in the GUI console with the desired results (producing a converting a .bde file to a .vhd file in the compile directory as well as using the VHDL compiler on the .vhd file).
'''
acom -w -O3 -e 100 -work gen8_ieee -2008 -d <path to compile directory> -s <path to library.cfg> -j <path to projlib.cfg> <BDE file path>
'''
When issuing the same command in the VSimSA shell, I receive the following error. This error occurs for multiple lines in the bde file. If I run the same command in VSimSA on a .vhd file, it compiles with no issues.
*Error: COMP96_0010: <BDE file path> : (12, 36) : Invalid literal.*
Line 12 in the BDE file :
#DEFAULT_RANGE0="<range<index=\"0\"><name=\"bist_cctrl\"><left=\"bcw-1\"><direction=\"downto\"><right=\"0\"><initial_value=\"\"><delay=\"\">>"
Is there another command that needs to be issued first to convert the BDE file to a VHDL file prior to running the 'acom' command? The help documentation states the 'acom' command works with BDE files.

the make.sh for fastdht not work.When running it shows "nm: /usr/lib/libc_r.so:no such file"

I want to install fastdht. So I download source code from github
https://github.com/happyfish100/fastdht
I follow the INSTALL file, run make.sh first.
./make.sh
However, it shows the following error messages.
[root#localhost fastdht]# sh make.sh
make.sh: line 142: warning: here-document at line 2 delimited by end-of-file
(wanted `EOF')
make.sh: line 2: ./a.out: No such file or directory
nm: '/usr/lib/libc_r.so': No such file
nm: '/lib64/libc_r.so': No such file
nm: '/usr/lib64/libc_r.so': No such file
[root#localhost fastdht]#
What's the matter?
Maybe there are some format errors in make.sh.
I create a new file a.sh and type the contents of make.sh in it. When I run a.sh it works!

Why can't ld called from MSYS find (existing static) library when arguments are read from a response #file containing backslashes?

This is basically the same issue as in mingw ld cannot find some library which is exist in the search path, MinGW linker can't find MPICH2 libraries - and I'm aware that there are heaps of posts on StackOverflow regarding the issue of static and dynamic linking with MinGW - but I couldn't find anything that explains how I can troubleshoot.
I am building a project with a huge linker command like (via g++) on MinGW, in a MSYS2 shell (git-bash.exe). The process fails with, among others:
/z/path/to/../../../../i686-w64-mingw32/bin/ld.exe: cannot find -lssl
I add -Wl,--verbose to the g++ linker call (to be passed to ld), and I can see for the -L/z/path/to/libs/openssl/lib/mingw -lssl:
...
attempt to open /z/path/to/libs/openssl/lib/mingw/libssl.a failed
...
/z/path/to/libs/openssl/lib/mingw/ssl.dll failed
attempt to open /z/path/to/libs/openssl/lib/mingw\libssl.a failed
...
But this is weird, because the file exists?
$ file /z/path/to/libs/openssl/lib/mingw/libssl.a
/z/path/to/libs/openssl/lib/mingw/libssl.a: current ar archive
(... and it was built with the same compiler on the same machine)?
Weirdly, once it attempts to open with forward slash .../libssl.a, once with backslash ...\libssl.a - but at least the first path checks out in a bash shell, as shown above?
It gets even worse if I try to specify -l:libssl.a -- or if I specify -L/z/path/to/libs/openssl/lib/mingw -Wl,-Bstatic -lssl -- instead; then all attempts to open are with a backslash:
...
attempt to open /z/path/to/scripts/other/build/openssl/build/mingw/lib\libssl.a failed
attempt to open /z/path/to/libs/openssl/lib/mingw\libssl.a failed
...
To top it all off, if I look it up manually through the command line using ld, it is found ?!:
$ ld -L/z/path/to/libs/openssl/lib/mingw -lssl --verbose
attempt to open Z:/path/to/libs/openssl/lib/mingw/libssl.dll.a failed
attempt to open Z:/path/to/libs/openssl/lib/mingw/ssl.dll.a failed
attempt to open Z:/path/to/libs/openssl/lib/mingw/libssl.a succeeded
Does anyone have an idea why this happens, and how can I get ld to finally find these libraries? Or rather - how can I troubleshoot, and understand why these libraries are not found, when they exist at the paths where ld tries to open them?
OK, found something more - not sure if this is a bug; but my problem is that I'm actually reading arguments from a file (otherwise I get g++: Argument list too long). So, to simulate that:
$ echo " -Wl,--verbose -L/z/path/to/libs/openssl/lib/mingw -lssl -lcrypto " > tmcd3
$ g++ #tcmd3 2>&1 | grep succeeded | grep ssl
# nothing
$ g++ `cat tcmd3` 2>&1 | grep succeeded | grep ssl
attempt to open Z:/path/to/libs/openssl/lib/mingw/libssl.a succeeded
attempt to open Z:/path/to/libs/openssl/lib/mingw/libcrypto.a succeeded
... it turns out, if the very same arguments are fed on the command line, then static library lookup succeeds - but if the arguments are read from file through the # at-sign, then static library lookup fails?! Unfortunately, I cannot use on my actual project, since even with cat, I'd still get g++: Argument list too long ... So how can I fix this?
MSYS has special handling of directories as arguments when they are used in the shell. This translates e.g. /<drive_letter>/blabla to the proper Windows style paths. This is to accomodate Unix programs that don't handle Z: style directory root.
What you see here is that MSYS isn't performing this interpretation for string read from a file. When you think about it, it's very logical, but as you have experienced first-hand, also sometimes annoying.
Long story short: don't put Unix style paths in files with command arguments. Instead, pass them through e.g. cygpath -w, which works in MSYS2 (which should be the MSYS that Git for Windows 2+ comes with).
Ok, with some more experiments, I noticed that:
-L/z/path/to/libs/openssl/lib/mingw, the Unix path specification, tends to fail - while if we specify the same, except starting with a Windows drive letter, that is:
-LZ:/path/to/libs/openssl/lib/mingw, then things work - also from an arguments file with # at-sign:
$ echo " -Wl,--verbose -LZ:/path/to/libs/openssl/lib/mingw -lssl -lcrypto " > tmcd3
$ g++ #tcmd3 2>&1 | grep succeeded | grep ssl
attempt to open Z:/path/to/libs/openssl/lib/mingw/libssl.a succeeded
attempt to open Z:/path/to/libs/openssl/lib/mingw/libcrypto.a succeeded
I guess, since the shell is MSYS2/git-bash.exe, entering full POSIX paths on the shell with /z/... is not a problem, because the shell will convert them - but in a file, there is nothing to convert them, so we must use Windows/MingW convention to specify them...

Resources