Clang/LLVM OpenMP program not spawning threads - clang

According to http://blog.llvm.org/2015/05/openmp-support_22.html, OpenMP support in Clang is completed. However, I'm having difficulties trying a simple program.
I've installed Clang/LLVM as explained in http://clang.llvm.org/get_started.html and the OpenMP runtime as explained in http://openmp.llvm.org/.
The test program is:
#include "omp.h"
#include <stdio.h>
#include <stdlib.h>
int main(int argc, char** argv)
{
#pragma omp parallel
{
printf("thread %d\n", omp_get_thread_num());
}
return 0;
}
The compilation line is:
clang -lomp -I/.../openmp/runtime/exports/common/include -L/.../openmp/runtime/exports/lin_32e/lib ./test-openmp.c -o ./test-openmp
Using which, I check I'm using the correct clang binary.
With ldd, I check I'm linking to the correct OpenMP library:
$ ldd ./test-openmp
linux-vdso.so.1 => (0x00007ffdaf6d7000)
libomp.so => /.../openmp/runtime/exports/lin_32e/lib/libomp.so (0x00007f7d47552000)
libc.so.6 => /lib64/libc.so.6 (0x00007f7d47191000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007f7d46f8d000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f7d46d71000)
/lib64/ld-linux-x86-64.so.2 (0x00007f7d477fb000)
But when running, it executes only one thread:
$ OMP_NUM_THREADS=4 ./test-openmp
thread 0
The reason I'm linking with lomp is because if I link with fopenmp, the code is wrongly linked to the gcc omp library. However in that case, the result is the same:
$ clang -fopenmp -I/.../openmp/runtime/exports/common/include -L/.../openmp/runtime/exports/lin_32e/lib ./test-openmp.c -o ./test-openmp
$ ldd ./test-openmp
linux-vdso.so.1 => (0x00007ffdf351f000)
libgomp.so.1 => /lib64/libgomp.so.1 (0x00007fbc1c3e1000)
librt.so.1 => /lib64/librt.so.1 (0x00007fbc1c1d9000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fbc1bfbd000)
libc.so.6 => /lib64/libc.so.6 (0x00007fbc1bbfc000)
/lib64/ld-linux-x86-64.so.2 (0x00007fbc1c5f8000)
$ OMP_NUM_THREADS=4 ./test-openmp
thread 0
When using gcc, it works as expected:
$ gcc -fopenmp ./test-openmp.c -o ./test-openmp
$ ldd ./test-openmp
linux-vdso.so.1 => (0x00007ffc444e0000)
libgomp.so.1 => /lib64/libgomp.so.1 (0x00007f7d425ce000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f7d423b2000)
libc.so.6 => /lib64/libc.so.6 (0x00007f7d41ff1000)
/lib64/ld-linux-x86-64.so.2 (0x00007f7d427e5000)
$ OMP_NUM_THREADS=4 ./test-openmp
thread 0
thread 2
thread 3
thread 1
I've used in the past the implementation described in http://clang-omp.github.io/ and I know that one works (a different repo is presented there for Clang and LLVM, but the OpenMP repo is the same). However that page was (apparently) updated in 2014, and the blog in http://blog.llvm.org/2015/05/openmp-support_22.html is from May 2015, which makes you think that you can use the latest Clang/LLVM for OpenMP.
So my question is, am I missing something or the blog from May 2015 is actually referring to Clang/LLVM implementation from in http://clang-omp.github.io and not to the latest one?
Thanks

Adding -fopenmp=libomp should do the trick.
This is a temporary situation; hopefully, soon clang will be changed to do what is described in the blog post.
Yours,
Andrey

Related

How to print something to the console from OpenMP source Code?

I am modifying OpenMP source Code, and I want to make sure that it is indeed working.
For example the following code:
#include "omp.h"
int main()
{
int i=0;
#pragma omp parallel for schedule(dynamic,4)
for(i=0;i<1000;++i)
{
int x = 4+i;
}
}
Should call __kmpc_dispatch_init_4()
And I have verified this by using emit-llvm option in clang.
To double check I added the following print statement:
void __kmpc_dispatch_init_4(ident_t *loc, kmp_int32 gtid,
enum sched_type schedule, kmp_int32 lb,
kmp_int32 ub, kmp_int32 st, kmp_int32 chunk) {
KMP_DEBUG_ASSERT(__kmp_init_serial);
printf("%s\n", "Hello OpenMP");
#if OMPT_SUPPORT && OMPT_OPTIONAL
OMPT_STORE_RETURN_ADDRESS(gtid);
#endif
__kmp_dispatch_init<kmp_int32>(loc, gtid, schedule, lb, ub, st, chunk, true);
}
But when I am compiling the code, I am not getting this output on terminal.
I am compiling the code like this:
llvm_build/bin/clang sc.c -L/usr/local/lib -fopenmp
And after building the openmp source code the output of sudo make install is this:
Install the project...
-- Install configuration: "Release"
-- Up-to-date: /usr/local/lib/libomp.so
-- Up-to-date: /usr/local/include/omp.h
-- Up-to-date: /usr/local/include/omp-tools.h
-- Up-to-date: /usr/local/include/ompt.h
-- Up-to-date: /usr/local/lib/libomptarget.so
-- Up-to-date: /usr/local/lib/libomptarget.rtl.x86_64.so
To cross check if this is used I used:
ldd a.out
And the output is:
linux-vdso.so.1 (0x00007fff25bb6000)
libomp.so => /usr/local/lib/libomp.so (0x00007f75a52c6000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f75a50a7000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f75a4cb6000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f75a4ab2000)
/lib64/ld-linux-x86-64.so.2 (0x00007f75a558a000)
So I am assuming it should be using the code that I have modified.
But not able to see the output of the print statement.
I have also tried using fprintf(stderr, ....)
But doesn't work.
Thank you in advance.
First, you should flush the output using fflush(stdout) after the printf to enforce the line to be printed in the console.
Moreover, you should also ensure that /usr/local/lib/libomp.so is the good one by just checking the modification date of the file (with the stat command).
If the loaded file is not the good one you can force it with the LD_LIBRARY_PATH, LIBRARY_PATH and LD_PRELOAD environment variables.
If this is not sufficient, you can use the nm or objdump tools to check the dynamic symbols: the function should be undefined in your program and provided by the modified LLVM OpenMP runtime.
PS: If none of this works, I strongly advise you to build your program and the OpenMP runtime library with debugging (and tracing) information so you can use debuggers like gdb in order to track function calls.

How do you install lua-resty-string in openresty docker image?

I want to use sha256 in my openresty code so tried installing lua-resty-string.
I am clearly missing some other package(s), but have no idea which ones. The lua-resty-string homepage offers no clue that I can see.
$ docker run --rm -it openresty/openresty bash
# apt-get update && apt-get install luarocks
# luarocks install lua-resty-string
Installing https://luarocks.org/lua-resty-string-0.09-0.rockspec
lua-resty-string 0.09-0 is now installed in /usr/local (license: )
root#28fe64c51c5a:/# luajit
LuaJIT 2.1.0-beta3 -- Copyright (C) 2005-2017 Mike Pall. http://luajit.org/
JIT: ON SSE2 SSE3 SSE4.1 fold cse dce fwd dse narrow loop abc sink fuse
> resty_sha256 = require "resty.sha256"
> sha256 = resty_sha256:new()
/usr/local/share/lua/5.1/resty/sha256.lua:41: luajit: undefined symbol: SHA256_Init
stack traceback:
[C]: in function '__index'
/usr/local/share/lua/5.1/resty/sha256.lua:41: in function 'new'
stdin:1: in main chunk
[C]: at 0x55cd1fdaf250
I would be more convinced that luarocks if it would build or install software correctly.
What I am missing (this time)?
It seems that the lua-resty-string package only works when code is executed by the nginx server.
root#62817b2fe1b2:/# ldd `which nginx`
linux-vdso.so.1 (0x00007ffe3b1db000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fa367a81000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fa367a60000)
libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 (0x00007fa367a26000)
libluajit-5.1.so.2 => /usr/local/openresty/luajit/lib/libluajit-5.1.so.2 (0x00007fa3679a3000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fa367820000)
libpcre.so.1 => /usr/local/openresty/pcre/lib/libpcre.so.1 (0x00007fa3677ad000)
libssl.so.1.1 => /usr/local/openresty/openssl/lib/libssl.so.1.1 (0x00007fa367739000)
libcrypto.so.1.1 => /usr/local/openresty/openssl/lib/libcrypto.so.1.1 (0x00007fa3674ad000)
libz.so.1 => /usr/local/openresty/zlib/lib/libz.so.1 (0x00007fa367490000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fa3672cf000)
/lib64/ld-linux-x86-64.so.2 (0x00007fa367c97000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fa3672b5000)
root#62817b2fe1b2:/# nm -D /usr/local/openresty/openssl/lib/libcrypto.so.1.1 | grep SHA256_Init
00000000001b12a0 T SHA256_Init
And that library comes with openresty:
root#62817b2fe1b2:/# dpkg -S /usr/local/openresty/openssl/lib/libcrypto.so.1.1
openresty-openssl: /usr/local/openresty/openssl/lib/libcrypto.so.1.1
So the real question I suppose it how do you tell luajit to load that library so that it can resolve SHA256_Init when resty.sha256 is imported.
And that it seems can be done with ffi:
local ffi = require("ffi")
local crypto = ffi.load("/usr/local/openresty/openssl/lib/libcrypto.so.1.1", true)

690mb memory overhead for openmp program compiled with ifort

I was running some tests using openmp and fortran and came to realize that a binary compiled with ifort 15 (15.0.0 20140723) has 690MB of virtual memory overhead.
My sample program is:
program sharedmemtest
use omp_lib
implicit none
integer :: nroot1
integer, parameter :: dp = selected_real_kind(14,200)
real(dp),allocatable :: matrix_elementsy(:,:,:,:)
!$OMP PARALLEL NUM_THREADS(10) SHARED(matrix_elementsy)
nroot1=2
if (OMP_GET_THREAD_NUM() == 0) then
allocate(matrix_elementsy(nroot1,nroot1,nroot1,nroot1))
print *, "after allocation"
read(*,*)
end if
!$OMP BARRIER
!$OMP END PARALLEL
end program
running
ifort -openmp test_openmp_minimal.f90 && ./a.out
shows a memory usage of
50694 user 20 0 694m 8516 1340 S 0.0 0.0 0:03.58 a.out
in top. Running
gfortran -fopenmp test_openmp_minimal.f90 && ./a.out
shows a memory usage of
50802 user 20 0 36616 956 740 S 0.0 0.0 0:00.98 a.out
Where is the 690MB of overhead coming from when compiling with ifort? Am I doing something wrong? Or is this a bug in ifort?
For completeness: This is a minimal example taken from a much larger program. I am using gfortran 4.4 (4.4.7 20120313).
I appreciate all comments and ideas.
I don't believe top is reliable here. I do not see any evidence that the binary created from your test allocates anywhere near that much memory.
Below I have shown the result of generating the binary normally, with the Intel libraries linked statically and with everything linked statically. The static binary is in the ballpark of 2-3 megabytes.
It is possible that OpenMP thread stacks, which I believe are allocated from the heap, could be the source of the addition virtual memory here. Can you try this test with OMP_STACKSIZE=4K? I think the default is a few megabytes.
Dynamic Executable
jhammond#cori11:/tmp> ifort -O3 -qopenmp smt.f90 -o smt
jhammond#cori11:/tmp> size smt
text data bss dec hex filename
748065 13984 296024 1058073 102519 smt
jhammond#cori11:/tmp> ldd smt
linux-vdso.so.1 => (0x00002aaaaaaab000)
libm.so.6 => /lib64/libm.so.6 (0x00002aaaaab0c000)
libiomp5.so => /opt/intel/parallel_studio_xe_2016.0.047/compilers_and_libraries_2016.0.109/linux/compiler/lib/intel64/libiomp5.so (0x00002aaaaad86000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00002aaaab0c7000)
libc.so.6 => /lib64/libc.so.6 (0x00002aaaab2e4000)
libgcc_s.so.1 => /opt/gcc/5.1.0/snos/lib64/libgcc_s.so.1 (0x00002aaaab661000)
libdl.so.2 => /lib64/libdl.so.2 (0x00002aaaab878000)
/lib64/ld-linux-x86-64.so.2 (0x0000555555554000)
Dynamic Executable with Static Intel
jhammond#cori11:/tmp> ifort -O3 -qopenmp smt.f90 -static-intel -o smt
jhammond#cori11:/tmp> size smt
text data bss dec hex filename
1608953 41420 457016 2107389 2027fd smt
jhammond#cori11:/tmp> ls -l smt
-rwxr-x--- 1 jhammond jhammond 1872489 Jan 12 05:51 smt
Static Executable
jhammond#cori11:/tmp> ifort -O3 -qopenmp smt.f90 -static -o smt
jhammond#cori11:/tmp> size smt
text data bss dec hex filename
2262019 43120 487320 2792459 2a9c0b smt
jhammond#cori11:/tmp> ldd smt
not a dynamic executable

How to resolve sqlplus conflicts?

I am install Oracle Instant Client: Basic, SDK and SQL*Plus packages v 12.1.0.1 for gem "ruby-oci8" than i realize that i need use Oracle Instant Client v 10.2.0.4. But whan i try to install rpm -ivh oracle-instantclient-sqlplus-10.2.0.4-1.i386.rpm i have error:
Preparing... ########################################### [100%]
file /usr/bin/sqlplus from install of oracle-instantclient-sqlplus-10.2.0.4-1.i386 conflicts with file from package oracle-instantclient12.1-sqlplus-12.1.0.1.0-1.i386
How to resolve this issue?
You are trying to have two different versions of Oracle client libs (10g and 12c).
You can use:
replace -i with -U (U stands for upgrade)
remove ver. 10 first (rpm -e)
If you insist on having two differt client versions remove .rpm packages. And use .zip packges. And simply unpack them into /opt
ldd sqlplus
linux-gate.so.1 => (0x006bf000)
libsqlplus.so => /usr/lib/oracle/10.2.0.4/client/lib/libsqlplus.so (0x00110000)
libclntsh.so.10.1 => /usr/lib/oracle/10.2.0.4/client/lib/libclntsh.so.10.1 (0x006c0000)
libnnz10.so => /usr/lib/oracle/10.2.0.4/client/lib/libnnz10.so (0x00218000)
libdl.so.2 => /lib/libdl.so.2 (0x001c4000)
libm.so.6 => /lib/libm.so.6 (0x001c9000)
libpthread.so.0 => /lib/libpthread.so.0 (0x0041e000)
libnsl.so.1 => /lib/libnsl.so.1 (0x00439000)
libc.so.6 => /lib/libc.so.6 (0x00469000)
/lib/ld-linux.so.2 (0x001f8000)
and ldd libclntsh.so
ldd: ./libclntsh.so: No such file or directory

Program segfaults when compiled with clang but not gcc

Summary: When I set the -mcmodel=large flag when compiling with clang my application segfaults when accessing thread local storage. This does not happen when compiling with gcc. Is this a bug with clang or something I am doing wrong?
Details:
The following code segment crashes when compiled with clang when setting the -mcmodel flag, but it runs fine when compiled with gcc
#include <stdio.h>
#include <pthread.h>
__thread int tlsTest;
int main(int argc, char **argv) {
printf("&tlsTest is %p\n", &tlsTest);
tlsTest = argc;
printf("tlsTest is %d\n", tlsTest);
return 0;
}
When I compile with: clang test.c -pthread -mcmodel=large the result is:
&tlsTest is 0x7fd24262c6fc
Segmentation fault (core dumped)
But with: gcc test.c -pthread -mcmodel=large the result is:
&tlsTest is 0x7f1cf785c6fc
tlsTest is 1
The program also works fine when compiled with: clang test.c -pthread
I read the following link about mcmodel but I'm not sure how this relates to the segfault that I've observed. Note that the problem occurs for -mcmodel=medium also, but not for -mcmodel=small.
Is this a bug with clang/llvm or is it a different interpretation of the standard or some unimplemented feature?
Also my system is Ubuntu 12.04. My version of gcc is 4.6.3 and the version of clang/llvm that I tested is a recent snapshot of 3.3 development, and I also tested with clang 3.2.

Resources