I want to install mediapipe on my Mac&Android by following the two tutorials below.
https://google.github.io/mediapipe/getting_started/install.html
https://google.github.io/mediapipe/getting_started/android.html
So I tried the bazel build command, but I get an error.
I tried the following command.
$ bazel build -c opt --config=android_arm64 mediapipe/examples/android/src/java/com/google/mediapipe/apps/handtrackinggpu:handtrackinggpu
The error message is as follows.
ERROR: /private/var/tmp/_bazel_mac/686390cc5137afebad5267456e3662e5/external/bazel_tools/tools/jdk/BUILD:350:14: Action external/bazel_tools/tools/jdk/platformclasspath.jar failed (Exit 1): java failed: error executing command external/remotejdk11_macos/bin/java -XX:+IgnoreUnrecognizedVMOptions '--add-exports=jdk.compiler/com.sun.tools.javac.platform=ALL-UNNAMED' -cp ... (remaining 4 argument(s) skipped)
The detailed build contents are as follows.
(base) Mac-mini:mediapipe mac$ export ANDROID_HOME=/Users/mac/Library/Android/sdk
(base) Mac-mini:mediapipe mac$ export ANDROID_NDK_HOME=/Users/mac/Library/Android/sdk/ndk/21.4.7075529
(base) Mac-mini:mediapipe mac$ bazel build -c opt --config=android_arm64 mediapipe/examples/android/src/java/com/google/mediapipe/apps/handtrackinggpu:handtrackinggpu
DEBUG: /private/var/tmp/_bazel_mac/686390cc5137afebad5267456e3662e5/external/org_tensorflow/third_party/repo.bzl:109:14:
Warning: skipping import of repository 'com_google_absl' because it already exists.
DEBUG: /private/var/tmp/_bazel_mac/686390cc5137afebad5267456e3662e5/external/org_tensorflow/third_party/repo.bzl:109:14:
Warning: skipping import of repository 'com_google_protobuf' because it already exists.
DEBUG: /private/var/tmp/_bazel_mac/686390cc5137afebad5267456e3662e5/external/org_tensorflow/third_party/repo.bzl:109:14:
Warning: skipping import of repository 'com_google_googletest' because it already exists.
DEBUG: /private/var/tmp/_bazel_mac/686390cc5137afebad5267456e3662e5/external/org_tensorflow/third_party/repo.bzl:109:14:
Warning: skipping import of repository 'com_github_gflags_gflags' because it already exists.
DEBUG: /private/var/tmp/_bazel_mac/686390cc5137afebad5267456e3662e5/external/org_tensorflow/third_party/repo.bzl:109:14:
Warning: skipping import of repository 'build_bazel_rules_apple' because it already exists.
DEBUG: /private/var/tmp/_bazel_mac/686390cc5137afebad5267456e3662e5/external/org_tensorflow/third_party/repo.bzl:109:14:
Warning: skipping import of repository 'build_bazel_rules_swift' because it already exists.
DEBUG: /private/var/tmp/_bazel_mac/686390cc5137afebad5267456e3662e5/external/org_tensorflow/third_party/repo.bzl:109:14:
Warning: skipping import of repository 'build_bazel_apple_support' because it already exists.
DEBUG: /private/var/tmp/_bazel_mac/686390cc5137afebad5267456e3662e5/external/org_tensorflow/third_party/repo.bzl:109:14:
Warning: skipping import of repository 'bazel_skylib' because it already exists.
DEBUG: /private/var/tmp/_bazel_mac/686390cc5137afebad5267456e3662e5/external/org_tensorflow/third_party/repo.bzl:109:14:
Warning: skipping import of repository 'pybind11' because it already exists.
INFO: Build options --compilation_mode, --cpu, --crosstool_top, and 3 more have changed, discarding analysis cache.
WARNING: API level 30 specified by android_ndk_repository 'androidndk' is not available. Using latest known API level 29
DEBUG: Rule 'android_opencv' indicated that a canonical reproducible form can be obtained by modifying arguments sha256 = "cdb0e190c3734edd4052a3535d9e4310af912a9f70a421b1621711942a1028d5"
DEBUG: Repository android_opencv instantiated at:
/Users/mac/mediapipe/WORKSPACE:218:13: in <toplevel>
Repository rule http_archive defined at:
/private/var/tmp/_bazel_mac/686390cc5137afebad5267456e3662e5/external/bazel_tools/tools/build_defs/repo/http.bzl:336:31: in <toplevel>
INFO: Analyzed target //mediapipe/examples/android/src/java/com/google/mediapipe/apps/handtrackinggpu:handtrackinggpu (0 packages loaded, 18053 targets configured).
INFO: Found 1 target...
ERROR: /private/var/tmp/_bazel_mac/686390cc5137afebad5267456e3662e5/external/bazel_tools/tools/jdk/BUILD:350:14: Action external/bazel_tools/tools/jdk/platformclasspath.jar failed (Exit 1): java failed: error executing command external/remotejdk11_macos/bin/java -XX:+IgnoreUnrecognizedVMOptions '--add-exports=jdk.compiler/com.sun.tools.javac.platform=ALL-UNNAMED' -cp ... (remaining 4 argument(s) skipped)
Use --sandbox_debug to see verbose messages from the sandbox java failed: error executing command external/remotejdk11_macos/bin/java -XX:+IgnoreUnrecognizedVMOptions '--add-exports=jdk.compiler/com.sun.tools.javac.platform=ALL-UNNAMED' -cp ... (remaining 4 argument(s) skipped)
Use --sandbox_debug to see verbose messages from the sandbox
Exception in thread "main" java.lang.IllegalArgumentException: external/local_jdk
at jdk.compiler/com.sun.tools.javac.file.Locations$SystemModulesLocationHandler.update(Locations.java:1853)
at jdk.compiler/com.sun.tools.javac.file.Locations$SystemModulesLocationHandler.handleOption(Locations.java:1798)
at jdk.compiler/com.sun.tools.javac.file.Locations.handleOption(Locations.java:2062)
at jdk.compiler/com.sun.tools.javac.file.BaseFileManager.handleOption(BaseFileManager.java:269)
at jdk.compiler/com.sun.tools.javac.file.BaseFileManager$2.handleFileManagerOption(BaseFileManager.java:222)
at jdk.compiler/com.sun.tools.javac.main.Option.process(Option.java:1138)
at jdk.compiler/com.sun.tools.javac.main.Option.handleOption(Option.java:1086)
at jdk.compiler/com.sun.tools.javac.file.BaseFileManager.handleOption(BaseFileManager.java:232)
at jdk.compiler/com.sun.tools.javac.main.Arguments.doProcessArgs(Arguments.java:390)
at jdk.compiler/com.sun.tools.javac.main.Arguments.processArgs(Arguments.java:347)
at jdk.compiler/com.sun.tools.javac.main.Arguments.init(Arguments.java:246)
at jdk.compiler/com.sun.tools.javac.api.JavacTool.getTask(JavacTool.java:185)
at DumpPlatformClassPath.dumpJDK9AndNewerBootClassPath(DumpPlatformClassPath.java:106)
at DumpPlatformClassPath.main(DumpPlatformClassPath.java:67)
Target //mediapipe/examples/android/src/java/com/google/mediapipe/apps/handtrackinggpu:handtrackinggpu failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 2.900s, Critical Path: 0.73s
INFO: 129 processes: 129 internal.
FAILED: Build did NOT complete successfully
I've tried everything, but the bazel build doesn't work and I can't install mediapipe. What should I do?
Any help is welcome.
Related
For easily (hopefully) reproducing this issue, I've created a minimal Github repo.
I am trying to get test coverage reporting working using clang, llvm-cov, etc. At one point it was working and I either don't remember what I could've done to cause this issue or this issue just started appearing after a clean and rebuild. The error is:
$> DIR=build && cmake --build $DIR --config Debug --target example-coverage
[2/3] Linking CXX executable example.exe
lld-link: warning: Cannot use debug info for 'clang_rt.profile-i386.lib(InstrProfilingFile.c.obj)' [LNK4099]
>>> failed to load reference 'D:\a\_work\1\s\Intermediate\vctools\libasan.nativeproj__384798980\objr\x86\projects\compiler-rt\lib\profile\CMakeFiles\clang_rt.profile-i386.dir\clang_rt.profile-i386.pdb': no such file or directory
lld-link: warning: Cannot use debug info for 'clang_rt.profile-i386.lib(WindowsMMap.c.obj)' [LNK4099]
>>> failed to load reference 'D:\a\_work\1\s\Intermediate\vctools\libasan.nativeproj__384798980\objr\x86\projects\compiler-rt\lib\profile\CMakeFiles\clang_rt.profile-i386.dir\clang_rt.profile-i386.pdb': no such file or directory
lld-link: warning: Cannot use debug info for 'clang_rt.profile-i386.lib(InstrProfilingBuffer.c.obj)' [LNK4099]
>>> failed to load reference 'D:\a\_work\1\s\Intermediate\vctools\libasan.nativeproj__384798980\objr\x86\projects\compiler-rt\lib\profile\CMakeFiles\clang_rt.profile-i386.dir\clang_rt.profile-i386.pdb': no such file or directory
lld-link: warning: Cannot use debug info for 'clang_rt.profile-i386.lib(InstrProfilingPlatformWindows.c.obj)' [LNK4099]
>>> failed to load reference 'D:\a\_work\1\s\Intermediate\vctools\libasan.nativeproj__384798980\objr\x86\projects\compiler-rt\lib\profile\CMakeFiles\clang_rt.profile-i386.dir\clang_rt.profile-i386.pdb': no such file or directory
lld-link: warning: Cannot use debug info for 'clang_rt.profile-i386.lib(InstrProfilingMerge.c.obj)' [LNK4099]
>>> failed to load reference 'D:\a\_work\1\s\Intermediate\vctools\libasan.nativeproj__384798980\objr\x86\projects\compiler-rt\lib\profile\CMakeFiles\clang_rt.profile-i386.dir\clang_rt.profile-i386.pdb': no such file or directory
lld-link: warning: Cannot use debug info for 'clang_rt.profile-i386.lib(InstrProfiling.c.obj)' [LNK4099]
>>> failed to load reference 'D:\a\_work\1\s\Intermediate\vctools\libasan.nativeproj__384798980\objr\x86\projects\compiler-rt\lib\profile\CMakeFiles\clang_rt.profile-i386.dir\clang_rt.profile-i386.pdb': no such file or directory
lld-link: warning: Cannot use debug info for 'clang_rt.profile-i386.lib(InstrProfilingWriter.c.obj)' [LNK4099]
>>> failed to load reference 'D:\a\_work\1\s\Intermediate\vctools\libasan.nativeproj__384798980\objr\x86\projects\compiler-rt\lib\profile\CMakeFiles\clang_rt.profile-i386.dir\clang_rt.profile-i386.pdb': no such file or directory
lld-link: warning: Cannot use debug info for 'clang_rt.profile-i386.lib(InstrProfilingMergeFile.c.obj)' [LNK4099]
>>> failed to load reference 'D:\a\_work\1\s\Intermediate\vctools\libasan.nativeproj__384798980\objr\x86\projects\compiler-rt\lib\profile\CMakeFiles\clang_rt.profile-i386.dir\clang_rt.profile-i386.pdb': no such file or directory
lld-link: warning: Cannot use debug info for 'clang_rt.profile-i386.lib(InstrProfilingValue.c.obj)' [LNK4099]
>>> failed to load reference 'D:\a\_work\1\s\Intermediate\vctools\libasan.nativeproj__384798980\objr\x86\projects\compiler-rt\lib\profile\CMakeFiles\clang_rt.profile-i386.dir\clang_rt.profile-i386.pdb': no such file or directory
lld-link: warning: Cannot use debug info for 'clang_rt.profile-i386.lib(InstrProfilingInternal.c.obj)' [LNK4099]
>>> failed to load reference 'D:\a\_work\1\s\Intermediate\vctools\libasan.nativeproj__384798980\objr\x86\projects\compiler-rt\lib\profile\CMakeFiles\clang_rt.profile-i386.dir\clang_rt.profile-i386.pdb': no such file or directory
lld-link: warning: Cannot use debug info for 'clang_rt.profile-i386.lib(InstrProfilingUtil.c.obj)' [LNK4099]
>>> failed to load reference 'D:\a\_work\1\s\Intermediate\vctools\libasan.nativeproj__384798980\objr\x86\projects\compiler-rt\lib\profile\CMakeFiles\clang_rt.profile-i386.dir\clang_rt.profile-i386.pdb': no such file or directory
lld-link: warning: Cannot use debug info for 'clang_rt.profile-i386.lib(InstrProfilingNameVar.c.obj)' [LNK4099]
>>> failed to load reference 'D:\a\_work\1\s\Intermediate\vctools\libasan.nativeproj__384798980\objr\x86\projects\compiler-rt\lib\profile\CMakeFiles\clang_rt.profile-i386.dir\clang_rt.profile-i386.pdb': no such file or directory
lld-link: warning: Cannot use debug info for 'clang_rt.profile-i386.lib(InstrProfilingBiasVar.c.obj)' [LNK4099]
>>> failed to load reference 'D:\a\_work\1\s\Intermediate\vctools\libasan.nativeproj__384798980\objr\x86\projects\compiler-rt\lib\profile\CMakeFiles\clang_rt.profile-i386.dir\clang_rt.profile-i386.pdb': no such file or directory
[3/3] Running coverage for example...
FAILED: CMakeFiles/example-coverage C:/src/build/CMakeFiles/example-coverage
cmd.exe /C "cd /D C:\src\build && "C:\Program Files\CMake\bin\cmake.exe" -E env LLVM_PROFILE_FILE=example.profraw C:/src/build/example.exe && "C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\Llvm\bin\llvm-profdata.exe" merge --sparse example.profraw --output example.profdata && "C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\Llvm\bin\llvm-cov.exe" report C:/src/build/example.exe -instr-profile=example.profraw && C:\Users\erich\AppData\Roaming\Python\Python38\Scripts\gcovr.exe --cobertura-pretty example-coverage.xml -r C:/src/"
1 + 2 = 3
warning: example.profraw: unsupported instrumentation profile format version
error: no profile can be merged
ninja: build stopped: subcommand failed.
I don't know how this path to a .pdb file on a D: drive occurs, but that path does not exist. I do however have a D: drive though.
After a few hours of searching, I have not found anything that has helped me figure this out. Where does this path to D:\a\_work\1\s\Intermediate\vctools\libasan.nativeproj__384798980... come from?
The other odd thing is that I'm specifying to link with clang_rt.profile-x86_64, not clang_rt.profile-i386.
This is an MSVC bug caused by them shipping the wrong version of clang_rt.profile-x86_64.lib. See https://developercommunity.visualstudio.com/t/MSVC-ships-with-old-version-of-clang_rt/10178093
I have been attempting to build Drake from source with Bazel on Ubuntu 18.04 but the following error occurs when I run
bazel build ...
from the Drake root directory:
ERROR: /home/username/dir/drake/bindings/pydrake/BUILD.bazel:56:37: Action bindings/pydrake/documentation_pybind.h failed (Exit 1) mkdoc failed: error executing command bazel-out/host/bin/tools/workspace/pybind11/mkdoc -DDRAKE_COMMON_SYMBOLIC_DETAIL_HEADER -DEIGEN_MPL2_ONLY -DHAVE_CSTDDEF '-DFMT_HEADER_ONLY=1' '-DFMT_NO_FMT_STRING_ALIAS=1' -DHAVE_SPDLOG ... (remaining 1096 argument(s) skipped)
Use --sandbox_debug to see verbose messages from the sandbox
external/eigen/include/_usr_include_eigen3/Eigen/Core:66:12: fatal error: 'new' file not found
Traceback (most recent call last):
File "/home/username/.cache/bazel/_bazel_username/6e98757561df7a931d098ab985a3e673/sandbox/linux-sandbox/1371/execroot/drake/bazel-out/host/bin/tools/workspace/pybind11/mkdoc.runfiles/drake/tools/workspace/pybind11/mkdoc.py", line 841, in <module>
main()
File "/home/username/.cache/bazel/_bazel_username/6e98757561df7a931d098ab985a3e673/sandbox/linux-sandbox/1371/execroot/drake/bazel-out/host/bin/tools/workspace/pybind11/mkdoc.runfiles/drake/tools/workspace/pybind11/mkdoc.py", line 805, in main
severities.count(cindex.Diagnostic.Fatal)))
RuntimeError: Parsing headers using the clang library failed with 0 error(s) and 1 fatal error(s)
----------------
Note: The failure of target //tools/workspace/pybind11:mkdoc (with exit code 1) may have been caused by the fact that it is running under Python 3 instead of Python 2. Examine the error to determine if that appears to be the problem. Since this target is built in the host configuration, the only way to change its version is to set --host_force_python=PY2, which affects the entire build.
If this error started occurring in Bazel 0.27 and later, it may be because the Python toolchain now enforces that targets analyzed as PY2 and PY3 run under a Python 2 and Python 3 interpreter, respectively. See https://github.com/bazelbuild/bazel/issues/7899 for more information.
----------------
INFO: Elapsed time: 2345.311s, Critical Path: 321.76s
INFO: 1378 processes: 1378 linux-sandbox.
FAILED: Build did NOT complete successfully
Prior to building, I ran:
sudo ./setup/ubuntu/install_prereqs.sh
as mentioned in the installation instructions.
I had also set the environment variables for the c and c++ compilers to $CC=/usr/bin/gcc and $CXX=/usr/bin/gcc because I was suspicious of compiler issues being the problem. To compare against this, I also tried to build with $CC=/usr/bin/clang-9 and $CXX=/usr/bin/clang++-9, with a different resulting error:
ERROR: /home/username/dir/drake/systems/framework/BUILD.bazel:213:17: C++ compilation of rule '//systems/framework:cache_and_dependency_tracker' failed (Exit 1) clang-9 failed: error executing command /usr/bin/clang-9 -U_FORTIFY_SOURCE -fstack-protector -Wall -Wthread-safety -Wself-assign -fcolor-diagnostics -fno-omit-frame-pointer -g0 -O2 '-D_FORTIFY_SOURCE=1' -DNDEBUG -ffunction-sections ... (remaining 77 argument(s) skipped)
Use --sandbox_debug to see verbose messages from the sandbox
In file included from systems/framework/cache.cc:1:
bazel-out/k8-opt/bin/systems/framework/_virtual_includes/cache_and_dependency_tracker/drake/systems/framework/cache.h:7:10: fatal error: 'cstdint' file not found
#include <cstdint>
^~~~~~~~~
I suspect that there is some issue with the include paths that are being used during build since both compilers were unable to find standard files, however I am unsure of what to try next or how to fix this. Any suggestions or advice would be greatly appreciated!
This seems to be the same problem that I encountered. My Question is here Encounter "RuntimeError: The operating system's C++ standard library is not installed correctly" when build drake from source using bazel and I solve my problem following #jwnimmer-tri's answer.
It turns out that Clang is looking for GCC's standard library and it seems that it looks for GCC with higher version (not sure). I guess the problem is that you have gcc-8 installed but not g++-8, so after you install g++-8 you can fix the issue. Another possible way to solve your problem is to follow #jwnimmer-tri's answer in my question to remove gcc-8 completely.
UBUNTU 20.04
bazel 3.0.0- (#non-git)
openjdk 11.0.8 2020-07-14
Actually, the error itself:
dmitry#dmitry-pc:~/mediapipe$ bazel build -c opt --define MEDIAPIPE_DISABLE_GPU=1 mediapipe/examples/desktop/hand_tracking:hand_tracking_cpu
Starting local Bazel server and connecting to it...
INFO: SHA256 (https://github.com/bazelbuild/rules_foreign_cc/archive/master.zip) = f358144776d3dfc5a928ef32a0e4fbe93c99b55772d70cca77a6478e34d96aa7
DEBUG: Rule 'rules_foreign_cc' indicated that a canonical reproducible form can be obtained by modifying arguments sha256 = "f358144776d3dfc5a928ef32a0e4fbe93c99b55772d70cca77a6478e34d96aa7"
DEBUG: Call stack for the definition of repository 'rules_foreign_cc' which is a http_archive (rule definition at /home/dmitry/.cache/bazel/_bazel_dmitry/870728c32b519bd8ea9ab1059ef39614/external/bazel_tools/tools/build_defs/repo/http.bzl:296:16):
- <builtin>
- /home/dmitry/mediapipe/WORKSPACE:39:1
INFO: SHA256 (https://github.com/bazelbuild/rules_cc/archive/master.zip) = d1d266f0ee34a413af9e70b27354faddc2f300d873bcc388508901121e08aff4
DEBUG: Rule 'rules_cc' indicated that a canonical reproducible form can be obtained by modifying arguments sha256 = "d1d266f0ee34a413af9e70b27354faddc2f300d873bcc388508901121e08aff4"
DEBUG: Call stack for the definition of repository 'rules_cc' which is a http_archive (rule definition at /home/dmitry/.cache/bazel/_bazel_dmitry/870728c32b519bd8ea9ab1059ef39614/external/bazel_tools/tools/build_defs/repo/http.bzl:296:16):
- <builtin>
- /home/dmitry/mediapipe/WORKSPACE:33:1
ERROR: /home/dmitry/.cache/bazel/_bazel_dmitry/870728c32b519bd8ea9ab1059ef39614/external/rules_cc/cc/private/rules_impl/cc_flags_supplier.bzl:28:21: rule() got unexpected keyword argument 'incompatible_use_toolchain_transition'
ERROR: While resolving toolchains for target //mediapipe/examples/desktop/hand_tracking:hand_tracking_cpu: com.google.devtools.build.lib.packages.BuildFileContainsErrorsException: error loading package '#bazel_tools//tools/cpp': in /home/dmitry/.cache/bazel/_bazel_dmitry/870728c32b519bd8ea9ab1059ef39614/external/rules_cc/cc/defs.bzl: Extension file 'cc/private/rules_impl/cc_flags_supplier.bzl' has errors
ERROR: Analysis of target '//mediapipe/examples/desktop/hand_tracking:hand_tracking_cpu' failed; build aborted: com.google.devtools.build.lib.packages.BuildFileContainsErrorsException: error loading package '#bazel_tools//tools/cpp': in /home/dmitry/.cache/bazel/_bazel_dmitry/870728c32b519bd8ea9ab1059ef39614/external/rules_cc/cc/defs.bzl: Extension file 'cc/private/rules_impl/cc_flags_supplier.bzl' has errors
INFO: Elapsed time: 12,845s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (2 packages loaded, 0 targe\
ts configured)
I came across the same error and the option incompatible_use_toolchain_transition doesn't seem to be supported in bazel version 3.0.0.
Try updating your bazel following their manual.
On more thing, the latest 3.5 version of bazel seem to have another issue. So consider getting 3.4 version or apply the workaround in the Github issue.
I'm trying to test the build of google/or-tools using the Bazel based build system on various GNU/Linux distro by using various common distros
When trying to use bazel inside an Alpine:edge based Dockerfile (i.e. in a RUN cmd), at "docker build" time.
I don't have a consistency build between my Archlinux and on Github action workflow runners (ubuntu 18.04 IIRC).
Dockerfile: https://github.com/google/or-tools/blob/master/bazel/docker/alpine/Dockerfile
I run it using my Makefile target alpine_build in google/or-tools/bazel
ref: https://github.com/google/or-tools/blob/master/bazel/Makefile
From GH ubuntu-latest (18.04 LTS ?) runner, I got this trace
$ make alpine_build
...
Step 11/11 : RUN bazel build --curses=no --copt='-Wno-sign-compare' //...:all
---> Running in 9a2f9b6f24c7
Extracting Bazel installation...
Starting local Bazel server and connecting to it...
Loading:
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
DEBUG: Rule 'com_google_protobuf' indicated that a canonical reproducible form can be obtained by modifying arguments commit = "fe1790ca0df67173702f70d5646b82f48f412b99", shallow_since = "1576187991 -0800"
DEBUG: Call stack for the definition of repository 'com_google_protobuf' which is a git_repository (rule definition at /root/.cache/bazel/_bazel_root/86fee77ec27da0053940f3f327a6fd59/external/bazel_tools/tools/build_defs/repo/git.bzl:195:18):
- /home/lib/WORKSPACE:22:1
Loading: 2 packages loaded
Analyzing: 301 targets (16 packages loaded, 0 targets configured)
Analyzing: 301 targets (25 packages loaded, 43 targets configured)
Analyzing: 301 targets (26 packages loaded, 43 targets configured)
...
ref: https://github.com/google/or-tools/runs/568849544?check_suite_focus=true
So everything seems fine up to this point (still need to figure out the jdk javac issue but it's an other topic)
On the contrary on my Archlinux, I've got:
$ make alpine_build
...
Step 11/11 : RUN bazel build --curses=no --copt='-Wno-sign-compare' //...:all
---> Running in e13ca9fd3e84
Extracting Bazel installation...
Starting local Bazel server and connecting to it...
Loading:
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
INFO: Call stack for the definition of repository 'com_google_protobuf' which is a git_repository (rule definition at /root/.cache/bazel/_bazel_root/86fee77ec27da0053940f3f327a6fd59/external/bazel_tools/tools/build_defs/repo/git.bzl:195:18):
- /home/lib/WORKSPACE:22:1
ERROR: An error occurred during the fetch of repository 'com_google_protobuf':
java.io.IOException: unlinkat(/root/.cache/bazel/_bazel_root/86fee77ec27da0053940f3f327a6fd59/external/com_google_protobuf/.git/logs/refs/remotes/origin) (Directory not empty)
ERROR: no such package '#com_google_protobuf//': java.io.IOException: unlinkat(/root/.cache/bazel/_bazel_root/86fee77ec27da0053940f3f327a6fd59/external/com_google_protobuf/.git/logs/refs/remotes/origin) (Directory not empty)
ERROR: no such package '#com_google_protobuf//': java.io.IOException: unlinkat(/root/.cache/bazel/_bazel_root/86fee77ec27da0053940f3f327a6fd59/external/com_google_protobuf/.git/logs/refs/remotes/origin) (Directory not empty)
INFO: Elapsed time: 29.713s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded)
FAILED: Build did NOT complete successfully (0 packages loaded)
The command '/bin/sh -c bazel build --curses=no --copt='-Wno-sign-compare' //...:all' returned a non-zero code: 1
make: *** [Makefile:121: alpine_build] Error 1
I've tried to look at the file /root/.cache/bazel/_bazel_root/86fee77ec27da0053940f3f327a6fd59/external/com_google_protobuf/.git/logs/refs/remotes/origin
using the previous step (alpine_devel) container:
$ make sh_alpine_devel
/home/lib # bazel build --curses=no --copt='-Wno-sign-compare' //...:all
Extracting Bazel installation...
Starting local Bazel server and connecting to it...
Loading:
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
INFO: Call stack for the definition of repository 'com_google_protobuf' which is a git_repository (rule definition at /root/.cache/bazel/_bazel_root/86fee77ec27da0053940f3f327a6fd59/external/bazel_tools/tools/build_defs/repo/git.bzl:195:18):
- /home/lib/WORKSPACE:22:1
ERROR: An error occurred during the fetch of repository 'com_google_protobuf':
java.io.IOException: unlinkat(/root/.cache/bazel/_bazel_root/86fee77ec27da0053940f3f327a6fd59/external/com_google_protobuf/.git/logs/refs/remotes/origin) (Directory not empty)
ERROR: no such package '#com_google_protobuf//': java.io.IOException: unlinkat(/root/.cache/bazel/_bazel_root/86fee77ec27da0053940f3f327a6fd59/external/com_google_protobuf/.git/logs/refs/remotes/origin) (Directory not empty)
ERROR: no such package '#com_google_protobuf//': java.io.IOException: unlinkat(/root/.cache/bazel/_bazel_root/86fee77ec27da0053940f3f327a6fd59/external/com_google_protobuf/.git/logs/refs/remotes/origin) (Directory not empty)
INFO: Elapsed time: 29.924s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded)
FAILED: Build did NOT complete successfully (0 packages loaded)
cat /root/.cache/bazel/_bazel_root/86fee77ec27da0053940f3f327a6fd59/external/com_google_protobuf/.git/logs/
refs/remotes/origin/revert-6272-MutableSequence-import
0000000000000000000000000000000000000000 c5fedd61a48a174054f98684d5ddbc2d11530367 root <root#Flex2.home> 1586336706 +0000 fetch origin refs/heads/*:refs/remotes/origin/* refs/tags/*:refs/tags/*: storing head
note Flex2 is my local machine...
So my questions:
Is it a known issue (ed don't find anything on java.io.IOException: unlinkat) ?
Does Bazel deal with the kernel (or uname -a etc...) which could explain why I don't have the same behaviour from one host to an other ?
May I have more trace to debug this issue ?
How can I fix it ?
Thanks,
Basically you need to tell to bazel to use the java locally installed by alpine than downloading one.
this can be done using the option --host_javabase=#local_jdk//:jdk
ref: https://github.com/google/or-tools/blob/2cb85b4eead4c38e1c54b48044f92087cf165bce/bazel/docker/alpine/Dockerfile#L26
Minimal working example:
https://github.com/Mizux/bazel-cpp/blob/main/ci/docker/alpine/Dockerfile
I'd like to use bazel in a locked-down environment, one without uncontrolled internet access. I'd like to force bazel to use the locally installed java, instead of downloading one. I could not figure out the required combination of flags - this is what I tried:
$ java -version
java version "1.8.0_171"
Java(TM) SE Runtime Environment (build 1.8.0_171-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.171-b11, mixed mode)
$ javac -version
javac 1.8.0_171
$ bazel clean --expunge
$ bazel info release
release 0.24.1
$ bazel build //hello
# This builds a 'hello world' cpp file.
# This rule also downloads some rules, would be nice to avoid it as
INFO: Build completed successfully, 6 total actions well
$ bazel test --nofetch //hello:hello_test
ERROR: /home/erenon/.cache/bazel/_bazel_erenon/afacf41c7e3fc3f4ea7510d344b4de38/external/bazel_tools/tools/jdk/BUILD:218:1: no such package '#remote_java_tools//': to fix, run
bazel fetch //...
External repository #remote_java_tools not found and fetching repositories is disabled. and referenced by '#bazel_tools//tools/jdk:JacocoCoverageRunner'
ERROR: Analysis of target '//hello:hello_test' failed; build aborted: no such package '#remote_java_tools//': to fix, run
bazel fetch //...
External repository #remote_java_tools not found and fetching repositories is disabled.
INFO: Elapsed time: 0.160s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (3 packages loaded, 27 targets configured)
FAILED: Build did NOT complete successfully (3 packages loaded, 27 targets configured)
I get the same results with a more intricate command line:
$ bazel test --nofetch --define=ABSOLUTE_JAVABASE=/usr/lib/jvm/java-8-openjdk-amd64 --host_javabase=#local_jdk//:jdk --javabase=#local_jdk//:jdk --host_java_toolchain=#bazel_tools//tools/jdk:toolchain_hostjdk8 --java_toolchain=#bazel_tools//tools/jdk:toolchain_hostjdk8 //hello:hello_test
INFO: Build options --define, --host_java_toolchain, --host_javabase, and 2 more have changed, discarding analysis cache.
ERROR: /home/erenon/.cache/bazel/_bazel_erenon/afacf41c7e3fc3f4ea7510d344b4de38/external/bazel_tools/tools/jdk/BUILD:218:1: no such package '#remote_java_tools//': to fix, run
bazel fetch //...
External repository #remote_java_tools not found and fetching repositories is disabled. and referenced by '#bazel_tools//tools/jdk:JacocoCoverageRunner'
ERROR: Analysis of target '//hello:hello_test' failed; build aborted: no such package '#remote_java_tools//': to fix, run
bazel fetch //...
External repository #remote_java_tools not found and fetching repositories is disabled.
INFO: Elapsed time: 0.215s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded, 50 targets configured)
FAILED: Build did NOT complete successfully (0 packages loaded, 50 targets configured)
I tried every combination of the java flags, no luck. I also tried to enable every incompatible change (--all_incompatible_changes), which yields to a different result:
INFO: Build options --incompatible_allow_python_version_transitions, --incompatible_auto_configure_host_platform, --incompatible_cc_coverage, and 24 more have changed, discarding analysis cache.
ERROR: While resolving toolchains for target //hello:hello_test: com.google.devtools.build.lib.packages.BuildFileNotFoundException: no such package '#local_config_platform//': to fix, run
bazel fetch //...
External repository #local_config_platform not found and fetching repositories is disabled.
ERROR: Analysis of target '//hello:hello_test' failed; build aborted: com.google.devtools.build.lib.packages.BuildFileNotFoundException: no such package '#local_config_platform//': to fix, run
bazel fetch //...
External repository #local_config_platform not found and fetching repositories is disabled.
INFO: Elapsed time: 0.228s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (1 packages loaded, 0 targets configured)
FAILED: Build did NOT complete successfully (1 packages loaded, 0 targets configured)
It now looks for #local_config_platform.
Is there a bazel target which I can build from source and includes these dependencies?
Is there a different command line, which enables bazel test --nofetch?
It appear that bootstrapping bazel from source can solve the issue. After downloading bazel-<version>-dist.zip from GitHub, in the unzipped folder:
env EXTRA_BAZEL_ARGS="--host_javabase=#local_jdk//:jdk" bash ./compile.sh
The compiled output is placed into output/bazel. This is a self-contained Bazel binary, without an embedded JDK.
https://docs.bazel.build/versions/master/install-compile-source.html