I am trying to run bazel from a company-installed location. It decides to use my home directory for some caching and extracting installation files. There is no much space (company policy). So, the question is: how to redirect it to a different location? I could not find much on the web.
WARNING: Output base '/home/name/.cache/bazel/_bazel_name/b05102fc551c33214ce89fb43ea90837' is on NFS. This may lead to surprising failures and undetermined behavior.
Extracting Bazel installation...
FATAL: Failed to extract embedded binaries: Failed to write zipped file '/home/name/.cache/bazel/_bazel_name/install/f6ca571514ebc1be8327564b4455aae2.tmp.10086/_embedded_binaries/A-server.jar': (error: 122): Disk quota exceeded
Also, not sure why it thinks that this is NFS. I try to run it onlinux.
found it: bazel --output_user_root=<loc> fixes the issue.
Related
I have a (relatively small) Bazel rule for some configure/make based project, say xmlsec1 - you can take any other, the important thing seems to be the external tooling behind foreign_cc:
xmlsec1.BUILD:
load("#rules_foreign_cc//foreign_cc:defs.bzl", "configure_make")
filegroup(name="all_srcs", srcs=glob(["**"]))
configure_make(
name="xmlsec1",
lib_name="xmlsec1",
lib_source=":all_srcs",
configure_command="configure",
configure_in_place=True,
out_binaries=["xmlsec1"],
targets=["install"],
)
xmlsec1.bzl:
load("#bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
def xmlsec1():
http_archive(
name = "xmlsec1",
url = "https://www.aleksey.com/xmlsec/download/xmlsec1-1.2.37.tar.gz",
sha256 = "5f8dfbcb6d1e56bddd0b5ec2e00a3d0ca5342a9f57c24dffde5c796b2be2871c",
build_file = "#//:xmlsec1.BUILD",
)
All works fine for me until Bazel's remote cache gets activated and I'm building among different Linux distributions.
To avoid cache collisions I'm running bazel build with --action_env=SYSTEM_DIGEST="$(cat /etc/os-release)", resulting in different hashes on different distros.
While this approach seems to work for the artifacts defined in xmlsec1 (I can see this from the execution logs and observing expected re-builds), the foreign_cc part seems to built without those action_env variables.
This is what I get when I try to build #xmlsec1//:xmlsec1 (line breaks for readability):
+ /home/me/.cache/bazel/_bazel_me/8f6a55c898f3ec22f87d9cee5890b9e5/sandbox/processwrapper-sandbox/5/execroot/my_project_packages/bazel-out/k8-opt-exec-2B5CBBC6/bin/external/rules_foreign_cc/toolchains\
/make/bin/make install
/home/me/.cache/bazel/_bazel_me/8f6a55c898f3ec22f87d9cee5890b9e5/sandbox/processwrapper-sandbox/5/execroot/my_project_packages/bazel-out/k8-opt-exec-2B5CBBC6/bin/external/rules_foreign_cc/toolchains\
/make/bin/make: /lib64/libc.so.6: version `GLIBC_2.34' not found (required by /home/me/.cache/bazel/_bazel_me/8f6a55c898f3ec22f87d9cee5890b9e5/sandbox/processwrapper-sandbox/5/execroot/my_project_packages/bazel-out/k8-opt-exec-2B5CBBC6/bin/external/rules_foreign_cc/toolchains/make/bin/make)
/home/me/.cache/bazel/_bazel_me/8f6a55c898f3ec22f87d9cee5890b9e5/sandbox/processwrapper-sandbox/5/execroot/my_project_packages/bazel-out/k8-opt-exec-2B5CBBC6/bin/external/rules_foreign_cc/toolchains\
/make/bin/make: /lib64/libc.so.6: version `GLIBC_2.33' not found (required by /home/me/.cache/bazel/_bazel_me/8f6a55c898f3ec22f87d9cee5890b9e5/sandbox/processwrapper-sandbox/5/execroot/my_project_packages/bazel-out/k8-opt-exec-2B5CBBC6/bin/external/rules_foreign_cc/toolchains/make/bin/make)
I get this linker error only with bazel remote cache being activated and (this is the interesting part) having xmlsec1 built on a recent distribution (say Ubuntu 22.04) and then trying to build it on Centos-8.
So I guess this is what's going on:
make from foreign_cc get's built and linked against a recent version of GLIBC, ignoring values provided with --action_env
foreign_cc artifacts are being stored in the bazel remote cache
another build on an older distro (Centos-8) also tries to build make and has no reason not to take the artifacts from the cache, since --action_env values are also ignored here, resulting in the same hash
since the binaries are linked against a version of GLIBC which is not available yet on Centos-8 they are not compatible and crash with the error you see above.
So my question(s) is(/are):
Is that intended behavior? why is --action_env being ignored for builds Bazel runs implicitly (and not for those defined explicitly)?
Is there a way to apply those rules for Bazels own dependencies?
Is there a better way to define system properties with effects on all Builds?
Cytoscape appears to expect it can write to its installation directory. On Linux, we do shared installs of software such the installation directories are root-owned and read-only.
Upon launch, the message:
Unable to update instance pid: Unable to create directory /usr/software/cytoscape/framework/instances
is displayed. Is this something to be concerned about? Is there an option to specify a different, usable writable directory for the instance data?
Good news. This is not a critical error message. You can ignore it.
The Cytoscape dev team now has this as a bug to fix in future versions (thanks). We'll write that properties file to the CytoscapeConfiguration folder (where all other properties are written).
here is a copy of a few erros but the are to many to list in the post maybe 50 total all same error just different file names
fatal error: file '/azerothcore/src/server/game/Entities/GameObject/GameObject.h' has been modified since the precompiled header '/azerothcore/var/build/obj/src/server/game/CMakeFiles/game.dir/cmake_pch.hxx.pch' was built: size changed
note: please rebuild precompiled header '/azerothcore/var/build/obj/src/server/game/CMakeFiles/game.dir/cmake_pch.hxx.pch'
1 error generated.
make[2]: *** [src/server/game/CMakeFiles/game.dir/build.make:154: src/server/game/CMakeFiles/game.dir/AI/CoreAI/GuardAI.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
how can i do this step and rebuild my precompiled headers in docker build of ac? also my world database keeps bloating up in size huge amounts went from 8gb to 16gb to 28 gb since most recent new build yesterday files from 10/11/2021
Probably this is because of the ccache, which is the cache that allows you to have super-fast compilation, but sometime the header cache fails because of a high amount of changes, and it must be cleaned.
So, Please try this:
./acore.sh docker dev:dash compiler ccacheClean
it should clean the ccache, then you can restart the compilation
PS The problem about the database size is related to the configuration of MySQL. InnoDB usually stores a large amount of Binglogs that should be cleaned. But please open a separated question for that. AC provides the default MySQL configuration and the system admin should tune it based on its needs
the only way i was able to fix my problems was at a personal cost to hours spent with creating so much custom content, only to be lost. i rebuilt a clean server and then only imported my AUTH and CHARACTER databases, problems solved. the build is 100% working as intended, but will have to slowly and painfully redo all my custom work that was done in the WORLD database.
According to How does AOSP 9.0 build system link the executable? and What does # mean in this clang command in AOSP build log? , when linking a module, AOSP seems to produce a .rsp file that contains all the obj files that the module need,and pass the file name as a parameter to the link command, for example:
prebuilts/clang/host/linux-x86/clang-4691093/bin/clang++ /OpenSource/Build/Android/9.0.0_r30/soong/.intermediates/bionic/libc/crtbegin_so/android_x86_64_core/crtbegin_so.o #/OpenSource/Build/Android/9.0.0_r30/soong/.intermediates/frameworks/base/libs/hwui/libhwui/android_x86_64_core_shared/libhwui.so.rsp ......
But the .rsp files seems to be removed after build.
The question is, how are these file generated and how to get these files? This may require to learn and modify the build scripts which is out of reach for me.
There maybe the answer for you, read the ninja build manual , in that manual .rsp file mentioned.
https://ninja-build.org/manual.html
the following is info that I copy out.
rspfile, rspfile_content
if present (both), Ninja will use a response file for the given command, i.e. write the selected string (rspfile_content) to the given file (rspfile) before calling the command and delete the file after successful execution of the command.
This is particularly useful on Windows OS, where the maximal length of a command line is limited and response files must be used instead.
I have a test which tries to start an embedded mysql.
It tries to write a socket file to Files.createTempFile(null, ".sock") which returns with the following error:
mysql start failed with error: [ERROR] The socket file path is too long (> 103): /private/var/tmp/_bazel_user/1c8ed8d84f6cb79483aa3cc4da758c86/bazel-sandbox/2478112867584790357/execroot/some_workspace/_tmp/dfebe48cda4dfdc8739653efedfa4933/394798020705754292.sock.
I worked around it by re-pointing java temp dir to /tmp using jvm_flags but this doesn't work when I try to use sandboxing since I guess the test isn't allowed to write there.
I've tried setting a symbolic link from the java code (like so Files.createSymbolicLink(Paths.get("/tmp/foo"),Paths.get(System.getProperty("java.io.tmpdir"))) but this doesn't seem to help.
I've also tried setting the output_base but that didn't help either.
Would really appreciate pointers and tips since I currently can't run my tests under sandbox and so can't parallelize them.
I think your approach of re-pointing Java to /tmp should work. The macOS sandbox always allows writing to a number of directories and /tmp and /private/tmp are included in this set. I tried to reproduce the failure with a genrule: genrule(name = "write_to_tmp", outs = ["out.txt"], cmd = "touch /tmp/something.sock && touch $#"), which works fine and creates the file /tmp/something.sock.
I think in general using /tmp should work fine, although it does seem to be a bit unusual on macOS. $TMPDIR is set to a user-specific folder with a randomized name underneath /var/folders by the OS and it seems to be generally encouraged to use that instead of /tmp. But if you know what you're doing, I don't see a real problem.
Note that we don't have tmpfs or similar mechanisms available on macOS, so we can't automatically guarantee that your usage of writable folders like /tmp by actions is hermetic, won't leak state between runs or that file names won't conflict. Make sure to generate file names in a secure way via mkstemp or similar.
In a bazel test, you can use the TEST_TMPDIR environment variable for a test-private writeable area.
See https://docs.bazel.build/versions/master/test-encyclopedia.html