Failure compiling glog with gflags support using Bazel - bazel

I'm getting a failure when I try to compile glog with gflags support using Bazel. A github repo reproducing this problem and showing the compilation error message is here: https://github.com/dionescu/bazeltrunk.git
I suspect that the problem occurs because glog is finding and using the "config.h" file published by gflags. However, I do not understand why this happens and why the current structure of the build files results in such errors. One solution I found was to provide my own BUILD file for gflags where the config was in a separate dependency (just how glog does it in my example).
I would appreciate any help with understanding the issue in this example.

The problem is that gflag's BUILD file is including its own config. Adding -H to glog.BUILD's copts yields:
. external/glog_archive/src/utilities.h
.. external/glog_archive/src/base/mutex.h
... bazel-out/local-fastbuild/genfiles/external/com_github_gflags_gflags/config.h
In file included from external/glog_archive/src/utilities.h:73:0,
from external/glog_archive/src/utilities.cc:32:
external/glog_archive/src/base/mutex.h:147:3: error: #error Need to implement mutex.h for your architecture, or #define NO_THREADS
# error Need to implement mutex.h for your architecture, or #define NO_THREADS
^
If you take a look at gflag's config.h, it went with a not-very-helful approach of commenting out most of the config:
// ---------------------------------------------------------------------------
// System checks
// Define if you build this library for a MS Windows OS.
//cmakedefine OS_WINDOWS
// Define if you have the <stdint.h> header file.
//cmakedefine HAVE_STDINT_H
// Define if you have the <sys/types.h> header file.
//cmakedefine HAVE_SYS_TYPES_H
...
So nothing is defined.
Options:
The easiest way is probably to generate the config.h in your glog.BUILD:
genrule(
name = "config",
outs = ["config.h"],
cmd = "cd external/glog_archive; ./configure; cd ../..; cp external/glog_archive/src/config.h $#",
srcs = glob(["**"]),
)
# Then add the generated config to your glog target.
cc_library(
name = "glog",
srcs = [...],
hdrs = [
":config.h",
...
This puts the .h file at a higher-precedence location than the gflags version.
Alternatively, you could do something like this in the genrule, if you want to use your //third_party/glog/config.h (#// is shorthand for your project's repository):
genrule(
name = "config",
outs = ["config.h"],
cmd = "cp $(location #//third_party/glog:config.h) $#",
srcs = ["#//third_party/glog:config.h"],
)
You'll have to add exports_files(['config.h']) to the third_party/glog/BUILD file, too.

Related

With Bazel how do I make part of one genrules' source files (e.g. header files) available to another genrule?

Maybe this is a no-brainer and I just didn't get the concept yet.
I have a genrule, basically wrapping an existing make/config workflow to integrate it into a Bazel-based build configuration. In my example I'd like to build openssl, and then (with the same approach) some library depending on openssl, say xmlsec1
My (shortened) rule for openssl looks like this:
genrule(
name = "build",
visibility = ["//visibility:public"],
srcs = glob(["**/*"], exclude=["bazel-*"]),
outs = [
"libssl.a",
"libcrypto.a",
"include/openssl/opensslconf.h",
],
cmd = """
OUT_DIR="$$(realpath $(RULEDIR))"
pushd "$$(dirname $(location config))"
./config
make
make -j6 DESTDIR="$$OUT_DIR" install_sw install_ssldirs
"""
)
This builds fine and $OUT_DIR contains all files I need to build against openssl
I'd now like to create another genrule building xmlsec1 which needs the path to openssls header files.
Now if I want to access a header, say include/opensslv.h it won't be part of #openssl//:builds artifacts since I didn't explicitly listing it in outs. But doing so results in
ERROR: Traceback (most recent call last):
File "/bla/blubb/.cache/bazel/_bazel_me/f68917ddf601b6533d6db04f8101d580/external/openssl/BUILD.bazel", line 37, column 8, in <toplevel>
genrule(
Error in genrule: rule 'build' has file 'include/openssl/opensslv.h' as both an input and an output
which is correct of course, but what can I do about it?
Removing those header files from srcs doen't work neither, since they wouldn't be available at build time.
One way would be to make install openssl to some destination directory, listing each of the dozens of header files explicitly and using that prefix in all dependent projects. But that doesn't feel right.
What's the recommended way to pass lists of files from one genrule to another?
xmlsec1 could have include/openssl/opensslv.h in its own srcs directly. The build genrule shouldn't really need include/openssl/opensslv.h in its outs both because that would be a circular dependency as bazel said, and because the genrule doesn't really build that file: it already exists on disk (I assume it's getting captured by the glob())
There may be nicer way to organize the library though, something like this:
genrule(
name = "build_openssl",
visibility = ["//visibility:private"],
outs = [
"libssl.a",
"libcrypto.a",
"include/openssl/opensslconf.h",
],
.....,
)
cc_library(
name = "openssl",
srcs = [":build_openssl"],
hdrs = [
"include/openssl/opensslv.h",
# other headers that openssl should provide
],
)
then your other rules can depend on the openssl cc_library and get both the .a files and the header files. (I have not tested this though)

How to fail a Bazel build on a rule failure?

I am using Bazel rules in NodeJS in my application. The aim is to simply lint a set of files and fail the build if linting fails. What I'm currently experiencing is that the build is successful despite lint errors.
Here's a part of my BUILD file:
load("#npm//htmlhint:index.bzl", "htmlhint")
filegroup(
name = "htmldata",
srcs = glob(["**/*.html"]),
)
htmlhint(
name = "compile",
data = [
"htmlhint.conf",
"//:htmldata"
],
args = [
"--config",
"htmlhint.conf",
"$(locations //:htmldata)"
]
)
I first load the hinting library, then I define a filegroup for all the HTML files that I want to lint. Afterward, I use the rule with its data and arguments.
To run the build, I use the default option via npm script: bazel build //...
Your build file is working as expected. Unfortunately it doesn't do what you want, because when you load the macro from #npm//htmlhint:index.bzl it sets up the nodejs binary which is a runnable target, which means that it will only create runfiles + executable when building. In this case, the build will not run the library.
There are several options to do what you want:
Use the htmlhint_test macro to create a test target.
Create a custom rule that will use the nodejs binary to build some artefacts. In this case, you can force the build to fail.
However, I suggest using the first approach, because if htmlhint is a linting tool, it won't produce any meaningful outputs and is best to keep it as part of the test suite.
Here's what you need to do to set up the compile target as a test target
diff --git a/BUILD.bazel b/BUILD.bazel
index 4e58ac5..3db5dbb 100644
--- a/BUILD.bazel
+++ b/BUILD.bazel
## -1,11 +1,11 ##
-load("#npm//htmlhint:index.bzl", "htmlhint")
+load("#npm//htmlhint:index.bzl", "htmlhint_test")
filegroup(
name = "htmldata",
srcs = glob(["**/*.html"]),
)
-htmlhint(
+htmlhint_test(
name = "compile",
data = [
"htmlhint.conf",
Then you can check it with bazel test //....
If you want to see the output just run your compile target with bazel run //path/to:compile

Why does bazel fail with header not under the specified strip prefix

I am using bazel 3.7.2 (the same project works OK with bazel 3.3.1)
In my build file I use:
cc_library(
name = "xft",
hdrs = ["#nixpkgs_xft//:include"],
strip_include_prefix = "/external/nixpkgs_xft/include",
)
Running bazel build target, bazel complains:
BUILD:71:11: in cc_library rule //target:xft: header 'external/nixpkgs_xft/include/X11/Xft/Xft.h' is not under the specified strip prefix 'external/nixpkgs_xft/include'
Somehow bazel got a different understanding of a header's prefix... How is this supposed to work?
external/nixpkgs_xft/include/X11/Xft/Xft.h
external/nixpkgs_xft/include
OK, the error message is really confusing. I looked through the code and found that there is a different thing compared to the prefix than the header file: the repository relative path is compared to the prefix (but the error message prints the "exec path").
Seems in newer Bazel, you do no longer need to include the exernal/foo part of the path, so this works:
cc_library(
name = "xft",
hdrs = ["#nixpkgs_xft//:include"],
strip_include_prefix = "/include",
)

"undeclared inclusion(s)" in bazel with headers in subdirectories

I'm trying to use ArduinoJson in a C++ bazel project. It's a header only library with headers in subdirectories under src. But I get an undeclared inclusion(s) error.
This is how the //Third-Party/ArduinoJson cc_library def looks like:
cc_library(
name = "ArduinoJson",
hdrs = glob(["5.12.0/src/**"]),
includes = [
"5.12.0/src",
"5.12.0/src/ArduinoJson",
],
visibility = ["//visibility:public"],
)
The target that uses it (Serialization) does have //Third-Party/ArduinoJson in it's deps
Here is the error:
ERROR: /[...]/Serialization/BUILD:1:1: undeclared inclusion(s) in rule '//[...]/Serialization:Serialization':
this rule is missing dependency declarations for the following files included by '[...]/Serialization/JsonDeserializer.cpp':
'[...]/Third-Party/ArduinoJson/5.12.0/src/ArduinoJson/Data/Encoding.hpp'
'[...]/Third-Party/ArduinoJson/5.12.0/src/ArduinoJson/Serialization/FloatParts.hpp'
'[...]/Third-Party/ArduinoJson/5.12.0/src/ArduinoJson/Polyfills/math.hpp'
'[...]/Third-Party/ArduinoJson/5.12.0/src/ArduinoJson/TypeTraits/FloatTraits.hpp'
Target //[...]/Serialization:Serialization failed to build
Since it is only complaining about files in subdirectories of src, e.g. src/Data/Encoding.hpp, I'm guessing that might have something to do with it?
This is not a duplicate of "How to resolve bazel “undeclared inclusion(s)” error?" since that one is solved by doing what I'm already doing with deps.
Make sure there are no BUILD files in the subdirectories of Third-Party/ArduinoJson, otherwise Bazel regards those as packages and the glob won't pick up files from them.
If you do have BUILD files in those subdirectories, you need to create cc_library rules in them, exporting the headers that you wanted in //Third-Party/ArduinoJson:ArduinoJson, and depending on these cc_libraries from ArduinoJson.

Bazel cc_library dependency on other cc_library when each compile with a different crosstool

I have a code generator tool that generates C/C++ code. This code generator tool is compiled with crosstool1. The generated C/C++ code needs to be compiled with crosstool2.
So the actions are:
Using Crosstool1 compile 'code_generator'.
Execute 'code_generator' and generate 'generated_code.cpp'
Using Crosstool2 compile 'generated_code.cpp'
Is it possible to make a cc_library() determine the crosstool to use? I saw that Skylark rules now allow a 'toolchains' parameter which I'm not sure how this is used, also I do not want to do the heavy lifting of C/C++ compiling bare bone with Skylark.
Is there an example of using a proper Host Crosstool and Target Crosstool except for the Tenserflow example? I get a headache each time I read it :D
Assume //crosstool1:toolchain is a label for cc_toolchain_suite rule describing first crosstool, //crosstool2:toolchain is a label for cc_toolchain_suite for second crosstool, and the build file for the project is:
cc_binary(
name = "generator",
srcs = [ "main.cc" ],
)
genrule(
name = "generate",
outs = ["generated.cc"],
cmd = "$(location :generator) > $#",
tools = [":generator"],
)
cc_binary(
name = "generated",
srcs = [ "generated.cc" ],
)
Then running:
bazel build --host_crosstool_top=//crosstool1:toolchain --crosstool_top=//crosstool2:toolchain :generated
will do exactly what you describe, it will use crosstool1 to build :generator, and crosstool2 to build generated. Genrules use host configuration by default, so all should just work.

Resources