Building a simple library with bazel, fixing include path - bazel

I have a very simple directory structure:
.
├── libs
│   └── foo
│   ├── BUILD
│   ├── include
│   │   └── foo
│   │   └── func.h
│   └── src
│   └── func.cxx
└── WORKSPACE
With func.h:
#pragma once
int square(int );
And func.cxx:
#include <foo/func.h>
int square(int i) { return i * i; }
And BUILD:
cc_library(
name = "foo",
srcs = ["src/func.cxx"],
hdrs = ["include/foo/func.h"],
visibility = ["//visibility:public"],
)
This fails to build:
$ bazel build //libs/foo
INFO: Analysed target //libs/foo:foo (0 packages loaded).
INFO: Found 1 target...
ERROR: /home/brevzin/sandbox/bazel/libs/foo/BUILD:1:1: C++ compilation of rule '//libs/foo:foo' failed (Exit 1)
libs/foo/src/func.cxx:1:22: fatal error: foo/func.h: No such file or directory
#include <foo/func.h>
^
compilation terminated.
Target //libs/foo:foo failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 0.299s, Critical Path: 0.02s
FAILED: Build did NOT complete successfully
How do I set the include path properly? I tried using include_prefix (whether include or include/foo) but that didn't change the behavior.

Hmm, the tricky part about including headers from another places is that you have to specify the header file from its relative location according to the workspace (where the WORKSPACE file resides).
Moreover, you should not use the angle-bracket including style #include <a/b.h> unless you are including the system header files.
The related specifications for #include can be found here: https://docs.bazel.build/versions/master/bazel-and-cpp.html#include-paths
TL;DR The only change you need to make to your func.cxx file is that, change the first line to #include "libs/foo/include/foo/func.h".
And then, when you run bazel build //... (build all targets in this workspace, similar to make makes all) from the root of the workspace, you will encounter no error.
However, this is not the only way that can solve your problem.
Another way to resolve this issue that does not involve changing the source code include statement would be specifying the include path in the attribute of the rule cc_library.
Which means, you can make changes to your BUILD in path libs/foo to make it look like this:
cc_library(
name = "foo",
srcs = ["src/func.cxx"],
hdrs = ["include/foo/func.h"],
copts = ["-Ilibs/foo/include"], # This is the key part
visibility = ["//visibility:public"],
)
After this change, the compiler will be able to figure out where to find the header file(s), and you don't have to change your source code, yay.
Related information can be found here: https://docs.bazel.build/versions/master/cpp-use-cases.html#adding-include-paths
Nevertheless, there is another hacky way to solve your problem, however, it involves in more changes to your code.
It makes uses of the rule cc_inc_library.
The rule cc_inc_library will strips prefix attribute passed to this rule from the relative path of the header files specified in the hdrs attribute.
The example on the website is a little confusing, your code and directory structure will yield a much better demonstration purpose.
In your case, you have to modify your BUILD file under libs/foo to something that looks like this:
cc_library(
name = "foo",
srcs = ["src/func.cxx"],
deps = [":lib"],
copts = ["-Ilibs/foo/include"],
visibility = ["//visibility:public"],
)
cc_inc_library(
name = "lib",
hdrs = ["include/foo/func.h"],
prefix = "include/foo",
)
In your case, the header file func.h which has a relative path from the package libs/foo as include/foo/func.h, which is specified in the hdrs attribute.
Since it has a relative path to the root of the workspace as libs/foo/include/foo/func.h, and the prefix attribute in the cc_inc_library is specified as include/foo: the value include/foo will be stripped form lib/foo/include/foo/func.h, making it libs/foo/func.h.
So, now you can include this header file in your func.cxx as #include "libs/foo/func.h".
And now, bazel will not report error saying that it was not able to find the header file.
You can find the information about this rule at: https://docs.bazel.build/versions/master/be/c-cpp.html#cc_inc_library.
However, as stated above, the explanation is confusing at best, possibly because the documentation for it is out of date.
I was puzzled by the office explanation on bazel.build for quite some time until I read the source code for this rule at: https://github.com/bazelbuild/bazel/blob/f20ae6b20816df6a393c6e8352befba9b158fdf4/src/main/java/com/google/devtools/build/lib/rules/cpp/CcIncLibrary.java#L36-L50
The comment for the actual code that implements the function does a much, much better job at explaining what this rule actually does.
cc_inc_library rule has been deprecated since Bazel release 0.12.
Use cc_library approach instead.
See: https://blog.bazel.build/2018/04/11/bazel-0.12.html

What you really want here is strip_include_prefix:
cc_library(
name = "foo",
srcs = ["src/func.cxx"],
hdrs = ["include/foo/func.h"],
# Here!
strip_include_prefix = "include",
visibility = ["//visibility:public"],
)
This will make the headers accessible via:
#include "foo/func.h"
This attribute has been available since at least Bazel 0.17.

Related

Unable to use a hermetic python in a Bazel Workspace as an external dependency

I've been working on a small project trying to teach myself how to use Bazel. The goal is to download a http_archive that contains a python interpreter. once the python interpreter has been added to the environment I want to run the command `python.exe --version and write the output of that into a file
The issue I have the most difficulty with at the moment are the following:
I am not confident that I am correctly injecting the hermetic python BUILD file into the hermetic python package (I keep getting the message "BUILD file not found in any of the following directories. Add a BUILD file to a directory to mark it as a package").
I'm pretty sure that when I pass in python_compiler = ["#hermetic_python"] in the BUILD file I'm just getting a string and not a reference to the files in the package
Here is an overview of my project and the code files. Any help would be appreciated! :D
Project structure:
|-- WORKSPACE
|-- BUILD
|-- custom_rules.bzl
|-- main.py
|-- custom-rules/
|-- BUILD.custom_python
|-- custom_python_rules.bzl
WORKSPACE
load("#bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
http_archive(
name = "hermetic_python",
urls = ["https://www.python.org/ftp/python/3.9.10/python-3.9.10-embed-amd64.zip"],
sha256 = "67161cab713a52f6658b76274f8dbe0cd2f256aab1e84f94cd557d4a906fa5d2",
build_file = "#//:custom-rules/BUILD.custom_python"
)
BUILD File:
load("//:custom_rules.bzl","build_with_custom_python")
build_with_custom_python(
name = "write-to-file",
python_compiler = ["#hermetic_python"]
)
custom_rules.bzl
def _build_with_custom_python_impl(ctx):
out_file = ctx.actions.declare_file("file_with_python_version.txt")
ctx.actions.run(
outputs = [out_file]
executable = ctx.attr.python_compiler,
arguments = [--version],
)
return DefaultInfo(files=[out_file])
build_with_custom_python = rule(
implementation = _build_with_custom_python_impl,
attrs = {
"python_compiler": attr.label_list(allow_files=True)
}
)
BUILD.custom_python
load("//:custom_python_rules.bzl","run_me")
run_me(
name="my_py_run",
python_files = glob(["**"]),
visibility = ["//visibility:public"],
)
custom_python_rules.bzl
def _run_me_impl(ctx):
pass
run_me = rule (
implementation = _run_me_impl,
attrs = {
"python_files" : attr.label_list(allow_files=True),
}
)
I've spent some more time on this I've managed to mostly do what I intended to do. Here is an overview of what I've learned that fixes the problems in the original question
1 - Mark the custom build file / custom rule folder as a package
Just by adding an empty BUILD file into the custom-rules folder I had market it as a bazel package. That allowed me to reference the files
|-- WORKSPACE
|-- BUILD
|-- custom_rules.bzl
|-- main.py
|-- custom-rules/
|-- BUILD # NEW
|-- BUILD.custom_python
|-- custom_python_rules.bzl
2 - The workspace file references a BUILD file. That build file needs to reference a custom rules file
The workspace should just reference the build file directly // means workspace root and then its just the path to the BUILD file
http_archive(
# build_file = "#//:custom-rules/BUILD.custom_python" // WRONG
build_file = "//custom-rules/BUILD.custom_python" // RIGHT
)
3 - BUILD file can not reference the custom rules (bzl file)
This is the one that took the longest time for me to figure out! The BUILD file that is loaded into the package (hermetic_python from the workspace file) NEEDS TO reference the current workspace in order to
# BUILD.custom_python
load("#root_workspace//custom-rules:custom-python-rules.bzl","execute_python_file")
Notice that this starts with #root_workspace which tells the BUILD file to look in that workspace. the WORKSPACE file in the project now contains this line
# WORKSPACE
workspace(name = "root_workspace")
4 - Passing in a build step as the dependancy
build_with_custom_python(
name = "write-to-file",
# python_compiler = ["#hermetic_python"] WRONG
python_compiler = ["#hermetic_python:my_py_run"] RIGHT
)
The key here is that I need to reference a build action which then makes the files in that build action available through the dependency. In this case #hermetic_python is the package and :my_py_run is the build action.
I still need to figure out how to properly use the files in the dependency but that's outside of the scope of this question

Up-level references ("..") in Bazel

In my bazel BUILD file, I have a line:
srcs = glob([<pattern1>, <pattern2>, ...])
I tried to have one of my patterns be "../dir/*.cc" but I get an error that I'm not allowed to use the .. sequence here.
Checking the documentation, I have found that it's not permitted, but I'm not sure what the expected substitute is.
Similarly, up-level references (..) and current-directory references (./) are forbidden.
How can I include these other source files in my srcs list given my current file structure? If I can't reference the up-level directory, is there a way to use the package name of the other directory instead?
Going "up" from your BUILD file would violate the package boundaries. If you really need that structure and cannot or don't want to change it, you have to make files from one package available to the other package by declaring the corresponding target(s) or at least export the files and making those visible. For instance assuming the following structure:
.
├── BUILD
├── WORKSPACE
├── hello.c
└── tgt
└── BUILD
It the // (top-level) package BUILD I could say:
filegroup(
name = "hello",
srcs = ["hello.c"],
visibility = ["//tgt:__pkg__"],
)
(Could also be: exports_files(["hello.c"], ["//tgt:__pkg__"]) instead in which case I would refer to the file by its name //:hello.c from tgt.)
And inside //tgt (tgt/BUILD) it can then read:
cc_binary(
name="tgt",
srcs=["//:hello"],
)
Which would give me:
$ bazel run //tgt
WARNING: /tmp/bzl1/tgt/BUILD:3:10: in srcs attribute of cc_binary rule //tgt:tgt: please do not import '//:hello.c' directly. You should either move the file to this package or depend on an appropriate rule there
INFO: Analyzed target //tgt:tgt (11 packages loaded, 68 targets configured).
INFO: Found 1 target...
Target //tgt:tgt up-to-date:
bazel-bin/tgt/tgt
INFO: Elapsed time: 0.247s, Critical Path: 0.09s
INFO: 2 processes: 2 linux-sandbox.
INFO: Build completed successfully, 6 total actions
INFO: Build completed successfully, 6 total actions
Hello World!
Note: bazel still flags this as something weird and noteworthy going on. I have to say I do not disagree with it. The tree structure does not seem to correspond to the content very well.
Perhaps in this example the tgt package boundary is artificial and not actually useful? Or hello.c is in the wrong place.

How to resolve a dependency in an external package workspace_file?

Trying to use a target in the build_file from an external package imported through http_archive that has dependencies defined in the external package workspace via the workspace_file attribute fails. I'm using Bazel 0.27.0 on Debian Testing.
The documentation only talks about referencing targets in the provided build_file, but I could not find any information how one could reference a dependency defined in the provided workspace_file in the provided build_file.
The usual #stringtemplate3//jar syntax fails, but I don't know how I could include a reference to the imported archive which according to the manual would have to start with #antlr3_runtimes.
The project layout looks like this:
├── antlr.BUILD
├── antlr.WORKSPACE
├── BUILD
├── external_dependency
│ └── src
│ └── main
│ └── java
│ └── bazel
│ ├── BUILD
│ └── Hello.java
├── LICENSE
└── WORKSPACE
The WORKSPACE definition looks like this:
workspace(name="bazel")
load("#bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
http_archive(
name = "antlr3_runtimes",
sha256 = "d4f7d3c38c5523f8009ff37528e5797c81adb454be6acc9af507cfcb41f2016f",
strip_prefix = "antlr3-master",
urls = ["https://github.com/ibre5041/antlr3/archive/master.tar.gz"],
build_file = "#//:antlr.BUILD",
workspace_file = "#//:antlr.WORKSPACE",
)
It seems that the provided workspace_file is not even analyzed. The build already fails when it tries to resolve the dependencies in the custom build file.
A repro can be found here: https://github.com/marcohu/bazel
bazel build //... shows this error message:
ERROR: /home/user/.cache/bazel/_bazel_user/64492308e78c9898c41f12c18dd29b63/external/antlr3_runtimes/BUILD.bazel:43:1: no such package '#stringtemplate3//jar': The repository '#stringtemplate3' could not be resolved and referenced by '#antlr3_runtimes//:antlr3_tool'
ERROR: Analysis of target '//external_dependency/src/main/java/bazel:hello' failed; build aborted: no such package '#stringtemplate3//jar': The repository '#stringtemplate3' could not be resolved
I reported this in the Bazel issue tracker, but it got rejected with a hint to post here.
Is this use case something that is just not possible? Or did I got the syntax wrong?
At least as of now (I suppose this statement may change in the future versions), bazel does not directly support transitive external dependencies. WORKSPACE file would still get read in even in your case and if it contained entirely broken syntax it'd still fail, but it does not get "acted upon" and you could (currently) for instance load from non-existing labels or call undefined functions and would still be left none the wiser for a nested WORKSPACE.
You essentially have two options:
Repeat your nested dependencies (http_archive rules) in your "parent"/top WORKSPACE.
You can define a function(s) with corresponding repository rules that you load and call in your "parent"/top WORKSPACE.
Bazel actually does support the transitive fetching of jvm dependencies. https://github.com/bazelbuild/rules_jvm_external
WORKSPACE
load("#bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
rules_jvm_external_tag = "2.0.1"
rules_jvm_external_sha = "55e8d3951647ae3dffde22b4f7f8dee11b3f70f3f89424713debd7076197eaca"
http_archive(
name = "rules_jvm_external",
sha256 = rules_jvm_external_sha,
strip_prefix = "rules_jvm_external-%s" % rules_jvm_external_tag,
url = "https://github.com/bazelbuild/rules_jvm_external/archive/%s.zip" % rules_jvm_external_tag,
)
load("#rules_jvm_external//:defs.bzl", "maven_install")
maven_install(
name = "maven",
artifacts = [
"io.grpc:grpc-netty-shaded:1.22.1",
"io.grpc:grpc-api:1.22.1",
"io.grpc:grpc-testing:1.22.1",
"io.grpc:grpc-core:1.22.1",
"junit:junit:4.12",
],
repositories = [
"https://jcenter.bintray.com/",
"https://repo1.maven.org/maven2",
],
)
BUILD
...
java_library(
name = "hyphenation-service",
srcs = ["src/test/java/com/example/hyphenation/HyphenationServiceTest.java"],
deps = ["#maven//:io_grpc_grpc_core"]
)
...
Example repository https://github.com/mancini0/bazel-grpc-playground

What does the "#" symbol mean in Bazel?

I'm studying Bazel building system at present. I always see the # symbol in Bazel script, but I cannot find any documentation about it. I searched it on the website of Bazel but the result seems useless.
# in Bazel.
For example:
filegroup(
name = "toolchain_fg",
srcs = [
":cc-compiler-amd64",
"#x86_64_unknown_linux_gnu_gcc_730//:compiler_components",
],
)
Could anybody explain the # symbol here for me?
This is to reference a remote repository.
From the doc, depending on other Bazel projects
local_repository(
name = "coworkers_project",
path = "/path/to/coworkers-project",
)
If your coworker has a target //foo:bar, your project can refer to it
as #coworkers_project//foo:bar.
See also the design doc of remote repository and bind example in workspace rules.
In Bazel, targets are referred by labels.
Bazel labels have the following format:
#repoName//packageName:target
For example, in the following packages found in myRepo:
myRepo
├── WORKSPACE
├── package1
│ └── BUILD
│ └── src
└── package2
├── BUILD
└── src
a target called myTarget in package1/BUILD can be labeled as as #myRepo//package1:myTarget globally.
If referenced from the same repo, for example from package2/BUILD, then the #myRepo prefix can be omitted and you can use //package1:myTarget.
If referenced from the same package, for example in another target from package1/BUILD, then the package name can be omitted and you can use :myTarget. The colon can also be omitted if it does not create confusion with a name. Such short form labels should not be confused with the names. Labels start with '//' or ':'. But names never do. For example, the name of the first package is package1 but its label is //package1.
Reference: https://docs.bazel.build/versions/master/build-ref.html

Error with multiple Bazel BUILD files: "Target 'bar' is not visible from target 'foo'"

My project as the following structure:
$ tree
.
├── bar
│   ├── bar.cpp
│   └── BUILD
├── BUILD
├── foo.cpp
└── WORKSPACE
Content of ./BUILD:
cc_binary(
name = "foo",
srcs = [ "foo.cpp" ],
deps = [ "//bar" ],
)
Content of bar/BUILD:
cc_library(
name = "bar",
srcs = ["bar.cpp"],
)
If I build foo, I get the following error:
Target '//bar:bar' is not visible from target '//:foo'. Check the visibility declaration of the former target if you think the dependency is legitimate.
What do I need to do so the dependency can be resolved and foo is built successfully?
From the Bazel docs:
However, by default, build rules are private. This means that they can only be referred to by rules in the same BUILD file.
[...]
You can make a rule visibile to rules in other BUILD files by adding a visibility = level attribute.
In this case, bar/BUILD should look as follows:
cc_library(
name = "bar",
srcs = ["bar.cpp"],
visibility = ["//__pkg__"],
)
The additional line visibility = ["//__pkg__"] allows all BUILD files in the current WORKSPACE to access the target bar.
visibility = ["//__pkg__"] didn't work for me.
But I've managed to make it work by adding
package(default_visibility = ["//visibility:public"])
as the first line of the bar/BUILD file.

Resources