I'm following a tutorial on adding custom toolchains: https://bazel.build/tutorials/cc-toolchain-config
At step 6 I get errors:
ERROR: ... : name 'cc_common' is not defined
ERROR: ... : name 'rule' is not defined
ERROR: ... : name 'CcToolchainConfigInfo' is not defined
Step 6:
def _impl(ctx):
return cc_common.create_cc_toolchain_config_info(
ctx = ctx,
toolchain_identifier = "k8-toolchain",
host_system_name = "local",
target_system_name = "local",
target_cpu = "k8",
target_libc = "unknown",
compiler = "clang",
abi_version = "unknown",
abi_libc_version = "unknown",
)
cc_toolchain_config = rule(
implementation = _impl,
attrs = {},
provides = [CcToolchainConfigInfo],
)
Bazel version 4.1.0.
I see the global functions at [https://bazel.build/rules/lib/globals], but there is no indication I would have to load them, or how to even do that.
What did I miss?
Found out why after asking friends.
Bazel has a sort of declaration-definition file distinction. Functions like rule belong in definition files.
I got it working by moving the affected sections from declaration file (BUILD) to a definition file (.bzl), expose contents by macros etc, and finally have declaration file load it and use it.
filegroup similarly only works in declaration file, not definition.
Related
I have a rule like this
do_action = rule (
implementation = _impl,
attrs = {
...
"_cc_toolchain": attr.label(default = Label("#bazel_tools//tools/cpp:current_cc_toolchain")),
},
fragments = ["cpp"],
toolchains = [
"#bazel_tools//tools/cpp:toolchain_type",
],
)
I define custom cc_toolchain for a custom cpu:
toolchain(
name = "cc-toolchain-%{toolchain_name}",
toolchain = ":cc-compiler-%{toolchain_name}",
# can be run on this platform
target_compatible_with = [
"#platforms//os:windows",
"#platforms//cpu:x86_64",
],
toolchain_type = "#bazel_tools//tools/cpp:toolchain_type",
)
cc_toolchain_suite(
name = "toolchain",
toolchains = {
"%{cpu}": ":cc-compiler-%{toolchain_name}",
},
)
I use --crostool_top to select this toolchain when needed.
I want to allow my custom rule to be invoked only if --crostool_top matches one of my custom toolchains. How to do this?
Add a new constraint_setting with a constraint_value which only your toolchains are target_compatible_with, and then make all targets which use your rules target_compatible_with it.
Something like this in a BUILD file:
constraint_setting(name = "is_my_toolchain")
constraint_value(
name = "yes_my_toolchain",
constraint_setting = ":is_my_toolchain",
)
And then add yes_my_toolchain to target_compatible_with on all your toolchains.
Easiest way to force using it with every usage of your rule is with a macro. Rename the actual rule to _do_action (so it's private and can't be loaded directly from any BUILD file) and add:
def do_action(target_compatible_with = [], **kwargs):
_do_action(
target_compatible_with = target_compatible_with + [
"//your_package:yes_my_toolchain",
],
**kwargs
)
I'm trying to build a rule for bazel which emulates the CMake *.in template system.
This has two challenges, the first is generate the template output. The second is make the output available to both genrules, filegroups and cc_* rules. The third is to get that dependency to transitively be passed to further downstream rules.
I have it generating the output file version.hpp in genfiles (or bazel-bin), and I can get the initial library rule to include it, but I can't seem to figure out how to make my cc_binary rule, which depends on the cc_library and transitively the header_template rule to find the header file.
I have the following .bzl rule:
def _header_template_impl(ctx):
# this generates the output from the template
ctx.actions.expand_template(
template = ctx.file.template,
output = ctx.outputs.out,
substitutions = ctx.attr.vars,
)
return [
# create a provider which says that this
# out file should be made available as a header
CcInfo(compilation_context=cc_common.create_compilation_context(
headers=depset([ctx.outputs.out])
)),
# Also create a provider referencing this header ???
DefaultInfo(files=depset(
[ctx.outputs.out]
))
]
header_template = rule(
implementation = _header_template_impl,
attrs = {
"vars": attr.string_dict(
mandatory = True
),
"extension": attr.string(default=".hpp"),
"template": attr.label(
mandatory = True,
allow_single_file = True,
),
},
outputs = {
"out": "%{name}%{extension}",
},
output_to_genfiles = True,
)
elsewhere I have a cc_library rule:
load("//:tools/header_template.bzl", "header_template")
# version control
BONSAI_MAJOR_VERSION = '2'
BONSAI_MINOR_VERSION = '0'
BONSAI_PATCH_VERSION = '9'
BONSAI_VERSION = \
BONSAI_MAJOR_VERSION + '.' + \
BONSAI_MINOR_VERSION + '.' + \
BONSAI_PATCH_VERSION
header_template(
name = "bonsai_version",
extension = ".hpp",
template = "version.hpp.in",
vars = {
"#BONSAI_MAJOR_VERSION#": BONSAI_MAJOR_VERSION,
"#BONSAI_MINOR_VERSION#": BONSAI_MINOR_VERSION,
"#BONSAI_PATCH_VERSION#": BONSAI_PATCH_VERSION,
"#BONSAI_VERSION#": BONSAI_VERSION,
},
)
# ...
private = glob([
"src/**/*.hpp",
"src/**/*.cpp",
"proto/**/*.hpp",
])
public = glob([
"include/*.hpp",
":bonsai_version",
])
cc_library(
# target name matches directory name so you can call:
# bazel build .
name = "bonsai",
srcs = private,
hdrs = public,
# public headers
includes = [
"include",
],
# ...
deps = [
":bonsai_version",
# ...
],
# ...
)
When I build, my source files need to be able to:
#include "bonsai_version.hpp"
I think the answer involves CcInfo but I'm grasping in the dark as to how it should be constructed.
I've already tried add "-I$(GENDIR)/" + package_name() to the copts, to no avail. The generated header still isn't available.
My expectation is that I should be able to return some kind of Info object that would allow me to add the dependency in srcs. Maybe it should be a DefaultInfo.
I've dug through the bazel rules examples and the source, but I'm missing something fundamental, and I can't find documentation that discuss this particular.
I'd like to be able to do the following:
header_template(
name = "some_header",
extension = ".hpp",
template = "some_header.hpp.in",
vars = {
"#SOMEVAR#": "value",
"{ANOTHERVAR}": "another_value",
},
)
cc_library(
name = "foo",
srcs = ["foo.src", ":some_header"],
...
)
cc_binary(
name = "bar",
srcs = ["bar.cpp"],
deps = [":foo"],
)
and include the generated header like so:
#include "some_header.hpp"
void bar(){
}
The answer looks like it is:
def _header_template_impl(ctx):
# this generates the output from the template
ctx.actions.expand_template(
template = ctx.file.template,
output = ctx.outputs.out,
substitutions = ctx.attr.vars,
)
return [
# create a provider which says that this
# out file should be made available as a header
CcInfo(compilation_context=cc_common.create_compilation_context(
# pass out the include path for finding this header
includes=depset([ctx.outputs.out.dirname]),
# and the actual header here.
headers=depset([ctx.outputs.out])
))
]
elsewhere:
header_template(
name = "some_header",
extension = ".hpp",
template = "some_header.hpp.in",
vars = {
"#SOMEVAR#": "value",
"{ANOTHERVAR}": "another_value",
},
)
cc_library(
name = "foo",
srcs = ["foo.cpp"],
deps = [":some_header"],
...
)
cc_binary(
name = "bar",
srcs = ["bar.cpp"],
deps = [":foo"],
)
If your header has a generic name (eg config.h) and you want it to be private (ie srcs instead of hdrs), you might need a different approach. I've seen this problem for gflags, which "leaked" config.h and affected libraries that depended on it (issue).
Of course, in both cases, the easiest solution is to generate and commit header files for the platforms you target.
Alternatively, you can set copts for the cc_library rule that uses the generated private header:
cc_library(
name = "foo",
srcs = ["foo.cpp", "some_header.hpp"],
copts = ["-I$(GENDIR)/my/package/name"],
...
)
If you want this to work when your repository is included as an external repository, you have a bit more work cut out for you due to bazel issue #4463.
PS. You might want to see if cc_fix_config from https://github.com/antonovvk/bazel_rules works for you. It's just a wrapper around perl but I found it useful.
In my Java-based Bazel project I use a code generator written in Java. The generator is part of the root project, and used in sub-projects as well.
What I want to achieve is to include the output of the root project (a .jar file) as a dependency for the code generation in sub-projects to grant the code generator access to all compiled files of the root project (through the classpath). Is that possible in Bazel?
What I see is that the classpath for key generation in the child project only includes the dependencies of the code generator binary (//parent:SettingsGenerator in the script below).
In my custom rule I invoke the code generator basically like this:
def _generate_settings(ctx):
...
ctx.actions.run(
inputs = [ctx.file._src] + [ctx.files.deps],
outputs = [keys, settings, loader],
mnemonic = "GenKeysAndSettings",
arguments = [args],
executable = ctx.executable._tool,
)
return [DefaultInfo(
files=depset([keys, settings, loader]),
runfiles=ctx.runfiles(files=ctx.files.deps)
)]
gen_settings = rule(
implementation = _generate_settings,
attrs = {
"lang": attr.string(),
"deps": attr.label_list(
allow_files = True
),
"_tool": attr.label(
cfg = "host",
executable = True,
default = Label("//parent:SettingsGenerator"),
),
"_src": attr.label(
single_file = True,
default = Label("//parent:Name")
),
}
)
The parent project BUILD:
load("//parent:settings.bzl", "gen_settings")
gen_settings(
name = "GenerateSettings",
lang = ""
)
java_library(
name = "parent",
srcs = glob(["src/main/java/**/*.java"]) + [
":GenerateSettings",
],
...
)
java_binary(
name = "SettingsGenerator",
srcs = glob(["src/main/java/**/SettingsGenerator.java"]),
main_class = "my.company.SettingsGenerator",
...
)
The child project BUILD:
gen_settings(
name = "GenerateSettings",
lang = "Java",
deps = ["//parent"]
)
...
My workaround is to include the .jar file as input and use a custom classloader in the generator. But it would be nice if I could control the classpath directly from Bazel.
Any help would be appreciated. Thank you.
I'm working on my first custom Bazel rules. The rules allow the running of bats command line tests.
I've included the rule definition below verbatim. I'm pretty happy with it so far but there's one part which feels really ugly and non-standard. If the rule user adds a binary dependency to the rule then I make sure that the binary appears on the PATH so that it can be tested. At the moment I do this by making a list of the binary paths and then appending them with $PWD which is expanded inside the script to the complete execution path. This feels hacky and error prone.
Is there a more idiomatic way to do this? I don't believe I can access the execution path in the rule due to it not being created until the execution phase.
Thanks for your help!
BATS_REPOSITORY_BUILD_FILE = """
package(default_visibility = [ "//visibility:public" ])
sh_binary(
name = "bats",
srcs = ["libexec/bats"],
data = [
"libexec/bats-exec-suite",
"libexec/bats-exec-test",
"libexec/bats-format-tap-stream",
"libexec/bats-preprocess",
],
)
"""
def bats_repositories(version="v0.4.0"):
native.new_git_repository(
name = "bats",
remote = "https://github.com/sstephenson/bats",
tag = version,
build_file_content = BATS_REPOSITORY_BUILD_FILE
)
BASH_TEMPLATE = """
#!/usr/bin/env bash
set -e
export TMPDIR="$TEST_TMPDIR"
export PATH="{bats_bins_path}":$PATH
"{bats}" "{test_paths}"
"""
def _dirname(path):
prefix, _, _ = path.rpartition("/")
return prefix.rstrip("/")
def _bats_test_impl(ctx):
runfiles = ctx.runfiles(
files = ctx.files.srcs,
collect_data = True,
)
tests = [f.short_path for f in ctx.files.srcs]
path = ["$PWD/" + _dirname(b.short_path) for b in ctx.files.deps]
sep = ctx.configuration.host_path_separator
ctx.file_action(
output = ctx.outputs.executable,
executable = True,
content = BASH_TEMPLATE.format(
bats = ctx.executable._bats.short_path,
test_paths = " ".join(tests),
bats_bins_path = sep.join(path),
),
)
runfiles = runfiles.merge(ctx.attr._bats.default_runfiles)
return DefaultInfo(
runfiles = runfiles,
)
bats_test = rule(
attrs = {
"srcs": attr.label_list(
allow_files = True,
),
"deps": attr.label_list(),
"_bats": attr.label(
default = Label("#bats//:bats"),
executable = True,
cfg = "host",
),
},
test = True,
implementation = _bats_test_impl,
)
This should be easy to support from Bazel 0.8.0 which will be released in ~2 weeks.
In your skylark implementation you should do ctx.expand_location(binary) where binary should be something like $(execpath :some-label) so you might want to just format the label you got from the user with the $(execpath) and bazel will make sure to give you the execution location of that label.
Some relevant resources:
$location expansion in Bazel
https://github.com/bazelbuild/bazel/issues/2475
https://github.com/bazelbuild/bazel/commit/cff0dc94f6a8e16492adf54c88d0b26abe903d4c
We are generating a number of Go source files as part of our build. Previously we used a genrule (example here) which resulted in the generated files being stored in bazel-genfiles/.
We recently switched to using a custom rule as demonstrated in rules_go (https://github.com/bazelbuild/rules_go/tree/master/examples/bindata). This change means that the output source files are stored in bazel-bin/ instead of bazel-genfiles/.
This change of output location has broken Go integration in some of the IDEs used by our developers. Notably, gocode, the autocompletion engine used by vim-go and VSCode, when running in bzl (Bazel) lookup mode seems to expect to find generated sources in bazel-genfiles/, not bazel-bin/, and therefore fails.
How do I modify my rule to save the output to bazel-genfiles/ instead of bazel-bin/? My rule is equivalent to the example in rules_go:
def _bindata_impl(ctx):
out = ctx.new_file(ctx.label.name + ".go")
ctx.action(
inputs = ctx.files.srcs,
outputs = [out],
executable = ctx.file._bindata,
arguments = [
"-o", out.path,
"-pkg", ctx.attr.package,
"-prefix", ctx.label.package,
] + [src.path for src in ctx.files.srcs],
)
return [
DefaultInfo(
files = depset([out])
)
]
bindata = rule(
_bindata_impl,
attrs = {
"srcs": attr.label_list(allow_files = True, cfg = "data"),
"package": attr.string(mandatory=True),
"_bindata": attr.label(allow_files=True, single_file=True, default=Label("#com_github_jteeuwen_go_bindata//go-bindata:go-bindata")),
},
)
I would expect an argument to either ctx.new_file or ctx.action but cannot find anything relevant in the Skylark reference or tutorial.
Many thanks!
Try setting output_to_genfiles=True in the rule() definition. It is mentioned in the rule docs.
So:
bindata = rule(
_bindata_impl,
attrs = {
"srcs": attr.label_list(allow_files = True, cfg = "data"),
"package": attr.string(mandatory=True),
"_bindata": attr.label(allow_files=True, single_file=True, default=Label("#com_github_jteeuwen_go_bindata//go-bindata:go-bindata")),
},
output_to_genfiles = True,
)