I've a c target that always must be compiled for darwin_x86_64, no matter the --cpu set when calling bazel build. It's the only target that always must be compiled for a specific cpu in a big project.
cc_binary(
name = "hello-world",
srcs = ["hello-world.cc"],
)
In the bazel documentation it seems to be possible to do this using transitions. Maybe something like:
def _force_x86_64_impl(settings, attr):
return {"//command_line_option:cpu": "darwin_x86_64"}
force_x86_64 = transition(
implementation = _force_x86_64_impl,
inputs = [],
outputs = ["//command_line_option:cpu"]
)
But how do I tie these two together? I'm probably missing something obvious, but I can't seem to find the relevant documentation over at bazel.build.
You've got the transition defined, now you need to attach it to a rule.
r = rule(
implementation = _r_impl,
attrs = {
"_allowlist_function_transition": attr.label(
default = "#bazel_tools//tools/allowlists/function_transition_allowlist",
),
"srcs": attr.label(cfg = force_x86_64),
},
)
def _r_impl(ctx):
return [DefaultInfo(files = ctx.attr.srcs[0][DefaultInfo].files)]
This defines a rule that attaches the appropriate transition to the srcs you pass it, and then simply forwards the list of files from DefaultInfo. Depending on what you're doing, this might be sufficient, or you might also want to forward the runfiles contained in DefaultInfo and/or other providers.
Use target_compatible_with
Example (only compiled on Windows):
cc_library(
name = "win_driver_lib",
srcs = ["win_driver_lib.cc"],
target_compatible_with = [
"#platforms//cpu:x86_64",
"#platforms//os:windows",
],
)
Related
I am having a simple shell script that executes a prebuilt binary. The build file looks like this:
filegroup(
name = "generator_srcs",
srcs = glob([
"configuration/**",
"features/**",
"plugins/**"
]) + [
"commonapi-generator-linux-x86_64.ini",
"artifacts.xml"
],
)
filegroup(
name = "generator_binary",
srcs = ["commonapi-generator-linux-x86_64"],
data = [":generator_srcs"],
)
sh_binary(
name = "generator",
srcs = ["#//tools:generator.sh"],
data = [":generator_binary"],
toolchains = ["#//toolchain:jre_toolchain_definition"],
args = ["$(location :generator_binary)", "$(JAVA)"],
)
However the prebuilt binary depends on a specific Java Runtime Environment. Therefore I simply defined a custom java_runtime that fits the requirements of the binary. The corresponding build file looks like this:
java_runtime(
name = "jre8u181-b13",
srcs = glob([
"jre/**"
]),
java_home = "jre",
licenses = [],
visibility = [
"//visibility:public"
]
)
config_setting(
name = "jre_version_setting",
values = {"java_runtime_version": "1.8"},
visibility = ["//visibility:private"],
)
toolchain(
name = "jre_toolchain_definition",
target_settings = [":jre_version_setting"],
toolchain_type = "#bazel_tools//tools/jdk:runtime_toolchain_type",
toolchain = ":jre8u181-b13",
visibility = ["//visibility:public"],
)
When I am trying to build and run the target generator bazel throws the error:
//toolchain:jre_toolchain_definition does not have mandatory providers: 'TemplateVariableInfo'
This is the point where I am a little bit lost. As stated in this post the rule should provide toolchain specific make variables. Therefore I was looking around in the bazel github repository and found the rule java_runtime_alias which seems to provide some useful variables that I could use in my sh_binary target. But in this rule automatic toolchain resolution happens. I would like to rewrite the rule such that I can hand over my custom toolchain target as an argument but I don't know how. Should I define an attribute?
I'd like to build a simple application and to specify its output name: for debug version 'd' prefix should be added. I tried to use genrule for it, but I don't know how to change this option in different compilation modes. However I suppose that using genrule is not correct way.
So, how to specify output name?
cc_binary(
name = "TestApp",
srcs = [
"src/TestApp/main.cpp",
],
)
genrule(
name = "output_name_rule",
srcs = [":TestApp"],
outs = ["TestAppd.exe"],
output_to_bindir = True,
cmd_bat = "rename $(location TestApp) TestAppd.exe"
)
config_setting(
name = "release_build",
values = {
"compilation_mode":"opt"
}
)
config_setting(
name = "debug_build",
values = {
"compilation_mode":"dbg"
}
)
genrule.outs is documented as nonconfigurable, so you can't use it directly. I would create a genrule that copies the file that exists in all configurations, and then choose between the genrule and the original cc_binary based on the configuration. Something like this:
genrule(
name = "copy_with_d",
srcs = [":TestApp"],
outs = ["TestAppd.exe"],
output_to_bindir = True,
cmd_bat = "rename $(location TestApp) TestAppd.exe"
)
alias(
name = "maybe_with_d",
actual = select({
":debug_build": "TestAppd.exe",
"//conditions:default": "TestApp.exe",
}),
)
Some notes on things to tweak for your use case:
On Linux, I would write cmd = "cp $< $#". I'm not sure how Windows rename handles the input not being writable. You can definitely write $< and $# instead of repeating the filenames though, they are Bazel "Make" variables.
I put the non-d version as //conditions:default so it applies in fastbuild too, up to you which way you want as the default.
Up to you whether you use TestApp.exe or :TestApp.exe or :TestApp in alias.actual to refer to that version, they're all equivalent. Same with TestAppd.exe vs copy_with_d or :copy_with_d, etc.
I need to set some build options every time I invoke bazel for a specific target. For example, bazel build --collect_code_coverage //:target. How can I avoid providing the build options at the command line explicitly, so that bazel build //:target implicitly has the build option --collect_code_coverage applied?
The closest solution I found was using the bazelrc file, but it does not allow me to configure build options at a target level.
This is doable with user defined transitions, there are some performance considerations here that are worth reading over. It's also relatively complicated.
# transition.bzl
def _platform_transition_impl(settings, attr):
_ignore = settings
return {"//command_line_option:collect_code_coverage": True}
coverage_transition = transition(
implementation = _coverage_transition_impl,
inputs = [],
outputs = ["//command_line_option:collect_code_coverage"],
)
def _coverage_impl(ctx):
executable = ctx.actions.declare_file(ctx.attr.name)
ctx.actions.symlink(output = executable, target_file = ctx.file.target)
return [DefaultInfo(executable = executable)]
build_with_collect_coverage = rule(
implementation = _coverage_impl,
attrs = {
"target": attr.label(mandatory = True, allow_single_file = True, cfg = coverage_transition),
"_allowlist_function_transition": attr.label(
default = "#bazel_tools//tools/allowlists/function_transition_allowlist",
),
},
)
Then add the transition rule to BUILD.bazel e.g.
load(":transition.bzl", "build_with_collect_coverage")
cc_binary(
name = "target"
#...
)
build_with_collect_coverage(
name = "target_with_coverage_collection",
target = ":target",
)
Note: While this is a generic solution that is incredibly powerful and supports any language you might simply be able to get away with using the features attribute e.g.
cc_binary(
name = "target",
#...
features = ["coverage"],
)
I have a rule that creates a folder (untars a tar.gz) and I would like to use this folder directly as data for a cc_binary. The only way I could find to so is to create an intermediate filegroup with the created folder as source.
Though it works it requires the users of the rule to create an intermediate filegroup and introduces some naming issues as the name of the filegroup cannot be the same as the created folder name.
This is what I have
# rules.bzl
def _untar_impl(ctx):
tree = ctx.actions.declare_directory(ctx.attr.out)
ctx.actions.run_shell(
inputs = [ctx.file.src],
outputs = [tree],
command = "tar -C '{out}' -xf '{src}'".format(src=ctx.file.src.path, out=tree.path),
)
return [DefaultInfo(files = depset([tree]))]
untar = rule(
implementation = _untar_impl,
attrs = {
"src": attr.label(mandatory=True, allow_single_file = True),
"out": attr.string(mandatory=True),
},
)
# BUILD
untar(
name = "media",
src = "media.tar.gz",
out = "media",
)
filegroup(
name = "mediafiles",
srcs = ["media"],
data = [":media"],
)
cc_binary(
name = "main",
srcs = ["main.cpp"],
data = [":mediafiles"],
)
Is there any way to avoid having the intermediate filegroup?
Based on the discussion that ensued, I see I've misunderstood the problem statement a bit. You probably still want to look into handling that tarball (and its entire processing) as an external dependency and a repository_rule, but for your immediate problem of need of intermediate filegroup.
If you noticed, you've defined both srcs and data to point to your media label, and that is exactly the missing bit to have data available for execution of your *_binary rule. Because the untar rule returned depset of files, but those when used data directly would resolve to being empty.
If you replace this line in your rule definition:
return [DefaultInfo(files = depset([tree]))]
with:
return [DefaultInfo(runfiles = ctx.runfiles([tree]))]
You can then in your BUILD file say:
cc_binary(
name = "main",
srcs = ["main.cpp"],
data = [":media"],
)
Because untar rule now provides runfiles of DefaultInfo. That filegroup wrapping and adding media through its data property has done that.
Situation
I have two Skylark extension rules: blah_library and blah_binary. All of a blah_library's transitive dependencies are propagated by returning a provider(transitive_deps=...), and are handled appropriately by any ultimate dependent blah_binary target.
What I want to do
I want each blah_library to also create a filegroup with all the transitive dependencies mentioned above, so that I can access them separately. E.g., I'd like to be able to pass them in as data dependencies to a cc_binary. In other words:
# Somehow have this automatically create a target named `foo__trans_deps`?
blah_library(
name = "foo",
srcs = [...],
deps = [...],
)
cc_binary(
...,
data = [":foo__trans_deps"],
)
How should I do this? Any help would be appreciated!
What I've tried
Make a macro
I tried making a macro like so:
_real_blah_library = rule(...)
def blah_library(name, *args, **kwargs):
native.filegroup(
name = name + "__trans_deps",
srcs = ???,
)
_real_blah_library(name=name, *args, **kwargs)
But I'm not sure how to access the provider provided by _real_blah_library from within the macro, so I don't know how to populate the filegroup's srcs field...
Modify the blah_library rule's implementation
Right now I have something like:
_blah_provider = provider(fields=['transitive_deps'])
def _blah_library_impl(ctx):
...
trans_deps = []
for dep in ctx.attr.deps:
trans_deps += dep[_blah_provider].trans_deps
return _blah_provider(trans_deps=trans_deps)
blah_library = rule(impl=_blah_library_impl, ...)
I tried adding the following to _blah_library_impl, but it didn't work because apparently native.filegroup can't be called within a rule's implementation ("filegroup() cannot be called during the analysis phase"):
def _blah_library_impl(ctx):
...
trans_deps = []
for dep in ctx.attr.deps:
trans_deps += dep[_blah_provider].trans_deps
native.filegroup(
name = ctx.attr.name + "__trans_deps",
srcs = trans_deps,
)
return _blah_provider(trans_deps=trans_deps)
You can't easily create a filegroup like that, but you can still achieve what you want.
If you want to use the rule in genrule.srcs, filegroup.srcs, cc_binary.data, etc., then return a DefaultInfo provider (along with _blah_provider) and set the files field to the transitive closure of files.
You can refine the solution if you want a different set of files when the rule is in a data attribute vs. when in any other (e.g. srcs): just also set the runfiles-related members in DefaultInfo. (Frankly I don't know the difference between them, I'd just set all runfiles-fields to the same value.)
I ended up making my own special filegroup-like rule, as discussed in the comments under #Laszlo's answer. Here's the raw code in case it's a useful starting point for anyone:
def _whl_deps_filegroup_impl(ctx):
input_wheels = ctx.attr.src[_PyZProvider].transitive_wheels
output_wheels = []
for wheel in input_wheels:
file_name = wheel.basename
output_wheel = ctx.actions.declare_file(file_name)
# TODO(josh): Use symlinks instead of copying. Couldn't figure out how
# to do this due to issues with constructing absolute paths...
ctx.actions.run(
outputs=[output_wheel],
inputs=[wheel],
arguments=[wheel.path, output_wheel.path],
executable="cp",
mnemonic="CopyWheel")
output_wheels.append(output_wheel)
return [DefaultInfo(files=depset(output_wheels))]
whl_deps_filegroup = rule(
_whl_deps_filegroup_impl,
attrs = {
"src": attr.label(),
},
)