I'm including a repository that has an extra_deps rule of the form:
maybe(
http_file,
name = "external_dependency",
downloaded_file_path = "foo.h",
sha256 = "<some_sha>",
urls = ["https://example.com/foo.h"],
)
If I have an existing repository, foo_repo, that provides foo.h, how can I substitute the target for it in place of external_dependency? http_file apparently provides #external_dependency//file, so I can't simply define an alias.
Using https://github.com/bazelbuild/bazel/blob/master/tools/build_defs/repo/http.bzl as a reference, you can define a custom repository rule that provides #external_dependency//file. For example:
def _repository_file(ctx):
ctx.file("WORKSPACE", "workspace(name = \"{name}\")".format(name = ctx.name))
ctx.file("file/BUILD.bazel", """
filegroup(
name = "file",
srcs = ["{}"],
visibility = ["//visibility:public"],
)
""".format(ctx.attr.source))
repository_file = repository_rule(
attrs = {"source": attr.label(mandatory = True, allow_single_file = True)},
implementation = _repository_file,
doc = """Analogue of http_file, but for a file in another repository.
Usage:
repository_file(
name = "special_file"
source = "#other_repo//path/to:special_file.txt",
)
""",
)
Now use:
repository_file(
name = "external_dependency",
source = "#foo_repo//path/to:foo.h",
)
Related
I created a first rule file to geneate a scirpt using the ctx.actions.expand_template and ran it. and I wanted that pass the result of bazel run with first rule to the next 2nd rule file as a resource. But, I could't get the result which is generated by bazel run 1st rule in the 2nd rule.
It does not mean the script created by the first rule, but the file created when the script is executed.
Below example is what I tested.
This is bazel rule file
def _1st_rule_impl(ctx):
...
out = ctx.outputs.executable
template = ctx.file.my_template
ctx.actions.expand_template(
output = out,
template = template,
substitutions = {
"{ARG1}: ctx.attr.my_file_name,
"{ARG2}: ctx.attr.my_file_content,
}
return [
DefaultInfo(
files = depset([out]),
runfiles = ctx.runfiles(files = [out])
)
]
1st_rule = rule(
implementation = _1st_rule_impl,
attrs = {
"my_template":attr.label(
allow_single_file = True,
default = Label("#my_test//my_rules:my_script.sh.template"),
),
"my_file_name": attr.string(
default = "myfile.txt",
),
"my_file_content": attr.string(
default = "hello world",
),
},
executable = True,
)
def _2nd_rule_impl(ctx):
...
for dep in ctx.attr.deps:
//
// HOW CAN I GET THE RESULT OF `bazel run` THE '1st_rule'?
// I WANT TO GET THE `myfile.txt` WHICH IS GENERATED BY '1st_rule'
//
return [
DefaultInfo(
files = depset([out]),
runfiles = ctx.runfiles(files = [out]),
)
]
2nd_rule = rule(
implementation = _2nd_rule_impl,
attrs = {
"dep":attr.label_list(
mandatory= True,
),
...
},
executable = True,
)
This is BUILD file
1st_rule(
name = "my_1st_rule",
my_file_name = "myfile.txt",
my_file_content = "hello world",
)
2nd_rule(
name = "my_2nd_rule",
dep = [
":my_1st_rule",
],
)
This is template shell script
...
function create_file() {
echo "{ARG2}" > "{ARG1}"
}
...
I tested with the example described above.
There are several ways to access the outputs of dependencies. Perhaps, the simplest is ctx.files. For example, print(ctx.files.dep) in _2nd_rule_impl should show the output of my_1st_rule when my_2nd_rule is analyzed.
I am trying to write a custom rule, where I first generate a file from a template, then pass this file to a script to generate some c++ headers that are the output of my rule.
def _msg_library_impl(ctx):
# For each target in deps, print its label and files.
for source in enumerate(ctx.attr.srcs):
print("File = " + str(source))
out_header = ctx.actions.declare_file("some_header.hpp")
out_arguments = ctx.actions.declare_file("arguments.json")
ctx.actions.expand_template(
template = ctx.file._arguments_file,
output = out_arguments,
substitutions = {
"{output_dir}": out_header.dirname,
"{idl_tuples}": out_header.path,
},
)
args = ctx.actions.args()
args.add("--arguments-file")
args.add(out_arguments)
ctx.actions.run(
outputs = [out_header],
progress_message = "Generating headers '{}'".format(out_header.short_path),
executable = ctx.executable._generator,
arguments = [args],
)
return [
CcInfo(compilation_context=cc_common.create_compilation_context(
includes=depset([out_header.dirname]),
headers=depset([out_header])))
]
msg_library = rule(
implementation = _msg_library_impl,
output_to_genfiles = True,
attrs = {
"srcs": attr.label_list(allow_files = True),
"outs": attr.output_list(),
"_arguments_file": attr.label(
allow_single_file = [".json"],
default = Label("//examples/generation_rule:arguments_template.json"),
),
"_generator": attr.label(
default = Label("//examples/generation_rule:generator"),
executable = True,
cfg = "exec"
),
},
)
Here, generator is a python library that, given an input file provided to srcs and an arguments file generates headers.
The issue that I am facing is that it seems that the expand_template doesn't actually run before run is called, so the generated file is nowhere to be found. What am I doing wrong here? Did I misunderstand how things work?
You need to indicate the file is an input to the action, in addition to passing its path in the arguments. Change the ctx.actions.run to:
ctx.actions.run(
outputs = [out_header],
inputs = [out_arguments],
progress_message = "Generating headers '{}'".format(out_header.short_path),
executable = ctx.executable._generator,
arguments = [args],
)
Error:
LINK : warning LNK4098: defaultlib 'LIBCMTD' conflicts with use of other libs; use /NODEFAULTLIB:library
I am trying to depend on libsodium: https://download.libsodium.org/libsodium/releases/libsodium-1.0.17-stable-msvc.zip
Using the following Bazel configuration:
# WORKSPACE
new_local_repository(
name = "org_libsodium_sodium",
build_file = "third_party/sodium.BUILD",
path = "third_party/sodium",
)
# sodium.BUILD
config_setting(
name = "windows_dbg_build",
constraint_values = ["#bazel_tools//platforms:windows"],
values = {"compilation_mode": "dbg"},
)
config_setting(
name = "windows_fastbuild_build",
constraint_values = ["#bazel_tools//platforms:windows"],
values = {"compilation_mode": "fastbuild"},
)
config_setting(
name = "windows_opt_build",
constraint_values = ["#bazel_tools//platforms:windows"],
values = {"compilation_mode": "opt"},
)
cc_library(
name = "sodium",
srcs = select({
":windows_dbg_build": ["lib/dbg/libsodium.lib"],
":windows_fastbuild_build": ["lib/dbg/libsodium.lib"],
":windows_opt_build": ["lib/opt/libsodium.lib"],
"//conditions:default": ["lib/opt/libsodium.a"],
}),
hdrs = glob(["sodium/**/*.h"]),
defines = ["SODIUM_STATIC"],
visibility = ["//visibility:public"],
)
Is it correct to use the /MT and /MTd Runtime Library for Windows precompiled static libraries?
Any idea what I am doing wrong?
Kind regards,
Ryan
I'm trying to build a rule for bazel which emulates the CMake *.in template system.
This has two challenges, the first is generate the template output. The second is make the output available to both genrules, filegroups and cc_* rules. The third is to get that dependency to transitively be passed to further downstream rules.
I have it generating the output file version.hpp in genfiles (or bazel-bin), and I can get the initial library rule to include it, but I can't seem to figure out how to make my cc_binary rule, which depends on the cc_library and transitively the header_template rule to find the header file.
I have the following .bzl rule:
def _header_template_impl(ctx):
# this generates the output from the template
ctx.actions.expand_template(
template = ctx.file.template,
output = ctx.outputs.out,
substitutions = ctx.attr.vars,
)
return [
# create a provider which says that this
# out file should be made available as a header
CcInfo(compilation_context=cc_common.create_compilation_context(
headers=depset([ctx.outputs.out])
)),
# Also create a provider referencing this header ???
DefaultInfo(files=depset(
[ctx.outputs.out]
))
]
header_template = rule(
implementation = _header_template_impl,
attrs = {
"vars": attr.string_dict(
mandatory = True
),
"extension": attr.string(default=".hpp"),
"template": attr.label(
mandatory = True,
allow_single_file = True,
),
},
outputs = {
"out": "%{name}%{extension}",
},
output_to_genfiles = True,
)
elsewhere I have a cc_library rule:
load("//:tools/header_template.bzl", "header_template")
# version control
BONSAI_MAJOR_VERSION = '2'
BONSAI_MINOR_VERSION = '0'
BONSAI_PATCH_VERSION = '9'
BONSAI_VERSION = \
BONSAI_MAJOR_VERSION + '.' + \
BONSAI_MINOR_VERSION + '.' + \
BONSAI_PATCH_VERSION
header_template(
name = "bonsai_version",
extension = ".hpp",
template = "version.hpp.in",
vars = {
"#BONSAI_MAJOR_VERSION#": BONSAI_MAJOR_VERSION,
"#BONSAI_MINOR_VERSION#": BONSAI_MINOR_VERSION,
"#BONSAI_PATCH_VERSION#": BONSAI_PATCH_VERSION,
"#BONSAI_VERSION#": BONSAI_VERSION,
},
)
# ...
private = glob([
"src/**/*.hpp",
"src/**/*.cpp",
"proto/**/*.hpp",
])
public = glob([
"include/*.hpp",
":bonsai_version",
])
cc_library(
# target name matches directory name so you can call:
# bazel build .
name = "bonsai",
srcs = private,
hdrs = public,
# public headers
includes = [
"include",
],
# ...
deps = [
":bonsai_version",
# ...
],
# ...
)
When I build, my source files need to be able to:
#include "bonsai_version.hpp"
I think the answer involves CcInfo but I'm grasping in the dark as to how it should be constructed.
I've already tried add "-I$(GENDIR)/" + package_name() to the copts, to no avail. The generated header still isn't available.
My expectation is that I should be able to return some kind of Info object that would allow me to add the dependency in srcs. Maybe it should be a DefaultInfo.
I've dug through the bazel rules examples and the source, but I'm missing something fundamental, and I can't find documentation that discuss this particular.
I'd like to be able to do the following:
header_template(
name = "some_header",
extension = ".hpp",
template = "some_header.hpp.in",
vars = {
"#SOMEVAR#": "value",
"{ANOTHERVAR}": "another_value",
},
)
cc_library(
name = "foo",
srcs = ["foo.src", ":some_header"],
...
)
cc_binary(
name = "bar",
srcs = ["bar.cpp"],
deps = [":foo"],
)
and include the generated header like so:
#include "some_header.hpp"
void bar(){
}
The answer looks like it is:
def _header_template_impl(ctx):
# this generates the output from the template
ctx.actions.expand_template(
template = ctx.file.template,
output = ctx.outputs.out,
substitutions = ctx.attr.vars,
)
return [
# create a provider which says that this
# out file should be made available as a header
CcInfo(compilation_context=cc_common.create_compilation_context(
# pass out the include path for finding this header
includes=depset([ctx.outputs.out.dirname]),
# and the actual header here.
headers=depset([ctx.outputs.out])
))
]
elsewhere:
header_template(
name = "some_header",
extension = ".hpp",
template = "some_header.hpp.in",
vars = {
"#SOMEVAR#": "value",
"{ANOTHERVAR}": "another_value",
},
)
cc_library(
name = "foo",
srcs = ["foo.cpp"],
deps = [":some_header"],
...
)
cc_binary(
name = "bar",
srcs = ["bar.cpp"],
deps = [":foo"],
)
If your header has a generic name (eg config.h) and you want it to be private (ie srcs instead of hdrs), you might need a different approach. I've seen this problem for gflags, which "leaked" config.h and affected libraries that depended on it (issue).
Of course, in both cases, the easiest solution is to generate and commit header files for the platforms you target.
Alternatively, you can set copts for the cc_library rule that uses the generated private header:
cc_library(
name = "foo",
srcs = ["foo.cpp", "some_header.hpp"],
copts = ["-I$(GENDIR)/my/package/name"],
...
)
If you want this to work when your repository is included as an external repository, you have a bit more work cut out for you due to bazel issue #4463.
PS. You might want to see if cc_fix_config from https://github.com/antonovvk/bazel_rules works for you. It's just a wrapper around perl but I found it useful.
I'm trying to write a custom rule to compile C++ code using the cc_common API. Here's my current attempt at an implementation:
load("#bazel_tools//tools/cpp:toolchain_utils.bzl", "find_cpp_toolchain")
load("#bazel_tools//tools/build_defs/cc:action_names.bzl", "C_COMPILE_ACTION_NAME")
def _impl(ctx):
cc_toolchain = find_cpp_toolchain(ctx)
feature_configuration = cc_common.configure_features(
cc_toolchain = cc_toolchain,
unsupported_features = ctx.disabled_features,
)
compiler = cc_common.get_tool_for_action(
feature_configuration=feature_configuration,
action_name=C_COMPILE_ACTION_NAME
)
compile_variables = cc_common.create_compile_variables(
feature_configuration = feature_configuration,
cc_toolchain = cc_toolchain,
)
compiler_options = cc_common.get_memory_inefficient_command_line(
feature_configuration = feature_configuration,
action_name = C_COMPILE_ACTION_NAME,
variables = compile_variables,
)
outfile = ctx.actions.declare_file("test.o")
args = ctx.actions.args()
args.add_all(compiler_options)
ctx.actions.run(
outputs = [outfile],
inputs = ctx.files.srcs,
executable = compiler,
arguments = [args],
)
return [DefaultInfo(files = depset([outfile]))]
However, this fails with the error "execvp(external/local_config_cc/wrapped_clang, ...)": No such file or directory. I assume this is because get_tool_for_action returns a string representing a path, not a File object, so Bazel doesn't add wrapped_clang to the sandbox. Executing the rule with sandboxing disabled seems to confirm this, as it completes successfully.
Is there a way to implement this custom rule without disabling the sandbox?
If you use ctx.actions.run_shell you can add the files associated with the toolchain to the input (ctx.attr._cc_toolchain.files). Also, you'll want to add the compiler environment variables. E.g.
srcs = depset(ctx.files.srcs)
tools = ctx.attr._cc_toolchain.files
...
compiler_env = cc_common.get_environment_variables(
feature_configuration = feature_configuration,
action_name = C_COMPILE_ACTION_NAME,
variables = compiler_variables,
)
...
args = ctx.actions.args()
args.add_all(compiler_options)
ctx.actions.run_shell(
outputs = [outfile],
inputs = depset(transitive = [srcs, tools]), # Merge src and tools depsets
command = "{compiler} $*".format(compiler = compiler),
arguments = [args],
env = compiler_env,
)
Bazel doesn't add files as action inputs automatically, you have to do it explicitly, as you did in your second approach (ctx.attr._cc_toolchain.files). With that, ctx.actions.run should work just fine.