Bazel -- get arguments to another target macro - bazel

I have two macros/targets: component and bundle (which packages several components). I would like to extend the bundle macro to accept a list of bundles in addition to a list of components and package all of the components directly-included or included in one of its included bundles.
For example, if I have the following BUILD file:
component(name = 'a')
component(name = 'b')
component(name = 'c')
component(name = 'd')
bundle(name = 'x', components = ['a'])
bundle(name = 'y', components = ['b', 'c'], bundles = ['x'])
bundle(name = 'z', components = ['d'], bundles = ['y'])
Bundle z should include components a, b, c, and d
The .bzl file right now is like this:
def component(name):
# implementation (it uses other args but they aren't relevant)
def bundle(name, components = []):
# complex logic on components
What I want is:
def bundle(name, components = [], bundles = []):
for bundle in bundles:
for component in TODO_get_components_in_bundle(bundle):
if component not in components:
components.append(component)
# complex logic on components
How can I implement TODO_get_components_in_bundle or achieve the same effect?

You cannot do that with a macro (alone):
A macro is a function called from the BUILD file that can instantiate rules. Macros don’t give additional power, they are just used for encapsulation and code reuse. By the end of the loading phase, macros don’t exist anymore, and Bazel sees only the set of rules they created.
In other words you'll need (custom) rule(s) that you can pass the inputs to and work with as you'll need to establish their relations during analysis phase and for execution phase. That is something macros cannot help with.
I've put this example together, providing necessary load is in place, it works with the BUILD file you've used in the question (these rules are written to that interface):
ComponentInfo = provider(fields = ["files", "name"])
BundleInfo = provider(fields = ["files", "name", "components"])
def _component_impl(ctx):
ctx.actions.write(
output = ctx.outputs.out,
content = "NAME: {}\n".format(ctx.attr.name),
)
return ComponentInfo(
files = depset([ctx.outputs.out]),
name = ctx.attr.name,
)
component = rule(
implementation = _component_impl,
outputs = {"out": "%{name}.txt"},
)
def _bundle_impl(ctx):
deps = depset(
[c[ComponentInfo] for c in ctx.attr.components] +
[c for b in ctx.attr.bundles for c in b[BundleInfo].components.to_list()],
)
content = "NAME: {}\n".format(ctx.attr.name)
for comp in deps.to_list():
content += "CONTAINS: {}\n".format(comp.name)
ctx.actions.write(
output = ctx.outputs.out,
content = content,
)
return BundleInfo(
files = depset([ctx.outputs.out]),
name = ctx.attr.name,
components = deps,
)
bundle = rule(
implementation = _bundle_impl,
attrs = {
"components": attr.label_list(),
"bundles": attr.label_list(),
},
outputs = {"out": "%{name}.txt"},
)
It doesn't do anything useful. It just creates a text file with component name for all component targets the same for bundle targets in which case it also lists all components bundled.
I've used custom providers to pass information such as component info around (assuming it's important) without having to resort to some magic divining it from file generated or label name.

Related

LUA indexed table access via named constants

I am using LUA as embedded language on a µC project, so the ressources are limited. To save some cycles and memory I do always only indexed based table access (table[1]) instead og hash-based access (table.someMeaning = 1). This saves a lot of memory.
The clear drawback of this is approach are the magic numbers thrughtout the code.
A Cpp-like preprocessor would help here to replace the number with named-constants.
Is there a good way to achieve this?
A preprocessor in LUA itself, loading the script and editing the chunk and then loading it would be a variant, but I think this exhausts the ressources in the first place ...
So, I found a simple solution: write your own preprocessor in Lua!
It's probably the most easy thing to do.
First, define your symbols globally:
MySymbols = {
FIELD_1 = 1,
FIELD_2 = 2,
FIELD_3 = 3,
}
Then you write your preprocessing function, which basically just replace the strings from MySymbols by their value.
function Preprocess (FilenameIn, FilenameOut)
local FileIn = io.open(FilenameIn, "r")
local FileString = FileIn:read("*a")
for Name, Value in pairs(MySymbols) do
FileString = FileString:gsub(Name, Value)
end
FileIn:close()
local FileOut = io.open(FilenameOut, "w")
FileOut:write(FileString)
FileOut:close()
end
Then, if you try with this input file test.txt:
TEST FIELD_1
TEST FIELD_2
TEST FIELD_3
And call the following function:
Preprocess("test.txt", "test-out.lua")
You will get the fantastic output file:
TEST 1
TEST 2
TEST 3
I let you the joy to integrate it with your scripts/toolchain.
If you want to avoid attributing the number manually, you could just add a wonderful closure:
function MakeCounter ()
local Count = 0
return function ()
Count = Count + 1
return Count
end
end
NewField = MakeCounter()
MySymbols = {
FIELD_1 = NewField(),
FIELD_2 = NewField(),
FIELD_3 = NewField()
}

How to concantenate select objects to a list in bazel

I have a bazel target with attribute that must be a list.
However, I need selectively add elements to the list based on the outcome of a select.
glob_tests(
# some stuff
exclude = [
"a.foo",
] + if_A([
"x.foo",
]) + if_B([
"y.foo",
]),
)
In the above code snippet, the functions if_A and if_B return select objects.
But when I run this as is, I get an error stating that a sequence object was expected but a select object was encountered instead.
How can I convert the select objects to sequence objects?
(I assume glob_test is a macro that calls the builtin function glob.) globs are evaluated when a BUILD file is loaded, which is before any configuration is known. This means glob cannot take any select objects as inputs because the knowledge to turn select objects into lists is not present.
The way to solve this is to lift the select calls above the globs like this
some_test(
name = "some_test",
srcs = select({
"//cond1": glob(["t*", "s*"], exclude=["thing"]),
"//cond2": glob(["t*", "s*"], exclude=["something else"]),
}),
)
instead of
some_test(
name = "some_test",
srcs = glob(
["t*", "s*"],
exclude=select({
"//cond1": ["thing"],
"//cond2": ["something else"],
}),
),
)

Bazel select fails inside ctx.file

I am trying to specify build conditions based on the os I'm running bazel from, so in my .bzl script I have a rule that makes all the simlinks from external sources and writes a BUILD file (with ctx.file), in which I'm declaring all the imports and libraries and in those I would like to add the select function. However, when I build I get this error message:
ERROR: no such package '#maya_repo//': Traceback (most recent call last):
File "/var/tmp/doNotRemove/mdilena_plugins/MayaMathNodes/src/maya.bzl", line 149
ctx.file("BUILD", _BUILD_STRUC.format(maya_...))
File "/var/tmp/doNotRemove/mdilena_plugins/MayaMathNodes/src/maya.bzl", line 149, in ctx.file
_BUILD_STRUC.format(maya_dir = maya_dir)
Invalid character '[' inside replacement field
so here's an example of my code and what I'm trying to achieve:
_BUILD_STRUC = \
"""
# Windows imports
cc_import(
name = "Foundation-win",
interface_library = "{maya_dir}/lib/Foundation.lib",
shared_library = "{maya_dir}/bin/Foundation.dll",
)
cc_import(
name = "OpenMaya-win",
interface_library = "{maya_dir}/lib/OpenMaya.lib",
shared_library = "{maya_dir}/bin/OpenMaya.dll",
)
# Linux imports
cc_import(
name = "Foundation-lnx",
shared_library = "{maya_dir}/bin/Foundation.so",
)
cc_import(
name = "OpenMaya-lnx",
shared_library = "{maya_dir}/bin/OpenMaya.so",
)
cc_library(
name = "Foundation",
deps = select({
"#bazel_tools//src/conditions:windows": [":Foundation-win"],
"//conditions:default": [":Foundation-lnx"],
}),
includes = ["{maya_dir}/include"],
visibility = ["//visibility:public"],
)
cc_library(
name = "OpenMaya",
deps = select({
"#bazel_tools//src/conditions:windows": [":OpenMaya-win"],
"//conditions:default": [":OpenMaya-lnx"],
}),
includes = ["{maya_dir}/include"],
visibility = ["//visibility:public"],
)
"""
def _impl(ctx):
maya_src = ctx.os.environ["MAYA_LOCATION"]
maya_ver = ctx.os.environ["MAYA_VERSION"]
maya_dir = "maya{}".format(maya_ver)
ctx.symlink(maya_src, maya_dir)
ctx.file("BUILD", _BUILD_STRUC.format(maya_dir=maya_dir))
link_maya = repository_rule(
implementation = _impl,
local = True,
environ = ["MAYA_LOCATION"],
)
does anyone have any idea why this is happening? I looked at select and configurable attributes docs and seems like that's the way to use it; I wonder if it's me doing something wrong or if there's a bug somewhere.
Thanks for any help!
EDIT:
looks like Bazel really doesn't like using select inside a ctx.file,
I'll leave the question open in case someone will be able to shed some
light on it. In the meantime I solved it by making all the cc_imports
and includes public from the linked repo, while leaving all the
cc_libraries with select to my plugin's BUILD file; from there I'm
able to use the condition and everything builds.
It looks like the error is coming from this line, specifically the call to string.format.
ctx.file("BUILD", _BUILD_STRUC.format(maya_dir=maya_dir))
string.format searches the template string for curly braces like {} or {key} and replaces them with positional or keyword arguments.
You're seeing this error because string.format is mistaking the dict argument to select within the template as something to replace because it starts with a curly brace. Escaping the braces within the template string by doubling them should fix the problem:
_BUILD_STRUC = \
"""
...
cc_library(
name = "Foundation",
deps = select({{
"#bazel_tools//src/conditions:windows": [":Foundation-win"],
"//conditions:default": [":Foundation-lnx"],
}}),
includes = ["{maya_dir}/include"],
visibility = ["//visibility:public"],
)
...
FYI, you might find repository_ctx.template easier to work with. It has slightly different semantics: it replaces strings literally, without looking for special characters like {, so escaping is not needed.

$location expansion in Bazel

I want to add $(location) expansion to rules_scala for jvm_flags attribute where I set the dependency in the data attribute but that fails with:
label '//src/java/com/google/devtools/build/lib:worker' in $(location) expression is not a declared prerequisite of this rule.
I define a dependency in my target on that label in the data attribute like this:
scala_specs2_junit_test(
...
data = ["//src/java/com/google/devtools/build/lib:worker"],
jvm_flags = ["-XX:HeapDumpPath=/some/custom/path", "-Dlocation.expanded=$(location //src/java/com/google/devtools/build/lib:worker)"],
)
I saw that when I add ctx.attr.data to the expand_location call expansion works but I wasn't really sure why this is not a hack. Is data indeed a special case?
location_expanded_jvm_flags = []
for jvm_flag in jvm_flags:
location_expanded_jvm_flags.append(ctx.expand_location(jvm_flag, ctx.attr.data))
Also tried looking in the java_* rules sources to see how this works (since $(location) expansion there supports the data attribute) but couldn't find the relevant place.
Full target:
scala_specs2_junit_test(
name = "Specs2Tests",
srcs = ["src/main/scala/scala/test/junit/specs2/Specs2Tests.scala"],
deps = [":JUnitCompileTimeDep"],
size = "small",
suffixes = ["Test"],
data = ["//src/java/com/google/devtools/build/lib:worker"],
jvm_flags = ["-XX:HeapDumpPath=/some/custom/path", "-Dlocation.expanded=$(location //src/java/com/google/devtools/build/lib:worker)"],
)
You're doing it right.
I looked at the source code and you're right: srcs, deps, and tools (if defined on the rule) are added to the set of labels that expand_locations understands. data is added only if LocationExpander is created with allowDataAttributeEntriesInLabel=true, which it isn't. That's why you must add it to expand_locations(targets).

How to merge AndroidManifest.xml in bazel

My android project contains some aar modules, which have their own AndroidManifest.xml. What should I do to have the aar's manifest to be merged into the final AndroidManifest.xml?
Thanks very much for any help!
My android_binary rule:
android_binary(
name="apk",
custom_package = "com.xtbc",
manifest_merger = "android",
manifest = "AndroidManifest.xml",
resource_files = glob(["res/**"], exclude=["res/.DS_Store"]),
assets = glob(["assets/**"], exclude=["assets/.DS_Store"]),
assets_dir = "assets",
multidex = "manual_main_dex",
main_dex_list = "mainDexList.txt",
dexopts = [
"--force-jumbo"
],
deps = [
":lib",
":base_lib",
":jni"
]
)
The :base_lib is a module (ie, an android_library rule):
android_library(
name = "base_lib",
srcs = glob(["base/src/**/*.java"]),
custom_package = "com.xtbc.base",
manifest = "base/AndroidManifest.xml",
resource_files = glob(["base/res/**"], exclude=["base/res/.DS_Store"]),
assets = glob(["base/assets/**"], exclude=["base/assets/.DS_Store"]),
assets_dir = "base/assets",
deps = [
"#androidsdk//com.android.support:support-annotations-23.0.1"
]
)
It has its own base/AndoridManifest.xml, what I want is that the :base_lib's AndroidManifest.xml will be merged into the final AndroidManifest.xml(ie, the :apk's AndroidManifest.xml).
I do not have enough stackoverflow reputation to respond to the comment chain, but it sounds like what you are after is the exports_manifest attribute of android_library.
The documentation at https://bazel.build/versions/master/docs/be/android.html#android_library.exports_manifest says that the default is 1, however, that documentation is based on source code changes that have not made it into a Bazel release yet. For now you will need to add exports_manifest = 1 onto your android_library. In the next Bazel release, this will no longer be necessary.
Also, regarding "AAR modules": If these are prebuilt .aar files, you will want to use the aar_import rule. It does not have an exports_manifest attribute, because it will always export by default. If these are Gradle Android library modules, then you can just use the android_library rule. If you were referring to the support libraries, #androidsdk//com.android.support:support-annotations-23.0.1 is actually a JAR, not an AAR.

Resources