I'm writing a post-build tool that needs the location of a list of target's jar files.
For these locations I have an aspect that runs on a list of targets (separately for each target using --aspects) and fetch the jar file path for each of them.
I've managed to get each jar file path in a custom output file (e.g. jar.txt) in each target's output folder.
But this will mean I would need to go over each jar.txt file separately to get the location.
Is there a way to accumulate the jar files paths in a single file?
Something like:
Try and write to the same output folder with append command in the aspect. I'm not sure if a shared output folder is possible.
Create a synthetic target which depends on all the relevant targets, then run an aspect on this target and accumulate the jars and only write them at the root after the recursion is back.
Are 1. or 2. valid options?
What is the recommended strategy to accumulate data in bazel aspects output files?
Bazel doesn't provide facitlities in Skylark for accumulating information between targets that are not related to each other in the target graph (e.g. ones that are mentioned on the command line next to each other).
One possibility would be to write a Skylark rule that depends on all the targets you usually mention on the command line and built that one; that rule will be able to collate the classpaths from each Java target to a single file.
Another possibility is to tell Bazel to write build events (that includes all the outputs of all targets the specified build pattern expands to) to a file using the --experimental_build_event_{json,text,binary}_file. (The "experimental" will be removed soon.). The files contain instances of this message:
https://github.com/bazelbuild/bazel/blob/master/src/main/java/com/google/devtools/build/lib/buildeventstream/proto/build_event_stream.proto
Natan,
If I understand correctly, you want to transitively propagate the information from each aspect node out into a single result. To do this, build the transitive set in your aspect rule implementation and pass it via the "provider" mechanism [^1]. I wrote up some examples on bazel aspects, perhaps you'll find it useful[^2].
https://github.com/pcj/bazel_aspects/blob/master/aspects.bzl#L94-L104
https://github.com/pcj/bazel_aspects
Related
We have a Filters framework that contains many image processing filters (written in Swift and Metal) and the resources they require (like ML models and static images). But not every app we have uses all the filters in Filters. Rather we want to only build and bundle the required filters and resources that are needed by the app.
The only way we can think of to achieve that is to create different framework targets in Xcode, one for each app. But that would require that the Filters framework project “knows” all of its consumers (apps) and we would rather like to avoid that. Especially since the filters are in a separate repository.
Is there a way to, for instance, pass some kind of configuration file to the framework that is used at build time to decide which files to build and bundle?
You can use Target Membership to assign files into specific targets. See image below.
Make sure you are on the File Inspector tab on the right side of Xcode. Select the files you want to limit to a target and in the Target Membership area, you can select which target the selected file belongs to or don't belong to. For public resources, make sure to select public for the visibility scope.
For files that don't require compilation, e.g. image files, once you've selected the target membership, the file will be automatically added to the Build Phases' Copy Bundle Resources area.
Alternative One
Alternatively, you can use add a Copy Files Phase in the Build Phases. With the Copy Files Phase, you can copy files to subdirectories instead of the root of the framework bundle.
Alternative Two
Yet another way is to add a Run Script Phase in the Build Phases. The script can be in any language but usually shell script. You can do whatever you need in the script including compiling code manually but you need to know where the files goes by using environment variables and placed the files in the correct location. I think this will be the most manual and most hassle to use for selecting files based on targets.
Alternative Three
If you really want to go fancy, you can even break down all the components into targets and use Aggregate target to tie the different components into the target you are building for. I would not recommend this usually and reserve this for very special needs that other methods could not achieve.
Is there any bazel convention for defining all of a project's test targets in some subfolder tests with its own BUILD file. Is that layout preferred to one main BUILD file that combines everything in one place: bins, libs, and tests?
An all in one build file could get lengthy, but it has the benefit of using in-package sibling target references, ie the ":siblingTarget" references.
Short answer: Yes.
Why would you like to couple your test with the implementation? You should always try to reduce dependencies between different parts of your code. That makes it easier to understand. See also here.
Imaging you find some BUILD file that contains implementation and test rules (e.g. cc_library, cc_binary, cc_test). Imagine some developer needed for some test some library - and only for testing proposes (e. g. cc_library(name="some_lib"). How would you know if this library is only needed for testing? If it is in a BUILD file contained in test folder then it would be clear from the folder structure that there is a high probability that this lib is only needed for testing proposes.
I have a .bzl file in the same directory as WORKSPACE. This .bzl file is loaded by the WORKSPACE and one other file in the source tree.
bazel query --universe_scope=//... --order_output=no 'rbuildfiles(variables.bzl)'
prints the paths of the two files I would expect, but also references to about 200 other files which are all external dependencies and cannot possibly depend on variables.bzl
for example:
#pypi__futures_3_2_0//:BUILD
#pypi__grpcio_1_14_1//:BUILD
#eigen//:BUILD.bazel
#io_bazel_rules_go//go/private:BUILD.bazel
Assuming I am doing something incorrectly and that this is not a bug. Any expertise would be greatly appreciated. How can I use rbuilddeps to return /only/ the files which load variables.bzl?
The WORKSPACE file of the main repo can arbitrarily affect external repositories. So, rbuildfiles is showing you because changes to variables.bzl could in fact affect all those external BUILD files indirectly through WORKSPACE.
If you don't actually want to see the BUILD files in external repositories, you could intersect the result of rbuildfiles with //....
I am aware that bazel accepts both BUILD and BUILD.bazel as valid filenames.
The android tools seem to also have a BUILD.tools file.
In general, does bazel have any restrictions for a BUILD file's extension? For example, could i have BUILD.generated to delineate generate BUILD files from non-generated BUILD files?
The .tools extension is part of building Bazel itself. From the perspective of Bazel, it's just any ordinary file. It gets picked up here: https://github.com/bazelbuild/bazel/blob/bbc8ed16aee07c3ba9321d58aa4c0ffc55fa2ba9/tools/android/BUILD#L197
then eventually gets processed here: https://github.com/bazelbuild/bazel/blob/c816b89a2224c3c318f1228755ef41c53975f45c/src/create_embedded_tools.py#L74
For the use case you mention, one way to go about it is to generate a .bzl file with a meaningful name that contains a macro that you can call from a BUILD or BUILD.bazel file. That way you can separate the generated rules from manually maintained rules. This is similar to how generate_workspace works: https://docs.bazel.build/versions/master/generate-workspace.html
I am trying to create a rule (maybe one already exists?), to un-tar a file during a bazel build step.
If I understand correctly all output files need to be known during the "Analysis Phase". To work around this I have a file lets call it manifest.txt which lists all the files in the tar file. However, I don't quite understand how I can read this file as a list outputs for my skylark rule? Is there an easy way to read a file and have each line be a generated output?
Thanks.
This is only possible if the manifest file is a source file, i.e. it is NOT generated by some rule in the build.
Rules must declare all their outputs without relying on the content of generated files, therefore it's not supported to create for example a genrule whose outs is computed based on a manifest file that's generated by another rule.
To work with a tar file input, the rule needs to unpack the tar with an action, and ultimately produce a predictable amount of outputs (i.e. independent of how many files are there in the tar). Typically this is done by repacking the outputs, that is, the rule would consume one tar and produce another.