For a cc_binary (or py_binary, sh_binary), when does bazel create its runfiles directory and have all symlinks correctly set up?
Is it created right after the cc_binary is built, and before any rule that takes this cc_binary as input, OR
Is it created after the whole build process of all targets is finished?
I'm trying to write a custom rule to pack the contents in the runfiles directory of a cc_binary into a tarball. This custom rule takes the cc_binary and all targets in its runfiles as input. If the runfiles directory is properly set up right after the cc_binary is built, then I just need to directly pack this directory. If not, I probably need to set up a tmp runfiles directory by myself in my custom rule.
Also, is this behavior guaranteed to keep in the future releases?
Thanks a lot!
Bazel does not guarantee any ordering; all it guarantees is that by the end of the build, both the binary and the runfiles tree should be there.
If you want an action to be executed after both of these have been built, you need to depend on both the binary and its runfiles manifest. This doesn't guarantee, though, that the runfiles tree will be in the sandbox the packing action rules on.
I think it's possibly to craft something from ctx.actions.run(input_manifests=) and ctx.resolve_command that does that, although it's very arcane at the moment.
Related
We have a Filters framework that contains many image processing filters (written in Swift and Metal) and the resources they require (like ML models and static images). But not every app we have uses all the filters in Filters. Rather we want to only build and bundle the required filters and resources that are needed by the app.
The only way we can think of to achieve that is to create different framework targets in Xcode, one for each app. But that would require that the Filters framework project “knows” all of its consumers (apps) and we would rather like to avoid that. Especially since the filters are in a separate repository.
Is there a way to, for instance, pass some kind of configuration file to the framework that is used at build time to decide which files to build and bundle?
You can use Target Membership to assign files into specific targets. See image below.
Make sure you are on the File Inspector tab on the right side of Xcode. Select the files you want to limit to a target and in the Target Membership area, you can select which target the selected file belongs to or don't belong to. For public resources, make sure to select public for the visibility scope.
For files that don't require compilation, e.g. image files, once you've selected the target membership, the file will be automatically added to the Build Phases' Copy Bundle Resources area.
Alternative One
Alternatively, you can use add a Copy Files Phase in the Build Phases. With the Copy Files Phase, you can copy files to subdirectories instead of the root of the framework bundle.
Alternative Two
Yet another way is to add a Run Script Phase in the Build Phases. The script can be in any language but usually shell script. You can do whatever you need in the script including compiling code manually but you need to know where the files goes by using environment variables and placed the files in the correct location. I think this will be the most manual and most hassle to use for selecting files based on targets.
Alternative Three
If you really want to go fancy, you can even break down all the components into targets and use Aggregate target to tie the different components into the target you are building for. I would not recommend this usually and reserve this for very special needs that other methods could not achieve.
I am aware that bazel accepts both BUILD and BUILD.bazel as valid filenames.
The android tools seem to also have a BUILD.tools file.
In general, does bazel have any restrictions for a BUILD file's extension? For example, could i have BUILD.generated to delineate generate BUILD files from non-generated BUILD files?
The .tools extension is part of building Bazel itself. From the perspective of Bazel, it's just any ordinary file. It gets picked up here: https://github.com/bazelbuild/bazel/blob/bbc8ed16aee07c3ba9321d58aa4c0ffc55fa2ba9/tools/android/BUILD#L197
then eventually gets processed here: https://github.com/bazelbuild/bazel/blob/c816b89a2224c3c318f1228755ef41c53975f45c/src/create_embedded_tools.py#L74
For the use case you mention, one way to go about it is to generate a .bzl file with a meaningful name that contains a macro that you can call from a BUILD or BUILD.bazel file. That way you can separate the generated rules from manually maintained rules. This is similar to how generate_workspace works: https://docs.bazel.build/versions/master/generate-workspace.html
cc_binary (on osx / linux) creates executables with no suffix. This makes sense, since the standard on those platforms is to not use extensions.
When using Bazel as a cross-compiler through a custom CROSSTOOL, though, I'd like Bazel to emit an elf file with an explicit .elf suffix.
Is this possible, either through CROSSTOOL or a custom "rename" rule?
You can name your cc_binary 'foo.elf' and bazel will just build it. Or you can use genrule to do the renaming afterwards.
Now if you need to build the same cc_binary with multiple toolchains, each time producing a different extension, that's a little bit more tricky. But genrule will work there too.
I'm writing a post-build tool that needs the location of a list of target's jar files.
For these locations I have an aspect that runs on a list of targets (separately for each target using --aspects) and fetch the jar file path for each of them.
I've managed to get each jar file path in a custom output file (e.g. jar.txt) in each target's output folder.
But this will mean I would need to go over each jar.txt file separately to get the location.
Is there a way to accumulate the jar files paths in a single file?
Something like:
Try and write to the same output folder with append command in the aspect. I'm not sure if a shared output folder is possible.
Create a synthetic target which depends on all the relevant targets, then run an aspect on this target and accumulate the jars and only write them at the root after the recursion is back.
Are 1. or 2. valid options?
What is the recommended strategy to accumulate data in bazel aspects output files?
Bazel doesn't provide facitlities in Skylark for accumulating information between targets that are not related to each other in the target graph (e.g. ones that are mentioned on the command line next to each other).
One possibility would be to write a Skylark rule that depends on all the targets you usually mention on the command line and built that one; that rule will be able to collate the classpaths from each Java target to a single file.
Another possibility is to tell Bazel to write build events (that includes all the outputs of all targets the specified build pattern expands to) to a file using the --experimental_build_event_{json,text,binary}_file. (The "experimental" will be removed soon.). The files contain instances of this message:
https://github.com/bazelbuild/bazel/blob/master/src/main/java/com/google/devtools/build/lib/buildeventstream/proto/build_event_stream.proto
Natan,
If I understand correctly, you want to transitively propagate the information from each aspect node out into a single result. To do this, build the transitive set in your aspect rule implementation and pass it via the "provider" mechanism [^1]. I wrote up some examples on bazel aspects, perhaps you'll find it useful[^2].
https://github.com/pcj/bazel_aspects/blob/master/aspects.bzl#L94-L104
https://github.com/pcj/bazel_aspects
I am trying to create a rule (maybe one already exists?), to un-tar a file during a bazel build step.
If I understand correctly all output files need to be known during the "Analysis Phase". To work around this I have a file lets call it manifest.txt which lists all the files in the tar file. However, I don't quite understand how I can read this file as a list outputs for my skylark rule? Is there an easy way to read a file and have each line be a generated output?
Thanks.
This is only possible if the manifest file is a source file, i.e. it is NOT generated by some rule in the build.
Rules must declare all their outputs without relying on the content of generated files, therefore it's not supported to create for example a genrule whose outs is computed based on a manifest file that's generated by another rule.
To work with a tar file input, the rule needs to unpack the tar with an action, and ultimately produce a predictable amount of outputs (i.e. independent of how many files are there in the tar). Typically this is done by repacking the outputs, that is, the rule would consume one tar and produce another.