I would like to have a file optionally provided to the data attribute and have my program to deal with the file being present or not.
If the file doesn't exist, bazel reports missing input file...
cc_binary(
name = "test",
srcs = ["main.c"],
data = [":conf.yml"]
)
Is there any way to make it optional ?
No. Bazel requires all inputs to be declared statically for hermeticity and determinism. The way forward here is to make an empty conf.yml and have the program determine if it's empty or not.
Related
Is there a way to control the Bazel build to generate wanted temp files for a list of source files instead of just using the command line option "--save_temps"?
One way is using a cc_binary, and add "-E" option in the "copts", but the obj file name will always have a ".o". This kind of ".o" files will be overwriten by the other build targets. I don't know how to control the compiler output file name in Bazel.
Any better ideas?
cc_library has an output group with the static library, which you can then extract. Something like this:
filegroup(
name = "extract_archive",
srcs = [":some_cc_library"],
output_group = "archive",
)
Many tools will accept the static archive instead of an object file. If the tool you're using does, then that's easy. If not, things get a bit more complicated.
Extracting the object file from the static archive is a bit trickier. You could use a genrule with the $(AR) Make variable, but that won't work with some C++ toolchains that require additional flags to configure architectures etc.
The better (but more complicated) answer is to follow the guidance in integrating with C++ rules. You can get the ar from the toolchain and the flags to use it in a custom rule, and then create an action to extract it. You could also access the OutputGroupInfo from the cc_library in the rule directly instead of using filegroup if you've already got a custom rule.
Thanks all for your suggestions.
Now I think I can solve this problem in two steps(Seems Bazel does not allow to combine two rules into one):
Step1, add a -E option like a normal cc_libary, we can call it a pp_library. It is easy.
Step2, in a new rules, its input is the target of pp_library, then in this rule find out the obj files(can be found via : action.outputs.to_list()) and copy them to the a new place via ctx.actions.run_shell() run_shell.
I take Bazel: copy multiple files to binary directory as a reference.
This answer to Looking for examples of Bazel genrules that generate data files states that "A genrule needs to know all its input files and output files".
Is it possible to write a genrule whose outputs depend on the input file? For example
genrule(
name = "creator",
cmd = "xargs -a $< touch",
srcs = [ "meow.txt" ],
)
This would create a file for each word in meow.txt, e.g.
Chico
Groucho
Harpo
would create three files.
The list of output files depends entirely on the content of meow.txt and is not known in the BUILD file.
That is beyond the capabilities of genrule. However, it is possible write a custom rule that outputs a directory, into which arbitrary files can be written when the action executes.
I'm having trouble understanding how to construct proper label forms when dealing with external repositories (directories with their own WORKSPACE).
What is the semantic meaning of characters like /, :, // or #?
For example:
#foo/bar
#foo:bar
//foo
foo
Do they preserve their meaning when used in an external repository? Also, is //external special in any way?
/ is a separator for package and target names.
relative/package/to/my:target
//absolute/package/to:my/file/target.java
A package is defined as a directory containing a BUILD or BUILD.bazel file.
: is the lexeme for selecting a rule or file target in a package.
//my/package:my_java_binary
Selects the target my_java_binary defined in <workspace root>/my/package/BUILD
//my/package:file.go
Selects the file <workspace root>/my/package/file.go if <workspace root>/my/package/BUILD exists, and if there's a rule in that BUILD file that references it.
//:my/nested/file.txt
Selects the file <workspace root>/my/nested/file.txt if <workspace root>/BUILD exists, but not in the my and my/nested subdirectories.
// is the location of the current or closest parent directory containing a WORKSPACE file.
Otherwise known as workspace root.
# is used for referencing a repository by its name when used to the left of //
#io_bazel_rules_scala//scala:scala.bzl: look into your WORKSPACE file for a repository named io_bazel_rules_scala. Usually defined using http_archive or git_repository.
#//my/package:target: # alone refers to the current workspace.
As of Bazel 0.16.0, # can be used in package names.
Do they preserve their meaning when used in an external repository?
Yes, think of the #<repository> syntax as a namespace mechanism.
Also, is //external special in any way?
Yes, it's used for the bind function, which is not recommended anymore. bind lets you give a target an alias in //external.
I want to extract some specific resource from jar (which, in turn, is downloaded as a part of http archive) and while I know how to achieve this in principle I don't know what is the most bazelish and minimal way to achieve this.
I've naïvely tried (after reading this answer) to do something like this:
new_http_archive(
name="some_jar_contents",
url="#some_archive//:lib/some_jar.jar",
build_file_content="""
filegroup(
name = "srcs",
srcs = glob(["*"]),
visibility = ["//visibility:public"]
)
"""
)
However, I'm predictably getting java.net.MalformedURLException: no protocol:
The problem you are getting is caused by the fact that Bazel does not understand the url #some_archive//:lib/some_jar.jar.
It cannot infer the protocol to use for fetching the Jar, and the exception is thrown.
The rule new_http_archive is intended to be used for fetching a compressed archive from certain remote location, then build & expose it as an external target to current repo.
You need to change the url parameter to the actual URL in order to let Bazel fetch the Jar with the resource you want to extract.
Then, in build_file_content parameter part, use genrule to move the desired resource file, and export it.
A working example can be found in this private gist.
The example does the following things:
Fetches the Jar for AutoValue in WORKSPACE
Uses genrule to extract the autovalue.vm file from the Jar
Have a Python Program that reads and prints content of autovalue.vm
Helpful resources:
genrule
"Make" Variables
Updating the runfiles tree structure
Problem
I wonder how to inform bazel about dependencies unknown at declaration time, but known at build time (a.k.a implicit dependencies, dynamic dependencies, ...). For instance when compiling C++ sources, the .cpp source file will depends on some header files and this information is not available when writing the BUILD file. It needs to be retrieve at build time. Whatever is the solution to get the information (dry-run, generating depfile, parsing stdout), it needs to be done at build time and the information need to be retrieved to bazel build graph.
Since skylark does not allow to do I/O, for instance to read a generated depfile or to parse stdout result containing a dependency list, I have no clue on how to deal with it.
Behind implicit dependencies, I am looking for correct incremental build.
Example
To experiment this problem I have created a simple tool, just_a_tool.exe which takes an input file, read a list of file from it, and concatenate the content of all these file to an output file.
command line example:
just_a_tool.exe --input input.txt --depfile dep.d output.txt
dep.d contains the list of all the read files.
Issue
If I change the content of test1.txt, test2.txt, or test3.txt, bazel does not rebuild output.txt file. Of course, because it does not know there were dependencies.
Example files
just_a_tool.bzl
def _impl(ctx):
exec_path = "C:/Code/JustATool/just_a_tool.exe"
for f in ctx.attr.source.files:
source_path = f.path
output_path = ctx.outputs.out.path
dep_file = ctx.actions.declare_file("dep.d")
args = ["--input", source_path, "--dep_file", dep_file.path, output_path]
ctx.actions.run(
outputs=[ctx.outputs.out, dep_file],
executable=exec_path,
inputs=ctx.attr.source.files,
arguments=args
)
jat_convert = rule(
implementation = _impl,
attrs = {
"source" : attr.label(mandatory=True, allow_files=True, single_file=True)
},
outputs = {"out": "%{name}.txt"}
)
BUILD
load("//tool:just_a_tool.bzl", "jat_convert")
jat_convert(
name="my_output",
source=":input.txt"
)
input.txt
test1.txt
test2.txt
test3.txt
Goal
I want to do correct and fast incremental build for the following situation:
Generate reflection data from C++ sources, this custom tool execution depends on header file included in my source files.
Use a internal tool to build asset file which can include other files
Run a custom preprocessor on my shaders allowing a #include feature
Thanks!
Bazel's extension language doesn't support creating actions with a dynamic set of inputs, where this set depends on the output of a previous action. In other words, custom rules cannot run an action, read the action's output, then create actions with those inputs or update (or prune the set of) inputs of already created actions.
Instead I suggest adding attribute(s) to your rule where the user can declare the set of files that the sources may include. I call this "the universe of headers". The actions you create depend on this user-defined universe, so the set of action inputs is completely defined. Of course this means these actions potentially depend on more files than the cpp files, which they process, include.
This approach is analogous to how the cc_* rules work: a file in cc_*.srcs can include other files in the srcs of the same rule and from hdrs of dependencies, but nothing else. Thus the union of srcs + hdrs of (direct & transitive) dependencies defines the universe of header files that a cpp file may include.