I have a command that compiles and runs a program, but the intermediate files are randomly named (but contained within a directory). E.g.
build foo.src bar.src -o output_dir
run output_dir
Bazel requires me to pre-declare all of the outputs of my rule, but I can't do that because they're randomly named. Can I somehow name an entire directory instead?
The only alternative I can think of is having the rule zip/unzip the directory before/after it runs the commands, which is a pretty awful solution.
Edit: I found an issue exactly describing the "just zip/unzip everything" solution here. The closing comment says to just use the rules from rules_pkg to zip/unzip stuff. Unfortunately it requires Python too.
Some of the comments in that thread suggest you can use declare_directory() but I don't think that really works.
There are tree artifacts. An example of how to use an tree artifact can be found here.
Tree artifacts are problematic for caching since Bazel is not aware of the content of the corresponding directory and if for some reason the content of a tree artifact is different between two machines that use the same Bazel cache and same Bazel configuration you are trouble.
Related
py_binary finally generates an executable file or an alias for a py script? What are its benefits? If it is an executable file, it will lose the meaning of python.
Making something executable can be just adding a chmod +x and slapping a #!/foo/bar line on top, the thing itself is still whatever interpreter code it was before.
In the case of bazel, it will add a wrapper script that will set up an execution environment before dispatching to the Python code. Consider e.g. Bazel's runfiles, but also other py_library targets.
In addition, you can use the target in places where an executable is required as attribute for another target. A single Python file doesn't have any dependencies Bazel knows about, so that would technically fit there but would not integrate well with Bazel.
Is there a list of tools that are assumed to be always in the PATH when a Bazel target runs a shell command?
This is relevant for creating isolated build environments. AFAIU (see https://github.com/NixOS/nixpkgs/pull/50765#issuecomment-440009735) by default Bazel picks up tools from /bin and /usr/bin when in strict mode.
But what can ultimately be assumed about the minimal content of those? For example, I saw awk to be used liberally. But then git as well, which sounds border-line.
I imagine the exact set might correspond to whatever Google-internal Bazel expects to find in Google's build images bin directories. At least for BUILD rules open-sourced by Google.
Is there such a definitive list? Thank you.
As far as I can tell, your assessment of the tool usage is correct, and unfortunately I'm not aware of such a list.
There should be one, and Bazel should treat the shell as a toolchain. Alas nobody is working on that at the moment. See https://github.com/bazelbuild/bazel/issues/5265.
Question
Is there any way I could use bazel query or aspects to identify where on the package path bazel is picking up a package? Something similar to the which command.
The documentation suggests using the --show_package_location. However that is deprecated and no longer supported, see #5592. Additionally, my attempts at using it have not uncovered much useful information. I have tried bazel query //some/target/... --output label_kind --show_package_location as well as other permutations with bazel build and it doesn't add output anything different to the console output.
Motivation
I have two different directories on my package path for fetch, query and build.
--package_path=%workspace%:%workspace%/__fuse__
This configuration supports a workflow where users perform sparse-checkouts of our large repository, while still being able to build code that has not been locally checked out. When building targets, Bazel checks for the locally checked out version of package, and if that doesn't exist, it searches a read only fuse mount.
Sometimes it's unclear to users where a package is getting picked up from, i.e. whether it's the locally checked out version or the one served from fuse. This becomes problematic when they delete or move a Bazel package, and Bazel picks up the version on the fuse mount.
It'd be nice if I could point them to a command that would map each package to where it's being picked up. For example, if i ran the command on ...
//some/package/foo --> package_path/some/package/foo
//some/package/bar --> other_package_path/some/package/bar
I completely missed this in the bazel query documentation.
With bazel query, I simply needed to add --output location, so provided I make a query like:
bazel query //some/package/... --output location
Then bazel query will output
/absolute/path/some/package/BUILD:lineno:colno target_kind label
for each target in //some/package/...
As part of our efforts to create a bazel-maven transition interop tool (that creates maven sized jars from more granular sized bazel jars),
we have written an aspect that runs on bazel build of the entire bazel repo and writes important information to txt files outputs (e.g.: jar file paths, compile deps targets and runtime deps targets, etc.)
We ran across an issue where the repo's code was changed such that some of the txt file were not written anymore. But the old txt file from previous runs (before the code change) remained!
Is there a way to know that these txt files are no longer relevant?
You should be able to run with --build_event_json_file=file.json and try to locate generated artifacts. For example we use it on ci.bazel.io to locate actual test.xml file that were generated: https://github.com/bazelbuild/continuous-integration/blob/09975cbb487a84a62ca1e43aa43e7c6fe078f058/jenkins/lib/src/build/bazel/ci/BazelUtils.groovy#L218
The definition of the protocol can be found in build_event_stream.proto
According to many different SO-questions, it should be possible to exclude files being copied/deployed using the Task "Copy and Publish Build Artifacts" in the new TFS build system.
However it doesn't work for me (it is not excluding anything). What could I be doing wrong:
This should work (I know the question is old but I needed an answer myself)
**\!(System.Windows.Interactivity.resources.dll|*.dll.config|*fluentassertions*)
This is a known issue of the build task “Copy and Publish Build Artifacts”,bitbonk.
Q: This step didn't produce the outcome I was expecting. How can I fix it?
This step has a couple of known issues:
Some minimatch patterns don't work.
It eliminates the most common root path for all paths matched.
Source Link: Utility: Copy and Publish Build Artifacts
Well, it's nothing business with your settings of minimatch. It's just not work for that build task. Certainly, you can also doulbe check your minimatch with Copy Files task to verify this.
For now, you can avoid these issues by instead using the Copy Files step and the Publish Build Artifacts step.
Note: If it's still not working on copy files step, you should pay attention to the architecture of file when using minimatch. There must be something wrong on it.
The task copies the files base on the contents you entered one line by one line and the "!" only exclude the files during the copy, it does not delete the files that already been copied. So with the "**\*" you entered in the first line, all the files have already been copied and published. You need to remove the first line in "Contents". And for the excluded files, if there are in the same folded, you need to exclude them in one line. For example: using
!?(1.txt|2.txt)
to exclude both 1.txt and 2.txt file instead of using
!1.txt
!2.txt