Is there a way to specify optional dependencies in Bazel?
I'd like to make a rule to somewhat mirror Kitware's ExternalData, but I would like to see if I can enable workflows where the developer edits the file in-tree, ideally without needing to modify the BUILD file.
Ideal Workflow
Define a rule, external_data, which can fetch a file from a given server given its SHA-512.
If the file already exists, check it's SHA-512.
If that is what is requested, symlink / copy this file (ensuring that no tests can modify the original file).
If it is different, print a warning, but proceed as normal, to allow for developers to quickly modify the large files as they need.
I would like to do this such that Bazel can switch between the file being present and not, and be robust to false-positives on caching. An example scenario that I would like to avoid, if I were to not include it as an optional dependency:
In a prior run, the file was in the workspace, Bazel built the target, everything's fine and dandy.
Developer removes the file from the workspace after uploading, satisfied with their changes and wanting to test the download process.
When running the downstream target, Bazel doesn't care about the change in the workspace since it's not an explicit dependency, and the symlink is invalidated, and the test crashes and burns.
To me, it seems like I'd run into this if I tried to implement a repository_rule rule which manually checks for the file existence, and conditionally executes (I'm not sure if analysis would retrigger this rule being "evaluated" if Step 2 happens.).
Workaround
My current thought for an alternative workflow is to have an explicit option for external_data, use_workspace: if False, it will download the file; if True, it will just mirror exports_files([]). The developer can then set this when modifying files.
(Ideally, I'd like to optionally include a file which indicates the SHA (${file}.sha512), but this seems to go back to the original ask.)
One workaround is to use Bazel's glob(...) method to effectively check for file existence.
If you have a file, say basic.bin.sha512, and you want a rule to switch modes based on that file's existence, you can use glob(["basic.bin.sha512"]), which will either match the package file exactly or return an empty list.
I had tinkered around with using this on larger sets of files, and it appears to work. However, for the time being, I've erred to having a sort-of explicit "development" mode for the target definition to keep the Bazel build relatively consistent, regardless of what files may be checked out.
Here's an example usage:
https://github.com/EricCousineau-TRI/external_data_bazel/blob/4bf1dff/WORKFLOWS.md#edit-files-in-a-sha512-group
Related
the app where I am working on was audited and the security team has a concern that consists in: so, for some reason if I extract the .ipa's payload I am able to identify which CI/CD tool was used to generate it. For example:
Terminal:
cd MyApp.app <- payload
strings * > ~/Desktop/file.txt
Inside the file I can find that we are using jenkins
/Volumes/jenkins-workspace/MyApp-Generic/MyApp/MyApp/SomeViewModel.swift
Has anyone faced this problem (if we can call it a problem)? Is there a way to obfuscate or completely remove this kind of references from the IPA?
Thanks
Does that file have a fatalError() call in it? If so, the file and line number will be reported when it runs. Here's a way to fix that if you really need the fatalError call.
If any of your source files have #file compiler directives in them, they will record the path to the file in the executable.
Swift 5.3 introduces a #fileID identifier which produces a shorter
string than #file. The #fileID string contains the filename and module
name, but leaves out the rest of the path to the file; this saves
space, improves performance, and avoids accidentally embedding private
information like the developer’s home directory name in binaries.
Compiler-generated error messages (like force-unwraps) and standard
library assertions like precondition and fatalError now use #fileID
strings, and we recommend you use them instead of #file in production
code. (SE-0285, 65514304)
This pretty much sums my problem. I am using the #file identifier in some methods (mostly for logs) and every file that calls that methods will appear embedded in the app binary.
Ref: https://developer.apple.com/documentation/xcode-release-notes/xcode-12-release-notes
I would like a set of rules from my_package.bzl to be accessible to all BUILD files of a workspace without having to load my_package.bzl in the BUILD files. Basically I want the rules in the package to look like native rules. How can I achieve this?
I was thinking maybe there's a line I could add to one of the .bazelrcs or to the WORKSPACE file of the the project.
This can be achieved by adding a prelude_bazel file at //tools/build_rules:prelude_bazel (this must be a package, so tools/build_rules must contain a BUILD file).
This will be loaded and prepended to all BUILD files loaded by Bazel.
However, there are a few things to consider before going this route. It's currently undocumented, and while doing some searching to find any info on this feature, it's unclear if it will remain a part of Bazel.
It may also have performance / scaling problems. If the prelude were to change (or any of its dependencies), every BUILD file would have to be reloaded, and this may take some time depending on the size of the build graph.
On my way to migrate an existing build to bazel, i have a submodule mod1 that has some JUnit tests reading files from a "testdata" directory. When trying to load those files, i have to use "mod1/testdata/test.txt" instead of "testdata/test.txt", i.e. the unit tests have to be aware of their corresponding bazel module directory.
(1) Is this the correct behaviour for bazel 0.23.2#debian and 0.23.2-homebrew?
(2) Is there a way to use the .java tests without changes, and to remove the need for a "mod1" prefix in bazel data/ runfiles?
My sample project is here: https://gitlab.com/jhinrichsen/bazel-data-test. I am looking for a way to use the same path "testdata/test.txt" for both root module and submodule. In my example project, bazel test AllTests suceeds, while bazel test mod1/AllTests fails because i need to prepend "mod1/" to "testdata/test.txt".
Not looking for a resources/classpath based solution as i cannot modify the existing test sources.
The behavior that you are seeing is indeed the correct behavior, and there is no way to strip the "mod1" prefix with the native Java rules. Anything you include with data will be scoped to its own package in the way you're seeing.
The reason for this is pretty straightforward. Let's say that your test target, //mod1:AllTests, also depended on a hypothetical //mod2:tests library. And let's say that hypothetical library also had a testdata/test.txt as a data dependency. The multiple test.txt files would conflict unless they were namespaced to their packages.
If you absolutely cannot modify the test source at all, then you are pretty much stuck. Here's a previous discussion about this:
https://groups.google.com/forum/#!topic/bazel-discuss/w6TDwSZvN0k
I would recommend if you're trying to work with Bazel, you accept the concept of runfiles and modify your tests to either work with the runfiles structure, or accept a command-line argument for where to find the test data.
I need to preprocess my launch storyboard file to change its background color based on same build time configuration. I've added the following rule under Xcode Build Rules:
But I'm having some problems getting this to work. Actual script is fine.
First, the rule is not being executed. I guess if fails to match the file name. When I specify to match *.storyboard, it gets executed, but I do not want it matching all storyboard files, just that one. Of course, I could check the file name in the script itself.
Secondly, when it gets executed, it does not cause any side effects. I guess that the rules that follow, more specifically Storyboard compiler, fail to pick this file up. It might be that I'm using wrong file paths, or that my change is being overwritten with original file sometime later in the build process. Does anyone have more experience in this?
How do I make the test host/bundle loader dynamic based on the current scheme? Right now the value is set to:
$(BUILT_PRODUCTS_DIR)/MyApp1.app/MyApp1
The problem is I have 4 apps in the workspace and I would like to use the same unit testing suite for all of them. How do I dynamically change the "MyApp1" part based on the current scheme? Is it an environment variable based during build? I tried setting it to things like $(PROJECT_NAME) but those seem to get the name of the test suite.
To do this you need to have a variable inside the build settings - which seems simple, but it's not. If you set an environment variable through a Pre- or Post- step in the application or test scheme, it does not seem as though it will be picked up here. The build settings, after all, happen before the build. Same is true of a preprocessor macro, though doing this using xcodebuild and passing in a custom option may work.
The only way I know of to do this is to use an xcconfig file. Create one and apply it to (at the least) your test target. The content should include something like this:
THINGUNDERTEST=FooBar
Now in your project settings, wether in an xcconfig or the project file, set BUNDLER_LOADER to:
$(BUILT_PRODUCTS_DIR)/$(THINGUNDERTEST).app/$(THINGUNDERTEST)
That will work. Now you can change THINGUNDERTEST through various means and get at least some dynamic behavior. This may work for you or may not, depending on your needs - but it probably only a starting point.