let's say that I want to generate multiple rules in a macro based on repository contents - something like:
def mymacro(dests) :
for d in dests:
myrule(name = d, ...)
# in a BUILD:
mymacro(dests = glob(["some/pkg/path/**"]))
So far, I've always gotten an empty list when I try this (although the path has many entries). Is such a thing possible or am I doing something wrong?
Using glob like that should work fine. Glob also works within a macro (though you have to do native.glob(...))
The glob pattern is probably just not matching anything. Glob will happily return an empty list if the pattern matches nothing (you can pass allow_empty=False to change that behavior).
Note that glob will not traverse into subpackages. So with your example some/pkg/path/**, if there's a build file in one of the subdirectories (some/BUILD, some/pkg/BUILD, some/pkg/path/BUILD, etc) then glob won't looking for anything in that subdirectory. To make files visible from one package to another, typically you'll have a filegroup (maybe with its own glob) in one package that another package depends on.
Related
I have rule A implemented with a macro that uses declare_directory to produce a set of files:
output = ctx.actions.declare_directory("selected")
Names of those files are not known in advance. The implementation returns the directory created by declare_directory with the following:
return DefaultInfo(
files = depset([output]),
)
Rule A is included in "srcs" attribute of rule B. Rule B is also implemented with a macro. Unfortunately the list of files passed to B implementation through "srcs" attribute only contains the "selected" directory created by rule A instead of files residing in that directory.
I know that Args class supports expansion of directories so I could pass names of all files in "selected" directory to a single action. What I need, however, is a separate action for every individual file for parallelism and caching. What is the best way to achieve that?
This is one of the intended use cases of directory outputs (called TreeArtifacts in the implementation), and it's implemented using ActionTemplate:
https://github.com/bazelbuild/bazel/blob/c2100ad420618bb53754508da806b5624209d9be/src/main/java/com/google/devtools/build/lib/actions/ActionTemplate.java#L24-L57
However, this is not exposed to Starlark, and has only a couple usages currently, in the Android rules AndroidBinary.java and C++ rules CcCompilationHelper.java. The Android rules and C++ rules are going to be migrated over to Starlark, so this functionality might eventually be made available in Starlark, but I'm not sure of any concrete timelines. It would probably be good to file a feature request on Github.
I'm writing an Apache Beam pipeline in python and trying to load multiple text files but encounter an error when using the pattern match. When I pass in an exact filename, the pipeline runs correctly.
For example:
files = p | 'Read' >> ReadFromText('lyrics.txt')
However, when using pattern match an error occurs:
files = p | 'Read' >> ReadFromText('lyrics*')
IOError: No files found based on the file pattern
In this example, I have several files that start with "lyrics".
I've tried many different pattern types but haven't had any success with anything except passing the complete file name. Is there a different way to apply pattern match in this case?
Updated with answer
If you're on Windows don't forget to use a backslash instead of forward slash when specifying directories. For example: ReadFromText('.\lyrics*')
This looks like a bug. I've filed https://issues.apache.org/jira/browse/BEAM-7560. In the meantime, try an absolute path or ReadFromText('./lyrics*').
I am trying to get all directories in a path. For example, from a/b/c/d.e I would like to get a, a/b, and a/b/c. I can achieve this by calling functions like posixdirname several times. The problem is with paths like a/b/c/. I would like to get a, a/b, and a/b/c. Since there is a / following c, c should be a directory that I would like to list. But functions like dirname return a/b instead of a/b/c when given a/b/c/ as input.
Can I just split on / to get the list of directories or is there an edge case where this wouldn't work?
The only special cases I know are
with file name at the end
with a / at the end
with a folder name called a.e (may or may be not at the end)
symbolic link
Windows path (which is using \ but I think it is not applying in your case)
User input error, put multiple /. e.g. /tmp//something which is supported in some programmes
If your function is supporting all of the above cases, I think it is fine.
For the import of my Sass files, I use sass-rails' (https://github.com/rails/sass-rails) glob feature. It says
Any valid ruby glob may be used
I want to exclude a directory and a file when using #import. Any ruby code using blocks don't work in this scenario. But even trying to exclude a single file doesn't work the way I want.
Consider this tree structure
/_bar.scss
/_foo.scss
/all.scss
For example, I want to exclude the file _foo.scss. I read here https://stackoverflow.com/a/27707682/228370, using a ! you can negate a pattern.
I tried the following:
Dir["{[!_foo]}*.scss"]
=> ["all.scss"]
But this skips _bar.scss. When looking into the glob reference of Ruby (http://ruby-doc.org/core-2.2.0/Dir.html#method-c-glob) it becomes clear why:
[set]
Matches any one character in set. Behaves exactly like character sets in Regexp, including set negation ([^a-z]).
(apparently, negation can be achieved with ! AND ^)
Because we have an underscore in our pattern, every file with an underscore gets excluded.
But what would be the solution, to exclude a fixed file?
There's probably a regex way of doing it. But if you're talking about one specific file, it might be easier to just do:
Dir["*.scss"].reject { |i| i == '_foo.scss' }
Is it possible using .tfignore to add a wildcard to directories? I assumed it would have been a case of just adding an asterisk wildcard to the directory segment. For example:
\path\*\local.properties
However this does not work and I am unsure how I would achieve such behaviour without explicitly declaring every reference that I need excluding. .
Documentation
# begins a comment line
The * and ? wildcards are supported.
A filespec is recursive unless prefixed by the \ character.
! negates a filespec (files that match the pattern are not ignored)
Extract from the documentation.
The documentation should more correctly read:
The * and ? wildcards are supported in the leaf name only.
That is, you can use something like these to select multiple files or multiple subdirectories, respectively, in a common parent:
/path/to/my/file/foo*.txt
/path/to/my/directories/temp*
What may work in your case--to ignore the same file in multiple directories--is just this:
foo*.txt
That is, specify a path-less name or glob pattern to ignore matching files throughout your tree. Unfortunately you have only those two options, local or global; you cannot use a relative path like this--it will not match any files!
my/file/foo*.txt
The global option is a practical one because .tfignore only affects unversioned files. Once you add a file to source control, changes to that file will be properly recognized. Furthermore, if you need to add an instance of an ignored name to source control, you can always go into TFS source control explorer and manually add it.
It seems this is now supported
As you see I edited tfignore in the root folder of the project such that any new branch will ignore its .vs folder when being examined for source control changes
\*\.vs
Directory/folder name wildcarding works for me in VS2019 Professional. For example if I put this in .tfignore:
*uncheckedToTFS
The above will ignore any folder named ending with "uncheckedToTFS", regardless of where the folder is (it doesn't have to be top level folder, can be many levels deep).