I'm setting up TFS 2015 on-prem and I'm having an issue on my last build step, Publish Build Artifacts. For some reason, the build agent appears to be archiving old binaries and I'm left with a huge filepath:
E:\TFSBuildAgent\_work\1a4e9e55\workspace\application\Development\project\WCF\WCF\obj\Debug\Package\Archive\Content\E_C\TFSBuildAgent\_work\1a4e9e55\workspace\application\Development\project\WCF\WCF\obj\Debug\Package\PackageTmp\bin
I'm copying the files using the example minimatch pattern to begin with:
**\bin
I'm only testing at the moment so this is not a permanent solution but how can I copy all binaries that are in a bin folder but not a descendant of obj?
From research I think that this should work, but it doesn't (It doesn't match anything):
**!(obj)**\bin
I'm using www.globtester.com to test. Any suggestions?
On a separate note, I'll look into the archiving issue later but if anyone has any pointers on it, feel free to comment. Thanks
In VSTS there are two kinds of pattern matching for URLs that are built-in to the SDKs. Most tasks nowadays use the Minimatch pattern as described in Matt's answer. However, some use the pattern that was used by the 1.x Agent's Powershell SDK. That format is still available in the 2.x Agent's Powershell SDK by the way.
So that means there are 5 kinds of tasks:
1.x agent - Powershell SDK
2.x agent - Node SDK
2.x agent - Powershell 1 Backwards compatibility
2.x agent - Powershell 3 SDK - Using find-files
2.x agent - Powershell 3 SDK - Using find-match
The ones in bold don't Minimatch, but the format documented in the VSTS-Task-SDK's find-files method.
The original question was posted in 2015, at which point in time the 2.x agent wasn't yet around. In that case, the pattern would, in all likelihood, be:
**\bin\$(BuildConfiguration)\**\*;-:**\obj\**
The -: excludes the items from the ones in front of it.
According to Microsoft's documentation, here is a list of
file matching patterns you can use. The most important rules are:
Match with ?
? matches any single character within a file or directory name (zero or one times).
Match with * or +
* or + matches zero or more characters within a file or directory name.
Match with # sign
# matches exactly once.
Match with Brackets (, ) and |
If you're using brackets with | it is treated as a logical OR, e.g. *(hello|world) means "Zero or more occurrances of hello or world"
Match with Double-asterisk **
** recursive wildcard. For example, /hello/**/* matches all descendants of /hello.
Exclude patterns with !
Leading ! changes the meaning of an include pattern to exclude. Interleaved exclude patterns are supported.
Character sets with [ and ]
[] matches a set or range of characters within a file or directory name.
Comments with #
Patterns that begin with # are treated as comments.
Escaping
Wrapping special characters in [] can be used to escape literal glob characters in a file name. For example the literal file name hello[a-z] can be escaped as hello[[]a-z].
Example
The following expressions can be used in the Contents field of the "Copy Files" build step to create a deployment package for a web project:
**\?(.config|.dll|*.sitemap)
**\?(.exe|.dll|.pdb|.xml|*.resx)
**\?(.js|.css|.html|.aspx|.ascx|.asax|.Master|.cshtml|*.map)
**\?(.gif|.png|.jpg|.ico|*.pdf)
Note: You might need to add more extensions, depending on the needs of your project.
Related
Im trying to copy a file in docker with below format
database-20.2.1.zip
I have tried using something like below, but it does not seems to work
COPY databse-+([0-9]).+([0-9]).+([0-9]).zip /docker-entrypoint-initdb.d/database.zip
Is this something that can be done in docker copy ?
Dockerfile COPY uses shell globs, not regular expressions. The actual implementation uses the Go filepath.Match syntax. That syntax doesn't allow some of the combinations that regular expressions do: you can match any single digit [0-9], or any number of characters *, but not any number of digits.
Depending on how strict you want to be about what files you'll accept and how consistent the filename format is, any of the following will work:
COPY database-[0-9][0-9].[0-9].[0-9].zip ./database.zip
COPY database-*.*.*.zip ./database.zip
COPY database-*.zip ./database.zip
In all cases note that the pattern can match multiple files in the build context. If the right-hand side of COPY is a single file name (not ending with /) but the glob matches multiple files you will get a build error. In this case that's probably what you want.
I want to replace a version number by a more current one (i.e. getting it from SVN, which is working fine). For testing purposes I try to replace what I find with something simple, but not even this is working.
This regular expression looks fine (e.g. tested in https://regex101.com/) and correctly matches my version number.
RegEx:
([1-9][0-9]|[0-9])\.([1-9][0-9]|[0-9])\.([1-9][0-9][0-9][0-9]|[1-9][0-9][0-9]|[1-9][0-9]|[0-9])
Text to match (that is not matched):
Title="MyTitle 1.0.0"
According to https://regex101.com/ this should work. But Visual Build cannot find any match and no replacement is done.
I can confirm that file access is working, because Visual Build adds the text "error while replacing" to the specified file.
I'm using Visual Build Professional v10.
I'm not exactly sure what's wrong with your existing regex, but when I've had to do similar replacements I've tried to manufacture a replacement strategy that is a little different - basically to keep the regex as simple as possible. You could try something like this:
Text or regex to find:
Title=.*
e.g., just match the correct line of the file. Don't worry about the existing version number values.
Text to replace matches with:
Title="MyTitle %v1%.%v2%.%v3%"
I'm assuming you have three macros v1, v2, v3 for your version number parts. If you have one for the app title you could obviously insert that there too.
The Lua specs say about package.config (numbering added by me):
The first line is the directory separator string. Default is '\' for Windows and '/' for all other systems.
The second line is the character that separates templates in a path. Default is ';'.
The third line is the string that marks the substitution points in a template. Default is '?'.
The fourth line is a string that, in a path in Windows, is replaced by the executable's directory. Default is '!'.
The fifth line is a mark to ignore all text before it when building the luaopen_ function name. Default is '-'.
My paraphrasing:
Absolutely clear (example for Windows/other systems makes it fool proof)
There can be multiple paths in a path string. They are separated by this symbol (; by default).
Wherever Lua finds this character in the path string (? by default), it will replace it with the module name supplied to the require or package.searchpath functions and check whether that file exists.
So far, so good, but the last two lines aren't entirely clear to me.
Why does it say "in a path in Windows"? Does that mean on other platforms, this doesn't have any significance? If so, why?
It took me a while to make sense of this, but eventually another part of the specs gave me a hint:
The name of this C function is the string "luaopen_" concatenated with a copy of the module name where each dot is replaced by an underscore. Moreover, if the module name has a hyphen, its prefix up to (and including) the first hyphen is removed. For instance, if the module name is a.v1-b.c, the function name will be luaopen_b_c.
So is this symbol (- by default) intended to make different versions of a library available at the same time – potentially with an unprefixed symlink to the newest version so that the same library would be accessible on two paths (i.e. under two module names), but with only one C symbol name?
4: Applications for Linux have libraries installed system-wide; however, for Windows, libraries can be installed in the current directory.
5: Versioning and project forking, I believe, would be the reason behind this.
I don't know how it happens that the option named "Restrict where this project can be run" from Jenkins seems to allow only a single value inside "Label Expression" field.
I tried lots of combinations in order to add more than one label and I wasn't able to find any way to put two.
I need to mention that I need AND between these labels.
The irony is that this option even has an Info button which loads some documentations, which is missing to say how an expression is supposed to look like. Another small nail in the Jenkins UX coffin. On this one neither Google helped.
There is a Jenkins bug causing the help text not to be shown. It is present since 1.585 and fixed since 1.621 (or 1.609.3 respectively).
Here is the help text:
If you want to always run this project on a specific node/slave, just specify its name. This works well when you have a small number of nodes.
As the size of the cluster grows, it becomes useful not to tie projects to specific slaves, as it hurts resource utilization when slaves may come and go. For such situation, assign labels to slaves to classify their capabilities and characteristics, and specify a boolean expression over those labels to decide where to run.
Valid Operators
The following operators are supported, in the order of precedence.
(expr) parenthesis
!expr negation
expr&&expr and
expr||expr or
a -> b "implies" operator. Equivalent to !a|b. For example, windows->x64 could be thought of as "if run on a Windows slave, that slave must be 64bit." It still allows Jenkins to run this build on linux.
a <-> b "if and only if" operator. Equivalent to a&&b || !a&&!b. For example, windows<->sfbay could be thought of as "if run on a Windows slave, that slave must be in the SF bay area, but if not on Windows, it must not be in the bay area."
All operators are left-associative (i.e., a->b->c <-> (a->b)->c ) An expression can contain whitespace for better readability, and it'll be ignored.
Label names or slave names can be quoted if they contain unsafe characters. For example, "jenkins-solaris (Solaris)" || "Windows 2008"
Is there an existing POSIX sh grammar available or do I have to figure it out from the specification directly?
Note I'm not so much interested in a pure sh; an extended but conformant sh is also more than fine for my purposes.
The POSIX standard defines the grammar for the POSIX shell. The definition includes an annotated Yacc grammar. As such, it can be converted to EBNF more or less mechanically.
If you want a 'real' grammar, then you have to look harder. Choose your 'real shell' and find the source and work out what the grammar is from that.
Note that EBNF is not used widely. It is of limited practical value, not least because there are essentially no tools that support it. Therefore, you are unlikely to find an EBNF grammar (of almost anything) off-the-shelf.
I have done some more digging and found these resources:
An sh tutorial located here
A Bash book containing Bash 2.0's BNF grammar (gone from here) with the relevant appendix still here
I have looked through the sources of bash, pdksh, and posh but haven't found anything remotely at the level of abstraction I need.
I've had multiple attempts at writing my own full blown Bash interpreters over the past year, and I've also reached at some point the same book appendix reference stated in the marked answer (#2), but it's not completely correct/updated (for example it doesn't define production rules using the 'coproc' reserved keyword and has a duplicate production rule definition for a redirection using '<&', might be more problems but those are the ones I've noticed).
The best way i've found was to go to http://ftp.gnu.org/gnu/bash/
Download the current bash version's sources
Open the parse.y file (which in this case is the YACC file that basically contains all the parsing logic that bash uses) and just copy paste the lines between '%%' in your favorite text editor, those define the grammar's production rules
Then, using a little bit of regex (which I'm terrible at btw) we can delete the extra code logic that are in between '{...}' to make the grammar look more BNF-like.
The regex i used was :
(\{(\s+.*?)+\})\s+([;|])
It matches any line non greedily .*? including spaces and new lines \s+ that are between curly braces, and specifically the last closing brace before a ; or | character. Then i just replaced the matched strings to \3 (e.g. the result of the third capturing group, being either ; or |).
Here's the grammar definition that I managed to extract at the time of posting https://pastebin.com/qpsK4TF6
I'd expect that sh, csh, ash, bash, would contain parsers. GNU versions of these are open source; you might just go check there.