I have a Jenkins job that has a test.excludes parameter that I would like to default to **/*$* (ie exclude all inner classes). Normally I would specify this value in a file, but in this case, I don't want to submit any files since this is investigative work (as I see tests that are failing, I will add them to test.excludes).
The problem is that the $* in **/*$* is being expanded to the command line variables. Using **/*$$* only changes the problem to $$ being expanded to the pid. Escaping * using \ doesn't work (inner classes are still run). Escaping by wrapping the entire value in ' does nothing to prevent the $* from being replaced.
Is there a way to get the behavior I want?
The following really ugly expression works: $(echo '**/*$*').
Related
I want to create some "build traceability" functionality, and include the actual bazel command that was run to produce one of my build artifacts. So if the user did this:
bazel run //foo/bar:baz --config=blah
I want to actually get the string "bazel run //foo/bar:baz --config=blah" and write that to a file during the build. Is this possible?
Stamping is the "correct" way to get information like that into a Bazel build. Note the implications around caching though. You could also just write a wrapper script that puts the command line into a file or environment variable. Details of each approach below.
I can think of three ways to get the information you want via stamping, each with differing tradeoffs.
First way: Hijack --embed_label. This shows up in the BUILD_EMBED_LABEL stamping key. You'd add a line like build:blah --embed_label blah in your .bazelrc. This is easy, but that label is often used for things like release_50, which you might want to preserve.
Second way: hijack the hostname or username. These show up in the BUILD_HOST and BUILD_USER stamping keys. On Linux, you can write a shell script at tools/bazel which will automatically be used to wrap bazel invocations. In that shell script, you can use unshare --uts --map-root-user, which will work if the machine is set up to enable bazel's sandboxing. Inside that new namespace, you can easily change the hostname and then exec the real bazel binary, like the default /usr/bin/bazel shell script does. That shell script has full access to the command line, so it can encode any information you want.
Third way: put it into an environment variable and have a custom --workspace_status_command that extracts it into a stamping key. Add a line like build:blah --action_env=MY_BUILD_STYLE=blah to your .bazelrc, and then do echo STABLE_MY_BUILD_STYLE ${MY_BUILD_STYLE} in your workspace status script. If you want the full command line, you could have a tools/bazel wrapper script put that into an environment variable, and then use build --action_env=MY_BUILD_STYLE to preserve the value and pass it to all the actions.
Once you pick a stamping key to use, src/test/shell/integration/stamping_test.sh in the Bazel source tree is a good example of writing stamp information to a file. Something like this:
genrule(
name = "stamped",
outs = ["stamped.txt"],
cmd = "grep BUILD_EMBED_LABEL bazel-out/volatile-status.txt | cut -d ' ' -f 2 >\$#",
stamp = True,
)
If you want to do it without stamping, just write the information to a file in the source tree in a tools/bazel wrapper. You'd want to put that file in your .gitignore, of course. echo "$#" > cli_args is all it takes to dump them to a file, and then you can use that as a source file like normal in your build. This approach is simplest, but interacts the most poorly with Bazel's caching, because everything that depends on that file will be rebuilt every time with no way to control it.
I would like to customize one of environment variable depending on branch that is being build.
environment {
CUSTOM_ENV='xyz_${GIT_BRANCH}'
}
I'd like to get CUSTOM_ENV=xyz_dev for origin/dev and CUSTOM_ENV=xyz_master for origin/master. Not sure if its important, but its multibranch project in Jenkins.
I tried things like xyz_${GIT_BRANCH} or xyz_env.GIT_BRANCH, but none of this worked out.
If your shell happens to be compatible with ksh or bash then you can use the variable-expansion modifier ## to discard everything up to and including the final / character, leaving get just the part of ${GIT_BRANCH} that comes after that /. That would look like:
CUSTOM_ENV="xyz_${GIT_BRANCH##*/}"
Note the double-quotes " rather than the single-quotes ' you used in your question. Single-quotes prevent the evaluation of variables inside the quoted string, and that's definitely not what you want in this case.
If your shell does not understand the ## modifier then you'll have to use something like sed to get just the last part of ${GIT_BRANCH}. That would look like this:
CUSTOM_ENV="xyz_$(echo ${GIT_BRANCH} | sed -e 's#.*/##')"
When you are doing a substitution in jenkinsfile for a variable. It should always be in "". From environment directive , I guess you are using pipelines. So, you can leverage the groovy syntax to achieve string manipulation.
Something like,
environment {
GIT_BRANCH = 'orgin/master'.split('/')[1]
CUSTOM_ENV="xyz_${GIT_BRANCH}"
}
Given the bin\ directory inside the Delphi project contains the files Cert.pem and Key.pem, the below Delphi post-build event only copies both files if C:\Binaries\Cert.pem does not exist:
if not exist $(OUTPUTDIR)Cert.pem (copy bin\Cert.pem $(OUTPUTDIR))
if not exist $(OUTPUTDIR)Key.pem (copy bin\Key.pem $(OUTPUTDIR))
As soon as C:\Binaries\Cert.pem exists, the Key.pem file is never copied.
How can I solve this in the post-build event?
Edit: unlike my 2014 post, this is indeed possible using parentheses. See my answer below.
The problem with Delphi post-build events is that they are not batch files.
It means that statements that look like lines are being concatenated by the Delphi IDE into one big & ampersand separated statement. This ensures the commands are executed in sequence, as per Command Redirection, Pipes - Windows CMD - SS64.com:
commandA & commandB Run commandA and then run commandB
So this is the actual statement that gets executed:
if not exist $(OUTPUTDIR)Cert.pem (copy bin\Cert.pem $(OUTPUTDIR))&if not exist $(OUTPUTDIR)Key.pem (copy bin\Key.pem $(OUTPUTDIR))
The problem here is that now the second if is seen as a continuation of the "then" part of the first if statement: the second if never executes when the $(OUTPUTDIR)Cert.pem exists.
What helps is a little known feature that you can wrap each command inside parentheses. Normally this is to allow one command to span multiple lines (especially for if, and for..do loops), but it also works on one line.
Wrapping each line having an if statement inside parentheses ensures they become standalone statements not affecting the other lines, even if they are being concatenated with & ampersand separators.
In the dialog it looks like this:
(if not exist $(OUTPUTDIR)Cert.pem (copy bin\Cert.pem $(OUTPUTDIR)))
(if not exist $(OUTPUTDIR)Key.pem (copy bin\Key.pem $(OUTPUTDIR)))
That way, the IDE translates it into one statement:
(if not exist $(OUTPUTDIR)Cert.pem (copy bin\Cert.pem $(OUTPUTDIR)))&(if not exist $(OUTPUTDIR)Key.pem (copy bin\Key.pem $(OUTPUTDIR)))
Now it works as intended:
When $(OUTPUTDIR)Cert.pem exists but $(OUTPUTDIR)Key.pem does not, only $(OUTPUTDIR)Cert.pem is copied
When $(OUTPUTDIR)Cert.pem does exists but $(OUTPUTDIR)Key.pem does, only $(OUTPUTDIR)Key.pem is copied
when neither exist, both are copied
when both exist, neither are copied
I did not know this "trick" when writing my 2014 post Delphi prebuild/prelink/postbuild events, so I need to write an update for it.
Searching for batch file parentheses site:microsoft.com -site:social.technet.microsoft.com -site:answers.microsoft.com did not reveal it in the official documentation, but I am not surprised as it grew hysterically, instead of being designed. Or like the Old New Thing attributes h2g2:
Much like the universe, if anyone ever does fully come to understand Batch then the language will instantly be replaced by an infinitely weirder and more complex version of itself. This has obviously happened at least once before ;)
The best documentation I could find was at Parenthesis/Brackets - Windows CMD - SS64.com:
Parenthesis can be used to split commands across multiple lines. This can make code more readable. Variables will be evaluated for the code block just as if the command was a single line.
(command)
(
command
command )
Things that break inside parenthesis
The CMD shell does not use any great intelligence when evaluating parenthesis, so for example the command below will fail:
IF EXIST MyFile.txt (ECHO Some(more)Potatoes)
...
Use multiple build events instead of putting both commands in the same event.
Executing the source lines of a build event.
I can cut this short very easily: build events are not batch files.
What happens is that all lines in your build event are concatenated together using ampersand (&) signs which are used to execute multiple commands on one command line.
This means that all the fancy control structures (if statements, setlocal, for loops) are not possible inside build events.
ref: Pasted from a blog post: Delphi prebuild/prelink/postbuild events written by Jeroen W. Pluimers
Makes me wonder why you asked since it looks like you wrote the answer in 2014. :)
I'm working on automating our software's build / deploy process. One major piece of this is executing msbuild to compile multiple Delphi projects.
Following numerous resources such as this one, I can do it successfully from the RAD Studio Command Line (which is simply calling rsvars.bat to set some environment variables). However, when trying to automate both this batch file and the msbuild command from Delphi, I cannot figure out how to proceed.
The key of the issue is that the batch file and the actual msbuild command are two entirely separate commands - although they need to be run together under the same environment. I found this question somewhat related, but I don't see a clear answer for my scenario.
How can I execute msbuild from Delphi while first preparing it with the environment variables as found in rsvars.bat?
As this answer showed, you can run cmd.exe itself with command-line parameters to execute commands.
Per the cmd.exe documentation:
Syntax
cmd [/c|/k] [/s] [/q] [/d] [/a|/u] [/t:{<B><F>|<F>}] [/e:{on|off}] [/f:{on|off}] [/v:{on|off}] [<String>]
Parameters
/c
Carries out the command specified by String and then stops.
...
<String>
Specifies the command you want to carry out.
...
Remarks
Using multiple commands
To use multiple commands for <String>, separate them by the command separator && and enclose them in quotation marks. For example:
"<Command>&&<Command>&&<Command>"
Processing quotation marks
If you specify /c or /k, cmd processes the remainder of String, and quotation marks are preserved only if all of the following conditions are met:
You do not use /s.
You use exactly one set of quotation marks.
You do not use any special characters within the quotation marks (for example: & < > ( ) # ^ | ).
You use one or more white-space characters within the quotation marks.
The String within quotation marks is the name of an executable file.
If the previous conditions are not met, String is processed by examining the first character to verify whether it is an opening quotation mark. If the first character is an opening quotation mark, it is stripped along with the closing quotation mark. Any text following the closing quotation marks is preserved.
So try using CreateProcess() or ShellExecute/Ex() to run an instance of cmd.exe with these parameters:
cmd.exe /C ""<path>\rsvars.bat" && msbuild "<path>\project" <msbuild parameters> ..."
Alternatively, you can have your app load rsvars.bat and parse out the values it defines (or just define the values in your own code), and then execute msbuild using CreateProcess(), passing the desired environment variables to its lpEnvironment parameter. This way, you do not need to actually execute rsvars.bat at all.
Another way which works is to put both commands together into another new batch file. This also allows you to build multiple projects using the same environment. For example DoBuild.bat with the following contents:
call "C:\Program Files (x86)\Embarcadero\Studio\17.0\bin\rsvars.bat"
msbuild "<path>\MyProject.dproj"
msbuild "<path>\MyOtherProject.dproj"
msbuild "<path>\YetAnotherProject.dproj"
Then, just execute your batch file:
Cmd.exe /K "<path>\DoBuild.bat"
I'm trying to set a variable which will identify the make of a laptop.
I am doing this by using the command
wmic csproduct get vendor
This gives the following output
Vendor
LENOVO
(blank)
So based on that command, I have used the for command in the way below, to try and set a variable with the value 'LENOVO'
for /f "skip=1 tokens=1 delims=" %i in ('wmic csproduct get vendor')
do set vendor=%i
however, problem is that the output of the wmic command actually produces a blank line, under the word LENOVO, so my variable gets set as a blank value. Is there anyway to stop the for command from parsing this 3rd line, therefore stopping once the variable has been set with the value of 'lenovo'?
The skip function works fine, and bypasses the first line completely. However it doesn't seem to give me the option to say for example, skip lines 1 and 3 but leave 2. I have experimented with the EOL parameter to try and ignore the blank line, but the for command still reads the empty 3rd line each time.
Many Thanks
A little whacky, but try using the following within your line. It will filter out the Vendor line and then sort alphabetically, putting the blank line first. If you really need to do stuff like this regularly, check into a Windows port of some Unix utils like sed and awk.
wmic csproduct get vendor | find /v /i "vendor" | sort