When I call a script in my gitlab-ci yml file, sometimes some characters within the call are skipped, such that the command cannot be executed anymore.
An example:
The call of
$ Scripts/build_all.bat Argument1 Argument2
results in:
Script/build_all.bat : The term 'Script/build_all.bat' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
As you can see, the shell omits the 's' in the path 'Scripts'.
I could post many other examples, where characters are omitted, it can happen at different locations in the call.
My environment:
Docker Windows with gitlab runner 13.10.3. The docker image is based on mcr.microsoft.com/windows/servercore:ltsc2019
Related
I am running a Dockerfile but every time it stops at one point.
RUN powershell %windir%\system32\inetsrv\appcmd.exe set config /section:system.webServer/handlers /+"[name='Test',path='Test.cgi',verb='*',modules="IsapiModule",scriptProcessor="c:\Test.dll",resourceType="Unspecified", preCondition="bitness64"]"
Failed to process input: The parameter 'verb=*' must begin with a / or -
I am struggling for hours. What could be the reason?
You're trying to invoke appcmd.exe via PowerShell (powershell.exe), even though the appcmd.exe command line doesn't require the features of that shell - it seems to be composed of literal strings only.
Your use of %windir% implies that your Dockerfile uses the default shell on Windows, namely cmd.exe
Therefore, you should be able to formulate your appcmd.exe command line as you would submit it from a cmd.exe session:
RUN %windir%\system32\inetsrv\appcmd.exe set config /section:system.webServer/handlers /+[name='Test',path='Test.cgi',verb='*',modules='IsapiModule',scriptProcessor='c:\Test.dll',resourceType='Unspecified',preCondition='bitness64']
Note:
All quoting in the remainder of the argument that starts with /+ now uses only '...', for consistency; since no argument-internal " chars. are therefore in play, the need to escape them goes away.
No spaces are allowed in the remainder of the argument.
As for what you tried:
When you use the -Command / -c PowerShell CLI parameter (which is implied in the absence of -File), " characters to be passed through as part of the PowerShell command must be escaped as \".
See this answer for an explanation.
Since your /+ argument also contains embedded " chars., as part of the argument, you would have to escape them twice, namely as `\" (sic)
Jenkins Cosole output :updated 2
When executing the Jmeter .JMX file using JMeter non GUI mode, it is working fine and I am able to get the .JTL file, but when I am trying to trigger the build using Jenkins it is getting failed giving Error: Unable to access jarfile ApacheJMeter.jar
errorlevel=1 as message from console output .
Please note that I have added Perfomance Plugin in Jenkins and made jmeter.save.saveservice.output_format=xml enabled in JMeter user properties. Please help if I am missing anything to setup the configuration.
Build cmd command
C:\jmeter\bin\jmeter -J jmeter.save.saveservice.output_format=xml -n -t C:\jmeter\bin\edueka.jmx -l C:\jmeter\bin\report3.jtl
code from CMD - JMeter non GUI
JMeter Jar file access from CMD
There is nothing wrong with your command, it should work normally.
Double check that the main JMeter executable .jar exists and the user which executes Jenkins process has correct permissions to access this file:
c:\jmeter\bin\ApacheJMeter.jar
The most common mistake is that users download source bundle of JMeter instead of the binary
Also the correct syntax for passing JMeter Properties via -J command line argument is:
J needs to be capital
remove the space between J and the property name
C:\jmeter\bin\jmeter -Jjmeter.save.saveservice.output_format=xml -n -t C:\jmeter\bin\edueka.jmx -l C:\jmeter\bin\report3.jtl
Also currently there is no need to switch JMeter results format to XML, Jenkins Performance Plugin is capable of processing JMeter results files in .CSV format as well
I'm trying to create a script that starts a docker container and mounts a local folder that contains spaces in the name. I can get it to work fine when I run a *.bat file with the docker run command:
docker run -p 8081:8081 -v "C:\Test Folder With Blanks":/zzz myimage jupyter lab --notebook-dir=/zzz--ip=0.0.0.0 --port=8081 --allow-root
But when I try to do the same in a Powershell script file, I get an error:
$CMD = 'docker run -p 8081:8081 -v "C:\Test Folder With Blanks":/zzz myimage jupyter lab --notebook-dir=/zzz--ip=0.0.0.0 --port=8081 --allow-root'
Invoke-Expression $CMD
docker: invalid reference format.
See 'docker run --help'.
I'm on Win10 and running Powershell in Visual Studio Code IDE.
Thanks for ideas.
First, the obligatory warning: Unless you have a command line stored as a single string somewhere and you either fully control or trust the content of the string, Invoke-Expression should generally be avoided.
You're seeing an inconsistency in how PowerShell treats compound tokens composed of directly concatenated quoted and unquoted parts.
Specifically, argument "C:\Test Folder With Blanks":/zzz is unexpectedly broken in two, i.e passed as two arguments.
The workaround is to quote the entire argument, i.e.
"C:\Test Folder With Blanks:/zzz"
Note: I'm assuming that docker doesn't actually require partial quoting in its arguments, which it shouldn't; however, there are high-profile CLIs on Windows that do, notably msiexec.
Alternatively, use an expression enclosed in (...) to compose your string; e.g.
("C:\Test Folder With Blanks" + ':/zzz')
There's no good reason to do so in this case, but it could be helpful if you need string interpolation in one part of your string ("..."), but not in another ('...').
General caveats:
Compared to cmd.exe and also POSIX-compatible shells such as bash, PowerShell has several additional metacharacters, notably # (at the start of a token), { / }, and ;. Therefore, you cannot always expect command lines written for these shells to work as-is in PowerShell.
As of PowerShell 7.2.2, passing arguments to external programs (such as docker) is fundamentally broken with respect to arguments that have embedded " characters and empty-string arguments - see this answer.
The general pattern of the inconsistency, as of PowerShell 7.2.2, is as follows:
If an argument:
starts with a quoted token - whether single- ('...') or double-quoted ("...") -
and has additional characters directly following the closing quote,
the first such character starts a separate argument.
E.g. "foo":bar / "foo"=bar / "foo"'bar' are passed as separate arguments foo and :bar / foo and =bar / foo and bar, respectively.
In other words:
You cannot compose a single string argument from a mix of quoted and unquoted / differently quoted tokens if the first token is quoted.
Conversely, it does work if the first token is unquoted, including an unquoted simple variable reference such as $HOME.
# OK: First token is unquoted.
PS> cmd /c echo foo"bar"'baz'last
foobarbazlast
# !! BROKEN: First token is quoted.
# !! Because each successive token is quoted too,
# !! each becomes its own argument.
PS> cmd /c echo 'foo'"bar"'baz'last
foo bar baz last
GitHub issue #6467 discusses this inconsistency; however, it has been closed, because the behavior is - surprisingly - considered by design.
This does not happen if the first token is unquoted; however, there are related bugs that start with unquoted tokens that similarly break arguments in two, related to their looking like named arguments to PowerShell (which is a PowerShell concept that doesn't apply when calling external programs):
GitHub issue #11646: an argument such as -foo=1,2 breaks parsing.
GitHub issue #6291: an argument such as -foo=bar.baz is broken in two at the (first) .
I'm attempting to build, launch, and link a set of docker containers using Rundeck. In short (for those not familiar with docker), when an image is launched, it returns a container ID. I would like to use this container ID in the launching of subsequent jobs.
When run from the command line, it would look something like this (example only!!):
# docker run -Pd 23ABCD45
34DEF123
# docker run -Pd --link 34DEF123:host1 ABC123EF
321CB456
(note the use of the first return value in the second command line)
At this point, there would be two containers running. The second would be linked to the first by the --link option, and it would be addressable using the hostname host1 from inside the second container. To be fair, docker generates (or may be given) a specific container name which can be used in place of the container id. I would prefer to use the container ID to avoid the hassle of having to create/track unique names.
I would like to be able to capture the output of the first command (the container ID) so that it can be reused in the second command. Is this possible?
Edit: These images are being used for testing immediately following a
"docker build" (which also outputs a similar ID I would like to
include in my chain) and might be followed by "docker rm" and "docker
rmi" commands, so there are a number of uses for capturing this type
of output and carrying it through a related set of operations. This
is not just about launching/linking containers.
There is no direct Rundeck implementation that allows you pass an output from one job to another job as an input, but there are work around I've tried in the past, and I've settled on the second approach.
1. Use a file to pass data
Save the ID/output into a tmp file in first job
Second job read that file
Things might go wrong since you depend on a file, but good code can improve.
2. Call two jobs using Rundeck CLI from another job
This is the approach I am using.
JobA printout two random numbers.
echo $RANDOM;echo $RANDOM
JobB print out the second random produced from JobA which is passed as an option "number"
echo "$RD_OPTION_NUMBER is the number JobB received"
JobC calls first job, save last line to a variable and pass it to JobB
#!/bin/bash
OUTPUT_FROM_JOB_A=`run -f --id <ID of JobA> | tail -n 1`
run -f --id <ID of JobB> -- -number $OUTPUT_FROM_JOB_A
Output:
[5394] execution status: succeeded
Job execution started:
[5395] JobB <https://hostname:4443/project/Project/execution/show/5395>
6186 is the number JobB received
[5395] execution status: succeeded
This is just primitive code sample. you can do alot with python subprocess or just use bash.
Under the link http://docs.buildbot.net/latest/manual/cfg-buildsteps.html#shellcommand,
in the shell command it states that
On Windows, commands are run via cmd.exe /c which works well. However, if you're running a batch file, the error level does not get propagated correctly unless you add 'call' before your batch file's name: cmd=['call', 'myfile.bat', ...].
It says that the commands are executed cmd.exe.
I assume the cmd.exe is located in the c:\windows\system32\cmd.exe, am i right or it calls from some other location like in the windows startup .
The cmd.exe is the one found in the system path variable PATH for the Jenkins process. On my machine it is in C:\windows\system32\cmd.exe.