I'm using PBSPro and am trying to use qsub command line to submit a job but can't seem to get the output and error files to be named how I want them. Currently using:
qsub -N ${subjobname_short} \
-o ${path}.o{$PBS_JOBID} -e ${path}.e${PBS_JOBID}
... submission_script.sc
Where $path=fulljobname (i.e. more than 15 characters)
I'm aware that $PBS_JOBID won't be set until after the job is submitted...
Any ideas?
Thanks
The solution I came up with was following the qsub command with a qalter command like so:
jobid=$(qsub -N ${subjobname_short} submission_script.sc)
qalter -o ${path}.o{$jobid} -e ${path}.e${jobid} ${jobid}
This way, PBS Pro does not need to resolve the variables, as it failed to do so in our install (this may be a configuration issue)
If you want the ${PBS_JOBID} to be resolved by PBSPro, you need to escape it on the command line:
qsub -o \$PBS_JOBID
Otherwise, bash will attempt to resolve $PBS_JOBID before it gets to the qsub command. I don't know if $subjobname_short and $path are actual environment variables or ones you want pbs to resolve, but if you want pbs to resolve them you'll also need to escape these ones or place it inside the job script.
NOTE: I also notice that your -o argument says {$PBS_JOBID} and I'm pretty sure you want ${PBS_JOBID}. I don't know if that's a typo in the question or what you tried to pass to qsub.
Related
I'm trying to get a Jenkins job to run sfdx force:data:soql:query commands in order to migrate configuration data sets between our production org and our sandboxes after a refresh. Certain configurations do not persist on a refresh so we need a way to move that data.
Running the queries from the command line on the Jenkins server work as expected, however the job when it runs fails with the following error:
'C:\Program' is not recognized as an internal or external command, operable program or batch file.
Build step 'Execute shell' marked build as failure
The job does three things:
Authorizes to the DevHub, lists out the connected orgs, and then performs a SQOL query to just print some data - 16 lines to be exact. Here are the commands in the shell script of the job:
sfdx force:auth:jwt:grant -i ${CONNECTED_APP_CONSUMER_KEY} -u ${DEV_HUB} -f ${JENKINS_HOME}/certs/prod/server.key -r [...] -a DevHub
sfdx force:org:list
sfdx force:data:soql:query -u ${DEV_HUB} -q "SELECT Id, Name FROM [...tablename...]" -r human
I am completely stumped on why this is happening. Again, running the SOQL command directly on the server through PowerShell or Command Line works as expected. I would appreciate any help with this.
This one stumped me for a long time but we finally got it figured out.
If you are seeing this error, make sure to check your machine's environmental variables. I saw a TON of other answers pointing to this as the issue where the install of SFDX path name had spaces in it as in C:|P:rogram Files\SFDX\bin but only showed some weird command line FOR loop that made no sense what so ever.
What we did was to completely uninstall all of SFDX making sure none of it was left on the machine and reinstalled into a folder we made where there was no spaces in the path name.
Once we did that, our job worked like it was supposed to. I hope this helps others who run into this same issue.
I have a variable in my jboss cli file, let's call it ${FOO}.
I can see that the variable is set in my Dockerfile, if I:
RUN echo ${FOO}
I see the value printed.
I have a foo.cli file which contains a line similar to:
/system-property=foo:add(value="${FOO}")
and when I run jboss-cli.sh foo.cli
I get:
Cannot resolve expression ${FOO}.
Is there a way to pass a variable from Docker to the file argument to jboss-cli.sh ?
I've tried removing the quotes around ${FOO} also in the system-property line but no luck.
This is due to the fact that the CLI does not treat values the same as names. In short, operation names, parameter names, header names and values, command names as well as command argument names are resolved automatically. Parameter and argument values aren’t. To fix this, we need to set resolve-parameter-values to true in jboss-cli.xml.
For e.g.:-
sed -i "s/<resolve-parameter-values>false<\/resolve-parameter-values>/\
<resolve-parameter-values>true<\/resolve-parameter-values>/" \
$JBOSS_HOME/bin/jboss-cli.xml
Note that the command above is meant to be a one-off solution. From the maintenance perspective, using sed for XML is not a good idea in general. If you’re automating this, consider using XPath instead.
Note:- This functionality requires WildFly 8.0.0.CR1 or newer.
you might need to set jboss-cli.xml to <resolve-parameter-values>true<\/resolve-parameter-values>
$ sed -i "s/<resolve-parameter-values>false<\/resolve-parameter-values>/\
<resolve-parameter-values>true<\/resolve-parameter-values>/" \
$JBOSS_HOME/bin/jboss-cli.xml
src: https://mirocupak.com/using-environment-variables-in-jboss-cli/
These are good hints how to solve the problem, but the sed command does not work this way on Mac. There is a lot of fun with "-i". More information here:
sed command with -i option failing on Mac, but works on Linux
On Mac this worked for me:
sed -i "" "s/false</resolve-parameter-values>/true</resolve-parameter-values>/" $JBOSS_HOME/bin/jboss-cli.xml
Unfortunately this way of using sed does not work on RHEL for example.
When using bash, if I have an environment variable called SOMEPATH which contains a path (e.g. SOMEPATH=/usr/local/lib), then if I type
ls $SOMEPATH/
and hit tab then bash expands the environment variable so my command line now says
ls /usr/local/lib/
If I do the same thing in zsh it instead escapes the $ sign so after tab completion the command line looks like
ls \$SOMEPATH
How can I get zsh to behave like bash does in this regard?
I've done a lot of googling but I haven't managed to find anything that exactly addresses this issue. I've also tried reading the zsh documentation but again I couldn't find anything that obviously relates to this.
Thanks for any pointers.
-D CMAKE_C_COMPILER is what I use to choose my compiler. However, if I have CMake options that are turned on/off like USEIPHONEFLAG, I need to do -DUSEIPHONEFLAG=1, -D USEIPHONEFLAG=1 does not work. I was wondering how the space after -D works in CMake.
CMake's command line parsing is not very consistent or robust unfortunately.
This issue is probably down to the order in which you pass the arguments.
Internally, CMake iterates over the command line arguments twice. The first time it's looking for non-cache arguments and skips any beginning with -D. Any that don't fit into the list of proper args are assumed to be the path to the CMakeLists.txt file (or the directory, or the CMakeCache.txt).
It's assuming that there will be only one path passed, and does nothing to validate that assumption. Herein lies the problem. If you've passed -D Foo=1, then -D is seen as a complete argument and is skipped, and Foo=1 is seen as a path.
On the second iteration through the args, it now grabs the values given via -D, but on this run it correctly handles spaces after the -D. So it understands that -D Foo=1 is setting Foo to 1.
So, the position of the path in your command line is all-important here.
cmake -D Foo=1 MyProject/CMakeLists.txt # --> Works
cmake MyProject/CMakeLists.txt -D Foo=1 # --> Fails
To complicate things further, you can wrap the space in quotes and have CMake parse it, but the variable name then includes the space and is unusable in the CMakeLists file.
cmake MyProject/CMakeLists.txt "-D Foo=1" # --> Defines the unusable var ${ Foo}
Another inconsistency is that other command line flags work with or without a space (e.g. -G), and others require a space (e.g. -E).
My own advice would be to always avoid adding a space after the flag unless it's required. I guess always passing the path last would help too, although that shouldn't be required if you don't add extra spaces.
This would be part of a reverse-engineering project.
To determine and document what a shell script (ksh, bash, sh) does, it is comfortable, if you have information about what other programs/scripts it calls.
How could one automate this task? Do you know any program or framework that can parse a shell script? This way for instance, I could recognize external command calls -- a step to the right direction.
For bash/sh/ksh, I think you can easily modify their source to log what has been executed. That would be a solution.
How about:
Get a list of distinct words in that script
Search $PATH to find a hit for each
?
bash -v script.sh ?
Bash's xtrace is your friend.
You can invoke it with:
set -x at the top of your script,
by calling your script with bash -x (or even bash --debugger -x),
or recursively by doing (set -x; export SHELLOPTS; your-script; )
If you can't actually run the script, try loading it into a text editor that supports syntax highlighting for Bash. It will color-code all of the text and should help indicate what is a reserved word, variable, external command, etc.