I just need a hint. I am trying to run the following command from the GNU parallel tutorial (GNU Parallel tutorial):
parallel -S $SERVER1,$SERVER2 echo ::: running on more hosts
I replaced $SERVERX with known hosts in my network. If I execute the command I'm getting asked for my password for each server and after that nothing happens anymore. The curser blinks all day long and I do not get any error message.
I tried different servers with the same result.
The verbose mode shows:
ssh $SERVER1 -- exec perl -e #GNU_Parallel\\=split/_/,\\"use_IPC::Open3\\;_use_MIME::Base64\\"\\;eval\\"#GNU_Parallel\\"\\;\\$SIG\{CHLD\}\\=\\"IGNORE\\"\\;my\\$zip\\=\(grep\{-x\\$_\}\\"/usr/local/bin/bzip2\\"\)\[0\]\\|\\|\\"bzip2\\"\\;open3\(\\$in,\\$out,\\"\>\\&STDERR\\",\\$zip,\\"-dc\\"\)\\;if\(my\\$perlpid\\=fork\)\{close\\$in\\;\\$eval\\=join\\"\\",\\<\\$out\>\\;close\\$out\\;\}else\{close\\$out\\;print\\$in\(decode_base64\(join\\"\\",#ARGV\)\)\\;close\\$in\\;exit\\;\}wait\\;eval\\$eval\\;
and Followed by random characters
Something similar appears four times. I guess for the four jobs I started. I'd be very happy for help.
I think you are expected to set up passwordless ssh logins to all the remotes so GNU Parallel can get into them. – Mark Setchell
This was the right suggestion. Setting up key authentication using ssh-keygen and ssh-copy-id did the job! Thank you very much now it works. A short hint in the tutorial would have been great.
Related
I'm trying to get a Jenkins job to run sfdx force:data:soql:query commands in order to migrate configuration data sets between our production org and our sandboxes after a refresh. Certain configurations do not persist on a refresh so we need a way to move that data.
Running the queries from the command line on the Jenkins server work as expected, however the job when it runs fails with the following error:
'C:\Program' is not recognized as an internal or external command, operable program or batch file.
Build step 'Execute shell' marked build as failure
The job does three things:
Authorizes to the DevHub, lists out the connected orgs, and then performs a SQOL query to just print some data - 16 lines to be exact. Here are the commands in the shell script of the job:
sfdx force:auth:jwt:grant -i ${CONNECTED_APP_CONSUMER_KEY} -u ${DEV_HUB} -f ${JENKINS_HOME}/certs/prod/server.key -r [...] -a DevHub
sfdx force:org:list
sfdx force:data:soql:query -u ${DEV_HUB} -q "SELECT Id, Name FROM [...tablename...]" -r human
I am completely stumped on why this is happening. Again, running the SOQL command directly on the server through PowerShell or Command Line works as expected. I would appreciate any help with this.
This one stumped me for a long time but we finally got it figured out.
If you are seeing this error, make sure to check your machine's environmental variables. I saw a TON of other answers pointing to this as the issue where the install of SFDX path name had spaces in it as in C:|P:rogram Files\SFDX\bin but only showed some weird command line FOR loop that made no sense what so ever.
What we did was to completely uninstall all of SFDX making sure none of it was left on the machine and reinstalled into a folder we made where there was no spaces in the path name.
Once we did that, our job worked like it was supposed to. I hope this helps others who run into this same issue.
I am unable to run repo non-interactively inside a container as part of a freestyle job.
It prompts for the user-name and email. I got round that by doing a git config --global inside the job.
But then it does the color test, and that hangs indefinitely.
Looking at the source code for repo I see this
if os.isatty(0) and os.isatty(1) and not self.manifest.IsMirror:
if opt.config_name or self._ShouldConfigureUser():
self._ConfigureUser()
self._ConfigureColor()
So, I ran the following inside the container:
python -C "import os; print os.isatty(0), os.isatty(1)"
and, sure enough, it printed out True True
Looking at the Jenkins log, it launches the container with --tty specified, and there seems no way to configure that option.
I can't find a bash option to force a script to be run in a non-interactive shell. If I put the above python line in a file and execute it with almost any combination of commands and options, it still prints out True True
The only way I see something different is if I use I/O redirection
bash <a.sh
which prints out False True - i.e. stdin is not a tty, and
bash <a.sh >a.log
which prints False False.
For a complex script, are there any problems using the bash <script approach?
Does anyone know any jenkins magic to prevent docker being launched using --tty?
I know that the --tty is the culprit. I built the container locally and ran the following
$ docker run repotest python -c "import os;print os.isatty(0), os.isatty(1)"
False False
$ docker run --tty repotest python -c "import os;print os.isatty(0), os.isatty(1)"
True True
Running Versions:
repo: 1.12.37 (per Ubuntu 16.04 apt-get)
Jenkins: 2.149
Cloudbees Docker Plugin: 1.7.3
Container base is ubuntu:xenial
I'm using the "Build inside a docker container" option.
To run bash script repo_script.sh "non-interactively", or more exactly speaking without having terminals associated with standard streams, you could run your script simply as
repo_script.sh < /dev/null 2>&1 | cat
assuming you want to see the output the way you would see it running simply as repo_script.sh. By piping the standard output and error to a different process the file descriptor appears as a pipe and not TTY to repo_script.sh. You could also direct output to a file, or even to /dev/null if you do not care about the output:
log_file=/dev/null
repo_script.sh < /dev/null > "${log_file}" 2>&1
Running the script as
bash < repo_script.sh | cat
might would work too, though it is very unorthodox and to my mind hackish way of running a script just to break the association of TTY to the standard input. From script engine point of view, it is different to read a script program from a file than from standard input (which typically, if it is a terminal, is not seekable), so there might be some subtle differences that could possibly bite you in unexpected ways. This way does not as clearly communicate your intention to the next person that need to understand your code, and may lead to partial hair loss in that person due to extraneous head scratching.
There is no need for any bash options, just using the output directions from within the interpreting shell as above described is an easy-to-comprehend, multi-platform compatible standard convention for changing the standard stream associations.
P.S. I think it should be enough for your repo script to just test if the standard input is a TTY. It looks to me like the author of that script did not think deeply enough there. There is simply no use waiting for input if you do not have terminal device associated with standard input, and you could determine that everything needs to run without user interaction from there or stop with an error if that is not possible.
I have a job that needs to run a script on a remote computer. I'm doing so by using psexec via "Execute windows batch command":
C:\PsExec.exe \\computername -u username -p password -accepteula c:\xxx.exe
When I run the job I get the following error:
c:\PsExec.exe is not recognized as an internal or external command
** PsExec.exe is located under c:\
Any ideas?
First Define psexec.exe path in environment varaiable "PATH" or else place psexec.exe file in C:\Windows\System32\
And to Download Psexec.exe file
https://download.sysinternals.com/files/PSTools.zip
One possible explanation is the version of PsExec.exe: 32bits or 64bits.
If you have the 32 one on a 64bits machine, that command would not be recognized indeed. PsExec64.exe would.
I can see the age of this question and my answer may not be relevant to this topic since I was technically trying to solve a different problem, but maybe this will help other people who are stuck.
c:\PsExec.exe is not recognized as an internal or external command
I was trying to disable the Maintenance Configurator with PSExec (my problem is the never ending maintenance bug) and kept running into the same error as the OP BUT I got PSexec64 to run this command:
C:\PsExec64.exe -s schtasks /change /tn >"\Microsoft\Windows\TaskScheduler\Maintenance Configurator" /DISABLE
BY checking the "Run this program as an administrator" option under the Compatibility settings for "PsExec64.exe"
Don't know if this has solved my problem yet, but I think the OP would have been able to run his process if he had done this. Dear OP did you ever solve that?
Good day. I am attempting to create/run a script that will allow me to send an updated IOS from a server to my network devices. The following code works when I put in a manual IP address right before the ":flash" command.
#!/user/bin/expect
set IOSroot "/xxxxx/xxx/c3750e-universalk9-mz.150-2.SE10a.bin"
set pw xxxxxxxxxxxxxxxxxxx
spawn scp $IOSroot 1.1.1.1:flash:/c3750e-universalk9-mz.150-2.SE10a.bin
expect "TACACS Password:"
send "$pw\r"
interact
The code there works great and as expected. The issue arises when I try to use a file called "ioshost" with a list of IP's and use that within this script to get some automation. I have tried various things to get this to work. Some of them are as follows:
Settings Variables
IPHosts=$(cat ioshost)
set IPHost 'cat ioshost'
Along with trying to use the read/do command...
while read line; do
spawn scp $IOSroot $line:flash:/c3750e-universalk9-mz.150-2.SE10a.bin
done < ioshost
None of these seem to work and I am looking for guidance. Please note I understand that setting a password is not best practice but setting RSA keys as mentioned in other articles is not allowed so I am forced to do it this way.
Thank you for your time.
You can use one Expect script and one Bash script.
First update your Expect script a bit:
#!/user/bin/expect
set IOSroot "/xxxxx/xxx/c3750e-universalk9-mz.150-2.SE10a.bin"
set pw xxxxxxxxxxxxxxxxxxx
spawn scp $IOSroot [lindex $argv 0]:flash:/c3750e-universalk9-mz.150-2.SE10a.bin
# ^^^^^^^^^^^^^^^^
expect "TACACS Password:"
send "$pw\r"
interact
Then write a simple Bash for loop:
for host in $(<ioshost); do
expect /your/script.exp $host
done
I would like to inherit environment variables in GNU Parallel. I have several 'scripts' (really just lists of commands, designed for use with GNU Parallel) with hundreds of lines each that all call different external programs. However, these external program (out of my control) requires that several environment variables be set before they will even run.
Setting/exporting them locally doesn't seem to help, and I don't see any way to add this information to a profile.
The documentation doesn't seem to have anything this, and similar SO pages suggest wrapping the command in a script. However, this seems like an inelegant solution. Is there a way to export the current environment, or perhaps specify the required variables in a script?
Thanks!
This works for me:
FOO="My brother's 12\" records"
export FOO
parallel echo 'FOO is "$FOO" Process id $$ Argument' ::: 1 2 3
To make it work for remote connections (through ssh) you need to quote the variable for shell expansion. parallel --shellquote can help you do that:
parallel -S server export FOO=$(parallel --shellquote ::: "$FOO")\;echo 'FOO is "$FOO" Process id $$ Argument' ::: 1 2 3
If that does not solve your issue, please consider showing an example that does not work.
-- Edit --
Look at --env introduced in version 20121022
-- Edit --
Look at env_parallel introduced in 20160322.