I'm following this tutorial here here, I installed GNU parallel and tested it using the ping command as per the read me here.
Trying:
parallel –j8 ./ A_asymm_refine.sh {} ::: {1..8}
Gives:
[user#machine Documents]$ parallel –j8 ./ A_asymm_refine.sh {} ::: {1..8}
Academic tradition requires you to cite works you base your article on.
If you use programs that use GNU Parallel to process data for an article in a
scientific publication, please cite:
O. Tange (2011): GNU Parallel - The Command-Line Power Tool,
;login: The USENIX Magazine, February 2011:42-47.
This helps funding further development; AND IT WON'T COST YOU A CENT.
If you pay 10000 EUR you should feel free to use GNU Parallel without citing.
To silence this citation notice: run 'parallel --citation'.
/usr/bin/bash: $'\342\200\223j8': command not found
/usr/bin/bash: $'\342\200\223j8': command not found
/usr/bin/bash: $'\342\200\223j8': command not found
/usr/bin/bash: $'\342\200\223j8': command not found
/usr/bin/bash: $'\342\200\223j8': command not found
/usr/bin/bash: $'\342\200\223j8': command not found
/usr/bin/bash: $'\342\200\223j8': command not found
/usr/bin/bash: $'\342\200\223j8': command not found
How can I troubleshoot the problem?
Try this instead:
parallel -j8 ./A_asymm_refine.sh {} ::: {1..8}
Related
Windows 10, Ryzen 3700x, gcc 8.1.0 (Posix, SEH-enabled)
I am building clang, llvm, and compiler-rt (the PGO tools) from source. I have downloaded the clang+llvm source for 14.0.0, and built it successfully with the following:
cmake -G "MinGW Makefiles" -DLLVM_ENABLE_PROJECTS="clang;lld;compiler-rt" -DCMAKE_BUILD_TYPE=Release -DLLVM_TARGETS_TO_BUILD=X86 ../llvm
After this, I can invoke clang, and build projects, ranging from simple "Hello, World", to a much more complex one. I am able to make use of -flto, with the addition of -fuse-ld=lld.
However, if I attempt to do ANY sort of PGO building, I fail. For example, here is the minimal example to demonstrate the problem.
Andrew#Ryzen3700x MINGW64 ~/Desktop
$ cat test.c
#include <stdio.h>
int main() {
printf("Hello, World!");
return 0;
}
Andrew#Ryzen3700x MINGW64 ~/Desktop
$ clang -fprofile-instr-generate test.c
Andrew#Ryzen3700x MINGW64 ~/Desktop
$ ./a.exe
Hello, World!
Andrew#Ryzen3700x MINGW64 ~/Desktop
$ llvm-profdata merge -output=test.profdata default.profraw
warning: default.profraw: malformed instrumentation profile data
error: no profile can be merged
Andrew#Ryzen3700x MINGW64 ~/Desktop
$ llvm-profdata show default.profraw
error: default.profraw: malformed instrumentation profile data: function name is empty
I am aware of many answers to this question, and none of them seem to apply. My .profraw file is not empty.
I will note that when I installed LLVM/Clang directly (not building on my own), the PGO portions DID work. However, after many hours I could not resolve linking issues regarding -flto.
Below are two command line strings I used to output to assembly language listing using --x86-asm-syntax=intel. Both command line strings work, but they both produce AT&T syntax, not Intel syntax.
sudo clang-8 -S -mllvm --x86-asm-syntax=intel Svx.c
sudo llc-8 --x86-asm-syntax=intel Svx.ll -o Svx.s
at the top of each file it says:
.intel_syntax noprefix
But the code it produces is AT&T syntax.
I've researched and haven't found an answer.
Thanks for any ideas on why these command strings do not produce Intel syntax.
I have the script run_md.py which produces the file test.dcd from the input file named test.pdb.
I want to execute the same command on multiple input files (test*.pdb) on a remote server using GNU parallel and transfer the result back to the local computer. Therefore, I'm using the following command:
parallel --trc {.}.dcd -j 2 -S $SERVER1 './run_md.py {} 1000' ::: test*.pdb
The command is running as expected on the server using 2 slots. However, the files are not transferred back and I get the following error:
rsync: link_stat "/home/bougui/{.}.dcd" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1655) [Receiver=3.1.1]
It looks like the replacement string is not working. How can I make it works?
Below is the output of parallel --version:
GNU parallel 20130922
Copyright (C) 2007,2008,2009,2010,2011,2012,2013 Ole Tange and Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
GNU parallel comes with no warranty.
Web site: http://www.gnu.org/software/parallel
When using GNU Parallel for a publication please cite:
O. Tange (2011): GNU Parallel - The Command-Line Power Tool,
;login: The USENIX Magazine, February 2011:42-47.
What you are doing is 100% correct. So something on your system is breaking this. Please try this on another system and if possible follow REPORTING BUGS from man parallel.
The bug reported in that thread has been fixed and this feature works well with the latest version of GNU parallel (20160622). The GNU parallel version 20130922 packaged with Debian 8.5 is buggy for the usage of {.} string replacement, as described below:
With more test I found that the output file must be specified with a replacement string in the command run in parallel.
For testing purpose, you can find below a complete example that others can run:
echo This is input_file > input_file && parallel --trc {}.out -S $SERVER1 cat {} ">"{}.out ::: input_file
The example above works well. When I use the substitution string {.} as below:
echo This is input_file > input_file.in && parallel --trc {.}.out -S $SERVER1 cat {} ">"{.}.out ::: input_file
It works, as well. However, if I didn't specify {.}.out in the command run in parallel as below:
echo This is input_file > input_file.in && parallel --trc {.}.out -S $SERVER1 cat {} ">"input_file.out ::: input_file
... I reproduce the error:
rsync: link_stat "/home/bouvier/{.}.out" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1655) [Receiver=3.1.1]
rsync: [Receiver] write error: Broken pipe (32)
Therefore the output file must be specified in the command run in parallel.
When input is read from terminal, GNU Parallel always displays a warning:
parallel: Warning: Input is read from the terminal. Only experts do this on purpose. Press CTRL-D to exit.
But sometimes I do want to read from terminal (e.g., when I'm copy & pasting stuff from elsewhere entry by entry). Is it possible to turn off this warning? I couldn't find such an option in man parallel or man parallel_tutorial.
Note that I don't want a cheap solution like 2>/dev/null, since warning messages from other programs will be turned off, too. For instance, consider the following simple script:
#!/bin/bash
function print12 () {
echo "printing $1 to stdout"
echo "printing $1 to stderr" >/dev/stderr
}
export -f print12
SHELL=/bin/bash parallel -k print12 2>/dev/null
Messages printed to stderr will all be suppressed.
Just realized that I can do a cat or some read </dev/tty to achieve my desired effect. But let's just focus on the original question.
It cannot be turned off. But see it as a praise: Since you are doing it on purpose, you are an expert (at least in the eyes of GNU Parallel).
As it is just a warning, you are free to paste your arguments and have them run: The warning does not stop GNU Parallel from reading your input.
If you really do not like the warning:
cat | parallel ...
I just discovered GNU parallel and I'm having some trouble running a simple parallel task. I have a simulation running over multiple values and I'd like to split it up to run in parallel using command line args. From the docs , it seems you can run parallel mycommand :::: myargfile in which myargfile contains the various arguments you would like to feed your command, in parallel. However, I didn't see any information on how the args should be listed and assumed a myargfile like this would work:
--pmin 0 --pmax 0.1
--pmin 0.1 --pmax 0.2
...
mycommand --pmin 0 --pmax 0.1 executes no problem. But when I run parallel mycommand :::: myargfile I get error: unknown option pmin 0 --pmax 0.1 (caught and decoded courtesy boost program options). parallel echo :::: myargfile correctly prints out the arguments. It's as if they are being wrapped in a string which the program can't read and not fed like they are from a standard bash script.
What's going on? How can I make this work?
Following #DmitriChubarov's link to https://stackoverflow.com/a/6258206/1328439 , I discovered that I was lacking the colsep flag:
parallel --colsep ' ' mycommand :::: myargfile
successfully executes.
After digging through manual and help pages I came up with this example. Perhaps it will save someone out there. :)
#!/usr/bin/env bash
COMMANDS=(
"cnn -a mode=flat"
"cnn -a mode=xxx"
"cnn_x -a mode=extreme"
)
parallel --verbose --progress --colsep ' ' scrapy crawl {.} ::: "${COMMANDS[#]}"