expect, assign output to buffer - buffer

#!/usr/local/bin/expect --
set env(TERM) vt100
set env(SHELL) /bin/sh
set env(HOME) /usr/local/bin
set PASSWORD eri
set DUL [lindex $argv 0]
match_max 100000
spawn ssh mashost
expect {
"assword" {send "$PASSWORD\r"}
}
expect "ranosusr#rn2osscs603"
send -- "cd /var/opt/bla/edd/ARNE_SIU \r"
expect "ranosusr#rn2osscs603"
send -- "grep -il $DUL *\r"
expect -re "$DUL.*\.xml"
set outcome $expect_out(0,string)
expect "ranosusr#rn2osscs603"
send -- "/opt/bla/arne/bin/import.sh -f $outcome -val:rall\r"
expect "ranosusr#rn2osscs603"
interact
and when I run it
ssh mahost
Password:
ranosusr#rn2osscs603> cd /var/opt/bla/edd/ARNE_SIU
ranosusr#rn2osscs603> grep -il FXLP89 *
FXLP89_FRTALZ_SIU_ARNE.xml
ranosusr#rn2osscs603> /opt/bla/arne/bin/import.sh -f FXLP89 *
Logging to file /var/opt/bla/arne/ARNE_Import_Log.2014-03-11_10:37:47
Failed to write to file. Writing to stdout instead.
The file 'FXLP89' can not be found.
ranosusr#rn2osscs603>
ranosusr#rn2osscs603>
ranosusr#rn2osscs603> FXLP89_FRTALZ_SIU_ARNE.xml -val:rall
FXLP89_FRTALZ_SIU_ARNE.xml: Command not found.
ranosusr#rn2osscs603>
and it shold be:
ranosusr#rn2osscs603> cd /var/opt/bla/edd/ARNE_SIU
ranosusr#rn2osscs603> grep -il FXLP89 *
FXLP89_FRTALZ_SIU_ARNE.xml
ranosusr#rn2osscs603>/opt/bla/arne/bin/import.sh -f FXLP89_FRTALZ_SIU_ARNE.xml -val:rall
so basicly, output of grep (name of file) I need to put in the line under as showed here, but I don't know how. I tried some suggestion from guys here, but I can't manage it still. Thanks for any help.

After you
send -- "grep -il $DUL *\r"
Expect sees all this output
ranosusr#rn2osscs603> grep -il FXLP89 *
FXLP89_FRTALZ_SIU_ARNE.xml
ranosusr#rn2osscs603>
The part that is matched by the regular expression "$DUL.*.xml" is in bold:
ranosusr#rn2osscs603> grep -il FXLP89 *
FXLP89_FRTALZ_SIU_ARNE.xml
ranosusr#rn2osscs603>
You should expect this:
set prompt {ranosusr#rn2osscs603> }
expect -re "($DUL\\S+\\.xml).*$prompt$"
set filename $expect_out(1,string)
Here, we're expecting the pattern: the value of $DUL followed by some non-whitespace chars followed by a literal dot and "xml" (capture all of that) followed by some chars and the prompt and the end of string.
The backslashes are doubled because we're using a double quoted string, and the backslashes have to be passed literally to the regex engine.

Related

Does Mercurial have a template to capture output of "hg grep"?

I was searching for a change that included "foreach" so I used this Mercurial command:
$ hg grep -r "user(mjh) & public() & date(-30)" --diff -i foreach
and it does return the hits where "foreach" was added and removed.
However, I'd like to know the actual commit hashes too. If I add a template:
$ hg grep ... -T '{date|shortdate}\n{node|short}\n{desc|firstline}\n\n'
then I get the commit hash and description as expected, but then I don't see the changed files listed.
Is there a template to capture the output of hg grep? The {files} template lists the files associated with a commit, but that's not the actual grep output. Is there an iterable template keyword available for the grep results?
Please, re-read carefully hg help grep -v (-v is important option), note the following part (new and unexpected for me also)
The following keywords are supported in addition to the common
template
keywords and functions. See also 'hg help templates'.
change String. Character denoting insertion "+" or removal "-".
Available if "--diff" is specified.
lineno Integer. Line number of the match.
path String. Repository-absolute path of the file.
texts List of text chunks.
After it you'll be able to repeat (so-so, because some details will differ slightly) default output of grep in you template
>hg grep --diff -i -r 1166 to_try
>hg grep --diff -i -r 1166 -T "{path}:{rev}:{change}:{texts}\n" to_try
hggit/compat.py:1166:-: for args in parameters_to_try:
hggit/compat.py:1166:+: for (args, kwargs) in parameters_to_try:
and after replacing {rev} by {node|short}
>hg grep --diff -i -r 1166 -T "{path}:{node|short}:{change}:{texts}\n" to_try
hggit/compat.py:f6cef55e6aeb:-: for args in parameters_to_try:
hggit/compat.py:f6cef55e6aeb:+: for (args, kwargs) in parameters_to_try:

grep exact match of string with alphabets and numbers

I am using grep to extract lines from file 1 that matches with string in file2. The string in file 2 has both alphabets and numbers. eg;
MSTRG.18691.1
MSTRG.18801.1
I used sed to write word boundaries for all the strings in the file 2.
file 2
\<MSTRG.18691.1\>
\<MSTRG.18801.1\>
and used grep -f file2 file1
but output has
MSTRG.18691.1.2
MSTRG.18801.1.3 also..
I want lines that matches exactly,
MSTRG.18691.1
MSTRG.18801.1
and not,
MSTRG.18691.1.2
MSTRG.18801.1.3
Few lines from my file1
t_name gene_name FPKM TPM
MSTRG.25.1 . 0 0
rna71519 . 93.398872 194.727926057583
gene34024 ND1 2971.72876 6195.77694943117
MSTRG.28.1 . 0 0
MSTRG.28.2 . 0 0
rna71520 . 33.235409 69.2927240732149
Updating the answer
You can use start with ^ and end with $ operator to match start with and begin with. To match exactly MSTRG.18691.1 you can add ^ & $ at both ends and remove the word boundaries, additionally . has special meaning in regex to match exactly . we need to escape that with a backslash \
Example pattern:
^MSTRG\.18691\.1$
^MSTRG\.18801\.1$
file1
MSTRG.18691.1
MSTRG.1311.1
MSTRG.18801.2
MSTRG.18801.3
MSTRG.18801.1.2
MSTRG.18801.1.1
MSTRG.18801.1
PrefixMSTRG.18801.1
Just create a normal file named file1 and paste the above content into it.
file2 (pattern file)
^MSTRG\.18801\.1$
Just create a normal file named file2 and paste the above content into it.
Run the below command from commandline
grep -i --color -f file2 file1
Result:
MSTRG.18801.1
Sed to add changes to the pattern file
Here is the sed command to escape . and add ^ and $ at the beginning and end of the pattern file you already have.
sed -Ee 's/\./\\./g' -e 's/^/\^/g' -e 's/$/\$/g' file2 > file2_updated
-E to support extended regex on BSD sed, you may need to replace -E with -r based on your system's sed
Updated patterns will be saved to file2_updated. Need to use the new pattern file in grep like this
grep -i -f file2_updated file1
The flag you're looking for is -F. From man grep:
-F, --fixed-strings
Interpret PATTERN as a list of fixed strings (instead of regular expressions), separated by newlines, any of which is to be matched.
You can use this quite comfortably in conjunction with -f:
grep -Ff file2 file1
To be clear, this will treat every line of file2 as an exact match against file1.

How to grep a text in a file with new/breaks line

I have to parse the content of multiple files with this content:
style=3D""><a href=3D"https://123456789.com/accounts/confirm_email/19AbCDx=
K/bWFyY29A1234529zYW50dWNjaS5ldQ/?app_redirect=3DFalse&ndid=3DHMTU1Mjk=
wODY5OTA1MDk2NTptYXJjb0BtYXJjb3NhbnR1Y2NpLmV1Ojg1OQ" style=3D"color:#3b599
I have to extract the https link, but my grep command can't ignore the new line return, and end with a trunk result:
COMMAND
grep -r -m1 -oh "https://123456789.com/accounts/confirm_email*\s*[^ ]*" /folder/
RESULT
https://123456789.com/accounts/confirm_email/19AbCDx=
DESIDERED RESULT
https://123456789.com/accounts/confirm_email/19AbCDx=K/bWFyY29A1234529zYW50dWNjaS5ldQ/?app_redirect=3DFalse&ndid=3DHMTU1MjkwODY5OTA1MDk2NTptYXJjb0BtYXJjb3NhbnR1Y2NpLmV1Ojg1OQ
PS: '=' character is not (always) part of link, but it is the format of the file when break the line.
NB: https://123456789.com/accounts/confirm_email/ is the only constant of the link repeated in all files.
IF I add -z option, -m1 option is ignored and the result is:
https://123456789.com/accounts/confirm_email/19AbCDx=
K/bWFyY29A1234529zYW50dWNjaS5ldQ/?app_redirect=3DFalse&ndid=3DHMTU1Mjk=
wODY5OTA1MDk2NTptYXJjb0BtYXJjb3NhbnR1Y2NpLmV1Ojg1OQ"https://123456789.com/accounts/confirm_email/19AbCDx=
K/bWFyY29A1234529zYW50dWNjaS5ldQ/?app_redirect=3DFalse&ndid=3DHMTU1Mjk=
wODY5OTA1MDk2NTptYXJjb0BtYXJjb3NhbnR1Y2NpLmV1Ojg1OQ"https://123456789.com/accounts/confirm_email/19AbCDx=
K/bWFyY29A1234529zYW50dWNjaS5ldQ/?app_redirect=3DFalse&ndid=3DHMTU1Mjk=
wODY5OTA1MDk2NTptYXJjb0BtYXJjb3NhbnR1Y2NpLmV1Ojg1OQ"
IF I add |head -3 after the command seem to work BUT http is repeated in the last line
COMMAND
grep -r -oh -z "https://123456789.com/accounts/confirm_email*\s*[^ ]*" /folder/ |head-3
https://123456789.com/accounts/confirm_email/19AbCDx=
K/bWFyY29A1234529zYW50dWNjaS5ldQ/?app_redirect=3DFalse&ndid=3DHMTU1Mjk=
wODY5OTA1MDk2NTptYXJjb0BtYXJjb3NhbnR1Y2NpLmV1Ojg1OQ"https://123456789.com/accounts/confirm_email/19AbCDx=
How can I exclude it?
man grep:
-z, --null-data
Treat the input as a set of lines, each terminated by a zero
byte (the ASCII NUL character) instead of a newline. - -
So:
$ grep -z -r -m1 -oh "https://123456789.com/accounts/confirm_email*\s*[^ ]*" file
Output:
https://123456789.com/accounts/confirm_email/19AbCDx=
K/bWFyY29A1234529zYW50dWNjaS5ldQ/?app_redirect=3DFalse&ndid=3DHMTU1Mjk=
wODY5OTA1MDk2NTptYXJjb0BtYXJjb3NhbnR1Y2NpLmV1Ojg1OQ"
The newlines will still be there but you could delete them with tr -d \\n

How do I 'grep -c' and avoid printing files with zero '0' count

The command 'grep -c blah *' lists all the files, like below.
% grep -c jill *
file1:1
file2:0
file3:0
file4:0
file5:0
file6:1
%
What I want is:
% grep -c jill * | grep -v ':0'
file1:1
file6:1
%
Instead of piping and grep'ing the output like above, is there a flag to suppress listing files with 0 counts?
SJ
How to grep nonzero counts:
grep -rIcH 'string' . | grep -v ':0$'
-r Recurse subdirectories.
-I Ignore binary files (thanks #tongpu, warlock).
-c Show count of matches. Annoyingly, includes 0-count files.
-H Show file name, even if only one file (thanks #CraigEstey).
'string' your string goes here.
. Start from the current directory.
| grep -v ':0$' Remove 0-count files. (thanks #LaurentiuRoescu)
(I realize the OP was excluding the pipe trick, but this is what works for me.)
Just use awk. e.g. with GNU awk for ENDFILE:
awk '/jill/{c++} ENDFILE{if (c) print FILENAME":"c; c=0}' *

Strange behaviour of grep command

When I do the following grep I get results I cannot explain to myself:
host:/usr/local/tomcat > grep '-XX:PermSize=256m' *
RELEASE-NOTES: http://www.apache.org/licenses/LICENSE-2.0
RUNNING.txt: http://www.apache.org/licenses/LICENSE-2.0
Afaik, none of the characters in my regular expression have a special meaning (inside square brackets, - has one, but there are none). I also put it into single quotes so it shouldn’t be modified by the shell.
Grep version: grep (GNU grep) 2.5.1
Tomcat version: 6.0.36 (binary
distro)
Since your pattern begins with a minus sign -, grep interprets it as an argument.
You could say:
grep -- '-XX:PermSize=256m' *
-- would tell grep to stop processing command line arguments.
Alternatively, you could say:
grep -- '[-]XX:PermSize=256m' *
(When you say [-] the hyphen is interpreted as a literal. Since you say (inside square brackets, - has one.., it seemed that it should be clarified.)
The shell expands the command to the following:
grep -XX:PermSize=256m LICENSE NOTICE RELEASE-NOTES RUNNING.txt bin conf lib …
Grep accepts a secret parameter -Xoptarg without reporting an invalid option. In the source code it reads
/* -X is undocumented on purpose. */
However, the next token, LICENSE is taken as regular expression. Typing grep -X * takes the LICENSE token as input string to the parameter -X and greps for NOTICE in RELEASE-NOTES and RUNNING.txt.
The meaning of the secret grep capital X parameter is to set the internal mode, i.e.
grep -X grep → behave normally (grep -G)
grep -X egrep → behave like grep -E
grep -X awk → uses RE_SYNTAX_AWK
grep -X fgrep → behave like grep -F
grep -X perl → behave like grep -P
grep -X 'unrecognized string' → behave normally
My grep complains about
grep: invalid matcher X:PermSize=256m
but what you can see here is that grep considers -X... as an option. To make it stop interpreting options, use --, i.e.
grep -- -XX:PermSize=256m *
The single-quotes are not necessary.

Resources