If I have a bash file with following content:
ls \
/tmp
Is there a way to grep this file and get
ls \
/tmp
instead of
ls \
?
Please notice that other behaviors of grep should remain same if input is a large file and has other match patterns.
Thanks!
Assuming you want to treat the final \ as a line continuation character, you can use awk to concatenate the lines:
awk '{ while( sub( "\\\\$", "" )) { getline n; $0 = $0 n; }} /ls/' input-file
This removes the line continuation character and combines everything into one line, then prints the line if if matches the regex ls.
You can use the context options, -A (lines after), -B (lines before), and -C (lines context, both before and after):
$ grep -A1 ls test.sh
ls \
/tmp
Related
by doing the following command in the folder
ls -d */ | cut -f1 -d'/'
I get entries like:
env1
env2
env3
env4
how I can use cat/grep or yq/jq or any other alternative command(s) instead of the above command?
for dir in */; do
echo "${dir%/}"
done
There are several options. You can use the tree command with the options:
# d: list only directories
# i: no print of indention line
# L: max display depth of the directory tree
tree -di -L 1 "$(pwd)"
Or you can also use the grep command to get the directories and the command awk:
# F: input field separator
# $9: print the ninth column of the output
ls -l | grep "^d" | awk -F" " '{print $9}'
Or you can use the sed command to remove the slash:
# structure: s|regexp|replacement|flags
# g: apply the replacement to all matches to the regexp, not just the first
ls -d */ | sed 's|[/]||g'
I found this solutions in this post.
File1 contains
hello
hello
I need to write a grep command to print the filename if this file contains more than one "hello". Otherwise, I need grep to exit on failure.
So far I have
grep -c "hello" File1 | grep -v :0
but it outputs
2. How do I get the desired output, which should either be filename File1 or no output at all (from what I understand, no match is a non zero exit code for grep)
with GNU grep for -z:
grep -lz 'hello.*hello' file
e.g.:
$ seq 15 | grep -lz '3.*3'
(standard input)
$ echo $?
0
$ seq 5 | grep -lz '3.*3'
$ echo $?
1
Like this:
#!/bin/bash
count=$(grep -c "hello" "$1")
if ((count > 1)); then
echo "$1"
else
exit 1
fi
Usage:
chmod +x script.sh
./script.sh File1
Explanations:
((...)) is an arithmetic command, which returns an exit status of 0 if the expression is nonzero, or 1 if the expression is zero. Also used as a synonym for "let", if side effects (assignments) are needed. See http://mywiki.wooledge.org/ArithmeticExpression
Using perl in a shell:
perl -0 -le '
my $filename = $ARGV[0];
print $filename if grep { /hello\nhello/ } <>
' file
I have a space-separated file that looks like this:
$ cat in_file
GCF_000046845.1_ASM4684v1_protein.faa WP_004920342.1 Chal_sti_synt_C
GCF_000046845.1_ASM4684v1_protein.faa WP_004927566.1 Chal_sti_synt_C
GCF_000046845.1_ASM4684v1_protein.faa WP_004919950.1 FAD_binding_3
GCF_000046845.1_ASM4684v1_protein.faa WP_004920342.1 FAD_binding_3
I am using the following shell script utilizing grep to search for strings:
$ cat search_script.sh
grep "GCF_000046845.1_ASM4684v1_protein.faa WP_004920342.1" Pfam_anntn_temp.txt
grep "GCF_000046845.1_ASM4684v1_protein.faa WP_004920342.1" Pfam_anntn_temp.txt
The problem is that I want each grep command to return only the first instance of the string it finds exclusive of the previous identical grep command's output.
I need an output which would look like this:
$ cat out_file
GCF_000046845.1_ASM4684v1_protein.faa WP_004920342.1 Chal_sti_synt_C
GCF_000046845.1_ASM4684v1_protein.faa WP_004920342.1 FAD_binding_3
in which line 1 is exclusively the output of the first grep command and line 2 is exclusively the output of the second grep command. How do I do it?
P.S. I am running this on a big file (>125,000 lines). So, search_script.sh is mostly composed of unique grep commands. It is the identical commands' execution that is messing up my downstream analysis.
I'm assuming you are generating search_script.sh automatically from the contents of in_file. If you can count how many times you'll repeat the same grep command you can just use grep once and use head, for example if you know you'll be using it 2 times:
grep "foo" bar.txt | head -2
Will output the first 2 occurrences of "foo" in bar.txt.
If you have to do the grep commands separately, for example if you have other code in between the grep commands, you can mix head and tail:
grep "foo" bar.txt | head -1 | tail -1
Some other commands...
grep "foo" bar.txt | head -2 | tail -1
head -n displays the first n lines of the input
tail -n displays the last n lines of the input
If you really MUST always use the same command, but ensure that the outputs always differ, the only way I can think of to achieve this is using temporary files and a complex sequence of commands:
cat foo.bar.txt.tmp 2>&1 | xargs -I xx echo "| grep -v \\'xx\\' " | tr '\n' ' ' | xargs -I xx sh -c "grep 'foo' bar.txt xx | head -1 | tee -a foo.bar.txt.tmp"
So to explain this command, given foo as a search string and bar.txt as the filename, then foo.bar.txt.tmp is a unique name for a temporary file. The temporary file will hold the strings that have already been output:
cat foo.bar.txt.tmp 2>&1 : outputs the contents of the temporary file. If none is present, will output an error message to stdout, (important because if the output was empty the rest of the command wouldn't work.)
xargs -I xx echo "| grep -v \\'xx\\' " adds | grep -v to the start of each line in the temporary file, grep -v something excludes lines that include something.
tr '\n' ' ' replaces newlines with spaces, to have on a single string a sequence of grep -vs.
xargs -I xx sh -c "grep 'foo' bar.txt xx | head -1 | tee -a foo.bar.txt.tmp" runs a new command, grep 'foo' bar.txt xx | head -1 | tee -a foo.bar.txt.tmp, replacing xx with the previous output. xx should be the sequence of grep -vs that exclude previous outputs.
head -1 makes sure only one line is output at a time
tee -a foo.bar.txt.tmp appends the new output to the temporary file.
Just be sure to clear the temporary files, rm *.tmp, at the end of your script.
If I am getting question right and you want to remove duplicates based on last field of each line then try following(this should be easy task for awk).
awk '!a[$NF]++' Input_file
So far, I have this command on my terminal and it doesn't do anything.
Essentially it's to look for any file that contains the word bango and move it to another directory.
grep -r ".*bango.*" /Users/user/Desktop/drums | xargs mv /Users/user/Desktop/bango
Grep has a function to list the filename only you should use that to list the name of the files.
Also xargs can build commands with positional arguments.
Try to use
grep -rlE ".*bango.*" /Users/user/Desktop/drums | xargs -I # mv # /Users/user/Desktop/bango
The option -E allows to use regular expressions.
However, a regular expression is not needed, you can activate a fast grep algorithm for fixed strings:
grep -rlF "bango" /Users/user/Desktop/drums | xargs -I # mv # /Users/user/Desktop/bango
I want to grep -R a directory but exclude symlinks how dow I do it?
Maybe something like grep -R --no-symlinks or something?
Thank you.
Gnu grep v2.11-8 and on if invoked with -r excludes symlinks not specified on the command line and includes them when invoked with -R.
If you already know the name(s) of the symlinks you want to exclude:
grep -r --exclude-dir=LINK1 --exclude-dir=LINK2 PATTERN .
If the name(s) of the symlinks vary, maybe exclude symlinks with a find command first, and then grep the files that this outputs:
find . -type f -a -exec grep -H PATTERN '{}' \;
The '-H' to grep adds the filename to the output (which is the default if grep is searching recursively, but is not here, where grep is being handed individual file names.)
I commonly want to modify grep to exclude source control directories. That is most efficiently done by the initial find command:
find . -name .git -prune -o -type f -a -exec grep -H PATTERN '{}' \;
For now.. here is how I would exclude symbolic links when using grep
If you want just file names matching your search:
for f in $(grep -Rl 'search' *); do if [ ! -h "$f" ]; then echo "$f"; fi; done;
Explaination:
grep -R # recursive
grep -l # file names only
if [ ! -h "file" ] # bash if not a symbolic link
If you want the matched content output, how about a double grep:
srch="whatever"; for f in $(grep -Rl "$srch" *); do if [ ! -h "$f" ]; then
echo -e "\n## $f";
grep -n "$srch" "$f";
fi; done;
Explaination:
echo -e # enable interpretation of backslash escapes
grep -n # adds line numbers to output
.. It's not perfect of course. But it could get the job done!
If you're using an older grep that does not have the -r behavior described in Aryeh Leib Taurog's answer, you can use a combination of find, xargs and grep:
find . -type f | xargs grep "text-to-search-for"
If you are using BSD grep (Mac) the following works similar to '-r' option of Gnu grep.
grep -OR <PATTERN> <PATH> 2> /dev/null
From man page
-O If -R is specified, follow symbolic links only if they were explicitly listed on the command line.