I am running this command "ls -lah -lF | grep /$ | awk '{print $2,$5,$9}' >> test.txt"
when it runs and sends the results to the text file the output is all in one row. How do I get it to formate each directory in a row?
It isn't in one line. Just checked. Check, how your editor handles Unix line endings.
Related
by doing the following command in the folder
ls -d */ | cut -f1 -d'/'
I get entries like:
env1
env2
env3
env4
how I can use cat/grep or yq/jq or any other alternative command(s) instead of the above command?
for dir in */; do
echo "${dir%/}"
done
There are several options. You can use the tree command with the options:
# d: list only directories
# i: no print of indention line
# L: max display depth of the directory tree
tree -di -L 1 "$(pwd)"
Or you can also use the grep command to get the directories and the command awk:
# F: input field separator
# $9: print the ninth column of the output
ls -l | grep "^d" | awk -F" " '{print $9}'
Or you can use the sed command to remove the slash:
# structure: s|regexp|replacement|flags
# g: apply the replacement to all matches to the regexp, not just the first
ls -d */ | sed 's|[/]||g'
I found this solutions in this post.
I have a space-separated file that looks like this:
$ cat in_file
GCF_000046845.1_ASM4684v1_protein.faa WP_004920342.1 Chal_sti_synt_C
GCF_000046845.1_ASM4684v1_protein.faa WP_004927566.1 Chal_sti_synt_C
GCF_000046845.1_ASM4684v1_protein.faa WP_004919950.1 FAD_binding_3
GCF_000046845.1_ASM4684v1_protein.faa WP_004920342.1 FAD_binding_3
I am using the following shell script utilizing grep to search for strings:
$ cat search_script.sh
grep "GCF_000046845.1_ASM4684v1_protein.faa WP_004920342.1" Pfam_anntn_temp.txt
grep "GCF_000046845.1_ASM4684v1_protein.faa WP_004920342.1" Pfam_anntn_temp.txt
The problem is that I want each grep command to return only the first instance of the string it finds exclusive of the previous identical grep command's output.
I need an output which would look like this:
$ cat out_file
GCF_000046845.1_ASM4684v1_protein.faa WP_004920342.1 Chal_sti_synt_C
GCF_000046845.1_ASM4684v1_protein.faa WP_004920342.1 FAD_binding_3
in which line 1 is exclusively the output of the first grep command and line 2 is exclusively the output of the second grep command. How do I do it?
P.S. I am running this on a big file (>125,000 lines). So, search_script.sh is mostly composed of unique grep commands. It is the identical commands' execution that is messing up my downstream analysis.
I'm assuming you are generating search_script.sh automatically from the contents of in_file. If you can count how many times you'll repeat the same grep command you can just use grep once and use head, for example if you know you'll be using it 2 times:
grep "foo" bar.txt | head -2
Will output the first 2 occurrences of "foo" in bar.txt.
If you have to do the grep commands separately, for example if you have other code in between the grep commands, you can mix head and tail:
grep "foo" bar.txt | head -1 | tail -1
Some other commands...
grep "foo" bar.txt | head -2 | tail -1
head -n displays the first n lines of the input
tail -n displays the last n lines of the input
If you really MUST always use the same command, but ensure that the outputs always differ, the only way I can think of to achieve this is using temporary files and a complex sequence of commands:
cat foo.bar.txt.tmp 2>&1 | xargs -I xx echo "| grep -v \\'xx\\' " | tr '\n' ' ' | xargs -I xx sh -c "grep 'foo' bar.txt xx | head -1 | tee -a foo.bar.txt.tmp"
So to explain this command, given foo as a search string and bar.txt as the filename, then foo.bar.txt.tmp is a unique name for a temporary file. The temporary file will hold the strings that have already been output:
cat foo.bar.txt.tmp 2>&1 : outputs the contents of the temporary file. If none is present, will output an error message to stdout, (important because if the output was empty the rest of the command wouldn't work.)
xargs -I xx echo "| grep -v \\'xx\\' " adds | grep -v to the start of each line in the temporary file, grep -v something excludes lines that include something.
tr '\n' ' ' replaces newlines with spaces, to have on a single string a sequence of grep -vs.
xargs -I xx sh -c "grep 'foo' bar.txt xx | head -1 | tee -a foo.bar.txt.tmp" runs a new command, grep 'foo' bar.txt xx | head -1 | tee -a foo.bar.txt.tmp, replacing xx with the previous output. xx should be the sequence of grep -vs that exclude previous outputs.
head -1 makes sure only one line is output at a time
tee -a foo.bar.txt.tmp appends the new output to the temporary file.
Just be sure to clear the temporary files, rm *.tmp, at the end of your script.
If I am getting question right and you want to remove duplicates based on last field of each line then try following(this should be easy task for awk).
awk '!a[$NF]++' Input_file
I want to print the filename if only ALL the matches are present... on different lines
grep -l -w '10B\|01A\|gencode' */$a*filename.vcf
this prints out the filename, but not only if ALL three matches are present.
Would you consider to try awk? awk may solve it in following method,
awk '/10B/&&/01A/&&/gencode/{print FILENAME}' */$a*filename.vcf
try following, just edited your solution a bit.
grep -l '10B.*01A.*gencode' Input_file
With grep and its -P (Perl-Compatibility) option and positive lookahead regex (?=(regex)), to match patterns if in any order.
grep -lwP '(?=.*?10B)(?=.*?01A)(?=.*?gencode)' /path/to/infile
grep -l 'pattern1' files ... | xargs grep -l 'pattern2' | xargs grep -l 'pattern3'
From the grep manual:
-l, --files-with-matches
Suppress normal output; instead print the name of each input file from which output would normally have been printed. The scanning will stop on the first match. (-l is specified by POSIX.)
I often have to look for specific strings in a big set of log files with grep. And I get lots of results, on what I must scroll a lot.
Today, the results of grep list the results in alphabetical order. I would like to have my grep results reversed ordered by time, like a ls -ltr would do.
I know I could take the result of ls -ltr and grep file by file. I do it like this:
ls -ltr ${my_log_dir}\* | awk '{print $9}' |xargs grep ${my_pattern}
But I wonder: Is there a simpler way?
PS: I'm using ksh on AIX.
The solution I found (thanks to Fedorqui) was to simply use ls -tr. It assumes the results are passed in the right order through the | to xargs allowing then to do the grep.
My misconception was that since when I us ls, the results arrive not as a single column list but as a multiple column list, it couldn't work as an input for xargs.
Here is the simplest solution to date, since it avoids any awk parsing:
ls -tr ${my_log_dir}\* | xargs grep ${my_pattern}
I checked, and every result of the ls -t are passed to xargs even though they look not as I expected they would enter easily in it:
srv:/papi/tata $ ls -t
addl ieet rrri
ooij lllr sss
srv:/papi/tata $ ls -t |xargs -I{} echo {}
addl
ieet
rrri
ooij
lllr
sss
This will work too use find command:-
find -type f -print0 | xargs -r0 stat -c %y\ %n | sort -r | awk '{print $4}' | sed "s|^\./||"
-print0 in find to preserve files having special characters(whitespaces, tabs)
Print file status (stat with %y (Time of last modification) and %n (%n File name) with output having new-separated (-c)
Reverse sort the output from previous command. (-r for reverse)
awk '{print $4}' printing only the file-name (can be optimized as needed)
Removing the leading ./ from the file-names.
I would like to pipe log file with tail -f but only the last line. My command is
tail -f logfile.log | while read line; do /usr/bin/gammu --sendsms TEXT ****** -text "$line"; done
I saw that I can use grep --line-buffered but my new line can't have any char and with what I have tried it doesn't work.
The error shows the problem is other place, which you don't show to us.
Below test code is fine for me.
tail -1f logfile.log | while read line
do
echo /usr/bin/gammu --sendsms TEXT ****** -text "$line"
done