i have a directory with a lot of files like this,
if i use grep to seach a string in those files, then it will search by this order, file log.0, then log.1....
but i want grep to search base on time order,
then i do like this,
grep -i 'stg_data.li51_cicmpdtap0521'
$(ls -ltr sequencer_cmbcs_seq_debug.log*) | less
but i get this error
grep：invalid option -- -
after i change to this, it worked,
grep -i 'stg_data.li51_cicmpdtap0521' $(ls -tr
sequencer_cmbcs_seq_debug.log*) | less
why ls -ltr do not work, but ls -tr work ? what's the difference between with -l and without -l here ?
The reason ls -ltr does not work is because grep is trying to use the entire "long" output of the returned directory listing. Essentially what that equates to is something like this:
-rw-rw-rw- 1 user staff 473 May 24 18:14 file
Which would give you a grep command like this:
grep -i 'string' -rw-rw-rw- 1 user staff 473 May 24 18:14 file | less
Notice the dashes in the first column (example 1); grep can't interpret what to make of the input file and returns "invalid option". When you changed your ls command to remove the -l long output you now just have filenames and grep is able to proceed.
Related
I am trying to learn RegEx, but it is hard.
For example, i have 3 files:
$ ls
thisisnothing12.txt Thisisnothing12.txt thisisnothing.txt
I want to use ls to grep out only the 2 files with digits on it..
These are what i have tried, but they doesn't show even a single file.. why ? What's wrong with em ?
$ ls | grep "^[\w]+[\d]+\.[\w]{3}$"
$ ls | grep "^[a-zA-Z]+[0-9]+\.[a-zA-Z]{3}$"
Thx.
There are different regex flavors, see https://stackoverflow.com/a/66256100/7475450
You need to use PCRE if you want to use \d:
$ touch thisisnothing12.txt Thisisnothing12.txt thisisnothing.txt
$ ls
Thisisnothing12.txt thisisnothing.txt thisisnothing12.txt
$ ls | grep '\d' # '\d' does not work in POSIX Basic regex
$ ls | grep -P '\d' # use PCRE regex
Thisisnothing12.txt
thisisnothing12.txt
$
As you can see you can search for just the characters you are interested in.
You can narrow down, such as finding files that start with a number:
$ touch 2feet.txt
$ ls | grep -P '\d'
2feet.txt
Thisisnothing12.txt
thisisnothing12.txt
$ ls | grep -P '^\d'
2feet.txt
$
Learn more with this tutorial: https://twiki.org/cgi-bin/view/Codev/TWikiPresentation2018x10x14Regex
^[\w]+[\d]+\.[\w]{3}$
^[a-zA-Z]+[0-9]+\.[a-zA-Z]{3}$
Let's simplify a bit. They are both essentially the same thing, because [\w] is the same as \w which is [A-Za-z]. And the same for \d.
So we can simplify to
^\w+\d+\.\w{3}$
The issue is that ^ asserts the start of the string, and $ is the end. grep works on each line. And ls returns all results on one line. You can use ls -1 to get one file per line. You also need the -P flag for grep to work with \w and \d.
$ ls -1 | grep -P "^\w+\d+\.\w{3}$"
You can try different regexes here: https://regexr.com/5mujo
I am trying to find out all directories and the overall size starting with pattern int-*
For this I am using the below command
$sudo ls -ld int-* | grep ^d | wc -l
3339
$ sudo ls -ld int-* | grep ^d | du -sh
204G .
Are my commands correct ? Any other command combination to gather the above information ?
Simply du -shc ./int-*/ should give the grand total of all directories under the pattern int-*. Add a trailing slash would do the trick for directories
AS
-s, report only the sum of the usage in the current directory, not for each directory therein contained
-h, is to obtain the results in human readable format
No, your commands are not okay (though the first is not outright wrong).
Both parse the output of ls which a dangerous thing to do as ls is supposed to produce human-readable output and the format might change in the future (and indeed it has several times over the years, and it differs across various Unix flavors). So generally, parsing the output ls is considered bad. See http://mywiki.wooledge.org/ParsingLs for details.
The second command also pipes this output into du while du isn't reading anything from stdin. It just ignores this kind of input and will do the same as it would have done if being called without the pipe: du -sh. This of course is not what you intended.
What you wanted can best be achieved in a proper fashion like this:
find -maxdepth 1 -type d -name 'int-*' -printf 'x\n' | wc -l
find -maxdepth 1 -type d -name 'int-*' -print0 | du --files0-from=- -c
Using the option --files0-from=- the command du does read NUL-separated file names from stdin. -c makes it print a total of all arguments.
Of course you can still add options -h for human-readable sizes (4G etc.) and -s if you do not want sizes for the subdirectories of your arguments.
If you want only the grand total, the best way is to chomp the output by piping it into tail -1.
I see that Linux tar has an option --overwrite. But overwriting seems to be the default. Moreover, specifying tar --no-overwrite does not change this behavior as the info file seems to suggest.
So what does that option actually do?
I test it with
ls -l >junk
ls -l junk
tar -cf junk.tar junk
>junk
ls -l junk
tar <option?> -xf junk.tar # option varies, results do not
ls -l junk
There are a few subtleties, but in general, here's the difference:
By default, "tar" tries to open output files with the flags O_CREAT | O_EXCL. If the file exists, this will fail, after which "tar" will retry by first trying to delete the existing file and then re-opening with the same flags (i.e., creating a new file).
In contrast, with the --overwrite option, "tar" tries to open output files with the flags O_CREAT | O_TRUNC. If the file exists, it will be truncated to zero size and overwritten.
The main implication is that "tar" by default will delete and re-create existing files, so they'll get new inode numbers. With --overwrite, the inode numbers won't change:
$ ls -li foo
total 0
5360222 -rw-rw-r-- 1 buhr buhr 0 Jun 26 15:16 bar
$ tar -cf foo.tar foo
$ tar -xf foo.tar # inode will change
$ ls -li foo
total 0
5360224 -rw-rw-r-- 1 buhr buhr 0 Jun 26 15:16 bar
$ tar --overwrite -xf foo.tar # inode won't change
$ ls -li foo
total 0
5360224 -rw-rw-r-- 1 buhr buhr 0 Jun 26 15:16 bar
$
This also means that, for each file overwritten, "tar" by default will need three syscalls (open, unlink, open) while --overwrite will need only one (open with truncation).
I often have to look for specific strings in a big set of log files with grep. And I get lots of results, on what I must scroll a lot.
Today, the results of grep list the results in alphabetical order. I would like to have my grep results reversed ordered by time, like a ls -ltr would do.
I know I could take the result of ls -ltr and grep file by file. I do it like this:
ls -ltr ${my_log_dir}\* | awk '{print $9}' |xargs grep ${my_pattern}
But I wonder: Is there a simpler way?
PS: I'm using ksh on AIX.
The solution I found (thanks to Fedorqui) was to simply use ls -tr. It assumes the results are passed in the right order through the | to xargs allowing then to do the grep.
My misconception was that since when I us ls, the results arrive not as a single column list but as a multiple column list, it couldn't work as an input for xargs.
Here is the simplest solution to date, since it avoids any awk parsing:
ls -tr ${my_log_dir}\* | xargs grep ${my_pattern}
I checked, and every result of the ls -t are passed to xargs even though they look not as I expected they would enter easily in it:
srv:/papi/tata $ ls -t
addl ieet rrri
ooij lllr sss
srv:/papi/tata $ ls -t |xargs -I{} echo {}
addl
ieet
rrri
ooij
lllr
sss
This will work too use find command:-
find -type f -print0 | xargs -r0 stat -c %y\ %n | sort -r | awk '{print $4}' | sed "s|^\./||"
-print0 in find to preserve files having special characters(whitespaces, tabs)
Print file status (stat with %y (Time of last modification) and %n (%n File name) with output having new-separated (-c)
Reverse sort the output from previous command. (-r for reverse)
awk '{print $4}' printing only the file-name (can be optimized as needed)
Removing the leading ./ from the file-names.
so I am running the following command
cat /Users/sars/logs/testlogs/2012-04-02*/*/top |grep -H "httpd"
I'm using the * because there are a bunch of directories (which is actually the information I am looking for) and looking for the phrase httpd in the top output
But when I do this I get (standard input): 4951 root 1 96 0 14052K 6844K select 2 0:12 0.00% httpd
instead of the filename
how do I go through these directories look in the top file and find the lines with httpd in them while maintaining the name and path of the file it is found in?
grep can take the filenames as arguments:
grep -H "httpd" /Users/sars/logs/testlogs/2012-04-02*/*/top
ack is a good tool for this sort of thing, but is nonstandard. To do it with grep, you probably want to use find:
find /Users/sars/logs/testlogs -type f \
-wholename '/Users/sars/logs/testlogs/2012-04-02*/*/top/*' \
-exec grep -H httpd {} \;