I clean hacked Wordpress installations for clients on a regular basis, and I have a set of scripts that I have written to help me track down where the sites are hacked. One code snippet that I use on a regular basis worked fine on Ubuntu, but since switching to Fedora on Friday has quite behaving as expected. The command is this:
grep -Iri --exclude "*.js" "eval\s*(" * | grep -rivf ~/safeevals.txt >../foundevals.txt;
What it is supposed to happen (and did happen when I was using Ubuntu): grep through all non-binary files, excluding Javascript includes, for all occurances of the eval() function, then perform a negative match on a line by line basis against all known occurances of the eval() function in a vanilla installation of Wordpress (the patterns of which are in ~/safeevals.txt).
What is actually happening: The first part is working fine, as I ran it separately and it did find all instances of eval() in the installation. However, instead of greping through those results, after the pipe is it re-grepping through all of the files, returning a negative match of ~/safeevals.txt (ie. pretty much every line of every file in the installation).
Any idea why the second grep isn't acting on the piped data, or what I need to do to fix it? Thanks.
-Michael
Just tested on my Debian box: apparently, grep -r likes to assume a default argument of .. I am really wondering if that behaviour is valid. Anyway, I guess dropping the -r option from the second grep command will fix it.
Edit: rgrep defaulting to $PWD seems to be a recent change in grep, see this discussion on unix.stackexchange and the link there to the commit in the upstream grep code repository.
Related
Dipping my toes into Bash coding for the first time (not the most experienced person with Linux either) and I'm trying to read the version from the version.php inside a container at:
/config/www/nextcloud/version.php
To do so, I run:
docker exec -it 1c8c05daba19 grep -eo "(0|[1-9]\d*)\.(0|[1-9]\d*)\.(0|[1-9]\d*)(?:-((?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)(?:\.(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*))?(?:\+([0-9a-zA-Z-]+(?:\.[0-9a-zA-Z-]+)*))?" /config/www/nextcloud/version.php
This uses a semantic versioning RegEx pattern (I know, a bit overkill, but it works for now) to read and extract the version from the line:
$OC_VersionString = '20.0.1';
However, when I run the command it tells me No such file or directory, (I've confirmed it does exist at that path inside the container) and then proceeds to spit out the entire contents of the file it just said doesn't exist?
grep: (0|[1-9]\d*).(0|[1-9]\d*).(0|[1-9]\d*)(?:-((?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-])(?:.(?:0|[1-9]\d|\d*[a-zA-Z-][0-9a-zA-Z-]))))?(?:+([0-9a-zA-Z-]+(?:.[0-9a-zA-Z-]+)*))?: No such file or directory
/config/www/nextcloud/version.php:$OC_Version = array(20,0,1,1);
/config/www/nextcloud/version.php:$OC_VersionString = '20.0.1';
/config/www/nextcloud/version.php:$OC_Edition = '';
/config/www/nextcloud/version.php:$OC_VersionCanBeUpgradedFrom = array (
/config/www/nextcloud/version.php: 'nextcloud' =>
/config/www/nextcloud/version.php: 'owncloud' =>
/config/www/nextcloud/version.php:$vendor = 'nextcloud';
Anyone able to spot the problem?
Update 1:
For the sake of clarity, I'm trying to run this from a bash script. I just want to fetch the version number from that file, to use it in other areas of the script.
Update 2:
Responding to the comments, I tried to login to the container first, and then run the grep, and still get the same result. Then I cat that file and it shows it's contents no problem.
Many containers don't have the GNU versions of Unix tools and their various extensions. It's popular to base containers on Alpine Linux, which in turn uses a very lightweight single-binary tool called BusyBox to provide the base tools. Those tend to have the set of options required in the POSIX specs, and no more.
POSIX grep(1) in particular doesn't have an -o option. So the command you're running is
grep \
-eo \ # specify "o" as the regexp to match
"(regexps are write-only)" \ # a filename
/config/www/nextcloud/version.php # a second filename
Notice that the grep output in the interactive shell only contains lines with the letter "o", but not for example the line just containing array.
POSIX grep doesn't have an equivalent for GNU grep's -o option
Print only the matched (non-empty) parts of matching lines, with each such part on a separate output line. Output lines use the same delimiters as input....
but it's easy to do that with sed(1) instead. Ask it to match some stuff, the regexp in question, and some stuff, and replace it with the matched group.
sed -e 's/.*\(any regexp here\).*/\1/' input-file
(POSIX sed only accepts basic regular expressions, so you'll have to escape more of the parentheses.)
Well, for any potential future readers, I had no luck getting grep to do it, I'm sure it was my fault somehow and not grep's, but thanks to the help in this post I was able to use awk instead of grep, like so:
docker exec -it 1c8c05daba19 awk '/^\$OC_VersionString/ && match($0,/\047[0-9]+\.[0-9]+\.[0-9]+\047/){print substr($0,RSTART+1,RLENGTH-2)}' /config/www/nextcloud/version.php
That ended up doing exactly what I needed:
It logs into a docker container.
Scans and returns just the version number from the line I am looking for at: /config/www/nextcloud/version.php inside the container.
Exits stage left from the container with just the info I needed.
I can get right back to eating my Hot Cheetos.
**NOTE: I've added updates in order, just keep reading, thanks. :) **
I've been very curious about this -- please see this screenshot of me running:
ls -lah build, and
yarn run assets, which runs ls -lah build.
Let me start by saying that this is a WIP build in webpack, so no need to tell me that a 31M bundle is less than optimal. :)
But why do I get the colors and the more detailed font with the native command and not when yarn executes the command? It may be relevant: this screen shot is:
- Windows 10
- Webstorm terminal
- logged in to a docker container running Ubuntu 14.4
Thanks! :)
** UPDATE: --color=always restores color **
As #Charles Duffy suggested, adding --color=always in the yarn script preserved the formatting:
If anyone has some specialized knowledge to share about what's going on here, I'm in the market to hear it! Thanks!
Short(ish) answer: What's actually going on?
The below answer assumes the GNU implementation of ls.
There are a few possibilities at play:
Your interactive terminal's options may be modified by a shell alias. Output from type ls will indicate whether this is true.
You may have ls --color=auto enabled, either via an alias or via an equivalent environment variable; regardless, this checks whether it's writing directly to a TTY, and only enables color if so.
If output is not direct to a TTY (for instance, if output is being captured by yarn before it's printed), ls --color=auto will not colorize.
To fix this, you can explicitly pass ls --color=always, or its equivalent, simply ls --color. This covers both cases: If you had an alias in use passing --color=auto on your behalf, passing it explicitly means you no longer need the alias. By contrast, if yarn is capturing content rather than passing it straight to the TTY, then --color=always tells ls to ignore isatty() returning false and colorize anyhow.
Background on what the above means:
A "TTY" is, effectively, a terminal. It provides bells and whistles (literally, for the bells) specialized for providing a device that a user is actually typing at. This means it has control sequences for inspecting and modifying cursor location, and -- pertinently for our purposes -- for changing the color with which output is rendered.
A "FIFO" is a pipe -- it moves characters from point A to point B, first-in, first-out. In the case of prog-one | prog-two, the thing that connects those two is a FIFO. It just moves characters, and has no concept of cursor location or colorization or anything else.
If ls tried to put color sequences in its output when that output is intended for any destination other than a terminal, those sequences wouldn't make any sense -- indeed, the very format in which colorization markers need to be printed is determined by the TERM variable specifying the currently active terminal type.
If you run ls --color, then, you're promising ls that its output really will be rendered by a terminal, or (at least) otherwise something that understands the color sequences appropriate to the currently configured TERM.
I stopped a process to troubleshoot something. Now, I would like to start the process where it left off in CentOS 6.4.
This script I stopped runs a perl script in a loop to process all of the files in /dev/shm/split/. These files were split into many parts from a larger file. An example of how they are named are as follows:
file.txt.aa
file.txt.ab
file.txt.ac
...and so on.
I have identified that the script left off at file.txt.fy. So, I would like to remove all of the files in /dev/shm/split/ that are from file.txt.aa through file.txt.fy.
I tried to create a whitelist for the rm command by doing:
ls /dev/shm/split/ > whitelist
cat whitelist | egrep -v 'file.txt.[aa-fz]' | tee whitelist.tmp
This did not do what I had intended.
Please help me! Thank you!
The problem with your command is that you cannot match two characters with the square bracket pattern in bash. You should use something like that instead:
ls file.txt.[a-e]? file.txt.f[a-y]
Basically decompose your range into two ranges, the first will match .aa to .ez, and the second .fa to .fy (included).
Note that I have used the ls command here. I always find it a good idea to first echo or ls the commands/files you're going to run when the operations you do are potentially destructive. When you're sure it produces the right output, go on and use rm instead of ls.
Reading a Rails log with vim, we can see a colored log.
But when we use tail -f or less to watch the log, it isn't colorized anymore. Is there any way to see the colored log with tail or less or whatever?
pass -R to less for it to let colour escape sequences pass through, i.e.
less -R log/development.log
This should result in them being displayed in colour, assuming you are using the proper terminal type
tail -f log/development.log | ccze -A
You may need to install ccze
sudo apt-get install ccze
it works better, not ideal but works
You can't do that out of the box, since tail and less know nothing about Rails, and logs are saved a simple text files. You could use regular expressions to colorize output but I doubt it's worth the trouble.
EDIT: see alfonso's comment for some alternatives.
I want to grep 2 patterns in a file on Solaris UNIX.
That is grep 'pattern1 OR pattern2' filename.
The following command does NOT work:
grep 'pattern1\|pattern2' filename
What is wrong with this command?
NOTE: I am on Solaris
What operating system are you on?
It will work with on systems with GNU grep, but on BSD, Solaris, etc., \| is not supported.
Try egrep or grep -E, e.g.
egrep 'pattern1|pattern2'
If you want POSIX functionality (i.e. Linux-like behavior) you can put the POSIX 2-compatible binaries at the beginning of your path:
$ echo $PATH
/usr/xpg4/bin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin:[...]
There is also /usr/xpg6 which is POSIX 1 compatible.
/usr/bin: SVID/XPG3
/usr/xpg4/bin: POSIX.2/POSIX.2a/SUS/SUSv2/XPG4
/usr/xpg6/bin: POSIX.1-2001/SUSv3
That command works fine for me. Please add additional information such as your platform and the exact regular expression and file contents you're using (minimized to the smallest example that still reproduces the issue). (I would add a comment to your post but don't have enough reputation.)
That should be correct. Make sure that you do or don't add the appropriate spaces i.e. "pattern1\|pattern2" vs "pattern1\| pattern2".
Are you sure you aren't just having problems with cases or something? try the -i switch.
That depends entirely on what pattern1 and pattern2 are. If they're just words, it should work, otherwise you'll need:
grep '\(pattern1\)\|\(pattern2\)'
An arcane method using fgrep (ie: fixed strings) that works on Solaris 10...
Provide a pattern-list, with each pattern separated by a NEWLINE, yet quoted so as to be interpreted by the shell as one word:-
fgrep 'pattern1
pattern2' filename
This method also works for grep, fgrep and egrep in /usr/xpg4/bin, although the pipe-delimited ERE in any egrep is sometimes the least fussy.
You can insert arbitrary newlines in a string if your shell allows history-editing, eg: in bash issue C-v C-j in either emacs mode or vi-command mode.
egrep -e "string1|string2" works for me in SunOS 5.9 (Solaris)