I've looked, and while this is going to simple I'm guessing, its eluding me.
Simple script:
RDATE=`date +"%m-%d-%y %T"`
echo $RDATE >> Repo_ChangeLog
repo sync | egrep '(From)|(->)' >> Repo_ChangeLog
If I run the egrep by itself on a test file, its fine. I'm missing something before the redirect >> but I'm not sure what.
Related
I'm building a student database in Bourne Shell Script, and this is literally the very first time I've ever even seen code written like this, so I'm terribly out of my element. I need to make it so that when the user inputs a course, the program checks the user input vs a database of courses I already have, and if the course doesn't exist, promps the user to input a new course. This is what I'm trying:
echo "course-1: \c"
read course1
while [[ grep -i "$course1" course3.dat == 1]]
do
echo "course does not exist"
echo "course-1: \c"
read course1
done
echo "course-2: \c"
read course2
while [[ grep -i "$course2" course3.dat == 1]]
do
echo "course does not exist"
echo "course-2: \c"
read course2
done
But I'm getting errors "conditional binary operator expected" and "syntax error near `-i' ". I've been trying to google answers but I'm not coming up with anything useful. So I was wondering if any of you could help me? Thanks so much.
[[ is a shortcut for /bin/test, which isn't what you want. (Here's a man page about it.)
Try this instead:
while ! grep -i "$course1" course3.dat
Or
until grep -i "$course1" course3.dat
The grep expression evaluates to true when grep is successful (i.e. matching lines), and the ! inverts that. Until has built in the opposite semantics from while.
[[ and [ are "test", which is what you want.
However, different shells have different syntaxes; ksh or bash would interpret "[[" okay, but Bourne shell (normally /bin/sh) would not.
I am trying to parse my mvn verify output to only show lines with INFO tags. Please note that maven outputs line to stdout in real time and not by batch. I do not think that it is a problem with maven.
At first I tried to do it with grep:
$ mvn verify | grep INFO
but didn't seem to output lines in real time, as I understand grep buffers its lines before outputting, so I have to wait a few seconds between each flush and then I have tens of lines being printed at the same time, not very convenient. Then I thought I would try with sed.
According to this link, the following command:
sed -n '/PATTERN/p' file
// is equivalent to
grep PATTERN file
and according to this link, the -l option should force sed to flush its output buffer after every newline. So now I am using this command:
$ mvn verify | sed -ln -e '/INFO/p'
but I'm still getting the same result as before, I get a ton of output flushed every 30s or so and I don't know what I've done wrong. Can someone point me in the right direction please?
Try this, if your grep supports it:
mvn verify | grep --line-buffered INFO
If you're doing this in a terminal and still seeing buffered results, it would probably be something earlier than grep doing the buffering, but I'm not familiar with mvn. (And, yes, the -l option to sed should have done the same thing, so the problem may be upstream.)
try this line:
mvn verify | while read line; do echo $line|grep INFO; done
I found what was the problem, I was using a script to colorise maven output (see here) and in fact it was that script that was buffering the output down the pipe. I forgot about it as I was using it as an alias, I guess this is a good lesson, I won't alias as easily in the future. Anyway here is the fix, I changed -e to -le in the last line of the sed call:
mvn $# | sed -e "s/\(\[INFO\]\ \-.*\)/${TEXT_BLUE}${BOLD}\1/g" \
-e "s/\(\[INFO\]\ \[.*\)/${RESET_FORMATTING}${BOLD}\1${RESET_FORMATTING}/g" \
-e "s/\(\[INFO\]\ BUILD SUCCESSFUL\)/${BOLD}${TEXT_GREEN}\1${RESET_FORMATTING}/g" \
-e "s/\(\[WARNING\].*\)/${BOLD}${TEXT_YELLOW}\1${RESET_FORMATTING}/g" \
-e "s/\(\[ERROR\].*\)/${BOLD}${TEXT_RED}\1${RESET_FORMATTING}/g" \
-le "s/Tests run: \([^,]*\), Failures: \([^,]*\), Errors: \([^,]*\), Skipped: \([^,]*\)/${BOLD}${TEXT_GREEN}Tests run: \1${RESET_FORMATTING}, Failures: ${BOLD}${TEXT_RED}\2${RESET_FORMATTING}, Errors: ${BOLD}${TEXT_RED}\3${RESET_FORMATTING}, Skipped: ${BOLD}${TEXT_YELLOW}\4${RESET_FORMATTING}/g"
In effect this is telling sed to flush its output at every new line, which is what I wanted. I am sorry I didn't find another workaround that is more generic. I tried playing around with empty (see man page) and script but none of these solutions worked for me.
I've got a mbox mailbox containing duplicate copies of messages, which differ only in their "X-Evolution:" header.
I want to remove the duplicate ones, in as quick and simple a way as possible. It seems like this would have been written already, but I haven't found it, although I've looked at the Python mailbox module, the various perl mbox parsers, formail, and so forth.
Does anyone have any suggestions?
This a small script, which I used for it:
#!/bin/bash
IDCACHE=$(mktemp -p /tmp)
formail -D $((1024*1024*10)) ${IDCACHE} -s
rm ${IDCACHE}
The mailbox needs to be piped through it, and in the meantime it will be deduplicated.
-D $((1024*1024*10)) sets a 10 Mebibyte cache, which is more than 10x the amount needed to deduplicate an entire year of my mail. YMMV, so adjust it accordingly. Setting it too high will cause some performance loss, setting it to low will let it slip duplicates.
formail is part of the procmail utility bundle, mktemp is part of coreutils.
I didn't look at formail (part of procmail) in enough detail. It does have such such an option, as mentioned in places like: http://hints.macworld.com/comment.php?mode=view&cid=115683 and http://us.generation-nt.com/answer/deleting-duplicate-mail-messages-help-172481881.html
'formail -D' and 'reformail -D' can only process one email per execution. Each mail needs to be separated from mbox first before being processed. I use reformail from maildrop instead since it's still in active development.
remove old idcache, tmpmail, nmbox
run dedup.sh .
nmbox is the output with duplicate messages removed.
dedup.sh
#! /bin/sh
# $1 = mbox, thunderbird mailbox
# wmbox.sh is called for each mail.
cat $1 | reformail -s ./wmbox.sh
wmbox.sh
#! /bin/sh
# stdin: a email
# called by dedup.sh
TM=tmpmail
if [ -f $TM ] ; then
echo error!
exit 1
fi
cat > $TM
# mbox format, each mail end with a blank line
echo "" >> $TM
cat $TM | reformail -D 99999999 idcache
# if this mail isn't a dup (reformail return 1 if message-id is not found)
if [ $? != 0 ]; then
# each mail shall have a message-id
if grep -q -i '^message-id:' $TM; then
cat tmpmail >> nmbox
fi
fi
rm $TM
Getting permission denied error while executing shell command from ruby console.
And the same shell command is working from shell.
From Shell..
tests#tests-workstation:~$ "`grep '^datadir=' /etc/mysql/my.cnf | cut -f 2 -d '='`/db_backups"
bash: /db_backups: is a directory
tests#tests-workstation:~$
From ruby console..
>> %x["`grep '^datadir=' /etc/mysql/my.cnf | cut -f 2 -d '='`/db_backups"]
sh: /db_backups: Permission denied
=> ""
Any Idea !
You're trying to execute a directory and the shells are saying no; bash says no by saying "/db_backups: is a directory" whereas sh says "/db_backups: Permission denied". If you just execute the backedticked part:
grep '^datadir=' /etc/mysql/my.cnf | cut -f 2 -d '='
You'll almost certainly see no output at all and the reason is probably that your regular expression is too tight, something like this:
grep '^[ ]*datadir[ ]*=' /etc/mysql/my.cnf | cut -f2 -d'='
Would serve you better; the character classes contain a space and a tab.
Now that you're looking for the right things we can move on to why it still won't work. The %x[] quoter tries to execute its argument using the shell. When you feed the backticked grep stuff:
`grep '^[ ]*datadir[ ]*=' /etc/mysql/my.cnf | cut -f2 -d'='`/db_backups
to the shell, you should get a directory name that ends with /db_backups but you can't execute a directory. I think you want this to produce the directory name:
d = %x[echo `grep '^[ ]*datadir[ ]*=' /etc/mysql/my.cnf | cut -f2 -d'='`/db_backups].strip
Note the leading echo and the .strip call on the returned string. The .strip is necessary to remove the newline from what echo produces.
I think you're going through a lot of trouble for something that could easily be done with just a couple lines of Ruby:
dir = nil
File.open('/etc/mysql/my.cnf').each do |line|
if(m = line.match(/^\s*datadir\s*=\s*(\S+)/))
dir = m[1] + '/db_backups'
break
end
end
You could probably tighten that up a bit if you wanted but I think that that's at least less confusing than putting shell backticks inside Ruby backticks.
It looks like you just want to get field 2 from the file. Then just do it in Ruby using split
File.open("file").each do |line|
if line[/^datadir/]
print line.split("=",2)[0]
end
end
There is no need to specifically shell out to call grep. This is inefficient and non-portable
I googled and couldn't find any could that would compare a webpage to a previous version.
In this case the page I'm trying to watch is link text. There are services that can watch a page, but I'd like to set this up on my own server.
I've set this up as a wiki so anyone can add to the code. Here's my idea
Check if previous version of file exists. If false then download page
If page exists, diff to find differences and email the new content along with dates of new and old versions.
This script would be called nightly via cron or on-demand via the browser (the latter is not a priority)
Sounds simple, maybe I'm just not looking in the right place.
Perhaps a simple sh-script like this, featuring wget, diff & test?
#!/bin/sh
WWWURI="http://foo.bar/testfile.html"
LOCALCOPY="testfile.html"
TMPFILE="tmpfile"
WEBFILE="changed.html"
MAILADDRESS="$(whoami)"
SUBJECT_NEWFILE="$LOCALCOPY is new"
BODY_NEWFILE="first version of $LOCALCOPY loaded"
SUBJECT_CHANGEDFILE="$LOCALCOPY updated"
SUBJECT_NOTCHANGED="$LOCALCOPY not updated"
BODY_CHANGEDFILE="new version of $LOCALCOPY"
# test for old file
if [ -e "$LOCALCOPY" ]
then
mv "$LOCALCOPY" "$LOCALCOPY.bak"
wget "$WWWURI" -O"$LOCALCOPY" -o/dev/null
diff "$LOCALCOPY" "$LOCALCOPY.bak" > $TMPFILE
# test for update
if [ -s "$TMPFILE" ]
then
echo "$SUBJECT_CHANGEDFILE"
( echo "$BODY_CHANGEDFILE" ; cat "$TMPFILE" ) | tee "$WEBFILE" | mail -s "$SUBJECT_CHANGEDFILE" "$MAILADDRESS"
else
echo "$SUBJECT_NOTCHANGED"
fi
else
wget "$WWWURI" -O"$LOCALCOPY" -o/dev/null
echo "$BODY_NEWFILE"
echo "$BODY_NEWFILE" | tee "$WEBFILE" | mail -s "$SUBJECT_NEWFILE" "$MAILADDRESS"
fi
[ -e "$TMPFILE" ] && rm "$TMPFILE"
Update: Pipe through tee, little spelling & remove of $TMPFILE
You can check This SO posting to get a few ideas and also information about the challenge of detecting "true" changes to a web page (with fluctuating advertisement block, and other "noise")