I need to tar -cvf but only tar the files that are added or changed within 48 hrs in our mail directory
How could I do that. Thank you for your help.
Try something like this
find MYMAILDIR -ctime 2 | xargs tar -cvf OUTPUT.tar
If your filenames have spaces in them, then use -print0 in find and -0 in xargs.
Related
I'd like to calculate MD5 for all files in a tar archive. I tried tar with --to-command.
tar -xf abc.tar --to-command='md5sum'
it outputs like below.
cb6bf052c851c1c30801ef27c9af1968 -
f509549ab4eeaa84774a4af0231cccae -
Then I want to replace '-' with file name.
tar -xf abc.tar --to-command='md5sum | sed "s#-#$TAR_FILENAME#"' it reports error.
md5sum: |: No such file or directory
md5sum: sed: No such file or directory
md5sum: s#-#./bin/busybox#: No such file or directory
tar: 23255: Child returned status 1
You don't have a shell so this won't work (you also might see that the | gets to md5sum as an argument). one way could be to invoke the shell yourself, but there is some hassle with nested quotes:
tar xf some.tar --to-command 'sh -c "md5sum | sed \"s|-|\$TAR_FILENAME|\""'
At first, it's better to avoid using sed, not only because it's slow, but because $TAR_FILENAME can contain magic chars to be interpreted by sed (you already noticed that, having to use # instead of / for substitution command, didnt you?). Use deadproof solution, like head, followed by echoing actual filename.
Then, as Patrick mentions in his answer, you can't use complex commands without having them wrapped with shell, but for convenience I suggest to use built-it shell escapement ability, for bash it's printf '%q' "something", so the final command be like:
tar xf some.tar \
--to-command="sh -c $(printf '%q' 'md5sum | head -c 34 && printf "%s\n" "$TAR_FILENAME"')"
"34" is number of bytes before file name in md5sum output format; && instead of ; to allow md5sum's error code (if any) reach tar; printf instead of echo used because filenames with leading "-" may be interpreted by echo as options.
I could do this for .zip files in the folder using the command below:
for f in "!"; do unzip -d "${f%*.zip}" "$f"; done
The above command extracts all .zip files in a given folder to subfolders, having content and name of respective .zip files.
But I couldn't find a command that would do the same for .tar files. Please help.
Btw, I am trying to do this on a remote server using WinSCP/putty. So, I cannot use a GUI software. I need a command, thus the question.
After a bit of fiddling I came up with for f in $(find -maxdepth 1 | grep .tar); do mkdir ${f%.tar}; tar -xaf $f -C ${f%.tar} ; done appears to work, so long as the file name does not contain any spaces. I assume you wanted the directory from foo.tar to be named foo (no file extension). If you want the directory to be named foo.tar (with file extension) then try using for f in $(find -maxdepth 1 | grep .tar); do mkdir $f ; tar -xaf $f -C $f ; done.
IIRC, the remote access client Cyberduck can handle compressed files in a GUI - so you can try that if you're fine with a GUI solution.
I have a file dagens_130325.tar.gz containing the folder dagens. In one folder I have hundreds of these daily info. I would like to unpack dagens_130325.tar.gz/dagens to 130325 with all the files inside. Then 130326 etc.
Is there a way to do it?
Not sure this is the right stack where to ask this kind of question, however try with
tar -zxvf dagens_130325.tar.gz -C /tmp/130325 dagens
This way, the folder dagens for the archive dagens_130325.tar.gz is going to be extracted into /tmp/130325. However, note that the target folder must exist, otherwise the command will fail
So, supposedly you have 4 archives in the form dagens_1.tar.gz, dagens_2.tar.gz, ..., you can write an extract.sh file containing
#!/bin/bash
for i in {1..4}
do
mkdir /tmp/$i
FILE="dagens_$i.tar.gz"
tar -zxvf $FILE -C /tmp/$i dagens
done
Having this file the execute permission, being in the same folder as your archives and executing it should produced the result you asked.
This was the solution I came up with in the end
#!/bin/bash
search_dir=/yourdir/with/tar.gz
for entry in "$search_dir"/*.tar.gz
do
substring=$(basename "$entry")
echo $substring
sub2=${substring:7:6}
tar -xvzf $substring
rm -rf $sub2
mv dagens $sub2
done
use
#!/bin/bash
for file in dagens_*.tar.gz
do
from=${file%_*} #removes chars after _
to=${file#*_} #removes chars before _
to=${to%.t*} #removes chars after .t (.tar.gz)
tar -zxf $file --show-transformed --transform "s/$from/$to/"
done
The problem:
I have a back-end process that at some point he collect and build a big tar file.
This tar receive few directories and an exclude files.
the process can take up to few minutes and i want to report in my front-end process (GUI) about the progress of the taring process (This is a big issue for a user that press download button and it seems like nothing is happening...).
i know i can use -v -R in the tar command and count files and size progress but i am looking for some kind of tar pre-run mode / dry run to help me evaluate either the expected number of files or the expected tar size.
the command I am using: tar -jcf 'FILE.tgz' 'exclude_files' 'include_dirs_and_files'
10x for everyone who is willing to assist.
You can pipe the output to the wc tool instead of actually making a file.
With file listing (verbose):
[git#server]$ tar czvf - ./test-dir | wc -c
./test-dir/
./test-dir/test.pdf
./test-dir/test2.pdf
2734080
Without:
[git#server]$ tar czf - ./test-dir | wc -c
2734080
Why don't you run a
DIRS=("./test-dir" "./other-dir-to-test")
find ${DIRS[#]} -type f | wc -l
beforehand. This gets all the files (-type f) one per line and counts the number of files. DIRS is an array in bash, so you can store the folders in a variable
If you want to know the size of all the stored files, you can use du
DIRS=("./test-dir" "./other-dir-to-test")
du -c -d 0 ${DIRS[#]} | tail -1 | awk -F ' ' '{print $1}'
This prints the disk usage with du, calculates a grand total (-c flag), gets the last line (example 4378921 total), and uses just the first column with awk
Is there any way I could use grep to ignore some files when searching something, something equivalent to svnignore or gitignore? I usually use something like this when searching source code.
grep -r something * | grep -v ignore_file1 | grep -v ignore_file2
Even if I could set up an alias to grep to ignore these files would be good.
--exclude option on grep will also work:
grep perl * --exclude=try* --exclude=tk*
This searches for perl in files in the current directory excluding files beginning with try or tk.
You might also want to take a look at ack which, among many other features, by default does not search VCS directories like .svn and .git.
find . -path ./ignore -prune -o -exec grep -r something {} \;
What that does is find all files in your current directory excluding the directory (or file) named "ignore", then executes the command grep -r something on each file found in the non-ignored files.
Use shell expansion
shopt -s extglob
for file in !(file1_ignore|file2_ignore)
do
grep ..... "$file"
done
I thinks grep does not have filename filtering.
To accomplish what you are trying to do, you can combine find, xargs, and grep commands.
My memory is not good, so the example might not work:
find -name "foo" | xargs grep "pattern"
Find is flexible, you can use wildcards, ignore case, or use regular expressions.
You may want to read manual pages for full description.
after reading next post, apparently grep does have filename filtering.
Here's a minimalistic version of .gitignore. Requires standard utils: awk, sed (because my awk is so lame), egrep:
cat > ~/bin/grepignore #or anywhere you like in your $PATH
egrep -v "`awk '1' ORS=\| .grepignore | sed -e 's/|$//g' ; echo`"
^D
chmod 755 ~/bin/grepignore
cat >> ./.grepignore #above set to look in cwd
ignorefile_1
...
^D
grep -r something * | grepignore
grepignore builds a simple alternation clause:
egrep -v ignorefile_one|ignorefile_two
not incredibly efficient, but good for manual use