I have a directory named dir with files file1 file2 file3 in it and I want to create a gzipped tar directory from it called dir.tar.gz but with the hierarchy as:
anotherdir/file1
anotherdir/file2
anotherdir/file3
There's always the option of writing a shell script to rename dir to anotherdir then tar it, but I was hoping for a more elegant solution.
Use a symlink and dereference during tarring.
$ mkdir dir && touch dir/file{1,2,3} && ln -sf dir anotherdir
$ tar -hcvpf a.tar anotherdir
anotherdir/
anotherdir/file1
anotherdir/file2
anotherdir/file3
$ tar -tf a.tar
anotherdir/
anotherdir/file1
anotherdir/file2
anotherdir/file3
You can make a symlink and then force tar to follow it:
ln -s dir anotherdir
tar -czf dir.tar.gz -L anotherdir
Beware that will also follow any symlinks inside the tree.
The following seems to work fine. Tar will not follow links, but the resulting tar will have additional ./ for the files (which I think is not a big deal).
mkdir dir another_dir && touch dir/file{1,2,3} another_dir/another_file{1,2,3}
tmp_dir=`mktemp -d /tmp/dXXXX` && ln -s `pwd`/dir $tmp_dir/another_dir &&
tar -C $tmp_dir -cvf result.tar another_dir/. &&
rm $tmp_dir/another_dir && rmdir $tmp_dir
Related
I am using security scan software in my Dockerfile and I need to add its bin folder to the path. Its path will contain the version part so I do not know the path until I download the software. My current progress is something like this:
1.Download the software:
RUN curl https://cloud.appscan.com/api/SCX/StaticAnalyzer/SAClientUtil?os=linux --output SAClientUtil.zip
RUN unzip SAClientUtil.zip -d SAClientUtil
2.The desired folder is located: SAClientUtil/SAClientUtil.X.Y.Z/bin/ (xyz mary vary from run to run). Get there using find and cd combination and try to add it to the PATH:
RUN cd "$(dirname "$(find SAClientUtil -type f -name appscan.sh | head -1)")"; \
export PATH="$PATH:$PWD"; # doesn't work
Looks like ENV command is not evaluating the parameter, so
ENV PATH $PATH:"echo $(dirname "$(find SAClientUtil -type f -name appscan.sh | head -1)")"
doesn't work also.
Any ideas on how to dynamically add a folder to the PATH during docker image build?
If you're pretty sure the zip file will contain only a single directory with that exact layout, you can rename it to something fixed.
RUN curl https://cloud.appscan.com/api/SCX/StaticAnalyzer/SAClientUtil?os=linux --output SAClientUtil.zip \
&& unzip SAClientUtil.zip -d tmp \
&& mv tmp/SAClientUtil.* SAClientUtil \
&& rm -rf tmp SAClientUtil.zip
ENV PATH=/SAClientUtil/bin:${PATH}
A simple solution would be to include a small wrapper script in your image, and then use that to run commands from the SAClientUtil directory. For example, if I have the following in saclientwrapper.sh:
#!/bin/sh
cmd=$1
shift
saclientpath=$(ls -d /SAClientUtil/SAClientUtil.*)
echo "got path: $saclientpath"
cd "$saclientpath"
exec "$saclientpath/bin/$cmd" "$#"
Then I can do this:
RUN curl https://cloud.appscan.com/api/SCX/StaticAnalyzer/SAClientUtil?os=linux --output SAClientUtil.zip
RUN unzip SAClientUtil.zip -d SAClientUtil
COPY saclientwrapper.sh /saclientwrapper.sh
RUN sh /saclientwrapper.sh appscan.sh
And this will produce, when building the image:
STEP 6: RUN sh /saclientwrapper.sh appscan.sh
got path: /SAClientUtil/SAClientUtil.8.0.1374
COMMAND SYNTAX
appscan <command> [options]
ADDITIONAL COMMAND HELP
appscan help <command>
.
.
.
Is there a way to append a file into a specific dir inside a tar file?
I have dir1/TarFile.tar. Inside the TarFile.tar there is dir2.
I want to append file.txt (in dir 1) to TarFile.tar under dir2.
Thanks in advance.
As far as i know you can't do this on the fly. You can create directory dir2, move there the file file.txt and update the archive:
cd dir1
mkdir -p dir2
mv file.txt dir2/
tar uvf TarFile.tar dir2/file.txt
I typically do :
tar czf $directory_path/file_${1}.tar.gz --directory=$specified_path/ *${1}.csv .
What if I just want to include only certain files of a directory, but not the directory itself?
cd [dir]; tar cz ./* >x.tgz
or
cd [dir]; tar cz file1 file2 >x.tgz
I'm new to Docker and ran into the following problem:
In my Dodckerfile I have these lines:
ADD dir/archive.tgz /dir/
RUN tar -xzf /dir/archive2.tar.gz -C /dir/
RUN ls -l /dir/
RUN ls -l /dir/dir1/
The first ls prints out files correctly and I can see that dir1 was created inside dir by the archive, with permissions drwxr-xr-x. But the second ls gives me:
ls: "cannot access /dir/dir1/: No such file or directory"
I thought that if the Docker can see a file, it can access it. Do I need to do some special magic here?
I thought that if the Docker can see a file, it can access it.
In a way you are right, but also missing a piece of info. Those RUN commands are not necessarily sequentially executed since docker operates in layers, and your third RUN command is executed while your first might be skipped. In order to preserve proper execution order you need to put them in same RUN command as such so they end up on the same layer (and are updated together):
RUN tar -xzf /dir/archive2.tar.gz -C /dir/ && \
ls -l /dir/ && \
ls -l /dir/dir1/
This is common issue, most often when this is put in Dockerfile:
RUN apt-get update
RUN apt-get install some-package
Instead of this:
RUN apt-get update && \
apt-get install some-package
Note: This is in line with best practices for usage of RUN command in Dockerfile, documented here: https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#run and avoids possible confusion with caches/layes...
To recreate your problem here is small test to resemble similar setup to yours, depending on actual directory structure in your archive this may differ:
Dummy archive 2 with dir/dir1/somefile.txt created:
mkdir -p ~/test-sowf/dir/dir1 && cd ~/test-sowf && echo "Yay" | tee --append dir/dir1/somefile.txt && tar cvzf archive2.tar.gz dir && rm -rf dir
Dockerfile created in ~/test-sowf with following content
from ubuntu:latest
COPY archive2.tar.gz /dir/
RUN tar xvzf /dir/archive2.tar.gz -C /dir/ && \
ls -l /dir/ && \
ls -l /dir/dir/dir1/
Build command like so:
docker build -t test-sowf .
Gives following result:
Sending build context to Docker daemon 5.632kB
Step 1/3 : from ubuntu:latest
---> 452a96d81c30
Step 2/3 : COPY archive2.tar.gz /dir/
---> Using cache
---> 852ef4f706d3
Step 3/3 : RUN tar xvzf /dir/archive2.tar.gz -C /dir/ && ls -l /dir/ && ls -l /dir/dir/dir1/
---> Running in b2ab281190a2
dir/
dir/dir1/
dir/dir1/somefile.txt
total 8
-rw-r--r-- 1 root root 177 May 10 15:43 archive2.tar.gz
drwxr-xr-x 3 1000 1000 4096 May 10 15:43 dir
total 4
-rw-r--r-- 1 1000 1000 4 May 10 15:43 somefile.txt
Removing intermediate container b2ab281190a2
---> 05b7dfe52e36
Successfully built 05b7dfe52e36
Successfully tagged test-sowf:latest
Note that extracted files are with 1000:1000 as opposed to root:root for the archive, so unless you are not running from some other user (non root) you should not have problems with user, but, depending on your archive you might run into path problems (/dir/dir/dir1 as shown here).
test that file is correct, and contains 'Yay' inside:
docker run --rm --name test-sowf test-sowf:latest cat /dir/dir/dir1/somefile.txt
clean the test mess afterwards (deliberatelynot using rm -rf but cleaning individual files):
docker rmi test-sowf && cd && rm ~/test-sowf/archive2.tar.gz && rm ~/test-sowf/Dockerfile && rmdir ~/test-sowf
For those using docker-compose:
Sometimes when you volume mount a folder/file from one container to another before it exists, it can have weird permissions after it's created
For example if one container is certbot and another is your webserver, certbot will take time to generate the /etc/letsencrypt folder and its contents
From the webserver you might be able to see the folder or its contents with an ls, but not open them. You can see the behavior with a cat * and you'll get back
cat: <files in question>: No such file or directory
One solution is generating the folder at build time with a RUN mkdir -p /directory/of/choice in your dockerfile for the container generating the folder/files. Then the folder will exist and docker will happily mount it to your other container or host machine the way you want it to
The following will extract the files in /root/ directory. But it also creates the parent directories under root. What I need is that the files should be exactly under root folder and not in /root/data/mysql/...
# tar -xvf company_raw_2012-02-22.tgz --directory=/root/
data/mysql/company_raw/data_archive_r_20120222.MYD
data/mysql/company_raw/data_archive_r_20120222.MYI
data/mysql/company_raw/data_archive_r_20120222.frm
If that is not possible, how do I write a program to move these files to the required folder?
I have tried the following and it does work.
--strip-components=3
But I do not know how many folder will be there. So the number 3 may change.
Extract everything to the temp directory with full path and then just walk it moving files to the desired destination?
destdir=/root
tmpdir=/root/tmp
rm -rf $tmpdir
mkdir $tmpdir
tar xf archive.tar.gz -C $tmpdir
find -H $tmpdir -type f -exec mv '{}' $destdir \;