For some tar adept, I'm sure all of the following could be achieved on one line?
mkdir site_media/media
cp fixtures/media.tar site_media/media/media.tar
cd site_media/media/
tar -xvf media.tar
rm media.tar
cd ../../
There is really no need to copy the tar file if you are only going to delete the copy anyway. This will work for you without the copying, deleting, and changing directories:
mkdir site_media/media; tar -xvf fixtures/media.tar -C site_media/media
Comedy answer:
mkdir site_media/media && cp fixtures/media.tar site_media/media/media.tar &&
cd site_media/media/ && tar -xvf media.tar && rm media.tar && cd ../../
Slightly more serious: I don't know of a way to create the base directory as part of a single tar, but it can be reduced somewhat:
mkdir -p site_media/media
tar -xvf fixtures/media.tar -C site_media/media
This avoids the delete by avoiding the copy in the first place. The C option in extract mode means 'change to this directory after opening the archive but before extracting anything'.
As #Steve-o humorously points out, you can combine lines with &&, which also has the effect of Boolean short-circuiting: stopping if a command fails.
Related
I have the following Dockerfile
FROM python:3.9-slim-buster
## DO SOMETHING HERE
RUN /bin/bash -c shopt -s extglob && rm -rfv !(".env")
I am getting
Step 42/49 : RUN /bin/bash -c shopt -s extglob && rm -rfv !(".env")
---> Running in 5b4ceacb1908
/bin/sh: 1: Syntax error: "(" unexpected
HOw to run this command. I need this
Every RUN line in your Dockerfile will launch a new container with a separate shell. The extglob shell option must be set before a line of input is parsed by the shell, so one way to go about this is to launch the shell with that option enabled:
RUN /bin/bash -O extglob -c 'rm -rfv !(".env")'
The main caveat is that you have to quote the command line as a string.
I'd recommend avoiding bash-specific syntax wherever possible.
In this case, the bash-specific glob pattern !(.env) means "every file except .env. So you can accomplish this specific task by moving .env out of the way, deleting the whole directory, recreating it, and moving .env back; all without worrying about which shell expansion objects are set.
RUN cd .. \
&& mv the_dir/.env . \
&& rm -rf the_dir \
&& mkdir the_dir \
&& mv .env the_dir
You also might consider whether you need a broad rm -rf at all. Because of Docker's layer system, the previous content of the directory is still "in the image". If you use a multi-stage build then the later stage will start from a clean slate, and you can COPY in whatever you need.
FROM python:3.9-slim-buster AS build
...
FROM python:3.9-slim-buster
WORKDIR /app
COPY .env .
COPY --from=build ...
/bin/bash seems to work for "shopt -s extglob" part but not the other. Separate the lines like this:
RUN /bin/bash -c shopt -s extglob
RUN /bin/bash -c rm -rfv !(".env")
or
RUN /bin/bash -c "shopt -s extglob && rm -rfv !(\".env\")"
I have the following code
RUN apt-get update
RUN apt-get install -y wget #install wget lib
RUN mkdir -p example && cd example #create folder and cd to folder
RUN WGET -r https://host/file.tar && tar -xvf *.tar # download tar file to example folder and untar it in same folder
RUN rm -r example/*.tar # remove the tar file
RUN MV example/foo example/bar # rename untar directory from foo to bar
But i get the following errors:
/bin/sh: 1: WGET: not found
tar: example/*.tar: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
I am a newbie in docker.
Each subsequent RUN command in your Dockerfile will be in the context of the / directory. Therefore your .tar file is not in the example/ directory, it would actually be in the / directory since your 'cd to the folder' would mean nothing for subsequent RUN commands. Instead of doing cd example, rather do WORKDIR example before running wget, eg:
RUN apt-get update
RUN apt-get install -y wget #install wget lib
RUN mkdir -p example # create folder and cd to folder
WORKDIR example/ # change the working directory for subsequent commands
RUN wget -r https://host/file.tar && tar -xvf *.tar # download tar file to example folder and untar it in same folder
RUN rm -r example/*.tar # remove the tar file
RUN mv example/foo example/bar # rename untar directory from foo to bar
Or alternatively, add cd example && ... some command before any command you'd like to execute within theexample directory.
As Ntokozo stated, each RUN command is a separate "session" in the build process. As such, Docker is really designed to pack as many commands into a single RUN as possible allowing for smaller overall image size and fewer layers. So the command could be written like so:
RUN apt-get update && \
apt-get install -y wget && \
mkdir -p example && \
cd example/ && \
wget -r https://host/file.tar && \
tar -xvf *.tar && \
rm -r example/*.tar && \
mv example/foo example/bar
From Spring Microservices in Action book: I am trying to use the Docker Maven Plugin to build a docker image for deploy a Java microservice as Docker container to the cloud.
Dockerfile:
FROM openjdk:8-jdk-alpine
RUN mkdir -p /usr/local/configserver
ADD jce_policy-8.zip /tmp/
RUN unzip /tmp/jce_policy-8.zip && \
rm /tmp/jce_policy-8.zip && \
yes | cp -v /tmp/UnlimitedJCEPolicyJDK8/*.jar /usr/lib/jvm/java-1.8-openjdk/jre/lib/security/
ADD #project.build.finalName#.jar /usr/local/configserver/
ADD run.sh run.sh
RUN chmod +x run.sh
CMD ./run.sh
Output related to step 4 in Dockerfile:
...
---> Using cache
---> dd33d4c12d29
Step 4/8 : RUN unzip /tmp/jce_policy-8.zip && rm /tmp/jce_policy-8.zip && yes | cp -v /tmp/UnlimitedJCEPolicyJDK8/*.jar /usr/lib/jvm/java-1.8-openjdk/jre/lib/security/
---> Running in 1071273ceee5
Archive: /tmp/jce_policy-8.zip
unzip: short read
Why do I get unzip: short read when I try to build the image?
Somehow, curl on alpine linux distro can't set cookie headers correctly while downloading jce zip file. It seems it downloads a zip file but in fact it is an html error page. If you view the file you can see that it is an html file. I've used wget instead of curl and it successfully downloaded file. Then unzip operation worked as expected.
FROM openjdk:8-jdk-alpine
RUN apk update && apk upgrade && apk add netcat-openbsd
RUN mkdir -p /usr/local/configserver
RUN cd /tmp/ && \
wget 'http://download.oracle.com/otn-pub/java/jce/8/jce_policy-8.zip' --header "Cookie: oraclelicense=accept-securebackup-cookie" && \
unzip jce_policy-8.zip && \
rm jce_policy-8.zip && \
yes |cp -v /tmp/UnlimitedJCEPolicyJDK8/*.jar /usr/lib/jvm/java-1.8-openjdk/jre/lib/security/
ADD #project.build.finalName#.jar /usr/local/configserver/
ADD run.sh run.sh
RUN chmod +x run.sh
CMD ./run.sh
It's possible your jce_policy-8.zip archive is being recognized as a compressed archive and expanded by the ADD instruction. If so, you can skip unzipping on the next line. Or, switch to the COPY instruction, which does no special processing of local archives.
In general, I recommend always using the COPY instruction to bring in files and directories from the build context. Only use ADD when you specifically want the extra unpacking behaviour.
I'm find solved link
FROM openjdk:8-jdk-alpine
RUN apk update && apk upgrade && apk add netcat-openbsd && apk add curl
RUN mkdir -p /usr/local/configserver
RUN cd /tmp/ && \
**curl -L -b "oraclelicense=a" http://download.oracle.com/otn-pub/java/jce/8/jce_policy-8.zip -O** && \
unzip jce_policy-8.zip && \
rm jce_policy-8.zip && \
yes |cp -v /tmp/UnlimitedJCEPolicyJDK8/*.jar /usr/lib/jvm/java-1.8-openjdk/jre/lib/security/
ADD #project.build.finalName#.jar /usr/local/configserver/
ADD run.sh run.sh
RUN chmod +x run.sh
CMD ./run.sh
Maybe it is related to the fact that the unzip command in alpine is provided busybox and not the standard unzip tool.
Busybox do have bugs related to this error:
https://bugs.busybox.net/show_bug.cgi?id=8821
Here is a related issue with more details:
https://github.com/wahern/luaossl/issues/103
As a workaround installing the standard unzip command should work.
I am trying to build the docker image with perl installation.
Dockerfile:
FROM amazonlinux
WORKDIR /shared
RUN yum -y install gcc
ADD http://www.cpan.org/src/5.0/perl-5.22.1.tar.gz /shared
RUN tar -xzf perl-5.22.1.tar.gz
WORKDIR /shared/perl-5.22.1
RUN ./Configure -des -Dprefix=/opt/perl-5.22.1/localperl
RUN make
RUN make test
RUN make install
all these steps are executed i am can see it executing the make, make test and make install commands but when i do :
docker run -it testsh /bin/bash
Error:
when I check perl -v it says command not found.
and I need to go the perl directory
'cd perl-5.22.1' and run 'make install' again then perl -v works
But I want the perl installation to work when I build it with docker image. can anyone tell me what is going wrong here?
perl was indeed installed, just wasn't added to the path.
export PATH=$PATH:/shared/perl-5.22.1 should do it -- but of course, you'd want to add a PATH update in the Dockerfile.
At first glance I thought that when you run make install second time, it adds perl's bin directory to PATH env, but when I compared output of env before and after make install it showed the same PATH variable content.
The reason you getting perl -v working after make install in running container is that make install puts perl binary to /usr/bin/perl. I don't know why it works such way, but it is just as it is. Also, it's almost useless to store sources inside of your image.
Anyway, I agree with #belwood suggestion about adding your perl's bin directiry to PATH environment variable. I just wanna correct the path: /opt/perl-5.22.1/localperl/bin
You need to add it in your Dockerfile (basically I've rewritten your file to make it produce more efficient image), for example:
FROM amazonlinux
RUN mkdir -p /shared/perl-5.22.1
WORKDIR /shared/perl-5.22.1
RUN yum -y install gcc \
&& curl -SL http://www.cpan.org/src/5.0/perl-5.22.1.tar.gz -o perl-5.22.1.tar.gz \
&& tar --strip-components=1 -xzf perl-5.22.1.tar.gz \
&& rm perl-5.22.1.tar.gz \
&& ./Configure -des -Dprefix=/opt/perl-5.22.1/localperl \
&& make -j $(nproc) \
&& make -j $(nproc) test \
&& make install \
&& rm -fr /shared/perl-5.22.1 /tmp/*
ENV PATH="/opt/perl-5.22.1/localperl/bin:$PATH"
WORKDIR /root
CMD ["perl","-de0"]
When you simply run container with this image, you'll immediately get into perl's shell. If you need bash, then use docker run -it --rm amazon-perl /bin/bash
It would be also good to look at Environment replacement section in the Dockerfile reference documentation, just to figure out how things work. For example, it isn't a best pratice to have that many RUN lines in your Dockerfile because of the RUN instruction will execute commands in a new layer on top of the current image and commit the results. So you'll get many unnecessary layers.
I am trying to mount the current working directory onto Docker container but isn't working. Here is my Dockerfile
FROM ubuntu:14.04.3
MAINTAINER Upendra Devisetty
RUN apt-get update && apt-get install -y g++ \
make \
git \
zlib1g-dev \
python \
wget \
curl \
python-matplotlib
ENV BINPATH /usr/bin
ENV HISAT2GIT https://upendra_35#bitbucket.org/upendra_35/evolinc.git
RUN git clone "$HISAT2GIT"
RUN chmod +x evolinc/evolinc-part-I.sh && cp evolinc/evolinc-part-I.sh $BINPATH
RUN wget -O- http://cole-trapnell-lab.github.io/cufflinks/assets/downloads/cufflinks-2.2.1.Linux_x86_64.tar.gz | tar xzvf -
RUN wget -O- https://github.com/TransDecoder/TransDecoder/archive/2.0.1.tar.gz | tar xzvf -
RUN wget -O- http://seq.cs.iastate.edu/CAP3/cap3.linux.x86_64.tar | tar vfx -
RUN curl ftp://ftp.ncbi.nlm.nih.gov/blast/executables/blast+/LATEST/ncbi-blast-2.2.31+-x64-linux.tar.gz > ncbi-blast-2.2.31+-x64-linux.tar.gz
RUN tar xvf ncbi-blast-2.2.31+-x64-linux.tar.gz
RUN wget -O- http://ftp.mirrorservice.org/sites/download.sourceforge.net/pub/sourceforge/q/qu/quast/quast-3.0.tar.gz | tar zxvf -
RUN curl -L http://cpanmin.us | perl - App::cpanminus
RUN cpanm URI/Escape.pm
ENV PATH /CAP3/:$PATH
ENV PATH /ncbi-blast-2.2.31+/bin/:$PATH
ENV PATH /quast-3.0/:$PATH
ENV PATH /cufflinks-2.2.1.Linux_x86_64/:$PATH
ENV PATH /TransDecoder-2.0.1/:$PATH
ENTRYPOINT ["/usr/bin/evolinc-part-I.sh"]
CMD ["-h"]
When i run the following to mount the current working directory to make sure everything is doing ok, what i see is that all those dependencies are getting installed in the current working directory.
docker run --rm -v $(pwd):/working-dir -w /working-dir ubuntu/evolinc:2.0 -c cuffcompare_out_annot_no_annot.combined.gtf -g Brassica_rapa_v1.2_genome.fa -r Brassica_rapa_v1.2_cds.fa -b TE_RNA_transcripts.fa
I thought, they should only be installed on the container and only the output is going to generate in the current working directory. Sorry, i am very new to Docker and i would need some help with this....
Mouting a volume in docker (-v) allows a container to share directories/volumes with the host. Therefore when changing the volume you are in fact changing the mounted directory. If you wanted to copy some files, rather than point at them, you may need to build your own container and use the COPY or ADD instructions.
What is the difference between the `COPY` and `ADD` commands in a Dockerfile?