Perl installation issue using Docker - docker

I am trying to build the docker image with perl installation.
Dockerfile:
FROM amazonlinux
WORKDIR /shared
RUN yum -y install gcc
ADD http://www.cpan.org/src/5.0/perl-5.22.1.tar.gz /shared
RUN tar -xzf perl-5.22.1.tar.gz
WORKDIR /shared/perl-5.22.1
RUN ./Configure -des -Dprefix=/opt/perl-5.22.1/localperl
RUN make
RUN make test
RUN make install
all these steps are executed i am can see it executing the make, make test and make install commands but when i do :
docker run -it testsh /bin/bash
Error:
when I check perl -v it says command not found.
and I need to go the perl directory
'cd perl-5.22.1' and run 'make install' again then perl -v works
But I want the perl installation to work when I build it with docker image. can anyone tell me what is going wrong here?

perl was indeed installed, just wasn't added to the path.
export PATH=$PATH:/shared/perl-5.22.1 should do it -- but of course, you'd want to add a PATH update in the Dockerfile.

At first glance I thought that when you run make install second time, it adds perl's bin directory to PATH env, but when I compared output of env before and after make install it showed the same PATH variable content.
The reason you getting perl -v working after make install in running container is that make install puts perl binary to /usr/bin/perl. I don't know why it works such way, but it is just as it is. Also, it's almost useless to store sources inside of your image.
Anyway, I agree with #belwood suggestion about adding your perl's bin directiry to PATH environment variable. I just wanna correct the path: /opt/perl-5.22.1/localperl/bin
You need to add it in your Dockerfile (basically I've rewritten your file to make it produce more efficient image), for example:
FROM amazonlinux
RUN mkdir -p /shared/perl-5.22.1
WORKDIR /shared/perl-5.22.1
RUN yum -y install gcc \
&& curl -SL http://www.cpan.org/src/5.0/perl-5.22.1.tar.gz -o perl-5.22.1.tar.gz \
&& tar --strip-components=1 -xzf perl-5.22.1.tar.gz \
&& rm perl-5.22.1.tar.gz \
&& ./Configure -des -Dprefix=/opt/perl-5.22.1/localperl \
&& make -j $(nproc) \
&& make -j $(nproc) test \
&& make install \
&& rm -fr /shared/perl-5.22.1 /tmp/*
ENV PATH="/opt/perl-5.22.1/localperl/bin:$PATH"
WORKDIR /root
CMD ["perl","-de0"]
When you simply run container with this image, you'll immediately get into perl's shell. If you need bash, then use docker run -it --rm amazon-perl /bin/bash
It would be also good to look at Environment replacement section in the Dockerfile reference documentation, just to figure out how things work. For example, it isn't a best pratice to have that many RUN lines in your Dockerfile because of the RUN instruction will execute commands in a new layer on top of the current image and commit the results. So you'll get many unnecessary layers.

Related

What is the correct way to write PATH variable in Docker ENV instruction?

I have tried to build a docker image and found that the PATH variable I set has some issues. A Minimal non-working example is:
FROM ubuntu:latest
SHELL ["/bin/bash", "-cu"]
ARG CTAGS_DIR=/root/tools/ctags
# Install common dev tools
RUN apt-get update --allow-unauthenticated \
&& apt-get install --allow-unauthenticated -y git curl autoconf pkg-config zsh
# Compile ctags
RUN cd /tmp \
&& git clone https://github.com/universal-ctags/ctags.git \
&& cd ctags \
&& ./autogen.sh \
&& ./configure --prefix=${CTAGS_DIR} \
&& make -j$(nproc) \
&& make install \
&& rm -rf /tmp/ctags
ENV PATH=$HOME/tools/ctags/bin:$PATH
RUN echo "PATH is $PATH"
RUN which ctags
In the above Dockerfile, the line ENV PATH=$HOME/tools/ctags/bin:$PATH does not work as expected. It seems that $HOME is not correctly expanded. The following two instructions also do not work:
ENV PATH=~/tools/ctags/bin:$PATH
ENV PATH="~/tools/ctags/bin:$PATH"
Only settings the absolute path works:
# the following setting works.
ENV PATH="/root/tools/ctags/bin:$PATH"
I have looked up the docker references but can not find document about this.
In general, when you're building a Docker image, it's okay to install things into the normal "system" directories. Whatever you're building will be isolated inside the image, and it can't conflict with other tools.
The easiest answer to your immediate question is to arrange things so you don't need to set $PATH.
In the example you give, you can safely use Autoconf's default installation directory of /usr/local. That will almost certainly be empty when you start your image build and only things you install will be there.
RUN ... \
&& ./configure \
&& make \
&& make install
(The Python corollary is to not create a virtual environment for your application; just use the system pip to install things into the default Python library directories.)
Don't expect there to be a home directory. If you have to install in some non-default place, /app is common, and /opt/whatever is consistent with non-Docker Linux practice. Avoid $HOME or ~, they aren't generally well-defined in Docker (unless you go out of your way to make them be).

Docker: Reverse Engineering of an Image

When we use Docker it's very easy push and pull image in a public repository in our https://hub.docker.com but this repository it's free only for public image(only one can be private).
Currently it's possible to execute a reverse engineering of a public image in repository and read the source code of project ?
You can check how an image was created using docker history <image-name> --no-trunc
Update:
Check dive which is a very nice tool that allows you to views image layers.
As yamenk said docker history is the key to this.
As https://github.com/CenturyLinkLabs/dockerfile-from-image is broken, you can use recent
https://hub.docker.com/r/dduvnjak/dockerfile-from-image/
Extract from the site
Note that the script only works against images that exist in your local image
repository (the stuff you see when you type docker images). If you want to
generate a Dockerfile for an image that doesn't exist in your local repo
you'll first need to docker pull it.
For example, you can run it agains itself, to see the code
$ docker run --rm -v /run/docker.sock:/run/docker.sock centurylink/dockerfile-from-image ruby
FROM buildpack-deps:latest
RUN useradd -g users user
RUN apt-get update && apt-get install -y bison procps
RUN apt-get update && apt-get install -y ruby
ADD dir:03090a5fdc5feb8b4f1d6a69214c37b5f6d653f5185cddb6bf7fd71e6ded561c in /usr/src/ruby
WORKDIR /usr/src/ruby
RUN chown -R user:users .
USER user
RUN autoconf && ./configure --disable-install-doc
RUN make -j"$(nproc)"
RUN make check
USER root
RUN apt-get purge -y ruby
RUN make install
RUN echo 'gem: --no-rdoc --no-ri' >> /.gemrc
RUN gem install bundler
ONBUILD ADD . /usr/src/app
ONBUILD WORKDIR /usr/src/app
ONBUILD RUN [ ! -e Gemfile ] || bundle install --system
You can use laniksj/dfimage to reverse engineering of an image.
For example:
# docker run -v /var/run/docker.sock:/var/run/docker.sock laniksj/dfimage <YOUR_IMAGE_ID>
FROM node:12.4.0-alpine
RUN /bin/sh -c apk update
RUN /bin/sh -c apk -Uuv add groff less python py-pip
RUN /bin/sh -c pip install awscli
RUN /bin/sh -c apk --purge -v del py-pip
RUN /bin/sh -c rm /var/cache/apk/*
RUN /bin/sh -c apk add --no-cache curl
ADD dir:4afc740ff29e4a32a34617d2715e5e5dc8740f357254bc6d3f9362bb04af0253 in /app
COPY file:b57abdb61ae72f3a25be67f719b95275da348f9dfb63fb4ff67410a595ae1dfd in /usr/local/bin/
WORKDIR /app
RUN /bin/sh -c npm install
ENTRYPOINT ["docker-entrypoint.sh"]
CMD ["node" "app.js"]
dfimage and dockerfile-from-image are broken
dedockify works
imageName=ruby:latest
docker pull $imageName
docker images # -> get imageId
imageId=xxxxxxxxxxxx
# write to Dockerfile
docker run --rm -v /var/run/docker.sock:/var/run/docker.sock mrhavens/dedockify $imageId >Dockerfile

Error in building docker image with perl installation

I am new to docker and i am trying to build the docker image with the perl installation but not sure exactly how to fix this error.
Dockerfile:
FROM amazonlinux
RUN mkdir /shared
RUN cd /shared
RUN yum -y install sudo
RUN cd /shared
RUN echo "Installing Perl."
RUN sudo yum -y update; yum -y install gcc
RUN yum -y install wget
RUN wget http://www.cpan.org/src/5.0/perl-5.22.1.tar.gz
RUN tar -xzf perl-5.22.1.tar.gz
RUN cd perl-5.22.1
RUN /shared/perl-5.22.1/Configure -des -Dprefix=/opt/perl-5.22.1/localperl
RUN make
RUN make test
RUN make install
RUN echo "Perl installation complete."
instead of /shared/perl-5.22.1/Configure i tried to give ./configure as well but i get the same error No such file or directory
Error:
/bin/sh: /shared/perl-5.22.1/Configure: No such file or directory
The command '/bin/sh -c /shared/perl-5.22.1/Configure -des -Dprefix=/opt/perl-5.22.1/localperl' returned a non-zero code: 127
ish-mac:testanalyse ish$
Can anyone tell me how to fix this issue.
Each Dockerfile RUN command runs in its own shell. So, when you do something like RUN cd /shared, the subsequent RUN commands will not be run inside that working directory.
What you want to use in this case is the WORKDIR instruction (https://docs.docker.com/engine/reference/builder/#workdir). You can also combine and shorten some things by taking advantage of the ADD instruction. A more concise Dockerfile to do what you are after might be:
FROM amazonlinux
WORKDIR /shared
RUN yum -y install gcc
ADD http://www.cpan.org/src/5.0/perl-5.22.1.tar.gz /shared
RUN tar -xzf perl-5.22.1.tar.gz
RUN /shared/perl-5.22.1/Configure -des -Dprefix=/opt/perl-5.22.1/localperl
RUN make -C /shared/perl-5.22.1
RUN make -C /shared/perl-5.22.1 test
RUN make -C /shared/perl-5.22.1 install
For example, the build context is already running as root so there is no need for sudo. With ADD we can add directly from URLs and no wget is required. And the make utility has a -C option to specify the working directory for make.
This should get you closer to what you are after. But the build still fails for other reasons (which you should probably open another question for if you are stuck).

Syntaxnet spec file and Docker?

I'm trying to learn Synatxnet. I have it running through Docker. But I really dont know much about either program Synatxnet or Docker. On the Github Sytaxnet page it says
The SyntaxNet models are configured via a combination of run-time
flags (which are easy to change) and a text format TaskSpec protocol
buffer. The spec file used in the demo is in
syntaxnet/models/parsey_mcparseface/context.pbtxt.
How exactly do I find the spec file to edit it?
I compiled SyntaxNet in a Docker container using these Instructions.
FROM java:8
ENV SYNTAXNETDIR=/opt/tensorflow PATH=$PATH:/root/bin
RUN mkdir -p $SYNTAXNETDIR \
&& cd $SYNTAXNETDIR \
&& apt-get update \
&& apt-get install git zlib1g-dev file swig python2.7 python-dev python-pip -y \
&& pip install --upgrade pip \
&& pip install -U protobuf==3.0.0b2 \
&& pip install asciitree \
&& pip install numpy \
&& wget https://github.com/bazelbuild/bazel/releases/download/0.2.2b/bazel-0.2.2b-installer-linux-x86_64.sh \
&& chmod +x bazel-0.2.2b-installer-linux-x86_64.sh \
&& ./bazel-0.2.2b-installer-linux-x86_64.sh --user \
&& git clone --recursive https://github.com/tensorflow/models.git \
&& cd $SYNTAXNETDIR/models/syntaxnet/tensorflow \
&& echo "\n\n\n" | ./configure \
&& apt-get autoremove -y \
&& apt-get clean
RUN cd $SYNTAXNETDIR/models/syntaxnet \
&& bazel test --genrule_strategy=standalone syntaxnet/... util/utf8/...
WORKDIR $SYNTAXNETDIR/models/syntaxnet
CMD [ "sh", "-c", "echo 'Bob brought the pizza to Alice.' | syntaxnet/demo.sh" ]
# COMMANDS to build and run
# ===============================
# mkdir build && cp Dockerfile build/ && cd build
# docker build -t syntaxnet .
# docker run syntaxnet
First, comment out the command line in the dockerfile, then create and cd into an empty directory on your host machine. You can then create a container from the image, mounting a directory in the container to your hard-drive:
docker run -it --rm -v /pwd:/tmp bash
You'll now have a bash session in the container. Copy the spec file into /tmp from /opt/tensorflow/syntaxnet/models/parsey_mcparseface/context.pbtxt (I'm guessing that's where it is given the info you've provided above -- I can't get your dockerfile to build an image so I can't confirm it; you can always run find . -name context.pbtxt from root to find it), and exit the container (ctrl-d or exit).
You now have the file on your host's hd ready to edit, but you really want it in a running container. If the directory it comes from contains only that file, then you can simply mount your host directory at that path in the container. If it contains other things, then you can use a, so called, bootstrap script to move the file from your mounted directory (in the example above, that's tmp) to its home location. Alternatively, you may be able to tell the software where to find the spec file with a flag, but that will take more research.

How to mount the current working directory onto Docker container?

I am trying to mount the current working directory onto Docker container but isn't working. Here is my Dockerfile
FROM ubuntu:14.04.3
MAINTAINER Upendra Devisetty
RUN apt-get update && apt-get install -y g++ \
make \
git \
zlib1g-dev \
python \
wget \
curl \
python-matplotlib
ENV BINPATH /usr/bin
ENV HISAT2GIT https://upendra_35#bitbucket.org/upendra_35/evolinc.git
RUN git clone "$HISAT2GIT"
RUN chmod +x evolinc/evolinc-part-I.sh && cp evolinc/evolinc-part-I.sh $BINPATH
RUN wget -O- http://cole-trapnell-lab.github.io/cufflinks/assets/downloads/cufflinks-2.2.1.Linux_x86_64.tar.gz | tar xzvf -
RUN wget -O- https://github.com/TransDecoder/TransDecoder/archive/2.0.1.tar.gz | tar xzvf -
RUN wget -O- http://seq.cs.iastate.edu/CAP3/cap3.linux.x86_64.tar | tar vfx -
RUN curl ftp://ftp.ncbi.nlm.nih.gov/blast/executables/blast+/LATEST/ncbi-blast-2.2.31+-x64-linux.tar.gz > ncbi-blast-2.2.31+-x64-linux.tar.gz
RUN tar xvf ncbi-blast-2.2.31+-x64-linux.tar.gz
RUN wget -O- http://ftp.mirrorservice.org/sites/download.sourceforge.net/pub/sourceforge/q/qu/quast/quast-3.0.tar.gz | tar zxvf -
RUN curl -L http://cpanmin.us | perl - App::cpanminus
RUN cpanm URI/Escape.pm
ENV PATH /CAP3/:$PATH
ENV PATH /ncbi-blast-2.2.31+/bin/:$PATH
ENV PATH /quast-3.0/:$PATH
ENV PATH /cufflinks-2.2.1.Linux_x86_64/:$PATH
ENV PATH /TransDecoder-2.0.1/:$PATH
ENTRYPOINT ["/usr/bin/evolinc-part-I.sh"]
CMD ["-h"]
When i run the following to mount the current working directory to make sure everything is doing ok, what i see is that all those dependencies are getting installed in the current working directory.
docker run --rm -v $(pwd):/working-dir -w /working-dir ubuntu/evolinc:2.0 -c cuffcompare_out_annot_no_annot.combined.gtf -g Brassica_rapa_v1.2_genome.fa -r Brassica_rapa_v1.2_cds.fa -b TE_RNA_transcripts.fa
I thought, they should only be installed on the container and only the output is going to generate in the current working directory. Sorry, i am very new to Docker and i would need some help with this....
Mouting a volume in docker (-v) allows a container to share directories/volumes with the host. Therefore when changing the volume you are in fact changing the mounted directory. If you wanted to copy some files, rather than point at them, you may need to build your own container and use the COPY or ADD instructions.
What is the difference between the `COPY` and `ADD` commands in a Dockerfile?

Resources